PaintTransformer is a neural network that approximates images using paint strokes. This gives it a very pretty and stylized watercolor/painterly aesthetic.
I like how this looks, and I'll sometimes just run random photos through it to see how they turn out.
Another cool thing you can do is save each step as a frame, which creates a cool animation similar to a watercolor timelapse.
Because of how PaintTransformer works, the brush size gets smaller and smaller as times goes on (so that it can add details), but this makes for a fun charades-like game: how quickly can you guess what it's painting?
PaintTransformer was developed in this paper by a team of researchers. It was later implemented in PyTorch by Huage001
I wanted to use PaintTransformer, but my makeshift solution of uploading files to a Colab runtime was too clunky to be used at scale. So I took this opportunity to brush up on my interface coding skills and code a client. I decided to name it ColourlessTransformer because I was going through a phase of naming everything eponymously after my username.
Streamlit
This was my first time working with Streamlit. I originally planned to use Gradio for this project because I had previous experience, but Gradio is an extremely heavy framework for my use case, and the 2-minute starting times were bothering me.
I'm quite happy with how the Streamlit app turned out. A detail I'm particularly proud of is how the previous output fades out slightly while the new one is processing.
Usage
If you want to try this out on your own, follow the instructions in the README. You'll need Git, Python, a GPU, and some familiarity with the terminal.
Conclusion
ColourlessTransformer was one of my favorite projects because I started with a complex task that took 3-5 minutes of tedious parameter filling and file reorganization and ended with a sleek, fast, and painless custom application.
Apparently it wasn't enough to automate watercolor; I also had to automate the automation.
~Ethan