PaintTransformer is a neural network that approximates images using paint strokes. This gives it a very pretty and stylized watercolor/painterly aesthetic.
I like how this looks, and I’ll sometimes just run random photos through it to see how they turn out.
Another cool thing you can do is save each step as a frame, which creates a cool animation similar to a watercolor timelapse.
PaintTransformer starts by adding with broad, vague brush strokes and ends with tiny details. It’s just a few blobs of colour at the start and it’s highly detailed at the end. This makes for a fun charades-like game: how quickly can you guess what it’s painting?
PaintTransformer was developed in this paper by a team of researchers. It was later implemented in PyTorch by Huage001
I wanted to use PaintTransformer, but my makeshift solution of uploading files to a Colab runtime was too clunky to be used at scale. So I instead decided to build a custom interface. I named it ColourlessTransformer because I was going through a phase of naming everything eponymously after my username.
Streamlit
This was my first time working with Streamlit. I originally planned to use Gradio for this project because I had previous experience, but Gradio is an extremely heavy framework for my use case, and the 2-minute starting times were bothering me. I’m quite happy with how the it turned out.
Usage
If you want to try this out on your own, follow the instructions in the README. You’ll need Git, Python, a GPU, and some familiarity with the terminal.
Conclusion
ColourlessTransformer was one of my favorite projects because I started with a complex task that took 3-5 minutes of tedious parameter filling and file reorganization and ended with a sleek, fast, and painless custom application.
Apparently it wasn’t enough to automate watercolor; I also had to automate the automation.
~Ethan