PaintBot: A deep learning student that trains then mimics old masters

Artificial intelligence has been showing us many ish tricks as apers of human-created art, and now a team of researchers have impressed AI watchers with PaintBot. They have managed to unleash their AI as a capable mimic of the old masters.

AI can deliver a Van Gogh-ish, Vermeer-ish, Turner-ish painting. The team, from the University of Maryland, the ByteDance AI Lab and Adobe Research, turned an algorithm into a mimic of the old masters.

"Through a coarse-to-fine refinement process our agent can paint arbitrarily complex images in the desired style."

PaintBot takes around six hours to learn to imitate a given painter, said the Daily Mail.

"Although there are existing filters which can transform digital photographs to make them similar to a painting," said the report, "the way that PaintBot's compositions are built up from thousands of individual brushstrokes makes the algorithms AI's works more realistic."

The authors stated their framework was novel. They referred to a "Simplified Simulated Painting Environment (SSPE)." Given a reference image, they wrote, their painting agent aims to reproduce the identical or transformed version of that image in the SSPE. Their PaintBot can study an artist's work and teach itself to paint in that very style. Pointillism? No prob. Post impressionism? Done.

Once trained on that style, it is suitably hooked: It can make new paintings in that style based on photos.

PaintBot moves along stroke by stroke.

What exactly has it picked up? Elements of techniques, such as "colour, density, position and size—along with the order in which each brushstroke should be made."

The paper stated: "We demonstrate that our painting agent can learn an effective policy with a high dimensional continuous action space comprising pen pressure, width, tilt, and color, for a variety of painting styles."

Ian Randall in Daily Mail stepped readers through how the PaintBot was trained for mimicry: "the researchers would present the algorithm with between 3 and 10 reference paintings."

The AI would practice reproducing reference paintings, said Randall, "which it would then compare with the original work to see how similar the two were and if it was improving its imitation of the artist's style."

The paper discussed presenting the agent in training with patches sampled from an ensemble of reference images. "To accelerate training convergence, we adopt a curriculum learning strategy, whereby reference patches are sampled according to how challenging they are using the current policy."

Their paper describing their work is up on arXiv.

"PaintBot: A Reinforcement Learning Approach for Natural Media Painting" is by Biao Jia, Chen Fang, Jonathan Brandt, Byungmoon Kim and Dinesh Manocha.

The authors call their tech "a painting agent" that is trained through reinforcement learning. They said their approach learns without human supervision, "and does not degrade after thousands of strokes which can handle a large dense reference image."

What's next? Randall said two individuals from Adobe Research are working to see if a future version of PaintBot could potentially be incorporated within Adobe's graphics editing application, Photoshop.

While this is all about AI as art-maker, not humans as art-makers, Daily Mail brought out that actually the path to art delivery has one similarity between the two, and that is apprenticeship: "Much like the pupils of the old masters, the new AI meticulously studies the work of virtuoso painters like Vermeer and Van Gogh and learns to reproduce their works."

As meticulous as the process might be, many people regularly point out that art by humans is not the same. Yes but AI can't chop its own ear off, said one reader comment in the Randall article.

Computer scientist Dinesh Manocha of the University of Maryland College Park, wrote Randall, "believes that algorithms are still no match for human creativity."