September 1, 2015 weblog
Neural algorithm gives photo masterpiece-style treatments
Computer scientists are intrigued about what computers can tell us about artistic masterpieces, from picking out forgeries to assessing artistic worth.
Now the intriguing question being explored by a team of scientists is how far an algorithm can go in turning a photograph into a work of art that in style evokes great names in painting such as Picasso, Turner and Kandinsky.
Researchers representing institutions in Tubingen, Germany and Houston, Texas (Baylor College of Medicine, Department of Neuroscience) have taken a look at fine art with surprising results.
Leon A. Gatys, Alexander S. Ecker and Matthias Bethge recently posted their paper, "A Neural Algorithm of Artistic Style," on the arXiv server, where they describe how far they got in unleashing the powers of deep learning for their project.
"In fine art, especially painting, humans have mastered the skill to create unique visual experiences through composing a complex interplay between the content and style of an image. Thus far the algorithmic basis of this process is unknown, they wrote, "and there exists no artificial system with similar capabilities."
Matt McFarland of The Washington Post translated what they achieved, an algorithm that can create a new 'Van Gogh' in just an hour. In other words, the style of art masters can be mimicked. Creative types could be delivering images leveraging the styles of the world's greatest artists.
They set out to introduce an artificial system based on a "Deep Neural Network" that creates artistic images which they said are of high perceptual quality. Deep Neural Networks are trained on object recognition. Adario Strange in Mashable likewise spoke of their experiment transforming any photo into an image that mimics the painting style of art masters.
The paper has a most interesting presentation of how different famous artists' styles transform a photograph. Your street's row of houses could be turned into a starry night portrait, then a dream-like geometric landscape, then an abstract-like screaming circus-like neighborhood, depending on the artistic rendering and all in the styles of world-famous artists.
The authors said that neural representations separate and recombine content and style of arbitrary images, "providing a neural algorithm for the creation of artistic images."
The Washington Post's McFarland described their process: "The researchers create their new images by taking one image, such as a bland photo of a row of houses, and another image, a painting from a great artist. They then use what's called a convolutional neural network to create a new image, in which the style of the great artist is applied to the other photo."
The authors stated that the key finding of their paper is that "the representations of content and style in the Convolutional Neural Network are separable. That is, we can manipulate both representations independently to produce new, perceptually meaningful images."
What do they mean by this "Convolutional Neural Network"? The class of Deep Neural Networks that are most powerful in image processing tasks, they said, are called Convolutional Neural Networks. These consist of layers of small computational units that process visual information hierarchically in a feed-forward manner. "Each layer of units can be understood as a collection of image filters, each of which extracts a certain feature from the input image."
McFarland shared his perspective on this: "Of course, the machines don't have a style all their own. They can only copy existing styles. True creativity is a greater challenge that machines have yet to master."
In fine art, especially painting, humans have mastered the skill to create unique visual experiences through composing a complex interplay between the content and style of an image. Thus far the algorithmic basis of this process is unknown and there exists no artificial system with similar capabilities. However, in other key areas of visual perception such as object and face recognition near-human performance was recently demonstrated by a class of biologically inspired vision models called Deep Neural Networks. Here we introduce an artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality. The system uses neural representations to separate and recombine content and style of arbitrary images, providing a neural algorithm for the creation of artistic images. Moreover, in light of the striking similarities between performance-optimised artificial neural networks and biological vision, our work offers a path forward to an algorithmic understanding of how humans create and perceive artistic imagery.
© 2015 Tech Xplore