Chitrakar: A system that can transform images of human faces into drawings

Researchers at the TCS Robotics Research Lab in India have recently developed an artificial intelligence (AI) system that can automatically convert an image of a person's face into a recognizable non-self-intersecting loop, known as a Jordan curve. Subsequently, this Jordan curve can be used to produce realistic drawings of the person in the original image, using a robotic hand.

"The idea for this paper came to me on a flight while reading collection of best mathematical writings, where I came across the article titled "Jordan Curve Theorem Is Non-trivial' by Fiona Ross and William T. Ross," Aniruddha Singhal, one of the researchers who carried out the study, told TechXplore. "I connected it with another book that I had read earlier, titled 'In Pursuit of the Travelling Salesman' by William J. Cook, discussing traveling salesman problem (TSP) art. I realized I could also interpret the article from a Vedantic (Indic or Hindu philosophical) point of view, which connects the idea of TSP art and the Jordan curve, turning it into an expression of the ultimate beauty."

A Jordan curve is essentially a single line that starts and ends in the same point in space, drawing an image without ever intersecting itself. TSP art, on the other hand, is a form of art in which images are produced by drawing a single, non-intersecting line.

After reading about Jordan curves and TSP art, Singhal, who is a roboticist, started investigating the possibility of automatically producing TSP drawings, which are very difficult for humans to create. In collaboration with his colleagues at the TCS Robotics Research Lab, he eventually created Chitrakar, a system that can transform an image into a drawing composed of a single, long line, ultimately turning it into a Jordan curve. In Hindi/Sanskrit, the word Chitrakar means 'painter," or the one who creates a painting.

"Our system uses state-of-the-art, deep-learning techniques to segment the human face from a photo and combined this with image enhancements techniques," Singhal said. "The enhanced image is stippled, and the points are connected by a TSP solver where each point is assumed to be a destination of a traveling salesman. The final route of traveling salesman is converted to a Jordan curve by an intersection removal technique."

Chitrakar, the system created by the researchers, can autonomously convert any image of a human face into a Jordan curve. Subsequently, this Jordan curve is drawn on a piece of paper by a robotic gripper, ultimately creating artistic TSP portraits.

"It is an engineering feat to be able to combine state-of-the-art deep-learning methods, image processing and robotics to enable a robotic arm to draw a portrait in its own unique style," Singhal explained. "It is satisfying to see a robot drawing your face in a unique style."

The robotic gripper designed by Singhal and his colleagues can hold a variety of different pens, which produce slightly different drawings (e.g., with thinner/thicker lines, of a different color, etc.). Remarkably, it can produce drawings with an acceptable result in less than 30 minutes, which is impossible to achieve by human artists. The researchers' system is currently being patented in several different countries and could eventually become widely available.

"We now plan to extend this for other mediums which are difficult for human to handle," Singhal explained. "For instance, we plan to use Style-GAN and other generative models to generate unique portraits from a given picture and then convert them into line or pencil sketches."

© 2021 Science X Network