August 24, 2022 feature
A deep learning framework to enhance the capabilities of a robotic sketching agent
In recent years, deep learning algorithms have achieved remarkable results in a variety of fields, including artistic disciplines. In fact, many computer scientists worldwide have successfully developed models that can create artistic works, including poems, paintings and sketches.
Researchers at Seoul National University have recently introduced a new artistic deep learning framework, which is designed to enhance the skills of a sketching robot. Their framework, introduced in a paper presented at ICRA 2022 and pre-published on arXiv, allows a sketching robot to learn both stroke-based rendering and motor control simultaneously.
"The primary motivation for our research was to make something cool with non-rule-based mechanisms such as deep learning; we thought drawing is a cool thing to show if the drawing performer is a learned robot instead of human," Ganghun Lee, the first author of the paper, told TechXplore. "Recent deep learning techniques have shown astonishing results in the artistic area, but most of them are about generative models which yield whole pixel outcomes at once."
Instead of developing a generative model that produces artistic works by generating specific pixel patterns, Lee and his colleagues created a framework that represents drawing as a sequential decision process. This sequential process resembles the way in which humans would draw individual lines using a pen or pencil to gradually create a sketch.
The researchers then hoped to apply their framework to a robotic sketching agent, so that it could produce sketches in real-time using a real pen or pencil. While other teams created deep learning algorithms for "robot artists" in the past, these models typically required large training datasets containing sketches and drawings, as well as inverse kinematic approaches to teach the robot to manipulate a pen and sketch with it.
The framework created by Lee and his colleagues, on the other hand, was not trained on any real-world drawing examples. Instead, it can autonomously develop its own drawing strategies over time, through a process of trial and error.
"Our framework also doesn't use inverse kinematics, which make robot movements a bit strict, instead it also lets the system find its own movement tricks (adjusting joint values) to make movement style as natural as possible," Lee said. "In other words, it directly moves its joints without primitives while many robotic systems commonly use primitives to move."
The model created by this team of researchers includes two "virtual agents," namely the upper class and lower class agent. The upper class agent's role is to learn new drawing tricks, while the lower class agent learns effective movement strategies.
The two virtual agents were trained individually using reinforcement learning techniques and were only coupled once they had completed their respective training. Lee and his colleagues then tested their combined performance in a series of real-world experiments, using a 6-DoF robotic arm with a 2D gripper on it. The results achieved in these initial tests were very encouraging, as the algorithm allowed the robotic agent to produce good sketches of specific images.
"We find that the reinforcement learning-based modules trained for each objective can be merged to achieve bigger collaborative objectives," Lee explained. "In a hierarchical setting, decisions from the upper agent can be the 'intermediate state,' which allows the lower agent to observe to make lower decisions. If each agent of levels is well-trained and generalized enough to each state space, then a whole system made of each module can do great things. However, the primal condition is that, as all reinforcement learning approaches have so, reward functions for each agent should be well-shaped (it's not easy)."
In the future, the framework created by Lee and his colleagues could be used to improve the performance of both existing and newly developed robotic sketching agents. In the meantime, Lee is developing similar creative reinforcement learning-based models, including a system that can produce artistic collages.
"We would also like to extend the task to more complicated robotic drawings such as paintings, but I am now focusing more on the practical issues of reinforcement learning applications itself than the robotic drawings," Lee added. "I hope our paper becomes a fun and meaningful example of pure reinforcement learning-based application especially equipped with robots."
More information: Ganghun Lee, Minji Kim, Minsu Lee, Byoung-Tak Zhang, From scratch to sketch: deep decoupled hierarchical reinforcement learning for robotic sketching agent. arXiv:2208.04833v1 [cs.RO], arxiv.org/abs/2208.04833
© 2022 Science X Network