Locomotion modeling evolves with brain-inspired neural networks

Locomotion modeling evolves with brain-inspired neural networks
A. Overview of the architecture of DMAP. Each component of the state history is processed independently by a temporal convolutional network (with shared weights). The extracted features are transformed into the value encoding matrix and the attention weights of each joint sensor and action component. The product of these matrices results in a joint-specific morphology encoding, which is the convex combination of the value encoding vectors, weighted by their attention score. Independent fully-connected networks control a single joint, based on the current proprioceptive state and the morphology encoding B. DMAP is trained for several episodes with randomly sampled perturbed bodies within the training distribution. At test time, the zero-shot performance of the model is assessed on a fixed set of body configurations, sampled either from within (IID) or outside (OOD) of the training distribution. Credit: arXiv (2022). DOI: 10.48550/arxiv.2209.14218

A team of scientists at EPFL have built a new neural network system that can help understand how animals adapt their movement to changes in their own body and to create more powerful artificial intelligence systems.

Deep learning has been fueled by , which stack simple computational elements on top of each other, to create powerful learning systems. Given enough data, these systems can solve challenging tasks like recognize objects, beat human's at Go and also control robots.

"As you can imagine, the architecture of how you stack these elements on top of each other might influence how much data you need to learn and what the ceiling performance is," says Professor Alexander Mathis at EPFL's School of Life Sciences.

Working with doctoral students Alberto Chiappa and Alessandro Marin Vargas, the three scientists have developed a new network architecture called DMAP for "Distributed Morphological Attention Policy." This incorporates fundamental principles of biological sensorimotor control, making it an interesting tool to study sensorimotor function.

Credit: Ecole Polytechnique Federale de Lausanne

The problem that DMAP is trying to address is that animals—including humans—have evolved to adapt to changes in both their environment and their own bodies. For example, a child can adapt its ability to walk efficiently throughout all the body changes in shape and weight from a toddler to adulthood—and do so on different types of surfaces, etc. When developing DMAP, the team focused on how an animal can learn to walk when its body is subject to these "morphological perturbations"—changes in the length and thickness of body parts.

"Typically, in Reinforcement Learning, so-called fully connected are used to learn ," says Mathis. Reinforcement Learning is a machine-learning training method that "rewards" desired behaviors and/or "punishes" undesired ones.

He continues: "Imagine you have some sensors that estimate the state of your body—for example, the angles of your wrist, elbow, shoulder, and so on. This sensor signals are the input to the motor system, and the output are the muscle activations, which generate torques. If one uses fully connected networks, then for instance in the first layer all sensors from across the body are integrated." In contrast, in biology is combined in a hierarchical way."

"We took principles of neuroscience, and we distilled them in a to design a better sensorimotor system," says Alberto Chiappa. In their paper, published at the 36th Annual Conference on Neural Information Processing Systems (NeurIPS), the researchers present DMAP that "combines independent proprioceptive processing, a distributed policy with individual controllers for each joint, and an attention mechanism, to dynamically gate sensory information from different body parts to different controllers."

DMAP was able to learn to "walk" with a body subject to morphological perturbations, without receiving any information about the morphological parameters such as the specific limb lengths and widths. Remarkably, DMAP could "walk" as well as a system that had access to those body parameters.

"So we created a Reinforcement Learning system thanks to what we know from anatomy," says Alberto Chiappa. "After we trained this model, we noticed that it exhibited dynamic gating reminiscent of what happens in the spinal cord, but interestingly this behavior emerged spontaneously."

Overall, models like DMAP serve two roles: building better artificial intelligence systems based on biological insights, and conversely building better models to understand the brain.

The findings are avaiable on the arXiv preprint server.

More information: Alberto Silvio Chiappa et al, DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Body, arXiv (2022). DOI: 10.48550/arxiv.2209.14218

Journal information: arXiv

Citation: Locomotion modeling evolves with brain-inspired neural networks (2022, December 6) retrieved 8 February 2023 from https://techxplore.com/news/2022-12-locomotion-evolves-brain-inspired-neural-networks.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

A novel integrated system of neuromorphic devices


Feedback to editors