Neuromorphic controllers to enhance control during landing in micro air vehicles
Flying insects are able to navigate their environments efficiently, processing visual stimuli to avoid obstacles and land safely on a variety of surfaces. Over the past decade or so, research teams worldwide have been trying to replicate these capabilities in autonomous micro air vehicles (MAVs) using mechanisms similar to those observed in insects.
Researchers at TU Delft's MAVLab have been trying to develop insect-inspired techniques that could enhance navigation and landing strategies in tiny drones for several years now. In a recent paper pre-published on arXiv, they introduced a new strategy for the creation of neuromorphic controllers that could improve landings in MAVs.
"At the MAVLab of TU Delft, we study the autonomous flight of tiny drones," Jesse Hagenaars, one of the researchers who carried out the study, told TechXplore. "This is a major challenge, because our drones (sometimes as light as 20 grams) have extremely limited resources in terms of energy, sensing and processing. This is why we draw a lot of inspiration from nature, and in particular, from flying insects."
In previous work, researchers at the MAVLab developed a series of bio-inspired techniques for vision-based motion estimation using spiking neural networks (SNNs). SNNs are a class of artificial neural networks that closely mimic neural networks in the human brain, using activation spikes to compute and analyze information.
In their new study, the Hagenaars and his colleagues wanted to take their techniques one step further, using them to control the flight and landing of MAVs. To do this, they started a collaboration with the Dutch National research institute for computer science and mathematics (CWI), which has a high level of expertise in the development of spiking neural networks.
"The final goal of our research is to connect motion estimation and control to end up with a completely bio-inspired pipeline, which will be much more efficient in terms of energy spending than traditional vision-based control approaches," Hagenaars said. "For now, to demonstrate the feasibility of the control part, we applied our approach to landing maneuvers."
Most previously developed techniques to control MAVs during vision-based landing are based on proportional controllers and conventional ANNs. Controllers based on SNNs have the potential of achieving similar or even better results with far greater energy efficiencies.
Unlike conventional artificial neural networks (ANNs), in which every individual neuron passes on a real value at every timestep, SNNs output a binary spike only when they receive sufficient stimulation. Given that each individual spike or computation requires a certain amount of energy, SNNs tend to be far more energy efficient than conventional ANNs, as they are generally implemented using what is known as 'neuromorphic hardware."
"While we did not implement our spiking controllers on neuromorphic hardware, we did go a step further in terms of energy efficiency, by minimizing the number of spikes used by the network to perform control," Hagenaars said. "This was done by including the number of spikes as an objective during the evolutionary optimization of the controllers."
Hagenaars and his colleagues trained their SNN-based controllers using simulation tools, and then evaluated their performance in real-world environments. Their experiments yielded very promising results, with the controllers enabling fast and safe MAV landings, while keeping SNN spikes and thus energy expenditure to a minimum.
Interestingly, the researchers also found that spiking controllers that produce fewer spikes (i.e., spend less energy) performed just as well as others that produce more spikes. In fact, limiting the number of incoming spikes seemed to simplify the transfer of a controller's landing capabilities from simulated environments to the real world.
"First of all, this work is the first to integrate spiking neural networks in the control loop of a real-world flying robot," Hagenaars said. "Second, we substantially minimize the spike rate of the controllers, which would result in considerable energy savings when implemented on neuromorphic hardware. Besides making the spiking networks as small as possible, we include network spike rate as an objective in the multi-objective evolution."
Hagenaars and his colleagues at TU Delft were among the first to use SNNs for controlling flying robots in real world environments. The controllers they created could ultimately help researchers to increase the performance and energy efficiency of both existing and newly developed MAVs, particularly during landing.
"Our recent paper only focused on bio-inspired control based on a given motion estimate," Hagenaars said. "The actual method of estimating this motion was, however, not very bio-inspired. Therefore, we now want to combine our controller with the bio-inspired motion estimation method (also based on spiking networks) developed previously by the MAVLab, to end up with a completely bio-inspired pipeline."
So far, the researchers have only tested their controllers on conventional chips, yet the energy savings they predicted can only be realized using neuromorphic hardware. In their future work, they thus also hope to implement them on neuromorphic chips, such as Intel's Loihi chip.
Hagenaars, J. J., Paredes-Vallés, F., Bohté, S. M., & de Croon, G. C. H. E. (2020). Evolved neuromorphic control for high speed divergence-based landings of MAVs.
arXiv:2003.03118 [cs.RO], 2020. arxiv.org/abs/2003.03118
Paredes-Vallés, F., Scheper, K. Y. W., & De Croon, G. C. H. E. (2019). Unsupervised learning of a hierarchical spiking neural network for optical flow estimation: From events to global motion perception. DOI: 10.1109/TPAMI.2019.2903179. ieeexplore.ieee.org/document/8660483
© 2020 Science X Network