A reservoir computing system for temporal data classification and forecasting

Over the past decade or so, deep-learning approaches have become increasingly efficient in processing static data such as images. However, these techniques have been found to be somewhat less effective in analyzing temporal data, such as videos, human speech and other streaming inputs. This is mainly because processing temporal data requires bigger artificial neural networks, which are more expensive to train and implement.

With this in mind, a team of researchers at the University of Michigan has recently developed a reservoir computing hardware system to process temporal data more effectively. Reservoir computing systems essentially consist of a reservoir that maps inputs into a high-dimensional space and a readout for pattern analysis based on the reservoir's high-dimensional states.

These systems have been found to be particularly effective for temporal or sequential data processing. The system developed by the researchers, which was presented in a paper published in Nature Electronics, is based on dynamic tungsten oxide (WO_x) memristors with internal short-term memory capabilities.

"A main reason for the large network size needed for processing temporal data is the large number of possible temporal features that need to be learned and stored by the network," Wei Lu, the senior author who led the study, told TechXplore. "To solve this problem, we employed a 'reservoir computing' concept, where the 'reservoir' in the system can process inputs without having to learn the features. This is enabled by the 'short-term memory' property of the reservoir, so that it can respond (be excited) accordingly to different inputs without having to explicitly store anything."

Most previously developed reservoirs were built using digital circuits that emulate short-term memory effects. This ultimately makes them difficult to implement physically, and thus highly impractical.

Lu and his colleagues, on the other hand, fabricated their reservoir computing system using WO_x memristor devices with intrinsic short-term memory properties. In other words, each individual memristor device is a dynamic system in itself and can process a broad range of temporal inputs.

Through these memristors, the reservoir system can nonlinearly map temporal inputs into reservoir states. Projected features can then be easily processed by a linear readout function.

"By taking advantage of the internal dynamics of the devices to naturally perform computing, we could build the reservoir network with only a small number of memristor devices leading to much smaller footprint, cost, and power consumption," Lu explained.

Lu and his colleagues demonstrated and evaluated their system on a standard speech recognition task that involves recognizing spoken digits. Their system was able to recognize digits spoken by humans with a remarkable accuracy of 99.2 percent.

"More interestingly, since the network can capture the temporal features of the input, we showed that we can also use the network to perform prediction/forecasting functions," Lu said. "For example, in speech recognition, we can predict the speaker's intended word before the speaker finishes it. In another example, we showed the ability of the network to capture the complex features of a chaotic system and reliably predict the evolution of the chaotic system over long term, which is a very challenging task."

In the future, the reservoir computing system for analyzing and forecasting temporal inputs devised by this team of researchers could have numerous interesting applications. For instance, it could help to improve human-machine interfaces, autonomous driving platforms, and other technology that requires the processing or forecasting of streaming inputs.

Moreover, using this new approach, the size and power consumption of artificial neural networks for processing temporal data can be significantly reduced. This could make embedding these networks into existing systems easier and cheaper, ultimately allowing researchers to equip a broader variety of devices with real-time temporal data analysis capabilities.

"We are now working on more complex systems and on further improving the network's performance," Lu added.

© 2019 Science X Network