Future dynamics prediction from short-term time series by anticipated learning machine

Making an accurate prediction based on observed data, in particular from short-term time series, is of much concern in various disciplines— from molecular biology, neuroscience, geoscience, and economics to atmospheric sciences—due to either data availability or time-variant non-stationarity. However, most of existing methods require sufficiently long measurements of time series or a large number of samples, and there is no effective method available for prediction with short-term time-series because of lack of information.

To address this issue, Prof. Chen Luonan (Institute of Biochemistry and Cell Biology, Chinese Academy of Sciences) with Dr. Chen Chuan (Sun Yat-sen University), Prof. Ma Huanfei (Soochow University) and Prof. Aihara Kazuyuki (University of Tokyo) proposed a new dynamics-based data-driven method, anticipated learning machine (ALM), for achieving precise future-state predictions based on short-term but high-dimensional data. ALM is a multi-layered neural network where high-dimensional variables are taken as input neurons (multiple variables but at a single time point) but a target variable is taken as output neurons (single variable but at multiple time points). In this way, ALM is able to transform the recent correlation/spatial information of high-dimensional variables to future dynamical/temporal information of any target variable, i.e. by spatial-temporal information transformation (STI) equations.

Specifically, ALM can be well trained to represent the randomly distributed embedding (RDE) map for STI equations by a large number of the generated training samples with the Dropout scheme and the proposed consistent-training scheme, thus predicting the target variable in an accurate and robust manner, even from short-term data. Extensive experiments on the short-term high-dimensional data from both synthetic and real-world systems demonstrated significantly superior performances of ALM over existing methods.

Compared with traditional neural networks (or other machine learning approaches) which excavate the historical statistics of the original high-dimensional system and thus require a large number of samples, ALM efficiently and robustly reconstructs its dynamics even with a small number of samples by constraining to a low-dimension space which is actually an inherent property of such a dissipative system. Based on nonlinear dynamics to transform the spatial information of the all measured high-dimensional variables into the temporal evolution of the target variable by learning the STI equations, ALM open a new path for dynamics-based machine learning or "intelligent" anticipated learning.

"How to consider the strong nonlinearity or/and stochasticity of the dynamical systems also with the observed noisy data, and further how to make more in-depth theoretical analysis and further develop an appropriate framework taking these issues into consideration remain an open and interesting problem in future," state the authors.