PREDICTION OF MULTIVARIATE TIME SERIES DATA USING ECHO STATE NETWORK AND HARMONY SEARCH

Muhammad Muharrom Al Haromainy, Chastine Fatichah, Ahmad Saikhu

Abstract


Multivariate time series data prediction is widely applied in various fields such as industry, health, and economics. Several methods can form prediction models, such as Artificial Neural Network (ANN) and Recurrent Neural Network (RNN). However, this method has an error value more significant than the development method of RNN, namely the Echo State Network (ESN). The ESN method has several global parameters, such as the number of reservoirs and the leaking rate. The determination of parameter values dramatically affects the performance of the resulting prediction model. The Harmony Search (HS) optimization method is proposed to provide a solution for determining the parameters of the ESN method. The HS method was chosen because it is easier to implement, and based on other research, the HS method gets the optimum value better than other meta-heuristic methods. The methods compared in this study are RNN, ESN, and ESN-HS. Root Mean Square Error (RMSE) and Mean Absolute Percent Error (MAPE) are used to measure the error rate of forecasting results. ESN got a smaller error value than RNN, and ESN-HS produced a minor error value among the other trials, namely 0.782e-5 for RMSE and 0.28% for MAPE. The HS optimization method has successfully obtained the appropriate global parameters for the ESN prediction model.


Full Text:

PDF

References


F. M. Bianchi, E. D. E. Santis, A. Rizzi, A. Sadeghian, and S. Member, “Short-Term Electric Load Forecasting Using Echo State Networks and PCA Decomposition,” IEEE Access, vol. 3, pp. 1931–1943, 2015.

T. Mikolov, W.-T. Yih, and G. Zweig, “Linguistic Regularities in Continuous Space Word Representations,” Association for Computational Linguistics, 2013.

Z. Ma, Y. Dong, C. Wang, and X. U. N. Shao, “Forecast of Non-Equal Interval Track Irregularity Based on Improved Grey Model and PSO-SVM,” IEEE Access, vol. 6, pp. 34812–34818, 2018.

K. M. Begam and S. N. Deepa, “Optimized nonlinear neural network architectural models for multistep wind speed forecasting ✩,” Comput. Electr. Eng., vol. 78, pp. 32–49, 2019.

M. O. Selbesoglu, “Prediction of tropospheric wet delay by an artificial neural network model based on meteorological and GNSS data,” Eng. Sci. Technol. an Int. J., 2019.

R. K. B. Navas, S. Prakash, and T. Sasipraba, “Artificial Neural Network based computing model for wind speed prediction: A case study of Coimbatore, Tamil Nadu, India,” Physica A, 2019.

O. I. Abiodun, A. Jantan, A. E. Omolara, K. V. Dada, N. A. Mohamed, and H. Arshad, “State-of-the-art in artificial neural network applications: A survey,” Heliyon, vol. 4, no. 11, p. e00938, Nov. 2018.

M. Daoud, M. Mayo, P. O. Box, and N. Zealand, “A survey of neural network-based cancer prediction models from microarray data,” Artif. Intell. Med., vol. 97, no. October 2017, pp. 204–214, 2019.

K. Ivanedra and M. Mustikasari, “Implementasi Metode Recurrent Neural Network Pada Text the Implementation of Text Summarization with Abstractive,” vol. 6, no. 4, 2019.

M. Usama, B. Ahmad, W. Xiao, M. S. Hossain, and G. Muhammad, “Self-attention based recurrent convolutional neural network for disease prediction using healthcare data,” Comput. Methods Programs Biomed., p. 105191, 2019.

S. Choi, J. Kim, and H. Yeo, “Attention-based Recurrent Recurrent Neural Network Network for Urban Vehicle Trajectory Prediction,” Procedia Comput. Sci., vol. 151, no. 2018, pp. 327–334, 2019.

S. A. Zagrebina, V. G. Mokhov, and V. I. Tsimbol, “Electrical Energy Consumption Prediction is based on the Recurrent Neural Network,” Procedia Comput. Sci., vol. 150, pp. 340–346, 2019.

X. Cai, N. Zhang, G. K. Venayagamoorthy, and D. C. Wunsch, “Time series prediction with recurrent neural networks trained by a hybrid PSO – EA algorithm,” vol. 70, pp. 2342–2353, 2007.

M. Xu, M. Han, and H. Lin, “Wavelet-denoising multiple echo state networks for multivariate time series prediction,” Inf. Sci. (Ny)., 2018.

J. Schimdhuber, “Deep Learning in Neural Networks : An Overview,” Neural Networks, vol. 61, pp. 85–117, 2015.

L. Shen, J. Chen, Z. Zeng, J. Yang, and J. Jin, “A novel echo state network for multivariate and nonlinear time series prediction,” Appl. Soft Comput. J., vol. 62, pp. 524–535, 2018.

G. Shi, D. Liu, and Q. Wei, “Energy consumption prediction of office buildings based on echo state networks,” Neurocomputing, pp. 1–11, 2016.

S. Løkse, F. Maria, and R. Jenssen, “Training Echo State Networks with Regularization Through Dimensionality Reduction,” Cognit. Comput., no. 1, 2017.

U. D. Schiller and J. J. S. Ã, “Analyzing the weight dynamics of recurrent learning algorithms,” vol. 63, pp. 5–23, 2005.

N. Chouikhi, B. Ammar, N. Rokbani, and A. M. Alimi, “PSO-based analysis of Echo State Network parameters for time series forecasting,” Appl. Soft Comput. J., vol. 55, pp. 211–225, 2017.

M. Lukoševičius, “A Practical Guide to Applying Echo State Networks,” pp. 659–686, 2012.

Y. Kim, Y. Yoon, and Z. Woo, “A comparison study of harmony search and genetic algorithm for the max-cut problem,” Swarm Evol. Comput., vol. 44, no. August 2017, pp. 130–135, 2019.

J. Saadat, P. Moallem, and H. Koofigar, “Training Echo Estate Neural Network Using Harmony Search Algorithm,” vol. 15, no. 1, pp. 163–179, 2017.

A. Klibisz, “A Primer on Reservoir Computing,” pp. 1–14, 2016.

D. Verstraeten, B. Schrauwen, M. D. Haene, and D. Stroobandt, “An experimental unification of reservoir computing methods,” vol. 20, pp. 391–403, 2007.

H. Jaeger, “Long Short-Term Memory in Echo State Networks : Details of a Simulation Study Long Short-Term Memory in Echo State Networks : Details of a Simulation Study,” no. 27, 2012.

A. Askarzadeh and E. Rashedi, Harmony Search Algorithm, no. April. 2017.




DOI: http://dx.doi.org/10.12962/j24068535.v19i2.a1051

Refbacks

  • There are currently no refbacks.