A Comprehensive Survey on Pi-Sigma Neural Network for Time Series Prediction
Keywords:
Higher Order Neural Network Time Series Forecasting, Pi-Sigma Neural Network, Recurrent Networks,Abstract
Prediction of time series grabs received much attention because of its effect on the vast range of real life applications. This paper presents a survey of time series applications using Higher Order Neural Network (HONN) model. The basic motivation behind using HONN is the ability to expand the input space, to solve complex problems it becomes more efficient and perform high learning abilities of the time series forecasting. Pi-Sigma Neural Network (PSNN) includes indirectly the capabilities of higher order networks using product cells as the output units and less number of weights. The goal of this research is to present the reader awareness about PSNN for time series prediction, to highlight some benefits and challenges using PSNN. Possible fields of PSNN applications in comparison with existing methods are presented and future directions are also explored in advantage with the properties of error feedback and recurrent networks.References
Jason Brownlee, [Online]. From: http://machinelearningmastery.com/ time-series-forecasting/, [Accessed on 2 December 2016].
A. J. Hussain, D. Al-Jumeily, H. Al-Askar, and N. Radi, “Regularized dynamic self-organized neural network inspired by the immune algorithm for financial time series prediction,” Neurocomputing, vol. 188, pp. 23–30, 2016.
J. Ballesteros, B. Carbunar, M. Rahman, N. Rishe, and S. S. Iyengar, “Towards safe cities: A mobile and social networking approach,” IEEE Transactions on Parallel & Distributed Systems, vol. 25, no. 9, pp. 2451–2462, 2014.
N. Sapankevych and R. Sankar, “Time series prediction using support vector machines: A survey,” IEEE Computational Intelligence Magazine, vol. 4, no. 2, pp. 24–38, 2009.
R. H. Shumway and D. S. Stoffer, Time Series Analysis and Its Applications. Springer Texts in Statistics, 2011, pp. 1-591.
D. Al-Jumeily, R. Ghazali, and A. Hussain, “Predicting physical time series using dynamic ridge polynomial neural networks,” PLoS One, vol. 9, no. 8, pp. 1–15, 2014.
R. Schwaerzel and T. Bylander, “Predicting currency exchange rates by genetic programming with trigonometric functions and high-order statistics,” in Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation (GECCO '06), ACM, New York, NY, USA, 2006, pp. 955–956.
A. A. Mahdi, A. J. Hussain, and D. Al-Jumeily, “The prediction of nonstationary physical time series using the application of regularization technique in self-organised multilayer perceptrons inspired by the immune algorithm,” in Proceedings of the Developments in E-Systems Engineering (DESE’10), 2010, IEEE Computer Society, Washington, DC, USA, pp. 213–218.
N. Malik, “Artificial Neural Networks and their applications,” in National Conference on ‘Unearthing Technological Developments & their Transfer for Serving Masses, GLA ITM, Mathura, India, 2005.
T. D. Chaudhuri and I. Ghosh, “Artificial neural network and time series modeling based approach to forecasting the exchange rate in a multivariate framework,” Journal of Insurance and Financial Management, vol. 1, no. 5, pp. 92–123, 2016.
B. Oancea and S. C. Ciucu, “Time series forecasting using neural networks,” in Proceedings of the International Conference CKS 2013, Cornell university library, pp, 1402–1408.
R. Ghazali, A. J. Hussain, P. Liatsis, and H. Tawfik, “The application of ridge polynomial neural network to multi-step ahead financial time series prediction,” Neural Computing and Applications, vol. 17, no. 3, pp. 311–323, 2008.
W. Sibanda and P. Pretorius, “Novel application of multi-layer perceptrons (mlp) neural networks to model HIV in South Africa using seroprevalence data from antenatal clinics,” International Journal of Computer Applications, vol. 35, no. 5, pp. 26–31, 2011.
N. A. Mat Isa and W. M. F. W. Mamat, “Clustered-hybrid multilayer perceptron network for pattern recognition application,” Applied Soft Computing, vol. 11, no. 1, pp. 1457–1466, 2011.
X. Yu, L. Tang, Q. Chen, and C. Xu, “Monotonicity and convergence of asynchronous update gradient method for ridge polynomial neural network,” Neurocomputing, vol. 129, pp. 437–444, 2014.
R. Ghazali and D. Al-Jumeily, “Application of pi-sigma neural networks and ridge polynomial neural networks to financial time series prediction,” Artificial Higher Order Neural Networks for Economics and Business IGI Global, pp. 271–273, 2009.
G. P. Zhang, B. E. Patuwo, and M. Y. Hu, “A simulation study of artificial neural networks for nonlinear time-series forecasting,” Computers & Operations Research, vol. 28, no. 4, pp. 381–396, 2001.
Y. M. M. Hassim and R. Ghazali, “Using artificial bee colony to improve functional link neural network training,” Applied Mechanics and Materials, vol. 266, pp. 2102–2108, 2013.
N. A. Husaini, R. Ghazali, N. M. Nawi, and L. H. Ismail, “The effect of network parameters on pi-sigma neural network for temperature forecasting,” International Journal of Modern Physics: Conference Series, vol. 9, pp. 440–447, 2012.
N. A. Husaini, R. Ghazali, N. M. Nawi, L. H. Ismail, M. M. Deris, and T. Herawan, “Pi-Sigma neural network for a one-step-ahead temperature forecasting,” International Journal of Computational Intelligence and Applications, vol. 13, no. 4, pp. 1450023-1-1450023– 16, 2014.
N. Yong and D. Wei, “A hybrid genetic learning algorithm for Pi-sigma neural network and the analysis of its convergence,” in 4th International Conference on Natural Computation, 2008, vol. 3, pp. 19–23.
A. J. Hussain and P. Liatsis, “Recurrent pi-sigma networks for DPCM image coding,” Neurocomputing, vol. 55, no. 1–2, pp. 363–382, 2002.
A. J. Hussain, P. Liatsis, H. Tawfik, A. K. Nagar, and D. Al-Jumeily, “Physical time series prediction using Recurrent Pi-Sigma Neural Networks,” International Journal Artificial Intelligence and Soft Computing, vol. 1, no. 1, pp. 130–145, 2008.
D. Wan, Y. Hu, and X. Ren, “BP neural network with error feedback input research and application,” in 2nd International Conference on Intelligent Computation Technology and Automation, 2009, no. 2, pp. 63–66.
W. Waheeb, R. Ghazali, and T. Herawan, “Ridge polynomial neural network with error feedback for time series forecasting,” PLoS One, vol. 458, pp. 1–34, 2016.
W. Waheeb, R. Ghazali, and T. Herawan, “Time series forecasting using ridge polynomial neural network with error feedback,” in Proceedings of Recent Advances on Soft Computing and Data Mining, SCDM, 2016, pp. 189-200.
R. Ghazali, A. J. Hussain, D. Al-jumeily, P. Lisboa, and B. Street, “Time series prediction using dynamic ridge polynomial neural networks,” in Proceedings of 2nd International Conference on Developments in eSystems Engineering, 2009, pp. 358–367.
N. A. Husaini, R. Ghazali, N. M. Nawi, and L. H. Ismail, “The Jordan pi-sigma neural network for temperature prediction,” in Proceedings of the International Conference on Ubiquitous Computing and Multimedia Applications, UCMA, 2011, pp. 547–558.
R. Ghazali, N. A. Husaini, L. H. Ismail, and N. A. Samsuddin, “An application of Jordan pi-sigma neural network for the prediction of temperature time series signal,” in Recurrent Neural Networks and Soft Computing, M. ElHefnawi and M. Mysara, Eds. INTECH Open Access Publisher, 2012, pp. 275–290.
J. Nayak, D. P. Kanungo, B. Naik, and H. S. Behera, “A higher order evolutionary jordan pi-sigma neural network with gradient descent learning for classification,” in International Conference on High Performance Computing and Applications, 2015, pp. 1–6.
Y. Shin and J. Ghosh, “The pi-sigma network: an efficient higher-order neural network for pattern classification and function approximation,” in Proceedings of the IEEE International Joint Conference on Neural Networks IJCNN-91-Seattle, 1991, pp. 1–18.
R. Ghazali, A. Hussain, and W. El-Deredy, “Application of ridge polynomial neural networks to financial time series prediction,” in Proceedings of the IEEE International Joint Conference on Neural Networks, 2006, pp. 913–920.
A. J. Hussain, A. Knowles, P. J. G. Lisboa, and W. El-Deredy, “Financial time series prediction using polynomial pipelined neural networks,” Expert Systems with Applications, vol. 35, no. 3, pp. 1186– 1199, 2008.
R. N. Yadav, P. K. Kalra, and J. John, “Time series prediction with single multiplicative neuron model,” Applied Soft Computing, vol. 7, no. 4, pp. 1157–1163, 2007.
S. Park, M. J. T. Smith, and R. M. Mersereau, “Target recognition based on directional filter banks and higher-order neural networks,” Digital Signal Processing, vol. 10, no. 4, pp. 297–308, 2000.
D. S. Huang, H. H. S. Ip, K. C. K. Law, and Z. Chi, “Zeroing polynomials using modified constrained neural network approach,” IEEE Transactions on Neural Networks, vol. 16, no. 3, pp. 721–732, 2005.
M. G. Epitropakis, V. P. Plagianakos, and M. N. Vrahatis, “Hardwarefriendly higher-order neural network training using distributed evolutionary algorithms,” Applied Soft Computing, vol. 10, no. 2, pp. 398–408, 2010.
R. Ghazali, A. J. Hussain, and P. Liatsis, “Dynamic ridge polynomial neural network: Forecasting the univariate non-stationary and stationary trading signals,” Expert Systems with Applications, vol. 38, no. 4, pp. 3765–3776, 2011.
R. Ghazali, A. J. Hussain, D. Al-Jumeily, and M. Merabti, “Dynamic ridge polynomial neural networks in exchange rates time series forecasting,” Adaptive and Natural Computing Algorithms, pp. 123– 132, 2007.
Milan Hajek, Neural Networks. University of KwaZulu-Natal, 2005, pp. 1-113.
S. C. Nayak, B. B. Misra, and H. S. Behera, “A pi-sigma higher order neural network for stock index forecasting,” Computational Intelligence in Data Mining, vol. 2, pp. 311–319, 2015.
W. Waheeb and R. Ghazali, “Multi-step time series forecasting using ridge polynomial neural network with error-output feed- backs,” in Proceedings of the International Conference on Soft Computing in Data Science (SCDS), 2016, Springer Singapore, pp. 48–58.
J. L. G. Nielsen, S. Holmgaard, N. Jiang, K. Englehart, D. Farina, and P. Parker, “Enhanced EMG signal processing for simultaneous and proportional myoelectric control,” in Proceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2009, pp. 4335–4338.
Downloads
Published
How to Cite
Issue
Section
License
TRANSFER OF COPYRIGHT AGREEMENT
The manuscript is herewith submitted for publication in the Journal of Telecommunication, Electronic and Computer Engineering (JTEC). It has not been published before, and it is not under consideration for publication in any other journals. It contains no material that is scandalous, obscene, libelous or otherwise contrary to law. When the manuscript is accepted for publication, I, as the author, hereby agree to transfer to JTEC, all rights including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author(s) specifically retain(s):
- All proprietary right other than copyright, such as patent rights
- The right to make further copies of all or part of the published article for my use in classroom teaching
- The right to reuse all or part of this manuscript in a compilation of my own works or in a textbook of which I am the author; and
- The right to make copies of the published work for internal distribution within the institution that employs me
I agree that copies made under these circumstances will continue to carry the copyright notice that appears in the original published work. I agree to inform my co-authors, if any, of the above terms. I certify that I have obtained written permission for the use of text, tables, and/or illustrations from any copyrighted source(s), and I agree to supply such written permission(s) to JTEC upon request.
