Sigmoid Function Implementation Using the Unequal Segmentation of Differential Lookup Table and Second Order Nonlinear Function

Authors

  • Syahrulanuar Ngah Faculty of Computer Systems & Software Engineering, UMP
  • Rohani Abu Bakar Faculty of Computer Systems & Software Engineering, UMP

Keywords:

Differential Look-Up Table, FPGA, Second Order Nonlinear Function, Sigmoid Function,

Abstract

This paper discusses the artificial neural network (ANN) implementation into a field programmable gate array (FPGA). One of the most difficult problem encounters is the complex equation of the activation function namely sigmoid function. The sigmoid function is used as learning function to train the neural network while its derivative is used as a network activation function for specifying the point at which the network should switch to a true state. In order to overcome this problem, two-steps approach which combined the unequal segmentation of the differential look-up table (USdLUT) and the second order nonlinear function (SONF) is proposed. Based on the analysis done, the deviation achieved using the proposed method is 95%. The result obtained is much better than the previous implementation that uses equal segmentation of differential look-up table.

References

H. M. Hasanien, “FPGA implementation of adaptive ANN controller for speed regulation of permanent magnet stepper motor drives,” Energy Convers. Manag., vol. 52, no. 2, pp. 1252–1257, Feb. 2011.

V. Saichand, N. D. M., A. S., and N. Mohankumar, “FPGA Realization of Activation Function for Artificial Neural Networks,” 2008 Eighth Int. Conf. Intell. Syst. Des. Appl., pp. 159–164, Nov. 2008.

A. H. Namin, K. Leboeuf, R. Muscedere, H. Wu, and M. Ahmadi, “Efficient hardware implementation of the hyperbolic tangent sigmoid function,” 2009 IEEE Int. Symp. Circuits Syst., pp. 2117–2120, May 2009.

X. Chen, G. Wang, W. Zhou, S. Chang, and S. Sun, “Efficient Sigmoid Function for Neural Networks Based FPGA Design,” in International Conference on Intelligent Computing, ICIC, 2006, pp. 672–677.

E. Z. Mohammed and H. K. Ali, “Hardware Implementation of Artificial Neural Network Using Field Programmable Gate Array,” Int. J. Comput. Theory Eng., vol. 5, no. 5, pp. 780–783, 2013.

A. Benjemmaa, I. Klabi, M. S. Masmoudi, J. El Ouni, and M. Masmoudi, “Implementations approaches of neural networks lane following system,” in Electrotechnical Conference (MELECON), 2012 16th IEEE Mediterranean, 2012, no. 3, pp. 515–518.

T. Orlowska-Kowalska and M. Kaminski, “FPGA implementation of the multilayer neural network for the speed estimation of the two-mass drive system,” IEEE Trans. Ind. Informatics, vol. 7, no. 3, pp. 436– 445, 2011.

M. T. Tommiska, “Efficient digital implementation of the sigmoid function for reprogrammable logic,” IEE Proc. - Comput. Digit. Tech., vol. 150, no. 6, p. 403, 2003.

B. M. Wilamowski, “Neural network architectures and learning algorithms - How to not get frustrated with neural networks,” Industrial Electronics Magazine, IEEE, vol. 3, no. 4, pp. 56–63, 2009.

J. L. J. Liu and D. L. D. Liang, “A Survey of FPGA-Based Hardware Implementation of ANNs,” 2005 Int. Conf. Neural Networks Brain, vol. 2, pp. 915–918, 2005.

A. Muthuramalingam, S. Himavathi, and E. Srinivasan, “Neural Network Implementation Using FPGA : Issues and Application,” Int. J. Inf. Technol., vol. 4, no. 2, pp. 396–402, 2008.

M. Krips, T. Lammert, and a. Kummert, “FPGA implementation of a neural network for a real-time hand tracking system,” Proc. First IEEE Int. Work. Electron. Des. Test Appl. ’2002, pp. 313–317, 2002.

Z. Xie, “A non-linear approximation of the sigmoid function based on FPGA,” in 2012 IEEE Fifth International Conference on Advanced Computational Intelligence (ICACI), 2012, pp. 221–223.

Y. Ago, Y. Ito, and K. Nakano, “An FPGA implementation for neural networks with the FDFM processor core approach,” Int. J. Parallel, Emergent Distrib. Syst., vol. Volume 28, no. No. 4, pp. 308 – 320, 2012.

R. R. Shrestha, S. Theobald, and F. Nestmann, “Simulation of flood flow in a river system using artificial neural networks,” Hydrol. Earth Syst. Sci., vol. 9, no. 4, pp. 313–321, 2005.

D. Costarelli and R. Spigler, “Approximation results for neural network operators activated by sigmoidal functions.,” Neural networks J. Int. Neural Netw. Soc., vol. 44, pp. 101–106, Aug. 2013.

A. Mohamed, A. Issam, B. Mohamed, and B. Abdellatif, “Parallel and Mixed Hardware Implementation of Artificial Neuron Network on the FPGA Platform,” Int. J. Eng. Technol., vol. 6, no. 5, pp. 2008–2016, 2014.

H. Yonaba, F. Anctil, and V. Fortin, “Comparing Sigmoid Transfer Functions for Neural Network Multistep Ahead Streamflow Forecasting,” J. Hydrol. Eng., vol. 15, no. 4, pp. 275–283, 2010.

A. Y. Shamseldin, A. E. Nasr, and K. M. O. Connor, “Comparison of different forms of the Multi-layer Feed-Forward Neural Network method used for river flow forecasting,” Hydrol. Earth Syst. Sci., vol. 6, no. 4, pp. 671–684, 2002.

T. M. Jamel and B. M. Khammas, “Implementation of a Sigmoid Activation Function For Neural Network Using FPGA,” in 13th Scientific Conference of Al-Ma’moon University College, 2012, no. April.

P. K. Meher, “An optimized lookup-table for the evaluation of sigmoid function for artificial neural networks,” in Proceedings of the 2010 18th IEEE/IFIP International Conference on VLSI and Systemon-Chip, VLSI-SoC 2010, 2010, pp. 91–95.

R. W. Duren, R. J. Marks, P. D. Reynolds, and M. L. Trumbo, “Real-time neural network inversion on the SRC-6e reconfigurable computer,” IEEE Trans. Neural Networks, vol. 18, no. 3, pp. 889–901, 2007.

H. Amin, K. M. Curtis, and B. R. Hayes-Gill, “Piecewise linear approximation applied to nonlinear function of a neural network,” in Circuits, Devices and Systems, IEE Proceedings -, 1997, vol. 144, no. 6, pp. 313–317.

M. Zhang, S. Vassiliadis, J. G. Delgado-Frias, and S. V. and J. G. D. F. M. Zhang, “Sigmoid generators for neural computing using piecewise approximations,” IEEE Trans. Comput., vol. 45, no. 9, pp. 1045–1049, 1996.

K. Leboeuf, A. H. Namin, R. Muscedere, H. Wu, and M. Ahmadi, “High Speed VLSI implementation of the hyperbolic tangent sigmoid function,” 2008 Third Int. Conf. Converg. Hybrid Inf. Technol., pp. 1070–1073, Nov. 2008.

C. Lin and J. Wang, “A digital circuit design of hyperbolic tangent sigmoid function for neural networks,” 2008 IEEE Int. Symp. Circuits Syst., pp. 856–859, May 2008.

J. E. Volder, “The CORDIC Trigonometric Computing Technique,” Electron. Comput. IRE Trans., vol. EC-8, no. 3, pp. 330–334, 1959.

E. Deprettere, P. Dewilde, and R. Udo, “Pipelined cordic architectures for fast VLSI filtering and array processing,” ICASSP ’84. IEEE Int. Conf. Acoust. Speech, Signal Process., vol. 9, pp. 250 – 253, 1984.

Y. H. Hu and S. Naganathan, “Novel implementation of a chirp Ztransform using a CORDIC processor,” IEEE Trans. Acoust., vol. 38, no. 2, pp. 352–354, 1990.

S. Ngah, R. A. Bakar, A. Embong, and S. Razali, “Two-Steps Implementation of Sigmoid Function for Artificial Neural Network in Field Programmable Gate Array,” ARPN J. Eng. Appl. Sci., 2015.

I. Aybay, “Classification of neural network hardware,” Neural Netw. World, IDG Co., vol. 6, no. 1, pp. 11 – 29, 1996.

Downloads

Published

2017-09-01

How to Cite

Ngah, S., & Abu Bakar, R. (2017). Sigmoid Function Implementation Using the Unequal Segmentation of Differential Lookup Table and Second Order Nonlinear Function. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 9(2-8), 103–108. Retrieved from https://jtec.utem.edu.my/jtec/article/view/2637

Issue

Vol. 9 No. 2-8: Towards Creative and Innovative Technologies for Humankind II

Section

Articles

License

TRANSFER OF COPYRIGHT AGREEMENT

The manuscript is herewith submitted for publication in the Journal of Telecommunication, Electronic and Computer Engineering (JTEC). It has not been published before, and it is not under consideration for publication in any other journals. It contains no material that is scandalous, obscene, libelous or otherwise contrary to law. When the manuscript is accepted for publication, I, as the author, hereby agree to transfer to JTEC, all rights including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author(s) specifically retain(s):

  1. All proprietary right other than copyright, such as patent rights
  2. The right to make further copies of all or part of the published article for my use in classroom teaching
  3. The right to reuse all or part of this manuscript in a compilation of my own works or in a textbook of which I am the author; and
  4. The right to make copies of the published work for internal distribution within the institution that employs me

I agree that copies made under these circumstances will continue to carry the copyright notice that appears in the original published work. I agree to inform my co-authors, if any, of the above terms. I certify that I have obtained written permission for the use of text, tables, and/or illustrations from any copyrighted source(s), and I agree to supply such written permission(s) to JTEC upon request.