An Enhanced Random Linear Oracle Ensemble Method using Feature Selection Approach based on Naïve Bayes Classifier

Authors

  • Boon Pin Ooi School of Mechatronic Engineering, University Malaysia Perlis, 02600 Arau, Perlis, Malaysia
  • Norasmadi Abdul Rahim School of Mechatronic Engineering, University Malaysia Perlis, 02600 Arau, Perlis, Malaysia
  • Ammar Zakaria School of Mechatronic Engineering, University Malaysia Perlis, 02600 Arau, Perlis, Malaysia
  • Maz Jamilah Masnan Institute of Engineering Mathematics, University Malaysia Perlis, 02600 Arau, Perlis, Malaysia
  • Shazmin Aniza Abdul Shukor School of Mechatronic Engineering, University Malaysia Perlis, 02600 Arau, Perlis, Malaysia

Keywords:

Ensemble, Feature Selection, Naïve Bayes, Pattern Recognition, Random Linear Oracle,

Abstract

Random Linear Oracle (RLO) ensemble replaced each classifier with two mini-ensembles, allowing base classifiers to be trained using different data set, improving the variety of trained classifiers. Naïve Bayes (NB) classifier was chosen as the base classifier for this research due to its simplicity and computational inexpensive. Different feature selection algorithms are applied to RLO ensemble to investigate the effect of different sized data towards its performance. Experiments were carried out using 30 data sets from UCI repository, as well as 6 learning algorithms, namely NB classifier, RLO ensemble, RLO ensemble trained with Genetic Algorithm (GA) feature selection using accuracy of NB classifier as fitness function, RLO ensemble trained with GA feature selection using accuracy of RLO ensemble as fitness function, RLO ensemble trained with t-test feature selection, and RLO ensemble trained with Kruskal-Wallis test feature selection. The results showed that RLO ensemble could significantly improve the diversity of NB classifier in dealing with distinctively selected feature sets through its fusionselection paradigm. Consequently, feature selection algorithms could greatly benefit RLO ensemble, with properly selected number of features from filter approach, or GA natural selection from wrapper approach, it received great classification accuracy improvement, as well as growth in diversity.

Downloads

Download data is not yet available.

Downloads

Published

2017-12-30

How to Cite

Ooi, B. P., Abdul Rahim, N., Zakaria, A., Masnan, M. J., & Abdul Shukor, S. A. (2017). An Enhanced Random Linear Oracle Ensemble Method using Feature Selection Approach based on Naïve Bayes Classifier. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 9(4), 69–77. Retrieved from https://jtec.utem.edu.my/jtec/article/view/1816