Parametric Feature Selection for an Enhanced Random Linear Oracle Ensemble Method

Authors

  • B. P. Ooi School of Mechatronic Engineering, University Malaysia Perlis, 02600 Arau, Perlis, Malaysia.
  • N. Abdul Rahim School of Mechatronic Engineering, University Malaysia Perlis, 02600 Arau, Perlis, Malaysia.
  • A. Zakaria School of Mechatronic Engineering, University Malaysia Perlis, 02600 Arau, Perlis, Malaysia.
  • M. J. Masnan Institute of Engineering Mathematics, University Malaysia Perlis, 02600 Arau, Perlis, Malaysia.
  • S. A. Abdul Shukor School of Mechatronic Engineering, University Malaysia Perlis, 02600 Arau, Perlis, Malaysia.
  • Paulraj M. P. Sri Ramakrishna Institute of Technology, Perur Chettipalayam, Pachapalayam, Coimbatore – 641010, Tamilnadu, India.

Keywords:

Ensemble, Feature Selection, Naïve Bayes, Pattern Recognition, Random Linear Oracle,

Abstract

Random Linear Oracle (RLO) utilized classifier fusion-selection approach by replacing each classifier with two mini-ensembles separated by an oracle. This research investigates the effect of t-test feature selection toward classification performance of RLO ensemble method. Naïve Bayes (NB) classifier has been chosen as the base classifier due to its elegant simplicity and computationally inexpensive. Experiments were carried out using 30 data sets from UCI Machine Learning Repository. The results showed that RLO ensemble could greatly improve the ability of NB classifier in dealing with more data with different properties. Moreover, RLO ensemble receives benefits from feature selection algorithm, with a properly selected number of features from ttest, the performance of ensemble can be improved.

References

F. Y. Shih, Image Processing and Pattern Recognition: Fundamentals and Techniques. 2010.

L. I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms: Second Edition. 2014.

F. Roli, G. Giacinto, and G. Vernazza, “Methods for Designing Multiple Classifier Systems,” Mult. Classif. Syst., vol. 1857, pp. 78–87, 2000.

T. G. Dietterich, “Ensemble Methods in Machine Learning,” Mult. Classif. Syst., vol. 1857, pp. 1–15, 2000.

I. Rish, “An Empirical Study of the Naive Bayes Classifier,” IJCAI 2001 Work. Empir. methods Artif. Intell., pp. 41–46, 2001.

L. Kuncheva and J. Rodriguez, “Classifier Ensembles with a Random Linear Oracle,” IEEE Trans. Knowl. Data Eng., vol. 19, no. 4, pp. 500– 508, Apr. 2007.

J. Rodrïguez and L. Kuncheva, “Naive Bayes Ensembles with a Random Oracle,” Mult. Classif. Syst., pp. 450–458, 2007.

C. Pardo and J. Rodríguez, “Random Oracles for Regression Ensembles,” Ensembles Mach., pp. 181–199, 2011.

K. Li and L. Hao, “Naive Bayes Ensemble Learning based on Oracle Selection,” Control and Decision Conference, 2009. CCDC ’09. Chinese. pp. 665–670, 2009.

A. Ahmad and G. Brown, “A Study of Random Linear Oracle Ensembles,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 5519 LNCS, pp. 488– 497, 2009.

G. Armano and N. Hatami, “Random Prototype-based Oracle for Selection-fusion Ensembles,” 2010 20th Int. Conf. Pattern Recognit., pp. 77–80, 2010.

A. K. Jain, R. P. W. Duin, and J. Mao, “Statistical Pattern Recognition: A review,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 1, pp. 4–37, 2000.

I. Guyon and A. Elisseeff, “An Introduction to Variable and Feature Selection,” J. Mach. Learn. Res., vol. 3, no. 3, pp. 1157–1182, 2003.

M. Ghaemi and M.-R. Feizi-Derakhshi, “Feature Selection using Forest Optimization Algorithm,” Pattern Recognit., vol. 60, pp. 121–129, 2016.

C. M. Bishop, Pattern Recognition and Machine Learning, vol. 4, no. 4. 2006.

D. Caprette, “‘Student’s’ t Test (For Independent Samples),” Rice University, 1999. [Online]. Available: http://www.ruf.rice.edu/~bioslabs/tools/stats/ttest.html. [Accessed: 30- Mar-2016].

N. Zhou and L. Wang, “A Modified T-test Feature Selection Method and Its Application on the HapMap Genotype Data,” Genomics, Proteomics Bioinforma., vol. 5, no. 3–4, pp. 242–249, 2007.

M. N. Aishah, M. Maz Jamilah, M. A. Nor Azrita, M. N. Nor Fashihah, and S. Syafawati, Engineering Statistics. Kedah: Institut Matematik Kejuruteraan, Fotocopy Ent., 2016.

J. Demšar, “Statistical Comparisons of Classifiers over Multiple Data Sets,” J. Mach. Learn. Res., vol. 7, pp. 1–30, 2006.

N. Nachar, “The Mann-Whitney U: A Test for Assessing Whether Two Independent Samples Come from the Same Distribution,” Tutor. Quant. Methods Psychol., vol. 4, no. 1, pp. 13–20, 2008.

K. Bache and M. Lichman, “UCI Machine Learning Repository,” University of California Irvine School of Information, 2013. [Online]. Available: http://www.ics.uci.edu/~mlearn/MLRepository.html

Downloads

Published

2018-05-29

How to Cite

Ooi, B. P., Abdul Rahim, N., Zakaria, A., Masnan, M. J., Abdul Shukor, S. A., & M. P., P. (2018). Parametric Feature Selection for an Enhanced Random Linear Oracle Ensemble Method. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 10(1-13), 111–116. Retrieved from https://jtec.utem.edu.my/jtec/article/view/4133

Issue

Vol. 10 No. 1-13: Exploring Man-Machines Compliance in Signal Processing and Artificial Intelligence

Section

Articles

License

TRANSFER OF COPYRIGHT AGREEMENT

The manuscript is herewith submitted for publication in the Journal of Telecommunication, Electronic and Computer Engineering (JTEC). It has not been published before, and it is not under consideration for publication in any other journals. It contains no material that is scandalous, obscene, libelous or otherwise contrary to law. When the manuscript is accepted for publication, I, as the author, hereby agree to transfer to JTEC, all rights including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author(s) specifically retain(s):

  1. All proprietary right other than copyright, such as patent rights
  2. The right to make further copies of all or part of the published article for my use in classroom teaching
  3. The right to reuse all or part of this manuscript in a compilation of my own works or in a textbook of which I am the author; and
  4. The right to make copies of the published work for internal distribution within the institution that employs me

I agree that copies made under these circumstances will continue to carry the copyright notice that appears in the original published work. I agree to inform my co-authors, if any, of the above terms. I certify that I have obtained written permission for the use of text, tables, and/or illustrations from any copyrighted source(s), and I agree to supply such written permission(s) to JTEC upon request.


Most read articles by the same author(s)