Hybrid Feature Selection Using Secretary Bird Optimization and Decision Tree Classifier
Abstract
Feature selection is one of the most crucial concepts of learning when constructing a machine learning algorithm. This paper proposes a new technique of using a blend of Secretary Bird Optimization (SBO) and Decision Tree for feature selection. SBO, with the consideration of hunting strategy of called secretary bird, can successfully search the space and find feature subsets. The first proposed framework involves identifying the relevant features by applying SBO then secondly deciding on a ranking of the features by using a decision tree model classifier. Evaluation is based on the most famous Iris dataset with the application of the 5-fold cross-validation to increase the reliability of the results. It is shown that the SBO-based approach succeeds in both objectives, and the average classification accuracy is equal to 0.9550 ± 0.0316, while the baseline selection methods have higher values of loss. This result has unveiled a theoretical and practical potential for future works that seek to combine metaheuristic optimization with decision trees and interpretability of selected features and machine learning models.
References
Abualigah, L. (2025). Particle swarm optimization: Advances, applications, and experimental insights. Computers, Materials & Continua, 82(2). https://10.32604/cmc.2025.060765
Ahmed, N. T., Mohialden, Y. M., & Abdulrazzaq, D. R. (2018). A new method for self-adaptation of genetic algorithms operators. International Journal of Civil Engineering and Technology, 9(11), 1279–1285.
Aljaidi, M., Jangir, P., Agrawal, S. P., Pandya, S. B., Parmar, A., Anbarkhan, S. H., & Abualigah, L. (2025). A quasi affine transformation evolution algorithm with evolution matrix selection operation for parameter estimation of proton exchange membrane fuel cells. Scientific Reports, 15(1), 1662. https://doi.org/10.1038/s41598-024-83538-6
Allgaier, J., & Pryss, R. (2024). Cross-validation visualized: A narrative guide to advanced methods. Machine Learning and Knowledge Extraction, 6(2), 1378–1388. https://doi.org/10.3390/make6020065
Chandrashekar, G., & Sahin, F. (2014). A survey on feature selection methods. Computers & Electrical Engineering, 40(1), 16–28. https://doi.org/10.1016/j.compeleceng.2013.11.024
Chen, F., Ye, S., Wang, J., & Luo, J. (2025). Multi-strategy improved binary secretarial bird optimization algorithm for feature selection. Mathematics, 13(4), 668. https://doi.org/10.3390/math13040668
Chen, P., Xiong, H., Cao, J., Cui, M., Hou, J., & Guo, Z. (2025). Predicting postoperative adhesive small bowel obstruction in infants under 3 months with intestinal malrotation: A random forest approach. Jornal de Pediatria. https://doi.org/10.1016/j.jped.2024.11.011
Fu, Y., Liu, D., Chen, J., & He, L. (2024). Secretary bird optimization algorithm: A new metaheuristic for solving global optimization problems. Artificial Intelligence Review, 57(5), 123. https://doi.org/10.1007/s10462-024-10729-y
Ghosh, S., Baranowski, E. S., Biehl, M., Arlt, W., Tino, P., & Bunte, K. (2025). Interpretable modeling and visualization of biomedical data. Neurocomputing, 626. https://doi.org/10.1016/j.neucom.2025.129405
Ghosh, S., Frase, H., Williams, A., Luger, S., Röttger, P., Barez, F., ... & Vanschoren, J. (2025). Ailuminate: Introducing v1.0 of the AI risk and reliability benchmark from MLCommons. arXiv preprint arXiv:2503.05731. https://doi.org/10.48550/arXiv.2503.05731
Grandhi, A., & Singh, S. K. (2025). Interrelated dynamic biased feature selection and classification model using enhanced gorilla troops optimizer for intrusion detection. Alexandria Engineering Journal, 114, 312–330. https://doi.org/10.1016/j.aej.2024.10.100
Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3(Mar), 1157–1182.
Hariyani, G. (2025). Enhancing sustainable AI with efficient machine learning models: An iris classification perspective. Vidhyayana-An International Multidisciplinary Peer-Reviewed E-Journal, 10(si4).
Hrizi, O., Gasmi, K., Alyami, A., Alkhalil, A., Alrashdi, I., Alqazzaz, A., ... & Yahyaoui, S. (2025). Federated and ensemble learning framework with optimized feature selection for heart disease detection. AIMS Mathematics, 10(3), 7290–7318. https://doi.org/10.3934/math.2025334
Huang, P., Yang, J., Zhao, D., Ran, T., Luo, Y., Yang, D., ... & Chen, C. (2025). Machine learning–based prediction of early complications following surgery for intestinal obstruction: Multicenter retrospective study. Journal of Medical Internet Research, 27, e68354. https://doi.org/10.2196/68354
Huang, Y., Chen, G., Gou, J., Fan, Z., & Liao, Y. (2025). A hybrid feature selection and aggregation strategy-based stacking ensemble technique for network intrusion detection. Applied Intelligence, 55(1), 28. https://doi.org/10.1007/s10489-024-06015-7
Jagdale, V. V., Khanaj, T. P., Merat, R. R., Patil, Y. A., & Tamkhade, J. (2024). Secretary bird optimization algorithm. Journal of Novel Research and Innovative Development, 2(12). https://tijer.org/jnrid/papers/JNRID2412003.pdf
Jiang, X., Zheng, C., Zhuo, Y., Kong, X., Ge, Z., Song, Z., & Xie, M. (2025). Advancing industrial data augmentation in AIGC era: From foundations to frontier applications. IEEE Transactions on Instrumentation and Measurement. https://doi.org/10.1109/TIM.2025.3572162
Johnson, J., Rajuroy, A., & Liang, W. (2025). Hybridizing mixing genetic algorithm with parallel metaheuristic ensembles for enhanced traveling salesman problem solving.
Jović, A., Brkić, K., & Bogunović, N. (2015, May). A review of feature selection methods with applications. In 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) (pp. 1200–1205). IEEE.
Kernbach, J. M., & Staartjes, V. E. (2020). Machine learning-based clinical prediction modeling—a practical guide for clinicians. arXiv preprint arXiv:2006.15069. https://doi.org/10.48550/arXiv.2006.15069
Khan, Z. A., Ullah, F. U. M., Yar, H., Ullah, W., Khan, N., Kim, M. J., & Baik, S. W. (2025). Optimized cross-module attention network and medium-scale dataset for effective fire detection. Pattern Recognition, 161, 111273. https://doi.org/10.1016/j.patcog.2024.111273
Kohavi, R. (1995, August). A study of cross-validation and bootstrap for accuracy estimation and model selection. In IJCAI (Vol. 14, No. 2, pp. 1137–1145).
Lind, E. (2025). A comparison of dimensionality reduction methods’ impact on model accuracy: Mitigating the curse of dimensionality in estimating retail customers' share of wallet.
Mallidi, S. K. R., & Ramisetty, R. R. (2025). Bowerbird courtship-inspired feature selection for efficient high-dimensional data analysis using a novel meta-heuristic. Discover Computing, 28(1), 6. https://doi.org/10.1007/s10791-025-09497-2
Mohammed, M. H., Kadhim, M. N., Al-Shammary, D., & Ibaida, A. (2025). EEG-based emotion detection using Roberts similarity and PSO feature selection. IEEE Access.
Mohialden, Y. M., & Hussien, N. M. (2025). A novel implementation of the secretary bird optimization algorithm for solving quadratic equations. In National Conference on New Trends in Information and Communications Technology Applications (pp. 207–221). Springer. http://dx.doi.org/10.1007/978-3-031-87076-7_13
Molnar, C. (2020). Interpretable machine learning. Lulu.com.
Nematzadeh, H., Mani, J., Nematzadeh, Z., Akbari, E., & Mohamad, R. (2025). Distance-based mutual congestion feature selection with genetic algorithm for high-dimensional medical datasets. Neural Computing and Applications, 1–16. https://doi.org/10.48550/arXiv.2407.15611
Qin, S., Liu, J., Bai, X., & Hu, G. (2024). A multi-strategy improvement secretary bird optimization algorithm for engineering optimization problems. Biomimetics, 9(8), 478. https://doi.org/10.3390/biomimetics9080478
Rezaie, F., Panahi, M., Kalantari, Z., Choi, M., & Kim, H. (2025). Enhancing detection of inundated areas using novel hybrid PolSAR-metaheuristic-deep learning models. Available at SSRN 5248912. https://dx.doi.org/10.2139/ssrn.5248912
Rostami, M., Berahmand, K., Nasiri, E., & Forouzandeh, S. (2021). Review of swarm intelligence-based feature selection methods. Engineering Applications of Artificial Intelligence, 100, 104210. https://doi.org/10.1016/j.engappai.2021.104210
Sadeghian, Z., Akbari, E., Nematzadeh, H., & Motameni, H. (2025). A review of feature selection methods based on meta-heuristic algorithms. Journal of Experimental & Theoretical Artificial Intelligence, 37(1), 1–51. https://doi.org/10.1080/0952813X.2023.2183267
Sanches, H. E., Possebom, A. T., & Aylon, L. B. R. (2025, April). AREF-Argumentative rule-based explanatory framework. In 2025 IEEE International Systems Conference (SysCon) (pp. 1–8). IEEE.
Sanjalawe, Y., Al-E’mari, S., Abualhaj, M., Makhadmeh, S. N., Alsharaiah, M. A., & Hijazi, D. H. (2025). Recent advances in secretary bird optimization algorithm, its variants and applications. Evolutionary Intelligence, 18(3), 1–32. http://dx.doi.org/10.1007/s12065-025-01054-6
Sanjaya, W. (2016). Strategi pembelajaran berorientasi standar proses pendidikan. Kencana.
Selvaraj, P., & Sivaprakash, S. (2025). Feature extraction and feature selection in medical images. In Intelligent Computing Techniques in Biomedical Imaging (pp. 83–97). Academic Press.
Settelmeier, J., Goetze, S., Boshart, J., Fu, J., Khoo, A., Steiner, S. N., ... & Wollscheid, B. (2025). MultiOmicsAgent: Guided extreme gradient-boosted decision trees-based approaches for biomarker-candidate discovery in multiomics data. Journal of Proteome Research. https://doi.org/10.1021/acs.jproteome.4c01066
Sharma, A., Kumar, D., & Gupta, G. (2025). Enhanced iris recognition with histogram cut selection and genetic algorithms for robust classification. Procedia Computer Science, 258, 2846–2859.
Shin, T. (2023). Understanding feature importance in machine learning. Accessed on, 26, 2024.
Shirali, M., Hatamiafkoueieh, J., Razoumny, Y., & Olegovich, D. D. (2025). Accuracy enhancement in land subsidence prediction using LightGBM and metaheuristic optimization. Earth Science Informatics, 18(3), 435. http://dx.doi.org/10.1007/s12145-025-01929-3
Sony, R., Farmanifard, P., Alzwairy, H., Shukla, N., & Ross, A. (2025). Benchmarking foundation models for zero-shot biometric tasks. arXiv preprint arXiv:2505.24214. https://doi.org/10.48550/arXiv.2505.24214
Tared, S., Roubehie Fissa, M., Khaouane, L., & Hanini, S. (2025). Prediction of diabetes using hybrid support vector machines with artificial bee colony. International Journal of Diabetes in Developing Countries, 1–16. http://dx.doi.org/10.1007/s13410-025-01478-x
Wang, L., Sheng, J., Zhang, Q., Song, Y., Zhang, Q., Wang, B., & Zhang, R. (2025). Diagnosis of Alzheimer’s disease using FusionNet with improved secretary bird optimization algorithm for optimal MK-SVM based on imaging genetic data. Cerebral Cortex, bhae498. https://doi.org/10.1093/cercor/bhae498
Xue, B., Zhang, M., Browne, W. N., & Yao, X. (2015). A survey on evolutionary computation approaches to feature selection. IEEE Transactions on Evolutionary Computation, 20(4), 606–626. https://doi.org/10.1109/TEVC.2015.2504420
Zhu, C., Wang, Z., Peng, Y., & Xiao, W. (2025). An improved red-billed blue magpie feature selection algorithm for medical data processing. PLoS ONE, 20(5), e0324866. https://doi.org/10.1371/journal.pone.0324866
Copyright (c) 2025 Journal La Multiapp
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
