Optimization of hybrid-based Collaborative Filtering using Matrix Factorization, Feedforward Neural Network, and XGBoost

Filimantaptius Gulo; Ronsen Purba; Muhammad Fermi Pasha

doi:10.56741/jnest.v5i02.1356

Optimization of hybrid-based Collaborative Filtering using Matrix Factorization, Feedforward Neural Network, and XGBoost

Authors

Filimantaptius Gulo Universitas Mikroskil
Ronsen Purba Universitas Mikroskil Medan https://orcid.org/0000-0002-1711-865X
Muhammad Fermi Pasha Universitas Mikroskil https://orcid.org/0000-0002-9848-8950

DOI:

https://doi.org/10.56741/jnest.v5i02.1356

Keywords:

Collaborative Filtering, Feedforward Neural Networks, Matrix Factorization, Recommender Systems, XGBoost

Abstract

Collaborative filtering recommendation systems are widely used in digital applications; however, they still face challenges such as cold-start and first-rater problems, as well as limited accuracy due to their inability to capture complex user–item relationships. This study proposes a hybrid recommendation model that integrates Matrix Factorization, MLP-based Feedforward Neural Network (MLP) and Extreme Gradient Boosting (XGBoost). Experiments were conducted on two real-world datasets, namely MovieLens (movies) and PT XYZ (hotels), to validate the effectiveness of the proposed approach. The results indicate that the hybrid model consistently outperforms baseline methods such as SGD-based Matrix factorization, Matrix factorization +MLP, and user/item-based Collaborative filtering. Specifically, the integration of nonlinear learning through MLP and feature enhancement via XGBoost significantly improves prediction accuracy while mitigating cold-start and first-rater issues. These findings suggest that hybrid machine learning–based approaches can advance the development of more adaptive, accurate, and personalized recommendation systems.

Downloads

Download data is not yet available.

Author Biographies

Filimantaptius Gulo, Universitas Mikroskil

has a degree in the Information Technology program at the Faculty of Informatics, Universitas Mikroskil, Medan, Indonesia. His current research focuses on the optimization of a hybrid collaborative filtering model, utilizing Matrix Factorization, Feedforward Neural Network, and XGBoost. (email: filmantapgullo@gmail.com).

Ronsen Purba, Universitas Mikroskil Medan

is a Lecturer in the Master of Information Technology (MTI) Program at the Faculty of Informatics, at Universitas Mikroskil, Medan, Indonesia. (email: ronsen@mikroskil.ac.id).

Muhammad Fermi Pasha, Universitas Mikroskil

is a Lecturer in the Master of Information Technology (MTI) Program at the Faculty of Informatics, at Universitas Mikroskil, Medan, Indonesia. (email: muhammad.pasha@mikroskil.ac.id).

References

M. Ibrahim, I. S. Bajwa, N. Sarwar, F. Hajjej and H. A. Sakr, “An Intelligent Hybrid Neural Collaborative Filtering Approach for True Recommendations,” IEEE Access, Vol. 11, p. 64831, 2023,

doi: 10.1109/ACCESS.2023.3289751. DOI: https://doi.org/10.1109/ACCESS.2023.3289751

K. K. Yadav, H. K. Soni, G. Y. Yadav and M. Sharman, “Collaborative Filtering Based Hybrid Recommendation System using Neural Network and Matrix Factorization Techniques,” International Journal of Intelligent Systems and Applications in Engineering, Vol. 12, pp. 695-701, 2023. [Online].

Available: https://ijisae.org/index.php/IJISAE/article/view/4307

Y. Li, J. Xu and M. Yang, “Collaborative Filtering Recommendation Algorithm Based on KNN and Xgboost Hybrid,” Journal of Physics: Conference Series, p. 1-6, 2021,

doi: 10.1088/1742-6596/1748/3/032041. DOI: https://doi.org/10.1088/1742-6596/1748/3/032041

M. Ibrahim, I. S. Bajwa, N. Sarwar, H. A. Waheed, M. Z. Hasanand and M. Z. Hussain, “Improved Hybrid Deep Collaborative Filtering Approach for True Recommendations,” Computers, Materials & Continua, Vol. 74 pp. 5301-5317, 2022,

doi: 10.32604/cmc.2023.032856. DOI: https://doi.org/10.32604/cmc.2023.032856

S. R. Mandalapu, B. Narayanan and S. Putheti, “A Hybrid Collaborative Filtering Mechanism for Product Recommendation System,” Multimedia Tools and Applications, Vol. 83, pp. 12776-12798, 2024. [Online]. Available: https://link.springer.com/article/10.1007/s11042-023-16056-8 DOI: https://doi.org/10.1007/s11042-023-16056-8

C. Lai, D. Liu and K. Lien, “A hybrid of XGBoost and Aspect-based Review Mining with Attention Neural Network for User Preference Prediction,” International Journal of Machine Learning and Cybernetics, Vol. 12, pp. 1203-1217, 2021. [Online]. Available: https://link.springer.com/article/10.1007/s13042-020-01229-w DOI: https://doi.org/10.1007/s13042-020-01229-w

Y. Qingmin, C. Xingyu, “Research on Collaborative Filtering Recommendation Algorithm,” 2022 IEEE 2nd International Conference on Power, Electronics and Computer Applications (ICPECA), p. 741-743, 2022, doi: 10.1109/ICPECA53709.2022.9719134. DOI: https://doi.org/10.1109/ICPECA53709.2022.9719134

K.K Jena, S. K. Bhoi, C. Mallick, S. R. Jena, R. Kumar, H. V. Long and N. T. K. Son, “Neural Model Based Collaborative Filtering for Movie Recommendation System”, International Journal of Information Technology, Vol. 14, pp. 2067-2077, 2022. [Online]. Available: https://link.springer.com/article/10.1007/s41870-022-00858-4 DOI: https://doi.org/10.1007/s41870-022-00858-4

R. Rismala, N. U. Maulidevi and K. Surendro, “Personalized Neural Network-based Aggregation Function in Multi-criteria Collaborative Filtering,” Journal of King Saud University - Computer and Information Sciences, Vol. 36, pp. 1-14, 2024,

doi: 10.1016/j.jksuci.2024.101922. DOI: https://doi.org/10.1016/j.jksuci.2024.101922

G. Behera, S. K. Panda, M. Hsieh and K. Li, “Hybrid Collaborative Filtering using Matrix Factorization and XGBoost for Movie Recommendation,” Computer Standards & Interfaces, Vol. 90, pp. 1-8, 2023, doi: 10.1016/j.csi.2024.103847. DOI: https://doi.org/10.1016/j.csi.2024.103847

S. Rendle, W. Krichene, L. Zhang and John Anderson, “Neural Collaborative Filtering vs. Matrix Factorization Revisited,” Computer Science and Engineering, pp. 240-248, 2020,

doi: 10.1145/3383313.3412488. DOI: https://doi.org/10.1145/3383313.3412488

F. Ullah, B. Zhang, R. U. Khan, T. Chung, M. Attique, K. Khan, S. E. Khediri and S. Jan, “Deep Edu: A Deep Neural Collaborative Filtering for Educational Services Recommendation,” IEEE Access, Vol. 8, pp. 110915-110926, 2020,

doi: 10.1109/ACCESS.2020.3002544. DOI: https://doi.org/10.1109/ACCESS.2020.3002544

K. Singh, S. Dhawan and N. Bali, “An Ensemble Learning Hybrid Recommendation System using Content-Based, Collaborative Filtering, Supervised Learning and Boosting Algorithms,” Automatic Control and Computer Sciences, Vol. 58, pp. 491-505, 2023, doi: 10.3103/S0146411624700615. DOI: https://doi.org/10.3103/S0146411624700615

J. Chen, C. Zhao, Uliji and L. Chen, “Collaborative Filtering Recommendation Algorithm Based on User Correlation and Evolutionary Clustering,” Complex & Intelligent Systems, Vol. 6, pp. 147-156, 2020, doi: 10.1007/s40747-019-00123-5. DOI: https://doi.org/10.1007/s40747-019-00123-5

R. Lara-Cabrera, Á. González-Prieto and F. Ortega, “Deep Matrix Factorization Approach for Collaborative Filtering Recommender Systems,” Applied Sciences, Vol. 10, pp. 1 - 14, 2020,

doi: 10.3390/app10144926. DOI: https://doi.org/10.3390/app10144926

P. Li, S. A. M. Noah and H. M. Sarim, "A Survey on Deep Neural Networks in Collaborative Filtering Recommendation Systems," 2024. [Online]. Available: https://arxiv.org/abs/2412.01378.

L. Zou, L. Xia, Y. Gu, X. Zhao, W. Liu, J. X. Huang and D. Yin, “Neural Interactive Collaborative Filtering,” SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Retrieval, p. 749 - 758, 2020,

doi: 10.1145/3397271.3401181. DOI: https://doi.org/10.1145/3397271.3401181

H. Chen, F. Qian, C. Liu, Y. Zhang, H. Su, and S. Zhao, "Training Robust Deep Collaborative Filtering Models via Adversarial Noise Propagation," ACM Transactions on Information Systems, vol. 42, pp 1-27, 2023, doi: 10.1145/3589000. DOI: https://doi.org/10.1145/3589000

W. Lin, M. Zhu, X. Zhou, R. Zhang, X. Zhao, S. Shen, and L. Sun, "A Deep Neural Collaborative Filtering Based Service Recommendation Method with Multi-Source Data for Smart Cloud-Edge Collaboration Applications," Tsinghua Science and Technology, vol. 29, pp. 897−910, 2024,

doi: 10.26599/TST.2023.9010050. DOI: https://doi.org/10.26599/TST.2023.9010050

Z. Shahbazi, D. Hazra, S. Park and Y. C. Byun, “Toward Improving the Prediction Accuracy of Product Recommendation System Using Extreme Gradient Boosting and Encoding Approaches,” Symmetry, Vol. 12, p. 1 - 20, 2020, doi: 10.3390/sym12091566. DOI: https://doi.org/10.3390/sym12091566

M. Nalluri, M. Pentela and N. R. Eluri, “A Scalable Tree Boosting System: XG Boost,” International Journal of Research Studies in Science, Engineering and Technology, Vol. 7, p. 36 - 51, 2020,

doi: 10.22259/2349-476X.0712005.

D. Deenadayalan and A. Kangaiammal, "User Feature Similarity Supported Collaborative Filtering for Page Recommendation Using Hybrid Shuffled Frog Leaping Algorithm," International Journal of Intelligent Engineering and Systems, vol.16, pp. 301-313, 2023.

doi: 10.22266/ijies2023.0228.27. DOI: https://doi.org/10.22266/ijies2023.0228.27

A. N. Ngaffo and Z. Choukair, “A Deep Neural Network-based Collaborative Filtering using a Matrix Factorization with a Twofold Regularization”, Neural Computing and Applications, Vol 34, pp. 6991-7003, 2021, doi: 10.1007/s00521-021-06831-9. DOI: https://doi.org/10.1007/s00521-021-06831-9

Y. Mao and C. Ye, “User Identification Across Online Social Networks Based on Gated Multi-feature Extraction,” Alexandria Engineering Journal, pp. 211 - 221, 2024,

doi: 10.1016/j.aej.2024.06.059. DOI: https://doi.org/10.1016/j.aej.2024.06.059