A Multimodal Graph-Based Recommendation Architecture for Vendor Discovery in Context-Aware Event Planning

Nuryani Mawar Putri; Irwansyah Saputra

doi:10.56741/jnest.v5i01.1050

A Multimodal Graph-Based Recommendation Architecture for Vendor Discovery in Context-Aware Event Planning

Authors

Nuryani Mawar Putri Universitas Nusa Mandiri https://orcid.org/0009-0009-1668-5351
Irwansyah Saputra Universitas Nusa Mandiri https://orcid.org/0000-0003-1851-1821

DOI:

https://doi.org/10.56741/jnest.v5i01.1050

Keywords:

Context-Aware Vendor Selection, Event Management Planning, Event Planning Systems, Graph Attention Networks, Knowledge Graph Reasoning, Local Large Language Model, Multimodal Recommendation System

Abstract

Context-aware vendor recommendation remains a critical challenge in dynamic event planning systems, particularly in multilingual and temporally sensitive markets such as Indonesia. This paper presents MAMRS (Multi-Agent Multimodal Recommendation System), a novel graph-based architecture that integrates semantic similarity, temporal availability, and user interaction history within a heterogeneous knowledge graph processed using Graph Attention Networks (GAT). The system uniquely combines CSR-aware attention mechanisms with a local LLM (Mistral-7B) to deliver explainable, sustainability-focused recommendations while preserving data privacy. MAMRS introduces three key innovations: (1) A GAT-based reasoning layer that dynamically weights vendor relevance using both structural relationships and corporate social responsibility (CSR) scores. (2) A fusion scoring engine that optimizes for semantic, temporal, behavioral, and sustainability constraints (achieving 93.4% CSR compliance in output, validated through grid search and rule-based scoring), (3) A locally deployed LLM that generates natural-language justifications with 18.7% higher BLEU scores, complemented by ROUGE-L (0.44) and METEOR (0.39), and validated through a small-scale Likert-scale user study (n=5). Evaluation on 250+ real user queries and 1,200+ vendor profiles from BuatEvent.id demonstrates that MAMRS achieves: (1) 21.3% improvement in top-3 accuracy over dense retrieval baselines (with p < 0.05, two-tailed t-test). (2) 1.6s average latency (1.9s p95) on local infrastructure. (3) 88.7% recall@5 in cold-start vendor scenarios, including new vendors with sparse metadata. This paragraph of the first footnote will contain the date on which you submitted your paper for review, which is populated by IEEE. It is IEEE style to display support information, including sponsor and financial support acknowledgments, here rather than in an acknowledgment section at the end of the article. The current implementation supports Indonesian and English queries, with future work planned for additional ASEAN languages and domains such as education and healthcare. These results position MAMRS as an effective solution for regulation-sensitive, multilingual event-planning platforms that require auditable, trustworthy, and scalable recommendation rationales.

Downloads

Download data is not yet available.

Author Biographies

Nuryani Mawar Putri, Universitas Nusa Mandiri

holds a Bachelor’s degree in Information Systems from Universitas Nusa Mandiri, Jakarta, and is currently pursuing a Master’s in Computer Science at the same institution. Her interests include artificial intelligence, system development, and digital transformation. She serves as Director of PT Inti Jasa Kreatif, focusing on smart event platforms, digital innovation, recommender systems, NLP, and machine learning applications (email: 14240017@nusamandiri.ac.id).

Irwansyah Saputra, Universitas Nusa Mandiri

is a lecturer at the Faculty of Computer Science, Universitas Nusa Mandiri, Jakarta. He earned his Doctoral degree from IPB University, focusing on blockchain technology. His interests include distributed systems, cybersecurity, and emerging technologies. He has published blockchain works and actively contributes to research in secure and decentralized computing systems. (email: irwansyah.iys@nusamandiri.ac.id).

References

W. Gao, “Research on optimization of library book recommendation system based on the collaborative fusion of transformer architecture and adaptive extreme learning machine,” Systems and Soft Computing, vol. 7, p. 200287, Dec. 2025, doi: 10.1016/j.sasc.2025.200287. DOI: https://doi.org/10.1016/j.sasc.2025.200287

A. H. Sutawidjaya and L. C. Nawangsari, “Integrated Project Management Planning and Event Management in the Service Industry,” in Proceedings of the 4th International Conference on Management, Economics and Business (ICMEB 2019), Paris, France: Atlantis Press, 2020. doi: 10.2991/aebmr.k.200205.012. DOI: https://doi.org/10.2991/aebmr.k.200205.012

J. Qu et al., “A hybrid time-and-event-driven strategy for integrated community energy system planning,” Appl Energy, vol. 384, p. 125274, Apr. 2025, doi: 10.1016/j.apenergy.2025.125274. DOI: https://doi.org/10.1016/j.apenergy.2025.125274

H.-R. Xu, N. Zhang, Z.-Y. Yin, and P. G. A. Njock, “Multimodal framework integrating multiple large language model agents for intelligent geotechnical design,” Autom Constr, vol. 176, p. 106257, Aug. 2025, doi: 10.1016/j.autcon.2025.106257. DOI: https://doi.org/10.1016/j.autcon.2025.106257

N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Stroudsburg, PA, USA: Association for Computational Linguistics, 2019, pp. 3980–3990. doi: 10.18653/v1/D19-1410. DOI: https://doi.org/10.18653/v1/D19-1410

G. Cheng, Yaxi Wu, D. Cao, Q. You, M. Yang, and Z. Wang, “A survey of multimodal retrieval: from traditional analytics to deep learning,” in Fourth International Conference on Digital Signal and Computer Communications (DSCC 2024), T. A. Rashid and Y. Yue, Eds., SPIE, Jul. 2024, p. 54. doi: 10.1117/12.3033335. DOI: https://doi.org/10.1117/12.3033335

N. Reimers and I. Gurevych, “Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Stroudsburg, PA, USA: Association for Computational Linguistics, 2020, pp. 4512–4525. doi: 10.18653/v1/2020.emnlp-main.365. DOI: https://doi.org/10.18653/v1/2020.emnlp-main.365

J. Lin, Y. Wan, J. Xu, and X. Qi, “Semantic graph neural network with multi-measure learning for semi-supervised classification,” Eng Appl Artif Intell, vol. 140, p. 109647, Jan. 2025, doi: 10.1016/j.engappai.2024.109647. DOI: https://doi.org/10.1016/j.engappai.2024.109647

N. Khan and D. S. Sisodia, “Exploiting diffusion-based structured learning for item interactions representations in multimodal recommender systems,” Inf Process Manag, vol. 62, no. 3, p. 104075, May 2025, doi: 10.1016/j.ipm.2025.104075. DOI: https://doi.org/10.1016/j.ipm.2025.104075

K. Santhanam, O. Khattab, J. Saad-Falcon, C. Potts, and M. Zaharia, “ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction,” Dec. 2021. DOI: https://doi.org/10.18653/v1/2022.naacl-main.272

R. Nogueira, W. Yang, K. Cho, and J. Lin, “Multi-Stage Document Ranking with BERT,” Oct. 2019.

X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, and M. Wang, “LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation,” Feb. 2020. DOI: https://doi.org/10.1145/3397271.3401063

R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec, “Graph Convolutional Neural Networks for Web-Scale Recommender Systems,” Jun. 2018, doi: 10.1145/3219819.3219890. DOI: https://doi.org/10.1145/3219819.3219890

P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph Attention Networks,” Oct. 2017.

B. Wu, T. Shi, L. Zhong, Y. Zhang, and Y. Ye, “Graph-coupled time interval network for sequential recommendation,” Inf Sci (N Y), vol. 648, p. 119510, Nov. 2023, doi: 10.1016/j.ins.2023.119510. DOI: https://doi.org/10.1016/j.ins.2023.119510

L. Mei, S. Mo, Z. Yang, and C. Chen, “A Survey of Multimodal Retrieval-Augmented Generation,” Mar. 2025.

P. Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” May 2020.

Q. Ye, M. Axmed, R. Pryzant, and F. Khani, “Prompt Engineering a Prompt Engineer,” Nov. 2023. DOI: https://doi.org/10.18653/v1/2024.findings-acl.21

L. Wu et al., “A Survey on Large Language Models for Recommendation,” May 2023.

Q. Peng, B. Cao, X. Xie, H. Ye, J. Liu, and Z. Li, “LLMSRec: Large language model with service network augmentation for web service recommendation,” Knowl Based Syst, p. 113710, May 2025, doi: 10.1016/j.knosys.2025.113710. DOI: https://doi.org/10.1016/j.knosys.2025.113710

Z. Wang et al., “Diversity-enhanced conversational recommendation via multi-agent reinforcement learning,” Knowl Inf Syst, May 2025, doi: 10.1007/s10115-025-02455-w. DOI: https://doi.org/10.1007/s10115-025-02455-w

A. Valencia-Arias, H. Uribe-Bedoya, J. D. González-Ruiz, G. S. Santos, E. C. Ramírez, and E. M. Rojas, “Artificial intelligence and recommender systems in e-commerce. Trends and research agenda,” Intelligent Systems with Applications, vol. 24, p. 200435, Dec. 2024, doi: 10.1016/j.iswa.2024.200435. DOI: https://doi.org/10.1016/j.iswa.2024.200435

F. Manessi, A. Rozza, and M. Manzo, “Dynamic graph convolutional networks,” Pattern Recognit, vol. 97, p. 107000, Jan. 2020, doi: 10.1016/j.patcog.2019.107000. DOI: https://doi.org/10.1016/j.patcog.2019.107000

N. Abughazalah and M. Khan, “Graph Neural Network-based multi-criteria group decision-making for optimizing radiation therapy plans in a Pythagorean fuzzy environment,” J Radiat Res Appl Sci, vol. 18, no. 2, p. 101535, Jun. 2025, doi: 10.1016/j.jrras.2025.101535. DOI: https://doi.org/10.1016/j.jrras.2025.101535

R. Wang, Z. Gao, L. Zhang, S. Yue, and Z. Gao, “Empowering large language models to edge intelligence: A survey of edge efficient LLMs and techniques,” Comput Sci Rev, vol. 57, p. 100755, Aug. 2025, doi: 10.1016/j.cosrev.2025.100755. DOI: https://doi.org/10.1016/j.cosrev.2025.100755

L. Chen, P. Peng, and L. Wang, “MAMRS: Mining Automatic Meter Reading System Based on Improved Deep Learning Algorithm Using Quadruped Robots,” Applied Sciences, vol. 14, no. 23, p. 10949, Nov. 2024, doi: 10.3390/app142310949. DOI: https://doi.org/10.3390/app142310949

H. Bekamiri, D. S. Hain, and R. Jurowetzki, “PatentSBERTa: A deep NLP-based hybrid model for patent distance and classification using augmented SBERT,” Technol Forecast Soc Change, vol. 206, p. 123536, Sep. 2024, doi: 10.1016/j.techfore.2024.123536. DOI: https://doi.org/10.1016/j.techfore.2024.123536