Assessment of Retrieval and Generative Chatbots in Tourism Information Service - Handayani, Sharfina Febbi; Dairoh, Dairoh; Af’idah, Dwi Intan (JUITA: Jurnal Informatika, 2025)
Chatbots are essential for improving the customer experience on tourism websites, especially when it comes to arranging travel and offering precise information. The purpose of this study is to evaluate the effectiveness of generative and retrieval-based chatbots in the tourism information service. Two retrieval-based models are MLP-based single QA and multi QA and two generative-based models namely LLaMA 2 and GEMMA were evaluated using confusion matrix, BLUE score, correctness response and naturalness response. The study found that LLaMA 2 outperformed other models, with the highest response Accuracy of 0.89, naturalness of 0.75, and BLEU score of 0.33. GEMMA received the lowest score, suggesting that it has trouble coming up with precise and organic answers. The retrieval-based models showed strong accuracy but were less natural in their responses. The ease of dataset creation for generative models, which only requires narrative text, further positions LLaMA 2 as the most suitable option for improving user experience in Tegal tourism services.