Comparison of Multinomial, Bernoulli, and Gaussian Naïve Bayes for Complaint Classification in Pro Denpasar Application

Ida Bagus Mahendra; I Made Gede Sunarya; I Made Agus Wirawan

doi:10.30595/juita.v13i1.24828

Authors

Ida Bagus Mahendra Universitas Pendidikan Ganesha
I Made Gede Sunarya Universitas Pendidikan Ganesha
I Made Agus Wirawan Universitas Pendidikan Ganesha

DOI:

https://doi.org/10.30595/juita.v13i1.24828

Keywords:

Naïve Bayes, Natural Language Processing (NLP), Public Complaints, Text Mining, TF-IDF

Abstract

Pelayanan Rakyat Online Denpasaror PRO Denpasar is a Denpasar City Government application intended as a public service mall to support Denpasar to become a smart city. This application was built since 2014 and is actively used in channeling public complaints so optimization is continuously needed to increase the efficiency of application use. Application optimization is carried out by developing decision support tools to determine complaint categories that are still done manually. The application of the properartificial intelligence method can be used as a solution in classifying complaint categories to become a decision support tool for operators. This study compares three classification methods including multinomial naïve bayes, bernoulli naïve bayes and gaussian naïve bayes by applying TF-IDF feature extraction to determine the best complaint category classification method. Based on eight comparison scenario results by applying a comparison of 25%, 50%, 75% and 100% of complaint descriptions with 5-fold cross validation and 10-fold cross validation, it was found that the multinomial naïve Bayes method provided the best result in seven combined comparisons involving the test parameters accuracy, precision, recall, f1-score and processing time.

Author Biographies

Ida Bagus Mahendra, Universitas Pendidikan Ganesha

Master of Computer Science Program

I Made Gede Sunarya, Universitas Pendidikan Ganesha

Master of Computer Science Program

I Made Agus Wirawan, Universitas Pendidikan Ganesha

Master of Computer Science Program

References

[1] A. D. Syafitri, E. Purnamaningsih, and P. D. Yudartha, “Pengaruh Kualitas Pelayanan Terhadap Kepuasan Pengguna pada Aplikasi Pro-Denpasar di Kota Denpasar,” Jurnal Administrasi Publik Citizen Charter, vol. 2, no. 1, Jun. 2022.

[2] H. A. Gozali and M. A. Rosid, “Classification of Student Complaints with Naive Bayes and Literary Methods Klasifikasi Keluhan Mahasiswa dengan Metode Naive Bayes dan Sastrawi,” 2020.

[3] C. Anam, “Klasifikasi Pengaduan Pelayanan Dispendukcapil Kota Malang Menggunakan Metode Naïve Bayes dan Seleksi Fitur Glasgow-II,” Jurnal Pengembangan Teknologi Informasi dan Ilmu Komputer, vol. 4, no. 9, pp. 3264–3271, Sep. 2020, [Online]. Available: http://j-ptiik.ub.ac.id

[4] F. A. Lestari, L. Efrizoni, E. Ali, and R. Rahmiati, “Sistem Klasifikasi Pengaduan Masyarakat pada BPJS Ketenagakerjaan Menggunakan Algoritma Naïve Bayes Berbasis Mobile,” Building of Informatics, Technology and Science (BITS), vol. 4, no. 1, Jun. 2022, doi: 10.47065/bits.v4i1.1685.

[5] G. Angiani, M. Mordonini, U. Pietroni, P. Fornacciari, G. Lombardo, and M. Tomaiuolo, “Automatic Processing and Classification of Citizens’ Reports,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Nov. 2018, pp. 310–311. doi: 10.1145/3284869.3284930.

[6] I. G. N. A. Kusuma, I. M. Pradipta, I. M. A. Santosa, and I. K. Dharmendra, “Penanganan Ketidakseimbangan Data pada Klasifikasi Pengaduan Masyarakat,” Jurnal Teknologi Informasi dan Komputer, vol. 9, no. 5, Oct. 2023, [Online]. Available: https://pengaduan.denpasarkota.go.id[6].

[7] J. P. D, Delizo, “Philippine Twitter Sentiments during Covid-19 Pandemic using Multinomial Naïve-Bayes,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 9, no. 1.3, pp. 408–412, Jun. 2020, doi: 10.30534/ijatcse/2020/6491.32020.

[8] A. Sabrani, I. W. G. P. W. Wedashwara, and F. Bimantoro, “Metode Multinomial Naïve Bayes untuk Klasifikasi Artikel Online Tentang Gempa Di Indonesia,” JTIKA, vol. 2, no. 1, Mar. 2020, [Online]. Available: http://jtika.if.unram.ac.id/index.php/JTIKA/

[9] F. A. Saputra and C. Dewi, “Klasifikasi Dokumen Pengaduan Sambat Online menggunakan Metode Multinomial Naïve Bayes dan N-Gram,” 2020. [Online]. Available: http://j-ptiik.ub.ac.id

[10] A. Y. Prayoga, A. Id Hadiana, and F. R. Umbara, “Deteksi Hoax pada Berita Online Bahasa Inggris Menggunakan Bernoulli Naïve Bayes dengan Ekstraksi Fitur TF-IDF,” Jurnal Syntax Admiration, vol. 2, no. 10, pp. 1808–1823, Oct. 2021, doi: 10.46799/jsa.v2i10.327.

[11] Widyawati and Sutanto, “Perbandingan Kinerja Variasi Naïve Bayes Multivariate Bernoulli dan Naïve Bayes Multinomial dalam Pengklasifikasian Dokumen Teks,” JURNAL OF INNOVATION AND FUTURE TECHNOLOGY (IFTECH), vol. 2, no. 1, Feb. 2020.

[12] K. V Deshmukh and S. S. Shiravale, “Ambiguity Resolution in English Language for Sentiment Analysis,” in IEEE Punecon, Pune, 2018.

[13] H. Zhang, J. Nie, and H. Wang, “Product Perspective: A Study of User Demand Priorities in Online Community Reviews,” 2019.

[14] N. M. Ranjan and R. S. Prasad, “Text Analytics: An Application of Text Mining,” Journal of Data Mining and Management, vol. 6, no. 3, pp. 1–6, Nov. 2021, doi: 10.46610/jodmm.2021.v06i03.001.

[15] E. K. Andana, M. Othman, and R. Ibrahim, “Comparative Analysis of Text Classification Using Naive Bayes and Support Vector Machine in Detecting Negative Content in Indonesian Twitter,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 8, no. 1.3 S1, pp. 356–362, 2019, doi: 10.30534/ijatcse/2019/6481.32019.

[16] R. Fajar, “Implementasi Algoritma Naive Bayes Terhadap Analisis Sentimen Opini Film Pada Twitter,” JURNAL INOVTEK POLBENG, vol. 3, no. 1, Jun. 2018.

[17] A. D. Adyatma, L. Afuan, and E. Maryanto, “The Effect of Unigram and Bigram in The Naïve Bayes Multinomial for Analyzing of Comment Sentiment of Gojek Application in Google Play Store,” Jurnal Teknik Informatika (JUTIF), vol. 4, no. 6, pp. 1535–1540, Dec. 2023, doi: 10.52436/1.jutif.2023.4.6.1310.

[18] K. S. Nugroho, I. Istiadi, and F. Marisa, “Naive Bayes Classifier Optimization for Text Classification on E-Government Using Particle Swarm Optimization,” Jurnal Teknologi dan Sistem Komputer, vol. 8, no. 1, pp. 21–26, Jan. 2020, doi: 10.14710/jtsiskom.8.1.2020.21-26.

[19] M. A. Suprayogi, “Klasifikasi Pengaduan Laras Online Berbasis Text Mining Menggunakan Algoritma K-Nearest Neighbor dan Naïve Bayes,” Jurnal Ilmiah Informatika Komputer, vol. 26, no. 1, pp. 65–77, 2021, doi: 10.35760/ik.2021.v26i1.3397.

[20] T. Taslim, S. Handayani, and F. Fajrizal, “Kinerja Komparatif Optimasi Algoritma Naive Bayes dalam Klasifikasi Teks untuk Uji Klinis Kanker,” Jurnal Eksplora Informatika, vol. 13, no. 1, pp. 113–123, Sep. 2023, doi: 10.30864/eksplora.v13i1.994.

[21] D. Aziz, N. Hayatin, C. Sri, and K. Aditya, “Perbandingan Algoritma Selection Feature Query Expansion Ranking dan Symmetrical Uncertainty pada Instagram Terkait Sengketa ZEE di Perairan Natuna,” REPOSITOR, vol. 3, no. 4, pp. 411–418, Aug. 2021.

[22] D. Maulana and R. S. S. Markiano, “Klasifikasi Analisis Sentimen Terhadap Calon Presiden 2019 pada Media Sosial Twitter Menggunakan Metode Algoritma Naïve Bayes,” Jurnal Teknologi Pelita Bangsa, vol. 11, no. 3, pp. 2407–3903, Mar. 2020.

[23] A. Y. Permana and D. N. Mahardhika, “Klasifikasi Analisis Sentimen Terhadap Kasus Penusukan Wiranto Media Sosial Twitter dengan Menggunakan Metode Naïve Bayes Classification,” Jurnal Teknologi Pelita Bangsa, vol. 10, no. 1, Mar. 2019.

[24] A. Lasarudin, “Klasifikasi Pengaduan Masyarakat Menggunakan Naive Bayes Berbasis Seleksi Atribut Information Gain,” 2018. [Online]. Available: http://research.

[25] M. B. Ressan and R. F. Hassan, “Naïve-Bayes Family for Sentiment Analysis During COVID-19 Pandemic and Classification Tweets,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 28, no. 1, pp. 375–383, Oct. 2022, doi: 10.11591/ijeecs.v28.i1.pp375-383.

[26] A. Nugraha and M. U. Siregar, “Application of the Naive Bayes Classifier Method in the Sentiment Analysis of Twitter User About The Capital City Relocation,” PROC. INTERNAT. CONF. SCI. ENGIN, vol. 4, Feb. 2021.

[27] W. M. P. Dhuhita and F. Zone, “Perbandingan Kinerja Algoritma Multinomial dan Bernoulli Naïve Bayes dalam Mengklasifikasikan Komentar Cyberbullying,” Komputika : Jurnal Sistem Komputer, vol. 12, no. 2, pp. 109–117, Oct. 2023, doi: 10.34010/komputika.v12i2.9767.

[28] N. A’ayunnisa, Y. Salim, and H. Azis, “Analisis Performa Metode Gaussian Naïve Bayes untuk Klasifikasi Citra Tulisan Tangan Karakter Arab,” Indonesian Journal of Data and Science (IJODAS), vol. 3, no. 3, pp. 115–121, 2022.

[29] I. W. Saputro and B. W. Sari, “Uji Performa Algoritma Naïve Bayes untuk Prediksi Masa Studi Mahasiswa Naïve Bayes Algorithm Performance Test for Student Study Prediction,” Citec Journal, vol. 6, no. 1, 2019.

[30] A. Aytekin, “Comparative Analysis Of Normalization Techniques in the Context of MCDM Problems,” Decision Making: Applications in Management and Engineering, vol. 4, no. 2, pp. 1–25, 2021, doi: 10.31181/dmame210402001a.

[31] Y. A. Singgalen, “Analisis Sentimen dan Sistem Pendukung Keputusan Menginap di Hotel Menggunakan Metode CRISP-DM dan SAW,” Journal of Information System Research (JOSH), vol. 4, no. 4, pp. 1343–1353, Jul. 2023, doi: 10.47065/josh.v4i4.3917.

[32] Y. Susanti, Y. Diah, and D. D. Hanum, “Penerapan Metode Naive Bayes dan Simple Additive Weighting dalam Penentuan Penerima Program Keluarga Harapan,” Seminar Nasional Sains dan Teknologi Terapan, vol. VI, 2018.

[33] I. M. A. Wirawan, R. Wardoyo, D. Lelono, and S. Kusrohmaniah, “Continuous Capsule Network Method for Improving Electroencephalogram-Based Emotion Recognition,” Emerging Science Journal, vol. 7, no. 1, pp. 116–134, Feb. 2023, doi: 10.28991/ESJ-2023-07-01-09.

[34] C. Naulak and P. Jindal, “A Comparative Study of Naive Bayes Classifiers with Improved Technique on Text Classification,” TechRxiv, May 2022.

[35] T. Simbolon, A. P. Wibawa, I. A. E. Zaeni, and A. R. Ismail, “Text Classification of Traditional and National Songs using Naïve Bayes Algorithm,” Science in Information Technology Letters, vol. 3, no. 2, pp. 59–72, Nov. 2022, doi: 10.31763/sitech.v3i2.1215.

[36] M. Abbas, K. Ali, A. Jamali, K. Ali Memon, and A. Aleem Jamali, “Multinomial Naive Bayes Classification Model for Sentiment Analysis,” IJCSNS International Journal of Computer Science and Network Security, vol. 19, no. 3, p. 62, 2019, doi: 10.13140/RG.2.2.30021.40169.

[37] S. M. Sadid, J. C. Young, and A. Rusli, “Spam Filtering on User Feedback via Text Classification using Multinomial Naïve Bayes and TF-IDF,” Ultimatics : Jurnal Teknik Informatika, vol. 13, no. 2, p. 108, 2021.