Sentiment Analysis of Pro-Israel Product Boycott Action Using IndoBERT Method  on Unbalanced Data

Auliana Rizky Puspita Dewi; Slamet Riyadi; Cahya Darmajati; Nor Ashidi Mat Isa; Annisa Divayu Andriyani

doi:10.30595/juita.v13i2.25976

Authors

Auliana Rizky Puspita Dewi Universitas Muhammadiyah Yogyakarta
Slamet Riyadi Universitas Muhammadiyah Yogyakarta
Cahya Darmajati Universitas Muhammadiyah Yogyakarta
Nor Ashidi Mat Isa Universiti Sains Malaysia
Annisa Divayu Andriyani Universiti Malaysia Pahang Al-Sultan Abdullah

DOI:

https://doi.org/10.30595/juita.v13i2.25976

Keywords:

Sentiment analysis, Product boycott, IndoBERT, Balancing, Twitter

Abstract

A boycott was an act taken to stop the purchase or use of a particular product or service as a form of public protest against a particular company or group committing a deviation. The Israeli-Palestinian conflict, which had been ongoing since 1948, peaked in October 2023 and had claimed more than 35,000 Palestinian lives. This conflict generated a wide range of public opinions in Indonesia, which were expressed through social media, especially Twitter. Thus, the sentiment analysis of public reactions on Twitter became important to understand the reactions and perspectives of society towards the boycott of Pro-Israel products. This study used the IndoBERT method, which was a variant of the BERT method specifically designed to understand Indonesian. Although many studies had applied the IndoBERT method for sentiment analysis and text classification in Indonesian, none had used the IndoBERT method along with data balancing techniques to analyze Indonesian sentiments regarding the boycott of Pro-Israel products on Twitter. Therefore, this study aimed to develop a sentiment analysis model using the IndoBERT method with more data to examine sentiments related to the boycott of Pro-Israel products on Twitter using imbalanced data, as well as to evaluate the effect of balancing methods using under sampling and oversampling on the model’s accuracy and performance. The methods used included data crawling, data preprocessing, labeling with a Lexicon-Based approach, data balancing, and data splitting. The IndoBERT model was trained with 20 epochs, a batch size of 16, and a learning rate of 2e-5. The results of the study showed that the model with balanced data using the oversampling method achieved an accuracy of 97% and an F1-Score of 97%, which was better compared to the model with imbalanced data and the undersampling method. Thus, data balancing using the oversampling method proved to be effective in improving accuracy in sentiment analysis. This research made a significant contribution to understanding the behavior of Indonesian society towards a product boycott supporting Israel and suggested further exploration in parameter optimization and evaluation with larger and more diverse data, as well as further development of data balancing methods to improve the generalization and capabilities of the model

Author Biography

Slamet Riyadi, Universitas Muhammadiyah Yogyakarta

Scopus ID: 6503991450
Google Scholar ID: bl1BHx8AAAAJ
Sinta ID: 5973361

References

[1] B. Palacios-Florencio, M. Á. Revilla-Camacho, D. Garzón, and C. Prado-Román, “Explaining Boycott Behavior: Proposal and validation of a conceptual model,” Journal of Consumer Behaviour, vol. 20, no. 5, pp. 1313–1325, Sep. 2021, doi: 10.1002/cb.1937.

[2] D. Suryani Sentosa and N. I. Sitepu, “Fatwas and the Urgency of Compliance,” 2024.

[3] T. Farhat, S. Ibrahim, Z. Abdul-Sater, and G. Abu-Sittah, “Responding to the Humanitarian Crisis in Gaza: Damned if You do. Damned if You don’t.,” Ann Glob Health, vol. 89, 2023, doi: 10.5334/aogh.3975.

[4] AJLabs, “Israel-Gaza war in maps and charts: Live tracker,” May 2024.

[5] A. Tiara Susilawati, N. Anjeni Lestari, and P. Alpria Nina, “Analysis of Public Sentiment on Twitter Toward Boycott of Israeli Products Using Naïve Bayes Method,” Jurnal Ilmiah Mahasiswa, vol. 2, no. 1, pp. 26–35, 2024, doi: 10.59603/niantanasikka.v2i1.240.

[6] Y. Matalon, O. Magdaci, A. Almozlino, and D. Yamin, “Using Sentiment Analysis to Predict Opinion Reversals in Political Communication Tweets,” Sci Rep, vol. 11, no. 1, Dec. 2021, doi: 10.1038/s41598-021-86510-w.

[7] M. Hamka and D. Ratna Sari, “Sentiment Analysis And Information Extraction Of Online Learning Using Lexicon Approach,” 2022.

[8] S. Damayanti, L. Dwi Oktaviana, and T. Bratakusuma, “Data Mining Using Support Vector Machine Model for Baturraden Tourism Visitor Satisfaction Prediction,” JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 8, 2024, doi: 10.30865/mib.v8i2.7346.

[9] H. Hassani, C. Beneki, S. Unger, M. T. Mazinani, and M. R. Yeganegi, “Text Mining in Big Data Analytics,” Big Data and Cognitive Computing, vol. 4, pp. 1–34, Mar. 2020, doi: 10.3390/bdcc4010001.

[10] M. F. Mubaraq and W. Maharani, “Sentiment Analysis on Twitter Social Media towards Climate Change on Indonesia Using IndoBERT Model,” JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 6, no. 4, p. 2426, Oct. 2022, doi: 10.30865/mib.v6i4.4570.

[11] R. G. Katryn, “Text Preprocessing: Tahap Awal dalam Natural Language Processing (NLP),” Mandiri Engineering. Accessed: May 17, 2024. [Online]. Available: https://medium.com/mandiri-engineering/text-preprocessing-tahap-awal-dalam-natural-language-processing-nlp-bc5fbb6606a

[12] Amazon, “What is Sentiment Analysis?,” Amazon. Accessed: May 23, 2024. [Online]. Available: https://aws.amazon.com/id/what-is/sentiment-analysis/

[13] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Oct. 2019, [Online]. Available: http://arxiv.org/abs/1810.04805

[14] H. Ramanizar, A. Fajri, R. Binsar Sinaga, H. Mubarok, A. D. Pangestu, and D. S. Prasvita, Sentiment Analysis of Twitter Users on the Conflict between Palestine and Israel Using Naïve Bayes Classification and Support Vector Machine Methods. 2021.

[15] H. Irsyad and A. Taqwiym, “Analyzing People’s Sentiment towards Palestinians with Naive Bayes Classification,” 2021.

[16] R. Azhar and M. F. Wijayanto, “Sentiment Analysis on Twitter: Uncovering Perceptions and Public Emotions Around the Palestinian-Israeli Conflict,” 2024.

[17] M. Makmun, Z. Mustofa, and A. Turmudi, “Bulletin of Information Technology (BIT) Sentiment Analysis of Indonesian People in Russia-Ukraine Conflict on Twitter,” vol. 3, no. 4, pp. 355–366, 2022, doi: 10.47065/bit.v3i1.

[18] T. Oswari, T. Yusnitasari, and S. Wijay, “Sentiment Analysis of Indonesian YouTube Reviews About Lesbian, Gay, Bisexual, and Transgender (LGBT) using IndoBERT Fine Tuning,” vol. 15, no. 1, pp. 2088–1541, 2024, doi: 10.24843/LKJITI.2024.v15.i01.p03.

[19] H. Imaduddin, F. Yusfida A’la, and Y. S. Nugroho, “Sentiment Analysis in Indonesian Healthcare Applications using IndoBERT Approach,” 2023. [Online]. Available: www.ijacsa.thesai.org

[20] H. Jayadianti, W. Kaswidjanti, A. T. Utomo, S. Saifullah, F. A. Dwiyanto, and R. Drezewski, “Sentiment analysis of Indonesian reviews using IndoBERT and R-CNN refinement,” ILKOM Jurnal Ilmiah, vol. 14, pp. 348–354, Dec. 2022, doi: 10.33096/ilkom.v14i3.1505.348-354.

[21] N. Mahfudiyah and A. Alamsyah, “Consumer Perception Analysis Of Gojek Service Quality Using Sentiment Analysis And Topic Modeling Based On Deep Learning IndoBERT,” 2022.

[22] R. Merdiansah and A. Ali Ridha, “Sentiment Analysis of Indonesian X Users Regarding Electric Vehicles Using IndoBERT,” Jurnal Ilmu Komputer dan Sistem Informasi (JIKOMSI, vol. 7, no. 1, pp. 221–228, 2024.

[23] Deepanshi, “Text Preprocessing in NLP with Python Codes,” Analytics Vidhya.

[24] H. M. Lee and Y. Sibaroni, “Comparison of IndoBERTweet and Support Vector Machine in Sentiment Analysis of Racing Circuit Development in Indonesia,” JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 7, no. 1, pp. 99–106, 2023, doi: 10.30865/mib.v7i1.5380.

[25] M. Ali, “NLTK Sentiment Analysis Tutorial for Beginners,” datacamp. Accessed: May 23, 2024. [Online]. Available: https://www.datacamp.com/tutorial/text-analytics-beginners-nltk

[26] J. Murel and E. Kavlakoglu, “What is stemming?,” IBM. Accessed: May 23, 2024. [Online]. Available: https://www.ibm.com/topics/stemming

[27] Y. Fauziah, B. Yuwono, and A. S. Aribowo, “Lexicon Based Sentiment Analysis in Indonesia Languages : A Systematic Literature Review,” RSF Conference Series: Engineering and Technology, vol. 1, no. 1, pp. 363–367, Dec. 2021, doi: 10.31098/cset.v1i1.397.

[28] M. Zheng et al., “UFFDFR: Undersampling framework with denoising, fuzzy c-means clustering, and representative sample selection for imbalanced data classification,” Inf Sci (N Y), vol. 576, pp. 658–680, Oct. 2021, doi: 10.1016/j.ins.2021.07.053.

[29] S. Fulvi Intan et al., “Perbandingan Algoritma KNN, NBC, dan SVM: Analisis Sentimen Masyarakat Terhadap Perparkiran di Kota Pekanbaru,” 2023.

[30] B. Wilie et al., “IndoNLU: Benchmarks and Resources for Evaluating Indonesian Natural Language Comprehension,” Sep. 2020, [Online]. Available: http://arxiv.org/abs/2009.05387

[31] S. Dharmawan, ) Viny, C. Mawardi, ) Novario, and J. Perdana, “Jurnal Ilmu Komputer dan Sistem Informasi Klasifikasi Ujaran Kebencian Menggunakan Metode FeedForward Neural Network (IndoBERT),” 2023.

[32] G. Z. Nabiilah, I. N. Alam, E. S. Purwanto, and M. F. Hidayat, “Indonesian multilabel classification using IndoBERT embedding and MBERT classification,” International Journal of Electrical and Computer Engineering (IJECE), vol. 14, no. 1, pp. 1071–1078, Feb. 2024, doi: 10.11591/ijece.v14i1.pp1071-1078.

[33] S. Riyadi, A. Divayu Andriyani, and S. Noraini Sulaiman, “Improving Hate Speech Detection Using Double-Layers Hybrid CNN-RNN Model on Imbalanced Dataset,” IEEE Access, vol. 12, pp. 159660–159668, 2024, doi: 10.1109/ACCESS.2024.3487433.

[34] S. Riyadi, F. Alfiana, M. I. Al-Habib, C. Damarjati, and E. Loniza, “Early Detection of Heart Disease on Imbalanced Data Using Gradient Boosting,” 2024 Ninth International Conference on Informatics and Computing (ICIC), pp. 1–6, Oct. 2024, doi: 10.1109/ICIC64337.2024.10956329.

[35] S. Riyadi, A. D. Andriyani, A. M. Masyhur, C. Damarjati, and M. I. Solihin, “Detection of Indonesian Hate Speech on Twitter Using Hybrid CNN-RNN,” Proceeding - International Conference on Information Technology and Computing 2023, ICITCOM 2023, pp. 352–356, 2023, doi: 10.1109/ICITCOM60176.2023.10442041.