Improving Sentiment Analysis Accuracy Using CRNN on Imbalanced Data: a Case Study of Indonesian National Football Coach

Slamet Riyadi; Muhammad Dzaki Mubarok; Cahya Damarjati; Asnor Juraiza Ishak

doi:10.30595/juita.v12i2.21847

Authors

Slamet Riyadi Universitas Muhammadiyah Yogyakarta
Muhammad Dzaki Mubarok Universitas Muhammadiyah Yogyakarta
Cahya Damarjati Universitas Muhammadiyah Yogyakarta
Asnor Juraiza Ishak Universiti Putra Malaysia

DOI:

https://doi.org/10.30595/juita.v12i2.21847

Keywords:

sentiment analysis, Deep Learning, overfitting, oversampling, imbalanced data.

Abstract

Conducting sentiment research on the perception of the Indonesian people towards Shin Tae Yong's (STY) role as coach of the Indonesian National Football Team (PSSI) is crucial as it can assist PSSI in determining whether to extend STY's contract. Prior studies have demonstrated that Deep Learning achieves a high level of accuracy when applied to sentiment analysis in many domains. Nevertheless, no investigation has been conducted thus far utilizing deep learning techniques to examine emotion towards STY. This study employs modified Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Convolutional Recurrent Neural Networks (CRNN), and CRNN models with and without data oversampling. The research findings indicate that the CRNN model, when combined with data oversampling and a redesigned architecture, achieves the highest level of accuracy (1.00) and consistently performs well. This research provides significant contributions in three areas: firstly, it utilizes Deep Learning techniques for sentiment analysis on STY; secondly, it modifies the CRNN architecture; and thirdly, it applies data oversampling to address the issue of imbalanced data.

Author Biographies

Slamet Riyadi, Universitas Muhammadiyah Yogyakarta

Information Technology Study Program, Faculty of Engineering

Muhammad Dzaki Mubarok, Universitas Muhammadiyah Yogyakarta

Information Technology Study Program, Faculty of Engineering

Cahya Damarjati, Universitas Muhammadiyah Yogyakarta

Information Technology Study Program, Faculty of Engineering

Asnor Juraiza Ishak, Universiti Putra Malaysia

Faculty of Engineering

References

[1] J. Dvorak, A. Junge, T. Graf-Baumann, L. Peterson, and F-MARC FIFA Medical Assessment and Research Center, “Football is the most popular sport worldwide,” Am. J. Sports Med., vol. 32, no. 1_suppl, pp. 3–4, Mar. 2004, doi: 10.1177/0363546503262283.

[2] B. Riffenburgh and B. Carroll, “THE BIRTH OF PRO FOOTBALL,” 1989.

[3] A. Fuller, “Approaching football in Indonesia,” Soccer Soc., vol. 16, no. 1, pp. 140–148, Jan. 2015, doi: 10.1080/14660970.2014.954387.

[4] M. J. Power, P. Widdop, D. Parnell, J. Carr, and S. R. Millar, “Football and politics: the politics of football,” Manag. Sport Leis., vol. 25, no. 1–2, pp. 1–5, Mar. 2020, doi: 10.1080/23750472.2020.1723437.

[5] I. Syahputra, “Terbentuknya Identitas Fans Sepak Bola sebagai Budaya Massa dalam Industri Media,” INFORMASI, vol. 46, no. 2, p. 205, Dec. 2016, doi: 10.21831/informasi.v46i2.11377.

[6] T. Aichner, “Football clubs’ social media use and user engagement,” Mark. Intell. Plan., vol. 37, no. 3, pp. 242–257, May 2019, doi: 10.1108/MIP-05-2018-0155.

[7] W. Akram, Department of Computer Applications, GDC Mendhar, Poonch, India, R. Kumar, and Department of Computer Applications, GDC Mendhar, Poonch, India, “A Study on Positive and Negative Effects of Social Media on Society,” Int. J. Comput. Sci. Eng., vol. 5, no. 10, pp. 351–354, Oct. 2017, doi: 10.26438/ijcse/v5i10.351354.

[8] F. Aftab, S. U. Bazai, S. Marjan, L. Baloch, S. Aslam, A. Amphawan, and T. K. Neo, “A Comprehensive Survey on Sentiment Analysis Techniques,” Int. J. Technol., vol. 14, no. 6, p. 1288, Oct. 2023, doi: 10.14716/ijtech.v14i6.6632.

[9] B. Constandt, M. M. Parent, and A. Willem, “Does it really matter? A study on soccer fans’ perceptions of ethical leadership and their role as ‘stakeowners,’” Sport Manag. Rev., vol. 23, no. 3, pp. 374–386, Jul. 2020, doi: 10.1016/j.smr.2019.04.003.

[10] Z. Hao, “Deep learning review and discussion of its future development,” MATEC Web Conf., vol. 277, p. 02035, 2019, doi: 10.1051/matecconf/201927702035.

[11] Moch. A. Nasichuddin, T. B. Adji, and W. Widyawan, “Performance Improvement Using CNN for Sentiment Analysis,” IJITEE Int. J. Inf. Technol. Electr. Eng., vol. 2, no. 1, Jul. 2018, doi: 10.22146/ijitee.36642.

[12] M. Thomas and L. C.A, “Sentimental analysis using recurrent neural network,” Int. J. Eng. Technol., vol. 7, no. 2.27, p. 88, Aug. 2018, doi: 10.14419/ijet.v7i2.27.12635.

[13] A. Onan, “Bidirectional convolutional recurrent neural network architecture with group-wise enhancement mechanism for text sentiment classification,” J. King Saud Univ. - Comput. Inf. Sci., vol. 34, no. 5, pp. 2098–2117, May 2022, doi: 10.1016/j.jksuci.2022.02.025.

[14] I. S. H. Almaqbali, “Web Scrapping: Data Extraction from Websites,” Jul. 2020.

[15] E. Haddi, X. Liu, and Y. Shi, “The Role of Text Pre-processing in Sentiment Analysis,” Procedia Comput. Sci., vol. 17, pp. 26–32, 2013, doi: 10.1016/j.procs.2013.05.005.

[16] S. Dewi and D. B. Arianto, “TWITTER SENTIMENT ANALYSIS TOWARDS QATAR AS HOST OF THE 2022 WORLD CUP USING TEXTBLOB,” 2022.

[17] V. R. Joseph, “Optimal Ratio for Data Splitting,” Stat. Anal. Data Min. ASA Data Sci. J., vol. 15, no. 4, pp. 531–538, Aug. 2022, doi: 10.1002/sam.11583.

[18] A. S. U., F. R. P. P., A. Abraham, and D. Stephen, “Deep Learning-Based BoVW–CRNN Model for Lung Tumor Detection in Nano-Segmented CT Images,” Electronics, vol. 12, no. 1, p. 14, Dec. 2022, doi: 10.3390/electronics12010014.

[19] K. Suzuki, S. G. Armato, F. Li, S. Sone, and K. Doi, “Massive training artificial neural network (MTANN) for reduction of false positives in computerized detection of lung nodules in low‐dose computed tomography,” Med. Phys., vol. 30, no. 7, pp. 1602–1617, Jul. 2003, doi: 10.1118/1.1580485.

[20] L. Alzubaidi, J. Zhang, A. J. Humaidi, A. Al-Dujaili, Y. Duan, O. Al-Shamma, J. Santamaría, M. A. Fadhel, M. Al-Amidie, and L. Farhan, “Review of deep learning: concepts, CNN architectures, challenges, applications, future directions,” J. Big Data, vol. 8, no. 1, p. 53, Mar. 2021, doi: 10.1186/s40537-021-00444-8.

[21] B. Soujanya and T. Sitamahalakshmi, “Optimization with ADAM and RMSprop in Convolution neural Network (CNN): A Case study for Telugu Handwritten Characters,” Int. J. Emerg. Trends Eng. Res., vol. 8, no. 9, pp. 5116–5121, Sep. 2020, doi: 10.30534/ijeter/2020/38892020.

[22] K. Team, “Keras documentation: Metrics.” Accessed: Mar. 28, 2024. [Online]. Available: https://keras.io/api/metrics/

[23] X. Wu and J. Liu, “A New Early Stopping Algorithm for Improving Neural Network Generalization,” in 2009 Second International Conference on Intelligent Computation Technology and Automation, Changsha, Hunan, China: IEEE, 2009, pp. 15–18. doi: 10.1109/ICICTA.2009.11.

[24] N. Rochmawati, H. B. Hidayati, Y. Yamasari, H. P. A. Tjahyaningtijas, W. Yustanti, and A. Prihanto, “Analisa Learning Rate dan Batch Size pada Klasifikasi Covid Menggunakan Deep Learning dengan Optimizer Adam,” J. Inf. Eng. Educ. Technol., vol. 5, no. 2, pp. 44–48, Dec. 2021, doi: 10.26740/jieet.v5n2.p44-48.

[25] M. Artur, “Review the performance of the Bernoulli Naïve Bayes Classifier in Intrusion Detection Systems using Recursive Feature Elimination with Cross-validated selection of the best number of features,” Procedia Comput. Sci., vol. 190, pp. 564–570, 2021, doi: 10.1016/j.procs.2021.06.066.

[26] A. H. Wicaksono, A. A. Supianto, S. H. Wijoyo, D. Krisnandi, and A. Heryana, “Klasifikasi Siswa Slow Learner untuk Mendukung Sekolah dalam Meningkatkan Pemahaman Siswa Menggunakan Algoritma Naïve Bayes,” J. Teknol. Inf. Dan Ilmu Komput., vol. 9, no. 3, pp. 589–596, Jun. 2022, doi: 10.25126/jtiik.2022935609.

[27] M. Sokolova, N. Japkowicz, and S. Szpakowicz, “Beyond Accuracy, F-Score and ROC: A Family of Discriminant Measures for Performance Evaluation,” in AI 2006: Advances in Artificial Intelligence, vol. 4304, A. Sattar and B. Kang, Eds., in Lecture Notes in Computer Science, vol. 4304. , Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 1015–1021. doi: 10.1007/11941439_114.

[28] Y. Sun, A. K. C. Wong, and M. S. Kamel, “CLASSIFICATION OF IMBALANCED DATA: A REVIEW,” Int. J. Pattern Recognit. Artif. Intell., vol. 23, no. 04, pp. 687–719, Jun. 2009, doi: 10.1142/S0218001409007326.

[29] R. Mohammed, J. Rawashdeh, and M. Abdullah, “Machine Learning with Oversampling and Undersampling Techniques: Overview Study and Experimental Results,” in 2020 11th International Conference on Information and Communication Systems (ICICS), Irbid, Jordan: IEEE, Apr. 2020, pp. 243–248. doi: 10.1109/ICICS49469.2020.239556.

[30] S. Pavlitskaya, J. Oswald, and J. M. Zöllner, “Measuring Overfitting in Convolutional Neural Networks using Adversarial Perturbations and Label Noise.” arXiv, Sep. 27, 2022. Accessed: Mar. 31, 2024. [Online]. Available: http://arxiv.org/abs/2209.13382