1. Home
  2. JUITA Vol. 10 No. 1, May 2022
  3. Murinto

Logarithm Decreasing Inertia Weight Particle Swarm Optimization Algorithms for Convolutional Neural Network

Murinto Murinto, Miftahurrahma Rosyda

Abstract


The convolutional neural network (CNN) is a technique that is often used in deep learning. Various models have been proposed and improved for learning on CNN. When learning with CNN, it is important to determine the optimal parameters. This paper proposes an optimization of CNN parameters using logarithm decreasing inertia weight (LogDIW). This paper is used two datasets, i.e., MNIST and CIFAR-10 dataset. The MNIST learning experiment, the CIFAR-10 dataset, compared its accuracy with the CNN standard based on the LeNet-5 architectural model. When using the MNIST dataset, CNN's baseline was 94.02% at the 5th epoch, compared to CNN's LogDIWPSO, which improves accuracy. When using the CIFAR-10 dataset, the CNN baseline was 28.07% at the 10th epoch, compared to the LogDIWPSO CNN accuracy of 69.3%, which increased the accuracy.


Keywords


CIFAR-10; CNN; Logarithm decreasing inertia weight; MNIST

References


[1] L. Deng and D. Yu, “Deep learning: Methods and applications,” Found. Trends Signal Process., vol. 7, no. 3–4, pp. 197–387, 2013.

[2] F. E. Fernandes Junior and G. G. Yen, “Particle swarm optimization of deep neural networks architectures for image classification,” Swarm Evol. Comput., vol. 49, no. June, pp. 62–74, 2019.

[3] A. A. Ahmed and S. M. Darwish, “A Meta-Heuristic Automatic CNN Architecture Design Approach Based on Ensemble Learning,” IEEE Access, vol. 9, pp. 16975–16987, 2021.

[4] B. Wang, B. Xue, and M. Zhang, “Particle Swarm optimisation for Evolving Deep Neural Networks for Image Classification by Evolving and Stacking Transferable Blocks,” 2020 IEEE Congr. Evol. Comput. CEC 2020 - Conf. Proc., 2020.

[5] Y. Wang, H. Zhang, and G. Zhang, “cPSO-CNN: An efficient PSO-based algorithm for fine-tuning hyper-parameters of convolutional neural networks,” Swarm Evol. Comput., vol. 49, pp. 114–123, 2019.

[6] V. Ayumi, L. M. R. Rere, M. I. Fanany, and A. M. Arymurthy, “Optimization of convolutional neural network using microcanonical annealing algorithm,” 2016 Int. Conf. Adv. Comput. Sci. Inf. Syst. ICACSIS 2016, pp. 506–511, 2017.

[7] M. N. Alam, B. Das, and V. Pant, “A comparative study of metaheuristic optimization approaches for directional overcurrent relays coordination,” Electr. Power Syst. Res., vol. 128, pp. 39–52, 2015.

[8] L. M. R. Rere, M. I. Fanany, and A. M. Arymurthy, “Simulated Annealing Algorithm for Deep Learning,” Procedia Comput. Sci., vol. 72, pp. 137–144, 2015.

[9] M. Dorigo, T. Stutzle, and M. Birattari, “Ant Colony Optimization,” IEEE Comput. Intell. Mag., no. November 2006, pp. 28–39, 2006.

[10] M. O. Okwu and L. K. Tartibu, “Firefly Algorithm,” Stud. Comput. Intell., vol. 927, pp. 61–69, 2021.

[11] X. S. Yang, “A new metaheuristic Bat-inspired Algorithm,” Stud. Comput. Intell., vol. 284, pp. 65–74, 2010.

[12] L. Zhang, Q. Fu, J. Chen, H. Bai, and X. Zhou, “A modified particle swarm optimization algorithm - CPSODE,” Proc. 29th Chinese Control Decis. Conf. CCDC 2017, vol. 1, no. 2, pp. 6659–6663, 2017.

[13] Y. Chhabra, S. Varshney, and Ankita, “Hybrid particle swarm training for convolution neural network (CNN),” 2017 10th Int. Conf. Contemp. Comput. IC3 2017, vol. 2018-Janua, no. August, pp. 1–3, 2018.

[14] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 2261–2269, 2017, doi: 10.1109/CVPR.2017.243.

[15] F. Wang, H. Zhang, K. Li, Z. Lin, J. Yang, and X. L. Shen, “A hybrid particle swarm optimization algorithm using adaptive learning strategy,” Inf. Sci. (Ny)., vol. 436–437, pp. 162–177, 2018.

[16] F. Qian, M. R. Mahmoudi, H. Parvïn, K. H. Pho, and B. A. Tuan, “An Adaptive Particle Swarm Optimization Algorithm for Unconstrained Optimization,” Complexity, vol. 2020, 2020.

[17] G. Ardizzon, G. Cavazzini, and G. Pavesi, “Adaptive acceleration coefficients for a new search diversification strategy in particle swarm optimization algorithms,” Inf. Sci. (Ny)., vol. 299, no. December, pp. 337–378, 2015.

[18] H. Cao, N. Feng, and J. Yang, “Improved Chaos Particle Swarm Optimization Algorithm For Wireless Sensor Networks Node Localization,” Revista de la Facultad de Ingenieria, vol. 32, no. 4, pp. 259–267, 2017.

[19] M. Li et al., “Particle Swarm Optimization Algorithm Based on Chaotic Sequences and Dynamic Self-Adaptive Strategy,” Journal of Computer and Communications , pp. 13–23, 2017.

[20] E. Çomak, “A particle swarm optimizer with modified velocity update and adaptive diversity regulation,” Expert Syst., vol. 36, no. 1, pp. 1–18, 2019.

[21] D. Yan, Y. Lu, M. Zhou, S. Chen, and D. Levy, “Empirically characteristic analysis of chaotic PID controlling particle swarm optimization,” PLoS One, vol. 12, no. 5, pp. 1–24, 2017.

[22] Y. L. Gao, X. H. An, and J. M. Liu, “A particle swarm optimization algorithm with logarithm decreasing inertia weight and chaos mutation,” Proc. - 2008 Int. Conf. Comput. Intell. Secur. CIS 2008, vol. 1, pp. 61–65.

[23] Y. LECUN, S. BOTTOU, Y. BENGIO, and P. HAFFNE, “Gradient-Based Learning Applied to Document Recognition,” Proc. IEEE, vol. 86, no. 1, pp. 2278–2322, 1998.

[24] E. R. Kennedy J, “Particle swarm optimization,” IEEE Int. Conf. Neural Networks, pp. 1942–8., 1995.

[25] Y. Shi and R. Eberhart, “Modified particle swarm optimizer,” Proc. IEEE Conf. Evol. Comput. ICEC, no. February, pp. 69–73, 1998.

[26] A. Krizhevsky, N. Vinod, and H. G. Hinton, "CIFAR-10 Dataset", 2021, https://www.cs.toronto.edu. [Accessed: 10-Jun-2021].


Full Text: PDF

DOI: 10.30595/juita.v10i1.12573

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

ISSN: 2579-8901