Optimizing Artificial Neural Networks Trough Weight Adjustments

Longer training times pose a significant challenge in artificial neural networks (ANNs) as it may leads to increasing the computational costs and decreasing the effectiveness of the model. Therefore, it is imperative to reduce training times in ANNs to enhance the computational efficiency. The initialization of the weights between the layers in ANN plays a vital role in reducing training times. Appropriate weight initialization can help the network converge faster during the training by providing an optimum starting point for the network. Therefore, weight initialization techniques are essential for efficient training of ANNs. This paper revisits and implements different popular weight initialization techniques in ANNs and analyzes their impact on training time. Specifically, this paper implements Gaussian-based, Kaming-based, and Xavier-based weight initiation atop a popular DNN-based network. The experiments are conducted by employing a well-known dataset. The results show that the scenario when no weight initiation is applied consumed the highest training time, whereas different weight initiation techniques contribute in reducing the training times for the network.