🪴 Aradinka Digital Garden

Search

Search IconIcon to open search

weights in deep learning

Last updated Dec 1, 2022

Weights are usually initialised randomly. The initialization takes a fair amount of repetitions to converge to the least loss and reach the ideal weight matrix.

The problem is that kind of initialization is prone to vanishing or exploding-gradient problesms

General ways to make it initialize better weight: Using relu activation function in the deep nets

Using tanh activation function