Random Initialization

  |   Source
Random Initialization
  • The last pieces in neural networks to be implemented

  • Init all theta to zero may eventually correct in logistic regression, but fail in neural networks

  • the result will be looping forever as the result(hidden units) is exactly is the same as input
  • The problem will be solved by random initialization

  • the random will be the value between 0 and 1
  • the epsilon is init that we manually set, different to what we know about epsilon in gradient checking

  • In summary, random init value close to zero
  • do backprop, gradient checking, do advanced animation , to minimize cost function j(theta) using random init with symmetry breaking
  • This will find a good random value of theta