**Dropout** is a regularization technique for reducing overfitting in neural networks by preventing complex co-adaptations on training data. It is a very efficient way of performing model averaging with neural networks.

**Why Dropout:** Dropout helps prevent weights from converging to identical positions. It does this by **randomly turning nodes off** when forward propagating.

In simple terms some of your neurons will not participate in the calculation.

In Tensorflow we have two dropout functions.

- tf.nn.dropout
- It has parameter
**keep_prop**which state the probability of neuron which will not be drop. If we dive keep_prop value to 0.6, it means 60% of neurons will remain and 40% will be drop.

- It has parameter
- tf.layers.dropout
- It has two main parameters, rate, and
**training.**Rate means the number of neurons which will be drop.If the rate is 0.6 then 60% neurons will be dropped and 40% will be used. - We can see
**keep_prop = 1- rate**. - Training parameter is to differentiate if the network is running for training or to get the result.We need dropout only when we are training the neural network, not while we are testing(inferencing) it.

- It has two main parameters, rate, and

**So what is the difference between these two functions?**

tf.layers.dropout is a wrapper over the tf.nn.droput, which give us the demarcation of whether to return the output in training mode (apply dropout) or in inference mode (return the input untouched).”

Advertisements