# What is the role of the bias in neural networks?

###### Posted By: Anonymous

I’m aware of the gradient descent and the back-propagation algorithm. What I don’t get is: when is using a bias important and how do you use it?

For example, when mapping the `AND`

function, when I use 2 inputs and 1 output, it does not give the correct weights, however, when I use 3 inputs (1 of which is a bias), it gives the correct weights.

## Solution

I think that biases are almost always helpful. In effect, **a bias value allows you to shift the activation function to the left or right**, which may be critical for successful learning.

It might help to look at a simple example. Consider this 1-input, 1-output network that has no bias:

The output of the network is computed by multiplying the input (x) by the weight (w_{0}) and passing the result through some kind of activation function (e.g. a sigmoid function.)

Here is the function that this network computes, for various values of w_{0}:

Changing the weight w_{0} essentially changes the “steepness” of the sigmoid. That’s useful, but what if you wanted the network to output 0 when x is 2? Just changing the steepness of the sigmoid won’t really work — **you want to be able to shift the entire curve to the right**.

That’s exactly what the bias allows you to do. If we add a bias to that network, like so:

…then the output of the network becomes sig(w_{0}*x + w_{1}*1.0). Here is what the output of the network looks like for various values of w_{1}:

Having a weight of -5 for w_{1} shifts the curve to the right, which allows us to have a network that outputs 0 when x is 2.

###### Answered By: Anonymous

Disclaimer: This content is shared under creative common license cc-by-sa 3.0. It is generated from StackExchange Website Network.