What is the Bellman Equation?Artificial Intelligence Free Tutorial

THE BELLMAN EQUATION

here are a few values that are going to be important trough  this course:-

• s- state of the AI
• a-action taken by AI
• R-the reward of the action
• y(gamma)-Discount factor 

This concept was bought forward by Richard Ernest Bellman

Lets us consider a maze our bot is in this maze here the finish stands for +1 and the fire or end stands for -1 as a result.

Our bot starts from the start then keeps exploring by the set of possible actions it can take .

Lets say,
Our bot begins from start, then it goes left and realizes that it can't go  left.then takes the action "up"
then it goes up then takes the next action and keeps going on so on then comes up  with the following path by valuing the path's each box as 1.

This happens as the algorithm analyses the the boxes before it went to the finish then values  it 1 as that box leads it to the reward of +1

now, lets us consider our bot starts from the top-left corner of the maze then the  values or messed up.
Hence this is not right

NOW WHAT??


That is how we use the bellman equation 

THIS IS IT :-  V(s) = max(R(s,a)+yV(S'))

V(s) stands for value of the state

the rest stands for maximum result of the action + the next state 

therefore when we apply the equation we see things in a more logical way we consider the maze in all point of views hence we get the above maze value where, when you are away the value decreases
and as you go near the value increases.Now let us say the AI starts in the bottom right corner the bot knows not to go up as the value is -1 and in left the value is +0.81 and hence goes left and so on to reach the best possible result +1.

Hope the knowledge will help,
Subscribe to my blog to be notified
Thank you
More Knowledge coming every alternative day!