Here is a simple deterministic problem, without discrete modes for for illustrating the process law. We consider a classical 1D integrator.
\(\displaystyle
\left \lbrace
\begin{array}{l}
\min \frac{1}{2}\int_{0}^{T} u(t)^2 dt +\frac{1}{2}{X(T)}^2\\
\dot{x}=u(t)\\
u, x \in[-1, 1] \\
x(0)=0.1, T = 1.
\end{array}
\right .
\)
The following figure shows the value function at initial time. As expected, the value function is 0 when x=0, and increases symmetrically when getting farther form it.
We set the state probabilities at initial time according to a discrete uniform distribution
\(\displaystyle\mathcal{P}_0(x_i)_{i=1, \ldots, N_{grid}} = \frac{1}{N_{grid}}\)
The following animated figure shows the evolution of the state probabilities over the grid for each time step. We observe a concentration of the distribution towards x=0.
