Adjoint ODE

Question: Minimize $F(x, p)$

$$F(x, p) = \int_{0}^{T}f(x,p,t)dt \tag1$$

Subject to

$$g(x_0,p) = x_0 - p =0 \tag2$$

-> If given $x_0$, we can compute $p$ at $t=0$ through $g(x_0, p)$, then substitue $x_t$ and $p$ in $h(x_t, \dot{x}_t, p, t)$ and can compute $\dot{x}_t$

Why is $\frac{\partial{x}}{\partial p}$ difficult to calculate? Because $fn(x,p)$ is unknown, need to solve ALL possible $x$ and $p$ using ODE

Apply Lagrangian function $\mathcal{L} (x, \lambda)= f(x)- \lambda g (x)$ and combine (1) (2) (3) in one loss function

$$Loss = \int_{0}^{T}[f(x,p,t) + \lambda^Th(x_t, \dot{x}_t, p, t)]dt + u^T g(x_0,p) \tag4$$

Substitute (2) (3)

$$Loss = \int_{0}^{T}[f(x,p,t) + \lambda^T0]dt + u^T 0 = \int_{0}^{T}f(x,p,t)dt = F(x, p) \tag5$$

Why does integral equals loss? Why is this loss function? Minimize loss?

$$\frac{\partial{L}}{\partial p} = \frac{\partial{F}}{\partial p} \tag6$$

Why calculate $\frac{\partial{L}}{\partial p}$during backprop? To use Newton's Method to approximate f(x) at a given point

approximation using updated derivative

$$\frac{\partial{L}}{\partial p} = \frac{\partial{F}}{\partial p} = \int_{0}^{T}[\frac{\partial{f}}{\partial x}\cdot\frac{\partial{x}}{\partial p}+\frac{\partial{f}}{\partial p}+\lambda^T(\frac{\partial{h}}{\partial x}\cdot\frac{\partial{x}}{\partial p}+\frac{\partial{h}}{\partial \dot{x}}\cdot\frac{\partial{\dot{x}}}{\partial p}+\frac{\partial{h}}{\partial p})]dt+u^T(\frac{\partial{g}}{\partial x_0}\cdot\frac{\partial{x_0}}{\partial p}+\frac{\partial{g}}{\partial p})$$

To avoid compute $\frac{\partial{\dot{x}}}{\partial p}$ , we apply integration by parts $\int u dv = u v - \int v du$

$$\frac{\partial{L}}{\partial p} = \frac{\partial{L}}{\partial p}$$