Adjoint Sensitive Method

Apply adjoint sensitive method to loss:

From $loss = (z_t - ODE(z_0))^2$ to $\frac{\partial a_t}{\partial t} = -a_t^T \frac{\partial f(z_t, t, \theta)}{\partial z}$ where $a_t = \frac{\partial loss}{\partial z_t}$

When there is a unknown vector u and two known matrix A and B, and we want to compute a product

$$u^T B \text{ such that } AB = C$$

If there is a vector v and compute

$$v^T C \text{ such that } A^Tv = u$$

Thus

$$v^T C = v^T AB = (A^T v)^T B = u^TB$$

Turn unknown B and unknown u to one known C and one unknown v