> For the complete documentation index, see [llms.txt](https://yinqi-h.gitbook.io/ml-models/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://yinqi-h.gitbook.io/ml-models/models/differential-equations/ordinary-differential-equations/neural-ordinary-differential-equations-ode.md).

# Neural Ordinary Differential Equations (ODE)

{% hint style="info" %}
return X, y #upper case: matrix, lower case: vector
{% endhint %}

## Traditional neural network:

$$
x \rightarrow f(x) \rightarrow y
$$

$$
f(x) = ax + b
$$

$$
loss = (f(x) - y)^2 = (ax + b -y)^2
$$

$$
a = a - \frac{\partial loss}{\partial a} \tag{gradient descent}
$$

$$
a = a - 2(ax + b - y) \cdot x \cdot LR
$$

$$
b = b - 2(ax + b - y) \cdot Y \cdot LR
$$

## Neural ODE:

$$
\theta = \[a,b]
$$

$$
\frac{\partial z}{\partial t} = f(z, t, \theta)
$$

$$
x \rightarrow g(x) \rightarrow y = z\_0 \rightarrow g(x) \rightarrow z\_t
$$

$$
loss = (z\_t - ODE(f(z\_0)))^2
$$

$$
\frac{\partial loss}{\partial z\_T} = 2 \times (z\_t - ODE(f(z\_0)))
$$

$$
\theta = \theta - 2\frac{\partial loss}{\partial \theta}  \cdot LR
$$

[ODE solver](https://github.com/rtqichen/torchdiffeq)