In this post, we will solve a simple Ordinary Differential Equation (ODE) in three different ways: analytically using SymPy, numerically using the Euler method, and with Physics-Informed Neural Networks (PINNs) in PyTorch. At the end, we'll compare their results and insights.

Problem Definition

We want to solve the following ODE:

$$\frac{dy}{dt} = 2t - y$$

with the initial condition:

$$y(0) = -1$$

1. Symbolic Solution with SymPy

Key steps in the symbolic approach:

  • Define symbolic variables (dependent and independent)
  • Create the differential equation using sp.Eq
  • Solve without initial conditions to see the general solution
  • Apply initial conditions to get the particular solution

Complete SymPy implementation:

Loading code from sympy_solution.py...

The exact solution is:

$$y(t) = 2t - 2 + e^{-t}$$

2. Numerical Solution with the Euler Method

The Euler method is one of the simplest numerical solvers for ODEs. It uses the iterative formula:

$$y_{i+1} = y_i + h \cdot f(t_i, y_i)$$

where h is the time step.

Think about it: instead of solving the entire equation, we just "walk along the slope" (gradient) at each step. It's like saying: I don't know the full path, but I can guess the next step if I know the slope right here.

That's the beauty of the gradient that it can act as a guide to approximate the solution.

The implementation includes:

  • ODE function definition: f(t, y) = 2t - y
  • Euler method algorithm with iterative approximation
  • Time grid setup and numerical solution calculation

Complete Euler method implementation:

Loading code from euler_method.py...

3. Physics-Informed Neural Networks (PINNs)

So far, we've solved the ODE with SymPy (analytical) and Euler (numerical). Now, let's see how a Physics-Informed Neural Network (PINN) can approximate the same solution.

The key idea of PINNs:

  • We don't give the network explicit training data for y(t)
  • Instead, we give it the physics of the problem:
    • The ODE residual (how well the network prediction satisfies the differential equation)
    • The initial condition (and boundary conditions if needed)

This means we are asking the ML model:

"I don't know the exact form of y(t), but I know what its derivative looks like (dy/dt), and I know how it should behave at specific points (initial/bounday conditions). Please learn a function that satisfies these."

Training Setup

We first prepare our training domain and initial condition, then define the PINN model with a simple neural network architecture.

Loss Function

We design the loss with two main components:

  1. Physics loss (ODE residual):
$$\mathcal{L}_{\text{phys}} = \left\| \frac{d\hat{y}}{dt} - (2t - \hat{y}) \right\|^2$$

This enforces the ODE itself.

  1. Initial condition loss:
$$\mathcal{L}_{\text{IC}} = \left( \hat{y}(0) - y(0) \right)^2$$

This enforces the known starting value.

Total loss:

$$\mathcal{L} = \mathcal{L}_{\text{phys}} + \mathcal{L}_{\text{IC}}$$

Complete PINN implementation:

Loading code from pinn_solution.py...

Compared to Euler, the PINN is more flexible: it doesn't need explicit discretization of the ODE, and it "learns" the solution space directly from physics.

Summary

Symbolic methods like SymPy are perfect when you need the exact, closed-form solution. They're mathematically rigorous but limited to equations that have analytical solutions.

Numerical methods like Euler's are workhorses for practically any ODE, even when no closed-form solution exists. The trade-off is accuracy vs. computational cost, smaller step sizes give better accuracy but require more computation.

PINNs represent a newer paradigm that combines the flexibility of neural networks with physics constraints. They're particularly powerful for complex geometries, unknown parameters, or when you have sparse data but strong physical knowledge.