Can Neural Networks Formulate Shock Wave?
PINN on Shock Waves
Physics-informed Neural Networks (PINNs) are a special type of neural networks. They estimate solutions to partial differential equations by incorporating the governing physical laws of a given dataset into the learning process.
An example of such an equation is the inviscid Burgers' equation, a prototype for conservation laws that can develop shock waves.
The current literature struggles to effectively tackle this issue. As shock waves are not continuous solutions, they only satisfy the equations in a weak sense. Continuous Time Models that depend solely on training samples, like the algorithmic differentiation method, cannot capture shock waves. These methods are only applicable to cases of functional regularity.
One could attempt to use Discrete Time Models where neural networks and time discretization work together to help the model formulate shocks. However, this method somewhat diminishes the advantages of Physics-informed Neural Networks (PINNs) and reverts to traditional numerical methods. This can be challenging for someone who understands equations but is not familiar with numerical solutions.
In this article, I will address the limitations of existing Continuous Time Models of PINN methods for the Burgers equation. I will introduce calculations for discontinuity and weak solutions based on algorithmic differentiation, enabling the equation to capture shocks. This article might inspire those who are interested in the intersection of neural networks and physics-based modeling, especially in domains related to conservation laws.
However, it should be noted that this method has only shown promising results for one of the simplest one-dimensional hyperbolic equations. Whether it can be extended to higher dimensions or more complex equations is an aspect that the author has not explored, and I invite readers to contribute their own ideas and resources on this topic.
PINN: Continuous Time Models for Burgers
According to the original paper: "Physics Informed Neural Networks (PINNs) are trained to solve supervised learning tasks whilst respecting any given laws of physics, described by general nonlinear partial differential equations (PDEs). "
These PDEs take the following form in general [1]:
ut + N [u] = 0, x ∈ Ω, t ∈ [0, T],
where u(t, x) represents the solution, N [·] is a nonlinear differential operator, and Ω is a subset of the d-dimensional space.
Let's denote by
L(u) = ut + N [u].
It can be immediately seen that f=0 if u is the solution of the equation. We will construct the solution u as a neural network
u = neural_net(t,x;weights)
where the inputs are the time and space variables. We determine the weights by minimizing the mean square error of f (as is said before, L(u) should be close to 0 if u is the solution of the equation) and certain initial and boundary conditions. For more details, please refer to the original paper.
Now, let's consider the 1-dimensional inviscid Burgers Equation:
The solution to the equation, adhering to the initial condition, can be constructed implicitly using the method of characteristics, that is, u=f(x-ut) with the characteristic curve x(t)= x0+f(x0)t. We see from the formula that the characteristics x(t) are straight lines without the same slope, so if there exists two points x1, x2 such that x1+f(x1)t= x2+f(x2)t at a finite time t, we will see the intersectfion of two characteristics and the wave breaks [2].
The following code is inspired by the git repository pinn-burgers. Here, a viscous Burgers' equation is considered for
