Question about Equations Containing Higher Order Derivatives

[braveyjc]

Thank you very much for your open-source code, it is truly helpful and inspiring！
Regarding the Kuramoto-Sivashinsky equation, I noticed that you employed a modified MLP structure and utilized the Taylor-mode AD method. I would like to inquire whether these methods are necessary. Can one achieve high-precision convergence results by noly using the causality PINN?
In my case, I have reproduced your algorithm using TensorFlow without installing jax, and I have been able to solve the Allen-Cahn equation, obtaining results that are close to those showcased in your open-source code. However, I did not employ the modified MLP structure or the Taylor-mode AD method you mentioned. As a result, I have been unable to converge to the exact solution, as the residual loss starts at a magnitude of 10^6 and then decreases to around 10^0 without further improvement.

[sifanexisted]

Thank you for your query!

From our observations, the plain MLPs often struggle to sufficiently minimize the PDE residuals for some of the more challenging benchmarks. This highlights the crucial role a good network backbone plays in enhancing the performance of PINNs. Furthermore, we found that a modified MLP tends to consistently outperform MLPs with minimal modifications. That being said, the focus should be on an architecture adept at minimizing PDE residuals, which doesn't necessarily have to be a modified MLP.

Regarding Taylor-mode AD, our primary use for it is to speed up the training process. However, it doesn't directly effect the predictive accuracy.

Hope this helps.

[braveyjc]

Thank you very much for your response!
Your answer has completely resolved my confusion. I greatly appreciate your help. I wish you all the best in your future research.

[braveyjc]

Dear Dr. Wang:
Thank you very much for your open-source code again.
1.The correct solution for the Navier-Stokes equations in the code you shared only includes the complete solution for vorticity. I need a reference solution for the velocity field (u, v). Could you please share this part of the results? 2.You mentioned the concept of "batch-size" in the paper. However, it seems from the code that you are not using a mini-batch strategy. Can you provide a more detailed explanation about this? 3.For two-dimensional time-dependent problems, the number of sampling points can become very large, and Casual PINN seems to do the simultaneous computation of all training points in a single training epoch. If the data exceeds the storage capacity of the computer, what can be done? Have you considered combining Casual PINN with a mini-batch strategy, apart from parallel computing methods?
Thanks for your attention. Looking forward your reply.
Yours, sincerely,
Jianchuan

[sifanexisted]

Hello,

Thank you for reaching us again.

I would recommend checking our latest work and implementation (https://github.com/PredictiveIntelligenceLab/jaxpi), which not only integrates causal training but autmatically supports multi-gpu and random sampling.

Also please check the ns_tori example. The problem setup is the same as this one. And the reference solution contains the velocity field.

Hope this helps! Please let me know if you any questions

[braveyjc]

Dear Dr. Wang:
I greatly appreciate your response; it has effectively resolved my issue. Your most recent achievements are both intriguing and inspiring. I eagerly anticipate further advancements from your team and wish you continued success in your research endeavors!