Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization
Published in International Conference on Machine Learning (ICML), 2026
We study multi-regime behavior in scientific machine learning (SciML) models through a regime-aware diagnostic framework that jointly analyzes performance, training dynamics, and loss-landscape geometry. We identify a consistent three-regime structure across PINNs, Neural Operators, and Neural ODEs, show that optimization effectiveness is regime-specific, and uncover fine-grained failure modes (e.g., deceptive sharpness and flatness) that challenge conventional loss-landscape interpretations.
Recommended citation: Yuxin Wang, Yuanzhe Hu, Xiaokun Zhong, Xiaopeng Wang, Haiquan Lu, Tianyu Pang, Michael W. Mahoney, Yujun Yan, Pu Ren, Yaoqing Yang. (2026). "Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization." ICML 2026.
Download Paper