Building Flexible Machine Learning Models for Scientific Computing at Scale

arXiv:2402.16014v1 Announce Type: new
Abstract: Foundation models have revolutionized knowledge acquisition across domains, and our study introduces OmniArch, a paradigm-shifting approach designed for building foundation models in multi-physics scientific computing. OmniArch's pre-training involves a versatile pipeline that processes multi-physics spatio-temporal data, casting forward problem learning into scalable auto-regressive tasks, while our novel Physics-Informed Reinforcement Learning (PIRL) technique during fine-tuning ensures alignment with physical laws. Pre-trained on the comprehensive PDEBench dataset, OmniArch not only sets new performance benchmarks for 1D, 2D …

abstract acquisition arxiv auto building computing cs.ai cs.lg data domains foundation knowledge knowledge acquisition machine machine learning machine learning models paradigm physics pipeline pre-training processes scalable scale study tasks temporal training type