Prithvi WxC: Foundation Model for Weather and Climate
akhaliq
Abstract
Triggered by the realization that AI emulators can rival the performance of traditional numerical weather prediction models running on HPC systems, there is now an increasing number of large AI models that address use cases such as forecasting, downscaling, or nowcasting. While the parallel developments in the AI literature focus on foundation models -- models that can be effectively tuned to address multiple, different use cases -- the developments on the weather and climate side largely focus on single-use cases with particular emphasis on mid-range forecasting. We close this gap by introducing Prithvi WxC, a 2.3 billion parameter foundation model developed using 160 variables from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2). Prithvi WxC employs an encoder-decoder-based architecture, incorporating concepts from various recent transformer models to effectively capture both regional and global dependencies in the input data. The model has been designed to accommodate large token counts to model weather phenomena in different topologies at fine resolutions. Furthermore, it is trained with a mixed objective that combines the paradigms of masked reconstruction with forecasting. We test the model on a set of challenging downstream tasks namely: Autoregressive rollout forecasting, Downscaling, Gravity wave flux parameterization, and Extreme events estimation. The pretrained model with 2.3 billion parameters, along with the associated fine-tuning workflows, has been publicly released as an open-source contribution via Hugging Face.
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- PuYun: Medium-Range Global Weather Forecasting Using Large Kernel Attention Convolutional Networks (2024)
- Efficient Localized Adaptation of Neural Weather Forecasting: A Case Study in the MENA Region (2024)
- FuXi Weather: An end-to-end machine learning weather data assimilation and forecasting system (2024)
- MetMamba: Regional Weather Forecasting with Spatial-Temporal Mamba Model (2024)
- Regional data-driven weather modeling with a global stretched-grid (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
๐ ๐๐ก๐ ๐๐ข๐ซ๐ฌ๐ญ ๐๐ฏ๐๐ซ ๐ ๐จ๐ฎ๐ง๐๐๐ญ๐ข๐จ๐ง ๐ฐ๐๐๐ญ๐ก๐๐ซ ๐ฆ๐จ๐๐๐ฅ: ๐๐ซ๐ข๐ญ๐ก๐ฏ๐ข ๐๐ฑ๐ ๐๐ง๐๐๐ฅ๐๐ฌ ๐ฅ๐ข๐๐-๐ฌ๐๐ฏ๐ข๐ง๐ ๐ฐ๐๐๐ญ๐ก๐๐ซ ๐๐จ๐ซ๐๐๐๐ฌ๐ญ๐ฌ
Hurricane Katrina killed hundreds of people as it made landfall on New Orleans in 2005 - many of these deaths could have been avoided if alerts had been given one day earlier. Accurate weather forecasts are really life-saving.
๐ฅ Now, NASA and IBM just dropped a game-changing new model: the first ever foundation model for weather! This means, it's the first time we have a generalist model not restricted to one task, but able to predict 160 weather variables!
Prithvi WxC (Prithvi, โเคชเฅเคฅเฅเคตเฅโ, is the Sanskrit name for Earth) - is a 2.3 billion parameter model, with an architecture close to previous vision transformers like Hiera.
But it comes with some important tweaks: under the hood, Prithvi WxC uses a clever transformer-based architecture with 25 encoder and 5 decoder blocks. It alternates between "local" and "global" attention to capture both regional and global weather patterns.
And boy, does it deliver.
๐๐ฒ๐ ๐ถ๐ป๐๐ถ๐ด๐ต๐๐:
๐ฎ ๐ก๐ฎ๐ถ๐น๐ ๐๐ต๐ผ๐ฟ๐-๐๐ฒ๐ฟ๐บ ๐ณ๐ผ๐ฟ๐ฒ๐ฐ๐ฎ๐๐๐ - Prithvi WxC crushed it on 6-12 hour predictions, even outperforming some traditional numerical weather models
๐ ๐ง๐ฟ๐ฎ๐ฐ๐ธ๐ ๐ต๐๐ฟ๐ฟ๐ถ๐ฐ๐ฎ๐ป๐ฒ๐ ๐น๐ถ๐ธ๐ฒ ๐ฎ ๐ฐ๐ต๐ฎ๐บ๐ฝ - For Hurricane Ida, it predicted the landfall location within 5 km (vs 20+ km errors from other AI models), which is a huge progress!
๐ ๐ฒ๐
๐ฑ๐ผ๐๐ป๐๐ฐ๐ฎ๐น๐ถ๐ป๐ด ๐ฝ๐ผ๐๐ฒ๐ฟ - Can zoom in on weather data to 6x higher resolution with 4x lower error than basic methods
๐ ๐ ๐ผ๐ฑ๐ฒ๐น๐ ๐ฒ๐น๐๐๐ถ๐๐ฒ ๐ด๐ฟ๐ฎ๐๐ถ๐๐ ๐๐ฎ๐๐ฒ๐ - Accurately simulates these crucial but hard-to-capture atmospheric oscillations
๐ The coolest part? ๐ฃ๐ฟ๐ถ๐๐ต๐๐ถ ๐ช๐
๐ ๐ถ๐๐ป'๐ ๐ฎ ๐ผ๐ป๐ฒ-๐๐ฟ๐ถ๐ฐ๐ธ ๐ฝ๐ผ๐ป๐. Its flexible design lets researchers fine-tune it for all kinds of specialized tasks. They've already adapted it for things like detailed regional climate projections, modeling tiny atmospheric gravity waves, and hurricane tracking.
This opens up tons of possibilities for improving climate models, severe weather prediction, and more. As climate change intensifies, tools like Prithvi WxC will become more and more crucial to avoid disasters!
Models citing this paper 3
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper