How to Build Your Own AI Weather Forecasting System
Written on
Chapter 1: Introduction to AI Weather Forecasting
Inexpensive and data-driven weather forecasts can now be created using pre-trained models that yield results comparable to traditional numerical models. Companies and research institutions, such as Huawei with PanguWeather, NVIDIA with FourCastNet, and Google DeepMind with GraphCast, are at the forefront of developing these AI weather models. The European Centre for Medium-Range Weather Forecasts (ECMWF) offers tools for generating forecasts using these systems, which can even run on a laptop, although utilizing a GPU is recommended for optimal performance.
This video covers the fourth part of building your own AI assistant, focusing on the weather skill, providing a practical example of how AI can enhance weather forecasting.
Section 1.1: Background on Weather Forecasting
Traditionally, weather forecasting relies on complex numerical models that require substantial computational resources, limiting the number of services capable of generating global forecasts. In contrast, AI weather models utilize historical weather data, known as reanalysis data, to predict future conditions, significantly reducing the computational burden.
AI weather forecasting diagram illustrating the prediction process using current weather data as input.
Section 1.2: Getting Started with AI Models
To begin, you will need to set up your coding environment. Acquire the necessary code from the ECMWF GitHub repository and follow the setup instructions. I recommend using version 0.2.5 for this guide. Create a conda environment to manage the required Python packages:
conda create -n ai-models python=3.10
conda activate ai-models
conda install cudatoolkit
pip install ai-models
To install specific AI models, use:
pip install ai-models-panguweather
pip install ai-models-fourcastnet
The installation of GraphCast has specific steps that will be discussed later.
Chapter 2: Generating Your AI Weather Forecast
Before you generate a weather forecast, you need to download the pre-trained model weights for PanguWeather, GraphCast, and FourCastNet:
ai-models --download-assets --assets assets-panguweather panguweather
ai-models --download-assets --assets assets-fourcastnet fourcastnet
ai-models --download-assets --assets assets-graphcast graphcast
Initialization data is essential for the AI models to make accurate predictions. You can obtain this data from either the ECMWF service MARS or the Copernicus Climate Data Store (CDS), both of which offer free access for non-commercial use. For this tutorial, I'll be utilizing reanalysis data from the CDS.
This video demonstrates how to build your own AI weather bot, showcasing the step-by-step process and potential applications in real-world scenarios.
Section 2.1: Creating Your Forecast
To create an AI weather forecast, select a date and time for initialization and commence model inference with the pre-trained models. For instance, to request a PanguWeather forecast starting on 09/20/2023 at 00:00 UTC based on CDS data, execute:
ai-models --input cds --date 20230920 --time 0000 --assets assets-panguweather panguweather
The logs generated during this process will provide detailed information about the model’s performance and inference times.
Section 2.2: Comparing AI Weather Models
When running forecasts with FourCastNet, similar commands can be used, and the performance will vary based on whether a GPU is utilized. For example, using the GPU will significantly speed up the inference process, allowing for quicker generation of forecasts.
After generating your forecasts, you can visualize the output files, which are available in GRIB format. For a 240-hour forecast, file sizes will differ between the models, with PanguWeather producing larger files due to its extensive pressure level data.
Summary
In summary, this guide has outlined the steps to create AI weather forecasts using models from the ECMWF repository. By leveraging initialization data from the Climate Data Store, you can generate forecasts using models like Huawei's PanguWeather and NVIDIA's FourCastNet. The efficiency of these AI models allows even individuals and smaller weather services to produce their forecasts, marking a significant advancement in weather prediction technology.