Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,18 @@ Want to learn more about the project? We've presented Quartz Solar Forecast at t
- **LF Energy 2024**: Exploring Open Quartz's developments - new models, inverter APIs, and our Open Source journey at Open Climate Fix [Watch the talk](https://www.youtube.com/watch?v=YTaq41ztEDg)


The current model uses GFS or ICON NWPs to predict the solar generation at a site
The current model uses GFS, ICON, UKMO, or ECMWF (IFS/AIFS) NWPs to predict the solar generation at a site.
### NWP Data Sources

You can set the NWP source by passing `nwp_source` to `run_forecast`:

| Model | Provider | Country | Resolution | `nwp_source` |
|-------|-----------------|---------|------------|------------------|
| ICON | DWD | Germany | 2 - 11 km | `icon` (default) |
| GFS. | NOAA | USA | 3 - 25 km | `gfs` |
| IFS | ECMWF | EU | 9 - 25 km | `ecmwf_ifs` |
| AIFS | ECMWF | EU | ~25 km | `ecmwf_aifs` |
| UKMO | UK Met Office | UK | 2 - 10 km | `ukmo` |

```python
from quartz_solar_forecast.forecast import run_forecast
Expand Down Expand Up @@ -106,7 +117,7 @@ Two models are currently available to make predictions.

**Gradient Boosting Model** (default)

The model uses GFS or ICON NWPs to predict the solar generation at a site.
The model uses NWP data from GFS, ICON, UKMO, or ECMWF (IFS/AIFS) to predict the solar generation at a site.
It is a gradient boosted tree model and uses 9 NWP variables.
It is trained on 25,000 PV sites with over 5 years of PV history, which is available [here](https://huggingface.co/datasets/openclimatefix/uk_pv).
The training of this model is handled in [pv-site-prediction](https://github.com/openclimatefix/pv-site-prediction)
Expand Down Expand Up @@ -166,7 +177,7 @@ _Predictions using the two different models and different data sources._
## Known restrictions

- The model is trained on [UK MetOffice](https://www.metoffice.gov.uk/services/data/met-office-weather-datahub) NWPs, but when running inference we use [GFS](https://www.ncei.noaa.gov/products/weather-climate-models/global-forecast) data from [Open-meteo](https://open-meteo.com/). The differences between GFS and UK MetOffice could led to some odd behaviours.
- Depending, whether the timestamp for the prediction lays more than 90 days in the past or not, different data sources for the NWP are used. If we predict within the last 90 days, we can use ICON or GFS from the open-meteo Weather Forecast API. Since ICON doesn't provide visibility, this parameter is queried from GFS in any case. If the date for the prediction is further back in time, a reanalysis model of historical data is used (open-meteo | Historical Weather API). The historical weather API doesn't provide visibility at all, that's why it's set to a maximum of 24000 meter in this case. This can lead to some loss of precision.
- Depending, whether the timestamp for the prediction lays more than 90 days in the past or not, different data sources for the NWP are used.If we predict within the last 90 days, we can use ICON, GFS, UKMO, or ECMWF (IFS/AIFS) from the open-meteo Weather Forecast API. Since ICON doesn't provide visibility, this parameter is queried from GFS in any case. If the date for the prediction is further back in time, a reanalysis model of historical data is used (open-meteo | Historical Weather API). The historical weather API doesn't provide visibility at all, that's why it's set to a maximum of 24000 meter in this case. This can lead to some loss of precision.
- The model was trained and tested only over the UK, applying it to other geographical regions should be done with caution.
- When using the XGBoost model, only hourly predictions within the last 90 days are available for data consistency.

Expand Down Expand Up @@ -255,6 +266,10 @@ To set up the development environment for this project, follow these steps:
- MAE: Mean Absolute Error
- [ICON](https://www.dwd.de/EN/ourservices/nwp_forecast_data/nwp_forecast_data.html): ICOsahedral Nonhydrostatic
- KW: Kilowatt
- UKMO: United Kingdom Met Office
- ECMWF: European Centre for Medium-Range Weather Forecasts
- IFS: Integrated Forecasting System
- AIFS: Artificial Intelligence Forecasting System

## FOSDEM

Expand Down
6 changes: 3 additions & 3 deletions quartz_solar_forecast/data.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

:param site: the PV site
:param ts: the timestamp for when you want the forecast for
:param nwp_source: the nwp data source. Either "gfs", "icon" or "ukmo". Defaults to "icon"
:param nwp_source: the nwp data source. Either "gfs", "icon" "ukmo" or "ecmwf". Defaults to "icon"

Check failure on line 24 in quartz_solar_forecast/data.py

View workflow job for this annotation

GitHub Actions / branch_ci / lint-typecheck

ruff (E501)

quartz_solar_forecast/data.py:24:101: E501 Line too long (102 > 100)
:return: nwp forecast in xarray
"""
now = datetime.now()
Expand Down Expand Up @@ -61,9 +61,9 @@
else:
# Getting NWP from open meteo weather forecast API by ICON, GFS, or UKMO
# within the last 3 months
url_nwp_source = {"icon": "dwd-icon", "gfs": "gfs", "ukmo": "ukmo_seamless"}.get(nwp_source)
url_nwp_source = {"icon": "dwd-icon", "gfs": "gfs", "ukmo": "ukmo_seamless","ecmwf":"ecmwf"}.get(nwp_source)

Check failure on line 64 in quartz_solar_forecast/data.py

View workflow job for this annotation

GitHub Actions / branch_ci / lint-typecheck

ruff (E501)

quartz_solar_forecast/data.py:64:101: E501 Line too long (116 > 100)
if not url_nwp_source:
raise Exception(f'Source ({nwp_source}) must be either "icon", "gfs", or "ukmo"')
raise Exception(f'Source ({nwp_source}) must be either "icon", "gfs", "ukmo" or "ecmwf"')

Check failure on line 66 in quartz_solar_forecast/data.py

View workflow job for this annotation

GitHub Actions / branch_ci / lint-typecheck

ruff (E501)

quartz_solar_forecast/data.py:66:101: E501 Line too long (102 > 100)
url = (
f"https://api.open-meteo.com/v1/"
f"{url_nwp_source if nwp_source != 'ukmo' else 'forecast'}"
Expand Down
Loading