Blaylockbk Herbie Versions Save

Download numerical weather prediction datasets (HRRR, RAP, GFS, IFS, etc.) from NOMADS, NODD partners (Amazon, Google, Microsoft), ECMWF open data, and the University of Utah Pando Archive System.

0.0.10

2 years ago

⭐ Over 110 GitHub stars!

Wow, I had no idea so many people would find Herbie useful. Thanks everyone.

What's Changed

Full Changelog: https://github.com/blaylockbk/Herbie/compare/0.0.9...0.0.10

0.0.9

2 years ago

Changelog

Full Changelog: https://github.com/blaylockbk/Herbie/compare/0.0.8...0.0.9

0.0.8

2 years ago

Add access to ECMWF open data forecast products

The main feature of this release is the ability to retrieve ECMWF open data forecast products (see tweet)

The big change was implementing a method to read the grib_ls-style index files to get the byte ranges for specific variables/parameters. This means that the searchString argument will need to be specified differently than that used for other models. Read more about the searchString argument for grib_ls-style index files: https://blaylockbk.github.io/Herbie/_build/html/user_guide/searchString.html#grib-ls-style-index-files

For example:

from herbie.archive import Herbie
# Create Herbie object to discover ECMWF operational forecast product
H = Herbie("2022-01-26", model="ecmwf", product="oper", fxx=12)

# Download the full grib2 file
H.download()

# Download just the 10-m u and v winds
H.download(searchString=":10(u|v):")

# Retrieve the 500 hPa temperature as an xarray.Dataset
ds = H.xarray(searchString=":t:500:")

🏹 More examples for retrieving ECMWF open data

Changelog

Full Changelog: https://github.com/blaylockbk/Herbie/compare/0.0.7...0.0.8

Pretty Pictures

image image image

0.0.7

2 years ago

Changelog

  • Default config value for priority is now None, which will make the download source priority to be the order of the SOURCE in the models template files.
  • isort all imports
  • Fixed Issue #24: Use hash labels to name subset files to make unique file names so filenames don't get too long (Pull Request #26).
  • When loading data into xarray, Herbie will parse CF grid_mapping from grib file with pygrib/pyproj/metpy. (see CF Appendix F: Grid Mapping for more info). 2789859120883e32592f5559a1e211696b58cf3e
  • Lay the groundwork for local GRIB2 files that are not downloaded from remote sources. Suppose you ran a model locally (like WRF) and have that data stored locally. Herbie can be configured to find and read those local files! This assumes an index file also exists in the same directory with .idx appended to the file name.
    • Documentation needed.
  • Added model template for rap_ncei. This is poorly implemented because the NCEI data doesn't match the other RAP model URLs.
  • #24, implemented hashed filename for subset by @blaylockbk in https://github.com/blaylockbk/Herbie/pull/26
  • Add a Gitter chat badge to README.md by @gitter-badger in https://github.com/blaylockbk/Herbie/pull/21
  • Remove Gitter chat badge. I'm not committed to chatting on Gitter; just focus on discussions.
  • Add support for NCEI historical RAP analyses by @djgagne in https://github.com/blaylockbk/Herbie/pull/30
  • Handle different idx styles by @blaylockbk in https://github.com/blaylockbk/Herbie/pull/35 (more changes in addition to those by djgagne)
  • Fixed #33 so that the herbie.tools.bulk_download will return a dict of Herbie objects that were successful and those that failed.

Full Changelog: https://github.com/blaylockbk/Herbie/compare/0.0.6...0.0.7

0.0.6

2 years ago

Changelog

  • #18 Use TOML as config file format instead of INI
  • Expanded setting that can be set in the configuration file
  • Adopt Black formatting
  • Moved PyPI project from hrrrb to herbie-data

0.0.5

2 years ago

New Name! HRRR-B 🠖 Herbie

I updated the GitHub repository name to Herbie and I'm slowly removing the old hrrrb API (but it's still there).

The most significant change is that the vision of Herbie has expanded. Herbie is being built do download many different model types, not just the HRRR model.

  • Rename package to herbie. "Herbie is your model output download assistant with a mind of its own." Yes, this is named after a favorite childhood movie series.
  • Implement new Herbie class
  • Drop support for hrrrx (experimental HRRR no longer archived on Pando and ESRL is now developing RRFS)
  • Added ability to download and read RAP model GRIB2 files.
  • Less reliance on Pando, more on aws and google.
  • New method for searchString index file search. Uses same regex search patterns as old API.
  • Filename for GRIB2 subset includes all GRIB message numbers.
  • Moved default download source to config file setting.
  • Check local file copy on init. (Don't need to look for file on remote if we have local copy)
  • Option to remove grib2 file when reading xarray if didn't already exist locally (don't clutter local disk).
  • Attach index file DataFrame to object if it exists.
  • If full file exists locally, use remote idx file to cURL local file instead of remote. (Can't create idx file locally because wgrib2 not available on windows)
  • Added GFS data, though it isn't implemented as cleanly as HRRR or RAP
  • Renamed 'field' argument to 'product'
  • ✨ Moved the source URL templates to their own classes in the models folder
  • Renamed GitHub repository to Herbie (changed from HRRR_archive_download)
  • Added RRFS, NBM, GFS, RAP as models Herbie can download
  • Reworked read_idx() to support index files with additional info (specifically for the NBM).

0.0.4

3 years ago

New Herbie API

There are a few things about the hrrrb API that make it difficult to update, so I started changing things under the new name "herbie." But don't worry, both the hrrrb and herbie APIs are included. The setup.py file also is fixed.

To use the new Herbie API, refer to the documentation for some usage examples.

0.0.3

3 years ago

Welcome to the world, HRRR-B 🎂

This is my first initial GitHub release ever! I have published on PyPi before, but this is my first here on GitHub. Certainly, a happy birthday for the HRRR-B package.

Be aware, this is v0.0.3, meaning it is subject to change at my leisure. The purpose of this repository is to serve as an example of how you can download HRRR data from archives, but I try to keep this package in a workable state that might be useful for you.

📔 Documentation