Blaylockbk Herbie Versions Save

Download numerical weather prediction datasets (HRRR, RAP, GFS, IFS, etc.) from NOMADS, NODD partners (Amazon, Google, Microsoft), ECMWF open data, and the University of Utah Pando Archive System.

2024.7.0

2 days ago

I'm excited about this release because I brought in some tools for plotting data that I've developed over the years from Carpenter_Workshop into Herbie. I hope people will test it, tell me if they are helpful for you, and tell me if I've broken anything.

Optional Dependencies

The "core" function of Herbie is to download data and read it with xarray. Thus, extra features, like plotting and extracting data at a point, now require installing "extra" dependencies that are not automatically installed. These optional dependencies include cartopy, metpy, matplotlib, and scikit-learn.

  1. Installing Herbie from conda-forge will always install extra dependencies.
  2. Install from pip pip install herbie-data will only install core dependencies.
  3. Install from pip pip install herbie-data[extras] will install dependencies for extra features.

I anticipate this might cause some breaking changes for people. If this change didn't get it right, please open an issue (and offer some help).

Numpy <2.0

Related to dependencies, it seems pygib isn't working with Numpy 2.0 https://github.com/jswhit/pygrib/issues/251, so I've tagged Herbie to require numpy<2.0. Please let me know if this changes or you have an idea to fix this. Actually, pygrib is only used by Herbie to parse the coordinate reference system from the grib files. It would be nice if Herbie didn't need to depend on pygrib (PR anyone??). Maybe pygrib should be another optional dependency.

Mature functions from Carpenter_Workshop moved into Herbie

I use my Carpenter Workshop repository as a workshop for building new stuff and testing ideas. Some of the tools I made there I use nearly every day (my EasyMap class for making Cartopy more simple), so I've migrated some of the mature features into Herbie.

  1. from herbie.toolbox import EasyMap EasyMap lets you make a cartopy axes without a lot of boilerplate code. I find it useful. Read more in the docs. 
    ax = EasyMay('50m', crs=..., figsize=[10,8]).STATES().LAND().OCEAN().BORDERS().ax
  2. from herbie.toolbox import ... Other stuff in the toolbox, like unit conversions
  3. from herbie import paint Paint is a bunch of additional colormaps I've collected over the years that I think look nice. For example, here are the National Weather Service standard color curves: image I'm not completely satisfied with how these are implemented and organized, so this is considered an unstable feature. The documentation is fresh and incomplete.

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2024.5.0...2024.7.0

2024.5.0

2 months ago

This release of Herbie has some new features and changes I've been developing and experimenting with, so I hope to hear from you if something isn't working as expected or if you have other ideas.


💣 Deprecation / Changes

I renamed the argument searchString to search. This change only affects you if you are using the keyword argument when calling the inventory or xarray methods. Herbie will now give a warning if you use searchString, so please update your scripts.

 from herbie import Herbie
 H = Herbie('2024-01-01')

- H.inventory(searchString="TMP:2 m")
- H.xarray(searchString="TMP:2 m")

+ H.inventory(r"TMP:2 m")
+ H.xarray(r"TMP:2 m")

The reason for the name change is 1) simplicity and 2) the search string isn't an exact match search but is a regex search. (I would have named it filter, but that is a python built-in and couldn't use that). I should also mention that it is recommended to use the r or raw string representation (i.e. search=r":TMP:\d+ mb") since the string is used as a regular expression.


✨ New Feature: ds.herbie.pick_points xarray accessor

I'm very excited to share this new feature. I created a new xarray accessor to get nearest neighbor point data from a model grid. I've had this on my mind for a long time and am happy with what I have made so far. This uses the Ball Tree algorithm, and thus adds scikit-learn as a new required dependency.

Here is a very simple example getting two values from the HRRR grid:

from herbie import Herbie
import pandas as pd

# Get HRRR 2-m temperature analysis
ds = Herbie('2024-1-1', model='hrrr').xarray("TMP:2 m")


# DataFrame of points to get from model grid
points = pd.DataFrame(
    {
        "longitude": [-100.25, -99.4],
        "latitude": [44.25, 45.4],
    }
)

# Pick points from HRRR grid
ds_points = ds.herbie.pick_points(points)

See more in-depth description and examples in the docs. There are options to get values from $k$ nearest neighbors from the requested point and compute the distance-weighted average of nearby points.

If you give this new feature a try, please let me know if you see any unexpected results.

📄 Documentation

I overhauled the organization of the documentation pages. Hopefully you find them organized a little better. Feel free to contribute to the docs if you see some details missing/misleading/confusing 😄


What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2024.3.1...2024.5.0



Future Changes

I am planning to rename the argument fxx for the forecast lead time to step to match the terminology used by cfgrib.

- Herbie('2023-01-01', model='hrrr', fxx=6)
+ Herbie('2023-01-01', model='hrrr', step=6)

Again, this isn't implemented yet, but I plan to implement it eventually and wanted to let you know. Provide any comments if you have any in https://github.com/blaylockbk/Herbie/issues/160

2024.3.1

3 months ago

Just two minor changes; wanted to get these out there for the few it may impact...

What's Changed

Note: the ds.herbie.plot() accessor is not implemented well. It could use a lot of work to make some standard, quick-look plots, which would be really cool, if anyone has some big ideas of what that should look like. I'm open to suggestions.

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2024.3.0...2024.3.1

2024.3.0

4 months ago

ECMWF shared this exciting news a few days ago about their open data products for the Integrated Forecast System (IFS)

ECMWF now provides a much larger open dataset to the public, representing weather forecasts at a higher resolution and a reduction in some release times.
- ECMWF: Media Centre

This Herbie release updates the ECMWF model templates to access the new IFS and AIFS 0.25 degree datasets. Historical access to the 0.4 degree datasets is still available for dates before February 1, 2024. Thanks @alcoat!

Deprecation

Now that ECMWF provides open access for two different models, the argument model='ecmwf' is deprecated. Instead, you should access the Integrated Forecast System (IFS) with model='ifs' or model='aifs for the Artificial Intelligence IFS.

- Herbie('2024-3-1', model='ecmwf')  # Deprecated

+ Herbie('2024-3-1', model='ifs')

  Herbie('2024-3-1', model='aifs')  # New Model

New Feature

Herbie has two new experimental functions that may be helpful to some people.

  • HerbieLatest Find the most recent available model run.
  • HerbieWait Wait for a particular model run to become available.

You can see examples in the docs: Herbie Latest Data. If these are helpful to you but you want them to be improved, I'm looking for people to help make these better, (along with FastHerbie which could use some work too).

Herbie at the American Meteorological Society

I gave a talk on Herbie at the American Meteorology Society in January. You can see my slides here. Soon, you should be able to go to the conference website and watch the recorded presentation.

It was a lot of fun talking to some of you at AMS this year. Its always a surprise to me when I'm wandering around the poster hall and someone says, "Hey Brian, Herbie was used to make this poster!" Thanks everyone for your kindness.

Please feel free to open issues when you see problems, share how you use Herbie in the discussions, and feel free to contribute your ideas and PRs.

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2023.12.4...2024.3.0

2023.12.4

6 months ago

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2023.12.3...2023.12.4

2023.12.3

6 months ago

What's Changed

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2023.12.2...2023.12.3

2023.12.2

6 months ago

What's Changed

Enhancements

Documentation

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2023.12.1...2023.12.2

2023.12.1

6 months ago

It's been a while since the last release, and I wanted to get out all the changes that have built up.

Please open an issue if you see problems. I'm trying a new "auto release on new tag" workflow action, so I may have some micro releases coming.

What's Changed

New model templates

Enhancements

Bug fixes

Documentation

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2023.3.0...2023.12.0

2023.3.0

1 year ago

Thanks again for using Herbie and your kind words. More than a few of you found me at the AMS annual meeting in Denver to thank me for publishing and maintaining Herbie. It's always encouraging to hear how I've helped someone. I'm still in shock how putting some code I wrote in grad school on the internet was discovered by so many. Hebie is still a project I work on in my free time, and I don't claim it is the most efficient or cleanly coded package, but the lessons I learn with Herbie have helped me in many ways in other work I do. So, I thank you for your involvement and use of Herbie and for helping me develop my skills.

Now I want to give a big "thank you" to these two people

  1. Ryan May (@dopplershift), thanks for instructing me on how to publish Herbie on conda-forge. This really simplifies installing Herbie given its dependency on cgfrib and cartopy. Herbie can now be installed like this:
conda install herbie-data

Note: The name of this package on pypi and conda is herbie-data because herbie is taken on PyPI.

  1. Andreas Motl (@amotl), thanks for your help with migrating the documentation to readthedocs and adding GitHub Actions to run the suite of tests with each pull request. Automating the doc builds and tests really helps.

These changes and other updates help make Herbie feel a little more "grown up" in this release.

Breaking Changes

  • I renamed the file herbie/archive.py to herbie/core.py because the name "archive" was starting to lose its meaning. Please pay attention to how you import Herbie...
# DON'T do this anymore
from herbie.archive import Herbie

# DO this
from herbie import Herbie
  • I renamed the Herbie method H.read_idx() to H.inventory() because the single word is easier for me to remember. If you use read_idx() you will get a warning for now; it will be removed later. Note that the inventory method is also available for FastHerbie. In fact, I'm migrating away from the term "index" to "inventory" because it I think it's a more accurate description of what a ".idx" file is--in inventory of the file contents. And besides, "index" means other things in python if you are a heavy Pandas user.
## DON'T do this anymore
H = Herbie(date, model)
H.read_idx()

# DO this
H = Herbie(date, model)
H.inventory()

New/Updated Model Templates

Note: There will be additional changes to the GEFS and GFS model templates to make the more consistent. I also plan to add a template for the URMA model since it exists.

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/2022.09.0...2022.x.x

2022.09.0

1 year ago

Huge shoutout to Ryan May for instructing me on how to publish Herbie on conda-forge. You can now install Herbie 2022.9.0.post1 with Conda

conda install -c conda-forge herbie-data

What's Changed

New Contributors

Full Changelog: https://github.com/blaylockbk/Herbie/compare/0.0.10...2022.09.0