Pangeo Forge Recipes Versions Save

What's Changed

• Retain original _FillValue in encoding by @maxrjones in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/711

New Contributors

• @maxrjones made their first contribution in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/711

Full Changelog: https://github.com/pangeo-forge/pangeo-forge-recipes/compare/0.10.6...0.10.7

What's Changed

• Ab/consolidate release notes by @abarciauskas-bgse in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/706
• Minimal multidim rechunk fix by @cisaacstern in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/708.

Full Changelog: https://github.com/pangeo-forge/pangeo-forge-recipes/compare/0.10.5...0.10.6

What's Changed

• Add pytest-xdist to speed up tests by @jbusecke in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/651
• Checkout tags for local install by @jbusecke in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/650
• Bump to myst-nb==1.0.0 by @cisaacstern in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/641
• Kerchunk WriteCombinedReference return fsstore by @norlandrhagen in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/660
• Consolidate Metadata Transform by @norlandrhagen in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/665
• Reuse FSSpecTarget Auth Credentials by @ranchodeluxe in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/668
• Pass zarr encoding through StoreToZarr() by @thodson-usgs in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/674
• Pinning pytest to fix ci test failures by @norlandrhagen in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/681
• ConsolidateMetadata2 by @norlandrhagen in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/678
• ConsolidatedDimensionCoords Transform -- cleaned up version of PR #556 by @norlandrhagen in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/671
• Update index.md by @abarciauskas-bgse in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/664
• Make mypy happy (type is not a class instance) by @moradology in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/693
• Add logging to StoreToZarr by @jbusecke in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/657
• Use MapTuple, put higher-order index type in types file by @moradology in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/694
• Tests to illustrate CombineReferences.bucket_by_position by @ranchodeluxe in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/697
• Fix CombineReference merge ordering by @moradology in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/689
• Assert Uniform by @ranchodeluxe in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/698
• Remove add_keys_iter by @moradology in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/700
• Fix WriteCombinedReference FSSpec Credentials Yet Again by @ranchodeluxe in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/703

New Contributors

• @thodson-usgs made their first contribution in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/674

Full Changelog: https://github.com/pangeo-forge/pangeo-forge-recipes/compare/0.10.4...0.10.5

What's Changed

• Fix broken contributing link by @cisaacstern in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/640
• Bump sphinxext-opengraph from 0.8.2 to 0.9.0 by @dependabot in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/642
• Add minio to Integration Tests by @norlandrhagen in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/638
• Disable fsspec Cache on Integration Tests by @ranchodeluxe in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/646
• Dynamic chunking interface for StoreToZarr by @cisaacstern in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/595

New Contributors

• @ranchodeluxe made their first contribution in https://github.com/pangeo-forge/pangeo-forge-recipes/pull/646

Full Changelog: https://github.com/pangeo-forge/pangeo-forge-recipes/compare/0.10.3...0.10.4

• Assign injection spec values for command line interface https://github.com/pangeo-forge/pangeo-forge-recipes/pull/566
• Docs rewrite https://github.com/pangeo-forge/pangeo-forge-recipes/pull/610
• Simplify reference recipe composition by nesting transforms https://github.com/pangeo-forge/pangeo-forge-recipes/pull/635
• Reference transforms dependency and bugfixes https://github.com/pangeo-forge/pangeo-forge-recipes/pull/631, https://github.com/pangeo-forge/pangeo-forge-recipes/pull/632

• Fix bug preventing use of multiple merge dims https://github.com/pangeo-forge/pangeo-forge-recipes/pull/591
• Add parquet storage target for reference recipes https://github.com/pangeo-forge/pangeo-forge-recipes/pull/620
• Support addition of dataset-level attrs for zarr recipes https://github.com/pangeo-forge/pangeo-forge-recipes/pull/622
• Integration testing upgrades https://github.com/pangeo-forge/pangeo-forge-recipes/pull/590 https://github.com/pangeo-forge/pangeo-forge-recipes/pull/605 https://github.com/pangeo-forge/pangeo-forge-recipes/pull/607 https://github.com/pangeo-forge/pangeo-forge-recipes/pull/611
• Add missing py.typed marker for mypy compatibility https://github.com/pangeo-forge/pangeo-forge-recipes/pull/613

• Add sentinel as default for transform keyword arguments that are required at runtime and which recipe developers may not want to set in recipe modules. This allows recipe modules to be importable (i.e., unit-testable) and type-checkable during development. https://github.com/pangeo-forge/pangeo-forge-recipes/pull/588
• StoreToZarr now emits a zarr.storage.FSStore which can be consumed by downstream transforms. This is useful for opening and testing the completed zarr store, adding it to a catalog, etc. https://github.com/pangeo-forge/pangeo-forge-recipes/pull/574
• Concurrency limiting transform added. This base transform can be used to limit concurrency for calls to external services. It is now used internally to allow OpenURLWithFSSpec to be limited to a specified maximum concurrency. https://github.com/pangeo-forge/pangeo-forge-recipes/pull/557
• Various packaging, testing, and maintenance upgrades https://github.com/pangeo-forge/pangeo-forge-recipes/pull/565 https://github.com/pangeo-forge/pangeo-forge-recipes/pull/567 https://github.com/pangeo-forge/pangeo-forge-recipes/pull/576
• Patched deserialization bug that affected rechunking on GCP Dataflow https://github.com/pangeo-forge/pangeo-forge-recipes/pull/548

Major breaking change: This release represents a nearly complete rewrite of the package, removing the custom recipe constructor classes and executors, and replacing them with a set of modular, domain-specific Apache Beam PTransforms, which can be flexibly composed and executed on any Apache Beam runner. The documentation has been updated to reflect this change. As the first release following this major rewrite, we expect bugs and documentation gaps to exist, and we look forward to community support in finding and triaging those issues. A blog post and further documentation of the motivations for and opportunities created by this major change is forthcoming.

Fixed bug in 0.9.3 which broke all recipe object imports if scipy (an optional dependency) was missing from environment. https://github.com/pangeo-forge/pangeo-forge-recipes/pull/466

• Bugfix to allow opening of zarr files. This fix allows using Zarr stores as an input source for recipes. https://github.com/pangeo-forge/pangeo-forge-recipes/pull/462
• Add netcdf3 support for opening source files with kerchunk. Resolves a long-standing issue wherein netcdf3 source files could not be loaded lazily, which effectively blocked the use of large netcdf3 files as recipe sources. https://github.com/pangeo-forge/pangeo-forge-recipes/pull/383
• Fix zarr reference bug https://github.com/pangeo-forge/pangeo-forge-recipes/pull/455
• Add dataset_type class attribute for recipe classes https://github.com/pangeo-forge/pangeo-forge-recipes/pull/437