Core Utilities

This reference covers the helper functions exported from firecube.core.api for use inside plugin hooks: URI and filesystem handling, dataset preparation, and time conversion.

RESERVED_ARRAY_ATTRS, assert_attrs_safe(), and FIRECUBE_STATIC_WRITTEN_ATTR describe the array attribute keys Firecube owns for Zarr writes. The sealing marker constants stamped by firecube zarr preallocate and firecube zarr consolidate-time-coord are operator-tooling surface; they are documented with those commands in CLI reference.

compare_zarr_stores() performs a read-only, driver-aware comparison of two Zarr stores and returns ZarrCompareReport.

Name	Description
`parse_uri`	Parse a URI/path into a minimal fsspec-like structure.
`is_remote_target`	Return True when `target` is a non-local URI (e.g. s3://...).
`infer_target_protocol`	Infer the fsspec protocol for a target string.
`local_path_from_target`	Resolve a target string into an absolute local path.
`create_filesystem_for_uri`	Create a driver-aware filesystem for a concrete product/output URI.
`path_stats`	Get bytes and file count for a path (local or remote).
`ensure_directory`	Ensure a directory exists, creating parents if needed.
`delete_path`	Delete a path (file or directory) from storage.
`discover_input_files`	Discover input files from a local path or remote URI.
`prepare_netcdf_for_zarr`	Convenience wrapper: rename time dimension then clean encodings.
`clean_netcdf_encoding`	Strip HDF5 chunk hints from all variable encodings.
`normalize_string_vars`	Normalize string-typed variables in a NetCDF-loaded Dataset for Zarr writes.
`rename_time_dim`	Rename the time dimension to Firecube's append convention.
`read_hdf5_array`	Read a named array from a local HDF5(-like) file, with xarray fallback.
`materialize_hdf5_path`	Return a local HDF5 path for a source path (ZIP or direct HDF5-like file).
`extract_all_from_zips`	Extract every member of each ZIP archive, optionally in parallel.
`epoch_s_to_iso`	Convert whole seconds-since-epoch to a canonical `"YYYY-MM-DDTHH:MM:SSZ"`.
`iso_to_epoch_s`	Convert an ISO-8601 UTC timestamp to whole seconds since Unix epoch.
`coerce_to_epoch_s`	Coerce a coordinate value to seconds since the Unix epoch (UTC).
`normalize_epoch_iso`	Round-trip an ISO-8601 UTC string through `epoch_s` and back to `"...Z"`.
`RESERVED_ARRAY_ATTRS`	Reserved Zarr array attribute keys managed by Firecube.
`assert_attrs_safe`	Raise ValueError if attrs contains any reserved key.
`FIRECUBE_STATIC_WRITTEN_ATTR`	Zarr array attr Firecube stamps after a static array commit.
`BatchResourceRegistry`	Track closeable resources by batch and close each batch idempotently.
`physical_chunk_keys_for_region`	Return physical chunk keys touched by one region write.
`chunk_axis_range`	Return chunk indices intersected by a half-open axis selection.
`axis_selection_is_chunk_aligned`	Return whether a half-open axis selection aligns to chunk boundaries.
`ZarrCompareReport`	Summary of a read-only comparison between two Zarr stores.
`compare_zarr_stores`	Compare two Zarr stores through the configured storage abstraction.

URI And Filesystem

firecube.core.api.parse_uri

parse_uri(target)

Parse a URI/path into a minimal fsspec-like structure.

Returns a dict with

protocol: "file", "s3", ...
path: protocol-specific path (e.g. "bucket/prefix/key" for s3, "/abs/path" for file)

firecube.core.api.is_remote_target

is_remote_target(target)

Return True when target is a non-local URI (e.g. s3://...).

firecube.core.api.infer_target_protocol

infer_target_protocol(target)

Infer the fsspec protocol for a target string.

Returns "file" for local paths and file:// URIs.

firecube.core.api.local_path_from_target

local_path_from_target(target)

Resolve a target string into an absolute local path.

Supports relative paths and file:// URIs.

firecube.core.api.create_filesystem_for_uri

create_filesystem_for_uri(uri, storage_config, *, format)

Create a driver-aware filesystem for a concrete product/output URI.

firecube.core.api.path_stats

path_stats(
    uri, *, storage_config=None, exclude_substrings=None
)

Get bytes and file count for a path (local or remote).

Works with both local paths and S3 URIs.

Parameters:

Name	Type	Description	Default
`uri`	`str`	Local path or remote URI	required
`storage_config`	`Any \| None`	Optional StorageConfig for S3 credentials	`None`
`exclude_substrings`	`Iterable[str] \| None`	Path substrings to exclude (default: manifest paths)	`None`

Returns:

Type	Description
`dict[str, int]`	Dict with 'bytes' and 'files' keys

firecube.core.api.ensure_directory

ensure_directory(path)

Ensure a directory exists, creating parents if needed.

firecube.core.api.delete_path

delete_path(
    base_uri,
    subpath,
    *,
    storage_config=None,
    allow_manifest_paths=False,
    dry_run=False,
    filesystem=None,
)

Delete a path (file or directory) from storage.

Parameters:

Name	Type	Description	Default
`base_uri`	`str`	Base URI (e.g., 's3://bucket/store.zarr' or '/data/store.zarr')	required
`subpath`	`str`	Relative path within base_uri to delete (e.g., 'F072')	required
`storage_config`	`Any \| None`	Optional StorageConfig for S3 credentials	`None`
`allow_manifest_paths`	`bool`	If True, allow deleting manifest paths	`False`
`dry_run`	`bool`	If True, only check existence without deleting	`False`

Returns:

Type	Description
`dict[str, Any]`	Dict with keys: path, exists, deleted

Source Discovery

firecube.core.api.discover_input_files

discover_input_files(
    source,
    *,
    storage_config=None,
    include_suffixes=(".zip", ".h5", ".nc"),
    preferred_globs=None,
    recursive=True,
    sniff_hdf5=True,
    exclude=None,
)

Discover input files from a local path or remote URI.

Selection is intentionally conservative and format-agnostic:

Accept files matching include_suffixes.
Optionally accept extensionless files that look like HDF5.
Optionally add files matched by preferred_globs. Patterns add to the suffix selection; they do not replace it.
Drop anything matching exclude before selection runs, so an excluded path is never considered by suffix, sniffing, or patterns.

Glob patterns in preferred_globs and exclude are matched against the file's base name, its path relative to source, and its full path or URI, so both "*.nc4" and "subdir/*.nc4" are usable.

Parameters:

Name	Type	Description	Default
`source`	`str \| Path`	Discovery root: a local path or a remote URI such as `s3://bucket/prefix`.	required
`storage_config`	`Any \| None`	Storage settings used to reach a remote `source`.	`None`
`include_suffixes`	`Sequence[str]`	File suffixes accepted by default.	`('.zip', '.h5', '.nc')`
`preferred_globs`	`Iterable[str] \| None`	Extra glob patterns whose matches are added.	`None`
`recursive`	`bool`	Search below `source`; when `False`, only entries directly in `source` are returned.	`True`
`sniff_hdf5`	`bool`	Accept extensionless files whose content looks like HDF5. Applies to local sources only.	`True`
`exclude`	`Iterable[str] \| None`	Glob patterns whose matches are dropped.	`None`

Returns:

Type	Description
`list[str]`	URI/path strings (for example `/tmp/data/file.nc` or
`list[str]`	`s3://bucket/prefix/file.nc`), sorted for deterministic batching.

Raises:

Type	Description
`ValueError`	If `source` cannot be opened or listed.

Dataset Preparation

firecube.core.api.prepare_netcdf_for_zarr

prepare_netcdf_for_zarr(
    ds, time_dim="time", target_time_dim="timestamp"
)

Convenience wrapper: rename time dimension then clean encodings.

Applies all standard NetCDF→Zarr V3 preparation steps in order: 1. Rename time dimension from time_dim to target_time_dim. 2. Strip HDF5 chunk hints from all variable encodings.

firecube.core.api.clean_netcdf_encoding

clean_netcdf_encoding(ds)

Strip HDF5 chunk hints from all variable encodings.

Removes 'chunks', 'chunksizes', and 'preferred_chunks' from each variable's encoding dict so they do not conflict with the Zarr chunk layout chosen by the core writer. Modifies encoding dicts in place; returns the same dataset object.

firecube.core.api.normalize_string_vars

normalize_string_vars(
    ds,
    *,
    iso_targets=None,
    logger=None,
    preserve_cf_time_attrs=False,
)

Normalize string-typed variables in a NetCDF-loaded Dataset for Zarr writes.

Detects string vars by dtype kind (S, U, or O with all-string content) and applies one of:

ISO conversion: vars named in iso_targets are converted from ISO-8601 UTC strings to datetime64[s] or datetime64[us] (auto-detected by fractional-second presence). By default, units and calendar attrs (if present on the source variable) are moved from .attrs to .encoding on the converted variable — matching xarray's convention for decoded CF time metadata (metadata consumed during conversion is relocated to encoding rather than deleted). Set preserve_cf_time_attrs=True to keep them verbatim in .attrs; note that firecube's CF advisor may then flag the variable. The returned ISO-converted variable always receives fresh .encoding — source encoding hints (chunks, dtype, pre-existing time-decode keys) are NOT copied.
UTF-8 decode: other string vars are decoded to <U*> (bytes decoded, object arrays consolidated to fixed-width).

Variables whose dtype is datetime64 are silently skipped even if named in iso_targets (already decoded — nothing to do). Variables with non-string object content are also silently skipped, with a DEBUG log when logger is provided.

Raises ValueError only when a name in iso_targets exists in the dataset AND its data cannot be processed (empty object array, non-string object content, unparseable ISO string, or invalid UTF-8 bytes). A name in iso_targets that is absent from the dataset is silently skipped.

This utility normalizes detectable string variables. It does NOT validate whole-dataset Zarr writability; callers who require strict schemas should run a separate validator.

Parameters:

Name	Type	Description	Default
`ds`	`Dataset`	Input dataset (post-concat is fine; handles vlen-string widening).	required
`iso_targets`	`Collection[str] \| None`	Variable names to convert to `datetime64`. `None` skips ISO conversion (UTF-8 decode is still applied to other string vars).	`None`
`logger`	`Logger \| None`	Optional logger for DEBUG-level skip diagnostics.	`None`
`preserve_cf_time_attrs`	`bool`	If True, keep `units` and `calendar` attrs on ISO-converted variables verbatim in `.attrs` and do NOT write them to `.encoding`. Default False moves them to `.encoding` (xarray convention). Ignored for CF-time attr handling when no variable is ISO-converted; other string normalization still runs.	`False`

Returns:

Type	Description
`Dataset`	A new dataset. The input is not mutated.

firecube.core.api.rename_time_dim

rename_time_dim(ds, source='time', target='timestamp')

Rename the time dimension to Firecube's append convention.

If source exists as a dimension or coordinate, rename it to target. If source is not present, return the dataset unchanged (no error).

firecube.core.api.read_hdf5_array

read_hdf5_array(
    hdf5_path, *, variable, dtype=None, logger=None
)

Read a named array from a local HDF5(-like) file, with xarray fallback.

Parameters:

Name	Type	Description	Default
`hdf5_path`	`Path`	Path to the HDF5 file.	required
`variable`	`str`	Name of the variable/dataset to read.	required
`dtype`	`dtype[Any] \| str \| None`	Optional dtype to cast to. If omitted, the source dtype is preserved.	`None`
`logger`	`Logger \| None`	Optional logger for debug messages.	`None`

Returns:

Type	Description
`ndarray[Any, Any]`	Numpy array containing the requested dataset.

Raises:

Type	Description
`KeyError`	If the variable is not found.
`RuntimeError`	If dependencies are missing or read fails.

Examples:

Read a dataset without changing its dtype:

>>> read_hdf5_array(Path("product.h5"), variable="counts")

Cast explicitly when the caller needs a different dtype:

>>> read_hdf5_array(Path("product.h5"), variable="counts", dtype="float32")

firecube.core.api.materialize_hdf5_path

materialize_hdf5_path(
    file_path, *, extract_root=None, logger=None
)

Return a local HDF5 path for a source path (ZIP or direct HDF5-like file).

If the input is a ZIP file, it extracts the content to a temporary directory. If it is already an HDF5-like file, it is returned as-is.

Parameters:

Name	Type	Description	Default
`file_path`	`Path`	Source file path.	required
`extract_root`	`Path \| None`	Optional root directory for temporary extraction.	`None`
`logger`	`Logger \| None`	Optional logger.	`None`

Returns:

Type	Description
`tuple[Path, TemporaryDirectory \| None]`	Tuple of (path_to_hdf5_file, temporary_directory_object_or_None).

ZIP Extraction

firecube.core.api.extract_all_from_zips

extract_all_from_zips(
    zip_paths, dest_dir_for, *, workers=1
)

Extract every member of each ZIP archive, optionally in parallel.

Destination directories are resolved by calling dest_dir_for once per archive, serially and in input order, before any extraction starts, so the callable needs no locking. Each archive is then fully extracted into its directory; member names that could escape it (.. segments, absolute paths, Windows drive prefixes) are rejected before anything is written.

A failing archive never raises and never aborts the batch: its partially extracted directory is removed and the failure is reported in the result. Callers MUST check the returned failures mapping — an unsafe member name or a corrupt archive is reported there, not as an exception. workers=1 extracts serially; higher values extract concurrently with identical failure semantics. Extraction is disk-bound, so workers composes with, and is independent of, the engine's pipeline_workers option; a plugin running several pipeline workers multiplies the two, so keep workers modest.

Parameters:

Name	Type	Description	Default
`zip_paths`	`Sequence[Path]`	Archives to extract.	required
`dest_dir_for`	`Callable[[Path], Path]`	Callable mapping each archive path to its destination directory. Called once per archive before extraction begins.	required
`workers`	`int`	Upper bound on concurrent extractions, capped at the number of archives. Defaults to serial extraction. Ingestion plugins conventionally pass the engine's `extract_workers` option here so operators control it with `--option extract_workers=N`.	`1`

Returns:

Type	Description
`dict[Path, Path]`	An `(extracted, failures)` pair: `extracted` maps each
`dict[Path, str]`	successfully extracted archive to its destination directory, and
`tuple[dict[Path, Path], dict[Path, str]]`	`failures` maps each failed archive to its error message. Every
`tuple[dict[Path, Path], dict[Path, str]]`	input path appears in exactly one of the two mappings.

Raises:

Type	Description
`ValueError`	If `workers` is less than 1.

Examples:

Extract a batch of archives next to each archive:

>>> from pathlib import Path
>>> extracted, failures = extract_all_from_zips(
...     [Path("a.zip"), Path("b.zip")],
...     lambda zip_path: zip_path.parent / zip_path.stem,
...     workers=4,
... )
>>> if failures:
...     raise RuntimeError(f"failed archives: {failures}")

Time Conversion

firecube.core.api.epoch_s_to_iso

epoch_s_to_iso(seconds)

Convert whole seconds-since-epoch to a canonical "YYYY-MM-DDTHH:MM:SSZ".

firecube.core.api.iso_to_epoch_s

iso_to_epoch_s(iso)

Convert an ISO-8601 UTC timestamp to whole seconds since Unix epoch.

Only UTC inputs are accepted. The trailing "Z" or the explicit "+00:00" offset are both honoured; any other offset raises ValueError. Numpy is imported lazily so that slot_index stays cheap to import for code paths that never touch the epoch helpers.

firecube.core.api.normalize_epoch_iso

normalize_epoch_iso(iso)

Round-trip an ISO-8601 UTC string through epoch_s and back to "...Z".

Equivalent to epoch_s_to_iso(iso_to_epoch_s(iso)); raises ValueError for non-UTC inputs via iso_to_epoch_s.

coerce_to_epoch_s() is documented with the direct-Zarr index types in Index Specification.

Reserved Array Attributes

firecube.core.api.RESERVED_ARRAY_ATTRS `module-attribute`

RESERVED_ARRAY_ATTRS = frozenset(
    {
        _ARRAY_DIMENSIONS_ATTR,
        _FILL_VALUE_ATTR,
        _FIRECUBE_RUN_ID_ATTR,
        _FIRECUBE_SPAN_ID_ATTR,
        _FIRECUBE_INTERNAL_ATTR,
        FIRECUBE_STATIC_WRITTEN_ATTR,
        FIRECUBE_GROUP_IDENTITY_HASH_ATTR,
        "firecube_coord_managed",
        "firecube_preallocated",
        "firecube_consolidated_at",
    }
)

Reserved Zarr array attribute keys managed by Firecube.

Examples:

>>> "firecube_static_written" in RESERVED_ARRAY_ATTRS
True
>>> "my_custom_attr" in RESERVED_ARRAY_ATTRS
False

firecube.core.api.assert_attrs_safe

assert_attrs_safe(attrs)

Raise ValueError if attrs contains any reserved key.

Parameters:

Name	Type	Description	Default
`attrs`	`Mapping[str, Any]`	Candidate attribute mapping for `ZarrArraySpec.attrs`.	required

Returns:

Name	Type	Description
`None`	`None`	The function returns nothing.

Raises:

Type	Description
`ValueError`	If any key is reserved for Firecube bookkeeping.

Examples:

>>> assert_attrs_safe({"my_custom_attr": 42})
>>> assert_attrs_safe({"firecube_static_written": True})
Traceback (most recent call last):
...
ValueError: Reserved attr 'firecube_static_written' is managed by firecube, not the plugin. Remove it from ZarrArraySpec.attrs.

firecube.core.api.FIRECUBE_STATIC_WRITTEN_ATTR `module-attribute`

FIRECUBE_STATIC_WRITTEN_ATTR = 'firecube_static_written'

Zarr array attr Firecube stamps after a static array commit.

Examples:

>>> FIRECUBE_STATIC_WRITTEN_ATTR
'firecube_static_written'

The sealing markers record who owns a time-coordinate array's values. firecube zarr preallocate stamps ATTR_PREALLOCATED on grid-valued coordinates and ATTR_COORD_MANAGED on observed-values coordinates it materializes; firecube zarr consolidate-time-coord stamps ATTR_PREALLOCATED together with ATTR_CONSOLIDATED_AT.

firecube.core.api.ATTR_PREALLOCATED `module-attribute`

ATTR_PREALLOCATED = 'firecube_preallocated'

Zarr array attr stamped on a time coord array after dense preallocate materialization.

Presence of this attr (with value True) means the coord array was written dense during firecube zarr preallocate. Subsequent write_timestamp calls perform an equality drift check instead of creating new chunk files.

firecube.core.api.ATTR_COORD_MANAGED `module-attribute`

ATTR_COORD_MANAGED = 'firecube_coord_managed'

Zarr array attr for an engine-managed unsealed coordinate.

Presence of this attr (with value True) means the materializer may fill NaT holes during staged seeding, while pods verify-or-error on writes. It is mutually exclusive with ATTR_PREALLOCATED.

firecube.core.api.ATTR_CONSOLIDATED_AT `module-attribute`

ATTR_CONSOLIDATED_AT = 'firecube_consolidated_at'

Zarr array attr stamped on a time coord array after consolidate-time-coord.

Value is an ISO 8601 UTC timestamp string recording when consolidation ran. A ConsolidatedTimeCoord WAL event accompanies this attr; ResumeGuard reads that event to block further ingest on the sealed cube.

firecube.core.api.assert_coord_markers_consistent

assert_coord_markers_consistent(attrs, coord_path)

Raise SchemaDriftError when a coordinate carries both sealing markers.

firecube_preallocated and firecube_coord_managed are mutually exclusive lifecycles for a time-coordinate array; their combined presence means a run crashed mid-transition or the store was edited out of band. Tooling that is about to stamp or trust either marker calls this first with the array's attribute dict and its store path (used in the error message). A consistent attribute set returns None.

Batch Resources

firecube.core.api.BatchResourceRegistry

Track closeable resources by batch and close each batch idempotently.

The registry is intentionally small: plugin mixins may register per-batch resources during batch_setup and ask the registry to close everything for the same batch id during batch_teardown. Teardown pops the batch before closing resources, so repeated teardown calls for the same batch are no-ops even when an earlier close raises.

Examples:

Register and close a per-batch resource:

>>> class Resource:
...     def __init__(self):
...         self.closed = False
...     def close(self):
...         self.closed = True
>>> registry = BatchResourceRegistry()
>>> resource = registry.register("batch-1", Resource())
>>> registry.teardown("batch-1")
>>> resource.closed
True

Methods:

register

register(batch_id, resource)

Register a closeable resource for a batch id.

Parameters:

Name	Type	Description	Default
`batch_id`	`Hashable`	Hashable batch identifier. The same value must be passed to `teardown` to close the registered resources.	required
`resource`	`_ResourceT`	Object exposing `close()`.	required

Returns:

Type	Description
`_ResourceT`	The same `resource` object, so callers can create and register in
`_ResourceT`	one expression.

Raises:

Type	Description
`TypeError`	If `batch_id` is not hashable or `resource` does not expose a callable `close` method.

Examples:

Register a file-like object and keep using it:

>>> registry = BatchResourceRegistry()
>>> handle = registry.register("batch-1", open(__file__))
>>> callable(handle.close)
True
>>> registry.teardown("batch-1")

teardown

teardown(batch_id)

Pop and close all resources registered for one batch id.

Parameters:

Name	Type	Description	Default
`batch_id`	`Hashable`	Hashable batch identifier to close and remove.	required

Returns:

Type	Description
`None`	`None`.

Raises:

Type	Description
`Exception`	Re-raises the first exception raised by a resource's `close()` after attempting to close every resource. Later close exceptions are logged and suppressed behind the first.

Examples:

Repeated teardown is safe and closes the resource once:

>>> registry = BatchResourceRegistry()
>>> resource = registry.register("batch-1", open(__file__))
>>> registry.teardown("batch-1")
>>> registry.teardown("batch-1")

teardown_all

teardown_all()

Tear down every registered batch id.

Intended for pipeline-start drains of batches whose per-batch teardown never ran (crash paths). Like teardown, every close is attempted and the first exception is re-raised at the end.

Examples:

>>> registry = BatchResourceRegistry()
>>> registry.register("batch-1", open(__file__))
>>> registry.register("batch-2", open(__file__))
>>> registry.teardown_all()

Zarr Chunk Geometry

firecube.core.api.physical_chunk_keys_for_region

physical_chunk_keys_for_region(
    *, group, intent, shape, chunks, selection
)

Return physical chunk keys touched by one region write.

Parameters:

Name	Type	Description	Default
`group`	`str`	Logical Zarr group used as the first element of each key.	required
`intent`	`Any`	Write intent with `group` and `array` attributes for error messages and chunk-key construction.	required
`shape`	`tuple[int, ...]`	Full target array shape. Rank 3 is interpreted as `(time, y, x)`; rank 4 as `(time, y, x, channel)`.	required
`chunks`	`tuple[int, ...]`	Target array chunk shape with the same rank as `shape`.	required
`selection`	`Any`	Region selection with `ts_index`, `y_start`, `y_stop`, and optional `channel_index` attributes.	required

Returns:

Type	Description
`set[tuple[str, str, tuple[int, ...]]]`	A `(keys, aligned)` pair. `keys` contains ``(group, array,
`bool`	chunk_coords)`tuples.`aligned`` is true when the write is aligned
`tuple[set[tuple[str, str, tuple[int, ...]]], bool]`	to whole physical chunks along the selected non-X axes and the time
`tuple[set[tuple[str, str, tuple[int, ...]]], bool]`	chunk size is one.

Raises:

Type	Description
`ValueError`	If `selection.ts_index` is outside `shape`.

Examples:

Compute the one physical chunk touched by a chunk-aligned 2-row write:

>>> from types import SimpleNamespace
>>> intent = SimpleNamespace(group="data", array="values")
>>> selection = SimpleNamespace(ts_index=0, y_start=2, y_stop=4, channel_index=None)
>>> physical_chunk_keys_for_region(
...     group="data",
...     intent=intent,
...     shape=(1, 6, 4),
...     chunks=(1, 2, 4),
...     selection=selection,
... )
({('data', 'values', (0, 1, 0))}, True)

firecube.core.api.chunk_axis_range

chunk_axis_range(start, stop, chunk_size)

Return chunk indices intersected by a half-open axis selection.

Parameters:

Name	Type	Description	Default
`start`	`int`	Inclusive selected element index.	required
`stop`	`int`	Exclusive selected element index.	required
`chunk_size`	`int`	Physical chunk length along the axis.	required

Returns:

Type	Description
`range`	A `range` of intersected chunk indices. Empty or zero-length
`range`	selections return an empty range.

Raises:

Type	Description
`ZeroDivisionError`	If `chunk_size` is zero.

Examples:

A misaligned [1, 5) selection with chunk size two touches three chunks:

>>> list(chunk_axis_range(1, 5, 2))
[0, 1, 2]

firecube.core.api.axis_selection_is_chunk_aligned

axis_selection_is_chunk_aligned(
    start, stop, axis_len, chunk_size
)

Return whether a half-open axis selection aligns to chunk boundaries.

Parameters:

Name	Type	Description	Default
`start`	`int`	Inclusive selected element index.	required
`stop`	`int`	Exclusive selected element index.	required
`axis_len`	`int`	Full axis length. A selection ending at this value is aligned even when the final chunk is partial.	required
`chunk_size`	`int`	Physical chunk length along the axis.	required

Returns:

Type	Description
`bool`	`True` when `start` begins on a chunk boundary and `stop` is
`bool`	either a chunk boundary or the axis end; otherwise `False`.

Raises:

Type	Description
`ZeroDivisionError`	If `chunk_size` is zero.

Examples:

A whole final partial chunk is aligned when it reaches the axis end:

>>> axis_selection_is_chunk_aligned(4, 5, 5, 2)
True

Zarr Store Comparison

firecube.core.api.ZarrCompareReport `dataclass`

Summary of a read-only comparison between two Zarr stores.

Attributes:

Name	Type	Description
`equivalent`	`bool`	Whether every compared array path, schema field, selected attribute, static marker, and value payload matched.
`mismatches`	`list[str]`	Terse mismatch descriptions grouped by array path and category.

Examples:

>>> ZarrCompareReport(equivalent=True, mismatches=[]).equivalent
True

Methods:

to_dict

to_dict()

Return a JSON-serializable representation of the report.

firecube.core.api.compare_zarr_stores

compare_zarr_stores(
    a_uri, b_uri, *, storage_type, storage_driver
)

Compare two Zarr stores through the configured storage abstraction.

The comparison is read-only and checks array paths, shape, dtype, chunks, native Zarr dimension names, public attrs, the Firecube static-array marker, and full array values. Runtime-managed attrs such as firecube_run_id and firecube_span_id are ignored.

Parameters:

Name	Type	Description	Default
`a_uri`	`str`	First Zarr store URI.	required
`b_uri`	`str`	Second Zarr store URI.	required
`storage_type`	`str`	Storage locality, either `"local"` or `"s3"`.	required
`storage_driver`	`str`	Storage driver, either `"fsspec"` or `"obstore"`.	required

Returns:

Name	Type	Description
`ZarrCompareReport`	`ZarrCompareReport`	`equivalent=True` when no mismatches were found;
	`ZarrCompareReport`	otherwise `equivalent=False` with one terse message per mismatch.

Raises:

Type	Description
`FileNotFoundError`	If either store or a discovered array is missing.
`ValueError`	If the storage configuration is invalid.

Examples:

>>> report = compare_zarr_stores(
...     "file:///tmp/a.zarr",
...     "file:///tmp/b.zarr",
...     storage_type="local",
...     storage_driver="fsspec",
... )
>>> isinstance(report.equivalent, bool)
True

Core Utilities

URI And Filesystem

firecube.core.api.parse_uri

firecube.core.api.is_remote_target

firecube.core.api.infer_target_protocol

firecube.core.api.local_path_from_target

firecube.core.api.create_filesystem_for_uri

firecube.core.api.path_stats

firecube.core.api.ensure_directory

firecube.core.api.delete_path

Source Discovery

firecube.core.api.discover_input_files

Dataset Preparation

firecube.core.api.prepare_netcdf_for_zarr

firecube.core.api.clean_netcdf_encoding

firecube.core.api.normalize_string_vars

firecube.core.api.rename_time_dim

firecube.core.api.read_hdf5_array

firecube.core.api.materialize_hdf5_path

ZIP Extraction

firecube.core.api.extract_all_from_zips

Time Conversion

firecube.core.api.epoch_s_to_iso

firecube.core.api.iso_to_epoch_s

firecube.core.api.normalize_epoch_iso

Reserved Array Attributes

firecube.core.api.RESERVED_ARRAY_ATTRS module-attribute

firecube.core.api.assert_attrs_safe

firecube.core.api.FIRECUBE_STATIC_WRITTEN_ATTR module-attribute

firecube.core.api.ATTR_PREALLOCATED module-attribute

firecube.core.api.ATTR_COORD_MANAGED module-attribute

firecube.core.api.ATTR_CONSOLIDATED_AT module-attribute

firecube.core.api.assert_coord_markers_consistent

Batch Resources

firecube.core.api.BatchResourceRegistry

Methods:

register

teardown

teardown_all

Zarr Chunk Geometry

firecube.core.api.physical_chunk_keys_for_region

firecube.core.api.chunk_axis_range

firecube.core.api.axis_selection_is_chunk_aligned

Zarr Store Comparison

firecube.core.api.ZarrCompareReport dataclass

Methods:

to_dict

firecube.core.api.compare_zarr_stores

See Also

firecube.core.api.RESERVED_ARRAY_ATTRS `module-attribute`

firecube.core.api.FIRECUBE_STATIC_WRITTEN_ATTR `module-attribute`

firecube.core.api.ATTR_PREALLOCATED `module-attribute`

firecube.core.api.ATTR_COORD_MANAGED `module-attribute`

firecube.core.api.ATTR_CONSOLIDATED_AT `module-attribute`

firecube.core.api.ZarrCompareReport `dataclass`