Index Specification

This reference covers the public types used to declare and resolve direct-Zarr index specs. TimeAxis provides the recommended intent-named constructors for time axes. IndexSpec, RegularTimeAxis, IntegerAxis, and IrregularTimeAxis describe the layout. AUTO is the sentinel that tells the engine to discover coordinates at planning time. inspect_item() returns ItemInfo, and resolve_index_spec() turns the declarative spec into a cached ResolvedIndex for the run.

coerce_to_epoch_s() normalizes supported timestamp values to Unix epoch seconds.

ResolvedIndexRecord is the on-disk control-plane record written after the engine resolves an IndexSpec.

Name	Description
`TimeAxis`	Intent-named constructors for time-axis declarations.
`AxisSpec`	Marker base for axis specifications.
`IndexSpec`	Declarative index specification for a multi-group DirectZarr product.
`IntegerAxis`	A zero-based integer axis with a fixed slot count.
`IrregularTimeAxis`	A time axis with explicit coordinate values.
`AUTO`	Sentinel that tells `IrregularTimeAxis` to discover coordinate values at planning time.
`RegularTimeAxis`	A regularly-spaced time axis with a fixed epoch and cadence.
`ItemInfo`	Metadata returned by `inspect_item` describing a single source item.
`ResolvedIndex`	A resolved, immutable index for a multi-group DirectZarr product.
`ResolvedIndexRecord`	On-disk record for an engine-resolved index payload.
`coerce_to_epoch_s`	Coerce a coordinate value to seconds since the Unix epoch (UTC).
`resolve_index_spec`	Resolve an `IndexSpec` into a `ResolvedIndex`.

Index Types

Import these from firecube.ingestor.api.

firecube.ingestor.api.TimeAxis

Intent-named constructors for time-axis declarations.

The four constructors cover every supported time-axis shape and are the recommended way to declare one; each returns a plain axis dataclass, so the explicit RegularTimeAxis / IrregularTimeAxis forms remain available as an escape hatch.

Pick by answering one question -- what defines your product's timeline?

A fixed cadence, and the coordinate should carry the grid labels: :meth:grid.
A fixed cadence, but the coordinate should carry each slot's real observation time: :meth:observed.
A known list of timestamps with no fixed cadence: :meth:explicit.
Timestamps only your source items can reveal: :meth:discovered.

Examples:

>>> axis = TimeAxis.observed(
...     coordinate="time",
...     epoch="2025-07-01T00:00:00Z",
...     cadence_s=600,
...     slot_count=4320,
... )
>>> spec = IndexSpec(name="my_product_v1", groups={"data": axis})

Methods:

grid `staticmethod`

grid(
    *,
    coordinate,
    epoch,
    cadence_s,
    slot_count=None,
    end_date=None,
    placement="exact",
)

A fixed-cadence axis whose coordinate carries the grid labels.

The coordinate values are epoch + n * cadence_s, known before ingest, so the engine materializes and seals them at preallocate. Item timestamps must sit exactly on grid boundaries; off-grid observation times need :meth:observed instead, because sealed grid values cannot pass drift verification against off-grid writes.

Parameters:

Name	Type	Description	Default
`coordinate`	`str`	Name of the time coordinate dimension.	required
`epoch`	`str`	UTC-explicit ISO 8601 axis origin.	required
`cadence_s`	`int`	Slot cadence in seconds (positive integer).	required
`slot_count`	`int \| None`	Total slots; mutually exclusive with `end_date`.	`None`
`end_date`	`str \| None`	Exclusive UTC-explicit axis end; mutually exclusive with `slot_count`. Leave both `None` for serial mode.	`None`
`placement`	`Literal['exact', 'floor']`	Must be `"exact"`; `"floor"` raises `ValueError` (use :meth:`observed` for floor placement).	`'exact'`

observed `staticmethod`

observed(
    *,
    coordinate,
    epoch,
    cadence_s,
    slot_count=None,
    end_date=None,
)

A fixed-cadence axis whose coordinate carries real observation times.

Items are placed on the grid by flooring, but the coordinate stores each slot's actual observation time (for example a sensing start such as 00:00:02 in a ten-minute cadence). Those values only exist once the source items are known, so the engine writes them in a single-writer materialization step; ingest verifies them.

Parameters:

Name	Type	Description	Default
`coordinate`	`str`	Name of the time coordinate dimension.	required
`epoch`	`str`	UTC-explicit ISO 8601 axis origin.	required
`cadence_s`	`int`	Slot cadence in seconds (positive integer).	required
`slot_count`	`int \| None`	Total slots; mutually exclusive with `end_date`.	`None`
`end_date`	`str \| None`	Exclusive UTC-explicit axis end; mutually exclusive with `slot_count`. Leave both `None` for serial mode.	`None`

explicit `staticmethod`

explicit(*, coordinate, values)

An axis whose timeline is a known, explicit list of timestamps.

Use when the product has no fixed cadence but the full timeline is known when the plugin is configured. The engine sorts the values ascending, assigns slot indices in that order, and materializes the coordinate at preallocate. The rest of the plugin is unchanged: resolved_index(ctx).position(group, timestamp) maps each value to its declared slot.

Parameters:

Name	Type	Description	Default
`coordinate`	`str`	Name of the time coordinate dimension.	required
`values`	`Sequence[Any]`	Non-empty sequence of distinct coordinate values (`datetime`, `numpy.datetime64`, or integers). Strings, bytes, duplicates, and empty input are rejected.	required

Examples:

>>> import numpy as np
>>> axis = TimeAxis.explicit(
...     coordinate="timestamp",
...     values=(
...         np.datetime64("2026-01-01T00:00:00", "ns"),
...         np.datetime64("2026-01-01T00:17:30", "ns"),
...         np.datetime64("2026-01-01T00:42:00", "ns"),
...     ),
... )

discovered `staticmethod`

discovered(*, coordinate)

An axis whose timeline is discovered from the source items.

Use when only the inputs can reveal the timestamps. Before preallocate the engine calls inspect_item on every discovered source item, sorts the coordinates ascending, and freezes the resulting axis. That costs one full source pass at planning time. The declaration is IrregularTimeAxis(values=AUTO), the spelling error messages use.

Discovery makes inspect_item load-bearing:

Return ItemInfo(coordinate=timestamp) for every item that belongs on the axis.
Return None to skip an item entirely.
Return ItemInfo(coordinate=None) only when the item exists but its coordinate cannot be resolved; discovery fails with MissingIrregularCoordinateError for it.

inspect_item runs during discovery and again during the write phase, so it must be idempotent and independent of discovery order, and items must resolve through references that stay valid for the run because the discovered axis is handed to parallel workers. Duplicate coordinates raise DuplicateIrregularCoordinateError; zero discovered items raise NoDiscoveredItemsError.

Parameters:

Name	Type	Description	Default
`coordinate`	`str`	Name of the time coordinate dimension.	required

firecube.ingestor.api.AxisSpec

Marker base for axis specifications.

RegularTimeAxis, IntegerAxis, and IrregularTimeAxis are the current implementations. Additional axis types are planned as additive extensions in future releases. The resolver in index_resolve.py dispatches on isinstance -- no abstract methods are required here.

firecube.ingestor.api.IndexSpec `dataclass`

Declarative index specification for a multi-group DirectZarr product.

When groups has more than one entry and the ingestion caller does not pass slot_group to name one authoritative group, every group must reference the same axis object (Python is identity, not value equality): slot-range filtering operates on one canonical axis, and two structurally equal axes give the engine no way to pick one. Separately constructed axes with identical fields are rejected with ConfigurationError at bind time; share the instance instead, or pass --slot-group on the command line.

Attributes:

Name	Type	Description
`name`	`str`	Stable identifier for this index configuration. Used as the `SlotIndexModel.name` for byte-identity checks.
`groups`	`Mapping[str, AxisSpec]`	Mapping from group name to axis specification. Must be non-empty. Normalized to a sorted tuple internally for hashability.
`time_unit`	`str \| None`	Optional time unit string forwarded to the legacy `SlotIndexModel` unchanged (e.g. `None` for the default).

Examples:

>>> axis = RegularTimeAxis(
...     coordinate="timestamp",
...     epoch="2024-01-01T00:00:00Z",
...     cadence_s=600,
...     end_date="2024-01-08T00:00:00Z",  # or slot_count=1008
... )
>>> spec = IndexSpec(
...     name="my_product_v1",
...     groups={"data_1km": axis, "data_2km": axis},  # same object
... )

Attributes

name `instance-attribute`

name

Stable identifier for this index configuration.

groups `instance-attribute`

groups

Mapping from group name to axis specification.

time_unit `class-attribute` `instance-attribute`

time_unit = None

Optional time unit forwarded to the legacy SlotIndexModel.

firecube.ingestor.api.IntegerAxis `dataclass`

Bases: AxisSpec

A zero-based integer axis with a fixed slot count.

Declares an integer position with no epoch or cadence. inspect_item returns the integer position as the item's coordinate, and resolved_index(ctx).position(group, key) maps it to the slot index. A single IndexSpec may mix IntegerAxis and time-axis groups; each group resolves its own axis independently.

Attributes:

Name	Type	Description
`slot_count`	`int`	Total number of integer positions on the axis (positive integer). Sets the shape of the axis dimension in the preallocated Zarr store: arrays indexed by this axis are created with shape `(slot_count, ...)`.

Examples:

>>> spec = IndexSpec(
...     name="my_mixed_product_v1",
...     groups={
...         "data": TimeAxis.observed(
...             coordinate="timestamp",
...             epoch="2024-01-01T00:00:00Z",
...             cadence_s=600,
...             end_date="2024-01-08T00:00:00Z",
...         ),
...         "lookup": IntegerAxis(slot_count=64),
...     },
... )

Attributes

slot_count `instance-attribute`

slot_count

Total number of integer positions on the axis (positive integer).

firecube.ingestor.api.IrregularTimeAxis `dataclass`

Bases: AxisSpec

A time axis with explicit coordinate values.

Attributes:

Name	Type	Description
`coordinate`	`str`	Name of the time coordinate dimension. Must match `time_dim_name` when the axis is resolved.
`values`	`Sequence[Any] \| _AutoSentinel`	Explicit coordinate values for the axis, or `AUTO` to discover them later. Accepts any non-empty `Sequence` of hashable comparable values, typically `datetime` objects, `numpy.datetime64` values, or integers. Duplicate values raise `ValueError`. A non-`Sequence` raises `TypeError`. Empty input raises `ValueError`. The engine sorts concrete values ascending and assigns slot indices in that order.

Note

AUTO triggers planning-time discovery: the engine scans the full source set via inspect_item before preallocate so it can discover coordinates and sort them. That costs a full source pass.

Attributes

coordinate `instance-attribute`

coordinate

Name of the time coordinate dimension.

values `instance-attribute`

values

Explicit coordinate values, or AUTO for planning-time discovery.

firecube.ingestor.api.AUTO `module-attribute`

AUTO = _AutoSentinel()

Sentinel that tells IrregularTimeAxis to discover coordinate values at planning time.

Set IrregularTimeAxis(coordinate=..., values=AUTO) to let the engine call inspect_item on every source item before preallocate and build the axis from the returned coordinates. The engine sorts discovered coordinates and assigns each a slot index in ascending order.

Importable from firecube.core.api and firecube.ingestor.api.

firecube.ingestor.api.RegularTimeAxis `dataclass`

Bases: AxisSpec

A regularly-spaced time axis with a fixed epoch and cadence.

Attributes:

Name	Type	Description
`coordinate`	`str`	Name of the time coordinate dimension (e.g. `"time"`).
`epoch`	`str`	UTC-explicit ISO 8601 string for the axis origin (e.g. `"2024-01-01T00:00:00Z"`). Naive strings are rejected.
`cadence_s`	`int`	Slot cadence in seconds. Must be a positive integer.
`mode`	`Literal['exact', 'floor']`	Alignment mode. `"exact"` requires timestamps to fall exactly on slot boundaries; `"floor"` maps each timestamp to the nearest preceding boundary.
`end_date`	`str \| None`	Optional UTC-explicit ISO 8601 string for the axis end (exclusive). Must be aligned to the cadence and strictly after `epoch`. At most one of `end_date` and `slot_count` may be set.
`slot_count`	`int \| None`	Optional total number of slots. Must be positive. At most one of `end_date` and `slot_count` may be set.

Note

Both end_date and slot_count may be None for serial-mode plugins that do not declare a fixed horizon. The parallel gate will raise ConfigurationError if it cannot determine the extent.

Attributes

coordinate `instance-attribute`

coordinate

Name of the time coordinate dimension.

epoch `instance-attribute`

epoch

UTC-explicit ISO 8601 epoch string.

cadence_s `instance-attribute`

cadence_s

Slot cadence in seconds (positive integer).

mode `class-attribute` `instance-attribute`

mode = 'exact'

Alignment mode: "exact" or "floor".

end_date `class-attribute` `instance-attribute`

end_date = None

UTC-explicit ISO 8601 timestamp for the axis end (exclusive).

The last written slot's left edge is at end_date - cadence_s; the axis covers the half-open interval [epoch, end_date).

Must be strictly after epoch and aligned to cadence_s (i.e. (end_date - epoch) must be a whole multiple of cadence_s). Misalignment raises ValueError with the two nearest aligned boundaries in the error message.

Mutually exclusive with slot_count. Leave both None for serial-mode plugins without a fixed horizon.

slot_count `class-attribute` `instance-attribute`

slot_count = None

Total number of slots on the axis (positive integer).

Equivalent to (end_date - epoch) // cadence_s when end_date is provided instead; the two spellings are algebraically identical, pick whichever expresses the product horizon most naturally.

Sets the shape of the time dimension in the preallocated Zarr store: time-indexed arrays are created with shape (slot_count, ...).

Mutually exclusive with end_date. Leave both None for serial-mode plugins without a fixed horizon.

firecube.ingestor.api.ItemInfo `dataclass`

Metadata returned by inspect_item describing a single source item.

The engine uses coordinate to map the item to a position on its assigned axis.

Attributes:

Name	Type	Description
`coordinate`	`Any`	The item's coordinate value. For `RegularTimeAxis` this is a UTC-explicit timestamp. For `IntegerAxis` this is an integer coordinate.

Attributes

coordinate `class-attribute` `instance-attribute`

coordinate = None

Coordinate value used by the resolver to compute a position.

firecube.ingestor.api.ResolvedIndex

A resolved, immutable index for a multi-group DirectZarr product.

Built by resolve_index_spec and cached per run context and spec on the DirectZarrIngestor instance, so repeated lookups within a run return the same object.

Plugin code reaches it through resolved_index(ctx). Use size(group) when a schema needs the declared axis extent before writes begin, position(group, coordinate) when a write needs the slot index for a timestamp or integer coordinate, and coordinate(group, index) for the reverse lookup.

The identity_hash is content-addressed from the canonical resolved-index payload. It intentionally does not track the legacy slot-index model hash.

Attributes

groups `property`

groups

Sorted tuple of group names.

identity_hash `cached` `property`

identity_hash

Content-addressed hash of the canonical resolved-index payload.

Methods:

filtered_spec

filtered_spec(groups=None)

Return an IndexSpec rebuilt from the resolved state.

bound_axes()-only reconstruction was rejected because callers need the product name and time_unit too, not just axis objects. Filtering as_resolved_index_record() was rejected because mixed specs still include unbounded axes, and that path re-raises ExtentUnknownError when it meets them.

canonical_index_payload

canonical_index_payload()

Return the canonical resolved-index payload.

as_resolved_index_record

as_resolved_index_record(*, run_id, recorded_at=None)

Build the on-disk resolved-index record for this resolved spec.

size

size(group)

Total number of slots for the given group.

Parameters:

Name	Type	Description	Default
`group`	`str`	Group name.	required

Returns:

Type	Description
`int`	Total slot count.

Raises:

Type	Description
`KeyError`	If the group is not in this index.
`ExtentUnknownError`	If the axis has no fixed extent.

position

position(group, coordinate)

Map a coordinate to its slot index within the given group.

Parameters:

Name	Type	Description	Default
`group`	`str`	Group name.	required
`coordinate`	`Any`	The coordinate value.	required

Returns:

Type	Description
`int`	Zero-based slot index.

coordinate

coordinate(group, index)

Map a slot index to its coordinate value within the given group.

Parameters:

Name	Type	Description	Default
`group`	`str`	Group name.	required
`index`	`int`	Zero-based slot index.	required

Returns:

Type	Description
`Any`	Coordinate value (type depends on axis kind).

axis_for

axis_for(group)

Return the axis spec for the named group, or None if not present.

as_legacy_slot_index_model

as_legacy_slot_index_model()

Build a SlotIndexModel for byte-parity with existing cubes.

Returns non-None ONLY when every axis is a RegularTimeAxis. Mixed-kind specs return None; persistence for those specs is handled by a future release.

Returns:

Type	Description
`SlotIndexModel \| None`	A `SlotIndexModel` byte-identical to what the legacy mixin
`SlotIndexModel \| None`	produced, or `None` for non-regular specs.

firecube.ingestor.api.ResolvedIndexRecord `dataclass`

On-disk record for an engine-resolved index payload.

The engine writes it to .firecube/index/current.json after the first successful resolution. Subsequent runs read it back and verify that the declared IndexSpec produces the same identity_hash before writing. Inspect the record with firecube zarr index show and regenerate it from a plugin declaration with firecube zarr index rebuild.

Optional items carries a content-addressed manifest for IrregularTimeAxis cubes. It is omitted from the wire format and identity_hash for regular / integer axes so those cubes stay byte-identical to pre-manifest records (see compute_resolved_index_identity_hash).

Methods:

to_json_bytes

to_json_bytes()

Serialise to the on-disk wire format (deterministic, UTF-8 JSON).

ASYMMETRIC: the items key is omitted when items is None so records for regular / integer axes serialise byte-identically to pre-manifest records.

from_json_bytes `classmethod`

from_json_bytes(data)

Parse and validate an on-disk resolved-index record.

firecube.ingestor.api.resolve_index_spec

resolve_index_spec(spec, *, time_dim_name, items=None)

Resolve an IndexSpec into a ResolvedIndex.

Validates that each RegularTimeAxis.coordinate matches time_dim_name, then builds one resolver per group.

Parameters:

Name	Type	Description	Default
`spec`	`IndexSpec`	The index specification to resolve.	required
`time_dim_name`	`str`	The expected time-coordinate dimension name (e.g. `"time"` or `"timestamp"`).	required

Returns:

Type	Description
`ResolvedIndex`	A `ResolvedIndex` ready for slot-index computation.

Raises:

Type	Description
`ConfigurationError`	If any axis coordinate does not match `time_dim_name`.
`NotImplementedError`	If any axis kind is not supported.

Examples:

>>> from firecube.core.api import IndexSpec, RegularTimeAxis
>>> axis = RegularTimeAxis(
...     coordinate="timestamp",
...     epoch="2024-01-01T00:00:00Z",
...     cadence_s=600,
...     slot_count=2,
... )
>>> spec = IndexSpec(name="demo", groups={"data": axis})
>>> resolved = resolve_index_spec(spec, time_dim_name="timestamp")
>>> resolved.groups
('data',)

Core API

coerce_to_epoch_s() is exported from firecube.core.api only; the ingestor facade does not re-export it.

firecube.core.api.coerce_to_epoch_s

coerce_to_epoch_s(value, *, mode='floor')

Coerce a coordinate value to seconds since the Unix epoch (UTC).

Accepts the following types:

str: UTC-explicit ISO 8601 string (via iso_to_epoch_s).
datetime.datetime: naive treated as UTC (FCI pattern); aware converted to UTC.
numpy.datetime64: any unit, converted to seconds.
pandas.Timestamp: naive treated as UTC; aware converted to UTC.

Parameters:

Name	Type	Description	Default
`value`	`Any`	The coordinate value to coerce.	required
`mode`	`str`	`"floor"` (default) or `"exact"`. In `"exact"` mode, fractional seconds raise `ValueError`.	`'floor'`

Returns:

Type	Description
`int`	Integer seconds since the Unix epoch (UTC).

Raises:

Type	Description
`TypeError`	If `value` is not one of the accepted types.
`ValueError`	In `"exact"` mode, if the value has sub-second precision.

Index Specification

Index Types

firecube.ingestor.api.TimeAxis

Methods:

grid staticmethod

observed staticmethod

explicit staticmethod

discovered staticmethod

firecube.ingestor.api.AxisSpec

firecube.ingestor.api.IndexSpec dataclass

Attributes

name instance-attribute

groups instance-attribute

time_unit class-attribute instance-attribute

firecube.ingestor.api.IntegerAxis dataclass

Attributes

slot_count instance-attribute

firecube.ingestor.api.IrregularTimeAxis dataclass

Attributes

coordinate instance-attribute

values instance-attribute

firecube.ingestor.api.AUTO module-attribute

firecube.ingestor.api.RegularTimeAxis dataclass

Attributes

coordinate instance-attribute

epoch instance-attribute

cadence_s instance-attribute

mode class-attribute instance-attribute

end_date class-attribute instance-attribute

slot_count class-attribute instance-attribute

firecube.ingestor.api.ItemInfo dataclass

Attributes

coordinate class-attribute instance-attribute

firecube.ingestor.api.ResolvedIndex

Attributes

groups property

identity_hash cached property

Methods:

filtered_spec

canonical_index_payload

as_resolved_index_record

size

position

coordinate

axis_for

as_legacy_slot_index_model

firecube.ingestor.api.ResolvedIndexRecord dataclass

Methods:

to_json_bytes

from_json_bytes classmethod

firecube.ingestor.api.resolve_index_spec

Core API

firecube.core.api.coerce_to_epoch_s

See Also

grid `staticmethod`

observed `staticmethod`

explicit `staticmethod`

discovered `staticmethod`

firecube.ingestor.api.IndexSpec `dataclass`

name `instance-attribute`

groups `instance-attribute`

time_unit `class-attribute` `instance-attribute`

firecube.ingestor.api.IntegerAxis `dataclass`

slot_count `instance-attribute`

firecube.ingestor.api.IrregularTimeAxis `dataclass`

coordinate `instance-attribute`

values `instance-attribute`

firecube.ingestor.api.AUTO `module-attribute`

firecube.ingestor.api.RegularTimeAxis `dataclass`

coordinate `instance-attribute`

epoch `instance-attribute`

cadence_s `instance-attribute`

mode `class-attribute` `instance-attribute`

end_date `class-attribute` `instance-attribute`

slot_count `class-attribute` `instance-attribute`

firecube.ingestor.api.ItemInfo `dataclass`

coordinate `class-attribute` `instance-attribute`

groups `property`

identity_hash `cached` `property`

firecube.ingestor.api.ResolvedIndexRecord `dataclass`

from_json_bytes `classmethod`