Plugin Templates

This reference covers the template classes plugin authors subclass and the members each template expects. Import all symbols from firecube.ingestor.api.

Plugins that own the complete batch pipeline subclass BaseIngestor directly; see Hooks & Lifecycle for that surface.

Name	Description
`register_ingestor`	Register an ingestor class under a public plugin name.
`GenericZarrIngestor`	Thin facade over `AppendStrategy` for Zarr-based batch ingestion.
`GenericParquetIngestor`	Generic Pipelined Ingestor for Parquet outputs.
`DirectZarrIngestor`	Abstract template for direct-Zarr region-based ingestors.
`GenericTensogramIngestor`	Template for plugins that write directly to Tensogram `.tgm` archives.

Registration

firecube.ingestor.api.register_ingestor

register_ingestor(name)

Register an ingestor class under a public plugin name.

Applied as a class decorator. The decorated class is added to the registry consulted by discover_ingestors() and get_ingestor(), and its name attribute is set to name.

Parameters:

Name	Type	Description	Default
`name`	`str`	Public name the plugin is looked up by, e.g. the value passed to `firecube ingest <name>`.	required

Returns:

Type	Description
`Callable[[type[_IngestorT]], type[_IngestorT]]`	A class decorator that registers the class and returns it unchanged.

Raises:

Type	Description
`TypeError`	If the decorated class does not satisfy the `Ingestor` protocol. Raised when the decorator is applied, not at call time.

Examples:

Register a plugin under the name used by firecube ingest:

@register_ingestor("my_product")
class MyProductIngestor(GenericZarrIngestor):
    PRODUCT_NAME = "my_product"

    def build_dataset(self, group, items, ctx):
        ...

Every concrete plugin class must also declare a non-empty PRODUCT_NAME: ClassVar[str]. Firecube checks this when the class is defined. Typed options are declared through PluginConfig and the template config classes listed in the Configuration Reference.

GenericZarrIngestor

Subclass GenericZarrIngestor and implement build_dataset. The template appends the returned dataset to the target Zarr store along the time dimension, once per write group per batch.

firecube.ingestor.api.GenericZarrIngestor

Thin facade over AppendStrategy for Zarr-based batch ingestion.

Resolves URIs/storage, acquires write claims, then delegates all append logic to AppendStrategy.write_groups().

Subclasses implement build_dataset(group, items, ctx) -> xr.Dataset | None.

Methods:

build_dataset `abstractmethod`

build_dataset(group, items, ctx)

Convert a sub-batch of items into an Xarray Dataset for the given group.

items is the time-grouped slice for this batch iteration. Returns None to skip writing for this group/batch.

The returned dataset must carry the ingestor's time_dim_name dimension, ordered on that dimension, with values that do not overlap another batch; it is appended along that dimension. Variables, dimensions, coordinates, and data types must remain compatible across batches.

Examples:

Build one dataset per batch from the discovered items:

def build_dataset(self, group, items, ctx):
    paths = [ctx.materialize(item) for item in items]
    ds = xr.open_mfdataset(paths, combine="by_coords")
    return ds[["temperature"]].sortby(self.time_dim_name)

Route variables per group when get_batch_groups declares more than one; every group receives the same items:

def build_dataset(self, group, items, ctx):
    ds = self._open(items, ctx)
    if group == "quality":
        return ds[["quality_level"]]
    return ds[["temperature"]]

get_batch_groups

get_batch_groups(items, ctx)

Return the logical write groups covered by a batch of items (Hook).

A group names a logical write destination: for Zarr templates it becomes the group path inside the store, for Parquet it becomes the output partition (subdirectory). Called once per planned batch; build_dataset is then invoked once per group with the full batch item list, and the plugin routes items to the group it is building.

Implementations MUST be deterministic and return a stable, sorted list so grouping stays consistent across runs.

Parameters:

Name	Type	Description	Default
`items`	`Sequence[Any]`	Items assigned to this batch.	required
`ctx`	`PluginContext`	Read-only plugin context.	required

Returns:

Type	Description
`list[str]`	Group names for the batch. Default: `["default"]`.

Examples:

Write two Zarr groups from the same source items. Each group name becomes the group path in the store, and build_dataset selects what belongs in the group it is called for:

def get_batch_groups(self, items, ctx):
    return ["quality", "sst"]

def build_dataset(self, group, items, ctx):
    ds = self._open(items, ctx)
    if group == "sst":
        return ds[["sea_surface_temperature"]]
    return ds[["quality_level"]]

get_zarr_config

get_zarr_config(ctx)

Return Zarr storage options from validated template config.

Options are declared through ZarrTemplateConfig.

Write Groups

get_batch_groups returns the logical write groups for a batch and defaults to ["default"]. Each group name is used directly as the Zarr group path in the store, so nested paths such as "sst/quality" are valid. build_dataset is called once per group per batch and receives the complete batch item list for every group; the plugin selects per group which variables or items to return. Returning None skips the group for that batch. The returned list must be deterministic across runs.

DirectZarrIngestor derives its groups from the declared schema instead; see DirectZarrIngestor.

Time Dimension

The inherited time_dim_name: ClassVar[str] selects the append dimension and defaults to "timestamp". It is a class declaration, not a --option field. When the target store already exists, Firecube verifies the declared dimension against the store before writing:

firecube.ingestor.api.verify_dim_compatibility

verify_dim_compatibility(
    target_uri, declared_dim, group_paths, storage_config
)

Verify each group in group_paths has a time dim matching declared_dim.

No-op for groups that do not yet exist on disk (they will be created on first write) and for a target_uri that does not exist at all (new cube).

Parameters:

Name	Type	Description	Default
`target_uri`	`str`	URI of the (possibly existing) Zarr store to inspect.	required
`declared_dim`	`str`	Time dimension name the plugin declares for its arrays.	required
`group_paths`	`Sequence[str]`	Group paths to check within the store.	required
`storage_config`	`Any`	Storage configuration or binding used to open the store.	required

Raises:

Type	Description
`ConfigurationError`	If any existing group uses a different time dimension than `declared_dim` (with migration guidance), if a data array carries both `time` and `timestamp` dimensions, or if arrays within one group disagree on the time dimension.

GenericParquetIngestor

Subclass GenericParquetIngestor and implement build_dataset. The remaining methods are optional customizations.

firecube.ingestor.api.GenericParquetIngestor

Generic Pipelined Ingestor for Parquet outputs.

Methods:

build_dataset `abstractmethod`

build_dataset(group, batch, ctx)

Convert a batch of items into an in-memory tabular dataset for the given group.

Supported return types for the default Parquet writer

pyarrow.Table
pandas.DataFrame (if pandas is installed)

Returns None to skip writing for this group/batch.

Unlike the Zarr template, this hook receives the PipelineBatch itself rather than a list of items.

Examples:

Return one table per batch:

def build_dataset(self, group, batch, ctx):
    rows = []
    for item in batch.items:
        rows.extend(read_detections(ctx.materialize(item)))
    if not rows:
        return None
    return pyarrow.Table.from_pylist(rows)

get_batch_groups

get_batch_groups(items, ctx)

Return the logical write groups for a batch (Hook).

Each group produces one build_dataset call (receiving the full batch) and one Parquet file; non-default group names become subdirectories of the dataset root via output_relpath. Must be deterministic and stable-sorted. Default: ["default"].

output_relpath

output_relpath(group, batch, ctx)

Return a relative output path (within the dataset root) for a group/batch.

write_parquet

write_parquet(
    dataset, *, output_path, storage_options, ctx
)

Write a supported dataset to Parquet and return number of rows written.

ParquetTemplateConfig currently declares parquet_partition_by and parquet_row_group_size, but the default writer does not apply either field; see the Configuration Reference.

GenericTensogramIngestor

Subclass GenericTensogramIngestor and implement build_dataset. The template writes each batch through the Tensogram strategy.

firecube.ingestor.api.GenericTensogramIngestor

Template for plugins that write directly to Tensogram .tgm archives.

Subclasses implement build_dataset(group, items, ctx) returning an xarray.Dataset (or None to skip); the template writes each group's dataset to the local .tgm target using the options declared in TensogramTemplateConfig. Remote targets are not supported.

Methods:

build_dataset `abstractmethod`

build_dataset(group, items, ctx)

Convert a sub-batch of items into an xarray Dataset.

Returns None to skip writing for this group/batch.

Examples:

Build the dataset the archive writer receives:

def build_dataset(self, group, items, ctx):
    paths = [ctx.materialize(item) for item in items]
    if not paths:
        return None
    return xr.open_mfdataset(paths, combine="by_coords")

get_batch_groups

get_batch_groups(items, ctx)

Return the logical write groups for a batch (Hook).

Each group produces one build_dataset call receiving the full batch item list. Must be deterministic and stable-sorted. Default: ["default"].

Options are declared through TensogramTemplateConfig.

DirectZarrIngestor

Subclass DirectZarrIngestor and implement index_spec, inspect_item, zarr_schema, and build_write_intents.

firecube.ingestor.api.DirectZarrIngestor

Abstract template for direct-Zarr region-based ingestors.

Plugins that write directly to Zarr (bypassing xarray) should subclass this template and implement:

zarr_schema — declare groups and arrays.
build_write_intents — convert a batch into write operations.

The template orchestrates store setup, write execution via a region write strategy, coverage tracking, and metrics aggregation.

Methods:

index_spec

index_spec(ctx)

Override to enable slot-range parallel ingestion.

Return an IndexSpec describing the product's index shape, or None for serial-only plugins (no --slot-start/--slot-end).

index_spec MUST be resolvable from typed config alone. The implementation may read self.plugin_config and self.template_config; it MUST NOT depend on ctx.source contents (source listing, file peek, or --input-data). Reason: firecube zarr slots and firecube zarr preallocate call index_spec without any --input-data; if your product's epoch or size derives from source, expose an explicit config override (e.g. MyConfig.time_epoch) and raise ConfigurationError naming the missing config field when the override is absent.

Default returns None (serial-only plugin).

inspect_item

inspect_item(item, ctx)

Override to enable slot-range parallel ingestion.

Called by the engine for each source item to determine its slot index. Return an ItemInfo with the item's time coordinate, or None to drop the item from this worker's slot range.

Default raises NotImplementedError — override this method to enable parallel ingestion.

Parameters:

Name	Type	Description	Default
`item`	`Any`	A source item from the batch.	required
`ctx`	`PluginContext`	The plugin context for this run.	required

Returns:

Type	Description
`ItemInfo \| None`	An `ItemInfo` with the item's coordinate, or `None` to drop.

Raises:

Type	Description
`NotImplementedError`	If not overridden.

resolved_index

resolved_index(ctx)

Return the resolved index for this run, cached per context.

Raises ConfigurationError if index_spec(ctx) returns None (serial-only plugin).

Parameters:

Name	Type	Description	Default
`ctx`	`PluginContext`	The plugin context for this run.	required

Returns:

Type	Description
`ResolvedIndex`	The `ResolvedIndex` for slot-index computation.

Raises:

Type	Description
`ConfigurationError`	If `index_spec(ctx)` returns `None`.

zarr_schema `abstractmethod`

zarr_schema(ctx)

Declare the Zarr store layout for this ingestor.

Returns a list of group specifications describing every group and array that the ingestor may write to. Called once per batch to ensure groups and arrays exist before writes begin.

The declared groups are also the ingestor's write groups, so the schema must cover every group any WriteIntent targets.

Examples:

Declare one group with a time-indexed array and its time axis:

def zarr_schema(self, ctx):
    n_times = self.resolved_index(ctx).size("FWI")
    # Or use a literal for serial mode.
    return [
        ZarrGroupSpec(
            group="FWI",
            arrays=[
                ZarrArraySpec(
                    name="fire_risk",
                    shape=(n_times, 550, 475),
                    dtype="float32",
                    chunks=(1, 550, 475),
                    dimension_names=("timestamp", "y", "x"),
                ),
                ZarrArraySpec(
                    name="timestamp",
                    shape=(n_times,),
                    dtype="int64",
                    dimension_names=("timestamp",),
                ),
            ],
        )
    ]

build_write_intents

build_write_intents(batch, ctx)

Convert a batch into a list of write operations.

Return one flat list that may freely mix two element types:

IndexedWrite — a coordinate-keyed write. Build it with coordinate=<timestamp or integer key> and the engine resolves the slot index for you; an unmappable coordinate raises IndexedWriteCompilationError before any write.
WriteIntent — a fully resolved write. Use it when you have computed the index yourself, and for writes that carry no slot coordinate (WriteIntent.coordinate and WriteIntent.static).

Skip an item by not appending anything for it; emit several writes for one item by appending several elements. Return an empty list to skip the whole batch. Every element's group must exist in zarr_schema(ctx), and resolved indexes must fall inside the worker's slot range when slot-range parallelism is enabled. Compilation of IndexedWrite elements runs at the call site, outside this hook, so no override can bypass it.

For every compiled IndexedWrite, the engine also emits the slot's time-coordinate verify-write automatically (one per resolved slot, skipped when the list already carries an explicit WriteIntent.coordinate for that slot), so plugins on this path never resolve or emit coordinate writes themselves.

Examples:

One coordinate-keyed write per item plus one static array:

def build_write_intents(self, batch, ctx):
    out = []
    for item in batch.items:
        timestamp, values = read_product(ctx.materialize(item))
        out.append(IndexedWrite.slot(
            group="data", array="value",
            coordinate=timestamp, data=values,
        ))
    out.append(WriteIntent.static(
        group="grid", array="lat", data=self._lat_grid,
    ))
    return out

Parameters:

Name	Type	Description	Default
`batch`	`PipelineBatch`	The pipeline batch to convert.	required
`ctx`	`PluginContext`	The plugin context for this run.	required

Returns:

Type	Description
`Sequence[WriteIntent \| IndexedWrite]`	A list mixing `WriteIntent` and `IndexedWrite` elements.

Raises:

Type	Description
`NotImplementedError`	If the plugin does not override this hook.

get_batch_groups

get_batch_groups(items, ctx)

Return the sorted set of groups declared by zarr_schema(ctx).

On this template the group set is schema-declared, not item-derived: writes are routed by WriteIntent.group. Plugins should NOT override this method — a hand-rolled group list can disagree with the declared schema and break group/schema agreement.

On this template get_batch_groups is derived from the groups declared in zarr_schema and is not an override point; overriding it breaks the agreement between groups and schema.

Use resolved_index(ctx).size(group) when the declared index extent controls array shape, and resolved_index(ctx).position(group, coordinate) when a write needs the slot index for a timestamp value.

For parallel writes across disjoint slot ranges, see Parallelism.

Schema And Write Types

firecube.ingestor.api.ZarrGroupSpec `dataclass`

Specification for a Zarr group and its arrays.

Returned by DirectZarrIngestor.zarr_schema() to describe the full layout.

attrs is optional, convention-agnostic group-level metadata stamped onto the group's zarr.json at schema setup — e.g. dataset-level attributes a plugin chooses to publish. Firecube writes the mapping verbatim and does not interpret it (no convention is assumed); reserved firecube-internal attribute names are rejected at write time.

firecube.ingestor.api.ZarrArraySpec `dataclass`

Specification for a single Zarr array within a group.

Attributes

name `instance-attribute`

name

Name of the Zarr array within its group.

shape `instance-attribute`

shape

Full shape of the array, with time as the first axis when indexed.

dtype `instance-attribute`

dtype

NumPy dtype or dtype-like value used when creating the array.

chunks `class-attribute` `instance-attribute`

chunks = None

Chunk shape for the array, or None to use the template default.

fill_value `class-attribute` `instance-attribute`

fill_value = None

Fill value written into unused cells when the array is preallocated.

expected_time_count `class-attribute` `instance-attribute`

expected_time_count = None

Expected number of time slots for time-indexed arrays, if known.

shards `class-attribute` `instance-attribute`

shards = None

Optional Zarr sharding shape for the array, or None to disable sharding.

attrs `class-attribute` `instance-attribute`

attrs = None

Array-level attributes stamped into zarr.json verbatim.

dimension_names `class-attribute` `instance-attribute`

dimension_names = None

Dimension names for the array, ordered to match shape.

time_indexed `class-attribute` `instance-attribute`

time_indexed = True

Whether the array participates in time-axis preallocation and slot writes.

filters `class-attribute` `instance-attribute`

filters = None

Per-array codec filters.

Each entry is a Zarr v3 ArrayArrayCodec config dict. None inherits the template default; an explicit tuple overrides the template filter pipeline for this array only.

serializer `class-attribute` `instance-attribute`

serializer = None

Per-array serializer codec.

This is a Zarr v3 ArrayBytesCodec config dict. None inherits the template default; an explicit value overrides the template serializer for this array only.

compressors `class-attribute` `instance-attribute`

compressors = None

Per-array compressor codecs.

Each entry is a Zarr v3 BytesBytesCodec config dict. None inherits the template default. An empty tuple means explicitly uncompressed for this array.

firecube.ingestor.api.WriteIntent `dataclass`

A single write operation to execute against the Zarr store.

Produced by DirectZarrIngestor.build_write_intents() to describe what data should be written, where, and at which timestamp index.

The kind field selects the write method on RegionZarrWriter. Here, "timestamp" means the conceptual time/index axis, not the on-disk dimension name; the latter is configured separately via IndexedRegionStrategy.time_coord_name.

"region" → write_region(group, array, ts_index, y_slice, data, channel_index=...)
"1d" → write_1d(group, array, ts_index, data)
"timestamp" → write_timestamp(group, ts_index, timestamp_val)
"static" → write_static(group, array_name, data) (non-time-indexed; ts_index is ignored)

data may be an eager numpy.ndarray or a zero-arg callable that returns one. It is resolved exactly once at dispatch time, in dispatch order. The callable must close over stable inputs (paths, configuration), not open file handles or per-batch scratch objects. Callable data is supported for kind="region" and kind="static" only. Passing a callable for other kinds raises TypeError at construction. Any callable exception propagates with the same error surface as an eager payload rejection.

Attributes

timestamp_val `class-attribute` `instance-attribute`

timestamp_val = None

Conceptual time/index axis value for kind="timestamp" writes.

"timestamp" is a stable plugin-contract token, not the on-disk dim name; the actual dim/coord name comes from IndexedRegionStrategy.time_coord_name.

Methods:

slot `classmethod`

slot(*, group, array, index, data)

Write a 1-D array slice at a single time slot.

Use this for 1-D arrays that grow along the time axis — per-slot scalars, per-slot vectors, or any array where each slot contributes one row. The array must be declared with time_indexed=True in ZarrArraySpec.

data must be an eager np.ndarray; callable payloads are not supported for kind="1d" and raise TypeError at construction.

Parameters:

Name	Type	Description	Default
`group`	`str`	Zarr group name matching a `ZarrGroupSpec` in the schema.	required
`array`	`str`	Array name within the group.	required
`index`	`int`	Time-slot index for this write.	required
`data`	`Any`	Array data to write at this slot.	required

Returns:

Type	Description
`WriteIntent`	A `WriteIntent` with `kind="1d"` and `ts_index=index`.

Examples:

>>> import numpy as np
>>> intent = WriteIntent.slot(group="data", array="counts", index=7, data=np.zeros((4,)))
>>> intent.kind
'1d'
>>> intent.ts_index
7

region `classmethod`

region(
    *,
    group,
    array,
    index,
    data,
    y_slice,
    channel_index=None,
)

Write a 2-D spatial region at a single time slot.

Use this for the main image arrays — counts, radiances, quality flags, pixel times — where each slot contributes a spatial tile. The array must be declared with time_indexed=True in ZarrArraySpec.

data may be an eager np.ndarray or a zero-arg callable Callable[[], np.ndarray]; the callable is resolved at dispatch time.

Parameters:

Name	Type	Description	Default
`group`	`str`	Zarr group name matching a `ZarrGroupSpec` in the schema.	required
`array`	`str`	Array name within the group.	required
`index`	`int`	Time-slot index for this write.	required
`data`	`Any`	2-D array data, or a callable that returns it.	required
`y_slice`	`slice`	Row slice within the array.	required
`channel_index`	`int \| None`	Channel dimension index, or `None` for non-channel arrays.	`None`

Returns:

Type	Description
`WriteIntent`	A `WriteIntent` with `kind="region"`.

Examples:

>>> import numpy as np
>>> intent = WriteIntent.region(
...     group="data_1km", array="counts", index=3,
...     data=np.zeros((100, 2048)), y_slice=slice(0, 100),
... )
>>> intent.kind
'region'
>>> intent.y_slice
slice(0, 100, None)

coordinate `classmethod`

coordinate(*, group, index, value)

Write the time-axis coordinate value for a single slot.

Use this to record the actual timestamp (or integer index) that corresponds to ts_index. The engine writes it into the time coordinate array so the output cube is self-describing.

Parameters:

Name	Type	Description	Default
`group`	`str`	Zarr group name matching a `ZarrGroupSpec` in the schema.	required
`index`	`int`	Time-slot index for this coordinate.	required
`value`	`Any`	The coordinate value for this slot — typically a `datetime`, `numpy.datetime64`, or integer. Must not be `None`.	required

Returns:

Type	Description
`WriteIntent`	A `WriteIntent` with `kind="timestamp"`.

Raises:

Type	Description
`ValueError`	If `value` is `None`.

Examples:

>>> from datetime import datetime, timezone
>>> intent = WriteIntent.coordinate(
...     group="data", index=5,
...     value=datetime(2024, 1, 1, tzinfo=timezone.utc),
... )
>>> intent.kind
'timestamp'
>>> intent.ts_index
5

static `classmethod`

static(*, group, array, data)

Write a static (non-time-indexed) array — coordinate grids, lookup tables, masks.

Use this for arrays that are the same across every time slot: latitude/ longitude grids, channel names, calibration tables, spatial references. The array must be declared with time_indexed=False in ZarrArraySpec; the engine pre-creates it at its declared shape during schema setup.

Write-once contract: the engine writes the array on the first ingest run and stamps a marker attribute. On any subsequent run (resume or re-ingest) the incoming data must be byte-identical to what was already written, or the ingest fails with SchemaDriftError. There is no partial-update path for static arrays.

data may be an eager np.ndarray or a zero-arg callable Callable[[], np.ndarray]; the callable is resolved at dispatch time.

Parameters:

Name	Type	Description	Default
`group`	`str`	Zarr group name matching a `ZarrGroupSpec` in the schema.	required
`array`	`str`	Array name declared with `time_indexed=False` in that group.	required
`data`	`Any`	Data to write, or a callable that returns it.	required

Returns:

Type	Description
`WriteIntent`	A `WriteIntent` with `kind="static"`.

Raises:

Type	Description
`SchemaDriftError`	On resume, if `data` does not match the already-committed array byte-for-byte (NaN-aware).
`TypeError`	If `data` is callable and `kind` is not `"static"` (cannot happen via this factory; raised by `__post_init__` only when constructing `WriteIntent` directly with a mismatched kind).

firecube.ingestor.api.IndexedWrite `dataclass`

A coordinate-keyed write intent whose slot index is resolved at compile time.

Returned from DirectZarrIngestor.build_write_intents() to describe a write whose target slot is expressed as a raw coordinate value (typically a datetime, numpy.datetime64, or integer) rather than a pre-resolved integer index. The engine maps coordinate to a slot index at compile time against the plugin's declared IndexSpec, then materializes an equivalent WriteIntent.

Use the region classmethod for 2-D spatial region writes and slot for 1-D per-slot writes. These are the only two builders by design — there is intentionally no .static() or .coordinate() factory:

Static (non-time-indexed) arrays remain WriteIntent.static because they carry no slot coordinate.
Time-coordinate verify-writes are emitted automatically by the engine for every compiled slot; plugins never resolve or emit them on this path.

For region, data may be an eager numpy.ndarray or a zero-arg callable Callable[[], np.ndarray] resolved exactly once at dispatch time, in dispatch order, on the same terms as WriteIntent — so large payloads are not held in memory while the batch is assembled. slot requires an eager array; a callable is rejected at compile time.

Attributes

group `instance-attribute`

group

Zarr group name matching a ZarrGroupSpec in the schema.

array `instance-attribute`

array

Array name within the group.

coordinate `instance-attribute`

coordinate

Raw slot key resolved to an integer index at compile time.

Typically a datetime, numpy.datetime64, or int. The engine dispatches the value through the plugin's declared IndexSpec; unresolvable coordinates raise at compile time. Must not be None.

data `instance-attribute`

data

Array payload, or a zero-arg callable that returns one.

Callable payloads are resolved exactly once at dispatch time under the same rules as WriteIntent — the callable must close over stable inputs (paths, configuration), not open file handles or per-batch scratch.

y_slice `class-attribute` `instance-attribute`

y_slice = None

Row slice within the array; required for region, None for slot.

channel_index `class-attribute` `instance-attribute`

channel_index = None

Channel dimension index for region writes, None otherwise.

Methods:

region `classmethod`

region(
    *,
    group,
    array,
    coordinate,
    data,
    y_slice,
    channel_index=None,
)

Build an indexed 2-D spatial region write.

Use for the main image arrays — counts, radiances, quality flags, pixel times — where each slot contributes a spatial tile. The array must be declared with time_indexed=True in ZarrArraySpec.

Parameters:

Name	Type	Description	Default
`group`	`str`	Zarr group name matching a `ZarrGroupSpec` in the schema.	required
`array`	`str`	Array name within the group.	required
`coordinate`	`Any`	Raw slot key (e.g. `datetime`, `numpy.datetime64`, or `int`). Resolved to a slot index by the engine at compile time. Must not be `None`.	required
`data`	`ndarray \| Callable[[], ndarray] \| Any`	2-D array data, or a zero-arg callable that returns it.	required
`y_slice`	`slice`	Row slice within the array.	required
`channel_index`	`int \| None`	Channel dimension index, or `None` for non-channel arrays.	`None`

Returns:

Type	Description
`IndexedWrite`	An `IndexedWrite` describing a 2-D spatial region write.

Examples:

>>> import numpy as np
>>> from datetime import datetime, timezone
>>> iw = IndexedWrite.region(
...     group="data", array="counts",
...     coordinate=datetime(2024, 1, 1, tzinfo=timezone.utc),
...     data=np.zeros((100, 2048)), y_slice=slice(0, 100),
... )
>>> iw.y_slice
slice(0, 100, None)

slot `classmethod`

slot(*, group, array, coordinate, data)

Build an indexed 1-D per-slot write.

Use for 1-D arrays that grow along the time axis — per-slot scalars, per-slot vectors, or any array where each slot contributes one row. The array must be declared with time_indexed=True in ZarrArraySpec.

Parameters:

Name	Type	Description	Default
`group`	`str`	Zarr group name matching a `ZarrGroupSpec` in the schema.	required
`array`	`str`	Array name within the group.	required
`coordinate`	`Any`	Raw slot key (e.g. `datetime`, `numpy.datetime64`, or `int`). Resolved to a slot index by the engine at compile time. Must not be `None`.	required
`data`	`ndarray \| Callable[[], ndarray] \| Any`	Eager 1-D array payload. Unlike `region`, a callable is rejected when the compiled write is validated.	required

Returns:

Type	Description
`IndexedWrite`	An `IndexedWrite` describing a 1-D per-slot write.

Examples:

>>> import numpy as np
>>> from datetime import datetime, timezone
>>> iw = IndexedWrite.slot(
...     group="data", array="counts",
...     coordinate=datetime(2024, 1, 1, tzinfo=timezone.utc),
...     data=np.zeros((4,)),
... )
>>> iw.array
'counts'

Plugin Templates

Registration

firecube.ingestor.api.register_ingestor

GenericZarrIngestor

firecube.ingestor.api.GenericZarrIngestor

Methods:

build_dataset abstractmethod

get_batch_groups

get_zarr_config

Write Groups

Time Dimension

firecube.ingestor.api.verify_dim_compatibility

GenericParquetIngestor

firecube.ingestor.api.GenericParquetIngestor

Methods:

build_dataset abstractmethod

get_batch_groups

output_relpath

write_parquet

GenericTensogramIngestor

firecube.ingestor.api.GenericTensogramIngestor

Methods:

build_dataset abstractmethod

get_batch_groups

DirectZarrIngestor

firecube.ingestor.api.DirectZarrIngestor

Methods:

index_spec

inspect_item

resolved_index

zarr_schema abstractmethod

build_write_intents

get_batch_groups

Schema And Write Types

firecube.ingestor.api.ZarrGroupSpec dataclass

firecube.ingestor.api.ZarrArraySpec dataclass

Attributes

name instance-attribute

shape instance-attribute

dtype instance-attribute

chunks class-attribute instance-attribute

fill_value class-attribute instance-attribute

expected_time_count class-attribute instance-attribute

shards class-attribute instance-attribute

attrs class-attribute instance-attribute

dimension_names class-attribute instance-attribute

time_indexed class-attribute instance-attribute

filters class-attribute instance-attribute

serializer class-attribute instance-attribute

compressors class-attribute instance-attribute

firecube.ingestor.api.WriteIntent dataclass

Attributes

timestamp_val class-attribute instance-attribute

Methods:

slot classmethod

region classmethod

coordinate classmethod

static classmethod

firecube.ingestor.api.IndexedWrite dataclass

Attributes

group instance-attribute

array instance-attribute

coordinate instance-attribute

data instance-attribute

y_slice class-attribute instance-attribute

channel_index class-attribute instance-attribute

Methods:

region classmethod

slot classmethod

See Also

build_dataset `abstractmethod`

build_dataset `abstractmethod`

build_dataset `abstractmethod`

zarr_schema `abstractmethod`

firecube.ingestor.api.ZarrGroupSpec `dataclass`

firecube.ingestor.api.ZarrArraySpec `dataclass`

name `instance-attribute`

shape `instance-attribute`

dtype `instance-attribute`

chunks `class-attribute` `instance-attribute`

fill_value `class-attribute` `instance-attribute`

expected_time_count `class-attribute` `instance-attribute`

shards `class-attribute` `instance-attribute`

attrs `class-attribute` `instance-attribute`

dimension_names `class-attribute` `instance-attribute`

time_indexed `class-attribute` `instance-attribute`

filters `class-attribute` `instance-attribute`

serializer `class-attribute` `instance-attribute`

compressors `class-attribute` `instance-attribute`

firecube.ingestor.api.WriteIntent `dataclass`

timestamp_val `class-attribute` `instance-attribute`

slot `classmethod`

region `classmethod`

coordinate `classmethod`

static `classmethod`

firecube.ingestor.api.IndexedWrite `dataclass`

group `instance-attribute`

array `instance-attribute`

coordinate `instance-attribute`

data `instance-attribute`

y_slice `class-attribute` `instance-attribute`

channel_index `class-attribute` `instance-attribute`

region `classmethod`

slot `classmethod`