Skip to content

Delete And Reingest

Use deletion commands when you need to remove old data, retry a failed range, or replace a written span.

PRODUCT_URI="file:///data/products/MY_PRODUCT.zarr"
PRODUCT_NAME="MY_PRODUCT"

Preflight

The full URI locates the product. Deleting storage objects also requires the storage type and driver. For a local product:

export FIRECUBE_STORAGE_TYPE=local
export FIRECUBE_STORAGE_DRIVER=fsspec

--manifest-only does not require this storage configuration because it keeps the product data.

Confirm no writer is using the product, then inspect the tracked records:

firecube chunks list --product-name "$PRODUCT_URI" --include-span

Delete Tracked Records And Storage

Preview first:

firecube chunks delete \
  --product-name "$PRODUCT_URI" \
  --dry-run

Expected output:

DRY RUN - Would delete:
  Chunks: 2
  Products: MY_PRODUCT.zarr

Delete after the dry-run matches your intent:

firecube chunks delete \
  --product-name "$PRODUCT_URI" \
  --yes-i-really-mean-it

Expected output:

Deleted 2 chunks

Verify:

firecube chunks list --product-name "$PRODUCT_URI"

Expected output:

No chunks found matching criteria.

Delete Records Only

Use --manifest-only when the storage data should remain but Firecube should forget the tracked records:

firecube chunks delete \
  --product-name "$PRODUCT_URI" \
  --manifest-only \
  --yes-i-really-mean-it

Expected output:

Deleted 2 chunks

Delete By Date Range

Preview a date range:

firecube chunks delete \
  --product-name "$PRODUCT_URI" \
  --range 2024-03-01,2024-03-02 \
  --dry-run

Delete it:

firecube chunks delete \
  --product-name "$PRODUCT_URI" \
  --range 2024-03-01,2024-03-02 \
  --yes-i-really-mean-it

--start-date, --end-date, and --range filter by ChunkManager record timestamps. Use chunks list --include-span first when you need to confirm semantic coverage.

To filter by the data time range covered by each span (rather than the record timestamp), use --time-range instead:

firecube chunks delete \
  --product-name "$PRODUCT_URI" \
  --time-range 2024-03-01:2024-03-02 \
  --dry-run

--time-range deletes spans whose data-time window overlaps the given range. Use it when you know the data timestamps you want to remove but not the exact record timestamps stored in the ChunkManager.

Delete Storage Chunks From Spans

Use delete-span when you want batch/span-level deletion from Zarr storage.

Preview:

firecube chunks delete-span \
  --product-name "$PRODUCT_URI" \
  --run-id docs-span-run \
  --dry-run

Expected output:

DRY RUN: would delete 1 chunk keys from storage across 1 spans

Delete:

firecube chunks delete-span \
  --product-name "$PRODUCT_URI" \
  --run-id docs-span-run \
  --yes-i-really-mean-it

Expected output:

Deleted 1 chunk keys from storage across 1 spans

Use --force only when you intentionally want to delete spans that are not time-chunk aligned:

firecube chunks delete-span \
  --product-name "$PRODUCT_URI" \
  --run-id docs-span-run \
  --force \
  --yes-i-really-mean-it

Reingest A Range

Two approaches are available. Choose based on whether you need physical chunk removal or just data replacement.

Delete then reingest — removes the physical Zarr chunks before writing replacement data. Use this when you want a clean slate or need to reclaim storage space.

firecube chunks delete \
  --product-name "$PRODUCT_URI" \
  --range 2024-03-01,2024-03-02 \
  --dry-run

firecube chunks delete \
  --product-name "$PRODUCT_URI" \
  --range 2024-03-01,2024-03-02 \
  --yes-i-really-mean-it

firecube ingest <plugin> \
  --input-data /data/source/2024-03-01 \
  --target "$PRODUCT_URI" \
  --product-name "$PRODUCT_NAME" \
  --storage-type local \
  --storage-driver fsspec \
  --output-format zarr \
  --write-mode direct \
  --option force_reingest=true

Reingest in place — overwrites overlapping timestamps in-place (region overwrite) and appends any new tail timestamps. No separate delete step is required.

firecube ingest <plugin> \
  --input-data ./replacement-data \
  --target "$PRODUCT_URI" \
  --product-name "$PRODUCT_NAME" \
  --storage-type local \
  --storage-driver fsspec \
  --output-format zarr \
  --write-mode direct \
  --option force_reingest=true

Reingesting in place refuses insert attempts (timestamps that would land in the middle of existing data), incoming or existing duplicates, and NaT values. State=2 (deleted) and state=3 (failed_batch) slots are refilled. Delete first when you need physical chunk removal or when the replacement data does not align with the existing coordinate range.

Rebuild the snapshot after a large cleanup or reingest:

firecube chunks snapshots rebuild --product-name "$PRODUCT_URI"

Failure Recovery

Symptom Meaning Recovery
Storage configuration not available Storage deletion needs a storage type and driver even when the product URI is explicit. Set FIRECUBE_STORAGE_TYPE and FIRECUBE_STORAGE_DRIVER; also configure credentials for S3.
Span is not aligned It does not map cleanly to Zarr time chunks. Use --force only if expanded deletion is acceptable.
does not contain expected time dimension The cube uses a custom time dimension that older span records do not capture. Re-run with --time-dim <name> matching the cube layout.
explicit time dimension ... contradicts --time-dim disagrees with the span record or cube metadata. Drop --time-dim, or fix the span records if they are wrong.
Active claim blocks deletion Another writer owns a write domain. Use Recover Runs And Claims before deletion.

Next Steps