Skip to content

Sentinel-3 SLSTR/OLCI Zarr stores inaccessible via HTTP for certain months #39

@emmanuelmathot

Description

@emmanuelmathot

Summary

Several monthly buckets of Sentinel-3 data (SLSTR LST, OLCI LFR) in the EOPF Zarr Sample Service are inaccessible via standard HTTP/Zarr clients (xarray.open_datatree, zarr.open_group, fsspec). Two distinct issues were identified:

Issue 1: Literal /../ in S3 object keys (Jun 2025)

The June 2025 buckets (202506-s03slslst, 202506-s03olcil2lfr) contain objects whose keys include a literal /../ path segment:

30/../products/cpm_v256/S3A_SL_2_LST____20250630T224247_....zarr/.zgroup

The STAC catalog returns these paths as-is in asset hrefs. HTTP clients (requests, fsspec, urllib3) resolve /../ during URL normalization, turning 30/../products/ into products/ — but no object exists at the resolved key. This makes all data in these buckets inaccessible via any standard HTTP-based Zarr client.

Affected buckets (confirmed):

  • 202506-s03slslst
  • 202506-s03olcil2lfr (sentinel-3-olci-l2-lfr)

Not affected (clean keys like 28/products/...):

  • 202504-s03slslst
  • 202505-s03slslst
  • 202510-s03slslst through 202603-s03slslst-eu

Issue 2: Missing Zarr metadata files (Sep 2025)

The September 2025 bucket (202509-s03slslst) has clean key paths (no /../) and chunk data files exist, but the root Zarr metadata files (.zgroup, .zattrs, .zmetadata) are missing. This means the Zarr store cannot be opened by any client.

# These all return 404:
21/products/cpm_v256/S3A_SL_2_LST____20250921T...zarr/.zgroup
21/products/cpm_v256/S3A_SL_2_LST____20250921T...zarr/.zattrs
21/products/cpm_v256/S3A_SL_2_LST____20250921T...zarr/.zmetadata
21/products/cpm_v256/S3A_SL_2_LST____20250921T...zarr/zarr.json

# But chunk data exists:
21/products/cpm_v256/S3A_SL_2_LST____20250921T...zarr/conditions/auxiliary/biome/0.0  ← exists

For comparison, working buckets (e.g., Apr 2025) have proper Zarr v2 metadata:

28/products/cpm_v256/S3B_SL_2_LST____20250428T...zarr/.zgroup  ← 200 OK, {"zarr_format": 2}

Reproducibility

import pystac_client, requests

catalog = pystac_client.Client.open("https://stac.core.eopf.eodc.eu")
search = catalog.search(
    collections=["sentinel-3-slstr-l2-lst"],
    bbox=[0, 40, 10, 50],
    datetime="2025-06-01/2025-06-30",
    max_items=1,
)
items = list(search.items())
href = items[0].assets["product"].href
print(href)  # Contains /../
print(requests.get(href + "/.zgroup").status_code)  # 404

Impact

This issue was discovered during the EOPF Zarr Community Notebook Competition judging. At least one submission (northflow_sentinel3_lst_climval) was unable to read Sentinel-3 SLSTR LST data and reported a GroupNotFoundError in their submission feedback.

Monthly status summary (sentinel-3-slstr-l2-lst)

Month Key format .zgroup Status
Apr 2025 28/products/... 200 ✅ OK
May 2025 clean 200 ✅ OK
Jun 2025 30/../products/... 404 ❌ Broken (literal /../)
Sep 2025 21/products/... 404 ❌ Broken (missing metadata)
Oct 2025 clean 200 ✅ OK
Nov 2025 clean 200 ✅ OK
Dec 2025 clean 200 ✅ OK
Jan 2026 clean 200 ✅ OK
Feb 2026 clean 200 ✅ OK
Mar 2026 clean 200 ✅ OK

Suggested fix

  1. Jun 2025: Re-ingest or rename the objects to remove the /../ from the key paths, and update the STAC catalog accordingly
  2. Sep 2025: Re-ingest the Zarr stores with proper root metadata files (.zgroup, .zattrs, .zmetadata)
  3. Consider adding a CI check to validate that newly ingested Zarr stores are accessible via xr.open_datatree(href, engine="zarr")

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions