Subsetting Sentinel-3 products based on footprints given in the STAC metadata#81
Subsetting Sentinel-3 products based on footprints given in the STAC metadata#81konstntokas merged 9 commits intomainfrom
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests.
🚀 New features to boost your workflow:
|
thomasstorm
left a comment
thomasstorm
left a comment
There was a problem hiding this comment.
Looks good to me, nice work! You may want to address my comments, but they're minor mostly. But please check the notebook comment.
| "to compute a valid grid mapping. Returning clipped dataset as-is.", | ||
| "Clipping with the specified bounding box " | ||
| "resulted in a dataset too small to compute a valid grid " | ||
| "mapping. Returning clipped dataset as-is.", |
There was a problem hiding this comment.
| "mapping. Returning clipped dataset as-is.", | |
| "mapping. Returning dataset as-is.", |
There was a problem hiding this comment.
But the dataset is clipped, so it returns a dataset with an axis of size less than 2; Should I still remove "clipped" @thomasstorm ?
There was a problem hiding this comment.
My point is: if the dataset is clipped, then it is not "as-is", or vice versa :)
There was a problem hiding this comment.
Then I better remove "as-is". Thanks for clarifying.
There was a problem hiding this comment.
Very nice showcase!
I have run into a problem, though: I could not "Open Sentinel-3 OLCI L1 ERR with Default Parameters" - I tried multiple times, but always received "ClientResponseError: 504, message='Gateway Time-out', url='https://data.eodc.eu/collections/EOPF_ZARR/products/cpm_v270/S03OLCERR/2026/03/13/S3B_OL_1_ERR____20260313T095241_20260313T103636_20260314T095142_2635_117_350______ESA_O_NT_004.zarr/.zmetadata'". This is probably not on our side, though.
Also: there are several cells that do xr.open_dataset(item.assets["product"].href, .... This always takes quite long. In the case of the step "Open Sentinel-3 SLSTR Level-1B RBT with Default Parameters" it took > 9 minutes. Isn't this step supposed to read only metadata?
There was a problem hiding this comment.
ClientResponseError: 504 is on EODC side, I also experienced it sometimes in the past, however in the last days during development it was stable.
The Sentinel-3 SLSTR Level-1B RBT product is complicated. There are multiple subgroups for oblique and nadir view and in different resolution. The footprint of the stac metadata cannot be used, as it does not represent all groups. So instead of doing the cutout based on the footprint, the entire lat/lon grid needs to be loaded for each subgroup to allow for a sub-setting. However 9min is very long. I just double checked on my side and it took 23sec to open that example and 4sec to plot.
There was a problem hiding this comment.
Alright. I just tested again from the office, and it was loaded in 12 secs, and it took 7sec to plot. So it's all on the server side, and "Very nice showcase!" is the only part of my comment that stays relevant :)
2 participants
metadata, improving performance by avoiding full latitude/longitude grid downloads
during subsetting.
Below, you can see example cutouts (and a notebook producing these plots). The light-blue polygon represents the full footprint, while the red points indicate the footprint samples derived from the STAC metadata. The solid green box shows the desired bounding box, and the dashed green box represents the bounding box of the dataset after clipping, which is then used for further processing. The orange outline corresponds to the SLSTR angle data used in the terrain correction step.
Note: The angles dataset has coarser sampling along the longitude axis and partially overlapping coverage, while the latitude dimension matches the resolution of the input dataset. Therefore, only the latitude dimension is subset to align the angles with the target region.
subset_sen3.ipynb