Refactor read_ldata() and update docs

[jonasschlegel]

Refactor read_ldata to drop ChannelId from its public API, add
per-detector slicing on event tiers (:jlevt, :jlskm, :jlpmt),
and introduce a cross-tier filter via the filtertier kwarg.
LegendHDF5IO extension docs are updated to match.

Changes

1. DetectorId throughout. read_ldata accepts DetectorIdLike only.
   Channel helpers (_get_channelid, _get_detectorid,
   _is_valid_id_or_tier) and the channel-consistency check in the
   partition method are removed.
2. Tier classification. _evt_tiers = [:jlevt, :jlskm, :jlpmt] and
   _perdet_tiers = [:jlpeaks, :jlhit, :jlpls] drive dispatch. raw and
   jldsp use _resolve_perdet_path, which tries tier/<det> (current),
   <det>/tier (legacy), and tier (struct-view) in order.
3. Per-detector event-tier slicing. With a detector,
   read_ldata(_, l200, (:jlevt, fk, det)) returns a flat-prefixed Table
   (geds_e_cusp_ctc_cal, geds_trig_e_cusp_ctc_cal,
   aux_pulser_aux_trig, ged_spm_is_valid_lar, …). The name-map is
   built dynamically by walking the HDF5 group; per-trigger and
   per-det-list VoVs are distinguished by their inner-length signature
   and unwrapped at the detector's per-event slot.
4. Cross-tier filter. A new filtertier kwarg lets a per-detector
   raw/jldsp read be filtered by an event-tier predicate (slicing via
   the per-det *_dataidx column, 1-based for raw and jldsp) or by a
   sibling per-trigger-tier predicate (1:1 row alignment per detector).
   Works for phy and cal.
5. Sub-group descent. subgroup=:dataQC for
   tier/<det>/group-style layouts (e.g. jlhit).
6. Missing-tolerant filter. Rows where filterby returns missing
   are dropped (same semantics as false).

What now works

# Per-detector event-tier read with a trigger-energy cut
read_ldata(@pf((; $geds_e_cusp_ctc_cal, $geds_t50)), l200, (:jlevt, fk, det);
           filterby = @pf $geds_is_valid_qc && $geds_trig_e_cusp_ctc_cal > 1500u"keV")

# Cross-subgroup QC filter (geds + ged_spm + !aux_muonveto + !aux_pulser)
read_ldata((:geds_e_cusp_ctc_cal,), l200, (:jlevt, fk, det);
           filterby = @pf $geds_is_valid_qc && $ged_spm_is_valid_lar &&
                          !$aux_muonveto_aux_trig && !$aux_pulser_aux_trig)

# Raw waveforms of phy events that pass an event-level QC + energy cut
read_ldata(@pf((; $waveform_presummed)), l200, (:raw, fk_phy, det);
           filterby   = @pf $geds_is_valid_qc && $geds_trig_e_cusp_ctc_cal > 1500u"keV",
           filtertier = :jlevt)

# Cal raw waveforms with a dsp-energy cut (no jlevt for cal)
read_ldata((:waveform_presummed,), l200, (:raw, fk_cal, det);
           filterby   = @pf $e_cusp > 1000,
           filtertier = :jldsp)

Backward-compat note

read_ldata(l200, :jlevt, fk) (no detector) still returns the native
nested LH5 view (evt.aux.pulser.aux_trig). Filters there use dot syntax
(@pf $geds.is_valid_qc == true); flat-prefix names exist only on the
per-detector side.

Performance

PropSel + filterby + filtertier fast-path vs. eager-load-then-filter
on real L200 data (FK p18-r000, B00000C, min of 3 trials):

Case	speed-up	memory
A — raw column selection (waveforms)	1.3×	2.1×
B — jldsp column + filter (e_cusp > 0)	1.0×	1.0×
C — jlevt event-scalar filter (cross-subgroup QC)	40.0×	42.3×
D — jlevt trigger-energy filter (>1500 keV)	33.7×	31.7×
E — cross-tier (filtertier=:jlevt, phy)	32.4×	26.0×
F — cross-tier (filtertier=:jldsp, cal)	1.6×	2.0×

The big wins (C–E) come from never materializing the ~1.3 GB full
jlevt+det Table when only a handful of columns are needed. Per-det
raw/dsp reads (A, B, F) are smaller files to begin with, so the
absolute numbers are modest but still favorable.

MCE

ENV["LEGEND_DATA_CONFIG"] = "/path/to/your/config.json"
using LegendDataManagement, LegendHDF5IO, PropertyFunctions
using Unitful: @u_str

l200    = LegendData(:l200)
PER, RUN = DataPeriod(18), DataRun(0)
fks_phy = search_disk(FileKey, l200.tier[:raw, :phy, PER, RUN])
fks_cal = search_disk(FileKey, l200.tier[:raw, :cal, PER, RUN])
FK_PHY, FK_CAL = first(fks_phy), first(fks_cal)
chinfo  = channelinfo(l200, FK_PHY)
DET_GED = first(filter(c -> c.system == :geds && c.processable, chinfo)).detector
DET_SPM = first(filter(c -> c.system == :spms, chinfo)).detector

# --- Column selection: three equivalent forms ----------------------------
read_ldata(l200, :jldsp, FK_PHY, DET_GED)                                # full
read_ldata((:e_cusp, :timestamp), l200, (:jldsp, FK_PHY, DET_GED))       # tuple
read_ldata(@pf((; $e_cusp, $timestamp)), l200, (:jldsp, FK_PHY, DET_GED))# @pf

# --- Per-detector across tiers (system auto-detected) --------------------
read_ldata((:waveform_presummed,), l200, (:raw,   FK_PHY, DET_GED))
read_ldata((:waveform_bit_drop,),  l200, (:raw,   FK_PHY, DET_SPM))
read_ldata((:e_cusp, :t50),        l200, (:jldsp, FK_PHY, DET_GED))
read_ldata((:e_cusp,),             l200, (:jlhit, FK_CAL, DET_GED); subgroup=:dataQC)
read_ldata((:daqenergy,),          l200, (:jlpls, FK_CAL, "PULS01");  subgroup=:tags)

# --- Event tier with detector → flat-prefixed Table ----------------------
read_ldata((:geds_e_cusp_ctc_cal, :geds_trig_e_cusp_ctc_cal),
           l200, (:jlevt, FK_PHY, DET_GED))

# Trigger-energy cut + cross-subgroup QC, all on flat names
read_ldata(@pf((; $geds_e_cusp_ctc_cal, $geds_t50)),
           l200, (:jlevt, FK_PHY, DET_GED);
           filterby = @pf $geds_is_valid_qc && $ged_spm_is_valid_lar &&
                          !$aux_muonveto_aux_trig && $geds_trig_e_cusp_ctc_cal > 20u"keV")

# --- Event tier without detector → native nested LH5 (use dots) ----------
# Note: the flat-prefix names (geds_is_valid_qc, …) only exist with a det.
# Without a det, use dot-syntax — `@pf $geds_is_valid_qc` would error,
# `@pf $geds.is_valid_qc == true` works (generic getproperty path).
evt = read_ldata(l200, :jlevt, FK_PHY;
                 filterby = @pf $geds.is_valid_qc == true)
evt.geds.e_cusp_ctc_cal       # VoV — per event, list of energies
evt.aux.pulser.aux_trig       # Vector{Bool}

# Old call style still works:
read_ldata((:tags,), l200, DataTier(:jlpls), :phy, partinfo, det_puls)

# --- Cross-tier filter (filtertier kwarg) --------------------------------
# (1) raw waveforms of events passing a jlevt QC + energy cut (phy)
read_ldata(@pf((; $waveform_presummed, $timestamp)), l200, (:raw, FK_PHY, DET_GED);
           filterby   = @pf $geds_is_valid_qc && $geds_trig_e_cusp_ctc_cal > 1500u"keV",
           filtertier = :jlevt)

# (2) cal raw filtered via cal jldsp (no jlevt for cal)
read_ldata((:waveform_presummed,), l200, (:raw, FK_CAL, DET_GED);
           filterby   = (@pf $e_cusp > 1000),
           filtertier = :jldsp)

# (3) symmetric — jldsp filtered via raw
read_ldata((:e_cusp,), l200, (:jldsp, FK_PHY, DET_GED);
           filterby   = (@pf $daqenergy > 100),
           filtertier = :raw)

# --- Multi-filekey, period, partition ------------------------------------
read_ldata((:e_cusp,), l200, (:jldsp, fks_cal[1:3], DET_GED))
read_ldata((:e_cusp,), l200, (:jldsp, :phy, PER, RUN, DET_GED))
read_ldata((:e_cusp,), l200, (:jldsp, :phy, DataPartition(1), DET_GED))

# Distributed: parallel=true after addprocs + @everywhere using …
# read_ldata((:e_cusp,), l200, (:jldsp, fks_phy, DET_GED); parallel=true)

# --- Other kwargs --------------------------------------------------------
read_ldata((:e_cusp,), l200, (:jldsp, FK_PHY, DET_GED); n_evts=1000)
read_ldata((:e_cusp,), l200, (:jldsp, FK_PHY, "DOESNOTEXIST"); ignore_missing=true)

[codecov]

Codecov Report

❌ Patch coverage is 0% with 178 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.28%. Comparing base (d2134f7) to head (2e15d35).

Files with missing lines	Patch %	Lines
ext/LegendDataManagementLegendHDF5IOExt.jl	0.00%	178 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #172      +/-   ##
==========================================
- Coverage   47.85%   46.28%   -1.57%     
==========================================
  Files          27       27              
  Lines        2466     2547      +81     
==========================================
- Hits         1180     1179       -1     
- Misses       1286     1368      +82     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[DaGeibl]

Hi, a review has been requested, but this PR is still a draft and nothing has changed since two weeks. @jonasschlegel is your work finished and is this PR ready to reviewed? If yes please convert this into a proper PR.

[DaGeibl]

Can you maybe add a few test, to see that we can guarantee we can open the data in any combination of DataSelectors?
In the testdata there is a cal file with the new data format, so you can use that.