GLOWS - Add packet file name to L1B data#2761
GLOWS - Add packet file name to L1B data#2761sdhoyt wants to merge 12 commits intoIMAP-Science-Operations-Center:devfrom
Conversation
to the L1B products. All pytests pass, but still need to look at the actual CDF files to verify the new variables are correct.
for L1a string/single element attributes: ground_software_version, pkts_file_name, and flight_software_version.
with new variables. Need to refactor tests after validating CDFs.
product. Still need to refactor test code. Validated CDF file output with Matlab using this code to generate L1a and L1b Histogram data from 20260125.
Re-validated good CDF files with the updated code using MATLAB and flight 20260125 data.
Successfully generated new l1a and l1b histogram products (CDFs) and validated with MATLAB. Passing all existing GLOWS l1a and L1b tests.
tech3371
left a comment
tech3371
left a comment
There was a problem hiding this comment.
I have one major suggestion which might change remaining changes in this PR.
Sean, it might be helpful to add Maxine or few others besides David to get you quicker review in the future.
| FILLVAL: # TBD: what is fillval for strings? | ||
| FORMAT: S256 # TBC | ||
| LABLAXIS: File name | ||
| FILLVAL: " " |
There was a problem hiding this comment.
My first response was that we shouldn't have empty fillval but then saw above comment. Can you add this note for future reader?
| FILLVAL: " " | |
| FILLVAL: " " # fillval for strings |
| CATDESC: Version of ground software used for processing | ||
| DISPLAY_TYPE: no_plot | ||
| FIELDNAM: Ground Software Version | ||
| FILLVAL: " " |
There was a problem hiding this comment.
same here and other places
| pkts_file_name: | ||
| <<: *support_data_defaults | ||
| CATDESC: Name of input file with CCSDS packets data | ||
| DISPLAY_TYPE: no_plot |
There was a problem hiding this comment.
I believe, we don't need DISPLAY_TYPE for 'metadata' var_type. - https://imap-processing.readthedocs.io/en/latest/external-tools/cdf/cdf_requirements.html#required-metadata-data-attributes
| VAR_TYPE: metadata | ||
| CATDESC: Name of L0 packets file data originated | ||
| FIELDNAM: Packets File Name | ||
| FILLVAL: ' ' | ||
| FORMAT: A49 |
There was a problem hiding this comment.
can you make keys in alphabetic order?
| output["flight_software_version"] = xr.DataArray( | ||
| np.array([hist_l1a_list[0].flight_software_version], dtype=np.uint32), | ||
| coords={"scalar": [0]}, | ||
| name="flight_software_version", | ||
| dims=["scalar"], | ||
| attrs=glows_cdf_attributes.get_variable_attributes( | ||
| "flight_software_version", check_schema=False | ||
| ), | ||
| ) |
There was a problem hiding this comment.
I feel like we should make these kinds of one element array into global attrs, similar to our ground_software_version global attrs here. What do you think?
If we make these into global_attrs, we don't need to add CDF attrs in yaml file too.
There was a problem hiding this comment.
Personally, I think that makes a lot of sense, however, that is not what the GLOWS team is asking for.
3 participants
Change Summary
Overview
This PR has not been tested yet, but has code to pull the packet file name from the L1B Parent global attribute and create a variable in the L1B datasets. It also contains the metadata for the missing fields in L1B and L2. I'm dropping this ticket since the rest is not metadata related.
This partially implements ticket #2340