CMR Metadata

[cmarshak]

Describe the bug
This is related to operations and coordinating between @asjohnston-asf, @cirrusasf, https://github.com/asfadmin/Discovery-asf_search, and JPL. This is related to #102 which now includes: frame_id, temporal_baseline_days, and version.

I submitted 5 products using the operational user which are currently set to be delivered to the UAT venue. All succeeded.

To view the relevant metadata:

import asf_search
opts=asf_search.ASFSearchOptions(host='cmr.uat.earthdata.nasa.gov')
results = asf_search.search(processingLevel=asf_search.GUNW_STD,
                            relativeOrbit=[164],
                            maxResults=50, 
                            opts=opts)
len(results)

Should return 5.

It appears the new metadata is going correctly to umm json as:

r = results[0]
r.umm

Returns:

{'TemporalExtent': {'RangeDateTime': {'BeginningDateTime': '2022-02-24T22:28:28.000Z',
   'EndingDateTime': '2022-02-24T22:29:20.000Z'}},
 'OrbitCalculatedSpatialDomains': [{'OrbitNumber': 42061},
  {'OrbitNumber': 41886}],
 'GranuleUR': 'S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0',
 'AdditionalAttributes': [{'Name': 'GROUP_ID',
   'Values': ['S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0']},
  {'Name': 'ASCENDING_DESCENDING', 'Values': ['ascending']},
  {'Name': 'BEAM_MODE_TYPE', 'Values': ['slc']},
  {'Name': 'BEAM_MODE', 'Values': ['IW']},
  {'Name': 'BEAM_MODE_DESC', 'Values': ['interferogram']},
  {'Name': 'POLARIZATION', 'Values': ['VV']},
  {'Name': 'LOOK_DIRECTION', 'Values': ['right']},
  {'Name': 'PATH_NUMBER', 'Values': ['164']},
  {'Name': 'BYTES', 'Values': ['69908881']},
  {'Name': 'NEAR_START_LON', 'Values': ['-71.32807577094489']},
  {'Name': 'NEAR_START_LAT', 'Values': ['47.35015226169008']},
  {'Name': 'FAR_START_LON', 'Values': ['-67.47945065650076']},
  {'Name': 'FAR_START_LAT', 'Values': ['47.35015226169008']},
  {'Name': 'FAR_END_LON', 'Values': ['-67.47945065650076']},
  {'Name': 'FAR_END_LAT', 'Values': ['49.14396169772953']},
  {'Name': 'NEAR_END_LON', 'Values': ['-71.32807577094489']},
  {'Name': 'NEAR_END_LAT', 'Values': ['49.14396169772953']},
  {'Name': 'ASF_PLATFORM', 'Values': ['Sentinel-1 Interferogram (BETA)']},
  {'Name': 'PROCESSING_TYPE', 'Values': ['GUNW_STD']},
  {'Name': 'PROCESSING_TYPE_DISPLAY', 'Values': ['Standard Product, NetCDF']},
  {'Name': 'PROCESSING_DESCRIPTION',
   'Values': ['Sentinel-1 SLC interferometric products generated by JPL using ISCE v2.0.0, delivered by ASF']},
  {'Name': 'THUMBNAIL_URL',
   'Values': ['https://grfn-public-test.asf.alaska.edu/S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0.png']},
  {'Name': 'PERPENDICULAR_BASELINE', 'Values': ['-59.4961']},
  {'Name': 'MISSION_NAME', 'Values': ['S1 I-grams (BETA) - Other']},
  {'Name': 'VERSION', 'Values': ['3.0.0']},
  {'Name': 'TEMPORAL_BASELINE_DAYS', 'Values': ['12']},
  {'Name': 'FRAME_NUMBER', 'Values': ['25502']}],
 'SpatialExtent': {'HorizontalSpatialDomain': {'Geometry': {'GPolygons': [{'Boundary': {'Points': [{'Longitude': -67.7838829742832,
         'Latitude': 48.803513684598364},
        {'Longitude': -67.78063949294818, 'Latitude': 48.80414813817185},
        {'Longitude': -67.82862731394724, 'Latitude': 49.00911898266449},
        {'Longitude': -68.35882950083834, 'Latitude': 48.952510206796376},
        {'Longitude': -68.87209673148254, 'Latitude': 48.89524681271839},
        {'Longitude': -68.89930636351852, 'Latitude': 49.001024725991755},
        {'Longitude': -69.49265888652626, 'Latitude': 48.93246094155771},
        {'Longitude': -70.08244510661609, 'Latitude': 48.861110698101},
        {'Longitude': -70.1620736681961, 'Latitude': 49.14396169772953},
        {'Longitude': -70.72357418792893, 'Latitude': 49.073063410471164},
        {'Longitude': -71.32807577094489, 'Latitude': 48.9934664193567},
        {'Longitude': -71.17732771160051, 'Latitude': 48.497691935550456},
        {'Longitude': -71.1335260373605, 'Latitude': 48.33245909620376},
        {'Longitude': -70.87876693003453, 'Latitude': 47.47767780134041},
        {'Longitude': -70.29087990731554, 'Latitude': 47.55199732126649},
        {'Longitude': -69.78111489146, 'Latitude': 47.613618796674885},
        {'Longitude': -69.70919700836623, 'Latitude': 47.35015226169008},
        {'Longitude': -69.09896846019329, 'Latitude': 47.421616017501826},
        {'Longitude': -68.55727989287345, 'Latitude': 47.48198524930101},
        {'Longitude': -68.52987513855085, 'Latitude': 47.37275123696082},
        {'Longitude': -67.99492534034955, 'Latitude': 47.429567703811465},
        {'Longitude': -67.47945065650076, 'Latitude': 47.48175879127325},
        {'Longitude': -67.56389051312044, 'Latitude': 47.852140949747735},
        {'Longitude': -67.7838829742832,
         'Latitude': 48.803513684598364}]}}]}}},
 'ProviderDates': [{'Date': '2023-05-27T04:06:43.000Z', 'Type': 'Insert'},
  {'Date': '2023-05-27T04:06:43.000Z', 'Type': 'Update'}],
 'CollectionReference': {'EntryTitle': 'Sentinel-1 Interferograms (BETA)'},
 'RelatedUrls': [{'Format': 'Not provided',
   'Type': 'GET DATA',
   'URL': 'https://grfn-test.asf.alaska.edu/door/download/S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0.nc'},
  {'Format': 'Not provided',
   'Type': 'GET RELATED VISUALIZATION',
   'URL': 'https://grfn-public-test.asf.alaska.edu/S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0.png'}],
 'DataGranule': {'DayNightFlag': 'Unspecified',
  'Identifiers': [{'Identifier': 'S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0',
    'IdentifierType': 'ProducerGranuleId'}],
  'ProductionDateTime': '2023-05-27T02:44:14.106Z',
  'ArchiveAndDistributionInformation': [{'Name': 'Not provided',
    'Size': 66.67030429840088,
    'SizeUnit': 'MB',
    'Format': 'Not provided'}]},
 'Platforms': [{'ShortName': 'SENTINEL-1A',
   'Instruments': [{'ShortName': 'SENTINEL-1A C-Band SAR',
     'ComposedOf': [{'ShortName': 'IW'}]}]}]}

However,

r.properties

Returns

{'beamModeType': 'slc',
 'browse': ['https://grfn-public-test.asf.alaska.edu/S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0.png'],
 'bytes': 69908881,
 'centerLat': None,
 'centerLon': None,
 'faradayRotation': None,
 'fileID': 'S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0',
 'flightDirection': 'ascending',
 'groupID': 'S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0',
 'granuleType': None,
 'insarStackId': None,
 'md5sum': None,
 'offNadirAngle': None,
 'orbit': 42061,
 'pathNumber': 164,
 'platform': 'Sentinel-1 Interferogram (BETA)',
 'pointingAngle': None,
 'polarization': 'VV',
 'processingDate': '2023-05-27T02:44:14.106Z',
 'processingLevel': 'GUNW_STD',
 'sceneName': 'S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0',
 'sensor': 'SENTINEL-1A C-Band SAR',
 'startTime': '2022-02-24T22:28:28.000Z',
 'stopTime': '2022-02-24T22:29:20.000Z',
 'url': 'https://grfn-test.asf.alaska.edu/door/download/S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0.nc',
 'pgeVersion': None,
 'fileName': 'S1-GUNW-A-R-164-tops-20220224_20220212-222854-00071W_00047N-PP-6c60-v3_0_0.nc',
 'frameNumber': None}

The questions are:

1. Should frameNumber be different than frame_id? I thought we wanted them to be the same.
2. Is there a way to asf_search to find products by: frame_id, temporal_baseline_days, and a range of startTime? Are we going to have to CMR or python-cmr?

Thanks.

[asjohnston-asf]

Regarding asf_search:

• I believe searching by startTime should already be supported through asf_search.search's start and end parameters
• I believe searching by frame number and populating frameNumber in search results requires changes to asf_search. I've opened Populate frameNumber for S1 ARIA GUNW search results asfadmin/Discovery-asf_search#197 for that feature request.
• I believe searching by temporal_baseline_days would require an additional feature request.

I believe any of these searches are already supported if you query CMR directly, in case that's valuable in the iterim.

[cmarshak]

So would like to "de-duplicate" (i.e. find existing GUNWs uniquely in CMR using these attributes) - what would be the lift for getting temporal_baseline_days into asf_search as a search parameter?

[jhkennedy]

@cmarshak most of this will be on the @glshort's side of the house -- @krkristenson-asf is going to connect with him and see how best to engage there. :-D