Skip to content

Feature: Add option to input yml file to specify multiple patterns #273

@kobebryant432

Description

@kobebryant432

Is your feature request related to a problem? Please describe.
Currently 2 versions of the same yaml input file are needed to describe the same datasets.

CCLM:
  root: /dodrio/scratch/projects/2022_200/project_output/rcs/CORDEXBE2/postprocessing
  pattern: <experiment_id>/<variable_id2>/<frequency>/<variable_id>_<frequency2>_<aggregation>_<time_period>.nc
  meta_data:
    source_id: "CCLM"
    source_type: "RCM"
    domain_id: "europe"
    version: "v1"
    resolution: "~5km"
    driving_source_id: "ERA5"
    institution_id: "KU Leuven"
    data_format: "%Y%m%d"

# Currently only the daily CCLM data is loaded here!
CCLM_hourly:
  root: /dodrio/scratch/projects/2022_200/project_output/rcs/CORDEXBE2/postprocessing
  pattern: <experiment_id>/<variable_id>/<frequency>/<variable_id2>_hourly_<time_period>.nc
  input_convertor: "CCLM"
  meta_data:
    source_id: "CCLM"
    source_type: "RCM"
    domain_id: "europe"
    version: "v1"
    resolution: "2.8 km"
    driving_source_id: "ERA5"
    institution_id: "KU Leuven"
    data_format: "%Y%m%d"

Describe the solution you'd like
Ideally the solution would be to add the option to add a second pattern attribute on which to search (and include) data.

CCLM:
  root: /dodrio/scratch/projects/2022_200/project_output/rcs/CORDEXBE2/postprocessing
  pattern: 
 - <experiment_id>/<variable_id2>/<frequency>/<variable_id>_<frequency2>_<aggregation>_<time_period>.nc
 -  test/<experiment_id>/<variable_id2>/<frequency>/<variable_id>_<frequency2>_<aggregation>_<time_period>.nc
  meta_data:
    source_id: "CCLM"
    source_type: "RCM"
    domain_id: "europe"
    version: "v1"
    resolution: "~5km"
    driving_source_id: "ERA5"
    institution_id: "KU Leuven"
    data_format: "%Y%m%d"

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions