Skip to content

Remove post processing core#372

Open
mo-marqh wants to merge 2 commits into
MetOffice:mainfrom
mo-marqh:remove-post-processing-core
Open

Remove post processing core#372
mo-marqh wants to merge 2 commits into
MetOffice:mainfrom
mo-marqh:remove-post-processing-core

Conversation

@mo-marqh
Copy link
Copy Markdown
Member

@mo-marqh mo-marqh commented May 14, 2026

PR Summary

Sci/Tech Reviewer:
Code Reviewer: Steve Mullerworth (@stevemullerworth)

This change removes the post-processing logic for the majority of cases from core

  • This logic is not compatible with XIOS3.
  • This logic is not well engineered with regard to MPI, init_wait calls.
  • This logic is not well optimised for file storage access at scale.
  • lfric_apps metadata definitions have meant that the use of the code is sporadic and not well understood with regard to triggers within lfric_apps

In detail:

  1. the definition of global attributes is replaced by direct XIOS usage.
  2. the definition of time metadata, including forecast_reference_time is removed;
  3. the definition of scaling for UGRID planar coordinates is retained
    • to support specific test cases, that are not used in suites and large scale simulations
    • that are protected by careful logic to only run in these cases

Code Quality Checklist

  • I have performed a self-review of my own code
  • My code follows the project's style guidelines
  • Comments have been included that aid understanding and enhance the readability of the code
  • My changes generate no new warnings
  • All automated checks in the CI pipeline have completed successfully

Testing

  • I have tested this change locally, using the LFRic Core rose-stem suite
  • If required (e.g. API changes) I have also run the LFRic Apps test suite using this branch
  • If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes)
  • I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.)
  • Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes)

trac.log

Test Suite Results - lfric_core - remove-post-processing-core/run2

Suite Information

Item Value
Suite Name remove-post-processing-core/run2
Suite User mark.hedley
Workflow Start 2026-05-14T10:23:56
Groups Run all
Dependency Reference Main Like
lfric_core mo-marqh/lfric_core@remove-post-processing-core False
SimSys_Scripts MetOffice/SimSys_Scripts@cab3315 True

Task Information

✅ succeeded tasks - 402

also tested with lfric_apps main to confirm compatibility

Test Suite Results - lfric_apps - test_apps_remove-post-processing-core/run1

Suite Information

Item Value
Suite Name test_apps_remove-post-processing-core/run1
Suite User mark.hedley
Workflow Start 2026-05-14T10:32:11
Groups Run all
Dependency Reference Main Like
casim MetOffice/casim@2026.03.2 True
jules MetOffice/jules@2026.03.2 True
lfric_apps mo-marqh/lfric_apps@7f8f805 True
lfric_core mo-marqh/lfric_core@dec84ae True
moci MetOffice/moci@2026.03.2 True
SimSys_Scripts MetOffice/SimSys_Scripts@cab3315 True
socrates MetOffice/socrates@2026.03.2 True
socrates-spectral MetOffice/socrates-spectral@2026.03.2 True
ukca MetOffice/ukca@1cdb9c2 True

Task Information

❌ failed tasks - 4
Task State
rose_ana_lfricinputs_um2lfric-aquaplanet_lbc_azspice_gnu_fast-debug-64bit failed
rose_ana_lfricinputs_um2lfric-aquaplanet_lbc_azspice_gnu_full-debug-64bit failed
rose_ana_lfricinputs_um2lfric-aquaplanet_lbc_ex1a_gnu_fast-debug-64bit failed
rose_ana_lfricinputs_um2lfric-aquaplanet_lbc_ex1a_gnu_full-debug-64bit failed
✅ succeeded tasks - 1538
⌛ waiting tasks - 2
Task State
housekeep_azspice waiting
housekeep_ex1a waiting

Security Considerations

  • I have reviewed my changes for potential security issues
  • Sensitive data is properly handled (if applicable)
  • Authentication and authorisation are properly implemented (if applicable)

Performance Impact

  • Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

AI Assistance and Attribution

  • Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

Documentation

  • Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

PSyclone Approval

  • If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the TCD Team

Sci/Tech Review

  • I understand this area of code and the changes being added
  • The proposed changes correspond to the pull request description
  • Documentation is sufficient (do documentation papers need updating)
  • Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

  • All dependencies have been resolved
  • Related Issues have been properly linked and addressed
  • CLA compliance has been confirmed
  • Code quality standards have been met
  • Tests are adequate and have passed
  • Documentation is complete and accurate
  • Security considerations have been addressed
  • Performance impact is acceptable

mo-marqh added 2 commits May 14, 2026 10:11
…ing. functional switching; only post process on ugrid planar projected. XIOS3 compatibility.
@mo-marqh mo-marqh requested a review from a team as a code owner May 14, 2026 12:34
@mo-marqh mo-marqh requested review from Mike Hobson (mike-hobson) and removed request for a team May 14, 2026 12:34
@mo-marqh
Copy link
Copy Markdown
Member Author

mo-marqh commented May 14, 2026

there are 4 failing rose_ana tests in the lfric_apps test suite, where the diff from nccmp is highlighting the removal of forecast_reference_time and forecast_period

This is the expected change from this change Set.

It is not clear whether a linked PR is needed here, as the code and tests would not change to let these pass, instead the static files that support these rose_ana tests would need to be updated.
e.g.
/home/users/umadmin/standard_jobs/lfricinputs/kgo/lfricinputs_um2lfric-aquaplanet_lbc_azspice_gnu_fast-debug-64bit/vn3.1/checkpoint_um2lfric_000001.nc

this code is ready for Sci/Tech review, but this lfric_apps testing coordination needs addressing before it passes cove review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants