ED2 crashes at restart

[femeunier]

Describe the bug
While I was trying to setp up a restart simulation, I realized that turning on the Q output files (IQOUTPUT = 3) made the restart run fail at the first time step. The crash does not happen if the Q files were originally included in the first simulation creating the S files. This seemed like a classic initialization problem but compiling ED2 with stricter files did not help out. I can of course achieve what I need by writing the Q files from the start, but most of them are not needed (i.e. spinup) and I guess this is unexpected ED2 behavior.

To Reproduce
Steps to reproduce the behaviour:

1. Environment (check all that apply):

• [ x] Linux
• MacOS
• Containerised version
• Other

2. Compiler (check all that apply):

• [ x] GNU (gfortran/gcc)
• [ x] Intel (ifort/icc)
• PGI (pgf90/pgcc)
• Other

3. Relevant options in
   See attached ED2IN
   ED2IN_restart_Q.txt

4. Error message:
   See attached file
   Qnew_job.sh.o26736477.txt

[mpaiao]

Sorry @femeunier, I totally missed this issue.

The interesting/mysterious part is that the code is crashing at the budget step, whereas QMEAN variables should be used exclusively for diagnostics in the code. I saw your crash report and the error is coming from carbon budget. The suspicious one there is the "committed" term, which is the only "flux" term that has the same value as the the residual.

I cannot see why cbudget_committed would be impacted by turning QMEAN on/off. Things worth checking is whether or not the terms used to calculate cbudget_committed somehow are using QMEAN or (more likely) FMEAN variables, which are also enabled when running IQOUTPUT=3 if I remember correctly. The other alternative would be to double check the "today" variables, and whether or not they are using fmean.