Options for injecting custom transform/analysis algorithms

[peanutfun]

One typical use case I imagine for this package is the comparison of the filter/smoother methods implemented here with one implemented in another package. However, it is currently very complicated for users to add a new analysis scheme, or entire filter/smoother method to this package.

The "heavy-lifting" of filter and smoother operations is currently done by the EnsembleKalmanSchemes.transform function, which implements the different analysis schemes and discerns them via the analysis::String argument. To add a new analysis scheme, users have to alter this file and necessarily need to familiarize themselves with the source code of this package. Although this would require quite a bit of work, I think it would be great if the functions of this package supported the injection of analysis schemes via function arguments. This requires defining and documenting a proper interface for the analysis scheme functions.

In theory, this could be extended to all parts of the filter/smoother algorithms, most prominently also the state inflation.

Here's a crude example on how this could look like:

# ---
# DataAssimilationBenchmark.jl code

# Define module containing analysis functions
module AnalysisSchemes

function enkf(...)
    # EnKF analysis function...
    # This is the code that is currently located in the "analysis == enkf" branch of transform()
end

end  # module AnalysisSchemes


module EnsembleKalmanSchemes

# Re-define function argument
function ensemble_filter(analysis::Function, ...)
    # ...
    analysis(...)  # Call the passed analysis function here
    # ...
end

end  # module EnsembleKalmanSchemes

# ---
# User code
using DataAssimilationBenchmarks
using EnsembleKalmanSchemes

ensemble_filter(AnalysisSchemes.enkf, ...)  # Call with EnKF analysis scheme from this package
# ensemble_filter("enkf", ...)  # <-- This is what the same call currently looks like

function my_fancy_enkf(...)
    # My own EnKF analysis function!
end

ensemble_filter(my_fancy_enkf, ...)  # Call with custom EnKF analysis function, without modifying package code!

openjournals/joss-reviews#4129

[cgrudz]

This is a very nice suggestion, and I think it is a bit of a longer-term issue to handle, especially as the scope of this project is starting to change with additional types of estimators now included, such as classical variational methods. Once the dust has settled a bit, and it is more clear what will be needed in the code to handle multiple types of estimators with sub-flavors, I think I'll approach this again. For now, I think I am just defaulting to a case-by-case development, like was performed to include 3D-VAR and its extensions, as described here:

https://cgrudz.github.io/DataAssimilationBenchmarks.jl/dev/community/Contributing/#How-to-contribute-new-methods,-models-or-other-core-core