new predict method API, and reduced sampling in input space

[odunbar]

Purpose

• Closes "Predicted covariance" is not always what the user thinks it is. Perhaps a better API? #400
• Closes Add a machine learning tool for user-defined functions #380

Content

• For the predict() for both Emulator and ForwardMapWrapper
  • Added deprecation notice to transform_to_real=false kwarg
  • Added add_obs_noise_cov=false kwarg to predict. when set true, this adds the regularization to the machine learning tool
  • Added encode=nothing to the predict method, when set to "in", "out", "in_and_out" it will take in / push out encoded/decoded outputs.
  • When encoding inputs, with ForwardMapWrapper, samples are decoded into the output space before application

API is now improved for user experience

# (User) typical to predict at new points:
y_pred, y_pred_cov = predict(em_or_fmw, new_inputs) 

# (Internal) call within encoded MCMC, with full uncertainty
g_pred, g_pred_cov = predict(em_or_fmw, new_inputs, encode="in_and_out", add_obs_noise_cov=true) 

• When retrieving MCMC posterior, if a sufficiently lossy encoder is used noise is injected into the null space from the prior (and correlation-aware)
• Added utility get_encoder_from_schedule to return E,b from an encoder_schedule defining an affine encoding Ex + b
• Added new docstrings and improved existing docstrings
• Full unit testing.

Misc

• encoding in input space does lead to lost performance in the (already-slow) ReverseDiff-mode differentiable MCMC samplers. But otherwise no noticeable differences
• example testing. Works on Sinusoid example, [now testing on Darcy...]
• Bugfix for arguments of _predict , , -> ;

Detail on current encoding setup

Current settings: full (F) vs reduced (R) space

• Emulator space (R)
• MCMC: initial conditions (R)
• MCMC: computing likelihood (R)
• MCMC: stored prior distribution (F)
• MCMC: computing logpdf(prior,x) (F) (decoding current sample of subspace)
• MCMC: output posterior samples (F) (decoding each posterior sample + add correlated noise in null space from the prior)

Darcy example

We compare, lossless encoding. Lossy encoding without inflation, Lossy encoding with inflation, and the truth. We look for the lossy encoding to by increased by using the prior variance. The D utility indicates the amount of contraction from prior to posterior (by comparison of determinants)

Lossless - (20D->50D), D utility O(10^11)

Lossy - inflate - (18D->13D), D utility O(10^11)

Lossy - no inflate - (18D->13D), D utility O(10^43)

true

---

• I have read and checked the items on the review checklist.

[codecov]

Codecov Report

❌ Patch coverage is 77.55102% with 33 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.92%. Comparing base (c011a05) to head (faf88f5).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
src/Utilities.jl	36.36%	21 Missing ⚠️
src/MarkovChainMonteCarlo.jl	89.13%	5 Missing ⚠️
src/MachineLearningTools/GaussianProcess.jl	76.92%	3 Missing ⚠️
src/Utilities/elementwise_scaler.jl	0.00%	2 Missing ⚠️
src/Emulator.jl	97.50%	1 Missing ⚠️
src/MachineLearningTools/VectorRandomFeature.jl	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #404      +/-   ##
==========================================
- Coverage   94.20%   92.92%   -1.29%     
==========================================
  Files          10       10              
  Lines        1813     1950     +137     
==========================================
+ Hits         1708     1812     +104     
- Misses        105      138      +33     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.