Support allocation scoring via forecast_allocation class

[seabbs-bot]

Summary

Add a forecast_allocation class to support scoring of resource allocation forecasts using generalised piecewise linear (GPL) loss functions, complementing the allocation scoring framework in the alloscore package (Gerding et al.).

Motivation

Allocation scoring (Gerding et al., 2023) evaluates forecasts through the lens of constrained resource allocation.
The central question is whether a forecast leads to good resource allocation decisions when a decision-maker must distribute a limited budget K across targets (e.g. locations).

This is directly relevant to public health forecasting (distributing vaccines, antivirals, hospital beds across regions) and was discussed in #1069 alongside the variogram score.

The alloscore package implements both the optimisation step (finding optimal allocations from forecast distributions) and the scoring step (evaluating allocations against outcomes).
This proposal covers only the scoring step in scoringutils, with alloscore handling the optimisation.

Reference: Gerding, A., Reich, N.G., Rogers, B. and Ray, E.L. (2023). Evaluating infectious disease forecasts with allocation scoring rules. arXiv:2312.16201

Proposed design

New class: forecast_allocation

An allocation forecast stores, per target within an allocation group:

• The allocation amount (predicted)
• The observed demand (observed)
• GPL loss parameters for scoring

Column	Type	Description
observed	numeric	Observed demand at this target
predicted	numeric	Allocated amount at this target
.alloc_group_id	any	Groups targets sharing a budget constraint
K	numeric	Total resource budget for this group
kappa	numeric	GPL scale parameter (default: 1)
alpha	numeric	GPL asymmetry parameter (default: 0.5)
w	numeric	Per-unit cost weight (default: 1)

The .alloc_group_id column plays the same structural role as .mv_group_id in forecast_multivariate_sample.

Constructor

as_forecast_allocation(
  data,
  forecast_unit = NULL,
  alloc_across = NULL,  # columns defining what is allocated across
  K = NULL,             # column name (required)
  kappa = NULL,         # column name or default value
  alpha = NULL          # column name or default value
)

Scoring metrics

• gpl_loss(): realised GPL loss summed within each allocation group
• gpl_regret(): excess loss relative to oracle allocation with perfect foresight

User workflow

# User computes allocations via alloscore or their own method
# Then evaluates them in scoringutils:
forecast <- as_forecast_allocation(
  allocation_data,
  alloc_across = "location",
  K = "K", alpha = "alpha"
)

scores <- score(forecast)
scores |> summarise_scores(by = "model")

Relationship to alloscore

This design complements alloscore rather than replacing it:

Concern	alloscore	scoringutils
Compute optimal allocations from forecast distributions	Yes	No
Score pre-computed allocations against outcomes	Yes	Yes
Evaluation pipeline (summarise, compare, visualise)	Limited	Yes

Generality

The design works for any constrained vector prediction problem with asymmetric loss, not just epidemic resource allocation.

Open questions

1. Should as_forecast_allocation() strictly require sum(w * predicted) == K or allow under-utilisation?
2. How to handle the g (increment) function in GPL loss: support identity and log as built-in string options?
3. Should oracle computation live in scoringutils, or should users provide oracle scores from alloscore?
4. Should GPL parameters be data columns, metric arguments via purrr::partial(), or both?

Related issues

• Variogram scores #1069 (original variogram/allocation score discussion)
• Implement variogram scores #1111 (variogram score implementation)

This was opened by a bot. Please ping @seabbs for any questions.

[seabbs]

If were to do this we definitely don't want kappa etc in the allocation class as that is score specific. I need to read more to see if we would even need K. I need to read more in general to see if this is useful.

[nikosbosse]

Love the idea in general!