query re: bmf schema intent

[SignpostMarv]

Is it intended to be:

type BMF = {
  [key: string]: {
    [key: string]: (
      | {
        value: number,
        lower_value: number,
      }
      | {
        value: number,
        upper_value: number,
      }
      | {
        value: number,
        lower_value: number,
        upper_value: number,
      }
    ),
  }
};

or

type BMF = {
  [key: string]: {
    [key: string]: (
      | {
        value: number,
      }
      | {
        value: number,
        lower_value: number,
        upper_value: number,
      }
    ),
  }
};

p.s. avoiding this because it means number | undefined

type BMF = {
  [key: string]: {
    [key: string]: {
      value: number,
      lower_value?: number,
      upper_value?: number,
    },
  }
};

[epompeii]

The intent is closer to the first example:

type BMF = {
  [benchmark: string]: {
    [measure: string]: (
      {
        value: number,
      }
      | {
        value: number,
        lower_value: number,
      }
      | {
        value: number,
        upper_value: number,
      }
      | {
        value: number,
        lower_value: number,
        upper_value: number,
      }
    ),
  }
};

[SignpostMarv]

Does the inclusion of one of lower_value or upper_value but the absence of the other imply that value represents the missing data?

[epompeii]

Does the inclusion of one of lower_value or upper_value but the absence of the other imply that value represents the missing data?

No? So the value is the only part of a Metric that is used outside of visualizations. Thresholds only ever look at the value for example.

Different benchmark harnesses have very different semantics for how lower_value and upper_value relate to value, if they even have them at all.

For a custom benchmark harness you can basically add whatever semantics you want. Bencher is intentionally very unopinionated on this