This bug manifests when using --parameter-scan and --reference together, e.g.
hyperfine \
--runs 1 \
--reference "sleep 1" \
--parameter-scan secs 2 3 \
--export-csv out.csv 'sleep {secs}'
The first parameterised run after the reference gives the following error when complete:
Error: CSV error: found record with 9 fields, but the previous record has 8 fields
Looking at out.csv, I can see that the column name for the secs parameter is missing:
command,mean,stddev,median,user,system,min,max
sleep 1,1.00279931462,0,1.00279931462,0.0006699600000000003,0.0018847000000000004,1.00279931462,1.00279931462
I would expect the parameter columns to exist but their values for the reference run to all be missing, since there is doesn't seem to be a way to specify what their values are in the reference command.
This bug manifests when using
--parameter-scanand--referencetogether, e.g.The first parameterised run after the reference gives the following error when complete:
Looking at
out.csv, I can see that the column name for thesecsparameter is missing:I would expect the parameter columns to exist but their values for the reference run to all be missing, since there is doesn't seem to be a way to specify what their values are in the reference command.