Reproduce OLMO-3-Think-SFT

[abhash-er]

This issue thread is to report the reproducibility steps and results of OLMO-3-Think-SFT using OLMo-Core and open-instruct. We are trying to target the 7B parameter model.

• Specifically we start from the hugging face model, convert it into the OLMo-core format using this script.
• We also need to prepare data before we run the training script. Since OLMO uses packed training instances, we would need to convert the data from HF.
• Then we can use the script at OLMo-core repository to run the SFT.

One can already find the scripts for running it on SLURM over here.

[ferreirafabio]

Hi,
I reproduced OLMo3-7B-Think-SFT (starting from the official base checkpoint) and OLMo3-7B-Instruct-SFT (starting from my think cp) using the OLMo3 recipe together with the tokenized dolci datasets. Baseline reproduction results were evaluated using OpenJury with both winrate and rubric-based scoring across AlpacaEval, Arena-Hard, and m-Arena-Hard benchmarks. Full results are available here.

I'm currently evaluating 20 intermediate Think checkpoints using the same methodology to analyze performance over Think training time. I will update the results document once completed.

Uploaded resources:

• Tokenized dolci datasets (ready for use with OLMo-core):
  • dolci-think-sft-tokenized
  • dolci-instruct-sft-tokenized
• 20 intermediate Think checkpoints: OLMo-3-7B-Think-SFT
• 5 intermediate Instruct checkpoints: OLMo-3-7B-Instruct-SFT

[ferreirafabio]

here's the follow-up results from the previous post: I evaluated 20 intermediate think training stage-checkpoints of OLMo3-7B-Think-SFT for winrate and rubrics on alpaca eval and alpaca hard.

Observations:
Some aspects are in line with expectations. For example, naturalness and coherence already appear strong in the base checkpoint and do not improve much with additional training, whereas accuracy and instruction following show clearer gains over time. The time step 0 checkpoints start very strong on AlpacaEval but then drop, while this pattern is much weaker on Arena-Hard (possibly because Alpaca Eval is easier and may allow more forgiving responses from close-to-base-checkpoint capability). Regarding low performance on swap_mode=fixed, the weak performance may result from the LLM judge being used (Qwen3-30B-A3B-Instruct-250).