Record: Score-First TTT + Multi-Order N-gram Backoff (3-seed mean val_bpb=0.9581)#940

Open

antaloaalonso wants to merge 14 commits intoopenai:mainfrom

antaloaalonso:pr761

Commits on Mar 24, 2026

Record: 11L XSA-all + LeakyReLU(0.5)² + VR + GA (val_bpb=1.1164)

Asukabot0
and
claude
committed

Commits on Mar 25, 2026

Fix all default values to match p17 config

Asukabot0
and
claude
committed
Fix DDP 3x slowdown: replace find_unused_parameters with static_graph

Asukabot0
and
claude
committed
p21: auto-downgrade quant + warmdown=3000 + n-gram auto-enable

Asukabot0
and
claude
committed
Fix: progressive int5 fallback — one layer at a time from center

Asukabot0
and
claude
committed
Add n-gram parameter sweep script for 8xH100

Asukabot0
and
claude
committed
Optimal n-gram params: alpha=0.40 order=7 (8xH100 sweep)

Asukabot0
and
claude
committed
Multi-order n-gram backoff + entropy-adaptive alpha

Asukabot0
and
claude
committed
Score-first TTT + DDP static_graph fix for our_experiment

Asukabot0
and
claude
committed
Fix all training defaults in our_experiment to match p17 config

Asukabot0
and
claude
committed
Aggressive SGD TTT: LR=1.0, 20 epochs, all blocks unfrozen

Asukabot0
and
claude
committed
LeakyReLU slope 0.5 -> 0.9 (Issue #140 sweep: -0.013 BPP)

Asukabot0
and
claude
committed
Revert to p23 config for submission (3-seed mean=0.9581)

Asukabot0
and
claude
committed
Record: Score-First TTT + Multi-Order N-gram Backoff (3-seed mean val_bpb=0.9581)

Asukabot0
and
claude
committed