Non-record: XSA-all + mHC + Full QAT (val_bpb=1.1211)#928
Non-record: XSA-all + mHC + Full QAT (val_bpb=1.1211)#928autocode-rayes wants to merge 1 commit intoopenai:mainfrom
Conversation
Three changes on PR openai#549 stack: - XSA on all 11 layers (was last 4) - Manifold-constrained hyper-connections (22 extra params) - Full-training QAT (LATE_QAT_THRESHOLD=1.0) Seed 1337: sliding_window=1.1229, legal_ttt=1.1211 Artifact: 15.95 MB, 8xH100 SXM, 600s train + 482s eval Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
slope 0.75 + LR 0.027 + warmdown 3700 (PR openai#977) No SWA with QAT (PR openai#989) QAT from 50% + range fix [-31,31] mHC 22-param residual mixing (PR openai#928) VE128 + no gated_attn + no value_residual (PR openai#549) LZMA preset 7 compression (PR openai#999) Muon TTT with NS3 (PR openai#999) Entropy-adaptive TTT epochs 2/3/4 (PR openai#999) Per-layer TTT LR (PR openai#995) TTT momentum 0.95 (PR openai#995)
Community Review — Non-record: XSA-all + mHC + Full QAT (val_bpb=1.1211)BPB: 1.1211 | Compliance: LOOKS CLEAN — score-first-per-chunk TTT (legal #1416/#1423 pattern) What I found in the code (head SHA The TTT path at line 1083 implements the score-first-per-chunk pattern: each chunk is scored under Per Issue #402 and Issue #677, TTT is legal when each token is scored before the adapter updates on it, and that's what the code does here — chunk CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.06s, dim=512, layers=11, vocab=1024, code=90110 B, SMOKE_TEST_PASS Verdict: LOOKS CLEAN. Recommendation to @cocohearts @valerio-oai @0hq @yuzhougu-oai @notapplica: MERGE pending standard checks (3-seed validation, 16MB artifact cap, 10-min wallclock on 8×H100 SXM). The compliance picture matches the legal reference frontier and no flags were raised by the classification pass. Auto-classification caveat: this review was drafted by the AST-based classifier against a template derived from manually-reviewed cluster PRs (#1420, #1450, #1487, #1541, #1529, #1533, #1518). If I've misread a subtlety in your eval path — e.g., multi-epoch TTT that I mistook for single-pass, or a target-in-key lookup I missed in a helper function — please flag it and I'll re-run the audit manually. Reviewed by @MatoTeziTanka — The Agora. CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.06s, dim=512, layers=11, vocab=1024, code=90110 B, SMOKE_TEST_PASS. Classification via deterministic AST-based |
Three changes on PR #549 stack:
Seed 1337: sliding_window=1.1229, legal_ttt=1.1211
Artifact: 15.95 MB, 8xH100 SXM, 600s train + 482s eval