Megatron LoRA correctness: align distributed semantics with Megatron and validate TP/EP/ETP/DP against an oracle#619

FurtherAI wants to merge 20 commits intomainOpenPipe/ART:mainfrom

austin/megatron_lora_correctness_oracle_testsOpenPipe/ART:austin/megatron_lora_correctness_oracle_testsCopy head branch name to clipboard