Add CRAB Benchmark

[cemde]

CRAB is a multi-agent benchmark around cross-enviornment interactions: https://crab.camel-ai.org

Enabling it would be interesting

[ThinkOffApp]

Another benchmark to consider alongside CRAB: we have been building OMATS (OpenClaw Multi-Agent Test Suite) which focuses specifically on room-based agent communication failures.

It tests 20 scenarios across 5 capability stages, from basic agent behavior (loop avoidance, idle discipline) through multi-agent coordination (echo resistance, stop order compliance, prompt hygiene) up to team management (task delegation, conflict resolution, guardrail compounding resistance).

Different angle from CRAB's cross-environment focus. OMATS is about social/communication dynamics between multiple AI agents sharing a conversation space. Based on patterns observed running a 9-agent fleet in production daily.

Scenarios and rubrics: https://github.com/ThinkOffApp/openclaw-multi-agent-test-suite

[cemde]

@ThinkOffApp great work. Looks very interesting! Would you mind opening a new issue for it?

[ThinkOffApp]

Busy running tests now, for example for a set of the newest Qwen models from Max down to 4B. Super interesting! In addition to tests where the agents deal with a fixed script, we're running tests where 2 or 3 agents of the same kind need to work together live, which is much more easily loop triggering than static inputs.

Results streaming in: https://github.com/ThinkOffApp/openclaw-multi-agent-test-suite

What kind of issue were you thinking off?

[cemde]

@ThinkOffApp Please consider opening a new issue as feature request for this library. https://github.com/parameterlab/MASEval/issues/new