01
Detection
Notice the incident before it is handed a hint about where to look.
Mesh 94.12%
Benchmarks ›
Mesh beats frontier models and agents at 1/30th the cost.
AIOpsLab is Microsoft Research's benchmark for autonomous-cloud agents: 86 incident scenarios across detection, localization, diagnosis, and mitigation. Mesh is self-learning — it builds context from every verified outcome and carries that understanding into the next incident.
Benchmark average
Mesh 91.4%
Competitor avg 69.8%
AIOpsLab results
Cerebral Systems (Mesh) 91.4%
Claude Code 80.5%
V
Vorflux 72.4%
AOI 66.3%
Claude Code 60.0%
What's measured
Four tasks, one incident lifecycle.
AIOpsLab injects a real fault into a live microservice deployment, then scores each step an on-call engineer would normally perform.
02
Localization
Pinpoint the failing component inside the service graph.
Mesh 100%
03
Diagnosis
Identify the root cause instead of stopping at the symptom.
Mesh 100%
04
Mitigation
Choose the action that resolves the incident safely.
Mesh 93.3%
Full AIOpsLab board
Published baselines, kept visible.
The table preserves every current row and task score so the headline remains auditable.
| # | Agent | Model | Detection | Localization | Diagnosis | Mitigation | Average |
|---|---|---|---|---|---|---|---|
| 1 | MeshBest Cerebral Systems | DeepSeek V4 Flash | 94.12 | 100 | 100 | 93.3 | 91.39 |
| 2 | Claude Code native agentic | claude-opus-4-8 | 100 | 71.4 | 38.5 | 100 | 80.5 |
| 3 | Vorflux agentic | — | 75.0 | 67.9 | 46.2 | 100 | 72.4 |
| 4 | AOI open-weight + AOI | Qwen3-14B | 100 | 53.6 | 30.8 | 46.2 | 66.3 |
| 5 | Claude Code agentic CLI | claude-opus-4-7 | 88.2 | 71.4 | 23.1 | 57.1 | 60.0 |
| 6 | AOI frontier | GPT-4o-mini | 90.6 | 32.1 | 38.5 | 53.8 | 58.1 |
| 7 | AOL-agent frontier | Claude Sonnet 4.5 | 68.8 | 53.6 | 15.4 | 76.9 | 57.0 |
| 8 | STRATUS multi-agent | GPT-4o-mini | 78.1 | 25.0 | 15.4 | 23.1 | 43.0 |
| 9 | STRATUS multi-agent | Qwen3-14B | 75.0 | 32.1 | 7.7 | 15.4 | 41.9 |
| 10 | AOL-agent baseline | GPT-4o-mini | 25.0 | 9.5 | 7.7 | 7.7 | 14.7 |
Baseline figures as reported in the AIOpsLab, STRATUS, and AOI papers. Open-weight AOI / STRATUS rows report best@5 across five runs; frontier-model rows are single-run. Mesh figures are from our own AIOpsLab run on DeepSeek V4 Flash, averaging a 49.14s runtime per scenario. The Claude Code rows are our own AIOpsLab runs. The claude-opus-4-8 row is its native agentic mode, micro-averaged by task counts (70 of 87 passes = 80.5%); the claude-opus-4-7 row (89 tasks) is shown as the macro average across the four categories (68.5% micro / 61 of 89). The Vorflux row is our own AIOpsLab run, micro-averaged by task counts (63 of 87 = 72.4%; detection 24/32, localization 19/28, diagnosis 6/13, mitigation 14/14).
Benchmark 02 · Loghub
The log-analysis board, without hiding harness quality.
Mean reward over 180 log-parsing and analysis tasks. Mesh takes the top two spots and finishes with zero harness errors.
| # | Harness | Model / agent | Tasks | Mean reward | Fully solved | Errors |
|---|---|---|---|---|---|---|
| 1 | Mesh Loghub profileBest | deepseek/deepseek-v4-flash | 180 | 0.915 | 66 (37%) | 0 |
| 2 | Mesh Loghub profile | mimo-v2.5-pro | 180 | 0.892 | 64 (36%) | 0 |
| 3 | Claude Code | claude-opus-4-7 | 180 | 0.878 | 45 (25%) | 0 |
| 4 | Raw mini-swe-agent | mimo-v2.5-pro | 171/180 | 0.861 | 32 | 2 |
| 5 | Raw mini-swe-agent | deepseek/deepseek-v4-flash | 180 | 0.860 | 31 (17%) | 0 |
| 6 | Devin | devin | 180 | 0.747 | 38 (21%) | 26 |
Harness: harbor==0.13.1. Mean reward is the harness's per-task score averaged over all 180 tasks; “fully solved” counts tasks that reached maximum reward. Mesh runs the same harbor harness as the baselines — only the orchestration profile differs. Errors are tasks the harness failed to complete cleanly.