GLM-Legal and Harvey LAB: What the 19.2% Score Means

On October 12, Z.ai released GLM-Legal, a model adapted for legal work from its GLM-5.3-Flash base. The headline figure is 19.2% on Harvey's Legal Agent Benchmark (LAB). The base model scored 6.7% on the same test.

What Z.ai announced

The model is built on GLM-5.3-Flash, a large mixture-of-experts system that activates only part of its weights for each token.
The legal version was trained in two stages: legal-focused mid-training, then reinforcement learning in simulated legal settings.
Access is by application. Law firms, legal tech companies, in-house teams and independent practitioners can apply through a form. No price has been published.
Z.ai plans to add the model to AutoClaw, its desktop agent app. It hasn't given a date.

What LAB tests

LAB contains 1,749 tasks across 24 practice areas, based on work lawyers do for real clients. The tasks ask for finished documents such as memos, redlines and filings, rather than a short answer. A task passes only if every required element is correct. One missed issue fails the whole task.

That's a far stricter test than question-and-answer benchmarks like LegalBench, and it helps explain why even strong models post low scores.

How to read 19.2%

The score comes from Z.ai's own reporting, and no outside group has rerun it yet.

The most direct comparison in the announcement is against the base model, a gain of 12.5 points. Scores for other models appear on Z.ai's chart but not in the text. Third-party coverage reads the chart as Kimi K3 at 12.9%, Claude Fable 5 at 11.3% and GPT-6 Astra at 5.4%. Check those against the chart before you quote them.

Z.ai also says GLM-Legal matches frontier systems on LegalBench and PLawBench, but the announcement doesn't include scores for either. Vals AI, an independent evaluator, tested the model on LegalBench and ranked it first among open-weight models. That's Vals AI's result, not Z.ai's. PLawBench focuses on Chinese legal practice, so it tells you little about work under US, UK or EU law.

Questions to answer before a pilot

The model is in beta, pricing isn't public, and Z.ai hasn't said which legal systems the training covers. Ask about all three.

Because the model comes from a Chinese company, look harder at data handling than you would for most tools in this directory. Ask where requests are processed, how long inputs are kept, and whether client material feeds into training. Get the answers in writing.

Run your own test on your own documents, in your own jurisdiction. A benchmark of 1,749 tasks won't show how the model handles your templates or your clients' rules.

Checklist

Confirm which jurisdictions the training covers.
Get a data processing agreement before any client material is shared.
Get pricing in writing and compare it with the tool you use now.
Check whether the weights are available if you plan to self-host.

Update log

October 2026: First version, based on Z.ai's October 12 announcement and third-party coverage. Next review: November 2026.

Sources:

GLM-Legal's 19.2% Harvey LAB Score: What It Measures and What It Doesn't