Legal Citation Verification
Your AI hallucinates cases. We catch it.
In 2023, an attorney was sanctioned for filing a brief citing cases ChatGPT had invented (Mata v. Avianca). This pack catches that — it takes every case citation an AI agent emits, looks it up in CourtListener (the free public database of US court opinions), and fails the run if any cite doesn't exist.
Highlights.
Verifies every Bluebook-form citation against CourtListener's public database (Free Law Project, CC0 data)
218 fixtures spanning landmark SCOTUS authorities + Stanford-confirmed fabrications + real court-sanctioned incidents
Deterministic — no LLM judges, no calibration drift, reproducible run-to-run
Every fixture traces to an authoritative source (CourtListener cluster id, Stanford paper row, or Charlotin incident URL)
Pistachio attorney sign-off pending (Phase L-5)
Vertical harnesses
are co-built.
Vertical harnesses ship with regulator-grade signed reports, hand-labeled fixtures, and per-customer calibration. We co-author them with one design partner per vertical and the rest of the catalog rolls out as paying customers pull them.
Example checks.
Cites a real Supreme Court case
Cites a fabricated case
Judging criteria.
What a pass means
A pass means every case citation in the agent's final message resolves to a real opinion in CourtListener. The bar is binary — one fabricated cite fails the run, because a single fake cite in a real filing is sanctionable.
Data sources
- CourtListener
Free public database of US court opinions (Free Law Project, CC0). Every cite the agent emits is resolved here at run time. No match = fail.
- Stanford 'Hallucinating Law'
Stanford's catalog of LLM-fabricated case citations — 150 confirmed fakes that fooled production legal AIs. Pack rejects every one.
- Damien Charlotin's AI hallucination tracker
Public record of real court cases where attorneys were sanctioned for filing AI-generated fake citations. 50 incidents seeded into this pack.