Legal Issue Detection
Did your AI spot every legal issue?
The first thing a lawyer does when a client tells them a story is spot every legal issue lurking in it — including the ones the client didn't ask about. A founder asking about cofounder equity might also have securities, statute-of-limitations, and litigation-hold issues hiding in the same fact pattern. This pack runs your AI through 30 California Bar essay fact patterns and checks whether it names every issue that a passing CA bar student identified. Miss one and the client walks into trouble.
Highlights.
Backed by California Bar passing student answers — publicly graded as passing by CA bar graders
30 fixtures spanning three administrations (Feb 2024, Jul 2024, Feb 2025)
Issues extracted via versioned Claude prompt at temperature=0 (llm_assisted_from_authoritative_answer provenance tier)
Deterministic grader — normalized substring recall against label phrases, no LLM judges at run time
Pistachio attorney sign-off pending (Phase L-5)
Vertical harnesses
are co-built.
Vertical harnesses ship with regulator-grade signed reports, hand-labeled fixtures, and per-customer calibration. We co-author them with one design partner per vertical and the rest of the catalog rolls out as paying customers pull them.
Example checks.
Spots every issue in a community-property dispute
Misses latent issues
Judging criteria.
What a pass means
A pass means the agent's reply names at least 80% of the legal issues a passing CA bar student identified (case- and whitespace-insensitive substring of the issue label). Adjustable passing score.
Data sources
- California Bar published passing answers
California publishes the student essay answers its graders selected as passing on each bar exam — Feb 2024, Jul 2024, Feb 2025. By construction, those answers identify every legal issue the graders cared about. Pack extracts the issue lists at temperature=0 with deterministic caching.