Renewal & liability extractionRegression

Extracts renewal dates, notice periods and liability caps from 120 annotated contracts. Runs against contract-clauses-gold on Contract Analyst v6.

Pass rate

78.4%

▼ 6.8 pts

Avg. score

0.79

Failing cases

26 / 120

Last run

1284.0s

Pass rate by run

Last 10 runs · latest 40m ago by Aisha Bello

A case passes only when every scorer passes.

Valid JSON schema
86.7%
JSON schemacontract_terms@3
Output parses and validates against the strict schema.
Field accuracy
79.2%
Exact match≥ 4 / 5 fields
Renewal date, notice days, cap, governing law and auto-renew all match.
Faithfulness
88.3%
Groundedness≥ 0.85
Every extracted value is supported by a quoted clause.

Showing 5 of 120 cases from the latest run · 3 failing

Failing only

	Case	Input	Valid JSON schema	Field accuracy	Faithfulness	Latency
	tc_0004	MSA — Harrowgate Freight (34 pp). §4.2: “This Agreement renews for successive 12-month terms unless either party gives 90 days’ notice.”	Fail	Fail	Pass0.92	14.0s
Input MSA — Harrowgate Freight (34 pp). §4.2: “This Agreement renews for successive 12-month terms unless either party gives 90 days’ notice.” Expected { renewal_date: 2027-03-31, notice_days: 90, auto_renew: true } ActualFail { renewal_date: null, notice_days: 90, auto_renew: true } Valid JSON schema: renewal_date is required; got null. Field accuracy: Missed renewal date computed from effective date + term.
	tc_0009	SaaS order form — Copperleaf Studio. Liability capped at fees paid in the prior 12 months.	Pass	Pass	Pass0.97	9.4s
	tc_0016	DPA addendum — Marlow & Finch. No renewal language; term follows the main agreement.	Pass	Fail	Fail0.38	8.1s
	tc_0022	Reseller agreement — Tidepool Outfitters. §11.1 auto-renews yearly; 60 days’ notice; cap 2× annual fees.	Pass	Pass	Pass0.95	11.3s
	tc_0035	Consulting SOW — Brightwater Analytics. Fixed term ending 2027-06-30, no renewal.	Fail	Pass	Pass0.99	7.6s