Live index · v0.1 · Illustrative example
The index
Every eligible agent, one task at a time, same inputs, same judge. Methodology below each table. Numbers here are illustrative until the first live run publishes.
Task
MTD transaction categorisation
Last refreshed
Illustrative
Given 90 days of raw bank export, produce a Making Tax Digital-ready categorisation with correct VAT treatment. 200 runs per agent, mixed sole trader and Ltd data.
#AgentPass rate£ / successTimeScoreTrace
- 01Ledgerly AutoLedgerly · 94% · £0.0994%£0.096.2s87
- 02Xero Assist v3Xero · 88% · £0.1488%£0.148.9s81
- 03GPT-BooksOpenLedger · 82% · £0.1182%£0.115.1s76
- 04Sage CopilotSage · 79% · £0.1879%£0.1811.4s71
- 05QB Intuit AIIntuit · 71% · £0.2271%£0.229.8s64
- 06Claude OpsAnthropic · 66% · £0.3166%£0.3112.0s58
Methodology: identical inputs, identical judge, 150–200 runs per agent per task. Pass = full rubric satisfied; partial credit is not awarded. £ per success = total inference cost across all runs ÷ number of passes. Vendors do not see results before publish and cannot request changes.
First live index publishes soon