← All work
RegTech - architecture

Regulated AI, banking

Reads a bank's loan contracts against the regulator's circulars and flags the ones that break the rules, in a way a compliance officer can actually defend.

ArchitectComplianceRecall-gatedCompany unnamed

What it does

When a regulator issues a new circular, a bank has to work out which of its existing contracts just became non-compliant. Today that means people reading contracts. Thousands of them.

This system does the first pass. It scans contracts against the relevant circular and routes each one green or red, with the specific obligation it matched against.

I designed the architecture, the agent harness, the retrieval layer and the specs; my collaborator built it. The company stays unnamed here, but the engineering decisions are the interesting part anyway.

See it work

Click any stage to read what it does, or press the button to walk the whole thing.

The decision that mattered

A missed non-compliant contract is the exact regulatory failure the bank is buying protection against.

The team's accuracy target was gating on false positives. I argued that was backwards, and it's the decision I'd defend in any room.

Precision measures how often you cry wolf. Recall measures how often you miss the wolf. For a bank buying protection from a regulator, a missed non-compliant contract is the whole reason the product exists. So recall became the primary gate and precision the secondary metric.

Then I added a prerequisite nobody had asked for: measure how much two human adjudicators disagree before comparing the AI to either of them. Human disagreement is the ceiling on any accuracy you can honestly claim. Press Optimise for precision above to see what that trade-off actually costs.

50contracts in the gated test set
2independent adjudicators required
1gated metric: recall

Built with

Node.jsTypeScriptRAG pipelinePostgresDockerHuman-in-the-loop review
← SMVIT DebSoc All work LRNR →