01
A number before a demo
Nothing gets built until we agree how it will be scored. An evaluation set from your real data, a threshold that makes deployment worthwhile, and an explicit statement of what the system is not required to do.
Approach
Measure first. Then build.
Most AI projects fail on process, not on modelling. Ours is built to surface bad news early, while it is still cheap, and to leave you with something you could hand to someone else at the end of any stage.
Principles
Five commitments
These are the things we will not trade away under deadline pressure, because they are the reasons the work survives past launch.
02
Riskiest thing first
Whatever is most likely to sink the project gets attacked in week two, not week ten. Building the easy parts first feels productive and tells you nothing you needed to know.
03
The least AI that works
A model call is an expensive, non-deterministic function. We use one where the problem genuinely requires judgement, and ordinary code everywhere else. Restraint here is the difference between a system that runs profitably and one that doesn't.
04
Production shape from day one
Version control, tests, tracing, deployment and cost controls exist from the first commit. There is no phase where a prototype gets "productionised" — that phase is where projects go to die.
05
Designed for our own exit
Your code, your infrastructure, your model accounts, documented for your engineers. A consultancy that makes itself indispensable has misaligned itself with its client.
Timeline
What a typical build looks like
Illustrative shape for a mid-sized delivery engagement. Strategy and hardening engagements are shorter, and the boundaries move with scope — but the ordering does not.
01
Scope
Week 1
Workshops with the people who do the work today, not only the people sponsoring the project. We define success numerically, assemble a first evaluation set from your real cases, agree the integration surface, and write down the constraints — data residency, latency, spend, approval requirements. Output is a scope document precise enough to price against.
02
Prove
Weeks 2–3
We build the component most likely to fail and measure it against the evaluation set. No interface, no polish. At the end there is a score and a decision: proceed, adjust scope, or stop. Stopping here has cost you two weeks instead of two quarters, and we would rather that than the alternative.
03
Build
Weeks 4–10
Full system delivery: retrieval and data pipelines, model orchestration, integrations, interface, permissions, observability and cost controls. Weekly demos against a live environment. Evaluation scores run on every change, so a quality regression is visible the same day rather than discovered by a user a month later.
04
Pilot
Weeks 9–11
Overlapping with build. A real group of users on real work, instrumented for both quality and behaviour. This is where you learn what people actually do with it, which is reliably different from what the workshop predicted. Findings feed straight back into the remaining build weeks.
05
Hand over
Final two weeks
Architecture walkthroughs, runbooks, an eval suite wired into your CI, alerting configured, and pairing sessions with the engineers taking ownership. We consider the engagement finished when your team has shipped a change to the system without us.
Evaluation
The eval suite is the deliverable.
If you take one thing from working with us, we would rather it be this than any particular model or prompt.
A graded set of real cases with expected outcomes turns AI development from opinion into engineering. It is what lets you change a prompt without fear, switch to a cheaper model deliberately, upgrade to a new one on the day it launches, and prove to a regulator or a board that quality is monitored rather than assumed.
Prompts age. Models get replaced within months. The evaluation set is the asset that outlives both, and it is the thing teams most often skip.
$ npm run evals -- --suite extraction
cases 412
field accuracy 0.961 threshold 0.940 PASS
hallucination 0.004 threshold 0.010 PASS
refusal rate 0.021 threshold 0.050 PASS
p95 latency 2.4s threshold 4.0s PASS
cost / 1k docs $1.84 threshold $3.00 PASS
vs. main: +0.008 accuracy, -$0.41 cost
✓ safe to merge
Illustrative output. Metrics and thresholds are defined per engagement during scope.
Fit
Who this works for
Good fit
- You have a specific process in mind, not a mandate to "do something with AI"
- Someone senior owns the outcome and can make decisions inside a week
- The data the system needs already exists, even if it is untidy
- You have engineers who will take ownership after handover
- You would rather hear an uncomfortable assessment early than a comfortable one late
Poor fit
- The goal is a demo for a board meeting rather than a system for users
- Success can't be described in any measurable way
- The data required would have to be created from scratch first
- You need a supplier to own the system indefinitely
- The decision to build has already been made and only needs validating
Next step
Start with the scoping call.
Thirty minutes, no deck. We'll tell you whether we think the project is viable, what we would build first, and roughly what it costs.