The CRM is the hard part

Calling a model is a weekend project. Letting it write to the system that runs your business is a year of unglamorous work, and that is where the value actually lives.

Our CRM is twenty years of accreted assumptions: fields that mean different things depending on who filled them, a status enum nobody can fully enumerate, and three integrations that all believe they own the same record. Dropping an eager agent into that is how you generate confident nonsense at scale.

Write the contract first

Before the model touched anything, we wrote down what a valid record looks like — not as documentation, but as an executable schema the writer has to satisfy. Most of the “AI” work turned out to be data-modelling work that had been deferred for two decades.

Run it in shadow

For six weeks the agent wrote nowhere. It produced proposals next to the human decision, and we measured agreement. The gap between “looks good in review” and “matches what an expert actually did” was the whole ballgame.

The model was never the bottleneck. The bottleneck was that we’d never written down what correct meant.