We removed the claim from six files. It was in the system prompt.

Build Notes — what shipped, what broke, and what we got wrong.

Over the last two weeks I removed the same false sentence from six files. This week I found it in the system prompt, which means the product had been saying it out loud the whole time.

The sentence is “live market prices”. Only India’s feed is daily, so for two of our three countries that single word was simply not true.

Six places, six commits, each one found by going looking after the last:

Every one of those is prose a human wrote and a human might re-read. That is the thing they had in common, and it is why I thought I was finished.

ai_brain.py, line 110, told the model: “The app has live market prices the farmer can open for exact spot figures.” And a clause later: “for live market prices give trend and direction in words”. The header on the reference data said the same thing.

So the claim was not sitting in a file waiting to be read. It was being handed to the model on every single request, as a fact about its own product, in the one voice a farmer has no reason to doubt.

The six before it were things we told people. This one was a thing we told the model to tell people. It generates rather than sits, and that makes it worse than all of them put together.

A farmer in Iowa asking our assistant about prices was not reading stale marketing copy. They were being told, by the product, in direct answer to their question, something we had already corrected everywhere a human would look.

This is the part I did not expect, and it is the reason the fix got a commit of its own.

MarketPricesView carried a docstring saying the US and Canadian endpoints “return an honest ‘not configured’ payload until wired”. They were wired. USDA and StatCan had both been connected. The docstring described a product that no longer existed.

Now follow what that does. Suppose someone had done the right thing and checked the store listing’s claim against the code. They would have found a docstring saying those countries return nothing at all, concluded the listing was wrong in the other direction, and “corrected” a true statement into a false one.

Rotten prose does not only mislead the person reading it. It makes the accurate prose look like the error. A stale comment is worse than no comment, because no comment sends you to the code and a stale one sends you to a confident wrong answer.

Nothing, in the code. The model is told something true instead of something false. That changes what it says and nothing about what it does.

Verified the only way worth verifying: 754 backend tests pass, manage.py check is clean, and grep for “live market price” across the API returns nothing at all. The last one is the only check that would have caught this in the first place, and it did not exist until this week.

Grep your own claims. Not once, across everything, including the files you do not think of as writing.

A system prompt is documentation that executes. A docstring is a promise about code that no test enforces. Neither is covered by your test suite, both are read as authoritative, and the only thing standing between them and a farmer is somebody remembering to look.

An earlier edition of this newsletter called out a number accurate about a different question than the one I was asking it. Another was a careers page saying less than the email I had already sent. This is the same failure in its most awkward form: I corrected the claim in public six times while the product kept making it in private.

If you ship an AI product, the system prompt is the highest-leverage piece of unreviewed prose in your company. Mine said something false about my own product for long enough that I corrected it everywhere else first.