My Code, My Test, and My Prompt All Agreed. All Three Were Wrong.
I tried to replace my receipt model with one 18× cheaper. The bug was in my benchmark, and the real lesson was about trusting AI to verify itself.
Jul 8, 202612 min read5

Search for a command to run...
Articles tagged with #llm
I tried to replace my receipt model with one 18× cheaper. The bug was in my benchmark, and the real lesson was about trusting AI to verify itself.

Three examples from my AI-powered job board pipeline where the output looked right but wasn't.
Not all models are good at the same thing. Here's the data.
