/ Article
A short argument for treating evaluation harnesses as part of the product — not an afterthought once demos look good.
Demos lie. Evals don't — or at least they lie less.
Evals are part of the MVP
If an AI feature can't be checked against a small suite of cases you care about, you haven't shipped a product. You've shipped a performance.
I now treat the eval harness as part of the MVP, not a polish item after launch.