Defining What Good Looks Like
AI evals feel like a brand-new skill — but the hardest part is a question product teams have always had to answer: what does "good" actually look like?
Petra Wille and Teresa Torres dig into Teresa's recent article on AI evals and the one idea in it that hasn't changed at all. Teresa argues that measuring the quality of AI outputs is really the same work as defining outcomes, writing a PRD, or instrumenting a product — and that teams keep repeating an old mistake: measuring technical performance (uptime, speed, "was it concise?") instead of whether they actually created value...