Evals: Hard-to-evaluate products
For products whose outputs are difficult to judge, even for an expert.
Overview
Start by reading “It’s Hard to Eval” Is a Product Smell to see how product design can make difficult outputs easier to verify.
Watch How to Eval an AI Product That Seems Impossible to Eval for a discussion of what users need to verify. Evals for AI Writing covers a subjective use case.