Q: What does a good eval dashboard look like for sharing with stakeholders?
I have rarely found dashboards to be helpful because they place the burden on the reader to figure out what’s important and don’t tend to be actionable. Stakeholders benefit much more from an explanation of what changed, possible root causes, and next steps.
Instead of a cookie-cutter dashboard, craft a narrative about what is important supported by custom data visualizations. For example, if your support agent has improved at resolving refund requests but still struggles with order cancellations, show specific visualizations of each trend along with possible root causes.
AI makes building custom data visualizations feasible, so there is less of an excuse to push every update into the same dashboard.
A dashboard might be ok if you find yourself presenting the same visual accross many updates. However, even in this case, custom visuals are often more effective.
This article is part of our AI Evals FAQ, a collection of common questions (and answers) about LLM evaluation. View all FAQs or return to the homepage.