Test · Add-on
Evaluate an AI model's quality before launch
AI features can give different answers to the same question and can be confidently wrong. Before launch, define what 'good enough' means and test the AI against a set of realistic examples, then keep testing after release.
About 3-5 daysFast-movingLast reviewed 4 November 2026
Who this applies to
- Skills it builds
- Technical and AI fluencyData and measurement
- PM types
- AI
- Product types
- Data or AI product
- Starting points
- All
- Ways of working
- All
- Experience
- All
- Size of organisation
- All
Steps, by experience
Beginner (under 1 year)
- Write down what the AI feature must do well, in measurable terms. For example: 'answers are accurate for at least 95% of common questions', 'never gives personal advice', 'replies within 3 seconds'.
- Collect a test set: 30 to 50 realistic examples to start, taken from real user questions where you are allowed to use them. Include easy cases, hard cases, edge cases and attempts to misuse it.
- For each example, write what a good answer looks like, or the rules a good answer must follow.
- Run the test set and score the results. Simple checks can be automated; others need a person, or another AI model, to grade against clear rules. Check a sample of any AI grading by hand.
- Check for harm and fairness: does it treat different user groups equally, does it make things up, and does it refuse when it should? Involve your legal or data protection colleagues.
- Agree a launch decision with the team based on the scores. Keep the test set and rerun it whenever the model, prompt or data changes, and keep monitoring after launch.
Intermediate (1 to 4 years)
- Define specific, measurable success criteria across accuracy, safety, tone, speed and cost.
- Build a test set of realistic, edge and adversarial cases with expected outcomes.
- Grade with a mix of code checks, model grading and human review, and validate the model grading by hand.
- Make a launch decision against thresholds, and rerun the evaluation as a regression check on every change.
Experienced (5+ years)
- Agree multi-dimensional success criteria with thresholds.
- Build a representative test set, including edge, misuse and fairness cases.
- Validate the grading approach and make results reproducible.
- Run the evaluation on every model or prompt change, and monitor live for drift and hallucination.
Who can do this
Owner: Product manager
Can be delegated to
- Data Scientist / ML Engineer or Tech Lead or Backend Engineer: Build and run the evaluation and grading
- QA / Tester: Write edge cases and misuse cases for the test set
- Policy / Legal / Compliance: Review for data protection, fairness and harm risks
- User Researcher: Test the AI feature with real users and capture where it confuses them
Reviewed by: Data Scientist / ML Engineer, Tech Lead, Policy / Legal / Compliance
Tool examples
- Claude
- Ask Claude to help draft success criteria and edge cases for your feature, then review every one yourself. Anthropic's docs on defining success criteria and building evaluations give a worked approach.
- GitHub
- Keep the test set and expected answers in the repository, so the evaluation can run automatically when the prompt or model changes.
- Confluence
- Keep an evaluation page with the success criteria, thresholds, the latest scores and the launch decision.
- Jira
- Raise each failing example as a bug linked to the AI feature, so fixes can be retested against the same case.
Definition of done
- Measurable success criteria and thresholds agreed
- Test set covers normal, edge and misuse cases
- Results scored, with any AI grading checked by a person
- Launch decision recorded, and evaluation set to rerun on every change
Common pitfalls
- Testing with a handful of hand-picked examples
- Trusting AI-graded scores without checking them
- Evaluating once before launch and never again
If you lead the team
What good looks like: AI quality is measured against agreed tests before launch.
Ask your PM
- What does good enough look like, and how did we measure it?
- How does the model do on difficult or sensitive cases?
Warning signs
- Quality is judged by a few demos.
- There's no agreed bar for launch.