Any phase · Craft
Understand how AI products work and fail
AI is now part of most products. PMs need to understand what AI does well, where it fails, how it is evaluated and what risks it brings, whether or not they build AI features.
About 3-5 daysFast-movingLast reviewed 5 November 2026
Who this applies to
- Skills it builds
- Technical and AI fluency
- PM types
- All
- Product types
- All
- Starting points
- All
- Ways of working
- All
- Experience
- All
- Size of organisation
- All
Steps, by experience
Beginner (under 1 year)
- Learn the basics: AI models learn patterns from data and make predictions or generate content. They can be wrong with confidence.
- Use AI tools in your own work and notice where they help and where they fail.
- Learn what an evaluation is: a set of test cases that checks how well an AI performs.
- Learn the main risks: wrong answers, bias, privacy, security and over-reliance.
- Ask your team or organisation how AI is used and governed today.
Intermediate (1 to 4 years)
- Understand model choice, prompting, retrieval and fine-tuning at a high level.
- Know how to define success criteria and evaluations.
- Know the main risks and controls.
Experienced (5+ years)
- Make build, buy or partner decisions for AI capabilities.
- Design human oversight and fallback into AI features.
- Track cost, quality and risk of AI in production.
Who can do this
Owner: Product manager
Can be delegated to
- Data Scientist / ML Engineer: Explain how your team's AI features work
Reviewed by: Data Scientist / ML Engineer
Tool examples
- Claude
- Ask it to explain a term, then test its answer against official documentation.
- ChatGPT
- Compare answers between AI tools to see how they differ.
- Gemini
- Try the same task in several tools and note the failure patterns.
Definition of done
- You can explain how AI products work and fail
- You understand evaluations
- You know the main risks
Common pitfalls
- Treating AI output as fact
- Adding AI because it is fashionable
- Ignoring cost and risk