Launch · Core
Know what to do when something breaks
Things will go wrong in live service. Agree in advance how the team will spot, fix and communicate problems, and how you will learn from them without blame.
About 1-2 daysEvergreenLast reviewed 5 November 2026
Who this applies to
- Skills it builds
- Delivery and executionTechnical and AI fluency
- PM types
- All
- Product types
- All
- Starting points
- All
- Ways of working
- All
- Experience
- All
- Size of organisation
- All
Steps, by experience
Beginner (under 1 year)
- Agree what counts as an incident, and how serious each level is.
- Agree who is on call, how they are contacted and who leads when an incident happens.
- Decide who tells users and stakeholders, and prepare simple message templates.
- Make sure there is a way to roll back or switch off a feature quickly.
- After each serious incident, hold a blameless review: what happened, why, and what you will change.
Intermediate (1 to 4 years)
- Define incident severity levels and response times.
- Agree on-call, incident lead and communication roles.
- Check rollback and feature switch options.
- Run blameless post-incident reviews and track actions.
Experienced (5+ years)
- Rehearse incidents so the process works under pressure.
- Track incident trends and feed them into the roadmap.
- Agree public status page and customer communication rules.
- Build a culture where people report problems early without fear.
Who can do this
Owner: Product manager
Can be delegated to
- DevOps / SRE or Backend Engineer: Set up on-call, alerting and rollback
- Tech Lead: Lead technical response during incidents
- Content Designer: Write incident message templates
Reviewed by: Tech Lead, DevOps / SRE
Tool examples
- Jira
- Use an incident issue type with severity, timeline and actions.
- Confluence
- Use a post-incident review template and link actions to the backlog.
- GitHub
- Document rollback steps in the repository so anyone on call can follow them.
Definition of done
- Severity levels and roles agreed
- Rollback or feature switch available
- Communication templates ready
- Review process agreed
Common pitfalls
- Working it out during the first incident
- Blaming individuals, so people hide problems
- Fixing the symptom and not the cause
If you lead the team
What good looks like: Teams know in advance how to spot, fix and communicate problems, and learn from each incident.
Ask your PM
- Who's on call, and how do users hear about problems?
- What did we change after the last incident?
Warning signs
- Incidents are handled differently every time.
- Reviews look for someone to blame.