Measure and improve · Core
Measure results against your hypothesis
Shipping is not the finish line. Check whether the change did what your hypothesis predicted, learn why, and use that to decide what to do next.
About 2-3 daysEvergreenLast reviewed 4 November 2026
Who this applies to
- Skills it builds
- Data and measurement
- PM types
- All
- Product types
- All
- Starting points
- All
- Ways of working
- All
- Experience
- All
- Size of organisation
- All
Steps, by experience
Beginner (under 1 year)
- Go back to your hypothesis and the measures of success you agreed. Write down the baseline (before) for each one.
- Check the data is being collected properly. Ask your analyst or engineers to confirm the tracking works before you trust any numbers.
- Wait long enough to get a fair picture: at least a full week, so weekday and weekend use are both included, and longer for low-traffic services.
- Compare the results with the baseline. Look at different user groups separately, because an average can hide a group that got worse.
- Pair the numbers with research. Numbers tell you what changed; talking to users or reading feedback tells you why.
- Write a short summary: was the hypothesis right, partly right or wrong? What did you learn? What will you do next: keep it, improve it or remove it? Share it with the team and stakeholders.
Intermediate (1 to 4 years)
- Confirm tracking works and baselines are recorded.
- Measure over a fair period, broken down by user group.
- Combine the numbers with qualitative feedback to understand why.
- Publish a verdict on the hypothesis and the next decision.
Experienced (5+ years)
- Verify tracking, and define baselines and segments.
- Compare results with the hypothesis over a fair period.
- Combine quantitative and qualitative evidence.
- Share the verdict, learning and next decision, and update the backlog.
Who can do this
Owner: Product manager
Can be delegated to
- Data Analyst or Performance Analyst: Build the dashboard and report results against the baseline
- User Researcher: Run follow-up research to explain the numbers
- Frontend Engineer or Tech Lead or Backend Engineer: Check analytics events fire correctly
Reviewed by: Tech Lead, Delivery Manager
Tool examples
- Google Analytics 4
- Set the key user actions as key events, then compare the period before and after release in the reports, broken down by device and user type.
- Amplitude
- Build a funnel for the journey your hypothesis targets and compare conversion before and after the release date.
- Mixpanel
- Use a funnel report and segment it by user group to see whether any group got worse.
- Power BI
- Put the hypothesis measures and their baselines on one page, so the team can see progress at a glance.
- Looker
- Create a saved view of the hypothesis measures with a release-date marker.
- Claude
- Paste your results table (no personal data) and ask what other explanations could account for the change besides your feature.
Definition of done
- Results compared with the baseline over a fair period
- Results checked by user group, not just overall
- Clear verdict on the hypothesis, with the reasons
- Next decision agreed and the backlog updated
Common pitfalls
- Declaring success after a day or two of data
- Assuming the change caused the result without ruling out other causes
- Only looking at averages
If you lead the team
What good looks like: Teams check whether each change did what they predicted, and learn why.
Ask your PM
- Did this change move the metric we expected?
- What's our next step based on what we learned?
Warning signs
- Shipping is treated as success.
- Results are explained away.