It's important that the team only do one improvement at a time so that any effects can be attributed to the improvement. For example, if you're introducing pair-programming to reduce acceptance tests failures but you've also introduced a new test harness to help increase test coverage, you might find it difficult or impossible to determine which of these two affected acceptance test results and by how much.
The retrospective process steps:
- Problem Statement: Clearly state the exact problem the improvement will solve
- Desired Outcome: Clearly state what the results will be with the improvement in place
- Experiment Summary: Develop the falsifiable* hypothesis for the improvement
- Measures: Identify the metrics and measures to collect
- Data Collection: Identify how the metric data are collected. Identify how the collected data is presented
- Results Validation: Identify how the team will know the hypothesis is proven true beyond a reasonable doubt. Identify how the team will know the hypothesis is proven false beyond a reasonable doubt
The information gathered above is the output of the retrospective. The team should place the falsifiable hypothesis for improvement [retrospective experiment] in a prominent location so everyone in the organization can understand the problem the team hopes to solve, what the experiment is to solve it, and the hoped for results of the improvement.
*A falsifiable hypothesis is a statement that can be proven wrong through observation or experiment. The structure of a falsifiable hypothesis is, "The team will [make this improvement] by [doing/not doing this action or activity]."