Retrospectives With Scientific Validation

During retrospectives, agile teams would benefit from following the Lean Startup concept of testing their improvement hypotheses using scientific methods. My experience during agile retrospectives has been that for most 'improvements', there's no deliberate validation that the change actually makes the hoped for improvement. The trick is to select quantifiable measures that can be directly linked to the 'improvement' i.e., cause and effect are firmly linked. In essence, the Scrum team conducts a scientific experiment to determine if for some action/activity/change they get a measurable improvement.

It's important that the team only do one improvement at a time so that any effects can be attributed to the improvement. For example, if you're introducing pair-programming to reduce acceptance tests failures but you've also introduced a new test harness to help increase test coverage, you might find it difficult or impossible to determine which of these two affected acceptance test results and by how much.

The retrospective process steps:

1. Problem Statement: Clearly state the exact problem the improvement will solve
2. Desired Outcome: Clearly state what the results will be with the improvement in place
3. Experiment Summary: Develop the falsifiable* hypothesis for the improvement
4. Measures: Identify the metrics and measures to collect
5. Data Collection: Identify how the metric data are collected. Identify how the collected data is presented
6. Results Validation: Identify how the team will know the hypothesis is proven true beyond a reasonable doubt. Identify how the team will know the hypothesis is proven false beyond a reasonable doubt

The information gathered above is the output of the retrospective. The team should place the falsifiable hypothesis for improvement [retrospective experiment] in a prominent location so everyone in the organization can understand the problem the team hopes to solve, what the experiment is to solve it, and the hoped for results of the improvement.

*A falsifiable hypothesis is a statement that can be proven wrong through observation or experiment. The structure of a falsifiable hypothesis is, "The team will [make this improvement] by [doing/not doing this action or activity]."

Tweet