Long ago, I had the opportunity to work with a team that was really adamant about not using metrics. As I explored their reasons, it became evident that they were uncomfortable with the visibility. A few more wanted to ‘hold up’ on any metrics as well, but for a different reason. They didn’t want to standardize on anything that would be ‘gamed’. This prompted quite a few conversations about what we were trying to accomplish. If I have discovered anything, people can quickly become defensive around a metric. At this point I start wondering about the environment and the visibility, trust and openness we need for agile teams to thrive. This is my list to keep mindful, as I use and even explain metrics.
1. Not the finish-line, but the start. Any metric is not a goal; it is a side-effect. Treated as a primary goal it can probably be cheated or gamed. As a side-effect it simply is a characteristic of the work done. Doing something right has its own characteristics. Reminding the program, pointing the focus of the teams towards something to improve upon is usually the gold coin of the scrum master or coach. If we are not measuring without bias, it is often difficult to be thought full and ask ourselves a few more questions. When the results are empirical they may not always be intuitive. Especially at an organizational level. Before we even begin some may demand a target. Before giving a reasonable target it may be wiser to spend some time characterizing our current ability.
2. Are You A Pirate? Are we fearful, do we know the person monitoring compliance. So much concern for metrics revolves around how it can be used against me. No one should be looking to play hangman based on other’s fear of accusation. Nor should this be received as anything but information. I ever prefer the teams to be in a problem solving position rather than in a defensive posture. So how do you know if you’ve really improved if you don’t bring in some standards or a backstop to help prevent your teams from sliding? Metrics can be useful to identify, explain, question and problem solve. These metrics REALLY are for all of us. You are the person, as part of a team, program and product that is trying to continuously improve AND using the metrics. As much information as we have at the time to make the right decisions. Don’t just give yourself a pass, or even beat yourself up, just do something small to improve it.
3. Fast and Light Sailing. Is this an automated data collection? I get leery about anything manual unless it is very light weight! This is intended to be a side effect, unnoticed as we go. Large manual data collection is a pause and requires effort from someone every time. Tools often have features to integrate and import/export data in many formats. John Wooden tried never to mistake activity for achievement, and he certainly knew that if you didn’t have time to do it right, you should ask yourself when you will have time to do it over.
4. No Metric Is An Island. A metric hardly stands alone. It may be part of a larger picture that can indirectly be addressed. What questions might this metric answer? Why is it important? Details like this on the report itself often helps teach others. Is the information actionable? What is acceptable? Do I have enough information to improve it? Were there exceptions? Can the metric be corroborated, or correlated? Does it prove out what initial assumptions may have been? How does the behavior it should promote, affect other metrics? e.g. What about the average number of days to address bugs by severity; which seems much more useful in assessing our capabilities than a current bug count. Could we ever measure that in hours? Lastly, how do we balance the impact and prioritize these initiatives?
5. Buried Is Not Treasured. Make this visible to everyone in the company in 1 or 2 clicks. Find a place for it. Scrum is all about being visible to encourage feedback for improvement and a convergence that draws alignment. Being visible also includes being in a format that is intuitive. Many times, even in some task boards, information is so disorganized as to be buried. If I look for 5 seconds and then close my eyes, do I know and understand what was trying to be conveyed to me. Although the measurement may be empirical, the results should be fairly forward to understand. The teams should also be used to these metrics, know they are supported, and fearless about having everyone’s help.
6. Level Up. Take the time periodically to review and update the metrics based on the behaviors, and improvements you want in your organization. What level is it appropriate for… the team, the program, or the organization? How mature are we? Once a quarter we should ask of ourselves if this metric has been useful. Perhaps other metrics might actually promote better program behaviors. Iterate and update to a different set of metrics if you have to. Scrum is a continuous inspect and adapt. Are we improving? Which way is the needle trending?