Metrics tell us if a good job is being done or not and when things start to deteriorate or improve. They act as a yardstick to demonstrate improvements – or at least to ensure we are not regressing.
Without Metrics, we don’t really know:
- Where we are
- Where we’re going
- Where we’ve been
The total cost of the IT Department is a Metric, as is the number of faults in the last release, the average severity of faults found after a product has been released, the number of pages or words in documents that have to be reviewed and so on.
However, whilst any data item can be a Metric this does not mean that every Metric should be collected for any particular project (as some Metrics may not be useful within the context of the project/organisation).
Metrics should always:
- Be owned: Ensure a person/function is given responsibility to ensure they are collected and managed as required.
- Be collectable: It is all very well to be concerned about the complexity of each code element of a software release as a measure of quality, but if no one in the organisation has access to the source code, then there is no way of knowing what the complexity is.
- Be inexpensive to collect: If the collection and provision of Metrics is too expensive, then the cost will outweigh the benefits, and questions regarding effectiveness and efficiency will be asked.
- Be useful: A good Metric not only contains the right information, but it also reports that information at the right time to the right person in the right way. For this reason, Metrics should be regularly reviewed to ensure that they are still fit for purpose. Most Metrics develop and change with the differing focus of the organisation and the industry as time progresses.
- Be reported on: A good Metric is clear and concise, enabling regular reporting which is easily comprehensible. A Metric that is delivered every day, but which is ignored, is adding no value and a waste of effort to generate.
- Be kept safe: How can anyone tell if a software project is better than a previous one if there aren’t any historical Metrics figures to review against? Therefore, all Metrics should be kept for a period of time usually determined by what the Metric is, and its relative importance.
Implementing a Metrics programme can appear to be a daunting task, but the benefits of a well implemented programme of measurement (gathering and managing information that enables efficient monitoring, control and reporting of every stage of a software project) always makes it worthwhile.
So, how can an organisation instigate a Metrics programme? A good framework would be to utilise the “Goal, Question, Measurement” (GQM) approach. GQM will also:
- Clarify what the goals or objectives are
- Decide what questions (answered positively) will tell us we have achieved that goal or objective
- Identify what measurement scales, raw data to be used (with thresholds) to provide numeric, accurate information to answer questions
This approach will document the what, why and how of your Metrics Programme. Implementation will also:
- Identify what Metric reporting requirements exist
- Identify what data is already available within an organisation
- Identify the best way to collect and store that information (in a organisation-relevant fashion)
- Develop reporting mechanisms and analysis tools that provide ongoing measurement of effective delivery (and which facilitate the organisational objective to deliver the right information to the right people at the right time)
- Perform training and mentoring in collecting, storing, analysing and using the Metrics. This element should not be underestimated, as local Metrics Administrators need to ensure that the Metrics programme works, is used, and provides real ongoing benefit to the organisation.
Nathan Weller is a senior consultant at Experimentus