Cost of Quality (COQ)
Cost of Quality (COQ)
“Quality is free, but only to those who are willing to pay heavily for it” (Tom DeMarco, again).
In a previous article I mentioned two ways of demonstrating the value of testing to the business and discussed one of them: Return on Investment (ROI). The other is a more complete way to quantify the value and efficiency of the quality-related work that goes into software delivery, including but not limited to testing, and is called the Cost of Quality (COQ). It works by classifying all the money actually spent on preventing and fixing defects into four categories, the sum of which gives the total COQ.
-
Prevention costs include training for analysts / designers / developers etc. and process improvements in any of these build / implementation disciplines.
-
Detection costs include reviewing specifications, writing tests, configuring test environments and running each test once;
-
also includes training and process improvements in the testing function.
-
-
Internal failure costs include reporting, investigation, fixing, confirmation testing and regression testing for all defects found before going live.
-
External failure costs include all costs associated with defects found by users / customers after going live;
-
including all indirect costs such as those mentioned as examples in Well, does it? above, in so far as they can be identified or estimated.
-
Part of our testing budget covers the Detection costs: this is the money that will have been spent on static and dynamic testing even if we find no defects. The remainder of our testing budget, that is, our contribution to reporting and dealing with the defects that we find, is part of the Internal failure costs; the other component of the Internal failure costs is the money spent by developers and others on investigating and fixing the defects that we found (‘debugging’ costs).
One feature of COQ is that Prevention costs bring into focus the performance of other groups involved in delivering a quality product, not merely the testers. It’s an old engineering idiom that quality can’t be tested into a product, it has to be built into it; if yours is one of the [far too many] teams that seem to be expected to test it in, this might be useful.
So, how can we use COQ? In three ways: for an improved ROI calculation, and for predicting and then measuring the effects of process improvement.
-
The sum of Prevention, Detection and Internal failure costs can be put into the ROI calculation described previously, in place of “The cost of the testing and fixing that found and removed those bugs”, to give a more realistic ROI (it will be slightly lower if there are any Prevention costs).
-
If it‘s measured on a regular basis (say, half-yearly) across the entire IT product delivery portfolio it will show whether overall quality-related efficiency is improving or not. If any specific quality improvement intiatives have been made in the period being measured, the change in COQ from the last period can be compared to the business case that was made for them to show whether they have had the desired effect.
-
A business case for a proposed improvement can be based on modelling the COQ changes expected: add the extra Prevention costs and apply the hoped-for decrease in the Internal and External failure costs (remember, the Detection costs won‘t change unless it‘s a test process improvement, in which case the extra cost goes there instead of into Prevention).
The biggest challenge lies in collecting data that are sufficiently accurate to be credible. COQ is based on actuals, not on estimates. An essential pre-requisite is a time recording system that will allow time spent to be recorded against the individual bug that was being reported / fixed / retested / worked on in any other way and that people use it conscientiously.
This article is a part of a series. You can check the next one right now.
Author: Richard Taylor
Richard has dedicated more than 40 years of his professional career to IT business. He has been involved in programming, systems analysis and business analysis. Since 1992 he has specialised in test management. He was one of the first members of ISEB (Information Systems Examination Board). At present he is actively involved in the activities of the ISTQB (International Software Testing Qualifications Board), where he mainly contributes with numerous improvements to training materials. Richard is also a very popular lecturer at our training sessions and a regular speaker at international conferences.