This paper attempts to estimate the quality of a document or piece of source code after a software inspection has occurred. It uses two estimate techniques; subjective in which the reviewers estimate the number of faults remaining after they have completed their inspection, and capture-recapture in which the number of faults is estimated based on the overlap of defects found by individuals reviewing the same piece of code. In capture-recapture, a high overlap indicates that not many defects remain.
The documents being inspected were requirements and design documents for a taxi management system. Of note is that the Blekinge students recently developed a taxi management system for a previous course.
During the experiment, subjective estimates were taken at different intervals (before and after meetings or inspections). The capture-recapture estimates were only calculated after the inspections took place. In the documents used, the exact number of faults was known so that the predictions could be compared for accuracy.
The results show that capture-recapture estimates are more accurate at predicting the number of remaining faults in the document or system. However, the Blekinge students had much fewer false positives (possibly due to the previous course in which they built a taxi management system), and thus their subjective estimates were much lower (and accurate). This indicates that domain knowledge may play a large role in determining the accuracy of subjective estimates, thus challenging the validity of this studies findings. Adding to this, the authors note several other tests in which the opposite result is obtained (subjective estimates are more accurate). Thus, the authors plan to conduct an industry case study to further validate their claims.