The Accuracy of Fault Prediction in Modified Code –Statistical Model vs. Expert Estimation

Document type: Conference Papers
Peer reviewed: Yes
Author(s): Piotr Tomaszewski, Jim Håkansson, Lars Lundberg, Håkan Grahn
Title: The Accuracy of Fault Prediction in Modified Code –Statistical Model vs. Expert Estimation
Conference name: 13th Annual IEEE International Conference on the Engineering of Computer Based Systems (ECBS 2006)
Year: 2006
Pagination: 334-343
ISBN: 0-7695-2546-6
City: Potsdam, Germany
ISI number: 000237197200035
Organization: Blekinge Institute of Technology
Department: School of Engineering - Dept. of Systems and Software Engineering (Sektionen för teknik – avd. för programvarusystem)
School of Engineering S- 372 25 Ronneby
+46 455 38 50 00
Language: English
Abstract: Fault prediction models still seem to be more popular in academia than in industry. In industry expert estimations of fault proneness are the most popular methods of deciding where to focus the fault defection efforts. In this paper we present a study in which we empirically evaluate the accuracy of fault prediction offered by statistical models as compared to expert estimations. The study is industry based It involves a large telecommunication system and experts that were involved in the development of this system. Expert estimations are compared to simple prediction models built on another large system, also from the telecommunication domain. We show that the statistical methods clearly outperform the expert estimations. As the main reason for the superiority of the statistical models we see their ability to cope with large datasets, which results in their ability to perform reliable predictions for larger number of components in the system, as well as the ability to perform prediction at a more fine-grain level, e.g., at the class instead of at the component level.
Subject: Computer Science\Computersystems