Monitoring and Implementing Early and Cost-Effective Software Fault Detection

Document type: Licentiates
Full text:
Author(s): Lars-Ola Damm
Title: Monitoring and Implementing Early and Cost-Effective Software Fault Detection
Translated title: Övervakning och implementation av tidig och kostnadseffektiv feldetektering
Series: Blekinge Institute of Technology Licentiate Series
Year: 2005
Issue: 2
Pagination: 154
ISBN: 91-7295-056-0
ISSN: 1650-2140
Publisher: Blekinge Institute of Technology
City: Karlskrona
Organization: Blekinge Institute of Technology
Department: School of Engineering - Dept. of Systems and Software Engineering (Sektionen för teknik – avd. för programvarusystem)
School of Engineering S- 372 25 Ronneby
+46 455 38 50 00
http://www.tek.bth.se/
Authors e-mail: lars-ola.damm@bth.se
Language: English
Abstract: Avoidable rework constitutes a large part of development projects, i.e. 20-80 percent depending on the maturity of the organization and the complexity of the products. High amounts of avoidable rework commonly occur when having many faults left to correct in late stages of a project. In fact, research studies indicate that the cost of rework could be decreased by up to 30-50 percent by finding more faults earlier. However, since larger software systems have an almost infinite number of usage scenarios, trying to find most faults early through for example formal specifications and extensive inspections is very time-consuming. Therefore, such an approach is not cost-effective in products that do not have extremely high quality requirements. For example, in market-driven development, time-to-market is at least as important as quality. Further, some areas such as hardware dependent aspects of a product might not be possible to verify early through for example code reviews or unit tests. Therefore, in such environments, rework reduction is primarily about finding faults earlier to the extent it is cost-effective, i.e. find the right faults in the right phase.

Through a set of case studies at a department at Ericsson AB, this thesis investigates how to achieve early and cost-effective fault detection through improvements in the test process. The case studies include investigations on how to identify which improvements that are most beneficial to implement, possible solutions to the identified improvement areas, and approaches for how to follow-up implemented improvements.

The contributions of the thesis include a framework for component-level test automation and test-driven development. Additionally, the thesis provides methods for how to use fault statistics for identifying and monitoring test process improvements. In particular, we present results from applying methods that can quantify unnecessary fault costs and pinpointing which phases and activities to focus improvements on in order to achieve earlier and more cost-effective fault detection. The goal of the methods is to make organizations strive towards finding the right fault in the right test phase, which commonly is in early test phases. The developed methods were also used for evaluating the results of implementing the above-mentioned test framework at Ericsson AB. Finally, the thesis demonstrates how the implementation of such improvements can be continuously monitored to obtain rapid feedback on the status of defined goals. This was achieved through enhancements of previously applied fault analysis methods.
Summary in Swedish: Avhandlingen handlar om hur en mjukvaruutvecklingsorganisation kan hitta fel tidigare i utvecklingsprocessen. Fokus ligger på att hitta rätt fel i rätt fas, d.v.s. när det är som mest kostnadseffektivt. Avhandlingen presenterar en samling fallstudier utförda inom detta området på Ericsson AB. Nyckelord: processförbättring, felanalys, tidig feldetektering
Subject: Software Engineering\General
Keywords: Early fault detection, software process improvement, fault analysis
URN: urn:nbn:se:bth-00297
Edit