Evaluation and Measurement of Software Process Improvement - A Systematic Literature Review

Document type: Journal Articles
Article type: Original article
Peer reviewed: Yes
Full text:
Author(s): Michael Unterkalmsteiner, Tony Gorschek, A.K.M. Moinul Islam, Chow Kian Cheng, Rahadian Bayu Permadi, Robert Feldt
Title: Evaluation and Measurement of Software Process Improvement - A Systematic Literature Review
Journal: IEEE Transactions on Software Engineering
Year: 2012
Volume: 38
Issue: 2
Pagination: 398 - 424
ISSN: 0098-5589
Publisher: IEEE
URI/DOI: 10.1109/TSE.2011.26
ISI number: 000301915200010
Organization: Blekinge Institute of Technology
Department: School of Computing (Sektionen för datavetenskap och kommunikation)
School of Computing S-371 79 Karlskrona
+46 455 38 50 00
Authors e-mail: tony.gorschek@bth.se
Language: English
Abstract: Software Process Improvement (SPI) is a systematic approach to increase the efficiency and effectiveness of a software development organization and to enhance software products. OBJECTIVE—This paper aims to identify and characterize evaluation strategies and measurements used to assess the impact of different SPI initiatives. METHOD--The systematic literature review includes 148 papers published between 1991 and 2008. The selected papers were classified according to SPI initiative, applied evaluation strategies and measurement perspectives. Potential confounding factors interfering with the evaluation of the improvement effort were assessed. RESULTS--Seven distinct evaluation strategies were identified, whereas the most common one, "Pre-Post Comparison", was applied in 49% of the inspected papers. Quality was the most measured attribute (62%), followed by Cost (41%) and Schedule (18%). Looking at measurement perspectives, "Project" represents the majority with 66%. CONCLUSION—The evaluation validity of SPI initiatives is challenged by the scarce consideration of potential confounding factors, particularly given that "Pre-Post Comparison" was identified as the most common evaluation strategy, and the inaccurate descriptions of the evaluation context. Measurements to assess the short and mid-term impact of SPI initiatives prevail, whereas long-term measurements in terms of customer satisfaction and return on investment tend to be less used.
Subject: Software Engineering\General
Keywords: Process implementation and change, Process measurement, Metrics/Measurement, Systematic Literature Review
Note: http://www.computer.org/portal/web/csdl/doi/10.1109/TSE.2011.26