A Method for Evaluating Rigor and Industrial Relevance of Technology Evaluations
|Document type:||Journal Articles|
|Article type:||Original article|
|Author(s):||Martin Ivarsson, Tony Gorschek|
|Title:||A Method for Evaluating Rigor and Industrial Relevance of Technology Evaluations|
|Journal:||Empirical Software Engineering|
|Organization:||Blekinge Institute of Technology|
|Department:||School of Computing (Sektionen för datavetenskap och kommunikation)
School of Computing S-371 79 Karlskrona
+46 455 38 50 00
|Abstract:||One of the main goals of an applied research field such as software engineering is the transfer and widespread use of research results in industry. To impact industry, researchers developing technologies in academia need to provide tangible evidence of the advantages of using them. This can be done trough step-wise validation, enabling researchers to gradually test and evaluate technologies to finally try them in real settings with real users and applications. The evidence obtained, together with detailed information
on how the validation was conducted, offers rich decision support material for industry practitioners seeking to adopt new technologies and researchers looking for an empirical
basis on which to build new or refined technologies. This paper presents model for evaluating the rigor and industrial relevance of technology evaluations in software engineering. The model is applied and validated in a comprehensive systematic literature review of evaluations of requirements engineering technologies published in software engineering journals. The aim is to show the applicability of the model and to characterize how evaluations are carried out and reported to evaluate the state-of-research. The review shows that the model can be applied to characterize evaluations in requirements engineering. The findings from applying the model also show that the majority of technology evaluations in requirements engineering lack both industrial relevance and rigor.
In addition, the research field does not show any improvements in terms of industrial relevance over time.
|Keywords:||research methodology, empirical, case study, requirements engineering|