Searching for Cognitively Diverse Tests: Towards Universal Test Diversity Metrics

Document type: Conference Papers
Peer reviewed: Yes
Author(s): Robert Feldt, Richard Torkar, Tony Gorschek, Wasif Afzal
Title: Searching for Cognitively Diverse Tests: Towards Universal Test Diversity Metrics
Conference name: 1st IEEE International Conference on Software Testing, Verification and Validation (ICST 2008)
Year: 2008
Pagination: 178 - 186
ISBN: 978-0-7695-3388-9
Publisher: IEEE
City: Lillehammer, Norge
URI/DOI: 10.1109/ICSTW.2008.36
Organization: Blekinge Institute of Technology
Department: School of Engineering - Dept. of Systems and Software Engineering (Sektionen för teknik – avd. för programvarusystem)
School of Engineering S- 372 25 Ronneby
+46 455 38 50 00
Language: English
Abstract: Search-based software testing (SBST) has shown a potential to decrease cost and increase quality of testingrelated software development activities. Research in SBST has so far mainly focused on the search for isolated tests that are optimal according to a fitness function that guides the search. In this paper we make the case for fitness functions that measure test fitness in relation to existing or previously found tests; a test is good if it is diverse from other tests. We present a model for test variability and propose the use of a theoretically optimal diversity metric at variation points in the model. We then describe how to apply a practically useful approximation to the theoretically optimal metric. The metric is simple and powerful and can be adapted to a multitude of different test diversity measurement scenarios. We present initial results from an experiment to compare how similar to human subjects, the metric can cluster a set of test cases. To carry out the experiment we have extended an existing framework for test automation
in an object-oriented, dynamic programming language.
Subject: Software Engineering\General