Searching for Cognitively Diverse Tests: Towards Universal Test Diversity Metrics
| Document type: | Conference Papers |
|---|---|
| Peer reviewed: | Yes |
| Author(s): | Robert Feldt, Richard Torkar, Tony Gorschek, Wasif Afzal |
| Title: | Searching for Cognitively Diverse Tests: Towards Universal Test Diversity Metrics |
| Conference name: | 1st IEEE International Conference on Software Testing, Verification and Validation (ICST 2008) |
| Year: | 2008 |
| Pagination: | 178 - 186 |
| ISBN: | 978-0-7695-3388-9 |
| Publisher: | IEEE |
| City: | Lillehammer, Norge |
| URI/DOI: | 10.1109/ICSTW.2008.36 |
| Organization: | Blekinge Institute of Technology |
| Department: | School of Engineering - Dept. of Systems and Software Engineering (Sektionen för teknik – avd. för programvarusystem) School of Engineering S- 372 25 Ronneby +46 455 38 50 00 http://www.tek.bth.se/ |
| Language: | English |
| Abstract: | Search-based software testing (SBST) has shown a potential to decrease cost and increase quality of testingrelated software development activities. Research in SBST has so far mainly focused on the search for isolated tests that are optimal according to a fitness function that guides the search. In this paper we make the case for fitness functions that measure test fitness in relation to existing or previously found tests; a test is good if it is diverse from other tests. We present a model for test variability and propose the use of a theoretically optimal diversity metric at variation points in the model. We then describe how to apply a practically useful approximation to the theoretically optimal metric. The metric is simple and powerful and can be adapted to a multitude of different test diversity measurement scenarios. We present initial results from an experiment to compare how similar to human subjects, the metric can cluster a set of test cases. To carry out the experiment we have extended an existing framework for test automation in an object-oriented, dynamic programming language. |
| Subject: | Software Engineering\General |
