Quantifying the Impact of Learning Algorithm Parameter Tuning (short version)

Document type: Conference Papers
Peer reviewed: Yes
Author(s): Niklas Lavesson, Paul Davidsson
Title: Quantifying the Impact of Learning Algorithm Parameter Tuning (short version)
Conference name: 3rd Joint SAIS/SLSS Workshop
Year: 2005
Pagination: 107-113
Publisher: Mälardalen University
City: Västerås
Organization: Blekinge Institute of Technology
Department: School of Engineering - Dept. Mathematics and Science (*** Error ***)
School of Engineering S- 371 79 Karlskrona
+46 455 38 50 00
Authors e-mail: Niklas.Lavesson@bth.se, Paul.Davidsson@bth.se
Language: English
Abstract: The impact of learning algorithm optimization by means of parameter tuning is studied. To do this, two quality attributes, sensitivity and classification performance, are investigated,
and two metrics for quantifying each of these attributes are suggested. Using these metrics, a systematic comparison has been performed between four induction algorithms on eight data sets. The results indicate that parameter tuning is often more important than the choice of algorithm and there does not seem to be a trade-off between the two quality attributes.
Moreover, the study provides quantitative support to the assertion that some algorithms are more robust than others with respect to parameter configuration. Finally, it is briefly described how the quality attributes and their metrics could be used for algorithm selection in a systematic way.
Subject: Computer Science\Artificial Intelligence