Human Observer Confidence in Image Quality Assessment

Document type: Journal Articles
Article type: Original article
Peer reviewed: Yes
Author(s): Ulrich Engelke, Anthony Maeder, Hans-Jürgen Zepernick
Title: Human Observer Confidence in Image Quality Assessment
Journal: Signal Processing: Image Communication
Year: 2012
Volume: 27
Issue: 9
Pagination: 935-947
ISSN: 0923-5965
Publisher: Elsevier
URI/DOI: 10.1016/j.image.2012.07.007
ISI number: 000310480100001
Organization: Blekinge Institute of Technology
Department: School of Computing (Sektionen för datavetenskap och kommunikation)
School of Computing S-371 79 Karlskrona
+46 455 38 50 00
Language: English
Abstract: Mean opinion scores obtained from subjective quality assessment are widely used as a ground truth for the development of predictive quality models. The underlying variance between observer ratings is typically quantified using confidence intervals, which do not provide any direct insight into the underlying causes of the disagreement. For better understanding of human visual quality perception and to develop more accurate models, it is important to identify the factors that impact on the variations in quality ratings. This work considers one such factor: observer confidence. This consideration is motivated by the view that quality assessment is a difficult task and hence quality ratings are provided with varying levels of confidence. The first goal of this paper is to analyse the results of an experiment to determine association between observer confidence and image quality judgement. Secondly, models are developed that aim to predict mean observer confidence as a complementary measure to the widely used mean opinion scores. It is shown that there is indeed a strong interrelation between quality perception and confidence, resulting in predictive models of high accuracy.
Subject: Telecommunications\General
Keywords: Image quality; Observer confidence; Psychophysical experiment; Reaction time