WestminsterResearch Image Quality Evaluation in Lossy Compressed Images Allen, E.

309 Pages·2012·9.19 MB·English

WestminsterResearch
http://www.westminster.ac.uk/westminsterresearch

Image Quality Evaluation in Lossy Compressed Images
Allen, E.

This is an electronic version of a PhD thesis awarded by the University of Westminster.
© Ms Elizabeth Allen, 2017.
The WestminsterResearch online digital archive at the University of Westminster aims to make the
research output of the University available to a wider audience. Copyright and Moral Rights remain
with the authors and/or copyright owners.
Whilst further distribution of specific materials from within this archive is forbidden, you may freely
distribute the URL of WestminsterResearch: ((http://westminsterresearch.wmin.ac.uk/).
In case of abuse or copyright appearing without permission e-mail [email protected]
I Q E L
MAGE UALITY VALUATION IN OSSY
C I
OMPRESSED MAGES

Elizabeth Allen, BSc(Hons), MSc

A thesis submitted in partial fulfilment of the requirements of the
University of Westminster for the Degree of Doctor of Philosophy
This research programme was completed within the Imaging
Technology Research Group at the University of Westminster

February 2017

1
Abstract
This research focuses on the quantification of image quality in lossy
compressed images, exploring the impact of digital artefacts and scene
characteristics upon image quality evaluation.
A subjective paired comparison test was implemented to assess perceived
quality of JPEG 2000 against baseline JPEG over a range of different scene
types. Interval scales were generated for both algorithms, which indicated a
subjective preference for JPEG 2000, particularly at low bit rates, and these
were confirmed by an objective distortion measure. The subjective results did
not follow this trend for some scenes however, and both algorithms were
found to be scene dependent as a result of the artefacts produced at high
compression rates. The scene dependencies were explored from the interval
scale results, which allowed scenes to be grouped according to their
susceptibilities to each of the algorithms. Groupings were correlated with
scene measures applied in a linked study.
A pilot study was undertaken to explore perceptibility thresholds of JPEG 2000
of the same set of images. This work was developed with a further experiment
to investigate the thresholds of perceptibility and acceptability of higher
resolution JPEG 2000 compressed images. A set of images was captured using a
professional level full-frame Digital Single Lens Reflex camera, using a raw
workflow and carefully controlled image-processing pipeline. The scenes were
quantified using a set of simple scene metrics to classify them according to
whether they were average, higher than, or lower than average, for a number
of scene properties known to affect image compression and perceived image
quality; these were used to make a final selection of test images. Image fidelity
was investigated using the method of constant stimuli to quantify
perceptibility thresholds and just noticeable differences (JNDs) of
perceptibility. Thresholds and JNDs of acceptability were also quantified to
explore suprathreshold quality evaluation. The relationships between the two
thresholds were examined and correlated with the results from the scene
2
measures, to identify more or less susceptible scenes. It was found that the
level and differences between the two thresholds was an indicator of scene
dependency and could be predicted by certain types of scene characteristics.
A third study implemented the soft copy quality ruler as an alternative
psychophysical method, by matching the quality of compressed images to a set
of images varying in a single attribute, separated by known JND increments of
quality. The imaging chain and image processing workflow were evaluated
using objective measures of tone reproduction and spatial frequency response.
An alternative approach to the creation of ruler images was implemented and
tested, and the resulting quality rulers were used to evaluate a subset of the
images from the previous study. The quality ruler was found to be successful in
identifying scene susceptibilities and observer sensitivity.
The fourth investigation explored the implementation of four different image
quality metrics. These were the Modular Image Difference Metric, the
Structural Similarity Metric, The Multi-scale Structural Similarity Metric and
the Weighted Structural Similarity Metric. The metrics were tested against the
subjective results and all were found to have linear correlation in terms of
predictability of image quality.

3
Acknowledgements
I would like to extend sincere thanks to my supervision team for their support,
encouragement, and understanding and inspiration, during the time of my
research. Dr Sophie Triantaphillidou has been my friend, colleague and
supervisor, and has believed in, cajoled and encouraged me throughout.
Professor Ralph Jacobson has been my teacher, colleague, and mentor. Both
have helped to nurture my ideas and academic development for many years
and have provided their huge experience and expertise openly and generously.
I would also like to thank Dr Aleka Psarrou for her support, encouragement
and interest in the work of our research group.
I would further like to thank my colleagues, Dr Efthimia Bilissi and John Smith,
for the many hours of discussion and for their support in my research and their
contributions to the academic work and research of our team. There are many
other people within our department who have provided useful discussions,
assistance and guidance in technical issues, including Jae Young-Park who
deserves a special mention for all of his good-natured patience and help;
Gaurav Gupta for giving up laboratory space and assisting with MATLAB
implementation; Ana Tsifouti for discussion about methodology; Danny
Garside and James Pickett for technical support; the photographic technical
team and all the observers who have participated in my tests.
I also thank the many members of the graduate school for facilitating my
ongoing research, and to the Faculty Dean, Professor Kerstin Mey, for her
interest and support in the completion of my PhD and in the research of our
group.
Finally I want to thank my family: My parents, who have always supported me
in every way possible and in more ways than I could have appreciated; with
encouragement, but without expectation; to Charlotte and Ben for always
being there. Lastly I dedicate this to Jason, Anja and Freddie who provide their
love and appreciation no matter what, and who mean more to me than
anything.
4
Contents
1 Introduction ............................................................................................................ 20
1.1 Image Compression ................................................................................................... 20
1.2 Evaluating Compression .......................................................................................... 24
1.3 Aims and Overview of the Project ........................................................................ 25
2 Image Quality .......................................................................................................... 28
2.1 Image Quality Definitions ........................................................................................ 28
2.2 Factors Affecting Image Quality Evaluation ...................................................... 29
2.3 Objective and Subjective Image Quality Assessment .................................... 35
2.4 Distortion, Fidelity and Quality ............................................................................. 37
2.5 Physical and Perceptual Image Quality Attributes ........................................ 42
2.6 Objective measures of physical attributes ........................................................ 45
2.6.1 Tone Reproduction ......................................................................................................... 45
2.6.2 Colour Reproduction ...................................................................................................... 49
2.6.3 Resolution ........................................................................................................................... 59
2.6.4 Sharpness ............................................................................................................................ 62
2.6.5 Noise ..................................................................................................................................... 68
2.7 Modelling visual perception ................................................................................... 70
2.7.1 Contrast Sensitivity Function ..................................................................................... 72
2.7.2 Visual Masking .................................................................................................................. 74
2.7.3 Multichannel model of the HVS ................................................................................. 75
2.8 Types of Metrics .......................................................................................................... 75
2.8.1 Methods based on HVS models .................................................................................. 76
2.8.2 Methods based upon image structure ..................................................................... 77
2.8.3 Other Issues ....................................................................................................................... 78
2.9 Subjective Evaluation: Psychophysics ................................................................ 78
2.9.1 Types of Scales .................................................................................................................. 80
2.9.2 Scaling Methods ............................................................................................................... 81
2.9.3 Thresholds and Just Noticeable Differences ......................................................... 83
2.9.4 The Quality Ruler ............................................................................................................. 88
3 Image Compression and Image Quality ......................................................... 93
3.1 Redundancy in Images.............................................................................................. 93
3.2 Image structure and information content ......................................................... 94
5
3.3 Transform based Lossy Compression ................................................................. 97
3.3.1 JPEG Compression ........................................................................................................... 98
3.3.2 JPEG 2000 ......................................................................................................................... 102
3.4 Quantifying Distortion ........................................................................................... 106
3.5 Scene Dependency and Image Quality ............................................................. 109
3.5.1 Sources of Scene Dependency .................................................................................. 109
3.5.2 Scene Dependency and Compression.................................................................... 110
3.5.3 Scene selection ................................................................................................................ 112
3.6 Digital Imaging Artefacts ...................................................................................... 113
3.6.1 Variation of artefacts across an image .................................................................. 114
3.6.2 Visual saliency ................................................................................................................. 116
3.7 Scene Classification ................................................................................................ 117
4 Comparison of JPEG and JPEG 2000 ............................................................. 120
4.1 Background to the investigation ........................................................................ 120
4.2 JPEG and JPEG 2000 ................................................................................................ 121
4.3 Artefacts and Scene Dependency ....................................................................... 128
4.4 Subjective Image Quality Assessment .............................................................. 129
4.4.1 Test Images ...................................................................................................................... 130
4.4.2 Image Compression ...................................................................................................... 132
4.4.3 Display Characterisation ............................................................................................ 132
4.4.4 Psychophysical Display ............................................................................................... 133
4.4.5 Image Distortion ............................................................................................................ 134
4.5 Scene Characterisation .......................................................................................... 134
4.6 Interval Scale Generation ..................................................................................... 136
4.7 Results and Observations ..................................................................................... 138
4.8 Summary..................................................................................................................... 150
5 JPEG 2000 Thresholds ...................................................................................... 152
5.1 Perceptibility and Acceptability Thresholds ................................................. 152
5.2 Image Acquisition and Processing .................................................................... 155
5.2.1 Image Acquisition .......................................................................................................... 156
5.2.2 Image Processing ........................................................................................................... 157
5.3 Characterisation of Devices and Workflow ................................................... 159
5.3.1 Camera-Lens System Tone Reproduction ........................................................... 159
5.3.2 Camera-System-Image Processing Pipeline: Tone Reproduction ............. 162
5.3.3 Comparison of camera OECF before and after processing ........................... 165
6
5.3.4 Display Characterisation ............................................................................................ 168
5.4 Psychophysical Experiment................................................................................. 170
5.4.1 Quantification of Scene Characteristics ................................................................ 170
5.4.2 Scene Selection ............................................................................................................... 172
5.4.3 Image Compression ...................................................................................................... 173
5.4.4 Psychophysical Display and Viewing Conditions ............................................. 174
5.4.5 Paired Comparison Test.............................................................................................. 174
5.5 Determination of Thresholds .............................................................................. 176
5.5.1 Functional Form of the Psychometric Curve ...................................................... 176
5.5.2 Error estimation and goodness of fit of the psychometric curve .............. 177
5.6 Results ......................................................................................................................... 179
5.6.1 Scene ranking from objective measures .............................................................. 179
5.6.2 Perceptibility and acceptability thresholds ........................................................ 180
5.6.3 Grouping Scenes ............................................................................................................. 182
5.6.4 Correlations between thresholds and scene metrics ..................................... 183
5.6.5 Scene grouping and correlations ............................................................................ 184
5.7 Discussion .................................................................................................................. 186
5.7.1 Group 1 .............................................................................................................................. 186
5.7.2 Group II .............................................................................................................................. 187
5.7.3 Group III: ........................................................................................................................... 188
5.7.4 Group IV ............................................................................................................................. 189
5.7.5 Group V .............................................................................................................................. 190
5.7.6 Group VI ............................................................................................................................. 191
5.8 Summary..................................................................................................................... 191
6 Soft Copy Quality Ruler .................................................................................... 193
6.1 The Soft Copy Quality Ruler ................................................................................. 193
6.2 MTF Modification..................................................................................................... 194
6.3 Measurement of the system MTF ....................................................................... 197
6.3.1 Cascading the system MTF ........................................................................................ 197
6.3.2 SFR software and Test Target .................................................................................. 199
6.3.3 Image Acquisition .......................................................................................................... 201
6.3.4 Camera-Lens MTF ......................................................................................................... 202
6.3.5 Derivation of Display MTF ......................................................................................... 206
6.3.6 Camera-Lens-Processing MTF.................................................................................. 211
6.3.7 System MTF ...................................................................................................................... 213
6.4 Creation of Ruler Images ...................................................................................... 214
7
6.4.1 Determination of aim MTF and shaping function ............................................ 214
6.4.2 Development of the JND filters ................................................................................ 219
6.4.3 Application of the JND filters to the test images ............................................... 221
6.5 Psychophysical Investigation .............................................................................. 221
6.5.1 Psychophysical Display and Viewing Conditions ............................................. 221
6.5.2 Interface Design ............................................................................................................. 222
6.5.3 Observers and Test Images ....................................................................................... 223
6.6 Results and Discussion .......................................................................................... 225
6.6.1 Overall average, all scenes ......................................................................................... 225
6.6.2 Individual Scene Results Prior to Ruler Calibration ....................................... 227
6.6.3 Observer Sensitivity ..................................................................................................... 233
6.6.4 Calibration of the Rulers ............................................................................................. 234
6.7 Summary..................................................................................................................... 237
7 Image Quality Metrics ....................................................................................... 238
7.1 Selecting the metrics for use in this work ...................................................... 238
7.2 The Modular Image Difference Model ............................................................. 239
7.3 Implementation of MDIM in this work ............................................................ 241
7.3.1 Pre-processing: Colour space conversion ........................................................... 241
7.3.2 Application of CSFs and Spatial Frequency Adaptation................................. 242
7.3.3 Conversion to Non-Linear IPT space ..................................................................... 244
7.3.4 Edge enhancement ........................................................................................................ 244
7.3.5 Local contrast detection ............................................................................................. 247
7.3.6 Calculation of error metric ........................................................................................ 247
7.4 Structural Similarity Approaches to Image Quality .................................... 247
7.4.1 Structural Similarity Index Metric (SSIM) ........................................................... 249
7.4.2 Multi-scale Structural Similarity Index Metric (MSSIM) ............................... 251
7.4.3 The Three-Component Weighted SSIM (WSSIM) ............................................. 252
7.5 Results ......................................................................................................................... 254
7.6 Summary..................................................................................................................... 258
8 Discussions ........................................................................................................... 259
8.1 Quality Comparison of JPEG and JPEG 2000 .................................................. 259
8.1.1 Additional experimental work based upon the first image set .................. 263
8.1.2 Recent Research into JPEG, JPEG 2000 and other standards ...................... 264
8.2 Perceptibility and Acceptability of JPEG 2000 .............................................. 265
8.3 Soft Copy Quality Ruler ......................................................................................... 267
8
8.4 Image Quality Metrics ............................................................................................ 270
8.5 General Comments .................................................................................................. 274
9 Conclusions and Further Work ..................................................................... 276
9.1 Conclusions ................................................................................................................ 276
9.2 Recommendations for Future Work ................................................................. 278
10 Related Work ....................................................................................................... 280
10.1 List of publications (Primary Author) ........................................................... 280
10.2 Presentations at Conferences and Symposia .............................................. 280
10.3 Awards ...................................................................................................................... 281
10.4 Related Publications (not primary author) ................................................ 281
References ................................................................................................................... 282
Appendix A Test Images Chapter 5 .......................................................................... 295
Appendix B Display Characterisation ..................................................................... 297
Appendix C Polynomial Functions for JND filters ............................................... 303
Appendix D Observer Instructions for the SCQR ................................................. 304
Appendix E Colour Space Transformations .......................................................... 305
9