If there is one thing that all engineers and scientists can appreciate in common, it is objectivity. Even when these individuals are interacting with a subject matter like aesthetic imaging, where the final judgement relies on the subjective perception of the viewer, metrics which can be used to compute image quality in an objective and repeatable way are highly sought after. This is because the alternative, subjective evaluation from observers, is time consuming, difficult to execute, and methodology dependent. On the other hand, if there were a metric whose judgements of image quality correlated with those of the average observer, image processing methods could be iteratively evaluated and improved completely within computer environments, and could be compared to existing methods on a common basis.
The application of image compression is a perfect example, as it aims to abbreviate image data which does not contribute to the viewer’s perception of reproduction quality (e.g. when viewers see a subtle gradation as a single color, this can be described with a single value as opposed to many, reducing the amount of data required to communicate it.) Thus, the image quality metric must be engineered to simulate visual perception well enough that it can identify what subtle image details would not be missed by observers if compressed. Metrics for this application then take the original, uncompressed image, compare it to a compressed version, and then return a quality score which should be highly correlated to the detectability of the abbreviated data. Using this quality score, compression methods can be tested, adjusted and retested on an iterative basis, accelerating their development.
Example image (starting at top left): uncompressed, color information compressed only, both brightness and color information compressed, brightness information compressed only.
While a large number of image quality metrics of this type exist, none are perfect, particularly when presented with images of novel formats like High Dynamic Range and Wide Color Gamut. To address this problem, the Image Processing for Enhanced Cinematography group at UPF has been involved in a collaboration with the Japanese Broadcasting Company, NHK, aiming to evaluate and improve existing image quality metrics. An experiment was first conducted at NHK to collect observer quality ratings of a dataset of compressed, 4K HDR WCG images. Then, the dataset was passed to a series of existing image quality metrics and the degree to which their scores correlated with those of observers was evaluated. Surprisingly, the results showed that more modern metrics which were specifically trained using images of these novel formats were outperformed by simpler, earlier metrics. From this point, our work has focused on improving the perception-mimicking elements of these metrics, as well as expanding upon them such that they also consider color information in addition to brightness. So far, the work has led to a series of conference publications at venues such as QUOMEX, ICASSP, and SMPTE, as well as a recently accepted publication in SMPTE’s Motion Imaging Journal. The result of work will be applied to the AdMiRe project to evaluate and iteratively train the image processing functions in the capture wizard, and expertise on this topic will be helpful in the overall assessment and benchmarking of image quality throughout the pipeline.
Trevor Canham – UPF IP4EC