You are here

Computational image aesthetics, an automatic analysis

Team and supervisors
Department / Team: 
Team Web Site: 
http://www-percept.irisa.fr/
PhD Director
Olivier LE MEUR
Co-director(s), co-supervisor(s)
Contact(s)
PhD subject
Abstract

Description

Image aesthetics assessment/analysis is a crucial part for many image editing applications, such as image sorting, image browsing or image query. In the context of image editing, automatic image retargeting, such as tone mapping, adapts image pixel values to fit display features while preserving original image aesthetics. This requires to assess the retargeted image aesthetics and to ensure that this aesthetics is as close as possible to the original one.

For this purpose, we can use one of the many image aesthetics assessment methods (see the review [Deng et al., 2017] for more details). Unfortunately, these methods suffer from several problems. First, they are biased by the training dataset that are composed, for most of them, of consumers/amateurs photography. A second issue concerns the fact that these methods just assess the overall aesthetics quality without providing a thorough and comprehensive aesthetic report; it would be more than helpful to explain and to elaborate on well-defined and grounded dimensions, such as visual composition, color harmony.

Objectives

The main objective of the proposed Phd thesis is to go beyond current methods evaluating the aesthetic quality of an image. At least we have to deal with two main limitations. Rather than providing a unique aesthetic score, we aim to provide a detailed report of the aesthetic of an image. This report aims to discuss the strengths and weaknesses of the tested image. A very recent work is going in this direction [Wang et al., 2018]. The second point is to design a new aesthetic dataset by considering not only the aesthetic scores provided by a panel of observers but also new and rich features such as the gaze deployment. From the gaze deployment, a wealth of information can be inferred such as the visual dispersion, the visual engagement, the cognitive load (pupil diameter), to name the most important ones.

In summary, the expected contributions of the proposed Phd thesis are:

  • A new aesthetic metric allowing us to score an image and to report automatically the strengths and weaknesses of the tested image;
  • A new aesthetic dataset composed of a set of images associated to an aesthetic score associated with a number of information related to observers’ visual attention (e.g. aesthetic, visual engagement, cognitive load…);
  • New editing image processing method that would leverage the proposed aesthetic method.

The design of the aesthetic model will likely rely on supervised machine learning, and more specifically on deep learning. The candidate will then also contribute to the field of deep learning by investigating smart data augmentation approaches, transfer learning (develop model approach or pre-trained model approach) / domain adaptation (aesthetic model, saliency model…). In addition, automatically reporting information related to the aesthetic of an image requires to master image captioning methods. The main idea is to map aesthetics observations of the scenes to an aesthetic language, which will be defined throughout the thesis. In order words, in addition to the aesthetics scores, we aim to output succinct and accurate descriptions related to the image aesthetics features.

Bibliography
  • Deng, Y., Loy, C. C., & Tang, X. (2017). Image aesthetic assessment: An experimental survey. IEEE Signal Processing Magazine, 34(4), 80-106.
  • Bernardi, R., Cakici, R., Elliott, D., Erdem, A., Erdem, E., Ikizler-Cinbis, N., ... & Plank, B. (2016). Automatic description generation from images: A survey of models, datasets, and evaluation measures. Journal of Artificial Intelligence Research, 55, 409-442.
  • Wang, W., Yang, S.,  Zhang, W. & Zhang, J. (2018). Neural Aesthetic Image Reviewer, arXiv.
Work start date: 
Sept. 2019
Keywords: 
aesthetic, visual perception, computational model, image, deep learning
Place: 
IRISA - Campus universitaire de Beaulieu, Rennes