Improving Selfie Aesthetics with Interactive Guidance based on Empirical Models

Abstract

We introduce RealSelfie, a smartphone camera application providing interactive guid- ance to help people take better self-portrait photos (commonly called “selfies”). The appli- cation uses empirical models to estimate aesthetic quality built from data gathered by 2,700 Amazon Mechanical Turk (AMT) aesthetic quality assessments of synthetic photographs. The synthetic photographs are generated from 3D models of realistic human models by manipulating a virtual camera and virtual lighting to precisely explore the space of three photographic principle parameters: face size, face position, and light direction. The Re- alSelfie application calculates the current value for each parameter using computer vision techniques and then compares those values with each model’s aesthetic estimates to display directional hints overlaid on the live camera preview. As part of this system, we contribute an algorithm to estimate lighting direction using the pattern of light and shade near the nose. We conduct a study to evaluate the RealSelfie application with 20 participants in a controlled environment to eliminate background and lighting confounds. AMT ratings of the photos show that RealSelfie provides a 26% increase in aesthetics over providing no guidance.