In an online dating profile, the photo that highlights your physical beauty may not be the one that makes you look smart or honest — also important traits in a significant other. A new neural network helps pick the most appealing shots.
What’s new: Agastya Kalra and Ben Peterson run a business called Photofeeler that helps customers choose portraits for dating and other purposes. Their model Photofeeler-D3 rates perceived looks, intelligence, and trustworthiness in photos. You can watch a video demo here.
Key insight: Individuals have biases when it comes to rating photos. Some consistently give higher scores than average, while others may consistently give more random scores. By taking into account individual raters’ biases, a model can predict more accurately how a group would judge a photo.
How it works: Photofeeler-D3 scores the beauty, intelligence, and trustworthiness of a person in a photo on a scale of 1 to 10. The network was trained on more than 10 million ratings of over 1 million photos submitted by customers through the company website.
- Photofeeler-D3 learned each rater’s bias (that is, whether the person’s ratings tend to be extreme or middling) based on their rankings of photos in the training dataset. The model represents this individual bias as a vector.
- A convolutional neural network using the xception architecture learned to predict a score for each trait. (The score wasn’t used.) After training, the CNN used its knowledge to generate vector representations of input images.
- The model samples a random rater from the training dataset. An additional feed-forward layer predicts that rater’s scores using the bias vector and photo vector.
- Then it averages its predictions of 200 random raters to simulate an assessment by the general public.
Results: Tested on a dataset of face shots scored for attractiveness, Photofeeler’s good-looks rating achieved 81 percent correlation compared to the previous state of the art, 53 percent. On the researchers’ own dataset, the model achieved 80 percent correlation for beauty, intelligence, and trustworthiness.
Why it matters: Crowdsourced datasets inherit the biases of the people who contributed to them. Such biases add noise to the training process. But Photofeeler’s voter modeling turns raters’ bias into a benefit: Individuals tend to be consistent in the way they respond to other peoples’ looks, so combining individuals yields a more accurate result than estimating mean ratings while ignoring their source.
We’re thinking: We’d rather live in a world where a link to your Github repo gets you the most dates.