Need a quick wardrobe upgrade? Image generation to the rescue! This research project automatically visualizes small changes to clothing that make the wearer look more fashionable.
What’s new: Researchers built a neural net to answer the question: Given an outfit, what small changes would make it more fashionable? Fashion++ renders improvements, from rolling up sleeves to adding an accessory to replacing garments. This video explains.
How it works: Given a full-body image, the model:
- Evaluates various body regions (face, hair, shirt, pants, and so on)
- Scores fashionability per region separately for shape (fit and presentation such as whether or not a shirt is tucked in) and texture (color, pattern, and material)
- Finds higher-scoring alternatives
- Updates shapes as 2D segmentation maps and textures as 3D feature maps
- Renders a series of updated outfits balancing higher scores with minimal change.
Can AI have fashion sense? To train the fashionability classifier, Wei-Lin Hsiao and her collaborators represented high fashion using the Chictopia photo set. They created degraded alternatives automatically by swapping in dissimilar garments (as measured by Euclidean distance on CNN features). Judges on Mechanical Turk found 92% of most-changed to be more fashionable.
Takeaway: Fashion++ has the kind of smarts generally thought to be the province of humans. It has clear commercial uses in ecommerce. And who couldn’t use a style assist?