Color Compatibility From Large Datasets Peter O’Donovan University of Toronto Aseem Agarwala Adobe...

Color Compatibility From Large Datasets

Peter O’DonovanUniversity of Toronto

Aseem AgarwalaAdobe Systems, Inc.

Aaron HertzmannUniversity of Toronto

Choosing colors is hard for many people

How do designers choose colors?

Picasso


You the Designer


Krause [2002]


Goethe [1810]

Complementary Color Theory: colors opposite on the color wheel are compatible

Hue Templates: relative orientations producing compatible colors

Complementary Monochromatic Analogous Triad

Photo and Video Quality Evaluation:Focusing on the SubjectLuo and Tang 2008

Aesthetic Visual Quality Assessment of PaintingsLi and Chen 2009

Color Harmonization for VideosSawant and Mitra 2008

Color Harmonization Cohen-Or et al. 2006

Adobe Kuler

527,935 themes

Ratings: 1-5 stars

COLOURLovers

1,672,657 themes

Views and “Likes”

Goals

1. AnalysisTest hypotheses and compatibility models

2. Learn ModelsPredict mean ratings for themes

3. New ApplicationsDevelop new tools for choosing colors

104,426 themes Ratings: 1-5 stars

383,938 themes # Views and “Likes”

Kuler Dataset COLOURLovers Dataset

Mechanical Turk dataset

10,743 themes from Kuler40 ratings per theme1,301 total participants

Overall preference for warmer hues and cyans

Histogram of hue usage

Hue

% of all themes

Mean rating for themes containing a hue

Overall preference for warmer hues and cyansHue

Mean Rating

Histogram of hue adjacency (Kuler)

is more likely than


Significant structure


Significant structureWarm hues pair well with each other


Significant structureWarm hues pair well with each otherGreens and purples more compatible with themselves


Hue Template Analysis

Templates are rotationally invariant


Different templates equally compatible

Complementary Monochromatic Analogous Triad


Diagonal lines are hue templates (Kuler interface bias)

Hue adjacency in a theme (Kuler)

Complementary template


Diagonal lines are hue templates (Kuler interface bias)


Complementary:

Complementary: Data:


In template theory, diagonals should be uniform


In template theory, diagonals should be uniformLarge dark bands indicates no rotational invariance


Kuler CL

Hue adjacency in a theme

COLOURLovers’ has less interface biasTemplates are not present

Distance to template

Rating


Themes near a template score worse

Rating

Themes near a template score worse - “Newbie” factor - “Too simple” factor


Rating

MTurk has no interface bias: much flatter


Rating

Template Conclusions

1) Templates do not model color preferences2) Themes near a template do not score better

than those farther away3) Not all templates are equally popular

- Simple templates preferred (see paper)

Hue Entropy: entropy of hues along the hue wheel


Low Entropy

Few Distinct Colors


Low Entropy High Entropy

Few Distinct Colors Many Distinct Colors


Hue Entropy

Rating

Main Analysis Results

1. Overall preference for warmer hues and cyans



2. Strong preferences for certain adjacent colors




3. Hue templates a poor model for compatibility





4. People prefer simpler themes (but not too simple)





4. People prefer simpler themes (but not too simple)

See paper for other tests

Goals


2. Learn ModelsPredict mean ratings for themes


3.63

𝑓 (𝒙 )=𝑦

Mean rating over all users

3.63

𝑓 (𝒙 )=𝑦

𝑓 (𝒙 )=𝑦

Features (326 total)- Colors, sorted colors, differences, min/max,

max-in, mean/std dev, PCA features, hue probability, hue entropy

- RGB, HSV, CIELab, Kuler color wheel- “Kitchen Sink”

3.63

𝑓 (𝒙 )=𝑦

Models- Constant baseline: mean of training targets - SVM-R, KNN- Lasso

- Linear regression model with L1 norm on weights- Solutions have many zero weights: feature

selection

3.63

Dataset MAE

Constant Baseline

KNN SVM-R

Lasso Lasso over Baseline

Kuler 0.572 0.533 0.531 0.521 9%

COLORLovers

0.703 0.674 0.650 0.644 8%

MTurk 0.267 0.205 0.182 0.179 33%

Dataset MAE

Constant Baseline

KNN SVM-R


Kuler 0.572 0.533 0.531 0.521 9%

COLORLovers

0.703 0.674 0.650 0.644 8%

MTurk 0.267 0.205 0.182 0.179 33%

Many more ratings per theme in MTurk

Dataset MAE

Constant Baseline

KNN SVM-R


Kuler 0.572 0.533 0.531 0.521 9%

COLORLovers

0.703 0.674 0.650 0.644 8%

MTurk 0.267 0.205 0.182 0.179 33%

MTurk has an average std dev of 0.33Kuler has an average std dev of 0.72

MTurk Test Set

Human Rating

Lasso Rating

High-rated

𝑦=3.90 , f (𝐱 )=3.41𝑦=3.79 , f (𝐱 )=3.50

High-rated

Low-rated

𝑦=3.90 , f (𝐱 )=3.41𝑦=3.79 , f (𝐱 )=3.50

𝑦=1.71 , f ( 𝐱 )=2.27𝑦=1.78 , f (𝐱 )=2.25

High-rated

Low-rated

High prediction error

𝑦=3.90 , f (𝐱 )=3.41𝑦=3.79 , f (𝐱 )=3.50

𝑦=1.71 , f ( 𝐱 )=2.27𝑦=1.78 , f (𝐱 )=2.25

𝑦=2.74 , f ( 𝐱 )=1.78𝑦=2.22 , f ( 𝐱 )=3.16

Model Analysis

Important Lasso Features

Positive: high lightness mean & max, mean hue probability

Important Lasso Features

Positive: high lightness mean & max, mean hue probability

Negative: high lightness variance, min hue probability

Goals


2. Learn ModelsPredict mean ratings for theme


1. Improve a Theme

Maximize regression score

Stay within a distance of original (L2 in CIELab)

Select order which maximizes score

Optimize colors with CMA [Hansen 1995]

Original Best Order Color and Order

f (𝐱 )=2.92 f (𝐱 )=3.04 f (𝐱 )=3.35

f (𝐱 )=3.00 f (𝐱 )=3.11 f (𝐱 )=3.37

f (𝐱 )=3.50 f (𝐱 )=3.50 f (𝐱 )=3.70

Original Best Order Color and Order

f (𝐱 )=2.92, 𝑦=3.04f (𝐱 )=3.04 , 𝑦=2.99f (𝐱 )=3.35 , 𝑦=3.40

f (𝐱 )=3.00 , 𝑦=2.96f (𝐱 )=3.11 , 𝑦=3.21

f (𝐱 )=3.50 , 𝑦=3.72f (𝐱 )=3.50 , 𝑦=3.69f (𝐱 )=3.70 , 𝑦=3.82

MTurk A/B test with original and optimized themes

Order and Color

2. Choose 5 colors that best ‘represent’ an image

One approach: k-means clustering

One approach: k-means clustering

This ignores color compatibility

Optimize 5 colors that1) Match the image well2) Maximize regression

score

Optimize 5 colors that1) Match the image well2) Maximize regression

scoreSee paper for details

With Compatibility Model

W/O Compatibility Model

MTurk A/B testwith and withoutcompatibility model

3. Given 4 colors for foreground, suggest background

Given 4 colors, choose 5th color to maximize score

Want contrast with existing colors

Find next best color, away from previous choices

, , …

Model Suggestions

Random Suggestions

MTurk tests selecting ‘Worst’ and ‘Best’

4 model & 4 random

Model Limitations & Future Work

Hard to interpret

Features

Weights

Model has very few abstract colors, only 1-D spatial layout

VS

Model does not understand how colors are used

Conclusions

Color preferences are subjective, but analysis reveals many overall trends

Simple linear models can represent compatibility fairly well

Models can be useful for color selection tasks

Our datasets and learned models are available online

Color Compatibility From Large Datasets Peter O’Donovan University of Toronto Aseem Agarwala Adobe...

Documents

Transcript of Color Compatibility From Large Datasets Peter O’Donovan University of Toronto Aseem Agarwala Adobe...