Style based

31
d.tour: Style-based Exploration of Design Example Galleries Author: Daniel Ritchie, Ankita Arvind Kejriwal, Scott R. Klemmer Stanford University HCI Group Department of Computer Science Stanford, CA, USA Source: UIST’11, October 1619, 2011, Santa Barbara, CA, USA. ACM Advisor: Geeng Neng You Speaker: Ching Jung Chang http://www.youtube.com/watch?v=g4ggoccLO yI&feature=related 1

description

 

Transcript of Style based

Page 1: Style based

1

d.tour: Style-basedExploration of

Design Example GalleriesAuthor: Daniel Ritchie, Ankita Arvind Kejriwal, Scott R. KlemmerStanford University HCI GroupDepartment of Computer ScienceStanford, CA, USASource: UIST’11, October 1619,2011, Santa Barbara, CA, USA. ACMAdvisor: Geeng Neng You Speaker: Ching Jung Changhttp://www.youtube.com/watch?v=g4ggoccLOyI&feature=related

Page 2: Style based

2

INTRODUCTION• An interface and interaction techniques for style-

based gallery exploration.• Introduce exploratory techniques for finding

relevant and inspiring design examples. • A curated database of Web pages .• Methods for extracting and analyzing design

features from a curated corpus of Web pages.• A recommender system that processes queries

and returns relevant pages from the database.• It decreases the gulfs of execution and evaluation

for design example-finding.

Page 3: Style based

3

Web interface : (a) Search Results, (b) Search box, (c) Current Query, (d) Ambient Style Display, (e) Bookmarks

Page 4: Style based

4

USER INTERFACE

• (a) Search Results: Presents search results as interactive thumbnails.

• query interface by providing ‘Show more like’ Show less like’ clicking a page thumbnail to expand the page to full size bookmark and add notes to the page.

• (b) Search box: Supports query by text string, color keywords, style terms, or any combination of the three. (colorful image-heavy)

Page 5: Style based

5

Searching for ‘colorful image heavy’ retrieves colorful pages that make heavy use of images.

Page 6: Style based

6

• (c) Current Query: This right-hand pane displays the keyword and page queries that specify the current state of the design gallery.

• Building simple interfaces for users to convey rationale is an important open issue in recommender systems.

• (d) Ambient Style Display: When a user hovers over a page, the display provides instant feedback by showing stylistic attributes .

• (e) Bookmarks: allows the user to maintain a collection of interesting pages across several search sessions.

Page 7: Style based

7

The Ambient Style Display shows bar chart visualizes the aggregate style attributes of the current example queries. When the user mouse over a page thumbnail, it instead shows that page’s stylistic attributes.

Page 8: Style based

8

IMPLEMENTATION

• Database• The current prototype includes around

300 Web pages listed on sites such as webbyawards.com, designmeltdown.com, ilovetypography.com, sixrevisions.com, and the Alexa Top 100.

• The complete contents of the database: bricolage.stanford.edu/database.

Page 9: Style based

9

• System Architecture:• All utilities for processing pages and

extracting features are implemented in:• C++ with the Qt framework.• QtWebKit API to interface with the

WebKit browser engine.• Search interface : HTML, JavaScript, CSS.• Supports HTML5-compliant browsers :

Firefox, Safari, and Chrome.• Web server: JavaScript using node.js

Page 10: Style based

10

FEATURE EXTRACTION

• Focuse on global pages features that describe a design’s use of space, color, text, and images.

• These features compose to predict higher-level stylistic attributes.

• Many of these features are derived from the page’s Document Object Model (DOM) tree.

Page 11: Style based

11

• The HTML DOM views an HTML document as a tree structure. The tree structure is called a node-tree.

• The tree starts at the root node and branches out to the text nodes at the lowest level of the tree.

• Node Parents, Children, and Siblings. The nodes in the node tree have a hierarchical relationship to each other.

http://www.w3schools.com/htmldom/dom_nodetree.asp

Parents

Children Siblings

Page 12: Style based

12

<html> <head> <title>Hello DOM </title> </head> <body> <h1>DOM Lesson one</h1> <p>Hello world!</p> </body></html>

The <html> node has no parent node; it is the root node.The <html> node has two child nodes; <head> and <body>The <head> node has one child node; the <title> nodeThe <title> node also has one child node; the text node “Hello DOM "The <h1> and <p> nodes are siblings, and both child nodes of <body>

Page 13: Style based

13

Principles from the design literatureUse ofSpace

DOM tree depth (min, max, mean, stddev), # of DOM leaf nodes, document width/height, amount of separation between content blocks, foreground/background ratio, overlapping element area, connected components in rendered image (#, minarea, maxarea, meanarea)

Use ofColor

Color (mean, stddev), saturation (min, max, mean, stddev), value (min, max, mean, stddev), # of colors (in DOM, in rendered page), most dominant color, most dominant text color, # of dominant colors, text-to-background contrast (min, max, mean, stddev), histogram

Use ofText

# of words in page, # of words per block (min, max, mean, stddev), # of fonts, font size (min, max, mode, mean)

Use ofImages

# of images, aspectRatio (min, max, mean, stddev), area (min, max, mean, stddev), complexity (min, max, mean, stddev)

Page 14: Style based

14

• Other features require Segmentation• Apply the Bento page segmentation algorithm to

ensure that the DOM tree accurately reflects the page’s visual layout.

• Bento: data collection study, the mapping algorithm, and the machine learning method.

• The system also computes a color histogram of the rendered page, quantizing the image into 256 colors that evenly sample the RGB color cube.

• Additional features for the local structure web pages.

• Bricolage algorithm transfers the contents of a source page into the style of a target page by computing a mapping between the two pages’ DOM trees.

Page 15: Style based

15

Left: The colored boxes illustrate Bento’s segmentation.Right: Bento’s output tree and associated DOM nodes for this page.

Page 16: Style based

16

• Color histogram in the RGB color space • A Red–Blue chromaticity histogram can be formed by first

normalizing color pixel values by dividing RGB values by R+G+B.

• Quantizing the normalized R and B coordinates into N bins each.

• A two-dimensional histogram of Red-Blue chromaticity divide in to four bins (N=4) might yield a histogram that looks like this table:

 Red

0-63 64-127128-191

192-255

Blue

0-63 43 78 18 064-127 45 67 33 2128-191 127 58 25 8

192-255 140 47 47 13

256 colors

Original

Reference: Wiki

Page 17: Style based

17

Bricolage computes coherent mappings between Webpages by matching visually and semantically similar page elements.

Page 18: Style based

18

QUERY PROCESSING• Use the text contents of its pages and the combined

feature vectors described above to respond to different types of queries.

• 1. Show More/Less Like’• q weighted average of all feature vectors for pages

rated via ‘show more/less like.’• M be the set of feature vectors for all ‘show more

like’ pages• L be the set of vectors for all ‘show less like’ pages.

Page 19: Style based

19

• 2. Text Keyword Search• System computes a TF-IDF score for each word in the

page’s document text . • It returns a ranked list of the pages whose combined

scores for the query terms are greater than zero.• TF-IDF : Term Frequency–Inverse Document Frequency.• Consider a document containing 100 words where in the

word colorful appears 3 times.• Term frequency (TF) for colorful (3 / 100) = 0.03. • Assume 10 million documents and colorful appears in

1,000 of these. DF=(10, 000 ,000 / 1, 000) • Inverse document frequency is calculated as log(10,

000 ,000 / 1, 000) =10,000= 4. • tf–idf score : 0.03 × 4 = 0.12.

Page 20: Style based

20

• 3. Color Keyword Search• The system converts each color keyword into an RGB

color c* for that word using a hard-coded lookup table. / Six-digit hexadecimal

• The current prototype supports twelve common color words.

• To determine how well a page matches the query color.

• It computes a score s based on the page’s histogram h. h is a set of pairs (ci; fi),

• ci - RGB color and fi - fraction of pixels in the page whose nearest histogram color is ci.

S=S1+S2+S3……..+Sh

Page 21: Style based

21

• frequently-occurring colors that are close to query color should have a high score contribution si.

• use a value of one-third the diagonal length of the RGB color cube for the threshold.

• The system returns as search results only the pages with s greater than a heuristically determined threshold.

fraction of pixels

each histogram pair (ci; fi)

RGB color

Page 22: Style based

22

Diagonal length

Reference: Wiki

Page 23: Style based

23

4. Style Keyword Search

The current prototype invokes theserules when the user searches for the bold terms.35 terms—including synonyms—map to 14 unique rules.

All the features in any single rule are weighted equally.

Page 24: Style based

24

• EVALUATION• Method• An online study instructed participants to find

a set of inspirational design examples.• Participants were randomly assigned to use

either d.tour or a Web search engine to find examples.

• Provide feedback about their search strategies and experience.

Page 25: Style based

25

• Participants • 40 participants: 20 through student email lists

at university and 20 more via Amazon Mechanical Turk.

• Participants searched for design examples. • Finishing search, responded to a

questionnaire. • Provided four inspirational design examples

URLs and explanations for their chosen examples.

Page 26: Style based

26

• Results• Mainly reported querying with keywords such

as ‘science,’ ‘kids,’ and ‘education.’• Retrieved pages from their own mental search

history, recalling well-designed, potentially relevant sites they had seen recently.

• Lack of style-based search or the insufficiency of searching by page content as serious roadblocks.

• Search function “rated sites by mood or feel.” • “Google is only really useful for searching for

actual page content, not page style.

Page 27: Style based

27

• d.tour as a “really good tool to find quick examples.”

• Specific interface features as useful.

• Current Query as reminder of search history.

• ‘Ambient Style Display’ received mixed feedback, difficulty interpreting and operationalizing the information it provided.

• Low corpus size as a limitation for d.tour.

Page 28: Style based

28

• Participants using d.tour focused on the style and present- ction of their chosen designs.

Page 29: Style based

29

• Many participants using search engines chose pages on a topic relevant to the design task.

Page 30: Style based

30

• CONCLUSION AND FUTURE WORK• 1. d.tour currently standardizes all page

features, effectively giving them all the same weight.

• 2. Expanding the stylistic vocabulary. Apply machine learning techniques to a large corpus of stylistically-tagged Web pages.

• 3. Many of d.tour’s features rely on the HTML DOM, which simplifies a page to a hierarchy of rectangles.

• 4. d.tour does not attempt to capture non-static aspects of design such as animation or interaction.

• 5. A full-fledged system should scale to much large page databases—perhaps even the entire Web.

Page 31: Style based

31

3. The wine bottles and the type on this page overlap and span multiple DOM nodes (outlined in blue). Breaking the strong grid lines specified by the DOM. d.tour’s current feature set cannot support queries about ‘organic’ page layouts such as this.