Loud and Trendy: Crowdsourcing Impressions of Social Ambiance in Popular Indoor Urban Places

Darshan Santani, Daniel Gatica-PerezIdiap and EPFL Switzerland

28 October 2015

ACM MM 2015, Brisbane

Which place looks more “romantic”?

A B

Which place feels more “loud”?

A B

Understanding Social Ambiance of Urban Places

Place impressions define our favorite hangouts and shape our new discoveries

Social ambiance elicited by perceptual cues and prior knowledge

Connections between psychological features of cities and key indicators like well-being and prosperity

Our goal: to provide “a better idea of how people perceive and experience places”

Brave New Idea?

Social Media

Mobile Technologies Lots of images

Lighting

Spatial Layout

Flooring

Wall Decorations

Table Layout

Ceiling

Holistic study of place ambiance would involve interpretation of multiple sources of information, including subtle visual (and audio) cues → Many

challenges for multimedia research

Open Issue: Multimedia approaches to study social perception of urban places.

Research Questions

● RQ1: What types of social media images are perceived as being more informative of the ambiance of popular indoor places?

● RQ2: Can the ambiance of an indoor place be reliably assessed by observers of social media images? If so, for what dimensions of ambiance?

Dataset – Places

Singapore

NYCSeattle

Mexico City

Barcelona Paris

50 popular places per cityPlaces: cafes, restaurants, bars, or clubs

Dataset – Places

Singapore

NYC

Seattle

Mexico City

BarcelonaParis

50 popular places per cityPlaces: cafes, restaurants, bars, or clubs

Dataset – Places

Singapore

NYC

Seattle

Mexico City

BarcelonaParis

50 popular places per cityPlaces: cafes, restaurants, bars, or clubs

Image Dataset #1

● Random Image Corpus● 50,000+ images for all 300 places via 4SQ API● 55% iPhone; 19% Android; 22% via Instagram● Randomly selected 3 images per place for a total of 900 images

Image Dataset #2

● Physical Environment Image Corpus● Clear views of the environment (manually chosen)● 3 images per place for a total of 900 images

RQ1: What types of social media images best convey the ambiance of popular indoor

places?

Use image corpora to judge which image selection approach results in images being perceived as more

adequate to convey ambiance.

Methodology

● Crowdsourcing via Mechanical Turk● 5 images – 2 from the Physical Environment image

corpus, and 3 from Random image corpus● Tasks:

● Rank the images based on how informative they are of the ambiance of a place

● Categorize the images in one of the four classes: – Food/Drinks, – People/Group, – Physical Environment, and – None of these.

Results

Images from Phy.Env. corpus are in Top 2 ranks

91.7% for ambiance

Results

Images from Phy.Env. corpus are in Top 2 ranks

91.7% for ambiance

96% of Phy. Env. Image corpus describe the physical environment

67% Random image corpus describe either food or people

Results

Phy. Env. image corpus describe the physical environment in 96.2% of the cases

Random image corpus describe food items or people in 67% of the cases

Finding from RQ1: Images with clear views of the environment are perceived as more suitable to characterize indoor

ambiance.

RQ2: Can the ambiance of an indoor place be reliably assessed by observers of social

media images? If so, for what dimensions of ambiance?

Methodology

● Crowdsourcing via Mechanical Turk

● Data: Phy. Env. Corpus 900 images across 300 places

● Ambiance Labels: 13 dimensions [Graham, 2011]

● Task: Rate personal impressions of the place ambiance along 13 dimensions on a 5-point Likert scale

● 10 annotations for each dimension per place → 3,000 responses.

[Graham 2011] L. T. Graham, & S. D. Gosling, “Can the ambiance of a place be determined by the user profiles of the people who visit it?”. In the Fifth International AAAI Conference on Web and Social Media (ICWSM), 2011

Artsy

Bohemian

Conservative

Creepy

Dingy

Formal

Sophisticated

Loud

Old Fashioned

Off the beaten path

Romantic

Trendy

Upscale

Ambiance Labels

Inter-annotator Consensus

[0.8, 1.0)

[0.6, 0.8)

[0.0, 0.6)

ICCs across Cities

[0.8, 1.0) [0.6, 0.8) [0.0, 0.6)Color Legend

Ambiance across Cities – Correlation Analysis

Comparison between Cities

● Relatively few significant differences across cities

● Popular places in Seattle are perceived as less artsy compared to places in Barcelona.

● Popular places in Paris are perceived as less old fashioned com-pared to NYC and Seattle.

Comparison between Cities

● Popular places in Seattle are perceived as less artsy compared to places in Barcelona.

● Popular places in Paris are perceived as less old fashioned com-pared to NYC and Seattle.

Findings from RQ2:

Reliable estimates of ambiance can be obtained using social media images, suggesting the

presence of visual cues to form place impressions.

Most aggregate impressions of ambiance are similar across popular places in all cities

Future Work

● What specific cues are used to form place impressions?● Color? Lighting? Spatial layout? Interior Design?

● Automatic recognition of ambiance● Establishing links between visual cues and place

ambiance

Q & A

Email: dsantani@idiap.chTwitter: @SabMayaHai