Post on 21-Oct-2020
On Realism of
Architectural Procedural ModelsJan Beneš, Tom Kelly, Filip Děchtěrenko, Jaroslav Křivánek, Pascal Müller
Quick Quiz
2Jan Beneš - On Realism of Architectural Procedural Models
Quick Quiz
3Jan Beneš - On Realism of Architectural Procedural Models
Computer Generated Photograph
Overview
4Jan Beneš - On Realism of Architectural Procedural Models
Procedural Modeling&
Realism
Questions Previous Work
Experiment &
Data Acquisition
Answers
Procedural Modeling
• Procedural modeling– Tool for artists
– Or fully automatic
– Wide range of outputs
• Wide use
5Jan Beneš - On Realism of Architectural Procedural Models
[Independence Day]
[Palubicki09]
Procedural Modeling
• Procedural modeling– Tool for artists
– Or fully automatic
– Wide range of outputs
• Wide use
6Jan Beneš - On Realism of Architectural Procedural Models
[Independence Day]
[Palubicki09]
Realism
Realism
Plausibility
Immersion
• Model & material & display
• Should be studied
• For procedural modelling
7Jan Beneš - On Realism of Architectural Procedural Models
[Velocipedia, Gianluca Gimini]
Realism
Realism
Plausibility
Immersion
• Model & material & display
• Should be studied
• For procedural modelling
8Jan Beneš - On Realism of Architectural Procedural Models
[Velocipedia, Gianluca Gimini]
What we did
9Jan Beneš - On Realism of Architectural Procedural Models
Procedural
architecture
What we did
10Jan Beneš - On Realism of Architectural Procedural Models
User Study
Question #1
(1) Can people tell procedurally generated
buildings from real ones?
11Jan Beneš - On Realism of Architectural Procedural Models
Question #2
(2) Is realism carried in the detail or in the
larger structure?
12Jan Beneš - On Realism of Architectural Procedural Models
Question #2
(2) Is realism carried in the detail or in the
larger structure?
13Jan Beneš - On Realism of Architectural Procedural Models
Question #2
(2) Is realism carried in the detail or in the
larger structure?
14Jan Beneš - On Realism of Architectural Procedural Models
Question #3
(3) What factors do the users think influence
the perception of realism?
15Jan Beneš - On Realism of Architectural Procedural Models
I think…
Questions
(1) Can people tell procedurally generated buildings from real ones?
(2) Is realism carried in the detail or in the larger structure?
(3) What factors do the users think influence the perception of realism?
16Jan Beneš - On Realism of Architectural Procedural Models
I think…
Previous Work
17
Previous Work - Realism
18Jan Beneš - On Realism of Architectural Procedural Models
“There is no dirt, no dust, no fingerprints on the glass table…simply too beautiful, too clean and polished…” [Reinhard13]
Previous Work - Realism
19Jan Beneš - On Realism of Architectural Procedural Models
CG2Real [Johnson11]
Previous Work - Realism
20Jan Beneš - On Realism of Architectural Procedural Models
[Rademacher01]
Previous Work - Buildings
21Jan Beneš - On Realism of Architectural Procedural Models
• Building generation
– Grammar [Stiny80, Wonka03, Muller06, Schwarz15]
– Data [Fan16]
– Sketch [Nishida16]
– Predefined parts [Kalogerakis12, Talton12]
Previous Work - Miscellaneous
22Jan Beneš - On Realism of Architectural Procedural Models
• Machine Learning– What makes Paris look like Paris [Doersch12]
– Architectural Style Recognition [Mathias11]
• Image Quality & Similarity– Visible Differences Predictor [Daly92]
– Visual Equivalence and Aggregates [Ramanarayanan07,08]
– Structural Similarity - SSIM [Wang04]
Experimental Setup
23
Datasets
24Jan Beneš - On Realism of Architectural Procedural Models
Favela Medieval Paris Venice
Datasets
25Jan Beneš - On Realism of Architectural Procedural Models
Favela Medieval Paris Venice
Photos
Computer
Generated
X
Classification Screen
26Jan Beneš - On Realism of Architectural Procedural Models
X
Blurs
27Jan Beneš - On Realism of Architectural Procedural Models
13px blur7px blur1px – no blur
55px blur32px blur25px blur
Summary
28Jan Beneš - On Realism of Architectural Procedural Models
TRAINING
Summary
29Jan Beneš - On Realism of Architectural Procedural Models
Venice-55px
Medieval-55px
Paris-55px
Favela-55px
TRAINING
Summary
30Jan Beneš - On Realism of Architectural Procedural Models
Venice-55px
Medieval-55px
Paris-55px
Favela-55px
Venice-37px
Medieval-37px
Paris-37px
Favela-37px
TRAINING
Summary
31Jan Beneš - On Realism of Architectural Procedural Models
Venice-55px
Medieval-55px
Paris-55px
Favela-55px
Venice-37px
Medieval-37px
Paris-37px
Favela-37px
Venice-1px
Medieval-1px
Paris-1px
Favela-1px
TRAINING
…
Summary
32Jan Beneš - On Realism of Architectural Procedural Models
Venice-55px
Medieval-55px
Paris-55px
Favela-55px
Venice-37px
Medieval-37px
Paris-37px
Favela-37px
Venice-1px
Medieval-1px
Paris-1px
Favela-1px
DEBRIEFING
TRAINING
…
Data preparation
33
Photographs
34Jan Beneš - On Realism of Architectural Procedural Models
Renders
35Jan Beneš - On Realism of Architectural Procedural Models
Data Characteristics
36Jan Beneš - On Realism of Architectural Procedural Models
300px
300px
Data Characteristics
37Jan Beneš - On Realism of Architectural Procedural Models
300px
300px
Choose subset
Quick Peek at Results
38
Quick Results
39Jan Beneš - On Realism of Architectural Procedural Models
random choice = 0.5
Quick Results
40Jan Beneš - On Realism of Architectural Procedural Models
random choice = 0.5
Quick Results
41Jan Beneš - On Realism of Architectural Procedural Models
random choice
Favela
Best Dataset
Medieval
Worst Dataset
Q1: CG vs Real
42
Q1: CG vs Real
43Jan Beneš - On Realism of Architectural Procedural Models
• HDIFF: participants can tell photographs and generated buildings apart.
– Random choice = 0.5 accuracy
– No blur, overall – p
Q1: CG vs Real
44Jan Beneš - On Realism of Architectural Procedural Models
• HDIFF: participants can tell photographs and generated buildings apart.
– Random choice = 0.5 accuracy
– No blur, overall – p
Q2: Details vs Structure
45
Q2: Details vs Structure
46Jan Beneš - On Realism of Architectural Procedural Models
• HSCALE: the detail that allows participants
to tell photographs and generated
images apart is present at various scales.
• ANOVA
What’s ANOVA?
47Jan Beneš - On Realism of Architectural Procedural Models
Black box
Independent
variable
Independent
variable
Independent
variable
Dependent
variable
What’s ANOVA?
48Jan Beneš - On Realism of Architectural Procedural Models
Black box
Blur Size
Original
image class
Dataset
Accuracy
What’s ANOVA?
49Jan Beneš - On Realism of Architectural Procedural Models
• ANOVA
– Linear model
– Effects of independent
variables
Significant Effects
50Jan Beneš - On Realism of Architectural Procedural Models
• Significant effects
– Blur, p
HSCALE
51Jan Beneš - On Realism of Architectural Procedural Models
• Significant effects
– Blur, p
More on Blur
52Jan Beneš - On Realism of Architectural Procedural Models
• Significant effects
– Blur, p
Post-Hoc - Blur
53Jan Beneš - On Realism of Architectural Procedural Models
Not significantSignificant
Blurs
54Jan Beneš - On Realism of Architectural Procedural Models
13px blur7px blur1px – no blur 55px blur32px blur25px blur
HSCALE: the detail that allows participants to tell photographs
and generated images apart is present at various scales.
Two- & Three-Way Interactions
55Jan Beneš - On Realism of Architectural Procedural Models
• Three factors
• Interactions
• More in paper
• Significant
• Not Significant
Dataset
Original Image
Class
Blur Size
Q2: Summary
56Jan Beneš - On Realism of Architectural Procedural Models
• Blur
– pHSCALE accepted
– Realism at different scales
• Exploratory Analysis
– Further insight
– Two- & Three-way interactions
– Mostly significant
Q3: What users thought
57
Participants
58Jan Beneš - On Realism of Architectural Procedural Models
• 52 total
– 11 female
– 24.3 years old
What users thought
59Jan Beneš - On Realism of Architectural Procedural Models
• Manually tallied debriefings
1. Imperfections & small detail (30/52 ~ 58%)
2. Texture (19/52 ~ 37%)
3. Reflections in windows (18/52 ~ 35%)
4. “Weird” or uniform color (17/52 ~ 33%)
5. Things in & around windows (16/52 ~ 31%)
6. Model Structure (14/52 ~ 27%)
7. Lighting (12/52 ~ 23%)
8. Shadow (12/52 ~ 23%)
9. Regularity (11/52 ~ 21%)
1. Imperfections & Small Detail
60Jan Beneš - On Realism of Architectural Procedural Models
• Imperfections & small detail (30/52 ~ 58%)
PhotographComputer GeneratedComputer Generated
2. Texture
61Jan Beneš - On Realism of Architectural Procedural Models
• Texture (19/52 ~ 37%)
Computer GeneratedComputer Generated
5. Windows
62Jan Beneš - On Realism of Architectural Procedural Models
• Things in & around windows (16/52 ~ 31%)
Jan Beneš - On Realism of Architectural Procedural Models
Computer Generated PhotographPhotograph
6. Model Structure
63Jan Beneš - On Realism of Architectural Procedural Models
• Model Structure (14/52 ~ 27%)
Jan Beneš - On Realism of Architectural Procedural Models
Computer Generated Computer Generated
9. Regularity
64Jan Beneš - On Realism of Architectural Procedural Models
• Regularity (11/52 ~ 21%)
Jan Beneš - On Realism of Architectural Procedural Models
Computer Generated Computer Generated
Biases
65Jan Beneš - On Realism of Architectural Procedural Models
• Cut-Out Edges– 21/52 ~ 40% – “played role”
• Camera Angles– 18/52 ~35% – Influenced for at
least one
• Background– 8/52 ~ 15% – Made me choose
computer generated more often
Opinions Summary
66Jan Beneš - On Realism of Architectural Procedural Models
• Multi-faceted
– Guidelines, guesses
– Not hard facts
• Biases
Textures
Color
More Exploration
67
Confounding Buildings @ 55px
68Jan Beneš - On Realism of Architectural Procedural Models
Computer
Generated
Photographs
Summary and Future Work
69
Summary
70Jan Beneš - On Realism of Architectural Procedural Models
• Realism in – Procedural Modelling
– Buildings
• Methodology for experiment– Procedural content
• Verified– People can tell CG and Photos apart
– Realism is carried at different scales
• “Soft results”– What people consider
– Exploration of results
Future Work & Limitations
71Jan Beneš - On Realism of Architectural Procedural Models
• Future Work– Understand asset reuse better
– Understand importance of structure better
– Study buildings in context
– Neural networks to automate testing & drive rule generation
• Limitations– Design limitations & Biases
– Generalization
Acknowledgements
72Jan Beneš - On Realism of Architectural Procedural Models
• ESRI
• Charles University GrantSVV-2016-260332
• Czech Science Foundation grant16-18964S.
• Filip Děchtěrenko was supported by Czech Academy of Sciences Strategy AV21 –RVO 68081740
• vrbn.io
• Corona Renderer
Thank You!
73
http://JanBenes.net
Supplemental Material
http://janbenes.net/
Biases
74Jan Beneš - On Realism of Architectural Procedural Models
• Cut-Out Edges– 21/52 ~ 40% – “played role”
• Camera Angles– 18/52 ~35% – Influenced for at
least one
• Background– 8/52 ~ 15% – Made me choose
computer generated more often
What users thought
75Jan Beneš - On Realism of Architectural Procedural Models
• Manually tallied debriefings
1. Imperfections & small detail (30/52 ~ 58%)
2. Texture (19/52 ~ 37%)
3. Reflections in windows (18/52 ~ 35%)
4. “Weird” or uniform color (17/52 ~ 33%)
5. Things in & around windows (16/52 ~ 31%)
6. Model Structure (14/52 ~ 27%)
7. Lighting (12/52 ~ 23%)
8. Shadow (12/52 ~ 23%)
9. Regularity (11/52 ~ 21%)