What do Sketches say about Thinking?

Pictorial and Verbal Tools for Conveying Routes

(COSIT 1999)

Barbara Tversky and Paul Lee

Department of Psychology

Stanford University

Route Maps and Route Directions

• Share the same underlying structure and semantics.

• A limited schematic map and direction toolkit are sufficient.

• As a result, should allow for automatic translation between the two.

Depictions

• Pictures regarded as an iconic medium.– Represent real objects and real space through

similarity, but debatable.

• Diagrams represent elements and relations between them.– Not meant to represent the physically apparent

world.

Descriptions

• Language. A purely symbolic medium with arbitrary symbols combined in complex rule-governed stuctures.

Depictions and Descriptions are Complementary

• Depictions preferred to convey iconic or metaphorically iconic information.

• Descriptions preferred for abstract information.

• Both complement human memory and information processing.

Actual Practice

• Graphic and verbal media are rarely pure.

• Maps have legends and arbitrary symbols.

• Written language uses spatial devices.– Space between words.– Paragraph indentations.

External Representations

Advantages over Information Processing

• External representations are analogous to internal ones: storage and retrieval devices.

• Good external displays compensate for limitations of information processing.

• Good external displays take advantage of skills of information processing.

Advantages over Information Processing

• External representations are:– Virtually unlimited size.– Permanent.– Public, transportable and shareable.

• Enlarge human memory, enhance processing by ‘offloading’ to inspectible, rearrangeable space.

Unique Advantages of Depictions

• Related information may be spatially proximal, minimizing search and facilitating inferences.

• Grouping, ordering, and distance in space correspond to grouping, ordering and distance on some other dimension.

Limitations of Depictions

• Difficult to convey abstract concepts.

• May force concreteness where it may not be meaningful, encouraging false inferences.– Bishop Berkeley: ‘Can only depict a particular

triangle, with specific angles and sides, not a general abstract triangle.

• Don’t naturally convey all meanings and relations.

Route Maps and Directions

Structure of Route Directions• The findings of Denis and collaborators

discerned several components of ideal route directions.– First: Put listener at point of departure.– Second: Begin the progression– Third: Iterate until the goal is reached:

• designate landmark

• reorient

• start again by prescribing an action

Actions

• Actions are changes of orientation or continuations of the current direction.

• A sequence of segments, each segment a triple:– orientation– action– a landmark (start and end points)

Route Maps

• Tversky and Lee experiment: Stopped bypasses near a campus dorm to either sketch a map, or write directions to a popular fast food restaurant.

• Maps and directions were coded for Denis’ categories plus the extra information.

• 90% of both contained extra information.

Common Structures• Remarkably, the structures of both were the same.

• Both could be divided into segments of triples.

• Landmarks schematized as rough shapes in maps.

• Actions represented by lines that referred to paths.

• About 1/2 of maps used arrows for directions.– Usually redundant.– Maps only contained relevant streets, so no ambiguity.

More Common Structures

• Maps had three kinds of paths.– Intersections

• Corresponded to turns in directions.

• Drawn at 90 degree angles, irrespective of actual angle.

– Straight Paths– Curved Paths

More Common Structures

• Actions directing the traveler to turn were also indifferent to actual angle of turn.– Terms like “turn”, “take a”, “make a”, “go”,

“left”, “right”.

• Actions corresponding to straight paths used terms such as “go”, “head”, “continue”.

• Curved paths used the term “follow” rather than “go”.

Discretized Depictions

• Route maps are potentially an analog medium, however that fact was not taken advantage of.

• Map-makers discretized the environment much like as used in route directions.

Conclusions to the Experiment

• Both were composed of the same components, with similar and corresponding distinctions within each category.– Landmarks.– Orientations.– Actions.

Interesting Differences

• For each component there were more verbal options than pictorial.

• Language allows several ways to express the same action.

• Depictions can conflate concepts that descriptions often separate.– A crossed line indicates an intersection, start

point, end point, and a turn simultaneously.

Sufficiency

• All of the information was explicitly contained in the maps.

• Much information was missing from the directions.– 75% lacked either a start or end point– 45% lacked path/progression information

• Still, directions were adequate to arrive at the destination.

Inference Rules• Rule of continuity stipulates that if a start point

is missing, it is the same as the previous end point.

• Rule of forward progression stipulates that when two reorientations occur successively, a forward movement is implied between those two reorientations.

• Assuming these rules: 86% of directions were complete and sufficient.

Translating Between Depictions and Descriptions

• Can we automatically translate between route maps and route directions?

• They are both composed of similar components.

• Useful for navigation devices.

Experiment• Give participants a set of route finding

problems and toolkits (descriptive or depictive) to construct a route.

• Participants could supplement the toolkits whenever needed.

• No direct and obvious correspondence between map and direction components.

• Pictorial elements vs. verb phrases.

Method

• 14 students.

• Each constructed 7 maps and 7 directions from the toolkits.

• Black marks for toolkit elements, red marks for their own additions.

• Minimalist toolkits based on previous experiment.

Map Toolkit

Direction Toolkit

Map Results• All maps used the three types, X,T, and L

intersections, as well as straight-line paths.

• The intersection types weren’t always corresponding to the real map.– 93% used at least one incorrect intersection.– Misuse might have been deliberate.

• Landmarks were by default rectangular, unless landmark actually was round.

• 86% used arrows and curved paths.

Example Map

Added Elements

• 43% marked at least one intersection as differing from 90 degrees.

• 29% marked at least one path as differing from straight or curved.

• These deviations were not necessarily accurate.

More added elements• About 1/2 used special depictions for exit

ramps, overpasses, street signs or lights.

• 21% used other iconic landmarks

• 43% used landmarks neither rectangular nor square, and also not iconic.

• 50% added short descriptive information.

• 2 used compass directions.

• 1 used distance information.

Directions Results• Participants used all of the classes provided

in the toolkit.

• Some actions were modified spatially.– “Turn sharply”

• Some were modified temporally.– “after 3 minutes”– “turn immediately”

• Most participants added a second landmark to at least one action.

Example Directions

Start at Stanford Shopping Center, facing El Camino. Turn right onto El Camino. Turn right at the first light onto Galvez St. At the second stop sign, turn left onto Serra St. Follow Serra St. until you hit Campus Dr. Turn right onto Campus. Turn right at the first stop sign onto Escondido Rd. Branner should be on your right side

Conclusions• Were the toolkits sufficient?

– Yes, in this domain. Paths are named streets in a city-world.

• Were all components necessary?– Maybe not the arrows. Not everyone needed

them.

• Are the toolkits sufficient in general? – Probably not. Barren deserts, dense jungles are

without clear paths. However, conventional sketches may be deficient as well.

The Big Question

• Can we automatically translate maps and directions?

• The Toolkits map onto each other, so YES, this should be possible. (but not done)

• “a simulation of a simulation”