Interactive Visualisation of Sound Interactive Visualisation of Sound ObjectsObjects

• Visualise: (vb) Visualise: (vb) to form a mental image or vision ofto form a mental image or vision of

• Cognitive abilityCognitive ability

• Allows us to internalise dataAllows us to internalise data

– Gain insight and understandingGain insight and understanding

• Internal Map = Cognitive ModelInternal Map = Cognitive Model

5 basic approaches5 basic approaches

IndentationIndentation

Node-link diagramsNode-link diagrams

Containment

ClusteringClustering Geographic

Visualisation Techniques / Visualisation Techniques / MethodsMethods

IndentationIndentation

• Tree controlTree control

• FisheyeFisheye

ContainmentContainment

• TreemapsTreemaps

• ZUI’sZUI’s

IndentationIndentation

• Tree controlTree control

• FisheyeFisheye

ContainmentContainment

• TreemapsTreemaps

• ZUI’sZUI’s

ClusteringClustering

• Galaxy of NewsGalaxy of News

• ThemeScapeThemeScape

GeographicGeographic

• Floor plansFloor plans

• Street mapsStreet maps

ClusteringClustering

• Galaxy of NewsGalaxy of News

• ThemeScapeThemeScape

GeographicGeographic

• Floor plansFloor plans

• Street mapsStreet maps

Node-link diagramsNode-link diagrams

• 2D diagrams 2D diagrams

• Cone TreeCone Tree

• Fisheye Cone TreeFisheye Cone Tree

• Hyperbolic viewerHyperbolic viewer

• XML3DXML3D

Node-link diagramsNode-link diagrams

• 2D diagrams 2D diagrams

• Cone TreeCone Tree

• Fisheye Cone TreeFisheye Cone Tree

• Hyperbolic viewerHyperbolic viewer

• XML3DXML3D

Sonic Browser InterfaceSonic Browser Interface

Sonic Browser

Psychophysical

Visual Auditory

Actions

Cataloguing

Visual Auditory

Actions

Tool for PsychophysicsTool for Psychophysics• Conduct simple experimentsConduct simple experiments

– Based on a psychophysic scalingBased on a psychophysic scaling

– 1 to 4 dimension comparison of sounds (X,Y, Shape, 1 to 4 dimension comparison of sounds (X,Y, Shape, Colour)Colour)

– Supplements traditional discrimination methods by Supplements traditional discrimination methods by providing both increased participant motivationproviding both increased participant motivation

• The scaling of the natural sounds vs. the model soundsThe scaling of the natural sounds vs. the model sounds

– Do they scale?Do they scale?

– Might validate sound models in relation to recordings of Might validate sound models in relation to recordings of real soundsreal sounds

Psychophysical

VisualEncoding

AuditoryEncoding

Actions

Bouncing Balls Breaking Glass

Tool for CataloguingTool for Cataloguing• Utilise traditional IR methods such asUtilise traditional IR methods such as• Indexing and FilteringIndexing and Filtering

– Controlled by text filters of meta information Controlled by text filters of meta information and by and by alphaslidersalphasliders of sound properties

• Combine Sound Retrieval methods such asCombine Sound Retrieval methods such as– Automatic classification & clustering among Automatic classification & clustering among

several possible AIR methodsseveral possible AIR methods– Cue points analysis and browsing of sound Cue points analysis and browsing of sound

resourcesresources– Supplement with results of psychophysical Supplement with results of psychophysical

scalingscaling

Cataloguing

VisualEncoding

AuditoryEncoding

Actions

Domain/Task FocusDomain/Task Focus

• Work is user-centered informed by Work is user-centered informed by ethnographyethnography

• Trying to understand how real people might Trying to understand how real people might work with sound objectswork with sound objects– E.g. Foley artists, composers, HCI E.g. Foley artists, composers, HCI

designersdesigners• simplify user navigation to sound objectssimplify user navigation to sound objects• find and manipulate sound objectsfind and manipulate sound objects• design visual aspectsdesign visual aspects• design auditory aspectsdesign auditory aspects• evaluate effectiveness & quality of use with evaluate effectiveness & quality of use with

respect to real working contextsrespect to real working contexts

My SOHO ObjectivesMy SOHO Objectives• Perform a pilot experiment 3-5 participants comparing Perform a pilot experiment 3-5 participants comparing

natural sounds versus sound models in psychophysics natural sounds versus sound models in psychophysics scenarioscenario

• Investigate results and use this to complement future work Investigate results and use this to complement future work on sound modelson sound models

• Gain insight and feedback from participants on current and Gain insight and feedback from participants on current and future dimensions to this researchfuture dimensions to this research

• Provide a roadmap for the production of a tool which will be Provide a roadmap for the production of a tool which will be used to navigate sound model collectionsused to navigate sound model collections

Grey Areas / DimensionsGrey Areas / Dimensions

• How to map the parameters from the Sonic How to map the parameters from the Sonic Browser to the Pure Data models in real time?Browser to the Pure Data models in real time?

• How to do this for multiple models or model How to do this for multiple models or model instances simultaneous (as when under the instances simultaneous (as when under the Aura)?Aura)?

• How to catalogue dynamic models such as the How to catalogue dynamic models such as the impact model with multiple variable parameters?impact model with multiple variable parameters?