Multimedia Information Systems VU...
Transcript of Multimedia Information Systems VU...
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualisation in the Web
Multimedia Information Systems VU (707.020)
Vedran Sabol
KTI, TU Graz
7th Decmber 2015
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Overview
• Motivation and Definition
• Introduction to Visualization
• Visualization examples and demos
Document content
Multidimensional data
Structures
Temporal and geospatial data
Multiple-visualisation interfaces
• Visual Analytics
• Web Visualisation Technologies and Frameworks
2
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualisation - Motivation
• We are confronted with:
Massive amounts of information
Complex heterogeneous information
Dynamically changing data
Uncertain, incomplete and conflicting information
…
• Big Data
• The four V’s: Volume, Variety, Velocity, Veracity
• Difficult to process using traditional applications and methods
3
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualisation - Motivation
How can computers help us to understand data?
Interactive exploration and analysis of data
Unveiling facts and knowledge hidden within the data
4
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Knowledge Discovery Process
• Knowledge Discovery Process [Fayyad, 1996]
Mainly an automatic approach consisting of a chain of processing steps
Goal: discovery of new, relevant, previously unknown patterns and relationships in data
5
FeedbackTarget Data
Transformed
Data
Patterns &
Models
Preprocessed
Data
Data
USER
Knowledge
Preprocessing & Cleaning
Data Transformation
Data Mining & Pattern Discovery
Interpretation & Evaluation
Data Selection
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualisation - Motivation
• Machines are very powerful
Automatic processing methods for huge data sets
Exponential growth of computer-performance since 60 years
• Moor‘s Law: continues until 2020, 2030… ?
Distributed computing: Cloud, Grid, …
• Nevertheless, machines still behind humans in
Identification of complex patterns and relationships
Wide knowledge, experience, intuition
Abstract thinking
…
6
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualisation - Motivation
• Human visual apparatus is an extremely efficient „processing machine“
• Enormous amounts of information are transferred by the visual nerve into the brain cortex
Extremely high bandwidth
• Visual cortex remains unbeatable in recognition of objects and complex patterns (e.g. rotational invariance)
• Pre-attentive processing
7
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Pre-attentive Processing
• Capability to process certain visual information without focusing our attention
• Criterion 1: Processing time < 200 - 250ms
Eye movements in about 200ms single glimpse
Highly parallel processing
• Criterion 2: Processing time does not correlate with the amount of noise in the data
• Limited number of pre-attentive features
Size (length/width), number, colour, intensity, curvature, orientation, flicker, motion direction, 3D depth-cues…
8
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Pre-attentive Processing
It is immediately possible to determine which data set contains a red spot Pre-attentive processing possible
9
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Pre-attentive Processing
It is still possible to quickly determine where the red spot is Borderline case
10
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Pre-attentive Processing
Scanning is necessary to determine where the red spot is Pre-attentive processing not possible
11
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualization – Approach and Challenges
• Approach
Machines transform the data into a suitable graphical representation
Employ the human visual system for pattern recognition
• Challenges
How should the graphical representations look like (design)?
How to compute the graphical representation (algorithms)?
Which operations shall be supported on the graphical representation (interactivity)?
12
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualization - Definitions
Graphical representation of data, information and knowledge
Use of human visual system, supported by computer graphics, to analyze and interpret large amounts of data
The use of visual representation to aid cognition
“Transformation of the symbolic into the geometric.”[McCormick et al., 1987]
“The depiction of information using spatial or graphical representations to facilitate comparison, pattern recognition, change detection, and other cognitive skills by making use of the visual system. “ [Hearst, 2003]
…
13
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualization - Examples
14
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualization DesignRepresentation Forms
• Fundamental categories of visual representation:
Formalisms
Metaphors
Models
• Formalisms: abstract schematic representations
Defined by a designer
Users must learn how to read and interpret
Example: Percentage is represented by an arc
15
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualization DesignRepresentation Forms
• Metaphors: representations based on a real-world equivalent
Intuitive
User can understand the meaning through building analogies
Example: using the geographic map metaphor to represent similarity in non-spatial data
• Models: based on mental representations of the physical world
Data typically has a natural representation in the real world
Examples: visualization of sensory data in 3D, virtual 3D worlds
16
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
VisualizationData – Information - Knowledge
• Data/Scientific Visualization
• Information Visualization
• Knowledge Visualization
17
Data KnowledgeInformation
Representation complexity, applicability by humans
Machine processing capability
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
VisualizationData – Information - Knowledge
• Data
Formal representation of raw, basic facts
Have a fixed format: numbers, dates, strings,…
Have a fixed, predefined meaning (i.e. no interpretation required)
„3162“ – Hotel room number (not a telephone number)
• Information
Result of processing, manipulation and interpretation of data
May not have a fixed format (unstructured or semi-structured)
Meaning is determined by interpretation within some context
“A small mouse” – a computer or a field mouse? (determined by context)
18
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
VisualizationData – Information - Knowledge
• Knowledge
Identified, organized and as valid recognized information
Representations of reality through abstract, domain-dependent models
Represented by formalized conceptual systems: Taxonomies, Thesauri…
Ontologies are formally defined knowledge representations consisting of concepts, relations and rules (axioms)
19
Animal
Mouse is aLegs
has
Jerry is a
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
VisualizationSubdivision
• Data/Scientific Visualization
• Sensor data
• 3D spaces
• Knowledge Visualization
• Knowledge models, knowledge transfer
• Information Visualization
• Document content (text and multi-media)
• Multidimensional data sets
• Structures: hierarchies and networks (graphs)
• Temporal information
• Geo-spatial information
20
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Data/Scientific Visualization
• Visualization of simulation or sensory data
have a natural representation in the real, physical world
• Applications in physics, medicine, astronomy, automotive…
• Web technologies: HTML5 canvas/WebGL
21
Pressure coefficients [NASA] Coil magnetic field
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Knowledge Visualization
• Knowledge Visualization is about using visual representations to present and transfer existing (explicit) knowledge between people [Eppler]
• The focus is on structured knowledge spaces
Concepts, relations, facts, attributes
Navigation along structures present in the knowledge model
• Use of metaphors and formalisms
22
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Knowledge VisualizationExamples
23
Stairs of Visualisation [Eppler](Let‘s Focus: http://en.lets-focus.com/ )
Research Map [Bresciani]
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Knowledge VisualizationExamples
Gyro, Know-Center [Kienreich]
24
Cultural Heritage Visualization Ancient Theatres [Blaise]
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Knowledge VisualizationExamples
• Gyro: visualisation of Brockhaus encyclopedia articles
Viewed article in the center
Related articles around it
Icons encode article type (person article, geo-location article etc.)
• Cultural Heritage Visualization
Roman amphitheaters in France
Uses formalisms to describe characteristics and state
25
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Information Visualization
• Interactive visualization of abstract information spaces
Abstract information has no „natural“, real-world representation
Rely on metaphors and formalisms
• Goal: identifying patterns and relationships
Explorative analysis and navigation
Unveiling of implicit knowledge
• InfoVis Mantra [B. Shneiderman]
„overview first - zoom and filter - details on demand”
26
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Document Content Summary
MovieDNA [Ponceleon]TileBars [Hearst]TagClouds, Know-Center [Seifert]
27
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• Tag Cloud
• Keyword visualisation
• Size corresponds to importance of the keyword
• TileBars
• Shows where keywords/concepts occur in the document
• Darker color – more occurrences at a given position
• Easy to find interesting parts in large documents
• MovieDNA
• Analogous concepts for videos
28
Visualization Examples Document Content Summary
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Multidimensional Data
Scatterplot [Nowell] Parallel Coordinates [Inselberg]
29
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• Multidimensional data
• Each data item described by a large number of features
• Scatteplot
• Data elements represented by dots/icons
• Up to five dimensions
• Using 2 axes (x and y) and icon shape, color and size
• Parallel Coordinates
• Data elements represented by lines
• Each dimension is a separate y axis (many are possible)
• Line color an additional dimension
• Filtering along different dimensions
30
Visualization Examples Multidimensional Data
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Multidimensional Data Similarity - Text
Know-Center [Sabol et al.]
Galaxies (SPIRE), PNNL [Wise]
31
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• Example: unstructured data - text
• Text analysis to transform text to multidimensional data
• Dimension are words/concepts in the text
• Makes documents topically comparable
• Information landscape (metaphor)
• Provides a topical overview of a (large) document repository
• Encodes topical similarity through spatial proximity
• Spatial clusters (hills) represent group of documents with similar topics
• Visualises cluster distribution, outliers
• Automatically extracted labels specify topical regions of the info-
landscape
32
Visualization Examples Multidimensional Data Similarity
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Multidimensional Data Similarity - Images
Image Similarity Layouts [Rodden]
33
• 3 Layouts:
• Overlapping, non-overlapping, space-filling
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Hierarchies
TreeMaps [Shneiderman]Hyperbolic Tree (InXight) [Lamping]
34
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• Hyperbolic Tree
• Large hierarchies shown in hyperbolic space
• Information at the edges geometrically “compressed”
• Tree branches not in the focus are small but remain visible
• TreeMap
• Hierarchy represented by nested rectangles
• Size and color of a rectangle encode properties
35
Visualization Examples Hierarchies
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Hierarchies
36
InfoSky, Know-Center [Andrews et al.]
Circle Packing, D3 library
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• InfoSky
• Visualisation of large, hierarchically organised document collections
• Hierarchy shown as nested Voronoi areas
• Documents shown as dots
• Distance in the visualisation represents topical similarity
• Color-coding used to represent document properties
• Interactive navigation in deep, complex hierarchies
• Packed Circles
• Similar approach using nested circles
• Disadvantage (vs. InfoSky): screen area between circles is wasted
37
Visualization Examples Hierarchies
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Graphs
38
Gephy, https://gephi.org/
Narcissus (3D) [Hendley]
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• Node-link diagrams
• Nodes represent entities
• Links represent relationships between them
• Properties encoded by colour, icon, size/thickness
• Problem: link clutter for large number of relationships
• Graph layout methods
• Large, complex area of research
• Popular: force-directed placement
– Connected nodes exert attractive forces (spring model)
– Good for small graphs (not scalable)
– Does not solve the link clutter problem
39
Visualization Examples Graphs
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Graphs – Edge Bundling
40
Edge-Bundling [Holten & van Wijk]
Concept Networks [Kienreich] (Know-Center)
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• Edge Bundling reduces clutter enormously
• Bundle together edges that
• Propagate in the same direction
• Are close to each other
• Are of comparable length
• Overlap
41
Visualization Examples Graphs – Edge Bundling
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Temporal Data
42
LifeLines [Plaisant]Themeriver, PNNL [Havre]
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• Timeline / LifeLines
• Displays event or time intervals in chronological order
• Time flow along an axis
• LifeLines: application for patient histories
• ThemeRiver (StreamGraph)
• Thematic variations (trends) across several categories
• With of a “stream” represents the strength of the topical category at a
given time point
43
Visualization Examples Temporal Data
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Temporal Data
44
Spiral geometry [Carlis] Perspective Wall [Mackinlay]
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• Spiral geometry timeline
• Longer time intervals possible than with the classical timelines
• Suitable for periodic event visualisation
• Perspective Wall
• A timeline variant with 3D “distortion”
• Suitable for large time intervals
• Only the time interval in the focus is displayed in high-detail
45
Visualization Examples Temporal Data
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Spatial Data
46
LucentVision [Pingali 2001]
Planetarium, Know-Center [Kienreich]
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
Visualization Examples Geospatial Data
47
Google Maps
APA-Labs component, by Know-Center
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• Variable level of detail(LOD)
• Technique known from 3D environments
• Decrease complexity of representation for “far-away” objects
• Coarse-grained view of the whole data space
• Provide more details when zooming-in
48
Visualization Examples Geovisualisation
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Geovisualisation with Variable LODGoogle Maps
49
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Graph Visualisation with Variable LOD Mouse Anatomy Ontology (Overview)
50
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Graph Visualisation with Variable LOD Mouse Anatomy Ontology (zoomed in)
51
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualization ExamplesMultiple Data Aspects – Geo-Temporal
GeoTime, Oculus [Kapler]
52
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• Oculus GeoTime
• Combines geo-visualisation and a timeline in a 3D view
• Geovisualisation: x-y plane
• Time: z-axis
• Follow movements along space and time
• Easy to identify encounters/meetings
53
Visualization Examples Multiple Data Aspects – Geo-Temporal
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualization ExamplesMultiple Data Aspects – Visual Links in 3D Environments
Starlight, PNNL [Risch et al.]
54
Caleydo, ICG, TU Graz [Lex et al.]
Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)
• Visual Links
• Multiple visualisations showing different aspects of the data
• Shown in a 3D space
• Represent links between data elements in different visualisations
55
Visualization Examples Multiple Data Aspects – Visual Links in 3D Environments
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualization ExamplesMultiple Data Aspects – Coordinated Multiple Views
Coordinated Multiple Views
• Multiple visualizations “fused” into a single, coherent user interface
• Each visualization designed to convey a different aspect of the data
simultaneous navigation and analysis over multiple data aspects becomes possible
• Coordination of state and behavior
• interactions in one visual component influence all others
• Selection, filtering, visual properties, navigation, …
56
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Coordinated Multiple Views
Spotfire [Schneiderman]
57
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Coordinated Multiple Views
58
• Spotfire
One of the first commercial CMV-enabled InfoVis tools
Introduced coordination on database level (“Snap Together”)
• CODE Visualisation Wizard
Coordinated visualisation of linked data
Utilises semantic information to link data sets
Developed at Know-Center
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Coordinated Multiple ViewsCODE Visualisation Wizard
59
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Coordinated Multiple ViewsCODE Visualisation Wizard
60
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visual Analytics
61
• Motivation
• Machines and humans have complementary capabilities
• Abundance of data: problems too large to be addressed by visualization alone
• Limited resources of the visual front end
• Idea: combine machine processing with human capabilities in a suitable way and get the best of both worlds.
• Integrate humans in the analytical process
• Provide means for explorative analysis
• Let machines learn from humans
• Visual Analytics: a young research field (2005)
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visual Analytics
62
• Combines automatic methods with interactive information/data/knowledge visualisation to get the best of both worlds [Keim 2008]
• Focuses on interaction between humans and machines through visual interfaces to derive new knowledge
• Supports analytical reasoning facilitated by interactive visual interfaces [Thomas 2005]
Repository
New Insights and Knowledge
Algorithms Visualization
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visual Analytics
63
• Main Idea (Mantra): “analyse first – show the important – zoom, filter and analyse further – details on demand” [Keim 2008]
Initial analysis and visual pattern recognition
Posing a hypothesis
Further analysis steps (automatic and interactive)
Confirmation or rejection of the hypothesis: new facts
Confirm the expected, discover the unexpected
• Challenges [Keim 2009]
Balance between automatic and interactive analysis
Design of effective VA workflows
Scalability
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualisation in the Web
64
• AJAX lays the foundations – asynchronous requests
• Rich, responsive user interfaces
• HTML5 provides the basis for visualization in the Web
• New APIs
• Drag-and-drop
• WebWorkers – background processing
• Storage (FileSystem API, IndexedDB…)
• Cacheing (enables Web Apps)
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Visualisation in the Web
65
• Rendering
• Canvas
• SVG
• WebGL
• Logic and Interactivity
• JavaScript: high-performance engines (compile to native)
• Server-Client Web architectures fit the needs of Visual Analytics
• Model View Controller (MVC) architecture
• Data storage/crunching on the server
• WebSockets – bidirectional server-client communication
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Web Visualization Performance and Scalability
• Scalability limited by computing power of the client
SVG: hundreds of items, easy to program
Canvas: at least one order of magnitude better
WebGL: potentially millions of items
• How to scale to large (huge) data sets
Millions (or billions) of data elements
Utilise the power of the server (Visual Analytics)
66
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Web Visualization Toolkits and Rendering Libraries
• General visualisation
D3.js: "Data-Driven Documents" JavaScript visualization library libraryusing SVG
JavaScript InfoVis Toolkit: for creating Interactive Data Visualizations
• Drawing and rendering
Raphaël: JavaScript library for vector graphics
Paper.js: open source vector graphics scripting framework that runs on top of the HTML5 Canvas
EASELJS: makes working with HTML5 canvas easy (Flash-like API)
67
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Web Visualization Toolkits and Rendering Libraries
• Charting and plotting
NVD3: Re-usable charts for d3.js
gRaphaël: JavaScript library for creating charts
jqPlot: a plotting and charting plugin for the jQuery Javascriptframework
Flot: JavaScript plotting library for jQuery
Flotr2: JavaScript library for drawing HTML5 charts and graphs
Peity: jQuery plugin for converting element's content into a mini-chart
Google Chart Tools (free usage, but NOT OPEN SOURCE)
RGraph: free (for NON-COMMERCIAL use only) HTML5 charts
Highcharts JS: free (for NON-COMMERCIAL use only) javaScript chartslibrary
68
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Web Visualization Toolkits and Rendering Libraries
• Graphs and trees
arbor.js: a graph visualization library using web workers and jQuery
sigma.js: lightweight JavaScript library to draw graphs using HTML canvas
jsPhyloSVG open-source javascript library for rendering phylogenetic trees
GraphGL: a network visualization library for massive graphs based on WebGL and WebWorkers
• Maps and geovisualisation
Leaflet: Open-Source JavaScript Library for Mobile-Friendly Interactive Maps
Kartograph: lightweight framework for building interactive mapapplications
Polymaps: JavaScript library for image- and vector-tiled maps usingSVG
69
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Web Visualization Toolkits and Rendering Libraries
• Time series and temporal data
Rickshaw: D3-based JavaScript toolkit for creating interactive time series graphs
SIMILE Widgets: Open-Source Data Visualization Web Widgets
Cubism.js: D3 plugin for visualizing time series
dygraphs: open source JavaScript library for time series charts
Envision.js: library for creating interactive HTML5 visualizations
Timeline JS: beautiful, easy to use timelines
70
Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)
Web Visualization Toolkits and Rendering Libraries
• 3D
three.js: WebGL rendering back end with optional fallback on canvasor SVG
LibGdx: Java game development framework, compiles to all desktopand mobile platforms, including to JavaScript/Browser
• Multiple view coordination
Crossfilter: Fast Multidimensional Filtering for Coordinated Views
• Programming languages and environments
Processing.js: visual programming language and environment designedfor the web
71