PPL IVS ivs200915 - cs.uml.edugrinstei/InfoVisJournal-2009-2011/Information...the advantages of...

http://ivi.sagepub.com/Information Visualization

http://ivi.sagepub.com/content/8/3/180The online version of this article can be found at:

DOI: 10.1057/ivs.2009.15

2009 8: 180Information VisualizationKatja Einsfeld, Achim Ebert, Andreas Kerren and Matthias Deller

Knowledge Generation through Human-Centered Information Visualization

Published by:

http://www.sagepublications.com

can be found at:Information VisualizationAdditional services and information for

http://ivi.sagepub.com/cgi/alertsEmail Alerts:

http://ivi.sagepub.com/subscriptionsSubscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

http://ivi.sagepub.com/content/8/3/180.refs.htmlCitations:

by guest on May 22, 2011ivi.sagepub.comDownloaded from

http://ivi.sagepub.com/

http://ivi.sagepub.com/content/8/3/180

http://www.sagepublications.com

http://ivi.sagepub.com/cgi/alerts

http://ivi.sagepub.com/subscriptions

http://www.sagepub.com/journalsReprints.nav

http://www.sagepub.com/journalsPermissions.nav

http://ivi.sagepub.com/content/8/3/180.refs.html


Original Article

Knowledge generation throughhuman-centered information visualization

Katja Einsfelda

Achim Eberta

Andreas Kerrenb,∗ andMatthias Dellerc

aComputer Science Department, VisualizationLab, University of Kaiserslautern, PO BOX3049, D-67653, Kaiserslautern, Germany.bSchool of Mathematics and SystemsEngineering (MSI), Växjö University, VejdesPlats 7, Växjö, SE-351 95, Sweden.E-mail: [email protected] Center Human-CenteredVisualization (HCV), German ResearchCenter for Artificial Intelligence, DFKIGmbH, Trippstadter Strasse 122, D-67663,Kaiserslautern, Germany.

∗Corresponding author.

Received: 20 March 2009Revised: 25 April 2009Accepted: 27 April 2009

Abstract One important intention of human-centered information visual-ization is to represent huge amounts of abstract data in a visual representa-tion that allows even users from foreign application domains to interact withthe visualization, to understand the underlying data, and finally, to gain new,application-related knowledge. The visualization will help experts as well asnon-experts to link previously or isolated knowledge-items in their mental mapwith new insights. Our approach explicitly supports the process of linkingknowledge-items with three concepts. At first, the representation of data itemsin an ontology categorizes and relates them. Secondly, the use of variousvisualization techniques visually correlates isolated items by graph-structures,layout, attachment, integration or hyperlink techniques. Thirdly, the intensiveuse of visual metaphors relates a known source domain to a less known targetdomain. In order to realize a scenario of these concepts, we developed a visualinterface for non-experts to maintain complex wastewater treatment plants.This domain-specific application is used to give our concepts a meaningfulbackground.Information Visualization (2009) 8, 180--196. doi:10.1057/ivs.2009.15

Keywords: information visualization; semantic data; human-centered visualization;human-computer interaction; wastewater treatment

Introduction

While information visualization (InfoVis) research often concentrates onvisualization techniques for one isolated data type, some applications haveto deal with huge amounts of abstract and unstructured data of various datatypes. This might be one of the reasons why innovative InfoVis techniquesare only reluctantly used in some application domains. A human-centeredperspective on information visualization1,2 can help to get – among otherthings – an overview of isolated visualization techniques and to structurethem in such a way that they can be implemented in real-world applica-tions in a usable and modular manner. Therefore, one of the main chal-lenges is to integrate individual InfoVis techniques in a way that semanticrelations among them are intuitively clear even to non-expert users (thisis also a goal of the emerging field of Visual Analytics3).

In this paper, we describe several techniques and concepts we haveimplemented in a framework in order to meet this challenge. After a shortdiscussion on our domain background in the next subsection Domain back-ground, we present InfoVis techniques for the analysis of huge amountsof complex process data (section Information Visualization Techniques).Here, we have developed, integrated and made use of techniques that allowto relate individual techniques to each other. For example, we utilizedthe advantages of virtual reality and integrated InfoVis techniques intothis approach. By employing ontologies and semantic techniques, we wereable to order, structure and relate the heterogeneous input data into moremeaningful information.

© 2009 Palgrave Macmillan 1473-8716 Information Visualization Vol. 8, 3, 180–196www.palgrave-journals.com/ivs/



Knowledge generation through human-centered information visualization

In the section Human-Centered Design Aspects, wedescribe how our implementation meets human needs.The application of the InfoVis reference model4 allowsus to react on user interaction in a modular and context-sensitive way. With the help of additional conceptsderived from Human–Computer Interaction (HCI), suchas visual metaphors,5 we created a syntheses of all theseelements. An application that provides intuitive accessto information allows to gain knowledge on complexprocesses and smoothly reacts on user interaction. Thus,it provides an appropriate context-sensitive view.

Domain background

Our work is part of the research project KOMPLETT6

(‘komplett’ is German for ‘complete’). In this project, aconsortium of partners from universities and industrydevelops a small innovative WasteWater Treatment Plant(WWTP), which can be utilized in arid areas for decen-tralized water recycling purposes. WWTPs constitute aninnovative and strong scenario for applying and testingour interactive visualization concepts.

The overall idea of the WWTP is to collect the wastew-ater of a small housing area or hotel in two separate frac-tions – black water and gray water – and to treat thesefractions in two separate processes. Gray water can berecycled up to drinking water and treated black water canbe used, for example, as irrigation water or service waterfor toilets. From the perspective of the plant engineers,the challenge is to apply state-of-the-art wastewater treat-ment techniques like physical treatment, biological treat-ment (membranes) and chemical treatment to separatedwater fractions and – compared to ordinary WWTPs –really small amounts of water. Moreover, engineers had tofind out how to place all needed devices and tubes in thelimited area of, for example, four containers.

Plants like this will likely be located in remote areas andcannot be maintained on a daily basis by experts. Thus,the caretaker of a housing area or hotel has to do thedaily maintenance work. On the other hand, the tech-nical equipment and the processes in the plant are quitecomplex. Furthermore, the traditionally available infor-mation for WWTPs is unstructured, unconnected, hetero-geneous and available in diverse data types and mediadepending on manufacturers and previous maintainers.This makes it even more difficult for non-experts to under-stand, handle and maintain such a plant.

Visualization framework

The goal of our visualization tool is to support educa-tional, monitoring and optimization tasks of those tech-nically complex plants in an intuitive and user-adaptiveway. Bertelsen and Nielsen7 studied different theoreticalframeworks that offer ways of analyzing the work at amodern WWTP. They found that, in the case of WWTPs,deviations and situated action limits the formalization

in Computer Supported Cooperative Work (CSCW)-design. They concluded that ‘Design approaches enablingtechnical tailoring based on strong modularization andencapsulation [...] may help.’ Consequently, from theperspective of information visualization, the challengeof the project is to ensure the usability of the plant fornon-experts by developing a visualization system thatbridges the gap between the unstructured data and thenon-expert by creating intuitive visual interfaces andhuman-centered interaction possibilities.

Our framework is implemented in C++ and OpenGL.The functionality of the processes in the WWTP, thephysical layout of the plant, links to measured data fromthe plant, and additional information on the plant areorganized in an ontology that can be accessed by theapplication in order to visualize single items in theircontext. The process data acquired from the plant arestored in a database. In general, an ontology is a descrip-tion of a specific domain of interest, including concepts,categories, entities, attributes and interrelationships. Incomputer science, ontologies can be used for the retrievalof semantics (through the relations) or for automaticreasoning. A detailed explanation of ontologies can be

found in the work of Uschold and Grüninger.8 Ceccaroniet al9 made use of ontologies in the domain of WWTPs,and Bosca et al10 proposed a 3D visualization techniquefor ontologies.

Information Visualization Techniques

In this section, an overview of InfoVis techniques fordiverse data types available in our application is provided.This ranges from visualization techniques for large quan-titative data sets to graph visualizations to the visualiza-tion of semantics and relations.

Quantitative process data

There are large quantities of process data in our applica-tion area, such as temperature, pressure or nitrate concen-tration measured at the plant. Each value can be qualifiedwith a timestamp and an item-ID. The item-ID is, with thehelp of the ontology, associated to its semantic context.For example, using the device which recorded an item-ID, we can identify its type and value (temperature, pres-sure, ... ), the physical and the logical part of the plant orprocess to which the measuring device belongs, availabledocumentation for that device and so on.

Systems that have to accomplish with huge amountsof data usually have to decide whether they displayone detail view at a time (time multiplexing) or show allinformation in different parts of the screen (space multi-plexing) simultaneously.4,11,12 While space multiplexingcan lead to cluttered views, users of time multiplexingtechniques are in risk of losing the context. We proposefor our concrete problem the use of animation as a solu-tion to this space vs time multiplexing dilemma. 3D and

© 2009 Palgrave Macmillan 1473-8716 Information Visualization Vol. 8, 3, 180–196 181



Einsfeld et al

animation provide one possibility to visualize moreinformation in a ‘natural’ focus+context style in ourspecial case. By choosing metaphors derived fromcommon concepts of the wastewater treatment domain,a more intuitive and human-centered access to informa-tion can be accomplished.

Color scale is highly configurable in our framework. Theinterface designer can choose between various objects ofa class hierarchy, for example, two-color gradients, multi-color gradients and gradients that consist of two or threeindividual gradients divided by one or two thresholds. Thedefault gradient we used consists of a dark-blue to light-blue two-color gradient in the lower 20 per cent, a dark-green to light-green to brown multi-color gradient in themiddle, and a light-red to dark-red two-color gradient inthe upper 20 per cent of the value spectrum. This decisionwas taken by two considerations: the findings on colorscales by Rheingans13 and Light and Bartlein14 show thatrainbow color scales are hard to interpret because of thefact that the order of multiple colors is not perceived preat-tentively and that various degrees of luminance improvethe quality of a color scale. The second consideration wasthat the application area of process data monitoring andanalysis often requires a qualitative interpretation of thedata. To ensure a human-centered interface, we decidedto separate the color scale with two thresholds in threevisually distinctive parts to indicate that some values areprecariously low, high or in an acceptable range. All ourdiagrams use the same color scale that can be adjustedwith a color scale widget. While the programmer has thepossibility to modify the colors and the type of the colorscale as described above, the user can only change theupper and lower threshold with the help of the widget.

The concept of process visualization in our frameworkis hierarchical as well as modular. There is a toolbox ofmetaphors for process data visualization from which theuser can easily choose and combine the most appro-priate metaphors, depending on current context andpersonal preferences. Some metaphors are used to gainoverview over all parameters (Figure 1); some are used tocompare the behavior of selected parameters (Figure 2);some are used to analyze the behavior of one parameterover time (Figure 3); finally, there are also focus views fordetailed analysis.

In order to get an overview over process data from thewhole process, we developed a 4D (3D plus time) visu-alization with animated bars as shown in Figure 1. Eachbar represents one parameter. The parameter values aremapped to the height of the bars and their color. The usercan specify the time interval of interest. When the anima-tion is started, a time slider moves from start-time to end-time, the values of the parameters change according to thecurrent animation time, and the bars move up and downdepending on the current value. The spatial order of thebars is determined by semantic information gained fromthe ontology: on the ground plane, the x-axis lists theparameter types and the y-axis the physical plant parts.The items of both axes can be rotated in order to move

other bars to the front or focus position. The thickness ofbars represents the changing behavior of the values of thatparameter, that is, the result of the ratio deviation/averagevalue is mapped to the radius of the bars.

The lower part of Figure 1 shows a detail view of thismetaphor that helps to explain the additional historyrings: the last five values before the current time areindicated with rings. The elevation (z-position) of thecenter of the rings as well as their colors indicate pastvalues. To indicate the age of the data, rings are gettingbroader and less tall over time, using an intuitive visual‘dissolving’ metaphor. When animated, the history ringsfollow the corresponding bar as they move up and down.Their dynamic behavior reminds of the natural behaviorof concentric waves on a water surface emanating fromthe center. This visual metaphor allows the user to preat-tentively distinguish increasing values (cone shape),decreasing values (upside-down cone shape), constantvalues (compressed sphere) and fast changing values(stretched cone).

Spotting patterns and comparing parametersThe Color Plane visualization of process data, shown inFigure 2(a), is comparable to other well-known InfoVisapproaches, like the technique presented by Kincaid andLam.15 The idea is to map a value v not to the y-axis, but toa color code only. Thus, the y-axis can be used to visualizea consecutive sequence of these colored bars. Color Planesvisualize a value v of one parameter in the time interval i(for example, 4 weeks) that are subdivided in n equallysized intervals ji,k (for example, days) from k = 1, . . . , n.Each of the intervals ji,k is represented by a single color-coded horizontal bar in the plane. Thus, the x-axis showsthe time-line from the start to the end of each interval ji,k,while the y-axis depicts the order of the ji,k-intervals inthe i interval. Compared to traditional approaches wherevalues are mapped to the y-axis, this technique visual-izes more space-filling and allows the user to discovercommon patterns or irregularities when comparing theintervals ji,k.

We extended this approach by making use of the thirddimension and animation in order to be able to comparemultiple parameters: Figure 2(b) shows our so-calledProcess Data Rolls. Each roll represents one parameterwith its Color Plane mapped around a cylinder. Thecylinders can simultaneously be rotated which brings aspecific time of interest from an interval ji,k (for example,evenings of all days) to the front. Due to the 3D shapeof the cylinders, a natural and intuitive focus + contexteffect results from the perspective distortion. Positive userfeedback inspired us to add vertical lines and horizontalrings that further improve the analysis of interrelatedbehavior across various parameters.

Comparing consecutive intervals and average valuesFigure 3 shows two visual metaphors derived from dailylife objects: the Rotary Diagram (a) and the Conveyor

182 © 2009 Palgrave Macmillan 1473-8716 Information Visualization Vol. 8, 3, 180–196




Figure 1: (a) 4D process overview and (b) detail extract.

Belt (b). Using simple metaphors that are easy to interactwith mostly results in an intuitive use. Both metaphorspresented in Figure 3 visualize the value of one parametersubdivided in n intervals ji,k. In contrast to Color Planes,the traditional y-axis representation of process data iskept. Thus, users that prefer this representation can alsofind a suitable diagram type in our toolbox.

The time vs space multiplexing dilemma is again solvedwith the help of 3D and animation: the semitransparentstyle of each diagram allows the human visual system todirectly compare consecutive diagrams. The orange rings

in the Rotary Diagram and the rods in the Conveyor Belt,which represent the average value of all intervals ji,k in aspecific time segment i further enhance the comparabilityand the possibility to discover outliers.

Graph-based data

Data that can be structured in a graph-based form, thatis, in nodes and edges, are common in our applicationarea: there are instructions, the flow direction of thewastewater through the plant, the hierarchical order of




Einsfeld et al

Figure 2: (a) Color Plane and (b) Process Data Rolls.

plant parts and technical devices, or the order of processsteps. As the expert knowledge is stored in the form of anontology, there are a manifold of possibilities to extractgraph structures, such as class hierarchies, relations ofinstances of one class or relations between instances fromdiverse classes depending on the current context.

In our application, we used various visualizationapproaches for graph-based data. Depending on theirpurpose, they vary in their degree of detail or overview,dynamics or fixed positions, abstractness or virtualrealism, and 3D. Our framework architecture allows usto reuse some of the elements, for example, the edges





Figure 3: (a) Rotary Diagram and (b) Conveyor Belt.




Einsfeld et al

Figure 4: Visualization of graph-based data.





with animated arrows, in multiple graph types, see forexample Figure 4.

Figure 5(a), shows a 2D overview visualization of theprocess steps and their order in water-flow direction,which was generated with the help of the open-sourcelibrary Graphviz.16 In Figure 5(b), a 3D detail visu-alization of the process is visualized. The focus nodeis located in the front and center position, and theconnected nodes are positioned in circles in a cone-like shape around the center. To change the currentfocus, the user can click on one of the nodes on thetwo circles, which smoothly moves the selected node tothe focus position and draws connected nodes aroundit. This view allows a good perception of the focusednode and its direct context. The user can switch betweenfocus view and overview, triggering an animated tran-sition. Both of these graph layouts calculate the posi-tion of the nodes dynamically depending on the graphstructure.

Figure 4(a), shows the visualization of wastewater flow(edges) through the plant parts (nodes). In this case, thepositions of the nodes are fixed, as they reflect the phys-ical position of the plant parts. Compared to the realplant layout, the plant parts are visualized as abstractboxes and in reduced size to avoid occlusion of the flowdirection indicators. The VR visualization of one plantpart, as shown in Figure 7, can also be seen as a graphvisualization because it visualizes elements of the plantand their connection via tubes. The purpose of this fixednode 3D visualization with a high degree of realism is,however, to promote the recognition of the physical plantand the interaction with it. In contrast, dynamic andabstract graph layouts are more useful for understandingand browsing complex processes.

Chart-based diagrams in graph-based contextIn applications with complex semantic contexts, it isbeneficial for mental integration to visualize chart-baseddiagrams within their semantic context. This context canoften be represented as an abstract graph structure or a3D visualization of real-world geometry. Figure 5 showstwo examples of integrating charts in graph structures. Inthe upper part, an overview visualization of the WWTP-process steps is displayed which was generated fromour ontology with the help of Graphviz. All tempera-ture measurements of the process are visualized as smallRotary Diagrams in a separate area. These are moved tothe front depending on the discrepancy between theircurrent and their average value. In that way, the user canidentify irregular behaviors at one glance. The semanticconnection between the diagrams and their context inthe graph is visually represented by connecting curveswhen either a diagram or a graph node is clicked, cp. thenext section on semantic data.

The lower part of Figure 5 shows a focus view of theprocess. All parameters measured at a specific process stepare visualized with the help of Focus Circles around the

nodes. A Focus Circle moves the clicked element in ananimated way around a circle to the top position whereit has the largest magnification. Thus, the user can easilyscroll through all charts of the process to look for irreg-ularities. Other application areas of Focus Circles can beseen in Figure 4(b), and Figure 6, where they are used withicons that allow to access additional information.

Semantic data

Semantic data are data that explains the category, contextor meaning of other data. Semantic data can occur, forexample, in the form of a structured set of variablesthat describe one data item (metadata) or in the formof numerous, diverse triples in the Resource DescriptionFramework (RDF) model. As semantic data can be usedto describe the categories or attributes of data items, theyusually deal with heterogeneous, previously unstructureddata of different type.

As indicated in the Introduction section, our appli-cation has to deal with huge amounts of heteroge-neous, semantically structured data. Our domain experts,together with information scientists, have developed anontology to categorize, describe and relate the informa-tion items (small isolated pieces of information) to eachother. This ontology contains information about

• processes in the WWTP,• technical components and their order in the process,• measured data,• documentation linked to the respective components,• instructions for maintenance linked with involved plant

parts and documentation, and• diagnosis elements with proposed solutions for critical

plant conditions.

This semantic information is exported in RDF-formatand then imported in a database in order to be fast anduniformly accessible from various applications.

Figure 6 shows one example of visualizing semanticdata. The available metadata for one technical compo-nent is organized as icons on a Focus Circle around thecomponent (right upper corner of the screenshot). Whenthe user clicks the icon in the top and focus position ofthe circle, the respective metadata is opened, that is, aPDF document with manufacturer information, an image,a video or a Color Plane with data from an process itemis shown. Textual metadata can be attached to nodesin the form of transparent or semitransparent labels asindicated in Figure 4(a). Also Figure 5 shows two possi-bilities of how semantic relations between diverse typesof information can be visualized. In the upper part ofthe image, the process-step nodes and the process dataare organized in separated areas and connected with 3Dedges when the user clicks one of the elements. In thelower part, the process data is displayed within the FocusCircles. Another possibility to visualize semantic data in a




Einsfeld et al

Figure 5: (a) Side by side visualization and (b) integrated visualization with Focus Circles.





Figure 6: Accessing abstract data through virtual reality interfaces.

human-centered way is to use the concept of modifiedvirtual reality.

Modified virtual realityIn a previous publication,17 we proposed a modified virtualreality as a technique to integrate and access informationin the VR visualization of the physical application domain(in our case the WWTP). This is a human-centered wayof visualization, because it bridges the gap between thephysical plant and the available abstract information thatnon-experts usually have to face. The user perceives theinformation directly integrated into the item to which itbelongs. Figure 7 shows two examples of direct informa-tion integration. In the upper part, some of the objectsare highlighted by using a signal color to indicate theconnection between two components. This can be usefulfor error detection or educational purposes. In the lowerpart of the image, the geometry of the tubes is modifiedwith animated blue rings that move along the tubes, visu-alizing the flow direction of water or air in the tubes.Other possibilities of modified virtual reality are hierar-chical extracts, animation (for example, pulsing behavior)or attached icons (see Figure 6).

Connections between various data types

There are many reasons why the visual representationof semantic connections between diverse data types andviews results in more human-centered applications thanthe isolated visualization of single data types. In aninformal evaluation we found that users seem to dislikeswitching between various views. They are reluctant to

remember how to invoke single views and when to usethem. They feel more comfortable with views that canintegrate additional data on demand or connect them toadditional visual representations of data needed in thecurrent context.

We also found that visualizing individual data typesin separated views does not help the user to sufficientlyunderstand the semantic connections between theseviews. Without any visual connections, it is hard to under-stand the relations, for example, between the physicallayout of the plant, the process data and the visualizationof process steps. Another argument for the visual repre-sentation of semantic connections is that it supports thehuman way of sensemaking. In order to understand newand complex situations, humans create mental maps byconnecting and relating known facts with new ones.18

The sections Graph-based data and Semantic dataalready described techniques for integrating and visu-ally connecting various data type visualizations in oneview. The integration of chart-based diagrams in theirgraph-based semantic context adds additional informa-tion that enables the user to understand and interpretthe chart depending on its context in the system.19 Wealso discussed how diagrams can be integrated in the 3Dvisualization of our application in order to bridge thesemantic gap between the physical layout of the plantand abstract data.

Visual integration vs hyperlinks of connectionsAnother approach of the visual representation of connec-tions between various data types implemented in ourapplication is an architecture that is similar to a web




Einsfeld et al

Figure 7: Modified virtual reality.

browser: information from the ontology is dynamicallylayouted, that is, depending on the context, semanticallyrelated information like diagrams, images, documents ortextual information is displayed in the same view, openedexternally when the user clicks the item, or hyperlinked

in another view. Hyperlinks are visualized as labels withsemitransparent background in the affordance color (seesection Visual metaphors). This approach is especiallyuseful in contexts where textual information is of centralimportance. In our application area there are at least two





Figure 8: Browser-like architecture in the diagnosis system.

tasks where this kind of visualization can be deployed:the diagnosis system and the maintenance instructions.

In the diagnosis system, there are possibilities to browseinformation depending on its category (documents,process data, images and so on) and to search the databasewith the help of textual search patterns. Moreover, there isa set of diagnosis situations that can be searched in orderto understand the reasons for problematic situations andto get some suggestions for possible solutions togetherwith additional information (for example, manufacturerdocuments or expert contacts). If there are formal descrip-tions of diagnosis situations available in the ontology,these situations are checked automatically with the helpof a parser implemented on the basis of the boost::spiritframework.20 As described above, the textual informationis dynamically layouted, and the additional informationis either automatically integrated in the view, shown ondemand, or hyperlinked in other views. Figure 8 showsa screenshot of the diagnosis system and the expert-knowledge database (knowledge from the ontology):process parameters are visualized as hyperlinks in a scrolllist (left hand side), and the diagrams of the parametersappear when the user clicks on the hyperlinks.

Figure 9 shows two screenshots of the daily maintenanceinstructions the caretaker of the plant has to follow. Theuser can click through all the steps, which are dynamicallylayouted in separate views and linked together. If thereis additional information available, such as a diagnosis,the caretaker has to validate process parameters, tech-nical components of the plant, documentation, images

or anything else that is linked, as described above. In theupper screenshot in Figure 9, the task of the caretaker isto validate the functionality of a measuring device. Thedevice is shown as an automatically integrated picture.The chart with the past values of this device appears whenthe user clicks the link. The lower screenshot of Figure 9shows the task of checking the functionality of a pump.In this instruction step, visual integration is utilized byanimating the 3D plant part containing the highlightedpump to the outside of the overall plant structure.

The next section provides more details on the human-centered implementation of connections betweendifferent data types.

Human-Centered Design Aspects

Modern Process Control Systems (PCS; for example,see web page21) support the staff during operation ofwastewater plants, for example, by automatization ofcontrol strategies and documentation of process data.But, the staff also depends on additional information likemanuals, service instructions, technical maps, knowledgeof colleagues, and so on. Because of the high diversityand the large amount of information necessary to char-acterize and operate a complex WWTP, it is importantto have access to the information directly at the place ofwork. In addition, in our case not only experts need tooperate the WWTP, but also non-experts have to shouldersome of the daily work.




Einsfeld et al

Figure 9: Browser-like architecture in the walk-through check.

To achieve these goals, it is firstly necessary to trans-form the available data into usable information, whichis available at any location and at any time. Secondly,common PCS have been developed for expert users only,such making the monitoring and control of WWTPs acomplex task. Therefore, in decentralized waste watertreatment approaches, new concepts for the user inter-faces are needed. They have to be tailored on theintended user group of non-experts, thereby givingthem the possibility to operate the WWTP withouta time- and cost-intensive training. The visualizationsystem has to provide as much information as possibleto the user, simultaneously taking care of not over-whelming him or her and presenting the informationas meaningfully as possible. This clearly points outthe need for interactive, human-centered visualizationmetaphors.

Translating raw data into information

The data state model developed by Chi22 and describedas InfoVis reference model by Card et al4 can be seenas a kind of Model-View-Controller pattern for InfoVisapplications. According to Card et al, a data transforma-tion firstly converts raw, abstract data into structured datatables. The structure used to represent the data has to besuitable for the chosen visualization. In a second step, avisual mapping is used to convert data to visual structures.Thirdly, view transformations convert these structures tospecific views.

This model helps to separate the four layers andto isolate dependencies, which allows to easily reusedifferent parts of our implementation. Thus, the rawdata can be restructured to create multiple visualiza-tions of the data source, and the visual structures can be





transformed to allow the user to gain various perspectiveson the visualization, for example, by using zoom, rota-tion or other operations for reorganizing or modifyingthe elements of a visualization.

From the perspective of interaction, applicationsimplementing the InfoVis reference model have severaladvantages. The modular and reusable structure of theapplication allows to create flexible and context sensitiveviews that can be adapted to the current user needs. Thus,the user can interact with the application at three diverselevels: data transformation, visual mapping and viewtransformation. This kind of interaction can easily beimplemented and handled in a modular way, which leadsto applications with numerous flexible interaction possi-bilities and less interaction constrains. In consequence,the user is not forced to understand what the machineis doing, but can choose between several intuitive visu-alization metaphors, depending on his/her current tasksand individual preferences.

As already indicated in the Introduction section, theraw data of our application consists of large amounts ofcomplex heterogeneous data that are categorized, struc-tured and related in the form of an expert knowledgebase in our ontology. This knowledge management isperformed by domain experts and information scientiststogether with the help of the ontology editor Protégé.23

Additional information in diverse media formats (processdata in database tables, videos, images, documentation,3D objects) is referenced from the ontology. The ontologyis exported in RDF-format with the help of the RSSDBtool24 and then imported into multiple database tables.In this way, the ‘raw data’ are already pre-structuredin order to be able to access the semantic informationuniformly from any context.

At the user interaction layer of the data transformationstep, the user chooses the visualization type (for example,VR visualization, diagnosis system or process data visu-alization) and calls for connected visualization items ifnecessary. The required process data is loaded from thedatabase into DiagramInput objects, where some prepro-cessing calculations are done, for example, the calcula-tion of minimum and maximum values. Semantic datathat are needed for the abstract graph visualization orany other graph operations is loaded in the Common-Graph data structure that stores node types, edge types,nodes, edges and the connections between them. In othercases, semantic data will simply be loaded in associativecontainers that will, for example, store the ID of a tech-nical component item as a key and the ID of informationitems available for that component as values. Moreover,there are specialized classes that can store special purposeinformation from the ontology.

In the user interaction layer of the visual mapping, theuser can decide what type of process data visualization (forexample, Rotary Diagram or ColorPlane) he/she wants tomap to the abstract data and what color scale to use forthis mapping. He/she can also decide how to interact withthe VR visualization (navigating through the hierarchy,

using modified-virtual-reality techniques, and so on), howto navigate graph-based views (for example, focused viewor overview), or how to interact with the browser-likevisualization for the diagnosis system or the maintenanceinstructions (call for linked information, follow links toother layouts).

At the user interaction layer of view transformationsthere are also multiple possibilities: our general conceptof manipulating the 3D view is the thought-wizard-navigation metaphor described by Einsfeld et al.25 Theidea of this metaphor is to relieve the user from theburden of moving and trying to orientate in 3D space andinstead give him/her the power to use simple gestures tomove and manipulate the objects in front of him/her.There are possibilities to scale, translate and rotate theobjects either with mouse gestures or with navigationcontrol tools. Ware and Franck26 found that the motionparallax depth cue is very important for the humanperception of 3D. Due to this fact we added another inter-action possibility – the shake metaphor – that is moreeasy to use than conventional rotation of objects withthe mouse: with simple triggers (gestures or key presses)the user can ‘shake’ objects, that is, an object or scene willrotate within a small angle around the x- or y-axis. Thissmall movement provides the motion parallax depth cueand allows the user to perceive 3D quickly and withoutconscious effort.

The next section describes the advantages of visualmappings from structured data to visual metaphors ifmeaningful metaphors are used.

Visual metaphors

Metaphors are used to compare a phenomenon a ofdomain da (the metaphor’s target) to a phenomenon b ofdomain db (the metaphor’s source). This helps humans tocreate a vivid mental map of a – especially in cases wheredomain da is more abstract than domain db, or the indi-vidual is more familiar with domain db than with domainda. Thus, metaphors help to extend knowledge aboutthings we understand to new domains of experience.

A visual metaphor is a metaphor in which visualattributes of the source are used to facilitate the under-standing of or the interaction with the target. The targetdoes not necessarily need to have a visual representa-tion. Thus, an abstract target can be mapped to a visualrepresentation with the help of the visual attributes ofthe visual metaphor’s target. The usefulness of visualmetaphors in information visualization, for example,for decision-making, has also been emphasized byGörg et al.19

Carefully designed visual metaphors enable a morenatural representation of abstract data that can intu-itively be interpreted by domain experts and laymen,thereby resulting in a human-centered access to complexdata. To be obvious to human beings it is required thatthe metaphor’s source stems from domains that are




Einsfeld et al

well-known from the user’s daily life and thus intuitivelyto understand. A visual metaphor is even more effectiveand intuitive in the human-centered sense if not only thevisual attributes but also the interaction-related attributesof the source object are ascribed to the target object. Thisresults in more coherent and convincing metaphors thatare intuitive not only to understand, but also to interactwith.

Risch27 elaborates on the advantages of visual metaphorsin information visualization. He states that ‘the degreeof structural alignment of a particular visualization withone or more corresponding image schemas accountsfor its perceived degree of intuitiveness.’ Image schemasare, according to Risch, ‘cognitively entrenched patternsof physical experience that theoretically serve as a keybridge between perception and cognition’ and are, thus,ideal candidates for metaphor-sources. This is becauseof the fact that they ‘serve to structure and constrainabstract reasoning processes via metaphorical projectionoperations that are grounded in everyday cognitive andperceptual experiences.’

In our application, we use multiple visual and navi-gation metaphors. The section Quantitative process datadescribed some of the visual metaphors used for thevisualization of abstract process data: the rings in themaster diagram (Figure 1), Color Planes and Process DataRolls (Figure 2), and Rotary Diagram and Conveyor Belt(Figure 3). Additionally, the section Translating raw datainto information mentioned the thought-wizard andshake-navigation metaphors. The VR view of the plantis rather an analogy than a metaphor as it presents thereal physical layout of the plant in a very similar spaciallayout in the visualization. According to Risch, analogiesare, together with metaphors, one of the key aspects ofhuman cognition.

Gibson28 pointed out that affordances (visual items thatare perceived as ‘i can do something with it’) are impor-tant in the creation of user-centered interaction inter-faces. The affordances of the metaphor’s source are usuallyattached to its target. Thus, the user intuitively knowshow to interact with the visual metaphor. In cases wherethis is not possible, we propose to apply an affordance color:a color that is used for control elements and any otheritems the user can interact with. While not being asintuitively understandable as natural affordances andrequiring introductory information about the color, itnevertheless gives the user a preattentive cue aboutusable objects. Figure 6 shows how some of the technicalcomponents in the VR view in semantic-informationmode are colored with the affordance color (in this caseblue). When the user clicks one of the blue devices, aFocus Circle with icons appears. Through the icons, theplant documentation, diagrams for measured parametersand possibilities to control the devices are accessible.

The section Connections between various data typesalready explained the importance of visual representa-tions of semantic information for human-computer inter-faces. The semantic approach bridges the gap between

previously unrelated data items; it supports the humanway of sensemaking and thus helps to transform datato knowledge. Our application provides the user withcontext-information on demand, visually interrelatessemantically related visual items, and supports the userwith extensible semantic search possibilities.

Evaluation and Comparison with OtherApproaches

To confirm the benefits of our implementation, we didan evaluation with the involved parties, both domainexperts and non-expert users (for example, caretakers)of the system. Since most of the employed metaphorsand techniques have no equivalent solutions in ourapplication domain, we postponed a formal comparativeevaluation in favor of an informal one done by the desig-nated users of the system. We have presented our inter-faces to a group of WWTP experts, who are used to workwith conventional PCS, and to a group of laymen in thedomain. At the first glimpse, they were doubtful whetherthe new visual metaphors will really be applicable in theirdomain. However, after a closer inspection of the newvisual functionalities, they quickly recognized the poten-tial of our visualization and interaction metaphors. Theyalso testified that they expect that the new metaphorsshould give non-experts the chance to easily handlea large portion of daily routine work. Our system iscurrently installed in a pilot plant at the FraunhoferUMSICHT in Oberhausen, Germany.29 This installationwill give us the possibility to gather more data about thepotential of the new metaphors in a real-world scenario.

As already mentioned, we have not yet conducted aformal evaluation of our approach. Moreover, we thinkthat this formal step will be very difficult to accomplish.The reason for this is that we could only compare oursystem with existing PCS that are focused on domainexperts as users. So our main target group, non-experts,will not be able to control a PCS in a reasonable time.Therefore, a comparative evaluation in this applicationdomain is expected to be far less conclusive than a fieldtest of the system. Outside the domain of waste watertreatment, there are, however, some newer frameworksthat integrate and connect diverse InfoVis techniquesin a real-world application. One typical example is thework of Jern and Franzen.30 The authors use multiplewindows with color-coded maps, scatterplots, parallelcoordinates and other well-known techniques to visualizesociological, economical and ecological data of Sweden.Compared to our approach, Jern’s framework, which is inthis respect similar to many other InfoVis applications,utilizes the multiple-view concept. An excellent startingpoint for related work of coordinated and multiple viewvisualization techniques is the annual conference serieson Coordinated & Multiple Views in Exploratory Visual-ization or the work of Roberts.31 The possibilities to visu-ally relate individual visualizations to each other is often





not sufficient: instead of using brushing, glyphs or directfeedback of user interactions in all windows simultane-ously, our approach offers intuitive possibilities of visualrelations like spacial layout, attachment, integration orhyperlinked visualizations.

There are even less examples that combine InfoVistechniques with real-world geometry visualization ofthe application area: Kirner and Martins32 placed visualelements, such as circle diagrams, bars, text elements andmetaphors like books, in a walk-through environment inorder to visualize the behavior of visitors of a museum inthe virtual analogon of the real environment. Althoughthere is no formal evaluation available, their approach ofplacing information visualizations in a VR world seemsto be less intuitive than our techniques of integratinginformation.

Conclusion

In our informal evaluation, we found that our visualmetaphors help users to understand their meaning andinteraction possibilities and, additionally, increase thejoy of interacting with them. Especially, 3D supportsthe creation of perspective focus+context views and itscombination with transparency and/or animation allowsthe framework to visualize our application data intu-itively. It also facilitates mental integration of data insidea naturally 3D context (the plant simulation) as well asvisual comparison of data sets. Compared to commontime or space multiplexing techniques, details andcontext remain perceivable. Animation enables the userto understand and keep track of changes, or to compareconsecutive items. Moreover, the use of animation ampli-fies the impression of natural objects in 3D space whichresults in an immersive visualization and enables a morehuman-centered interaction with the application.

Besides these methods, we also explained how visualtechniques, for example integrating chart-based diagramsinto their (graph-based) semantic context, modifiedvirtual reality, information integration or browser-likevisualization, can help to understand the complexsemantic relations between isolated information items.The user can interpret the information item in its context,which leads to a deeper and more intuitive understandingof the whole information system and the interrelation ofindividual information items.

We used ontologies as a semantic data storage techniqueand Card’s InfoVis reference model to ensure the flex-ibility and natural user-interaction possibilities. Finally,we described how we integrated these techniques andconcepts in our application in order to create a flexibleand human-centered interface, intuitively usable by non-experts as well as experts of the domain.

This research is part of the interdisciplinary projectKOMPLETT, which is supported by the German FederalMinistry of Education and Research (BMBF, grant number02WD0685). We thank our colleagues from the Institute

of Urban Water Management of the Technical Universityof Kaiserslautern for sharing their expert knowledge andwork on the ontology.

References

1 Kerren, A., Ebert, A. and Meyer, J. (eds.) (2007) Human-CenteredVisualization Environments, GI-Dagstuhl Research Seminar,Dagstuhl Castle, Germany. 5–8 March 2006, Revised Lectures,Vol. 4417 of Lecture Notes in Computer Science. Berlin;Heidelberg: Springer.

2 Kerren, A., Stasko, J.T., Fekete, J.D. and North, C. (eds.) (2008)Information Visualization, Human-Centered Issues and Perspectives,Vol. 4950 of Lecture Notes in Computer Science. Berlin;Heidelberg: Springer.

3 Keim, D., Andrienko, G., Fekete, J.D., Görg, C., Kohlhammer, J.and Melançon, G. (2008) Visual analytics: Definition, process,and challenges. In: A. Kerren, J.T. Stasko, J.D. Fekete, &C. North (eds.), Information Visualization, Vol. 4950 of LectureNotes in Computer Science, Berlin; Heidelberg: Springer,pp. 154–175.

4 Card, S.K., Mackinlay, J.D. and Shneiderman, B. (1999) Readingsin Information Visualization: Using Vision to Think. San Diego, CA:Academic Press.

5 Catarci, T., Costabile, M.F. and Matera, M. (1995) Visualmetaphors for interacting with databases. SIGCHI Bull 27(2):15–17.

6 Project KOMPLETT (2009), http://www.komplett-projekt.de/index.php?id=3601&L=en, accessed March 2009.

7 Bertelsen, O.W. and Nielsen, C. (1999) Dynamics in wastewatertreatment: A framework for understanding formal constructsin complex technical settings. In: S. Bødker, M. Kyng andK. Schmidt (eds.) Proceedings of the Sixth European Conferenceon Computer Supported Cooperative Work. Norwell, MA, USA:Kluwer Academic Publishers, pp. 277–290.

8 Uschold, M. and Griininger, M. (1996) Ontologies: Principles,methods and applications. Knowledge Engineering Review.Cambridge University Press, pp. 93–136.

9 Ceccaroni, L., Cortés, U. and Sànchez-Marrè, M. (2004)OntoWEDSS: Augmenting environmental decision-supportsystems with ontologies. Environmental Modelling and Software19(9): 785–797.

10 Bosca, A., Bonino, D. and Pellegrino, P. (2005) OntoSphere:More than a 3D ontology visualization tool. In: P. Bouquetand G. Tummarello (eds.) Proceedings of SWAP 2005, the 2ndItalian Semantic Web Workshop; 14--16 December 2005, Trento,Italy. CEUR Workshop Proceedings. ISSN 1613-0073, onlinehttp://ceur-ws.org/Vol-166/.

11 Fekete, J.D. and Plaisant, C. (2002) Interactive informationvisualization of a million items. In: P. Chung Wong andK. Andrews (eds.) INFOVIS ’02: Proceedings of the IEEE Sympo-sium on Information Visualization (InfoVis’02) Washington DC,USA: IEEE Computer Society, p. 117.

12 Hart, D. and Tudoreanu, M.E. (2001) Visualization channels:Time multiplexing on a display. In: M.H. Hamza (ed.), VIIP ’01:Proceedings of the Conference on Visualization, Imaging andImage Processing. Washington DC: ACTA Press, pp. 95–100.

13 Rheingans, P. (1999) Task-based color scale design. In: Proceedingsof Applied Image and Pattern Recognition. Bellingham, WA:SPIE – The International Society for Optical EngineeringVol. 3657, pp. 35–43. citeseer.ist.psu.edu/rheingans99taskbased.html.

14 Light, A. and Bartlein, P.J. (2004) The end of the rainbow?Color schemes for improved data graphics. EOS Transactions 85:385–391.

15 Kincaid, R. and Lam, H. (2006) Line graph explorer: Scalabledisplay of line graphs using Focus+Context. In: A. Celentano andP. Mussio (eds.) AVI ’06: Proceedings of the Working Conferenceon Advanced Visual Interfaces, New York. USA: ACM Press,pp. 404–411.




Einsfeld et al

16 Graphviz–Free Open Source Graph Visualization Software (2009),http://www.graphviz.org/, accessed March 2009.

17 Einsfeld, K., Ebert, A. and Wölle, J. (2008) Modified virtual realityfor intuitive semantic information visualization. In: E. Banissi,L. Stuart, M. Jern, G. Andrienko, F.T. Marchese, N. Memon,T.G. Wyeld, R.A. Burkhard, G. Grinstein, D. Groth, A. Ursyn,C. Maple, A. Faiola and B. Craft (eds.) IV ’08: Proceedings ofthe 12th International Conference on Information Visualisation.Washington DC, USA: IEEE Computer Society, pp. 515–520.

18 Cosgrove, D. (1999) Mappings. London: Reaktion Books.19 Görg, C., Pohl, M., Qeli, E. and Xu, K. (2006) Visual

representations. In: A. Kerren, A. Ebert, & J. Meyer (eds.), Human-Centered Visualization Environments, Vol. 4417 of Lecture Notes inComputer Science. London: Springer, pp. 163–230.

20 boost::spirit – An object oriented recursive descent parserframework implemented using template meta-programmingtechniques (2008), http://spirit.sourceforge.net/, accessed March2009.

21 Siemens Water and Wastewater Technologies (2009),http://www.water.siemens.com, accessed March 2009.

22 Chi, E.H. (2000) A taxonomy of visualization techniques usingthe data state reference model. In: INFOVIS ’00: Proceedingsof the IEEE Symposium on Information Vizualization 2000.Washington DC, USA: IEEE Computer Society, pp. 69–76.citeseer.ist.psu.edu/chi00taxonomy.html.

23 Protégé – Free Open Source Ontology Editor (2009),http://protege.stanford.edu/, accessed March 2009.

24 RSSDB – The RDF Schema Specific DataBase (2006),http://139.91.183.30:9090/RDF/RSSDB/, accessed March 2009.

25 Einsfeld, K., Agne, S., Deller, M., Ebert, A., Klein, B. andReuschling, C. (2006) Dynamic visualization and navigation ofsemantic virtual environments. In: E. Banissi, R.A. Burkhard,

A. Ursyn, J.J. Zhang, M.W. Mck. Bannatyne, C. Maple, A.J.Cowell, G.Y. Tian and M. Hou (eds.) IV ’06: Proceedings ofthe 10th International Conference on Information Visualisation.Washington DC, USA: IEEE Computer Society, pp. 569–574.

26 Ware, C. and Franck, G. (1996) Evaluating stereo andmotion cues for visualizing information nets in threedimensions. ACM Transactions on Graphics 15(2): 121–140citeseer.ist.psu.edu/ware96evaluating.html.

27 Risch, J.S. (2008) On the role of metaphor in informationvisualization. The Computing Research Repository (CoRR)abs/0809.0884.

28 Gibson, J.J. (1979) The Ecological Approach to Visual Perception.Hillsdale, NJ: Lawrence Erlbaum Associates.

29 Fraunhofer UMSICHT – Installation of the pilot plant (2009),http://www.umsicht.fraunhofer.de/englisch/presse/bericht.php?titel=080521_komplett_lieferung, accessed March 2009.

30 Jern, M. and Franzen, J. (2007) Integrating Info Vis andGeoVis components. In: E. Banissi, R.A. Burkhard, G. Grinstein,L. Stuart, T.G. Wyeld, G. Andrienko, J. Dykes, M. Jern, A. Faiola,D. Groth, A. Ursyn, A.J, Cowell and M. Hou (eds.) IV ’07:Proceedings of the 11th International Conference InformationVisualization. Washington DC, USA: IEEE Computer Society,pp. 511–520.

31 Roberts, J.C. (2004) Exploratory visualization with multiplelinked views. In: A. MacEachren, M.J. Kraak, & J. Dykes (eds.),Exploring Geovisualization. Amsterdam: Elseviers, pp. 149–170.http://www.cs.kent.ac.uk/pubs/2004/1822.

32 Kirner, T.G. and Martins, V.F. (2000) Development of aninformation visualization tool using virtual reality. SAC ’00: In:J. Carroll, E. Damiani, H. Haddad and D. Oppenheim (eds.)Proceedings of the 2000 ACM Symposium on Applied Computing. NewYork, USA: ACM, pp. 604–606.




PPL IVS ivs200915 - cs.uml.edugrinstei/InfoVisJournal-2009-2011/Information...the advantages of...

Documents

Transcript of PPL IVS ivs200915 - cs.uml.edugrinstei/InfoVisJournal-2009-2011/Information...the advantages of...