E-Viz Towards an Adaptive Framework for Visualization on the Grid.

e-Viz

• Towards an Adaptive Framework for Visualization on the Grid

http://www.eviz.org/

e-Vizzers

UW Swansea

UW Bangor

U Manchester

U Leeds

• e-Viz is a three year joint research project funded by UK EPSRC

• Four partner universities:– University of Leeds– (Ken Brodlie, Jason Wood)

– University of Manchester – (John Brooke, Mark Riding)

– University of Wales, Bangor (Nigel John, Chris Hughes)

– University of Wales, Swansea– (Min Chen, David Chisnall, Mark

Jones, Nicolas Roard)


What problem are we trying to solve?

DataChoose

Visualization Algorithm

ChooseSoftware toImplementAlgorithm

ChooseHardwareto run on

Which areapplicable to

the data?

Which isthe mostsuitable?

What’sfastest?

Availability?

Domain Scientist Visualization Specialist e-Scientist


Why are we trying to solve this?

• Because not all potential Grid visualization users are experts in visualization and Grid technologies.


Why are we trying to solve this?

Visualization Experience

Grid Experience

Application Scientists

Visualization Specialists

Grid & Visualization Developers

e-Scientists


Putting Visualization on the Grid

• Can the Grid help? – it’s supposed to be:– heterogeneous in architecture– seamless and transparent in use– fault tolerant in operation – capable of adapting to the changing environment in

order to provide the best service.


What can the Grid offer?

• Heterogeneity: architectures

SMP

NUMA

Desktop

Specialised

Computation Rendering

Cluster



• Heterogeneity: operating systems

Linux Irix AIX

Solaris BSD HP/UX

Windows MacOS



• Heterogeneity: visualization software

AVS IRIS Explorer OpenDX

VTK rtrt Simian

Volumizer Matlab



• Heterogeneity: Implications

– There are a wide variety of architectures, operating systems and software applications used in visualization.

– Each has it’s own particular rewards and benefits, but we can’t expect every Grid user to be knowledgeable and proficient with each, or even to know which is the most appropriate for a given task.

– Want to be able to offer the power afforded by each application to all potential users of visualization.



• Seamlessness and transparency: Implications

– End users should only have to learn one interface, but still be able to benefit from the features offered by a wide range of applications.

– But each visualization application has it’s own user interface…

– … and every operating system looks and behaves differently



• Fault tolerance: Implications

– “A distributed system is one in which the failure of a computer you didn’t even know existed can render your own computer unusable” – Leslie Lamport

– “A Grid system is one in which the failure of a computer you didn’t even know existed goes by unnoticed”

– Users should be able to rely on the Grid



• Fault Tolerance - User Scenario… keyboard,

please



• Adaptation: Implications

– The world is dynamic and ever changing; so is the Grid.– Network loads and status– Queues on HPC machines– Runtime limits on HPC machines– CPU loading, both remotely and locally– A Grid system should adapt to cope with changes


How can we solve this problem?


What is Visualization?

VisualizationData Image



Filter Map RenderData Image




Controls


Common Abstract Interface


Controls




Controls

ComputationalSteering

FrameTransport

FormatConversion



VTKData Image

Controls


FrameTransport

FormatConversion AVS/ExpressIRIS Explorer



Data Image

Controls


FrameTransport

FormatConversion VTK rtrtAVS Simian



• Feasibility– can we create equivalent visualizations using different

software packages?


Equivalent Visualizations?


Abstraction

• Need to describe the pipeline itself – need an abstract visualization description language– gViz project gives us skML

Visualization


FrameTransport

FormatConversion


Abstraction

• Format Conversion– Most visualization software applications can already

read a wide range of data formats

Visualization


FrameTransport

FormatConversion


Abstraction

• Computational Steering

Visualization


FrameTransport

FormatConversion


Abstraction

• Computational Steering– RealityGrid or gViz APIs can be used to control a

running pipeline– Visualization applications must be instrumented to

expose their steerable pipeline parameters– Currently instrumented VTK and RTRT (real time ray

tracer)

– GUI created, which is really a specialised Computational Steering client

- Reads in a pipeline description and dynamically configures itself to show appropriate widgets


Abstraction

• Frame Transport

Visualization


FrameTransport

FormatConversion


Abstraction

• Frame Transport– Images can be rendered locally or remotely – in either

case, they need to be displayed on the user’s own machine, as well as any collaborators

– Have created a library to compress remote images and transport them back to the client for display. Visualization applications can be modified to use the library

– Range of image compression codecs– Local rendering supported via a rendering overlay– Linux and Windows clients


What do we still need?

• Decision making– What visualizations are applicable for a given input data

type– Which of the available hardware and software is most

suitable to implement such a pipeline– User may not know the answer to these questions –

provide assistance


Architecture

Software Store

Client Broker

Data Store

HPV Machine HPV Machine HPV Machine HPV MachineHPV Machine

WSRF

Globus

gViz

e-Viz


Architecture (Current)

Client Broker

HPV Machine HPV Machine HPV Machine HPV MachineHPV Machine

Software StoreData Store

WSRF

Globus

gViz

e-Viz


Demonstration 1

• gViz Pollution Demonstrator - shows– Active simulations as a data source– Computational steering of an active visualization– Common interface to servers


Demonstration 1


Demonstration 2

• Volume Render Demonstrator – shows– Heterogeneous access to multiple servers– (Almost) seamless switching between servers– Fault tolerance through redundancy


Demonstration 2


The Knowledge Gap

• The Broker has a knowledgebase which it will use to make informed decisions on pipeline choices.

• It’s empty!


How to Distribute?



How to Distribute?

Filter Map Render

Data Image


When to Distribute?

• Depends on– Where the dataset resides– How big the dataset is– Network bandwidth– Available machines– Chosen algorithm– Queue status– CPU capabilities– Graphical capabilities– …..


Other problems

• Single point of failure in the broker


Conclusions

• e-Viz has created a framework for visualization on the Grid that is:– Heterogeneous in architecture– Seamless and transparent in use– Fault tolerant– Adaptive (potentially)

• But much more to be done! (project runs another 16 months)


For more information

• See a live demo:– Tuesday 22nd, 14:00-16:30, White Rose Grid stand

• Project Web Site:– http://www.eviz.org

• email:– [email protected]

E-Viz Towards an Adaptive Framework for Visualization on the Grid.

Documents

Transcript of E-Viz Towards an Adaptive Framework for Visualization on the Grid.