Mouse BIRNMouse BIRN

- Jagadeeswaran Rajendiran- Jagadeeswaran Rajendiran

Pipeline Processing Pipeline Processing EnvironmentEnvironment

Motivation …Motivation …

Algorithms that have been implementedAlgorithms that have been implemented• for different computer architecturesfor different computer architectures• with varying input/ output needswith varying input/ output needs

Data characteristicsData characteristics• high volumeshigh volumes• located on different machineslocated on different machines• have different formatshave different formats

Computational needsComputational needs• Complex preprocessing/ analysis tasksComplex preprocessing/ analysis tasks• Single, flexible all-encompassing frameworkSingle, flexible all-encompassing framework• High volume of measurements/ resultsHigh volume of measurements/ results

It would be nice to auto-store intermediate results!It would be nice to auto-store intermediate results!

Motivation Motivation

Researchers are expected to have:Researchers are expected to have:• Expert knowledgeExpert knowledge

ToolsTools• Access to computing powerAccess to computing power

Collaboration is difficultCollaboration is difficult• data and analyses data and analyses

(communication)(communication) Seamless translation between Seamless translation between

data formatsdata formats Visualization frameworkVisualization framework

• PortablePortable• Context sensitiveContext sensitive

LONI Pipeline LONI Pipeline

Processing EnvironmentProcessing Environment• data flow execution environmentdata flow execution environment• links independent programslinks independent programs• auto-parallelizes sectionsauto-parallelizes sections• repeatable on different data-setsrepeatable on different data-sets• Uses:Uses:

Integrate your program as part of an Integrate your program as part of an analytic processanalytic process

Send a collaborator an analytic processSend a collaborator an analytic process Validate data formatsValidate data formats Parallel batch processing with error Parallel batch processing with error

handling handling

ArchitectureArchitecture

ModularModular Generic domain of applicationsGeneric domain of applications Platform-independent (Java)Platform-independent (Java) Extensible and flexible designExtensible and flexible design Clean code base Clean code base

• No unnecessary dependenciesNo unnecessary dependencies Rapid prototyping of new Rapid prototyping of new

functionalitiesfunctionalities Single encompassing GUISingle encompassing GUI

• User friendlyUser friendly• Data centricData centric

Feature rich GUIFeature rich GUI

• Zoom• Pan• Connections• Tooltips• Module Lists• Runtime updates• Workspaces

Execution EnvironmentExecution Environment Executes modulesExecutes modules ParallelizationParallelization

• Data-independent sectionsData-independent sections• If inputs are active and readyIf inputs are active and ready

Data-centricData-centric• Intermediate data-setsIntermediate data-sets• Can save for later analysis: OntologyCan save for later analysis: Ontology

Remote ExecutionRemote Execution• Client-Server using Java RMIClient-Server using Java RMI

Seamless integration (external Seamless integration (external apps)apps)

Data VisualizationData Visualization

Additional functionality

Is integrated via the extension

architecture.

• Viewer Extensions• Configurable

Data MediationData Mediation

• Seamless• Format conversion

Grid Engine IntegrationGrid Engine Integration

Horizontal vs. Vertical Parallelization

• VERTICAL— Pipeline

• HORIZONTAL— Grid

BREAK FOR DEMO!BREAK FOR DEMO!

Protocols and ApplicationsProtocols and Applications Data/ Functionality specificationData/ Functionality specification

• XMLXML Network protocolsNetwork protocols

• HTTP, FTPHTTP, FTP• Grid engine capabilitiesGrid engine capabilities

SUN Grid systemSUN Grid system SRB @ UCSDSRB @ UCSD

SecuritySecurity• Client - ServerClient - Server

Diffie Hellman key agreementDiffie Hellman key agreement Advanced Encryption Standard (AES)Advanced Encryption Standard (AES)

• User AuthenticationUser Authentication SHA-1 digest of passphraseSHA-1 digest of passphrase AES key from digestAES key from digest

ApplicationsApplications• SlicerSlicer• ShivaShiva

Upcoming features …Upcoming features …

Improved usabilityImproved usability Fault tolerance (system & Fault tolerance (system &

network)network) EfficiencyEfficiency

• Overlay grid implementationOverlay grid implementation ProvenanceProvenance

• Data validationData validation• Information recoveryInformation recovery

ResourcesResources

Wiki-PageWiki-Page• http://www.http://www.loniloni..uclaucla..edu/twiki/bin/view/Pipeline/edu/twiki/bin/view/Pipeline/

PersonnelPersonnel• Dr. Arthur Toga: Dr. Arthur Toga: toga@loni.ucla.edutoga@loni.ucla.edu• Michael Pan: Michael Pan: mjpan@loni.ucla.edumjpan@loni.ucla.edu• Ron Nor: Ron Nor: rnor@loni.ucla.edurnor@loni.ucla.edu• Jags: Jags: r.jagadeeswaran@loni.ucla.edur.jagadeeswaran@loni.ucla.edu

THANK YOU