Grid Computing Framework A Java framework for managed modular distributed parallel computing.
-
Upload
eustace-bryan -
Category
Documents
-
view
222 -
download
3
Transcript of Grid Computing Framework A Java framework for managed modular distributed parallel computing.
![Page 1: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/1.jpg)
Grid Computing FrameworkA Java framework for managed modular distributed parallel computing
![Page 2: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/2.jpg)
Presentation Plan
1. Discussion on distributed computing(Concept, examples, insight)
2. What are we doing (new?)(Our concept, innovation, vision)
3. Who is doing what(Technical details, responsibilities, SE Practices)
4. What remains to be done(Or as they say – ‘future scope’)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 2
Grid Computing Framework
![Page 3: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/3.jpg)
Parallel Processing (problem classes)1. SIMD
(Single Instruction Multiple Data)
Used mostly in DSP i.e.Digital Signal Processing applications
11/11/2010 Abhishek Ganguly | Apoorv Pandey 3
Grid Computing Framework – Discussion on distributed computing
Bass Boost
Left | Center | Right
![Page 4: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/4.jpg)
Parallel Processing (problem classes)1. SIMD
(Single Instruction Multiple Data)
2. MISD(Multiple Instruction Single Data)
Very few actually use or implement this aspect of parallelism
Task replication systems may use this technique for fault tolerance or error masking etc
11/11/2010 Abhishek Ganguly | Apoorv Pandey 4
Grid Computing Framework – Discussion on distributed computing
![Page 5: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/5.jpg)
Parallel Processing (problem classes)1. SIMD
(Single Instruction Multiple Data)
2. MISD(Multiple Instruction Single Data)
3. MIMD(Multiple Instruction Multiple Data)
Most commonly adopted distributed processing strategy
11/11/2010 Abhishek Ganguly | Apoorv Pandey 5
Grid Computing Framework – Discussion on distributed computing
![Page 6: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/6.jpg)
Parallel Processing (Solution Approaches)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 6
Grid Computing Framework – Discussion on distributed computing
![Page 7: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/7.jpg)
Parallel Processing (Solution Approaches)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 7
Grid Computing Framework – Discussion on distributed computing
Centralized parallelism
Common Memory (RAM)
Common Die(Multi Core)
Common Board(Multi Processor)
![Page 8: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/8.jpg)
Parallel Processing (Solution Approaches)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 8
Grid Computing Framework – Discussion on distributed computing
Centralized parallelism
Multi Core
Common Memory (RAM)
Common Die(Multi Core)
Common Board(Multi Processor)
• Common Die or Chip. Everything on a single Chip.• Common System bus.• Common L2 cache
• Individual L1 cache• Individual Register file• Individual ALU
• May or may not share a common Clock
![Page 9: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/9.jpg)
Parallel Processing (Solution Approaches)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 9
Grid Computing Framework – Discussion on distributed computing
Multi Processor
Common Memory (RAM)
Common Die(Multi Core)
Common Board(Multi Processor)
• Common Motherboard• Common OS instance• Common RAM
• Individual Clock• Individual System Bus• Individual RAM access
• Higher Throughput than multi core • Higher power consumption than multi core
![Page 10: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/10.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 10
Grid Computing Framework – Discussion on distributed computing
Distributed parallelismParallel Processing (Solution Approaches)
Reduced Coupling
![Page 11: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/11.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 11
Grid Computing Framework – Discussion on distributed computing
Distributed parallelism – SUPER COMPUTERParallel Processing (Solution Approaches)
• Highly coupled systems with specialized hardware built for the purpose• May be composed of hundreds of motherboards, each housing multiple CPUs• RAMs spread all over the housing• Interconnected by special high speed bus (Network)• Each board may have its own copy of the OS.
Requires• Special OS• Special software to run atop it• Special programming methodology
![Page 12: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/12.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 12
Grid Computing Framework – Discussion on distributed computing
Distributed parallelism – CLUSTERSParallel Processing (Solution Approaches)
•These fall between super computers and grid computers•These are networked computers•Computers are connected through high speed LAN•Each computer has its own instance of OS running for it
These are the highest deployed distributed systems.
Uses•Load balancing (as in web servers) •Availability, Robustness, fail-safe (DNS serves)•Throughput increase (database clusters)
![Page 13: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/13.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 13
Grid Computing Framework – Discussion on distributed computing
Distributed parallelism – GRID COMPUTERSParallel Processing (Solution Approaches)
• Most diffused, heterogeneous and independent form of multi node alliance. • Components of a grid computer can be anything from a commodity PC to clusters of supercomputers.• Components can be located al over the globe.• Internet is the basic networking medium between the components.
Requires• Special programming technique• Specially written software• Extensive GRID framework
![Page 14: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/14.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 14
Grid Computing Framework – Discussion on distributed computing
Grid Computer Architecture
Grid User(Programmer) Grid Coordinator Server
Workers
![Page 15: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/15.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 15
Grid Computing Framework – Discussion on distributed computing
Grid Computer Architecture
![Page 16: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/16.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 16
Grid Computing Framework – Discussion on distributed computing
Grid Computer Architecture
![Page 17: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/17.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 17
Grid Computing Framework – What are we doing (new?)
Project Proposal
To implement such a distributed framework in java
To adapt java multi-threading concepts for scalable distributed computing
![Page 18: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/18.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 18
Grid Computing Framework – What are we doing (new?)
Implementing Distributed Framework
Network
MachineMachine MachineMachine MachineMachine
JVMJVM JVMJVM JVMJVM JVMJVM
Grid Frame Work
UserProgram User
Sub-ProgramUser
Sub-Program
Grid CodeGrid Code Grid CodeGrid Code Grid CodeGrid Code Grid CodeGrid Code
Grid APIGrid API Grid APIGrid API Grid APIGrid API
![Page 19: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/19.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 19
Grid Computing Framework – What are we doing (new?)
Implementing Distributed Framework
Network
Machine Machine Machine
JVM JVM JVM JVM
Grid Frame Work
UserProgram User
Sub-ProgramUser
Sub-Program
Grid Code Grid Code Grid Code Grid Code
Grid APIGrid API Grid APIGrid API Grid APIGrid API
![Page 20: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/20.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 20
Grid Computing Framework – What are we doing (new?)
Adapting Multi Threading Concepts
Common Java Code
Common Java Code
![Page 21: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/21.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 21
Grid Computing Framework – What are we doing (new?)
Adapting Multi Threading Concepts
Grid Aware java
Code
![Page 22: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/22.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 22
Grid Computing Framework – Who is doing what, Technical details etc
Functional Objectives
Research
Design
Coding
Testing by using
![Page 23: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/23.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 23
Grid Computing Framework – Who is doing what, Technical details etc
Research on existing work
Basic understanding of parallel computingAbhishek & Apoorv
MPI, Java RMI , Directory and Naming servicesApoorv
Object Serialization, CORBA, Dot net RemotingApoorv
Existing grid computing frameworks. BOINC, OPEN-Grid, Abhishek
Distributed file systems, Distributed SynchronizationAbhishek
![Page 24: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/24.jpg)
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 24
Grid Computing Framework – Who is doing what, Technical details etc
Client
Job ServerWorker
![Page 25: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/25.jpg)
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 25
Grid Computing Framework – Who is doing what, Technical details etc
Client
Job ServerWorker
Client uses Grid APIorg.accurate.grid
org.accurate.grid.Job
Job
Runnable SerializableThread
void start()void Run()void join()
![Page 26: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/26.jpg)
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 26
Grid Computing Framework – Who is doing what, Technical details etc
Client
WorkerJob Server
ServerManages job Queue
&ManagesCompleted jobs’ bin
![Page 27: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/27.jpg)
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 27
Grid Computing Framework – Who is doing what, Technical details etc
Client
WorkerJob Server
ServerManages job Queue
&ManagesCompleted jobs’ bin
JobServer.submitJob( Job j );>>
![Page 28: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/28.jpg)
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 28
Grid Computing Framework – Who is doing what, Technical details etc
Client
WorkerJob Server
ServerManages job Queue
&ManagesCompleted jobs’ bin Job j=JobServer.getJob()>>
![Page 29: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/29.jpg)
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 29
Grid Computing Framework – Who is doing what, Technical details etc
Client
WorkerJob Server
ServerManages job Queue
&ManagesCompleted jobs’ bin JobServer.doneJob( Job j )>>
![Page 30: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/30.jpg)
Software DesignSystem Architecture
11/11/2010 Abhishek Ganguly | Apoorv Pandey 30
Grid Computing Framework – Who is doing what, Technical details etc
Client
WorkerJob Server
ServerManages job Queue
&ManagesCompleted jobs’ bin
Job j =JobServer.getCompleted(this)>>
![Page 31: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/31.jpg)
Software DesignData Flow Diagram 0
11/11/2010 Abhishek Ganguly | Apoorv Pandey 31
Grid Computing Framework – Who is doing what, Technical details etc
Grid FrameworkGrid Framework
Jobs
Completed Results
![Page 32: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/32.jpg)
Software DesignData Flow Diagram 1
11/11/2010 Abhishek Ganguly | Apoorv Pandey 32
Grid Computing Framework – Who is doing what, Technical details etc
1.2Job Queuing
and dispatching
1.2Job Queuing
and dispatching
1.1Job Creation
1.1Job Creation
1.3Job Execution
1.3Job Execution
Code
blockUser Program
User Program
Job Queue Completed Job Store`` ` `
User Program
User Program
Child job modules
![Page 33: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/33.jpg)
All that remains to be done
11/11/2010 Abhishek Ganguly | Apoorv Pandey 33
Grid Computing Framework – Future Scope
Virtual distributed file system.A distributed file system for high volume data interchange
Dedicated application servers.So that long running applications do notrequire their originating PCs to stay on.
Integration with existing grid computing systems.So that jobs can be interchanged between them.
Automatic parallelization.So that code can be parallelized withoutexplicit programmer effort.
![Page 34: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/34.jpg)
All that remains to be done…
11/11/2010 Abhishek Ganguly | Apoorv Pandey 34
Grid Computing Framework – Future Scope
Wrapper to precompiled codeSo that precompiled modules can bereused without reengineering
Worker credit trackingSo that workers can be gifted benefit a/c to work they do
Standardization of APISo that programmers of different platformscan feel comfortable.
![Page 35: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/35.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 35
Grid Computing Framework – References
90% of the research material has been obtained from the internetAnd rest 10% from text books
Text Books:
Herbert Schildt, Java 2- The complete reference, Osborne.Client Server, Most of practical java
Bruce Eckel, thinking in java, PearsonMost of theoretical java
Galvin, Operating system conceptsScheduling, multitasking, synchronization
![Page 36: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/36.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 36
Grid Computing Framework – References
90% of the research material has been obtained from the internetAnd rest 10% from text books
Websites :
WikipediaAbout everything has been consulted on Wikipedia
Oracle sun tutorial-trailjava code samples, in-depth class structure
boinc.berkeley.eduBasic understanding of one of the most successful grid.
![Page 37: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/37.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 37
Grid Computing Framework – Rebuttle
Questions
![Page 38: Grid Computing Framework A Java framework for managed modular distributed parallel computing.](https://reader036.fdocuments.in/reader036/viewer/2022062519/5697bf991a28abf838c9198c/html5/thumbnails/38.jpg)
11/11/2010 Abhishek Ganguly | Apoorv Pandey 38
Grid Computing Framework – Thank you
Thank You