CompuNetCompuNetGrid ComputingGrid Computing

Milena Natanov Keren Kotlovsky

Project Supervisor: Zvika BerkovichLab Chief Engineer: Dr. Ilana David

044167/044169 – SoftLab SpringSpring, 2007, 2007

Agenda

About Grid Computing About CompuNet Implementation Steps Technologies Used CompuNet Terms System Overview CompuNet Usage - Sample Job Future Extensions

About Grid Computing

- The term Grid computing emerge in the early 1990’s.- Originally it was a metaphor for making computer power as easy as an electric power

grid.- First appearance in Ian Foster & Carl Kesselmans work: "The Grid: Blueprint for a new computing infrastructure".

- This grid computing idea was created by Ian Foster, Carl Kesselman and Steve Tuecke, regarded as the "fathers of the grid”.

- Goal: create “Globus Toolkit” which include computation and storage management.

- Utilization:

- Used to denote a hardware and software infrastructure that enables applying the resources of many computers to a single problem.

- Used to denote a hardware and software infrastructure that enables coordinated resource sharing within organizations.

About Grid Computing (cont)

Definition:Using resources of many separated computers connected by a network to solve large-scale computationproblems that requires a great number of computerprocessing cycles or access to a large amount of data.

Projects examples:• LCG – Petabytes of data will be recorded and analyzed every year.

• ‘

SETI – (Search for Extraterrestrial Intelligence) @Home project in which thousands of people are sharing the unused processor cycles of their PCs in the vast search for signs of "rational" signals from outer space.

About CompuNet

- CompuNet is an implementation of the “grid computing” idea in .Net with extensions for generic projects.

- This application enables simultaneous running several .Net computation projects in parallel, utilizing the power of all the clients connected to the CompuNet grid.

Implementation steps

• Project requirements• Use-case diagram and specifications• Class and Package diagrams• Sequence diagram• Scheduling• Coding• Testing and bug fixing• Documentation (presentation,

project book)

Technologies

• .Net Remoting– Allows an application to make an object available across

remoting boundaries, which includes different appdomains, processes or even different computers connected by a network.

– In CompuNet, we use .Net Remoting to send messages between CompuNet actors (service, management and client), stealing the idea of post office.

• XML Serialization– Process of packaging data structures or saving an object

in a format that can be easily transported. This format is an xml file.

– In CompuNet, we use XML serialization to save client and management GUI configuration settings and server database and reports

Technologies (cont)

• Reflection– Reflection can be used to dynamically create an

instance of a type, bind the type to an existing object, or get the type from an existing object and invoke its methods or access its fields and properties.

– In CompuNet, we use Reflection to run small work units (chunks) on clients

• Windows Installer– Windows Installer enables customers to provide better

corporate deployment, and provide a standard format for component management.

– In CompuNet, we implemented Installer using Visual Studio 2005 Setup Project. It can install either client or server or both.

CompuNet terms

Job – big computation project divided into computing units (chunks).

Chunk – single computing unit that will be sent to the client computer to be processed.

Chunk result – computed value that is either used by dependant chunks or shown as a final

result, also known as Job Result

Server – responsible for jobs and clients management and chunks distribution.

Client – a computer registered within the system to run job’s single computing unit (chunk).

Chunk dependency – chunk can be dependant or independent:

Independent – no inputs, runs when the job starts.

Dependant – the chunk depend on the output of other chunks,

when all the input is collected.

c1

c2 c3

c4

job1

System overview

Object-oriented Analysis and DesignPackage Diagram

Client Module

CompuNet Core

Server Module

Object-oriented Analysis and DesignCompuNet Core Module

• CompuNetCore

– Holds the definitions of Job, Chunk, Client and other basic classes

of CompuNet

• CompuNetRemoting

– Allows sending messages between CompuNet actors over TCP

Remoting

• CompuNetResources

– Holds icons used by all the components

Object-oriented Analysis and DesignClient Module

• GUI and Tray Icon

– Allows the client to choose the server address and see which

chunk is running now

• Chunk Runner

– Runs the chunk sent by the server

Object-oriented Analysis and DesignServer Module

• Main Service module

– Holds server database that includes jobs and clients information

– Distributes the chunks between clients

• Management GUI

– Shows the server status

– Shows the jobs, their status and results and allows adding,

starting, stopping and reporting the job

– Shows the information about registered clients

CompuNet Usage

Job example

Chunks dependency:

c1

c2 c3

c4

Job xml file:

Possible extensions and improvements

• GUI for Project Creation– Now, user needs to create and edit XML file

• Improve GUI of Project Monitoring to show chunks as directed graph

• Support chunks running in long loops– clients will get only range of loop and server will combine

the result– no need to define lots of chunks

• Compile code on client– User adds chunks as formulas, client compiles the code

and runs it

Thank you!Thank you!

CompuNet Server Database

Server Database:- Jobs

Chunks Credentials (Name) Status Vertices

- Jobs in process- Clients

Client information Current chunk information

- Free clients- Busy clients

CompuNet Client information

Client information:

- Nickname

- Client status

- Configuration

- Server address

- Preferred jobs

- Current chunk

- Current job name

SampleJob3

c1

c2 c3

c4

c5

c6

c7

c8

c9c10

c11

c12 c13

New GUI – server control:

New GUI – Projects Monitoring:

New GUI – Client Monitoring: