This Presentation is protected by

Creative Common License v2.0

Agenda

-What is a Supercomputer?-Why do we need it?-Types of Supercomputers-The Recipe of building a cluster

-Basic Concepts-Identifying the Application-Selection of Raw Materials-Preparation-Configuration -Deployment-Testing -Maintenance

-Home made vs. commercial clusters-Making money from clusters-Other resources and links

Domain : High Performance ComputingPresentation Level : Beginner Prerequisite : Familiarity with Linux

What is a Supercomputer?

“ An extremely fast computer that can perform hundreds of millions of instructions per second.”

-A powerful system built from collection of special purpose hardware -It is designed for a specific application-Processing power range is very high-There is no standard for supercomputer specification-Works on parallel processing schema

Best Supercomputers of the world

Indian Supercomputers PARAM Padma is C-DAC's next generation high performance scalable computing cluster, currently with a peak computing power of One Teraflop.

KABRU is a 144 node (Xeon DP) Linux cluster. Though it is a very fast supercomputer, it is not the fastest in the world. With a sustained performance of 1002.3 GFlops(reached on October 13th 2004) of double precision arithmetic it is the second fastest supercomputer in India and the fastest supercomputer in India belonging to an academic institution.--IMSc

Why do we need it?

“Obviously, we need it for more processing power!!!”

- We use it where computation can be parallel.- We use it where “divide and conquer” is prominent in algorithms- We use it for High Performance/Availability computing- We use it for distributed computing

In India, CDAC uses supercomputers for research in:Bioinfomatics, Computational Structural Mechanics , Computational Atmospheric Science, Evolutionary Computing , Computational Chemistry etc.

Types of Supercomputers

“Two broad categories: Tightly coupled parallel systems & Loosely Coupled Clusters”

Modern supercomputing clusters:- High performance (HP) clusters- Load-leveling clusters- Web-service clusters- Storage clusters- Database clusters

A special type is Single System Image (SSI) Clusters

The Recipe for building a cluster

“…before you make soup, you need hunger to enjoy it…”

Let’s brush up the basic concepts:- Linux Installation Basics- DHCP- Network Boot (via PXE Boot or Etherboot)- Interconnect

The Recipe for building a cluster[Identifying the Application]

“Why would you need a supercomputer? Hey , I need it just for fun!”

Building a cluster for:- High Performance (HP) need- High Availability (HA) need

or-“just need it for fun”

“There is really lot of fun when you write & test your algorithms on a cluster…”

The Recipe for building a cluster[Selection of raw material]

“Innovators build great things from non-great elements !”

Selection of Hardware:- Few number of old mother boards- Enough number of processors to sit on the boards- Minimum of 32Mb RAM per board- Network support via On-Board or External NIC cards- At least one Hard disk & CD-ROM Drive- Either BIOS support for Network Boot or A floppy drive for each board “mean to say, just get few boxes from your friends if you feel lazy to build your own hardware…”

The Recipe for building a cluster[Selection of raw material]

“Innovators build great things from non-great elements !”

Selection of software:- The OpenSSI (http://www.openssi.org) - OSCAR (http://oscar.sourceforge.net/)- TFTP - Etherboot (http://rom-o-matic.net/5.2.4/ )

“mean to say, just get few boxes from your friends if you feel lazy to build your own hardware…”

The OpenSSI“the most fantastic product I have ever seen !”

It has internode communication, clusterwide process management, clusterwide devices, a cluster filesystem, clusterwide IPC (pipes, fifos, msgqueues, semaphores, etc.) and clusterwide tcp/ip networking.

The Recipe for building a cluster[Preparation]“Question : Dedicated Cluster or Temporary Cluster ?”

- A clean install of base OS (Fedora Core 3) on PCs that has bootable device - A clean Network Configuration

The Recipe for building a cluster[Configuration]

- Download and unpack OpenSSI (http://www.openssi.org) - Go through the DOCS- ./install does everything for year

Enter a clustername. Enter a node number between 1 and 125.

Select a Network Interface Card (``NIC'') for the cluster interconnect. Select (P)XE or (E)therboot as the network boot protocol for this node.

Select whether you want to enable root filesystem failover.- For Adding nodes, run openssi-config-node. Select ``Add a new node''.

“ Remember that node 1 is called the init node…”

The Recipe for building a cluster[Configuration]

- Essentials# cluster -v (To check the membership of nodes in cluster with their status)# bash-ll (The shell that performs load leveling./etc/sysconfig/loadlevellist can also be added to specify special processes to be loadlevelled and run service loadlevel restart )# ssi-ksync (To rebuild the ramdisk to include the driver and update the network boot images)

# onnode <node_number> <command> (To run a specific command on a specific node)

The Recipe for building a cluster[Testing]

while(no_of_processes < 1000){ if (fork()==0) { run_relevent_algorithm(); exit(1); } no_of_processes++;}

“…This piece of code can do wonders…Its fun to keep adding zeros in the loop...”

The Recipe for building a cluster[Maintenance]

- Make sure there is no IP conflict if network is shared - Perform regular efficiency audit on network- Put proper firewall for security

“ you actually don’t need to bother much for maintenance…”

Home made vs. commercial clusters “The obvious difference is in the looks…”

Making money from clusters “Are’nt you interested in this…”

-Host web servers , file servers etc.-Create supercomputer for fun, give access to yours friends-Provide low-cost high performance computing facility to research institutes-Convert offices and academic institutions into night-time research facility

Other resources and links “go ahead find out more…”

Download this presentation and various other interesting things at:http://www.parolkar.com/download.aspx

Other links:http://www.openssi.orghttp://www.beowulf.orghttp://sourceforge.net/projects/ci-linuxhttp://linux-ha.orghttp://www.openmosix.org

Questions

?

Thank You!!!

I am just a click away abhishek@parolkar.com