Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong...
-
Upload
daniela-lindsey -
Category
Documents
-
view
215 -
download
0
Transcript of Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong...
![Page 1: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/1.jpg)
Minimalist’s Linux Cluster
Changyoung Choi, Jeonghyun Kim, Seyong Kim
Department of PhysicsSejong University
![Page 2: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/2.jpg)
Reported in Lattice’99
![Page 3: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/3.jpg)
1. Usual Linux cluster
• Easy to build, use and maintain• Off-The-Shelf everything!
– Ordinary PC’s– Ordinary network components– Free OS (Linux)– Free parallel programming environment
(MPI)
![Page 4: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/4.jpg)
Room for improvement
• Occupies too much space• Consumes too much power• Not all the components in an ordinary
PC is necessary for computation– Possible further reduction in cost!– Possible reduction in power consumption
• Supporting full bisection bandwidth is costly!
![Page 5: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/5.jpg)
Conditions
• No custom-made hardwares– If it takes too long to develop ->
no advantage– If there is, should be recyclable
• Should be efficient enough• Should be do-able with small manpower
![Page 6: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/6.jpg)
2. Construction(hardware)
• Node – ATX size standard mother board– Intel Pentium IV-2.4 GHz– 512 Mbytes DDR SDRAM– 100 Mbps ethernet card (3+1)– 180W power
• Cage for ATX size mother board• 1 front end server
![Page 7: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/7.jpg)
CPU Intel P-4,2.4GHz
\236,500(=$198)x36
motherboard
ASUS P4PE \203,500(=$170)x36
memory 512MB PC2700 DDR
\112,200(=$93.5)x36
NIC 3+1 realtek \53,000(=$44)x36
power Sun ATX 180W \25,300(=$21)x36
cable \10,000(=$8)x36
![Page 8: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/8.jpg)
cabinet 2000x910x750
\880,000(=$733)
cabinet accessory \365,200(=$304)
Cost per node $534.5
Total cost $534.5x36+$1037 = $20,279
Price Includes Tax
![Page 9: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/9.jpg)
![Page 10: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/10.jpg)
![Page 11: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/11.jpg)
![Page 12: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/12.jpg)
![Page 13: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/13.jpg)
![Page 14: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/14.jpg)
![Page 15: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/15.jpg)
Software
• WOW Linux 7.1 (equiv. RedHat 7.1)• Linux Terminal Server Project (LTSP)
– Remote booting (DHCP + TFTP)– Diskless node system
• 2-D mesh network configuration– Script developed in-house
• MPI programming environment– MPICH/LAM
![Page 16: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/16.jpg)
Setup
• Front end wait• F-end send DHCP
info• send kernel using
TFTP service• send client file
system by NFS
• Node 1 power-up• NIC listens to DHCP
server• IP setup complete
and download kernel• setup file system on
memory and copy NFS mounted file system to memory
![Page 17: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/17.jpg)
Setup(continued)
• Node 1 wait• Node 1 send DHCP
info• Node 1 send kernel
image• Node 1 act as a
pipe line between front end and node 2 for NFS
• Node 2 power-up• NIC listens to node 1
DHCP server• IP setup complete and
download kernel from node 1
• setup file system on memory and copy NFS mounted file system to memory
![Page 18: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/18.jpg)
Parallel boot
![Page 19: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/19.jpg)
3. Performance
• 1-D ring full QCD code– Not suitable for 2-D mesh structure
• 8^3 x 512 lattice
![Page 20: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/20.jpg)
8^3 x 32 lattice testedon 1-node
1187 sec, CG = 181.2 -> 369 MFLOPS
8^3 x 512 lattice testedon 4-node
5522 sec, CG = 391.4 -> 1.24 GFLOPS
8^3 x 512 lattice testedon 8-node
3029 sec, CG = 390.9 -> 2.25 GFLOPS
![Page 21: Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University.](https://reader036.fdocuments.in/reader036/viewer/2022062806/5697bfad1a28abf838c9c35d/html5/thumbnails/21.jpg)
4. Discussion
• recyclable– cabinet takes standard motherboard size
-> replacing node is enough for a upgrade
• occupies smaller foot-print, consumes less power
• need to fine-tune• need to modify the code for 1-D ring
to 2-D mesh