MPI Raspberry pi 3 cluster
-
Upload
arafat-hussain -
Category
Documents
-
view
134 -
download
2
Transcript of MPI Raspberry pi 3 cluster
![Page 1: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/1.jpg)
MPI Raspberry pi 3 & Hadoop cluster
4 nodes
![Page 2: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/2.jpg)
MPI Raspberry pi cluster
Why MPI Hardware Needed Software Needed Machine File MPI Code Python Code Output Finished product
![Page 3: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/3.jpg)
Why MPI?
Scientist during World War II used the same method of Parallel Computing to solve mathematical problems in the Manhattan Project. reduce the amount of time it took to solve a large mathematical problem. the people who performed these calculations were called computers.
How far can you go with MPI?
![Page 4: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/4.jpg)
Hardware you need
X4 Raspberry pi 3X5 Ethernet cat 5 cablesX4 micro USB cablesX4 8 GB micro SDOne 5 port Ethernet switchOption but recommended one 4 port USB power hub
![Page 5: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/5.jpg)
Software Needed
MPICH
MPI4PY Python to MPI interpreter
![Page 6: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/6.jpg)
Machine File
![Page 7: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/7.jpg)
MPI code
![Page 8: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/8.jpg)
Python code
![Page 9: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/9.jpg)
Output
![Page 10: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/10.jpg)
Finished product
![Page 11: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/11.jpg)
Hadoop
Hadoop is a framework of tools based on Java that supports running Big Data.
Inspired from Google’s GFS and MapReduce. It is open source. Relatively cheap to build. Has Fault Tolerant System.
![Page 12: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/12.jpg)
Parts of Hadoop
Hadoop consists of two main elements, HDFS (Hadoop Distributed File System)
Name Node (Store the directories of all files in the cluster) Data Node (Store Data)
MapReduce Job Tracker(Responsible for taking requests from client and assign to
TaskTracker) Task Tracker(Perform MapReduce in DataNodes/Slave)
![Page 13: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/13.jpg)
HDFS (Hadoop Distributed File System)
![Page 14: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/14.jpg)
MapReduce
![Page 15: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/15.jpg)
Use of Hadoop in Tech World
Amazon Facebook Netflix eBay Twitter etc.
![Page 16: MPI Raspberry pi 3 cluster](https://reader036.fdocuments.in/reader036/viewer/2022062523/586f727d1a28ab10258b5425/html5/thumbnails/16.jpg)
Thank You