Hadoop mapreduce performance study on arm cluster
-
Upload
airbots -
Category
Technology
-
view
3.469 -
download
0
description
Transcript of Hadoop mapreduce performance study on arm cluster
Hadoop MapReduce Performance Study on ARM cluster
Yanjun [email protected]
Outlines
• Motivation• Introduction• Evaluation• Conclusion• Questions
Motivation
• A credit card size Raspberry Pi can run general Linux with very low power consumption
Motivation
• ARM cluster vs. x86_64 cluster
Introduction
• Hadoop MapReduce• Cubieboard2
Evaluation• Environment– ARM cluster: 4 cubieboard2, 1 head node, 3 worker
nodes; lubuntu for ARM, java-1.7 for ARM– X86_64 cluster: 2 firefly nodes. 1 head node, 1 worker
node; CentOS 6.3, java-1.7• Hadoop 1.2 [1]• Testcases– Loadgen– MDAD (Molecular Dynamics Simulation based on
Hadoop MapReduce [2])
• [1]Apache Hadoop• [2]Chen He, “Molecular Dynamics Simulation based on Hadoop MapReduce” , Master thesis, 2011
Evaluation
• Equivalent Performance– Run program on current device and get
turnaround time– To achive the same turnaround time, how many
new devices we need, or could we this?
• Energy consumption– Kill-a-Watt device to collect ARM cluster energy;– ServerTech PDU for gathering x86_64 cluster
energy consumption
Evaluation
• loadgen
Evaluation
• MDAD
Evaluation
• Loadgen Energy
Evaluation
• MDAD energy
Conclusion
• We build a ARM cluster which is composed of 4 cubieboard2 cards.
• We setup Hadoop cluster on the ARM cluster• We compared the performance and energy
consumption between two clusters• Based on our current data, we conclude that
ARM cluster is not an alternative choice to replace X86_64.