2012/07/06

NameNode HA in CDH4

Background

• Prior to Hadoop 2.0.0, the NameNode was a single point of failure (SPOF) in an HDFS cluster.

Approach and Terminology

• Initial goal is Active-Standby

• Terminology– Active NN: Actively serves the read/write operations from the

clients– Standby NN: Waits, becomes active when Active dies or is

unhealthy– Hot Standby: Standby has all most of the Active’s state and start

immediately

High-level Architecture

Hardware resources

• NameNode machines– Should have equivalent hardware to each other

• Shared storage– Both NameNode can have read/write access– Only a single shared directory is supported

• High-quality dedicated NAS appliance is recommended

• Secondary NameNode is not necessary

Automatic Failover

• Introduce two new components– ZooKeeper– ZKFailoverController (abbreviated as ZKFC)

• It’s a ZooKeeper client • Each of the machines which runs a NameNode also runs a ZKFC• Responsible for:

– Health monitoring– ZooKeeper session management– ZooKeeper-based election

Automatic Failover

ActiveNN

Hot StandbyNN

Shared dir on NFS

DN

ZKFC

ZK

Heartbeat

Block Reports

DN DN

ZK ZK

session session

ZKFC

Heartbeat

Appendix

• High Availability Framework for HDFS NN– HDFS-1623

• HDFS portion of ZK-based FailoverController– HDFS-2185

Questions?