Hadoop Essential SetupA big data proposal

Who is thisweirdo?

Hiking, Table Tennis, Kicker, Traveling, Foods, Cultures, The BigbangTheory, Family Guy, Sherlok, …

What's Big Data Really?

There are only two hard things in Computer Science: cache invalidation and naming things.

Phil Karlton

Hadoop?

From Wikipedia

The genesis of Hadoop came from the Google File System paper ..... This paper spawned another research paper from Google – MapReduce: …. in January 2006. Doug Cutting, who was working at Yahoo! at the time, named it after his son's toy elephan

Map/Reduce

Shuffle(Transfer & Merge)

HDFS

By the wayHadoop 2.0

HadoopEcosystyem

ETL or ELT? ?

Source

TargetTarget

Source

Source

Essential Setup

+

Flume Agent

Flume Agent

Flume Agent

HDFSHDFS

HDFSHDFS

Pig

Hive

SparkHbase

Essential Setup

Essential Setup

Fast Data & Big Data

Lambda Architecture

Fast Data & Big Data

Now your

Turn, What's

your idea?