Terracotta Hadoop & In-Memory Webcast

© 2013 Terracotta Inc. | Internal Use Only

In-Memory & Hadoop:Real-time

Big Data Intelligence

© 2013 Terracotta Inc. 2

Your speaker

Manish DevganDirector of Product

ManagementTerracotta


What we’ll cover in this webcast

• What’s Hadoop? (quick intro)

• Hadoop’s weaknesses

• Emerging best practices for combining Hadoop and in-memory data management

• Real-time intelligence example

• Getting started with in-memory and Hadoop

• Q & A


4© 2013 Terracotta Inc. | Internal Use Only

What is Hadoop?


What is ?

• Hadoop is open-source software data management framework used to draw insights from data

Components Benefits

HDFS*: Scalable & distributed Storage• Data distributed across cluster nodes• Name node keeps track of location

MapReduce: Parallel Processing of data• Splits a task for processing based on

data locality and then assembles results

• Comprises of Map() procedure for filtering & sorting and Reduce() procedure for summarizing

Scalable• Efficiently store and process large data

sets

Reliable• Get redundant storage, with failover

across cluster

Rich & Flexible• Complimentary set of tools & frameworks• Store data in any format

Economical• Deploy on commodity hardware

*Hadoop Distributed File System


What is ?

• With Hadoop, you can ask interesting questions about your data and get answers economically

Questions Hadoop can help answer

How can I target promotions to my customers for better sales?

How risky are each of my customers?

Which advertisement should I show to optimize return?

How relevant is a result for a given search?

When will my machinery likely have a malfunction?



Hadoop’s Weaknesses


Hadoop’s Weaknesses

• No support for real-time insights• No support to facilitate interactive and exploratory data analysis• Challenging framework for computation beyond Map Reduce• Lacks tools for business analysts



Emerging best practices for combining Hadoop and

in-memory data management


Combining Hadoop and In-memory Data Management

- Businesses are looking for ways to mine real-time insights to provide competitive advantages

- Increased adoption of transactional system data for analytics is blurring the line between OLTP and OLAP

- New frameworks and products are bringing in-memory technologies to the Hadoop ecosystem


Real-time Data Integration with Hadoop

WebApps

Mobile Apps

Dashboards & Mashups

In-memory Data Management Platform

Real-time Data Apps

Transactional Apps

Operational Intelligence

Log Data POS Data Social Media Sensors

Data Sources

EventsImages/Videos

Data Feeds

Real-timedata

Real-timeInsights



Real-time intelligence example


BigMemory & Hadoop in financial servicesBefore: Custom ETL connector pushing batch data

Hadoop Cluster

Big

Mem

ory

Sto

re

Short Term Transaction

Data

Long Term Transaction

Data

Rules & Triggers

Tagged Accounts

Credit Reference

Data

HDFS to BigMemory Processing

Hadoop M/R


BigMemory & Hadoop in financial servicesToday: Streaming Data insights

Hadoop Cluster

Insights Hadoop M/R

BigMemory- Hadoop

Connector

Big

Mem

ory

Sto

re

Short Term Transaction

Data

Long Term Transaction

Data

Rules & Triggers

Tagged Accounts

Credit Reference

Data



Getting started with in-memory and Hadoop


How to get started with In-memory and Hadoop?

• If you already have a Hadoop project, look for use cases where you want real-time access to insights

• Start with a small-to-medium sized (20-40 nodes) cluster with a well-defined use case requiring fast access to data

• Consider exploratory use cases where you’re doing iterative analysis on a data set to get answers faster


In-Memory & Hadoop

QuestionsPlease type yours in the “Questions” panel or in the chat window.


Connect with Terracotta

• Download “BigMemory & Hadoop” white paper− Visit: www.terracotta.org (Resources > White Papers)

• Download “BigMemory-Hadoop Connector”− Visit: www.terracotta.org/downloads/hadoop-connector

• Contact Manish Devgan− Email: [email protected]

• Follow us on Twitter− @big_memory

• Stay Tuned

http://www.terracotta.org/

http://www.terracotta.org/downloads/hadoop-connector

mailto:[email protected]

Terracotta Hadoop & In-Memory Webcast

Technology

Transcript of Terracotta Hadoop & In-Memory Webcast