Terracotta Hadoop & In-Memory Webcast
-
Upload
terracotta-a-product-line-at-software-ag -
Category
Technology
-
view
765 -
download
0
description
Transcript of Terracotta Hadoop & In-Memory Webcast
© 2013 Terracotta Inc. | Internal Use Only
In-Memory & Hadoop:Real-time
Big Data Intelligence
© 2013 Terracotta Inc. 2
Your speaker
Manish DevganDirector of Product
ManagementTerracotta
© 2013 Terracotta Inc. 3
What we’ll cover in this webcast
• What’s Hadoop? (quick intro)
• Hadoop’s weaknesses
• Emerging best practices for combining Hadoop and in-memory data management
• Real-time intelligence example
• Getting started with in-memory and Hadoop
• Q & A
© 2013 Terracotta Inc. 4
4© 2013 Terracotta Inc. | Internal Use Only
What is Hadoop?
© 2013 Terracotta Inc. 5
What is ?
• Hadoop is open-source software data management framework used to draw insights from data
Components Benefits
HDFS*: Scalable & distributed Storage• Data distributed across cluster nodes• Name node keeps track of location
MapReduce: Parallel Processing of data• Splits a task for processing based on
data locality and then assembles results
• Comprises of Map() procedure for filtering & sorting and Reduce() procedure for summarizing
Scalable• Efficiently store and process large data
sets
Reliable• Get redundant storage, with failover
across cluster
Rich & Flexible• Complimentary set of tools & frameworks• Store data in any format
Economical• Deploy on commodity hardware
*Hadoop Distributed File System
© 2013 Terracotta Inc. 6
What is ?
• With Hadoop, you can ask interesting questions about your data and get answers economically
Questions Hadoop can help answer
How can I target promotions to my customers for better sales?
How risky are each of my customers?
Which advertisement should I show to optimize return?
How relevant is a result for a given search?
When will my machinery likely have a malfunction?
© 2013 Terracotta Inc. 7
7© 2013 Terracotta Inc. | Internal Use Only
Hadoop’s Weaknesses
© 2013 Terracotta Inc. 8
Hadoop’s Weaknesses
• No support for real-time insights• No support to facilitate interactive and exploratory data analysis• Challenging framework for computation beyond Map Reduce• Lacks tools for business analysts
© 2013 Terracotta Inc. 9
9© 2013 Terracotta Inc. | Internal Use Only
Emerging best practices for combining Hadoop and
in-memory data management
© 2013 Terracotta Inc. 10
Combining Hadoop and In-memory Data Management
- Businesses are looking for ways to mine real-time insights to provide competitive advantages
- Increased adoption of transactional system data for analytics is blurring the line between OLTP and OLAP
- New frameworks and products are bringing in-memory technologies to the Hadoop ecosystem
© 2013 Terracotta Inc. 11
Real-time Data Integration with Hadoop
WebApps
Mobile Apps
Dashboards & Mashups
In-memory Data Management Platform
Real-time Data Apps
Transactional Apps
Operational Intelligence
Log Data POS Data Social Media Sensors
Data Sources
EventsImages/Videos
Data Feeds
Real-timedata
Real-timeInsights
© 2013 Terracotta Inc. 12
12© 2013 Terracotta Inc. | Internal Use Only
Real-time intelligence example
© 2013 Terracotta Inc. 13
BigMemory & Hadoop in financial servicesBefore: Custom ETL connector pushing batch data
Hadoop Cluster
Big
Mem
ory
Sto
re
Short Term Transaction
Data
Long Term Transaction
Data
Rules & Triggers
Tagged Accounts
Credit Reference
Data
HDFS to BigMemory Processing
Hadoop M/R
© 2013 Terracotta Inc. 14
BigMemory & Hadoop in financial servicesToday: Streaming Data insights
Hadoop Cluster
Insights Hadoop M/R
BigMemory- Hadoop
Connector
Big
Mem
ory
Sto
re
Short Term Transaction
Data
Long Term Transaction
Data
Rules & Triggers
Tagged Accounts
Credit Reference
Data
© 2013 Terracotta Inc. 15
15© 2013 Terracotta Inc. | Internal Use Only
Getting started with in-memory and Hadoop
© 2013 Terracotta Inc. 16
How to get started with In-memory and Hadoop?
• If you already have a Hadoop project, look for use cases where you want real-time access to insights
• Start with a small-to-medium sized (20-40 nodes) cluster with a well-defined use case requiring fast access to data
• Consider exploratory use cases where you’re doing iterative analysis on a data set to get answers faster
© 2013 Terracotta Inc. 17
In-Memory & Hadoop
QuestionsPlease type yours in the “Questions” panel or in the chat window.
© 2013 Terracotta Inc. 18
Connect with Terracotta
• Download “BigMemory & Hadoop” white paper− Visit: www.terracotta.org (Resources > White Papers)
• Download “BigMemory-Hadoop Connector”− Visit: www.terracotta.org/downloads/hadoop-connector
• Contact Manish Devgan− Email: [email protected]
• Follow us on Twitter− @big_memory
• Stay Tuned