Lightning Talk: Get Even More Value from MongoDB Applications

Post on 08-Jul-2015

493 views 1 download

Transcript of Lightning Talk: Get Even More Value from MongoDB Applications

Teradata + MongoDB Daniel.Graham@Teradata.com

Dec 3,2014

2

• Teradata Corporation – 30+ years experience– Global Leader in Enterprise Data Warehousing

- Teradata/Aster Database Technology- Teradata Applications- Consulting Services

• Top 10 US Software Company – Positioned in Gartner’s Leader’s Quadrant since 1999– InformationWeek’s Top 10 Most Strategic Vendors– Dow Jones Sustainability Index– World’s Most Ethical Companies – Ethisphere Institute– #2 in The 25 Best Companies in America - The Motley Fool

• Global presence and world-class customer list– 1,500+ customers in ~70 countries – 10,000+ data professionals world wide

Big Data Since 1984

3

Teradata in the Data Warehouse Market

4

• We added JSON to tables as columns

• Any user has easy access to JSON– Dashboards, ad hoc, OLAP

– Predictive analytics, statistics, etc.

– Discovery analytics

JSON in the Data Warehouse

Teradata

Databasestable

BI Tools

SQL

Sourcedata

ETL

JSON

5

{ "MFG_Line": {

"Product”: {

"Color”: "Red”,

"Size”: "Large”

"Prod_ID”: 100,

"Create_Time”: "2013-06-15 20:07:27”

},

"Machine”: {

"Temp”: 95,

"Warning”: null,

"FW_Version”: 1.2,

"Sensor_Code”: 152

}

} }

Scenario: Machines Packing Boxes Emits JSON

6

JSONPath in Teradata 15.0 SQL

Color Size Prod_ID Create_Time

----- ----- ------- -------------------

Blue Small 96 2013-06-17 20:07:27

SELECT

box.MFG_Line.Product.Color AS "Color",

box.MFG_Line.Product.Size AS "Size",

box.MFG_Line.Product.Prod_ID AS "Prod_ID",

box.MFG_Line.Product.Create_Time AS "Create_Time"

FROM mfgTable

WHERE CAST(box.MFG_Line.Product.Create_Time

AS TIMESTAMP) >= TIMESTAMP'2013-06-16 00:00:00'

AND box.MFG_Line.Product.Prod_ID = 96;

7

eCommerce in Action: A Virtuous Cycle

Buyer preferences

Sales catalog

Campaigns

Recent purchases

Profitability

DataWarehouse

Shard

Shard

Shard

Shard

Shard

Shard

Shard

Shard

8

Call Center Efficiency

Trouble tickets

Web clicks

Payment history

Claims

Up-sell offers

Data

Warehouse

Shard

Shard

Shard

Shard

Shard

Shard

Shard

Shard

web logs

9

Parallel Integration with MongoDB

10

TERADATA ASTER

SQL,SQL-MR,SQL-GR

OTHERDATABASES

Remote Data

Teradata and MongoDB: QueryGrid

Teradata Systems

TERADATA DATABASE

HADOOP

Push-down to Hadoop

IDW

TERADATA DATABASE

Discovery

ASTER DATABASE

Business users Data scientists

MONGODB

NoSQLDatabase

11

• Standard Server Grammar SELECT doc. item_id, doc.desc

FROM collection@MongoDBserver doc

WHERE doc.qty > 1;

• Foreign Table Select SELECT doc.id, doc.descFROM FOREIGN TABLE (

‘testdb.products.find({ qty: { $lt: 25 } }, { item_id: 1, qty: 1,

desc: 1 }’ )@MongoDBserver AS imported_products(doc JSON(160000)) );

• Insert / ExportINSERT INTO testdb.customers@MongoDBserver

SELECT JSON_compose( item_id, doc.qty ) as document FROM cust_data;

Example QueryGrid Syntax

12

• Parallel data transfer for queries without aggregation

• Foreign query pass-through for selection, projection of data in MongoDB

SQL Connecting to MongoDB

SELECT prod_doc.itemid, prod_doc.qty FROM FOREIGN TABLE (

‘testdb.products.find({ qty: { $lt: 25 } }, { item_id: 1,

qty: 1 }’ )

@MongoServer

RETURNS (prod_doc JSON(160000)) AS imported_products

INNER JOIN local_products ON

local_products.product.item=prod_doc.itemid;

13

Scale-out NoSQL + Scale-out DW SQL

Application

Shard 1 Shard 2 Shard NShard 3

Primary Primary PrimaryPrimary

Query router Query router Query router

NoSQL

SQL

AMPAMP

PE

AMPAMP

PE

AMPAMP

PE

AMPAMP

PE

14

Query Router

Shard 1

Shard 2

Shard 3

Shard 4

Contract Phase

SQL

Teradata

node

AMP

AMP

AMP

AMP

PEE

A

H

15

Teradata

node

AMP

AMP

AMP

AMP

PEE

A

H

Contract Phase

Query Router

Shard 1

Shard 2

Shard 3

Shard 4

16

Teradata

node

AMP

AMP

AMP

AMP

PEE

A

H

Import Data from Shards

Query Router

Shard 1

Shard 2

Shard 3

Shard 4

17

• Teradata needs a partner with strong

JSON data solutions

• MongoDB = leader in new JSON

applications

• Operational + analytical

– Complementary

• Customers accelerate JSON analysis

Why Teradata and MongoDB Partnership?

MongoDB Teradata

Application Data

Analytics

18

19

Math

and Stats

Data

Mining

Business

Intelligence

Applications

Languages

Marketing

ANALYTIC TOOLS

& APPS

USERS

INTEGRATED DISCOVERY

PLATFORM

INTEGRATED DATA

WAREHOUSE

ERP

SCM

CRM

Images

Audio

and Video

Machine

Logs

Text

Web and

Social

SOURCES

DATA

PLATFORM

ACCESSMANAGEMOVE

TERADATA UNIFIED DATA ARCHITECTURE

Marketing

Executives

Operational

Systems

Frontline

Workers

Customers

Partners

Engineers

Data

Scientists

Business

Analysts

TERADATA DATABASE

HORTONWORKSCLOUDERA

MAPR

TERADATA DATABASE

TERADATA ASTER DATABASE

21

Forrester Data Warehouse Wave December 2013

2222 © 2014 Teradata