Snaplogic Live: Big Data in Motion
-
Upload
snaplogic -
Category
Data & Analytics
-
view
388 -
download
2
Transcript of Snaplogic Live: Big Data in Motion
SnapLogic Live – Big Data in Motion
2Source: Internaphttp://www.internap.com/resources/infographic-data-motion-vs-data-rest/
3
The Data Lake
“Just as data integration is the foundation of the data warehouse, an end-to-end data processing capability
is the core of the data lake. The new
environment needs a new workhorse.”
- Mark Madsen, Third Nature
snaplogic.com/resources
z
Data Acquisition
Data Access
z
Data Management
Data Lake Components
Add information and improve
data
SparkPythonScalaJava
RPig
Collect and integrate data from multiple sources
HDFSAWS S3
MS Azure Blob
On Prem Apps and
Data• ERP• CRM• RDBMS
Cloud Apps and Data• CRM• HCM• Social
IoT Data• Sensors• Wearable
s• Devices
LakeshoreData Mart• MS Azure• AWS
Redshift
BI / Analytics
• Tableau• MS
PowerBI / Azure
• AWS QuickSight
Organize and prepare
data for visualizatio
n
HDFSAWS S3
MS Azure BlobHive
Batch
Streaming
Schedule and manage: Oozie, Ambari
Kafka, Sqoop, Flume
Real-time
Impala, HiveSQL, SparkSQL
Current Data Lake Architecture
z
Data Acquisition
Data Access
z
Data Management
The Modern Data LakePowered by SnapLogic
Sort,Aggregate,Join, Merge,Transform
SnapLogicabstracts and
operationalizes with
MapReduce or Spark pipelines
Collect and integrate data from multiple sources
SnapLogicpipelines with
standard mode
execution
Organize and prepare
data for visualizatio
n
SnapLogicpipelines with
standard mode
execution
On Prem Apps and
Data• ERP• CRM• RDBMS
Cloud Apps and Data• CRM• HCM• Social
IoT Data• Sensors• Wearable
s• Devices
LakeshoreData Mart• MS Azure• AWS
Redshift
BI / Analytics
• Tableau• MS
PowerBI / Azure
• AWS QuickSight
Schedule and manage: SnapLogic
Batch
Streaming
Real-time
Modern Data Lake Architecture
SnapLogic Pipeline
6
SnapLogic in the Modern Data FabricCo
nsum
eSt
ore
& Pr
oces
sSo
urce z z z z
HANA
Data Warehouses & Data Marts Big Data and Data
LakesINGEST INGEST
Data Integration and Transformation
On Prem Application
s
RelationalDatabases
CloudApplication
s
NoSQLDatabases
WebLogs
Internet of Things
DELIVER DELIVER
Modern Architecture: Hybrid and ElasticStreams: No data is stored/cachedSecure: 100% standards-basedElastic: Scales out & handles data and app integration use cases
MetadataData
Databases On Prem Apps
Big Data
Cloud Apps and DataCloud-Based Designer,
Manager, Dashboard
Cloudplex
Groundplex
Hadooplex
Firewall
Discussion
snaplogic.com
Unified Platform Self-Service UX
Modern Architecture Connected: 400+ Snaps