Post on 12-Jul-2020
Darwin Schweitzer
Big Data Analytics
C LO U D
D ATA A I
Organizations that harness Data, Cloud, and AI outperform
Security and performanceFlexibility of choiceReason over any data, anywhere
Data warehouses
Data Lakes
Operational databases
Hybrid
Data warehouses
Data Lakes
Operational databases
SocialLOB Graph IoTImageCRM
T H E M O D E R N D A T A E S T A T E
Classic AnalyticsTransaction-driven
Cloud-born AnalyticsEvent-driven
COSMOS DB Databricks
SQL DB/DW and Analysis Services
Big Data & Advanced Analytics in Azure
Model & ServePrep & Train
Databricks
HDInsight
Data Lake AnalyticsCustom
apps
Sensors
and devices
Store
Blobs
Data Lake
Ingest
Data Factory(Data movement, pipelines & orchestration)
Machine
Learning
Cosmos DB
SQL Data
Warehouse
Analysis Services
Event Hub
IoT Hub
SQL Database
Analytical dashboards
Predictive apps
Operational reports
Intelligence
B I G D ATA & A D VA N C E D A N A LY T I C S AT A G L A N C E
Business
apps
1001
SQLKafka
Azure DatabricksPowered by Apache Spark
A fast, easy and collaborative Apache® Spark™ based analytics platform optimized for Azure
Best of Databricks Best of Microsoft
Designed in collaboration with the founders of Apache Spark
One-click set up; streamlined workflows
Interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.
Native integration with Azure services (Power BI, SQL DW, Cosmos DB, Blob Storage)
Enterprise-grade Azure security (Active Directory integration, compliance, enterprise -grade SLAs)
Get started quickly by launching
your new Spark environment with
one click.
Share your insights in powerful
ways through rich integration with
Power BI.
Improve collaboration amongst
your analytics team through a
unified workspace.
Innovate faster with native
integration with rest of Azure
platform
Simplify security and identity control
with built-in integration with Active
Directory.
Regulate access with fine-grained user
permissions to Azure Databricks’
notebooks, clusters, jobs and data.
Build with confidence on the trusted
cloud backed by unmatched support,
compliance and SLAs.
Operate at massive scale
without limits globally.
Accelerate data processing with
the fastest Spark engine.
ENHANCE PRODUCTIVITY BUILD ON THE MOST COMPLIANT CLOUD SCALE WITHOUT LIMITS
Optimized Databricks Runtime Engine
DATABRICKS I/O SERVERLESS
Collaborative Workspace
Cloud storage
Data warehouses
Hadoop storage
IoT / streaming data
Rest APIs
Machine learning models
BI tools
Data exports
Data warehouses
Azure Databricks
Enhance Productivity
Deploy Production Jobs & Workflows
APACHE SPARK
MULTI-STAGE PIPELINES
DATA ENGINEER
JOB SCHEDULER NOTIFICATION & LOGS
DATA SCIENTIST BUSINESS ANALYST
Build on secure & trusted cloud Scale without limits
Demo Azure Databricks
https://github.com/Azure/data-ai-iot/tree/master/databricks
https://gallery.azure.ai/Solution/Azure-Databricks-Spark-Streaming-4
Additional Sample Azure Databricks Notebooks
Use Cases
Business / custom apps
(Structured)
Logs, files and media
(unstructured)
Azure storage
Polybase
Azure SQL Data Warehouse
Data factory
Data factory
Azure Databricks
(Spark)
Analytical dashboards
Model & ServePrep & TrainStoreIngest Intelligence
Web & mobile appsAzure Databricks
(Spark Mllib,
SparkR, SparklyR)
Azure Cosmos DB
Business / custom apps
(Structured)
Logs, files and media
(unstructured)
Azure storage
Polybase
Azure SQL Data Warehouse
Data factory
Data factory
Analytical dashboards
Model & ServePrep & TrainStoreIngest Intelligence
Unstructured data
Azure storage
Polybase
Azure SQL Data Warehouse
Azure HDInsight
(Kafka)
Azure Databricks
(Spark)
Analytical dashboards
Model & ServePrep & TrainStoreIngest Intelligence
Pricing @General Availability
Release Standard Premium
General availability Data analytics: $0.40/DBU/hr + VM
Data engineering: $0.20/DBU/hr + VM
Includes:
Compliance: SOC2, HIPAA, AAD Integration
Data connectors (Blob Storage, Data Lake, SQL DW,
Cosmos DB, Event Hub), GPU Instances
Data analytics: $0.55/DBU/hr + VM
Data engineering: $0.35/DBU/hr + VM
Includes:
Everything from standard +
Fine grained control for notebooks & clusters, structured
data controls
JDBC/ODBC endpoint
Governance logs
.NET Integration
Integrations with Azure apps like Power BI, etc.
Engage Microsoft experts for a workshop to help identify
high impact scenarios
Try a Quickstart or Tutorial at:
https://docs.microsoft.com/en-us/azure/azure-databricks/
https://gallery.azure.ai/Solution/Azure-Databricks-Spark-Streaming-4
https://github.com/Azure/data-ai-iot
Learn more about Azure Databricks www.azure.com/databricks
https://github.com/Azure/data-ai-iot
Accelerate learning and using Try → Learn → Build:
Try• Demos (IDEA)
• Introduce
• Demo
• Explain
• Attend
Learn• GitHub Samples
• Solution Templates
• Data Science VM
Build
• Documentation &
Solution Architectures
Transform individuals to transform business
Transformation of individuals → teams → organizations
Darwin Schweitzer | WW INTELLIGENT CLOUD – Big Data / AI Advanced Workload LeadWorldwide Commercial Business (WCB) – Intelligent Cloud(425) 638-9068 | darsch@microsoft.com | @DataSnowman | GitHub DataSnowmanPlease check out Data and AI and IoT resources at https://github.com/Azure/data-ai-iot
Thank you
Appendix