The Future of Microsoft Data Strategy : OpenSource, IoT, Analytics and Power BI ; Global Azure...
-
Upload
azug-fr -
Category
Technology
-
view
511 -
download
0
Transcript of The Future of Microsoft Data Strategy : OpenSource, IoT, Analytics and Power BI ; Global Azure...
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Vincent THAVONEKHAMThe Future of Microsoft Data Strategy : OpenSource, IoT, Analytics and Power BI
1
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
GAB 2016 LYON ORAGANIZER
LOCAL SPONSORS
THANK YOU ALL !!
www.iot-world.fr Prochaine édition
21 et 22 mars 2017 à Paris
2
GLOBAL SPONSORS
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
• Description of the Data, traditional vs. new
• What are the problems ?• Solutions, with a focus on Power BI• Conclusion
CONTENT
3
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Microsoft Practice Manager @VISEO
MVP Azure, P-Seller ALM/DevOps
www.thavo.com @vThavo
Microsoft User Group LyonAzure User Group France
Author & co-author DevOps et Azure(France & international)In 2015 :- White paper: Zend /
Programmez!- Round table: live streaming
for Electric Cloud, with Gene Kim (The Phoenix project)
Speaker : Vincent Thavonekham,
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
CONTENT
• Description of the Data (traditional vs. new)
• What are the problems ?• Solutions (with a focus on Power BI)• Conclusion
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCECredits : http://blogs.technet.com/b/dataplatforminsider/archive/2013/06/03/sql-server-2014-unlocking-real-time-insights.aspx
INTRODUCTIONThe Data is an “old” topic addressed by SQL Server
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
VALU
E
DIFFICULTY
INTRODUCTIONKey types of analysis in the Big DataTraditional BI has evolved. Microsoft has grouped them into 2 categories:
Descriptive and Advanced Analytics (part of the « Data platform »
Source :Gartner
Data Scientists (Senior Doctor in
Maths, Astrophysics, …)
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Data Platform leaps in innovation in 2015 !2012
SQL Server 2012
2013
HDInsight
“Our industry does not respect tradition – it only respects innovation.” – Satya Nadella
2015
Azure Machine Learning
Azure Document DB
HDInsight on Linux, HBase, Storm
Azure SQL DB with In-memory OLTP
Cortana Analytics Preview
IoT Services
2014
SQL Server 2014
APS
Azure SQL Database
Power BI
+ BI DataZen
+ Revolution R
Note : OLAP : On Line Analytic Processing OLTP : On Line Transaction Processing
Source : Microsoft
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
INTRODUCTIONKey actors in 2015
2015 2015
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Traditional data warehouse approach
BI & analyticsETL
Staging
+ Sharepoint + Power BI + O365
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Breaking points of the traditional approach
Staging
Increasing data volumes1
Real-time data2
New data types3
Cloud-born data4 $100
B spend on cloud
50% large orgs have hybrid by 2017
40% CRM sold are SaaS
50x Data growth 2010-2020
40ZB Digital Universe 2020
1Trillion Web pages
Source : Microsoft Convergence
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
INTRODUCTIONKey actors in 2016
2015 2016
Spectacular progress of the TOP 3
Leaders !
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
CONTENT
• Description of the Data (traditional vs. new)
• What are the problems ?• Solutions (with a focus on Power BI)• Conclusion
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
What are the problems ? And how to address them ?
• Many innovations in 2015• Too many data• Too many data types• Cold and hot data (i.e. traditional and real-
time)• Need for Agility (don’t know yet how to use
data)• Need really powerful compute to process
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
(Continued) What are the problems ? And how to address them ?
• EXCEL is no more the key peace of Microsoft Agile BI,
• Reporting Service is less and less used,
• OLAP cubes are no more the automatic solution
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Everyone
3rd waveEnd user BI
A new generation of BI Today, BI extends to everyone
Analyst to end user
2nd waveSelf-service BI
IT to end user1st waveTechnical BI
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
FRUSTRATIONS : The old Power BI and its « WOW » effect• Beyond the “WOW effect”, you’ll find the « Ouch » effect !
Ouch : EXCEL 250 Mb
limitations!
Ouch : what ?!?PowerQuery, PowerPivot, PowerView, PowerMap
Ouch : Sharepoint ?
Ouch : Silverlight ?
Ouch : Excel Plugins !
Ouch: Office licences
2010/2013/365 Pro
Ouch : Need admin rights ?
Ouch : … « and what else ?? »
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Event Queuing System
Collection Presentation and action
Event producers
Transformation
Long-term storage
Event Hubs
Storage adapters
Stream processi
ngCloud gateways(web APIs)
Field gateways
Applications
Search and query
Data analytics (Excel)
Web/thick client dashboards
Live Dashboards
Apache Storm on
HDInsight
Devices to take action
Kafka /RabbitMQ /ActiveMQ
Web and Social
Devices
Sensors / IoT
Azure Stream
Analytics
HDFS
Azure DBs
Azure storage
HBase
CURRENT BIG DATA ECOSYSTEM !
Source : Microsoft
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
IoT AS AN INPUT TO THE BIG DATAZOO
M
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
CONTENT
• Description of the Data (traditional vs. new)
• What are the problems ?• Solutions (with a focus on
Power BI)• Conclusion
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
SOLUTIONS (some of them!)• Microsoft is Opening even more to the
world,• Reinventing Microsoft• Many Open Source solutions supported• Following standards
• SaaS + VERY quick Onboarding• Start small with Free offers• “Suites” to simplify usage: Azure IoT Suites (with pre-packaged
solutions), Cortona Analytics Suites,• Focus on PowerBI.com and PowerBI Desktop
MICROSOFT
ANSWERS
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Robust Architecture : LambdaStandards + Generic architecture = OPEN to non Microsoft technologies
Storage Cold Path Analysis
Apache Spark
Apache StormHot Path AnalysisPresentation
ApacheKafka
IoT Hub
Apache Storm on HDInsight
Apache HBASEon HDInsight
Hybrid : MS + Open Source
“Lambda Architecture”
MICROSOFT
ANSWERS
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Azure IoT SuiteZOOM
« Think big, start small » : Microsoft referenced architecture
Pre-packaged solutions:• Asset management• Remote monitoring• Preventive
maintenanceMICROSOFT
ANSWERS
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Cortana Analytics Suite
http://pocavenet-azure-iot-dashboard.azurewebsites.net/
ZOOMMICROSOFT
ANSWERS
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Global Vision of the Microsoft IoTMICROSOFT
ANSWERS
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Event Queuing System
Collection Presentation and action
Event producers
Transformation
Long-term storage
IoT Hubs
Storage adapters
Stream processi
ngCloud gateways(web APIs)
Field gateways
Applications
Search and query
Data analytics (Excel)
Web/thick client dashboards
Live Dashboards
Apache Storm on
HDInsight
Devices to take action
Kafka /RabbitMQ /ActiveMQ
Web and Social
Devices
Sensors / IoT
Azure Stream
Analytics
HDFS
Azure DBs
Azure storage
HBase
CURRENT BIG DATA ECOSYSTEM !
Source : Microsoft
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Tendency with reporting / BI• Reporting Services are “brittle” • and needs IT experts to modify
• No more Reporting Services within TFS/VSTS• Reports not based on OLAP cube anymore• Replaced by a lightweight real-time reports and Power BI.com (1)
• No more Power BI inside EXCEL,• Replaced by a lightweight standalone Power
BI.com(1) https://blogs.msdn.microsoft.com/visualstudioalm/2016/01/11/announcing-reporting-capabilities-for-visual-studio-team-services
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
• The new SQL Server Reporting Services
https://blogs.msdn.microsoft.com/sqlrsteamblog/2016/03/18/sql-server-2016-rc1-whats-new-in-reporting-services
Tendency with reporting / BI
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Zoom on Power BIStarting with a definition
“Power BI is a suite of business analytics tools to analyze data and share insights.
Monitor your business and get answers quickly with rich dashboards available on every device.”
MICROSOFT
ANSWERS
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Zoom on Power BIStarting with a new Portal
===== December 31th 2015 =====
https://powerbi.com
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
New and easy prices
Version pro
Version gratuite et pro===== December 31th 2015 =====
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
IT challenges
Management of Data in the cloud Security, private data, refresh and data updates.
Need to ease the usage
Need an easy tool for both the IT and the business users, to decrease time spent to create reports and visuals.
Keep data up to date Complex to refresh data from various sources, and manage increasing data volumes due to real time.
Integrating with the existing Time and cost.
IT Administration tasks Time spent for high SLA. Cost of the servers and maintenance.
Various Data sources Complex to connect to both On Premise and Cloud (SaaS) Data.
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Power BI – Key benefits and differentiatorsPre-built content packs, consisting of
dashboards and reports, for popular SaaS solutions
Integration with familiar Microsoft products, and commitment for scale and availability in Azure
Intuitive data exploration by using natural language query
Real-time dashboard updates
Fast deployment, hybrid configuration, secure, and integrated with existing IT systems
Secure, live connection to your data sources, on-premises and in the cloud
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
How does Power BI addresses both the IT challenges and the business needs ?
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Data connections Version gratuite et pro
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Connectorsto popular SaaS applications
Today:
Soon:
Version gratuite et pro
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Quick start samples
Version gratuite et pro
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Create reports
Desktop
Version gratuite et pro
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Power BI DesktopPower BI Desktop
Desktop application to explore data and create interactive reports.Easy usage, with Drag & Drop experience.Publish directly to PowerBI.com.
Version gratuite et pro
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Reports
Version gratuite et pro
Azure Stream Analytics
Azure SQL/DW
HDInsight
AS Tabular
PBI Desktop
Excel
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
New custom visuals: Gallery (subset)Funnel
Combo Chart
Gauge
Tree Map
Fill Map
Version gratuite et pro
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Dashboard
Azure Stream Analytics
Azure SQL/DW
HDInsight
AS Tabular
PBI Desktop
Excel
Version gratuite et pro
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
« Ready to go » content packs :to be used against popular SaaS applications
Version gratuite et pro
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
« Ready to go » content packs :to be used against your data sources
Version pro
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Live data sources Version pro
Consume live data sources with full interactivity
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Power BI Personnal Gateway Version pro
Dataset
Access your on-premise data using the Data Connectivity Gataways
(Personal and Data Management)
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Data updates : Refresh strategies
Real time data streaming in your dashboard- Using Power BI REST API- Using Azure Stream Analytics
Live connectivity to Microsoft SSAS - Auto-refresh- The Data are updated when the user accesses a dashboard or a report
Planned or automatic refresh- SaaS connectors (updates every 24h or planned or on-demand)
- Personal Gateway : planned or on demand refresh
Data refresh architecture
Live query
Popular SaaS Solutions
Live dashboards and reports
Dynamics Marketing
Power BI
SQL Server Analysis Services (SSAS)
Secure Credential
Store
PRO Version1 million rows/hour
FREE Version10 000 rows/hour
PRO VersionEvery hour
FREE VersionEvery day
https://support.powerbi.com/knowledgebase/articles/474669-refresh-data-in-power-bi
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Sharing reports and personal Dashboards
Version gratuite et pro
Share dashboards with others within your orgianization. • Shared dashboards (and related reports) are Read-Only. • They cannot be saved volontarily• The users cannot access the underlaying datasource• The users always have an up to date view
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Team working collaboration
Version pro
Team working with Office 365 groups within Power BI
Create, publish and display contents packs of the company
Manage access control and sharing via Active Directory groupes
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Stay connected from any mobile devices
• Support for iPad, iPhone, and Windows
• Get alerts when your data changes
•
Share and collaborate with your team. Take collaborative décisions
Version gratuite et pro
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Query your data by asking a question in Natural language
Version gratuite et pro
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Push data
Push your data directly into Power BI from your apps.
Integration
Add Power BI to the workflows of your apps.
Develop a real-time connection from your data sources, applications, IoT devices, …
Real-time
{ name":{String}, "tables":[ ..." ]}
0110010100001010
“name”:[string],“columns”:[String]“daType”:[String]
{ "name":{String}, "columns":[ { "name":{String}, "dataType":{String} },
Visuals
Create your own visuals for Power BI
Power BI pour les developpeurs
Version gratuite et proDéveloppements spécifiques
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
Only ONE access point : Powerbi.com(for both Desktop and Cloud Service)
Sign up for a free Preview account
Take the Power BI Tour
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
EXAMPLE 1 : Integration with Azure servicesUse Azure Stream Analytics to push live ‘near-real time’, streaming data to Power BI
– Enables real-time dashboards at scale, over data from devices and applications
– Can stream millions of events per second– Can perform aggregation over time windows
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
EXAMPLE 2 : Integration with Azure servicesAdvanced analytics
Coalesce Azure services together to drive advanced analytics:
– Azure HDInsight: Big Data processing
– Azure Machine Learning: Predictive analytics
– Azure Data Factory: Orchestration at scale
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
CONCLUSION• STOP thinking Data the traditional way
• « Mobile First, Cloud First » (Nadella Satya)
• “Microsoft: opening the Windows to the world” ( Edouard Danel, 2016)
From the inside to the outside” & “From the outside to the inside”
• Microsoft is facilitating (again) the adoption of the new technologies• The NEW Microsoft Power BI (Power BI .com and Power BI Desktop)• Azure Machine Learning, Cortona Analytics, …
Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR
LYON - FRANCE
GAB 2016 LYON ORAGANIZER
LOCAL SPONSORS
THANK YOU ALL !!
www.iot-world.fr Prochaine édition
21 et 22 mars 2017 à Paris
57
GLOBAL SPONSORS