Enterprise on Blockchain Let you big data sing€¦ · Backend App Smart Contract Artificial...
Transcript of Enterprise on Blockchain Let you big data sing€¦ · Backend App Smart Contract Artificial...
Enterprise on Blockchain – Let you big data sing
Keynote presentation
by
Professor Dr. Mukesh Mohania
Prepared byE. Chang, S. Green, A. Ghildyal
UNSW@ADFA and Department of Defence7th International Conference on Big Data Analytics (BDA 2019)
Ahmedabad University Gujarat, India18-20 Dec 2019
Content
1. Enterprise Big Data Analytics
2. Why Enterprise on Blockchain
3. Shut your data up or let your data sing
4. Enterprise Blockchain and data migration
5. Examples – Blockchain and AI, let your data sing
6. Enterprise Blockchain – Implementation and future challenges
7. Conclusion - 50 years enterprise data management paradigm
1.1 Enterprise Big Data Analytics
setsBags
Text = Strings
Trees
DocumentsProfiles Folders
Containers
1.1 Enterprise Big Data Analytics
setsBags
Text = Strings
Trees
DocumentsProfiles Folders
Containers
1.1 Enterprise Big Data Analytics
Artificial Intelligence
Big data repositoryBackend
Big Data AnalyticsApplication
Big Data VisualizationFrontend
1.2 Enterprise Data Management
Artificial Intelligence
Big data repositoryBackend
Big Data AnalyticsApplication
Big Data VisualizationFrontend
Ledger
1.3 Future Enterprise on Blockchain
8
Memory
Simply
Environment Growing
Business Growing
Data Growing
Databases Growing
Enterprises Growing
2. Why Enterprise on Blockchain
Evolution of Enterprise ERP
9
BI
2. Why Enterprise on Blockchain
Evolution of Enterprise ERP
• ETL• Data Mart• Data Warehouses • Business Intelligence
Data Mart, Data Warehouse are the core of ERPAimed at efficiency, productivity, RoI, bottom line, customer satisfaction, etc.
However, it did not contribute to BI!
3 tier and 4 tier architecture
10
BI
2. Why Enterprise on Blockchain
Evolution of Enterprise ERP
Operational Issues1. ERP facilitate Silos
2. Propriety databases,
3. Interoperability is impossible
4. Manual data extraction, error-prone
5. Time consuming to extract data
6. Subject to manipulations and overwrite
7. Results are often rejected
8. unable to keep pace with the business growth
3 tier and 4 tier architecture
(CFO publishing, “The high cost of change for ERP” 2009; Gary Simon, CODA group publishing FSN, 2016).
11
BI
2. Why Enterprise on Blockchain
Evolution of Enterprise ERP
3 tier and 4 tier architecture
Enterprise Issue 1. Inflexible and inability to scale2. No single source of truth3. Require Expert to operate4. Heterogeneous data sources, not
for managers or end-users5. High Cost and low RoI6. Vendor in control of Data and DW7. No security, privacy, trust8. Data procurement is difficult
High cost, average $300m per year CFO publishing, “The high cost of change for ERP” 2009; Gary Simon, CODA group publishing FSN, 2016
12
2. Why Enterprise on BlockchainDW and DM has no future!
Interoperability should be forgotten
BI
3 tier and 4 tier architecture
Issue 1: Inflexible • Expensive One-off service,
build once, last once.• Don’t adapt to the changes
of business processes and policies.
• Requires DW and DM to change, when the business or processes are changed, but only Expert can do the job.
• Long lead time to get the change done
• Overtime, the one-off DW/DM only capture or use in-complete data, not the total data set
• hinder the org or enterprise development
13
BI
2. Why Enterprise on Blockchain
• Data lakes
• The end of data silos era
• Single source of truth
DW and DM has no future! Interoperability should be forgotten
14
BI
2. Why Enterprise on Blockchain
Blockchain1. Distributed BC Ledger2. Single Source of Truth3. Public and Private4. Smart Contract5. Transparency6. Immutability7. Open Source8. Traceability9. Consensus10. Anonymity11. Trust
Ledger
TrustSecurityRisks Accountability
DW and DM has no future! Interoperability should be forgotten
Traditional ERP/DB
Shut your data up
• Homogenous
• Schema controlled
• Data Relations, normalised
• No horizontal scale out
• High licensing cost
• Lacks high velocity
• Access through joining tables
• Centralised, single machine
Blockchain DB (NoSQL)
Let your data sing
• Heterogeneous
• Schema-less data
• Non-related and de-normalised
• Optimised to horizontal scale No license – pay as you go
• Rapid Growth
• Rapid access through one table
• Distributed and replicated
Traditional DB cannot meet the dynamic business and operational needs
3. Shut you data up or Let you data sing
15
# Traditional Structured DB/ERPSQL, shut your data up
Elastic and Blockchain DBNoSQL, let you data sing
1 Only SQL (Structured) No just SQL, but also Non-SQL
2 Table, Rows and Columns Collections and Documents
3 Fixed columns, cannot scale out Wide and no limit columns, can scale horizontally and vertically
4 Homogenous data type only (numbers inside table)
Heterogeneous (tables, bags, sets, arrays, strings, nested text, documents, images, maps, sound, scans, movies, PDF, excel, etc.)
5 Centralised server storage, single point of failure
Distributed storage (replication), fault tolerant
6 Consistency Consistency, Availability, Partition tolerance
7 Unable to analyse data quickly Real time data analytics
8 Costly to change and upgrade Dynamic upgrade
16
3. Shut you data up or Let you data sing
ERP & Legacy DBsBusiness Operation
Multi-system access
Blockchain ApplicationTrusted Data and Operation
People/Operation, Policy/Process, Workflow, Report/Data Data, Data/Process Flow, Smart Contract, User/System Interfaces
Data Migration
Process Automation
Smart Contract
Roles and Accountability
Target Business Re/Engineering
Target System Re/Engineering
Push data to Blockchain
Old OperationContinue
One Dashboard
DB1 DB2
SaveSave
New OperationNew Business
Business Logic
Run Blockchain in Hybrid Environment
Target Application and Data Migration
Heterogeneous DBs
Blockchain Data Lake
17
4. Enterprise Blockchain and Data Migration
Blockchain - How
Data Models
One of the two top key difference from the traditional DB
18
4. Enterprise Blockchain and Data Migration
Data Model – An Order in Traditional DB Models
Above, each box is a table. An order is a one-time operation, however, It has to be stored in 4-8 tables. This is due to the normalisation rules (relational calculus), that we need to split the data into tables.When we try to find a particular order, we need to get the data from several tables, called “joint”, this takes time.
19
4. Enterprise Blockchain and Data Migration
Order Table (Tax Invoice)
Tax Invoice #DateTimeSupplier CompanyAddressContact details Item #Item QuantityItem PriceTotal pricePurchaser NameSignatureetc
It is only one table. So, an Order is a No-SQL Table or a ledger. We do not need to have several tables. This is a natural way we do things. When we need to find an Order, we just give the Tax Invoice #, then you have all the information on one query, not several quires to several tables.20
Data Model – An Order in Blockchain Data Model
4. Enterprise Blockchain and Data Migration
For Elastic search, we can indicate what we want to search. Eg, by given an item#, I can find which order it is in; or by given a Total price, we can find out who ordered, where it coming from, and the entire information. The Red highlights, are called “Indexing”.
Order Table (Tax Invoice)
Tax Invoice #DateTimeSupplier CompanyAddressContact details Item #Item QuantityItem PriceTotal pricePurchaser NameSignatureetc
An Order is a No-SQL file or a “Document”, just like “Tax Invoice” is a document, or it is one row in a ledger. So, this is a natural way to treat them as a “document”.A collection of the documents or collection of the Tax Invoices, we call it “Collection” of the document, instead of call it a "Table”.
21
4. Enterprise Blockchain and Data Migration
Data Model – An Order in Blockchain Data Model
Block and Hash to Blockchain Ledger
Creating the DB is very similar to the traditional DB, instead of only structured tables, you can have structured and unstructured tables. You collect the data and “post” them to the corresponding tables. At the end of each data push session, say each Audit at the end of each day, you create a block and hash them /encrypt them using Blockchain infrastructure, it then stored as Ledger.
A Block is a set of collection of the data you want to store, or say a Block or say chunk of a collection, or say Audit at AMT is a Block, …you can set up the block by time, by location, by operation, by task, etc..
22
4. Enterprise Blockchain and Data Migration
Store and Retrieve in/from Blockchain DB
Everything can be stored in NoSQL Databases, and Blockchain works on top of all NoSQL databases. It provides Hash or Encryption for immutability and security. Hash provides us with traceability, we can always use it for verification of the authentication of the data.
The hash, store and retrieve are all build-in by the Blockchain System Development Kit. It is like a TV, you just need to learn the remote control buttons and set it up.
The queries language is still apply, but no need joins of multiple tables, but through Elasticsearch
23
4. Enterprise Blockchain and Data Migration
Security, privacy and permission
● The SDK - wallet systems● Wallets contain the public/private keys (identities)
24
4. Enterprise Blockchain and Data Migration
Business ExpertEnd-user and Business
Requirement Orchestration
Technical ExpertTechnology Implementation Orchestration
(libraries, open sources, AI, GUI, DB)
Front-end BIDashboardGeo-spatial and MapFlow of InteractionAugmented realityGame based
GUI Orchestration
Smart ContractPolicies alignmentProcesses automationOperations updateResponsibilities and Roles
Business Process Orchestration
Backend AppSmart ContractArtificial IntelligenceMachine LearningPrediction AnalyticsElastic search
Analytics Orchestration
Blockchain
BlockchainData lakeData migrationHyper ledger
Data Orchestration
SecurityData SecurityDB securityApp securityNetwork securityMobile security
Security Orchestration
Step-by-step to Blockchain Platform
4. Enterprise Blockchain and Data Migration
Step-by-Step push your data to Blockchain with the Platform
26
4. Enterprise Blockchain and Data Migration
Business Efficiency through Blockchain and AI
2
National Vehicle Sampling procedure
5. Example1 – National Vehicle Management
2
SPO
Tier 2
DAA
Tier 3
ANAO
DFG
CJLOG/LAB
Business Efficiency through Blockchain and AI
5. Example1 – National Vehicle Management
2
SPO
Tier 2
Key Risks• ANAO Category findings
• Threat of qualification
• Defence capabilities
• Financial accountability
DAA
Tier 3
ANAO
DFG
CJLOG/LAB
Business Efficiency through Blockchain and AI
5. Example1 – National Vehicle Management
DFG
CJLOG/LAB
ANAO
3
SPO
Tier 2
DAA
Tier 3
SMEInnovation Cell
Business Efficiency through Blockchain and AI
5. Example1 – National Vehicle Management
DFG
CJLOG/LAB
ANAO
3
Artificial Intelligence
LedgerLedger
Ledger
Ledger
Ledger
Risk 1
Risk 40+
Emerging risk
CASG
DefenceSustainment
SMEInnovation Cell
Ledger
Log bookImageConfirmation sheet
SmartContract
Business Efficiency through Blockchain and AI
5. Example1 – National Vehicle Management
4
AASPO D1713E 661631643Automobile Sedan Camry Toyota
Blockchain solution – let your data sing
5. Example1 – National Vehicle Management
6
NVS MobileApp
• Status Analysis• Status Reports• Error Detection• Problem Solving
Blockchain
• Document Storage• Smart Contracts• Accountability• Accessibility• Visibility • Security
Open-source technology integration
AI
• Data & Text Mining• Automatic Document Reasoning• Automatic Object Detection• Automatic License Plate, Price... Recognition
• Opérationnel Ledger everywhere
• Single source everywhere
• One Enterprise Policy everywhere
• One Enterprise Process everywhere
• One Interface everywhere
• Unified results everywhere
Blockchain solution – let your data sing
5. Example1 – National Vehicle Management
Low CostCapability up,Compliance up,Performance up,DSR up.
Cost down,Error by Value down,Error by Volume down,Discrepancy down,Risks and shame down.
Low Risk
Value for Money
Sustainability
Rapid Update
7
Blockchain solution – let your data sing
5. Example1 – National Vehicle Management
35
Business Efficiency through Blockchain and AI
5. Example2 – Inventory Pricing Management
36
Blockchain solution – let your data singQuarter 1 results
5. Example2 – Inventory Pricing Management
37
Blockchain solution – let your data sing
Quarter 2 results
5. Example2 – Inventory Pricing Management
Everybody Jump on It
38
6. Enterprise on Blockchain - Implementation
39
Private Blockchain
6. Enterprise on Blockchain - Implementation
40
6. Enterprise on Blockchain - ImplementationPeer vs Hierarchy
41
6. Enterprise on Blockchain - ImplementationPeople and Role are important
42
Tracking Asset Outside and Inside Enterprise
6. Enterprise on Blockchain - Implementation
43
6. Enterprise on Blockchain - ImplementationSmart Contract – No change
44
6. Enterprise on Blockchain - ImplementationManaging Smart Contract at both frontend and backend
45
6. Enterprise on Blockchain - Implementation
A hybrid environment and co-exist with ERP / other legacy DB
46
6. Enterprise on Blockchain - Implementation
Blockchain integration with existing DBs
47
6. Enterprise on Blockchain - Implementation
An ordered workflow sequence
Permissioned Blockchain
48
6. Enterprise on Blockchain - Implementation
1. Public Key Infrastructure2. Private keys and certificates for multi-channel private Blockchain3. Automated Smart Contract generation4. Multi-stake-holders and multi-Blockchain 5. Limitations of standards and policy
49
6. Future Challenges - Enterprise on Blockchain
1980 1990 2000 2010 2020
Time
Adaptability
50 years of Enterprise Information SystemThe future Business and Enterprises are with Blockchain
50
51
You should also Jump on It
52
UNSW and ADFA
Comments and Questions to: [email protected]