Post on 13-Feb-2021
13 October 2015 AllSeen Alliance 1
IoT Edge Processing
JEFF KIBLER (@jrkibler)
VP Tech Services, Infobright
Evolution of edge computing analytics and long-term data retention
13 October 2015 AllSeen Alliance 22
1. IoT Premise and Challenges
2. Exposing Opportunities
3. Directions to Consider
4. Moving from Possible to Practical
5. Wrap-up
Agenda
3
IoT Foundations
Premise and Challenges
13 October 2015 AllSeen Alliance 4
The Life of the End User
Athletics
Multi-billion dollar
industries where 1%
competitive edge
decides careers.
Infrastructure
Increasing reliance on
alternative energy,
permeable surfaces,
and environmental
metering.
Telemetry
Predicting and
improving health
outcomes.
13 October 2015 AllSeen Alliance 5
Premise
Leading Verticals
IoT Presents a Large Market Opportunity
Leading Challenges
• Data Security
• Infrastructure
• Privacy
• Governance
• Industrial Equipment
• Oil/Gas/Energy
• Automotive
• Retail / Restaurants
• Hospitals
• mHealth/teleHealth
• Infrastructure
13 October 2015 AllSeen Alliance 6
Premise
IoT Solutions Today are Sexy, Self-Contained
AlCloud Based
Central Repository
SensorsRules/
Workflow
Alerts,
Triggers,
Actions
Data:
NoSQL: Hadoop (Cloudera, Hortoworks, MapR), Cassanndra, MongoDB
Analytic: Sybase/IQ, HP Vertica, Amazon Redshift, Infobright, Pivitol
Standard Relational: Postgres, MySQL, Oracle, Sybase, Microsoft
Cloud: Amazon, Rackspace, Dimension
Data, Joyent, Cisco, EMC, IBM, Microsoft
Rules/Workflow: Apache Storm, Tibco Streambase,
Software AG Apama, Sybase Aleri, Various Coded in
(Java, Python, Ruby on Rails), TempoIQ
Closed Loop
Message-Response System
13 October 2015 AllSeen Alliance 7
Key Challenge
Added Complexity
Evolve with Simplicity
Centralized Volumes
• Gigabytes to Terabytes
• Terabytes to Petabytes
• Petabytes to Exabytes
• Data Exploitation Demands
• Edge Processing Demands
• Governance, ownership
• Privacy
8
Deliver an IoT Platform that
contemplates enormous
sophistication and complexity in a
delivery model that is intuitive,
accessible, and affordable.”
9
IoT Foundations
Exposing Opportunities
13 October 2015 AllSeen Alliance 10
Gaps Exposing the Opportunity
Data
Major Considerations
• How/where to leverage utility value of data
Edge Processing
• Drivers behind and rationale of edge processing both physical
and/or virtual
Architecture
• Meeting market requirements over time; getting it right today
IoT World Forum Reference Architecture
11
Many will overkill to address the
gaps. The result will be sophisticated
yet hardly elegant solutions.”
13 October 2015 AllSeen Alliance 12
Viewpoints: Now and Future
Vendors and Users
Sample Industry ViewpointsCurrent Equipment Vendor
Drivers Current IoT User Drivers
Vendor Assumptions about the
FutureUser Assumptions about the Future
Industrial Equipment
(Lutron Lighting / Glidden
Paints)
Better product, higher margins,
differentiation, stickiness
Higher uptime; easier servicing
when needed; better results
Control of the silo, data, and
devices. Customers will want the
value add of accessing the data
Devices supplied by multiple vendors
will work together and the data can be
leveraged
Oil & Gas
(FMC / Chevron)
Better products; safer products;
proactive servicing
Safety; efficiency; visibility; uptime;
compliance readiness
Gain product insight and control
devices; value added services
Integrated IoT devices; holistic view
from rig level up
Automotive
(Ford / you)
Better product info;
maintainability; increased
margins; more competitive
Ease of use; comfort, safety;
entertainment
Increasingly autonomous;
changing models; compliance
Fully integrated experience, “car as
device” including data; insurance;
ownership
Retail & Restaurants
(Viking Commercial /
McDonald’s)
Tracking (Beacons); better
equipment maintenance; higher
uptime; customer stickiness
Higher yields per customer; better
operational information; better
uptime; greater sales
Greater level of integration
required;
anonymization requirements
Leverage various IoT silos to create
operational efficiencies and greater
profits
Hospitals
(Lutron Lighting / Mercy Health
St. Louis)
Higher uptime; greater
efficiencies; enhanced supply
chain
Less shrinkage; better compliance;
greater visibility
Silo control with regulatory
oversight; Integrated product
suites
Exceptional level of integration of
patient data and resources; operational
efficiency
mHealth / Telehealth
(New England BioLabs /
McDonald’s)
“must have devices” for
consumers; highly cost effective
monitoring solutions
Health maintenance; physician
accessibility; reduced costs; better
outcomes
Lower cost delivery; shrinking
footprint – becoming invisible;
Lower energy; multi-point;
integration
Greater exposure to data; integration
with home systems; non-intrusive;
lifestyle insights
Smart City
Infrastructure
(Siemens / City of Chicago)
Specific silos
(lighting/rubbish/streets.
Increased efficiency and reduced
cost of service delivery for various
silos
Increasing footprint and product
suite offerings; Mega vendor
based service led engagements
Coordination and orchestration of
holistic data; lower cost and better
service delivery through analytics
13 October 2015 AllSeen Alliance 13
Viewpoints: Now and Future
Vendors and Users
Sample Industry ViewpointsCurrent Equipment Vendor
Drivers Current IoT User Drivers
Vendor Assumptions about the
FutureUser Assumptions about the Future
Industrial Equipment
(Lutron Lighting / Glidden
Paints)
Better product, higher margins,
differentiation, stickiness
Higher uptime; easier servicing
when needed; better results
Control of the silo, data, and
devices. Customers will want the
value add of accessing the data
Devices supplied by multiple vendors
will work together and the data can be
leveraged
Oil & Gas
(FMC / Chevron)
Better products; safer products;
proactive servicing
Safety; efficiency; visibility; uptime;
compliance readiness
Gain product insight and control
devices; value added services
Integrated IoT devices; holistic view
from rig level up
Automotive
(Ford / you)
Better product info;
maintainability; increased
margins; more competitive
Ease of use; comfort, safety;
entertainment
Increasingly autonomous;
changing models; compliance
Fully integrated experience, “car as
device” including data; insurance;
ownership
Retail & Restaurants
(Viking Commercial /
McDonald’s)
Tracking (Beacons); better
equipment maintenance; higher
uptime; customer stickiness
Higher yields per customer; better
operational information; better
uptime; greater sales
Greater level of integration
required;
anonymization requirements
Leverage various IoT silos to create
operational efficiencies and greater
profits
Hospitals
(Lutron Lighting / Mercy Health
St. Louis)
Higher uptime; greater
efficiencies; enhanced supply
chain
Less shrinkage; better compliance;
greater visibility
Silo control with regulatory
oversight; Integrated product
suites
Exceptional level of integration of
patient data and resources; operational
efficiency
mHealth / Telehealth
(New England BioLabs /
McDonald’s)
“must have devices” for
consumers; highly cost effective
monitoring solutions
Health maintenance; physician
accessibility; reduced costs; better
outcomes
Lower cost delivery; shrinking
footprint – becoming invisible;
Lower energy; multi-point;
integration
Greater exposure to data; integration
with home systems; non-intrusive;
lifestyle insights
Smart City
Infrastructure
(Siemens / City of Chicago)
Specific silos
(lighting/rubbish/streets.
Increased efficiency and reduced
cost of service delivery for various
silos
Increasing footprint and product
suite offerings; Mega vendor
based service led engagements
Coordination and orchestration of
holistic data; lower cost and better
service delivery through analytics
Connected Products
System of Systems
14
Deliver an IoT Platform that
accommodates evolving user needs
with minimal user requirements.”
13 October 2015 AllSeen Alliance 15
Lens of a Vendor
Need
Considerations
• Product that performs and adaptable
Data Characterization
• Framing view of product by dataData Use
• Predict and Evolve ProductConstituencies
• Understand User Segmentation
Ownership
• Retain rights to Data
Stewardship
• Data Access by usersManagement
• Controlling devices in the field
Drivers
• Decreased downtime, increased utilization and visibility, Upsell
Outlook
• Integration into larger system of systems
13 October 2015 AllSeen Alliance 16
Lens of a User
Need
Considerations
• Operate efficiently to gain better insight / make better decisions
Data Characterization
• Products and ServicesData Use
• Holistic understanding on data breadth / avoid silos
Constituencies
• Organization or consumer including various silo systems
Ownership
• Own the data
Stewardship
• Determine user permissionManagement
• Product companies manage assets
Drivers
• Cost savings, enhanced outcomes, increased revenue
Outlook
• Move from Silo to greater system
17
IoT Foundations
Directions to Consider
13 October 2015 AllSeen Alliance 18
IoT Direction
Pushing Suppliers for more Robust Analytic Stack
AlCloud Based
Central Repository
SensorsRules/
Workflow
Closed Loop
Message-Response SystemEnterprise Apps:
ERP, CRM, and
other enterprise
apps
Possible Specialized Store
Alerts,
Triggers,
Actions
Analytic Workbench: Operational,
Investigative, Predictive Analytics and
Machine Learning
13 October 2015 AllSeen Alliance 19
IoT Direction to the Edge
Increase in Edge Processing for filtering and increased capabilities
Cloud Based
Central RepositorySensorsRules/
Workflow
Closed Loop
Message-Response System
Enterprise Apps:
ERP, CRM, and
other enterprise
appsPossible Specialized Store
Alerts,
Triggers,
Actions
Analytic Workbench: Operational,
Investigative, Predictive Analytics and
Machine Learning
Edge Processor
• Apply rules and workflow against that data
• Take action as needed
• Filter and cleanse the data exhaust (increasing payload)
• Store local data for local use
• Enhance security
• Provide governance admin controls
Rules/
Workflow
13 October 2015 AllSeen Alliance 20
Edge Processing Assumptions
• Limited or no human resources for maintaining the database or other system capabilities at the edge – must be a hands off operation, with remote monitoring or
control only
• Hardware footprint will be limited
• Not all use cases apply – but many do
– Factories
– Retail/Restaurants
– Homes (but with far less data)
– Buildings
– Many aspects of smart cities
– Hospitals (but only marginally for personal health)
– Cars (Edge on board)
– Other transportation modalities (especially planes,, trains and ships)
– Oil and Gas
13 October 2015 AllSeen Alliance 21
Architectural Considerations
Cloud-Based Central
Repository
Cloud-Based Central
Repository
SensorsRules/
Workflow
(& Filtering)
Sensors
Closed Loop
Message-Response System
Edge Processor
Sensors
Sensors
External Data
Persisted Store
Publis
h
Analytic
Workbench:
Operational,
Investigative,
Predictive
ERP, CRM, etc.
Various Sensor Devices
First Receiver
Cloud-Based Central
Repository
Rules/
Workflow
Analytic Workbench
Enterprise Apps
Cloud-Based Central
Repository
Rules/
Workflow
Analytic Workbench
Enterprise Apps
Sensors
Sensors
Sensors
Sensors
Vendor Corporate (“Lutron Lighting”)
One of multiple vendor silos
User Remote Site (“McDonald’s/South Boston”)
User Corporate (“McDonald’s Head Office”)
Government (“USDA”)
Subscribe
Cloud-Based Central
Repository
Rules/
Workflow
Analytic Workbench
Enterprise Apps
13 October 2015 AllSeen Alliance 22
Local Back End Data Provisioning
SensorsRules/
Workflow
(& Filtering)
Sensors
Closed Loop
Message-Response System
Sensors
Sensors
External Data
Persisted Store
Analytic
Workbench:
Operational
Investigative
Predictive
ERP, CRM, etc.
Various Sensor
Devices & Silos
First Receiver
Cloud-Based Central
Repository
Rules/
Workflow
Analytic Workbench
Enterprise Apps
Cloud-Based Central
Repository
Rules/
Workflow
Analytic Workbench
Enterprise Apps
Cloud-Based Central
Repository
Rules/
Workflow
Analytic Workbench
Enterprise Apps
Sensors
Sensors
Sensors
Sensors
Vendor Corporate (“Lutron Lighting”)
McDonald’s
Vendor Corporate (“Honeywell HVAC”)
Vendor Corporate (“Bosch Appliances”)
23
Data beyond a certain scale becomes
impossible to accommodate and use
without vast infrastructure and
excessive administration.”
13 October 2015 AllSeen Alliance 24
Small Edge Node Volumes Today
� Most data volumes today and in the
near future are exceptionally low by
some standards (like Telco and
Networking)
� The key will be to provide the
underpinnings to service the full
analytic stack and feed enterprise
applications
Hotel Example
Sensors Deployed: 100,000
Avg. Message Interval: 5 seconds
Exhaust Rate: 100
Avg. Message size: 3kb
Data Retention Period: 30 days
Required Message Flow Capacity: 2.16M Messages/Hr
Required Storage: 2.59 TB
AlCloud Based
Central Repository
SensorsRules/
Workflow
Closed Loop
Message-Response SystemAlerts,
Triggers,
Actions
Analytic Workbench: Operational,
Investigative, Predictive Analytics
and Machine Learning
Enterprise Apps:
ERP, CRM, and
other enterprise
apps
Possible
Specialized Store
13 October 2015 AllSeen Alliance 25
Future Unknown Edge Node Volumes
� The combination of many silos with greater
reach along with the augmentation with
external data will create much higher
volumes over time, especially in certain
user cases
� The ability to practically accommodate
massive amounts of data in the future
will be a critical consideration of IoT
architectures
Sensors Rules/Workflow
(& Filtering)
Sensors
Closed Loop Message-Response System
Edge Processor
Sensors
Sensors
External Data
Persisted Store
Pu
blis
h
Analytic Workbench:
Operational,
Investigative,
Predictive
ERP, CRM, etc.
Various Sensor Devices First Receiver
Cloud-Based Central RepositoryRules/
Workflow
Analytic Workbench
Enterprise Apps
Cloud-Based Central RepositoryRules/
Workflow
Cloud-Based Central RepositoryRules/
Workflow
Sensors
Sensors
Sensors
Sensors
Vendor Corporate
User Corporate
Third Party – as needed
Su
bscri
be Analytic Workbench
Enterprise Apps
Analytic Workbench
Enterprise Apps
13 October 2015 AllSeen Alliance 26
Opportunity for Providers and Users
Increased Data Focus
& Analytic Capabilities
Edge & Tier Processing
wherever appropriate
Publish &
Subscribe Architecture
Leveraging the
Utility Value of
IoT Data
27
IoT Foundations
Moving from Possible to Practical
13 October 2015 AllSeen Alliance 28
Metadata Leveraged Architecture
� Establish Metadata at the point
of ingestion
� Provide comprehensive query
tools contemplating a variety of
needs
En
dp
oin
t D
evic
es
1st Receiver
Edge Processors
1st Receiver
Edge Processors
1st Receiver
Edge Processors
Mid-Tier
Edge Processors
Mid-Tier
Edge Processors
Includes
Infobright Store
integrated with
Hadoop for
enhancing
analysis of
machine data
Leverages Metadata throughout the architecture
13 October 2015 AllSeen Alliance 29
Metadata Leveraged Architecture
En
dp
oin
t D
evic
es
1st Receiver
Edge Processors
1st Receiver
Edge Processors
1st Receiver
Edge Processors
Mid-Tier
Edge Processors
Mid-Tier
Edge Processors
Includes
Infobright/Metadata
Store integrated
with Hadoop for
enhancing analysis
of machine data
Leverages Metadata throughout the architecture
Common Tool Sets, Minimal Administration
Affordable and Accessible
13 October 2015 AllSeen Alliance 3030
Gap: Leveraging Data and Analytics
Lack of data leverage and more robust analytic stack will become an increasing
impediment
Gap: Edge Processing
Edge processing is cloud based filtering and workflow for exhaust
Gap: Publish/Subscribe Model
Basic monolithic cloud architectures
Opportunity: Data Cleansed, Enriched, and Published
Analytic stack can be established to provide operational, investigative, predictive,
and machine learning.
Opportunity: Edge Processing / First Receiver
Extensible version of workflow and data cleansing for edge deployments
Opportunity: Event-driven Architecture
Flexible pub/sub architecture for adaptability in demands with constituencies in a
simple, secure, and accessible fashion
Gaps and Opportunities
13 October 2015 AllSeen Alliance 31
Thank youFollow us on
For more information on AllSeen Alliance, visit us at:
allseenalliance.org & allseenalliance.org/news/blogs