Post on 12-Aug-2020
Data
Intelligence for
Network
Operations
Grid Analytics Fiona Fulton/Alex White
SP Energy Networks
3www.spenergynetworks.co.uk
What is the Data Problem for a DNO?
Other Feeds
Smart Meter Data
Smart Grid
Monitor
Data Management
Too much data
Data SharingStandards
Visualisation
Decision Support
4www.spenergynetworks.co.uk
Yes, but the real problem is……
Work / Time Management
DTN Gateway
DUOSBilling
SPEN DTN Gateway
DocumentManagement
CRM
Outage & Distribution Management (OMS/DMS)
Plant Monitoring
GIS WorkManagement
Regulatory and Asset Volume Reporting
FieldWork Management
New Connections
SCADA
Land Agreements
ProjectPlanning
MPRS
Contractors Management
Mobile Communications
Telephony
Asset Management
Premise / Occupants Dbase
Network Modelling &
DesignStreetworks
I have all these data silos, not quite how I want it but I can’t
break anything……
5www.spenergynetworks.co.uk
I’d really like it to be more like this….
Model Viewer
Integrated Scheduling
Smart Data Management
DTN Gateway
Smart Meter Inventory
DUOSBilling
Data Integration Layers
Smart Gateway
SPEN DTN Gateway
Events Processor
DocumentManagement
SmartNetwork
Data StoreCRM OMS/DMS
Plant Monitoring
GISWork
Management
Business Intelligence Real Time
Business Intelligence
Analysis
FieldWork Management
Network Modelling & Design
New Connections
SCADA
Land Agreements
ProjectPlanning
MPAS
New Contractor
Portal
SmartDCC
Regulatory Reporting
Mobile Communications
WorkManagement
Telephony
Asset Management
Smart Grid Monitoring
Premises Database
Bits in red already in progress for smart meters…..
6www.spenergynetworks.co.uk
The Data Problem
Smart meter investment started me on the road to a better platform, but I’m still not using cross-
system data as effectively as I could
We often rush to analytics to use data more effectively and the data discovery/context
problem remains
Networks aren’t social media. We need to know your connectivity context to be useful. With just a
data lake, we drown…..
We need to look at a small example of the “data in the network context” problem to understand what
can really work effectively
We need DINO………..
7www.spenergynetworks.co.uk
Adding DINO to the masterplan….
Model Viewer
Integrated Scheduling
Smart Data Management
DTN Gateway
Smart Meter Inventory
DUOSBilling
Enterprise Service Bus (ESB)/Integration Layers
Smart Gateway
SPEN DTN Gateway
Events Processor
DocumentManagement
SmartNetwork
Data StoreCRM OMS/DMS
Plant Monitoring
GISWork
Management
Business Intelligence Real Time
Business Intelligence
Analysis
FieldWork Management
Network Modelling & Design
New Connections
SCADA
Land Agreements
ProjectPlanning
MPAS
New Contractor
Portal
SmartDCC
Regulatory Reporting
Mobile Communications
WorkManagement
Telephony
Asset Management
Smart Grid Monitoring
Premises Database
1. Complex Event Processing
2. Integrated Network model
3. Visualisation
1
2
3
8www.spenergynetworks.co.uk
DINO: Future Data Management Vision
Data Marshalling Strategy
• Intelligently recover data when
things happen – not all the time
• Build an Integrated Network
Model to link the data and
navigate between related items.
Data Mastering Strategy
• Use appropriate techniques to align
data between systems.
• Implement data quality KPIs to help
govern the data.
• Provide data stewardship, workflows
and tools.
Volume? Not so high, compared to e.g. social media
Velocity? Some high, but little needing true real-time response
Variety? Mostly structured, albeit with some complexity
Big Data?
9www.spenergynetworks.co.uk
DINO: Steps to addressing the data problems
1) Start somewhere
• Choose meaningful use cases,
that address real world problems.
• Try the techniques out in and with
the business.
• Start “thin” in the framework of the
wider vision.
2) Understand the use case
• Where the data is mastered, its
lifecycle, ownership?
• What is adequate data quality?
• Understand retention and
marshalling requirements?
• Understand the relationships
across systems.
3) Model the whole system
• Build an Integrated Network
Model across key systems
and domains
• Typically includes DMS, GIS,
AMS and Communications.
• This is the “data context
backbone” of the distribution
system.
4) Visualise the system
• Exploit the integrated network
model to build the rules.
• Use modern flexible technology to
expose the data meaningfully.
• Expose the structure, context and
state of the holistic system.
10www.spenergynetworks.co.uk
The initial use case
Use Case Example: NCP Alarms – RTU Battery Failure
• Unfolds over several days
• Scenario is cumulative
• 3 event types in complex scenario
• 1 virtual event generated
0 hrs 4 hrs
Operations OperationsOperations
RTU Loss Of Volts
DelayedNotify
24-48 hrs
BatteryAlarm
CommunicationsFailure
SCADA
+N hrs
www.spenergynetworks.co.uk 11Data Intelligence for Network Operations
Using data to improve network operations
Step 1: Start with the network context
What are they?
Where are they?
Network Assets
Distribution
Network Model
Distribution
Asset Model Communications
System Model
How are they connected?
How are they interrelated?
Network Topology
What can I see?
What can’t I see?
Communications
A whole system model is
required for whole system
understanding
Whole System Model
Requires master data
quality and data alignment
over the whole system
model
Information Confidence
Requires near real-time
visibility trusted against
whole system model
Situational Awareness
So I know - what’s happening now!
www.spenergynetworks.co.uk 12Data Intelligence for Network Operations
Using data to improve network operations
Step 2: Put events in context – Event Processing
Complex Events
Elem
ents
SmartModelling
Event sources
DynamicViews
Co
nce
pt
Visualisation1 2
3 4
51
24
y
z
z
z
x
yzz
Val
ue Volume
VelocityVariety
FilterCorrelate
Reduce Complexity
ConnectRelate
Understand& Action
www.spenergynetworks.co.uk 13Data Intelligence for Network Operations
Using data to improve network operations
What does this look like? Reduce Complexity - Visualise
Currently this picture is difficult to manage with
existing systems under extreme conditions.
In the future with the advent numerous new event
sources across multiple domains the situation is
likely to be unmanageable.
Detail
Location
aware
SummarySummary
Immediate
insights
25-Nov-15There are fewer actionable insights (complex
events) than the many raw alarms (simple events)
DINO is refining business cases to demonstrate
www.spenergynetworks.co.uk 14Data Intelligence for Network Operations
Using data to improve network operations
What does this look like? Reduce Complexity – Filter, Correlate
Drilldown to simple
events
www.spenergynetworks.co.uk 15Data Intelligence for Network Operations
Using data to improve network operations
What does this look like? Connect, Relate - Visualise
Topological
Context
Physical
Assets
Semantic
relationships
Logical
objects
www.spenergynetworks.co.uk 16Data Intelligence for Network Operations
Using data to improve network operations
Our Example - stage 1
RTU Loss Of Volts Received, after 4 hours Timeout Generated
www.spenergynetworks.co.uk 17Data Intelligence for Network Operations
Using data to improve network operations
Our Example - stage 1
RTU Loss Of Volts Received, after 4 hours Timeout Generated
www.spenergynetworks.co.uk 18Data Intelligence for Network Operations
Using data to improve network operations
Our Example - stage 2
Battery Alarm Occurs
www.spenergynetworks.co.uk 19Data Intelligence for Network Operations
Using data to improve network operations
Our Example - stage 2
Battery Alarm Occurs
www.spenergynetworks.co.uk 20Data Intelligence for Network Operations
Using data to improve network operations
Our Example - stage 3
Finally Comms Fails
Spatial
context
Progression
Situation
www.spenergynetworks.co.uk 21Data Intelligence for Network Operations
Using data to improve network operations
Our Example - stage 3
Topological
context
Progression
Impact
22www.spenergynetworks.co.uk
Conclusions: Build on success
We quickly demonstrated the use of an INM
• No handcrafting of data from legacy systems
• Parallel running with existing systems
Quickly showed the business user the value
• The initial use cases were ones raised by the business
• We were able to add new ones with the INM platform quickly to show flexibility
What now?
• This was a Proof of Concept
• We are evaluating how the INM concept can actually be built into our infrastructure
Any Questions?
Grid Analytics Fiona Fulton/Alex White
Fiona Fulton
SP Energy Networks
ffulton@spenergynetworks.co.uk
Alex White
CGI
alexander.white@cgi.com