August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream...

18
Spectrum Environment Awareness (SEA) System Innovative Spectrum Monitoring Solution August 2, 2016 Building a prosperous and innovative Canada Source: Communications Research Centre Canada

Transcript of August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream...

Page 1: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Spectrum Environment Awareness (SEA) System

Innovative Spectrum Monitoring Solution

August 2, 2016

Building a prosperous and innovative Canada

Source: Communications Research Centre Canada

Page 2: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Organisational chart

2

Innovation, Science and Economic Development Canada

(formerly Industry Canada)

… …Spectrum, Information Technology

and Telecommunications

Communications Research Centre

Spectrum Management

Operations

Engineering, Planning and

Standards Branch

Spectrum Licensing

Policy Branch…

Page 3: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Background - Monitoring

• Pre-1990: Manual operation using receivers

and handheld direction finding equipment

• >1990: radio spectrum getting more

congested, interference cases more frequent

and complex

• Creation of Integrated Spectrum Observation

Centres (ISOC)

3

Page 4: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Current Monitoring Infrastructure

• Infrastructure

• 80 Fixed monitoring sites (ISOC)

• 50 mobile monitoring vehicles

• 8 monitoring trailers

• 2 specialty vehicles

4

Page 5: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Why Monitor ?

5

Compliance

and

Enforcement

Spectrum Management

Operations

Policy

Licensing

Page 6: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Our Vision as a Canadian Regulator

Comprehensive Spectrum

Knowledge

Dynamic Spectrum

Assignment

Intelligent Sustainable Spectrum

Management

6

Page 7: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Spectrum Environment Awareness (SEA)

• Using spectrum knowledge to enable more

effective management and to maximize the use of

spectrum

7

Comprehensive Spectrum Knowledge SEA

Page 8: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

SEA System and Functional Concept

Spectrum Environment Awareness

SystemData

Aggregation

Analytics

Intelligent

Tasking

Visualization

Sensors

• Reporting right information at

right time

• Scalable & cost effective big

data analysis architecture

• Heterogeneous & expandable

smart sensing system

8

Page 9: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

SEA Prototype Overview

9

IBM IoTFoundation

IBM Bluemix

Sensor Resource Manager

Sensing Task Manager

Microsoft Azure

BLOB Storage

Event Hub

HDInsightHadoop

R&T DB

Spectrum Data &

Report DB

Stream Analytics

Batch Analysis & Analytics

Real-timeAnalytics

PowerBI

Visualization

Sensors

Control

Measurement Data

SEA Cloud

Infrastructure

To 3rd Party

Sensing

Data Cloud

Machine Learning

Studio

SEA Cloud Infrastructure

Page 10: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Sensors in The Prototype

10

• Spectrum coverage

o From 70 MHz to 6GHz, LMR, LTE, WiFi

• In-house, COTS, 3rd party collaboration

o CRC USRP, Rohde & Schwarz, Cognitive System Ltd.

• The new and the legacy, the stationary and the mobile sensors

o Stationary sensors in buildings, houses and mobile sensors in taxis

o Department’s legacy LMR sensing stations

Dedicated LTE Sensor R & S

Crowdsourcing AppLow Cost Spectrum Scanner WiFi Sniffer

Amera Sensor

CloudSpectrum Explorer

Page 11: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Sensing & Tasking

11

Page 12: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Generating Spectrum Metrics

12

• Use case defined spectrum metrics

• Signal power, noise floor, band/time occupancy, off-set,

loading/congestion, user experience, technology info. etc

Page 13: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Spectrum Data Analytics

13

• Machine Learning &

Analytics

• Clustering

• Time series analysis

• Anomaly detection

Anomaly Detection of Noise Power

Page 14: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Intelligence for Spectrum Management

14

Page 15: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Data Science for Spectrum Management

15

• Near realtime spectrum intelligence correlated and visualized

• Big data science based spectrum management

Page 16: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

Big Data Analytic Centre at CRC

16

Page 17: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating

The SEA Experience

17

• New way for acquiring and using the spectrum knowledge

• Game changer to spectrum regulator

• Spectrum management for 5G

Page 18: August 2, 2016 - ITS · Storage Event Hub HDInsight Hadoop R&T DB Spectrum Data & Report DB Stream Analytics Batch Analysis & ... Cloud Spectrum Explorer. Sensing & Tasking 11. Generating