AI for Smart City Management

Open Lab & University-as-a-Hub Model

Kim Khoa Nguyen, PhD

Associate Professor, University of Quebec’s École de technologie supérieure, Montréal, Canada

Sustainable Smart Cities for Innovation Enablement

“A smart sustainable city is an innovative city that

uses ICT and other means to improve quality of life,

efficiency of urban operation and services, and

competitiveness” – ITU Definition

Steps to build a smart city

Establishment of smart infrastructure: living labs, innovation

networks

Clear skills gap: education programs, industrial partnerships

Well developed business models: monetize data, financing models

Governance: optimized governance models

Making smart city inclusive: multidisciplinary, gender sensitive

2

Smart Infrastructure

Design principles:

People-Centered

and Inclusive

Infrastructure

Resilience and

Sustainability

Interoperability and

Flexibility

Managing Risks and

Ensuring Safety

3

Smart Cities Architecture

City cloud

Cloud BI Database Data center

Communication

Mobile network Internet Cyber-physic network

mobile PC camera RFID Sensor

network

IP

phone

internet Call

center

wireless Sensor

Emergency Application Digital

city

Enterpr

ise

portal

Govern

ment

service

Health

care

Environ

ment

control

Digital supply

chain Smart

traffic Brain of the

smart city

4

Networked Smart Infrastructure for Smart Cities

Networked infrastructure for innovations

Looking beyond the Internet

Multiple, federated sites

Share experiences, data, technologies

Identify exciting, challenging research

problems

5

6

Canadian Smart City Labs

Toronto Waterfront: waterfrontoronto.ca

Partners

Google (Sidewalk Lab), city of Toronto, Federal government

Focus on: smart transport, smart living, digital government

7

Canadian Smart City Labs

Montreal Lab-VI: labvi.ca

Partners

Ericsson, Videotron, ETS

Focus on 5G smart city applications, ecosystem of startups

Networked Smart Infrastructure for Smart Cities: A Canadian Example

8

Networked Smart Infrastructure for Smart Cities: A Canadian Example

Quebec-Windsor corridor (ENCQOR)

Virtual mobile test corridor following the Quebec to Windsor route

• Utilize available test frequencies

Connect and extend university test beds such as Aurora, SAVI

and GreenStar Networks

Utilize key technologies developed in Canada

• Cloud, LTE, Small Cells, WME, CCIC, Broadband, etc. 9

Universities may lead community testbeds projects

For Researchers

Explore next-gen ideas (advanced wireless, low latency cloud, IoT,

etc.)

Explore large-scale systems

For Industrial Partners

Explore new concepts (like 5G or 5G++)

Trials of potential industrial service offerings

For Governments

Try out next-generation policies: spectrum allocation, digital privacy,

digital currency, etc.

University testbeds are at the heart of new Internet transformation

Communication: from Users to Things

Services: from Network-centric to Data-centric

Universities and Smart City Cyber-physical Testbeds

10

Proliferation of network traffic (volume and type)

Highly real-time streaming data

Increased complexity of network and traffic monitoring and analysis

Difficulty in predicting and generalizing application behavior

Too many sources of knowledge to process by humans

Too many black boxes tasks that cannot be well-defined other

than by I/O examples

Need for aggregated value solutions: getting the most out of our

data

etc.

11

AI/ML and Smart City Infrastructure Management: Challenges

12

Smart is all about Data!

Data captured

through sensors Movement

Environmental

quality

Force

Acceleration

Flow

Position

Light

etc.

How is data generated?: Data collection in smart city

13

Smart is all about Data!

What are we monitoring in the network? System & Services

Available, reachable, energy consumption

Resources Expansion planning, maintain availability

Performance Round-trip-time, throughput

Changes and configurations Documentation, revision control, logging

Most used network monitoring software Availability: Nagios

Services, servers, routers, switches

Reliability: Smokeping Connection health, rtt, service response time,

latency

Performance: Cacti Total traffic, port usage, CPU, RAM, disk,

processes

Network monitoring

NOC: Network Operations Centre Coordinate tasks

Report status of network and

services

Process network-related

incidents and complaints

Host tools (ex. monitoring)

Generate documentation

14

Hierarchy of Data Analytics

Network data analytics can be looked at in multiple segments • Historical Analytics: Build data warehouses / run batch queries to

predict future events / generate trend reports

• Near Real-Time Analytics: Analyze indexed data to provide visibility into current environment / provide usage reports

• Real-Time Analytics: Analyze data as it is created to provide instantaneous, actionable business intelligence to affect immediate change

• Predictive Analytics: Build statistical models that can classify/predict the near future

15

Hierarchy of Data Analytics

Each segment of analytics serves specific purposes • Historical Analytics: Campaign & service plan creation, network planning, subscriber

profiling, customer care

• Near Real-time Analytics: Network optimization, new monetization use-cases, targeted services (ex. location-based)

• Real-time Analytics: Dynamic policy, self-optimizing networks, traffic shaping, topology change, live customer care

• Predictive Analytics: traffic demand forecasting, fault avoidance, planed service provisioning

Data is richer when associated to context – layer, location, time of day, etc.

For each type of data, there is a window / meaningful time period of which the data is relevant

16

Data profiling

Profile Profiling

User

Analytics

Content

Analytics

Infrastructure

Analytics

Active subscriber demographics

Crowdsourced data

Geographic segmentation

Network Performance / Quality

Network sensor data (IoT/M2M)

Usage (from DPI)

Consumption data

Content reach

Asset popularity / revenue

Distribution/Retention/Archival

Search / Discover / Recommend

Usage Data (from content source)

Device sensor data

Persistent Location / Presence

Behavioral / Search / Social

Purchasing / Payments

Mobility patterns

Usage data (from device)

Bandwidth and latency

Access types

IP pools

Routes / topology / Path

QoS / Policy Rulesets

Network Service Capabilities

Identity (Persistent)

Demographics

Explicit profile (interests, etc.)

Device(s) and capabilities

Billing / Subscription plan

Catalog / Title

Topic / Keywords

CA / Rights management

Encryption / DRM

Format(s) / Aspect ratio(s)

Resolution(s) / Frame rate(s)

17

Artificial Intelligence

• Ultimate goal is to automate the management of smart city network

• And make it more efficient

• AI is required in all layers of next-generation networks

• AI models are improved with big data collected over time

Sample network data analytics application Traffic differentiation and QoS provisioning with traffic analyzer –

Elephants and mice flows

Sample network data analytics application Bandwidth defragmentation based on real-time monitoring and forecasting

Packet drops pattern detected

Link going down shortly

Predicting and avoiding failures

Load balancing

Optimize resource allocation

Sample network data analytics application Prediction of faults

Sample network data analytics application Network access visualization

UDP traffic in network

TCP traffic in network

Port analyzing

Sample network data analytics application Application traffic visualization

Network Traffic Analytics Framework

Sampling Capturing all packets across the network is no longer appropriate

Overhead (resource consumption, computational time)

Random, deterministic, or hash-based sampling

Flow sampling vs. packet sampling

Sample tools: NetFlow, sFlow

Characterization Port-based characterization

Ex: HTTP, FTP ports

Header-based characterization

Ex: IP packet header

Payload-based analytics

Ex: Deep Packet Inspection (DPI)

Application behaviour-based characterization

Ex. Video, voice, text messaging

Network Traffic Analytics Framework

Modeling Stationary vs. non stationary

Random model (e.g., Poisson) cannot capture traffic accurately

There is self-similarity in traffic

Two factors affecting traffic patterns

Amount of multiplexing on the link: how many flows are sharing the link?

Where flows are bottlenecked: Is each flow’s bottleneck on, or off the link? Do all bottlenecks have similar rate?

Network Traffic Analytics Framework

• Marginals: highly variable

• Autocorrelation: low

Low multiplexed traffic Highly Multiplexed, Bottlenecked Traffic

• Marginals: tending to Gaussian

• Autocorrelation: high

Prediction and Machine learning Linear models

• ARMA, ARIMA (Auto Regressive Integrated Moving Average)

Non-linear models

• GARCH (Generalized Auto Regressive Conditional Heteroskedasticity)

• Gaussian Regression Framework (GRP)

• Neural network: ANN (Artificial Neural Network), FNN (Feedforward Neural Network), RNN (Recurrent Neural Network), ENN (Elman Neural Network), PNN (Propagation Neural Network), MLP (Multi-Layer Perception), etc.

Challenges for Machine learning • Unlabeled vs. Labeled Data

o Most commercial successes in ML have come with deep supervised learning

o There is no large labeled network data sets

• Training vs. {prediction, classification} complexity

o Stochastic (online) vs. Batch vs. Mini-batch

o Real-time requirements

Network Traffic Analytics Framework

Real-time traffic visualization

Network Traffic Analytics Framework

LabVI smart city project University-as-a-Hub Model

28

Big Picture

Smart Grid

Smart

Water

Smart

Assistant

Smart Heating

Smart Air

WiFi SON + Internet GIGA

Student rooms

Public space Lachine Canal

Bus station

AppIoT Platform TCSEP Platform

Green Cloud

Application

providers

Endusers

Data extraction

Visualization

Modelling

Optimization

Statistics

Control

Monitor

Innovate

29

Big Picture University as a Hub model

Synchromedia Lab

Résidence de l’ÉTS

Pub 100 genies

Habitat Évolutif

Quartier de l’Innovation

Videotron/ Quebecor

Ericsson/ Vaudreuil-Dorion

Toronto (SAVI)

30

Enabling Technologies for Sustainable Smart

Cities

31

Technology 1: Smart Sensing

Capture: movement, environmental quality, force, acceleration, flow, position, light etc

Technology 2: City-wide communications

Both fixed and mobile; licensed and unlicensed cellular networks; low power communications (LoRa, NB-IoT, LTE-M).

Technology 3: Cloud computing

Storage, analytics, economic scalability, access anywhere, anytime, high performance, reliability

Technology 4: Big data

IoT is King, Big data is Queen and Cloud is Palace

Technology 5: Artificial Intelligence (AI)

Automate the management of city and make it more efficient

Technology 6: Security & Privacy

Communication encryption, authentication & key, role-based authorization, blockchain

Platform 1: Open Sensing Data Network

• First-ever open sensing network in Montreal and Canada

• All sensing data can be used by SMBs and people

• Currently provide environmental data: air & quality • Will extend to cover other parameters

• Public data will help enable: • Business promotion

• Health care

• Administration / regulation / policy-making

• Etc.

32

Platform 1: Open Sensing Data Network

Enabling R&D for a variety of applications

Smart utility: through temperature, humidity and user behavior analytics, develop new algorithms to save energy consumption (e.g., garage door control of buildings)

Environmental health: analyzing the impacts of environmental indicators on human health (e.g., stress, productivity, behaviors)

Positioning and localization: spotting lost objects, directions, etc.

33

Platform 2: Open next-generation communication network

Global controller

Regional

controller

WiFi SON (Videotron/ETS) Self-Configuration (plug and play)

Auto-setup

Auto- neighbor detection

Self-Optimization (auto-tune) Coverage & capacity

Mobility robustness

Load balancing

Self-Healing (auto-repair) HW/SW failure detection

Cell outage detection

34

Platform 2: Open next-generation communication network

Picocell (Ericsson/Videotron) Increase coverage

Increase bandwidth

Lower latency

35

Platform 2: Open next-generation communication network

Public LoRa network (ÉTS) Low-power, long distance

Publicly accessible to all IoT objets

Additional service: localization, spotting

Smart Residence

Habitat Evolutif

36

Platform 2: Open next-generation communication network

LiFi (Videotron/GlobalLiFi) Internet access via visible light

Ultrahigh security and speed

Low-power consumption

37

Platform 3: Open cloud infrastructure

Synchromedia cloud (ÉTS) Based on Green Sustainable Telco Cloud & GreenStar

Network & SAVI

Featuring software-defined networking (SDN) and network function virtualization (NFV)

Green and awareness

38

Platform 3: Open cloud infrastructure

Ericsson AppIoT (powered by MS Azure) The Application Platform for Internet of Things

Calculations based on device sensors

Acting on data

Analyzing the data

AppIoT

39

Platform 4: Big data analytics

New data analytics techniques

Reconstruction of incomplete data

Complex event processing (CEP)

Fuzzy clustering and real-time classification

40

Platform 5: Artificial intelligence

Various models developed by many partners Ericsson: B2B AI model

Synchromedia: AI platform for cloud and IoT network management

NyX-R: Environmental pollution learning

Evey: User experience learning and adaptation

etc.

41

Platform 6: Data-centric security

42

Trust-as-a-service using blockchain (Ericsson) Focus on data integrity

Data-centric: every data asset is tagged, tracked, located, verified

Immutable validation of endpoints: every user and all devices

Perimeter-centric: access, control, encryption

Thank you!

43