University of Bologna, Italy. March 3, 2009 - unibo.it · University of Bologna, Italy. March 3,...

Towards Sensible Reality: Ubiquitous Intelligent Multimedia Services

University of Bologna, Italy. March 3, 2009

Prof. Theo KanterDepartment of Information technology and Media

Mid Sweden University, Sundsvall, Sweden

2009-03-04 Sensible Reality; Theo © 2008 2

What is this talk about?

• In our daily life we interact with many things whilewe need to get things done

• Rather seldom things understand what we want to achieve

• If only information technology and communicationcould make things easier and more effective.


Overview

• Mid-Sweden University• Research Computer Engineering• Motivations• Approach incl. Project overview• Early results• Conclusions

Mid Sweden University (MiUn)Facts and Figures


Mid Sweden University• Close links with industry and commerce, local authorities, governmental

authorities and other bodies in the region.

• Multi-campus network university with centres in Härnösand, Sundsvall andÖstersund.

• One of the ten largest institutions of higher education in Sweden. – Undergraduate courses and research are organised principally under the two

faculties of the Human Sciences and Technology and the Natural Sciences.

• Research– Six priority areas in which it already has an extensive commitment. – Clearer role in the research community in co-operation with regional actors


Facts

• 3 campuses • 700 independent courses• 50 programs of study• 30 master programs• 15 000 students • 900 employees • 184 graduate students• 54 professors


New Undergraduate Program

• International Bachelor's Programme in Computer Engineering, 180 higher education credits

• Weblink


Sundsvall


Main Campus Sundsvall

ResearchComputer Engineering


Challenges

Sensor and User Information

EverywhereConnectivity

Multimedia Networked MediaPOTS, GSM, IPTV, 3DTV…

Social MultimediaFacebook, Twitter, WoW,…

Internet of ThingsSensible Things that Communicate

(STC@MIUN)


Internet

Sensible Reality (SR)

Intelligent, interactive services combining multimedia, sensor networks, fixed & wireless communication

VisualizationInteraction

User & Sensorinformation

SensorsActuators

SeamlessMultimodalServices


Sensors

Ubiquitous Context-Aware

ServicesRealistic 3DTV

Digital Communities

Wireless Sensor Networks

[Computer Engineering]

Sensible Reality

[Archive]

[Electronics]

[Media]

MediaSense

[Informatics]


MiUn SR MiUn SR

HIG, FOV

Luleå TU

BTHFraunhofer

Bergen

Cooperation

KTH, ER, Acreo

Moldova

IIT Madras

BTU,BJTU

LTH

LiU,CMIV

ZU

XTU, XJU

JU

Bologna


Cooperation

MiUn MiUn

HIG, FOV

Luleå TU

BTH

Fraunhofer

Bergen

KTH, ER, Acreo

Moldova

IIT Madras

BTU,BJTU

LTH

LiU,CMIV

ZU

XTU, XJU

JU

Bologna

VTT

UCLConcordia

EJL


Testbed

Uppland/Södra Sverige

Gävle

SöderhamnBollnäsEdsbyn

SandvikenDalarna/Västra Sverige

Bergsjö

Ljusdal

Västernorrland/Norra Sverige

Hudiksvall

Hofor

Ockelbo

Jättendal

Fiberson Lab

GävleborgSundsvall

Härnösand

Bispgården

ÅngeJämtland

Örnsköldsvik

Sollefteå

Västerbotten

Kramfors

Norrbotten

KTH, CIRC

FOV

LTU

Sensible Reality - Motivations


Abstract• Sensible Reality : providing users with intelligent applications and services

that are available everywhere• Mobile telecommunications increasingly rapid development enables

communication between individuals irrespective of time and place.• Furthermore, an increasing range of types of wireless communications

and sensors allows us to gather information about our environment as well as users.

• Wireless Sensor Networks makes these issues even more urgent.• Hence, we can utilize such information for enabling services and

applications more intelligently for what we want to achieve whenperforming tasks.

• Conversely, communication networks can become flexible in delivering services and applications to us, thus minimizing our own efforts to reach services and applications.

• Moreover, we can also exchange and access information that is important to the tasks we wish to perform.

• Thus, we can take control and be more effective in our lives, mandating the study of the utility and usability of the new support.


Bus Mysteries

• Getting off the bus is the easy part. • Getting off at the right location can

already be a challenge.• And now watch the reverse…


Like driving in the dark?

• I really don’t…• What was that shape on

the right side?• Can you read the sign?• What’s that light coming

towards us?• And more importantly,

should I do somethingbefore it is too late?


Who needs a parking meter and why?


How informational are public displays?

Am I payingfor this!?


Who needs TV anyway?


Manage this pile?

• Pick up your pen and paper and start counting…


Does it really have to be this way?

• We need to rethink how we do things• Every example so far has items that can behave far

more intelligent and rational• Information and communication technologies exist

and some are lacking. • Also we need to understand how to connect them

and integrate them into everyday life• That is where the research is.

Approach


Current Research Projects

• MediaSense (2008-2010) - EU Mål 2: 19.7 MSEK– Fiber Optic Valley, Ericsson Research, ServaNet, Åkroken Science Park,

MediaTeknik, Swedish Connection, Arkaden, Länsstyrelsen, Sundsvalls kommun

• InterakTiV (2007-2010) – VINNOVA: 3 MSEK– ITV Arena, Northport, Rivermen, Arena for digital services, IHT, WII

• Testbed for Medical 3D (2008-2010) – KK+VINNOVA, 3 Mkr– Center for Medical Image Science and Visualization (CMIV), Lund

University Graphics Group , Setred AB, KK-stiftelsen, SFS, VINNOVA, Vårdalstiftelsen, Invest in Sweden Agency

• Digital 3D Signage (2008-2009) - EU Mål 2: 3.4 MSEK– NetCorner Europe AB, Nordic Fast Food AB, Länsstyrelsen


InterakTiV + MediaSense• InterakTiV• Development of a technical

infrastructure for multi-distribution of TV-based services.

• Development of software-based tools for integration of interactive services in multi-distribution via various TV-media.

• Development of seamless distribution of interactive services via various media.

• MediaSense• Creating a technical platform

combining– Multimedia, – Sensor- & User information– Fixed and mobile infrastructure

• Creating interactive multimedia services focusing on user needsand utility– Logistics, transport, environment

• Investments in broadbandnetworks to insure QoS and availability


MediaSense Sponsors & Partners• System Vendors

• Access Providers

• Service Brokers

• Business Innovation

• Public Sector

• Service Provisioning


Platform

Profiles

Service Provider

EnablersAccess Networks

STB HGW Mobile

User Domain

Access Service Service Broker

Sensors

MediaSense Approach

Media

Multimedia Delivery & Infrastructure

seamless

Sensor & User Information

Sensor information

Multimedia Service Control

Proxy Agents PAs


Pilot Project

• Provisioning of Sensor and User Information to Mobile Users and the Internet– MJCF for Mobile Clients and Application Servers– Associate Sensor Data with Users via IMS Presence– Assume reasonable update rates for Sensor Data– Utilize Sensor Data from Presence in a Mobile Application

• Make Sensor Information available in Mobile Network– Sensor Data Gateway connecting Sensor Network to

Mobile Phone via Bluetooth


IMS Innovation

Provisioning Development

MiUn

MediaSense Testbed3G cell phones can communicate with the IMS networks through the SIP protocol

To the cell phones we may attach different sensors, e.g. by Bluetooth

InternetIMS

SBG

Presence & Group Management

MiUn

SIP

HTTP

MediaServer

Sensor data

MiUn


Staff Related to SR/MediaSenseSenior Faculty• Prof. Theo Kanter, Project leader• Prof. Tingting Zhang• Dr. Stefan Pettersson• Dr. Patrik Österberg• Dr. Ulf Jennehag (post doc F-IIS)

Associate• Prof. Youzhi Xu• Prof. Karl W. Sandberg• Prof. Oliver Popov• Doc. Mårten Sjöström• Dr. Roger Olsson• Dr. Katarina Lindblad-Gidlund• Fil. Mag. Sheila Zimic• Fil. Lic. Aron Larsson• Prof. Love Ekenberg• Prof. Bengt Oelmann

Graduate Students• Techn. Lic. Linda Karlsson• Techn. Lic. Rahim Rahmani• M.Sc. Jamie Walters• M.Sc. Xin Huang

Industrial Graduate Students• M.Sc. Hans Eric Sandström• M.Sc. Karin Nygård-Skalman• M.Sc. NN• M.Sc. NN

Junior Lecturers• Nayeb Maleki• Martin Kjellqvist• Mikael Hasselmalm• Magnus Eriksson


MediaSense Pilot 2008 Student Projects

• Mobile Distributed Context eXchange Proxy• DCXP and SIP interworking• Fast Location-Scoped Context Dissemination• Mobile Context Modelling (>GML)• Seamless Media Session transfers• Low Power BT link between Mobile and Sensors• Sensor Community broker service• Secure Anonymous Sessions• Intelligent Calender

Early Results


Mobile Awareness Application

• Sensor Information about your Friends in GoogleMaps on your Mobile or in your Laptop

• Mobile or Laptop connects via 3G to IMS• Store and retrieve Sensor Information to and from

Presence Profile• Visualization in Mobile or on the Web• Bottom-line: once this is working you can utilize this

knowledge in creating useful services


3G IMSINTERNET


Operator XSBG Presence

Operator Y

Google Maps

Sensor distribution (flow)IMS communication (IMS)

Sensors in Google Earth / Maps

GPS, H2O, °C, …

ApplicationServer

CO2

H2O Acc

GPS¥


Google Earth

Röd bil passerade


MiUn

Internet

IMS

SBG

Presence & Group Management

MiUn

SIP

UDPContext information

MiUnMDPGSN

DCXP

P2P


uci://user[:password]@domain[/path[?options]]uci://[email protected]/weather/temperature?unit=celsius


Context Meta Model

Value

Source

20,34

Representation

Object

Aspect

Temperature

CarTemp=20,34Type=Celcius

Temperature

0..*

0..*

11

1

0..*

Representation1 1

1..* F(x) =standardF-1(standard)=x

Dimension1 1

RealNumbers


Bluetooth Sensor Data Gateway• Low power, flexible development sensor platform with BT link

– Several interfaces to measure, verify and enlarge the functionality of the platform

• BT link between mobile phone and sensors– Class 1 BT module; output power can freely reduced by software (max.

14.5dBm)– Sensor data is sent with a standard protocol using ASCII via BT link to the

mobile• Sensors

– An internal temperature sensor offers temperature values.– A wide range of sensors can be connected to the platform – A microSD card socket for future purposes. E.g., storage.

• Power control– Power supply currently a rechargeable Lithium battery. – Ultra low power capabilities: deep sleep = 40uA or when sending out every

second the internal temp. value: 20mA– Significant for overall power consumption are the BT module and sensors. By

reducing their duty cycles, the battery lifetime can tremendously increased.


Bluetooth Sensor Data Gateway

MCBT

USB

(2) USART1_UART

(1) USART1_UART (3) USART1_UART

USART1_SPI

interrupt

analogTemp.Sensor

MemoryCard

interrupt

USCI_A0

UAR

T & SPI

USCI_B0

SPI

SPISPI & I2C

Sensor Connector

Conclusions


So is a bus worth waiting for?

• A bus stand protects usfrom weather and wind, sure…

• But now this also talks to me and tells me how I am going to get to my destination.


Other bus mysteries

• You can avoid…• Bus drivers knows that we waiting

for that bus • The bus stand tells us where to wait


Darkness? So what?

Decelleration! Road works 1km Building


Need parking meters? Now we know.


Public displays show relevant information!

This is what I neededto know.

ProfilesSTB HGW Mobile Media


IT merges with everyday objects


Conclusions

• Mobile Awareness Application as a proof of conceptof the utility of Sensor Data in Mobile Applications

• Enable sensor information in mobile networks via a Sensor Data Gateway attached to mobile with BT

• Utilize these capabilities in Sensor & Userinformation driven Multimedia Applications via fixedand mobile.

• Connect WSNs directly to 3G networks, not requiringa mobile phone (i.e., mobile data).


Further Steps (In Progress)

• Enabling scalable, massive distribution of sensor data for mobile users and mobile services.

• Context-Aware Networked Media– Interactive homes and personal media

• Environmental Awareness• Commuting Multimedia• FP7 Call 4 and Call 5 Proposals

– Seamless Media Services– Internet of Things: M2M Context-aware Sensing

EndContact InformationTheo G. Kanter Professor of Computer Science, Distributed SystemsMobile +46 (70) 691 8859Mail [email protected]

University of Bologna, Italy. March 3, 2009 - unibo.it · University of Bologna, Italy. March 3,...

Documents

Transcript of University of Bologna, Italy. March 3, 2009 - unibo.it · University of Bologna, Italy. March 3,...