Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media...

26
Budapest University of Technology and Economics, BME Networked Media in the Future nternet

Transcript of Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media...

Page 1: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Budapest University of Technology and Economics, BME

Networked Media

in the Future nternet

Page 2: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Motivation

• Future Internet --- Internet of media

• content delivery should rely on service-

centric networks

• New requirements in knowledge

representation and analysis

2

Page 3: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Media Informatics Outline

Networked electronic media

Information and knowledge management

Human-computerinteraction

Media technologyContent delivery

Content productionweb 2.0 technologies

DatabasesKnowledge representation

Data- and text miningIR & CBIR

Personalized access to media

Natural language interfaces(Voice & visual ) Dialog systems

Cognitive interfacesContext identification

3

Page 4: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Networked, electronic media

Media technologyContent delivery

Content productionweb 2.0 technologies

Digital audiovisual archiving

Content Management Systems

IPTV apps.

Mobile TV apps.

3Play apps.

Streaming media analysis and miningVideo based trackingVideo based traffic countAudio and video based in-home health care solutions

4

Page 5: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Information and knowledge management

DatabasesKnowledge representation

Data- and text miningIR & CBIR

Personalized access to mediaText processing

Intelligent text retrieval

Domain specific search engines

Text categorization

Collaborative filtering

- in information filtering

- in recommender systems

- localization based services

- advertisement and content targeting

IndexingKnowledge retrieval by fuzzy

pseudo thesaurus Multi Criteria Decision Analysis

5

Page 6: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Human-computerinteraction Natural language interfaces

(Voice & visual ) Dialog systemsCognitive interfaces

Context identificationContext identification in language processing

Question to SQL Query translationDeep Web Search EngineAnaphora resolutionGenitive phrase meaning discoveryText to ontology building

Inferring from natural language corpusPersonal rating (‘vox populi’)Context targeting and content recommendationsTerm extractionTextual corpus based decision support systems

Pattern recognition in images- recognition of pedestrians, signals- identification and classification in time series- photo based recognition- signature, face, handwriting identification- impression identification of imagesVoice, tone and feeling identification in audio

6

Page 7: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Context recognition

7

Partneroldali vékonykliens

Szavak hálójában

Magas szintű modell

Nyelvi

elemzőKontextus-

felismerő

Mély háló

motor

Mediátor

rétegMély háló

adatbázis

Válasz-

elemzés

Kérdés

Formális mondat

CL kifejezés

Általános SQL

Eredmények

Rangsor

SQL

Abstract model

Context

recognizer

Morpho

analyzer

Deep web

engine

Answer

analyzer

Mediator

layerDeep web

DB

Page 8: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Context recognizer

• Determines the context(s) of queries

• Identifies the focus and the schema of the query

• Determines the role of each informationgranule of the query and assigns them toappropriate attributes of the identifiedschema

• Creates so-called CL (Context Language) expressions that is the basis of SQL queries

8

Page 9: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Multi Criteria Decision Analysis in engineering areas

• Info-communication networks

• Fault spreading problem

• Routing tasks

Mathematical modelling toolsAnalytic Hierarchy Process (AHP)

Analytic Network Process (ANP)

Bayesian Network

Method combining expert knowledge

and probabilistic theory to analyse fault

spreading problem

9

Page 10: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

• Task: extraction of feature vectors

characteristic to objects

• Referential comparison of different

categorization methods

Page 11: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Method

• Originally used for text documents

• Hierarchical taxonomy tree

• Each node has a processing unit

• Each processing unit outputs a sigmoid

function of the linear combination of its

inputs

Page 12: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Page 13: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Data Mining Competitions

• ACM KDD Cup (classification of internet search query; automatic identification of pulmonary embolisms (PE) from 3D computed tomography data)

– First place in patient classification

– Runner-up in query categorization precision

– Runner-up in PE identification

– Runner-up in creativity

• ACM SIGKDD Challenge on Time Series Classification (general purpose time series classification task)

– 3rd place

• IEEE Conference on Data Mining – 5th in precision on high precision in-door localization

• Netflix Prize Competition– 2nd place

13

KDD Cup is the annual Data Mining and Knowledge Discovery competition organized by ACM, USA

Page 14: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

integrated 3D mouse

Integrated touch-based actuator

haptic vibration feedback

Audio (speach recognition and synthesis 3D input devices

Cognitive interfaces

14

Page 15: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

- RT middleware

- ICE based TRT

middle ware

-Platform independent

- 3D Internet

-Collaboration Arena

VirCA operation system - network

15

Page 16: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Co-operation networks

• Emerging Information Technology in

Manufacturing (ITM) www.itm-lab.eu– Norwegian University of Science and Technology,

Narvik University College, BME, MTA-SZTAKI,

Budapest Tech (BMF), Széchenyi István University

• Intelligent Integrated Systems

Japanese-Hungarian Joint Laboratory

www.iisl-lab.org– University of Tokyo, BME

16

Page 17: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

7% - verbal communication; 38% - paraverbal communication; 55% - non-verbal signals, such

as gesticulation, posture

3D Motion capture

systemsSpeech recognition

17

Page 18: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

CAD

matching

Norway Hungary

18

Page 19: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Two approaches:

Cell phone actuator in CAVE

CAVE-enhanced cell phone

Cell phone equipped with:

Motion sensors (integrated 3D mouse)

Touch sensors (integrated touch-based actuator)

Vibrating components (haptic feedback)

<===>

Data glove for position

capture and haptic

feedback

Wireless input device

Mobile 3D Internet

19

Page 20: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Foundation of

Intelligent Space Network

• Dept. of Electrical and Computer Engineering North Carolina State

University

• Computer and Automation Research Institute of Hungarian Academy of

Sciences

• Eötvös University

• Budapest University of Technology and Economics

• National Institute of Advanced Industrial Science and Technology

• Hashimoto Laboratory, Institute of Industrial Science University of Tokyo

• Department of Electrical and Electronic Engineering Chuo University

• College of Information Science and Engineering Ritsumeikan University

• Department of Electrical Engineering Meiji University

• Department of Industrial Engineering, Narvik University College

• Department of Production and Quality Engineering Norwegian University

of Science and Technology

20

Page 21: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Speech activities

• Multi-lingual artificial speech generation

• Speech recognition

• Speech databases

21

Page 22: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

• Multi-lingual artificial speech generation– limited vocabulary (e.g., numbers, date, address)

– multi-lingual TTS (Hungarian, German, Polish, Spanish)

– speech profiles (prosody, variability, individual features)

– expression/emotion presentation (user’s manual <-> news)

• Speech recognition– noise handling (telephone, in-car, …)

– dictation (good quality medical transcription, continuous)

– audio indexing (e.g. radio archives, broadcast news)

– speech segmentation

– emotion detection

– keyword spotting

• Speech databases

• Health, rehabilitation, e.g. Audio-visual speech-pronunciation teaching system

• Acoustic company image, eto-informatics

22

Page 23: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

ETO-comm

• Adaptive artificial character („personality”)

in man-machine communications

– adaptive to the mood, style, activity of the

human being

– Personal gadgets personal partners

• Analogy: the complex communication of

a dog and a human being

23

Page 24: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research 24

Page 25: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

ETO-comm

• Adaptive artificial character („personality”)

in man-machine communications

– adaptive to the mood, style, activity of the

human being

– Personal gadgets personal partners

• Analogy: the complex communication of

a dog and a human being

• E.g. smartphone, robot (sound, vibration,

light, style of messages)

25

Page 26: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual

Israeli Hungarian Workshop on Future Internet Research

Networked Media in TMIT

Media/Content Delivery PlatformsIPTV, mobile internet

User Centric Mediainformation-retrieval, community content,personalisation, integration of UGC and professional content,

Multimedia Searchtagged media, multimedia content analysis,automated annotation

3D Media3D internet, feeling of physical presence

26