Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media...
Transcript of Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media...
Budapest University of Technology and Economics, BME
Networked Media
in the Future nternet
Israeli Hungarian Workshop on Future Internet Research
Motivation
• Future Internet --- Internet of media
• content delivery should rely on service-
centric networks
• New requirements in knowledge
representation and analysis
2
Israeli Hungarian Workshop on Future Internet Research
Media Informatics Outline
Networked electronic media
Information and knowledge management
Human-computerinteraction
Media technologyContent delivery
Content productionweb 2.0 technologies
DatabasesKnowledge representation
Data- and text miningIR & CBIR
Personalized access to media
Natural language interfaces(Voice & visual ) Dialog systems
Cognitive interfacesContext identification
3
Israeli Hungarian Workshop on Future Internet Research
Networked, electronic media
Media technologyContent delivery
Content productionweb 2.0 technologies
Digital audiovisual archiving
Content Management Systems
IPTV apps.
Mobile TV apps.
3Play apps.
Streaming media analysis and miningVideo based trackingVideo based traffic countAudio and video based in-home health care solutions
4
Israeli Hungarian Workshop on Future Internet Research
Information and knowledge management
DatabasesKnowledge representation
Data- and text miningIR & CBIR
Personalized access to mediaText processing
Intelligent text retrieval
Domain specific search engines
Text categorization
Collaborative filtering
- in information filtering
- in recommender systems
- localization based services
- advertisement and content targeting
IndexingKnowledge retrieval by fuzzy
pseudo thesaurus Multi Criteria Decision Analysis
5
Israeli Hungarian Workshop on Future Internet Research
Human-computerinteraction Natural language interfaces
(Voice & visual ) Dialog systemsCognitive interfaces
Context identificationContext identification in language processing
Question to SQL Query translationDeep Web Search EngineAnaphora resolutionGenitive phrase meaning discoveryText to ontology building
Inferring from natural language corpusPersonal rating (‘vox populi’)Context targeting and content recommendationsTerm extractionTextual corpus based decision support systems
Pattern recognition in images- recognition of pedestrians, signals- identification and classification in time series- photo based recognition- signature, face, handwriting identification- impression identification of imagesVoice, tone and feeling identification in audio
6
Israeli Hungarian Workshop on Future Internet Research
Context recognition
7
Partneroldali vékonykliens
Szavak hálójában
Magas szintű modell
Nyelvi
elemzőKontextus-
felismerő
Mély háló
motor
Mediátor
rétegMély háló
adatbázis
Válasz-
elemzés
Kérdés
Formális mondat
CL kifejezés
Általános SQL
Eredmények
Rangsor
SQL
Abstract model
Context
recognizer
Morpho
analyzer
Deep web
engine
Answer
analyzer
Mediator
layerDeep web
DB
Israeli Hungarian Workshop on Future Internet Research
Context recognizer
• Determines the context(s) of queries
• Identifies the focus and the schema of the query
• Determines the role of each informationgranule of the query and assigns them toappropriate attributes of the identifiedschema
• Creates so-called CL (Context Language) expressions that is the basis of SQL queries
8
Israeli Hungarian Workshop on Future Internet Research
Multi Criteria Decision Analysis in engineering areas
• Info-communication networks
• Fault spreading problem
• Routing tasks
Mathematical modelling toolsAnalytic Hierarchy Process (AHP)
Analytic Network Process (ANP)
Bayesian Network
Method combining expert knowledge
and probabilistic theory to analyse fault
spreading problem
9
Israeli Hungarian Workshop on Future Internet Research
• Task: extraction of feature vectors
characteristic to objects
• Referential comparison of different
categorization methods
Israeli Hungarian Workshop on Future Internet Research
Method
• Originally used for text documents
• Hierarchical taxonomy tree
• Each node has a processing unit
• Each processing unit outputs a sigmoid
function of the linear combination of its
inputs
Israeli Hungarian Workshop on Future Internet Research
Israeli Hungarian Workshop on Future Internet Research
Data Mining Competitions
• ACM KDD Cup (classification of internet search query; automatic identification of pulmonary embolisms (PE) from 3D computed tomography data)
– First place in patient classification
– Runner-up in query categorization precision
– Runner-up in PE identification
– Runner-up in creativity
• ACM SIGKDD Challenge on Time Series Classification (general purpose time series classification task)
– 3rd place
• IEEE Conference on Data Mining – 5th in precision on high precision in-door localization
• Netflix Prize Competition– 2nd place
13
KDD Cup is the annual Data Mining and Knowledge Discovery competition organized by ACM, USA
Israeli Hungarian Workshop on Future Internet Research
integrated 3D mouse
Integrated touch-based actuator
haptic vibration feedback
Audio (speach recognition and synthesis 3D input devices
Cognitive interfaces
14
Israeli Hungarian Workshop on Future Internet Research
- RT middleware
- ICE based TRT
middle ware
-Platform independent
- 3D Internet
-Collaboration Arena
VirCA operation system - network
15
Israeli Hungarian Workshop on Future Internet Research
Co-operation networks
• Emerging Information Technology in
Manufacturing (ITM) www.itm-lab.eu– Norwegian University of Science and Technology,
Narvik University College, BME, MTA-SZTAKI,
Budapest Tech (BMF), Széchenyi István University
• Intelligent Integrated Systems
Japanese-Hungarian Joint Laboratory
www.iisl-lab.org– University of Tokyo, BME
16
Israeli Hungarian Workshop on Future Internet Research
7% - verbal communication; 38% - paraverbal communication; 55% - non-verbal signals, such
as gesticulation, posture
3D Motion capture
systemsSpeech recognition
17
Israeli Hungarian Workshop on Future Internet Research
CAD
matching
Norway Hungary
18
Israeli Hungarian Workshop on Future Internet Research
Two approaches:
Cell phone actuator in CAVE
CAVE-enhanced cell phone
Cell phone equipped with:
Motion sensors (integrated 3D mouse)
Touch sensors (integrated touch-based actuator)
Vibrating components (haptic feedback)
<===>
Data glove for position
capture and haptic
feedback
Wireless input device
Mobile 3D Internet
19
Israeli Hungarian Workshop on Future Internet Research
Foundation of
Intelligent Space Network
• Dept. of Electrical and Computer Engineering North Carolina State
University
• Computer and Automation Research Institute of Hungarian Academy of
Sciences
• Eötvös University
• Budapest University of Technology and Economics
• National Institute of Advanced Industrial Science and Technology
• Hashimoto Laboratory, Institute of Industrial Science University of Tokyo
• Department of Electrical and Electronic Engineering Chuo University
• College of Information Science and Engineering Ritsumeikan University
• Department of Electrical Engineering Meiji University
• Department of Industrial Engineering, Narvik University College
• Department of Production and Quality Engineering Norwegian University
of Science and Technology
20
Israeli Hungarian Workshop on Future Internet Research
Speech activities
• Multi-lingual artificial speech generation
• Speech recognition
• Speech databases
21
Israeli Hungarian Workshop on Future Internet Research
• Multi-lingual artificial speech generation– limited vocabulary (e.g., numbers, date, address)
– multi-lingual TTS (Hungarian, German, Polish, Spanish)
– speech profiles (prosody, variability, individual features)
– expression/emotion presentation (user’s manual <-> news)
• Speech recognition– noise handling (telephone, in-car, …)
– dictation (good quality medical transcription, continuous)
– audio indexing (e.g. radio archives, broadcast news)
– speech segmentation
– emotion detection
– keyword spotting
• Speech databases
• Health, rehabilitation, e.g. Audio-visual speech-pronunciation teaching system
• Acoustic company image, eto-informatics
22
Israeli Hungarian Workshop on Future Internet Research
ETO-comm
• Adaptive artificial character („personality”)
in man-machine communications
– adaptive to the mood, style, activity of the
human being
– Personal gadgets personal partners
• Analogy: the complex communication of
a dog and a human being
23
Israeli Hungarian Workshop on Future Internet Research 24
Israeli Hungarian Workshop on Future Internet Research
ETO-comm
• Adaptive artificial character („personality”)
in man-machine communications
– adaptive to the mood, style, activity of the
human being
– Personal gadgets personal partners
• Analogy: the complex communication of
a dog and a human being
• E.g. smartphone, robot (sound, vibration,
light, style of messages)
25
Israeli Hungarian Workshop on Future Internet Research
Networked Media in TMIT
Media/Content Delivery PlatformsIPTV, mobile internet
User Centric Mediainformation-retrieval, community content,personalisation, integration of UGC and professional content,
Multimedia Searchtagged media, multimedia content analysis,automated annotation
3D Media3D internet, feeling of physical presence
26