Annual Project Review 12 October 2000 IST-1999-10500.
-
Upload
landon-blackburn -
Category
Documents
-
view
216 -
download
1
Transcript of Annual Project Review 12 October 2000 IST-1999-10500.
Annual Project Review
12 October 2000IST-1999-10500
Introduction
Nick LodgeITC
Presentation Structure• Introduction• Project Management Overview• Linguistic Advances for Deaf People• Progress on Internet Services• Progress on Broadcast Services• Commercial Strategies• Video of Counter Services
Project Management Overview
John GlauertUEA, Norwich, UK
Project Structure
• User Focus• Application Focus• Technology Focus
WWW High Street Broadcast
Language
Animation
Evaluation Exploitation
Application Focus
• Internet– Information and Education for Deaf People (IvD, IDGS)
• High Street– Counter Services – Post Office (UKPO)
• Television– Regulation and Standards (ITC, INT, IRT)
WWW High Street Broadcast
User Focus
• Evaluation– Close Involvement of Deaf People (RNID)
• Exploitation– Broadcasters and Service Providers (ITC, UKPO, Televirtual)
Evaluation Exploitation
Technology Focus
• Sign Language Linguistics– Use of natural sign language (IDGS, UEA, IvD, RNID)
• Animation– Increased realism in sign generation (Televirtual, INT, UEA)
Language
Animation
Workplan Structure
Project Phases
• Technology Transfer• Prototype Applications• Advanced Applications
Technology Transfer
• First 12 Months• Exchange Existing Prototypes• Establish Collaborative Working• Largely Complete
Prototype Applications
• Months 9 to 24• Enhancement of Prototypes• Extended Flexibility• Focus on Well Defined Domains• Excellent Progress
Advanced Applications
• Months 18 to 36• Increased Animation Realism• Flexible Sign Generation• Advanced Linguistic Processing• Address Wide Ranging Domains
Workpackage Progress
• Achievements in Review Period– All Deliverables achieved– Milestones achieved or rescheduled with no impact
• Progress on Deliverables– Reported here for Prototype Applications
• Planned Changes– Response to Exploitation Opportunities will be presented
Management System
• Workpackage Leadership• Consortium Meetings• Workpackage Meetings
Workpackage Leadership
• 9 Partners• 8 Workpackages• Workpackage Leadership
– Coordination of Partners– Workplan in Single Partner Tasks– Lightweight use of Microsoft Project
Consortium Meetings
• Team of Partner Leaders• Quarterly Meetings
– First day for Management and Planning
– Second day allows for Technical Discussions
Workpackage Meetings
• Meetings as required by Workpackage Leader
• Extensive use of Email• Website for Document Archive and
Exchange
Addressing the needs of Deaf people using natural sign language - Overview of linguistic processing in
ViSiCAST
Thomas Hanke, IDGS, U Hamburg
Why Sign Language?
• Natural language of most Deaf, i.e. not an artificial language
• 1‰ in Europe pre-lingually deaf, only 10% of them Deaf of Deaf
• Most of them leave school functionally illiterate
What is Sign Language?
• A language with its own syntax & lexicon
• Not a 1:1 translation from spoken language as is SSE
• Not universal, but much closer to each other than spoken languages
What is so special about Sign Language?
• Parallel use of articulators• Use of space to encode
grammatical function• Iconicity and productivity
Sign Language and Text
• No established written form for any sign language
• “phonetic” transcription systems such as HamNoSys
• Glossing: GIVE-1-3a
HamNoSys
• Flexible enough to handle at least the European sign languages, will be verified for DGS, NGT, BSL
• Further developed in the project to include nonmanual activities
• Embedded into XML: SiGML
SiGML• Three Levels:
– Glosses: mapping more or less 1:1 to the animation machines’ database of signs
– HamNoSys: phonetic description– Motion capture files: for uninterpreted
direct transmission thru SiGML
SiGML (2)• <signed_utterance>
– <sign>• <manual_activity gloss=“GIVE-BOOK-1-3a”>• <face attitude=IRONIC>
– </sign>– <sign>…
• </signed_utterance>
How does it fit in?
Web pages
EnglishSpoken lang to text
English text toSign Notation
Transport &Animation
Standard AI Machine Translation Approach
Semantic Structure
English text Sign Notation (XML)
LexiconGrammar
LexiconGrammar
To reduce complexity
• Restricted Domains– Handling objects in a kitchen– Appartment descriptions
• Semi-automatic process– User can help both with signs and
word order
Progress on Internet Services
Margriet VerlindenIvD, The Netherlands
Workpackage 2• objective
– produce tools that will allow a deaf citizen to access multimedia resources through sign language
• deliverables– Browser plug-in (month 12)– Web-page-sign (month 30)– Signing tutor (month 30)
Signing on the WWW
• What will it look like?• How does it work?• How is it created?• How far are we now?
Het weer in gebaren
Weerbericht van donderdag 17 augustus2000, 17:34 uur
Vanavond en vannacht opklaringen, in hetbinnenland mogelijk een mistbank en aan denoordwestkust een kleine kans op onweer.Minimumtemperatuur rond 13 graden. Morgengeleidelijk meer bewolking en in de avond vanhet zuiden uit enkele regen- en onweersbuien.Middagtemperatuur morgen ongeveer 22graden.
De wind: zuidwest, matig, kracht 4, aan de kusten op het IJsselmeer vrij krachtig, 5, dekomende uren af en toe krachtig, 6. Vanavondafnemend en morgen draaiend naar zuid.
daily updated
link to project info.
signing avatar
controls for signing
What will it look like?
PLAY PAUSE STOP
MOCK-UP
How does it work?text from meteo-rological institute
standard sentences
playlist of motion data
SER
VER C
LIEN
T
Internet-page with SiGML
Internet-browser +
plugin for motion player (=avatar)
How is it created?text from meteorological institute
standard sentences
playlist of motion data
Internet-page with SiGML
• model for weather forecasts• mapping from stand. senten-
ces to sign sequences (3x)• recording + post-editing of
signs (3x)• XML for signs: SiGML• user-interface (webpage+plug-
in)
• update-facility
How far are we?• basic functionality implemented• motion data is being post-edited• additional functionality in progress
DEMO
Progress on Broadcast Services
Mark WellsTelevirtual, Norwich, UK
The Problem:– Legislation (UK, soon European)
requires on-screen signing– “Open Signing” (ever present) is
obtrusive– Even “Closed Signing” uses bandwidth:
Broadcasters run a tight bandwidth economy
The Solution: Virtual Human Signing
– Can be Open or Closed– Picture Quality not compromised by
compression– Potential for User Interaction– Very Cheap in terms of bandwidth
Virtual Humans / Avatars
• Entertainment• Communications• Transactions• Guides
Virtual Humans / Avatars
• Entertainment
Virtual Humans / Avatars
• Communications
Signing Virtual Humans
Recording Signing
Recording SigningReal timeFace,Hand,Body, Tracking
Recording SigningReal timeFace,Hand,Body, Tracking - & Playback
On Television:
Transmit theMovement -not the Video
Motion Data + Calibration
Motion Data + Calibration
IHoIstCOM
Mesh
Mesh Attachment Description
Bone Set
Renderer
Avatar Codec
DSP
Compressed Motion Stream
DSP
Avatar Codec
Compressed Motion Stream
Bone Set
MPEG 2 Broadcast Stream
Scene
Transmission System
- The Demo...
• Integrated TX system for broadcast to STBs– demonstrator complete end of 2000
• Implementing virtual human s/w in STB• Optimising compression algorithm• Building multiplexer for carriage in MPEG-2 • SiGML streaming
Broadcast VH Signing:
• Low transmission rate < 30 kbit/s• Compatibility with signing on other media and
foreign deaf languages• Precise, sharp representation of signer• Many display options•Can conform to MPEG-4 standards• Future-proof:
–cost saving–allows vast no. of signed programmes–no transition from video-based to VH signing
VH on TV: The Advantages
Commercial Strategies
Nick LodgeITC
What is the Product or Service?
• Diversity of applications– Broadcast - information displays, plays, movies
– Multimedia - dictionaries, EPGs, learning tools
– Face-to-face - videoconferencing, UMTS phones
• Diversity of languages– German, Dutch, English signs– Future: ASL, Makaton, Paget Gorman
Theatre / Cinema Application
• Display allows visibility of external world and signer
What is the Product or Service?
• Diversity of applications– Broadcast - information displays, plays, movies
– Multimedia - dictionaries, EPGs, learning tools
– Face-to-face - videoconferencing, UMTS phones
• Diversity of languages– German, Dutch, English signs– Future: ASL, Makaton, Paget Gorman
What is the Product or Service?
• Increase awareness of sign languages• Promote standards for signing avatars
• Advance linguistic representation & processing of
deaf languages• Provide consultancy support
Building Brand Awareness
• Establish acceptability of virtual humans• Establish presence of ViSiCAST across
different media• Establish Visia as a character with
personality and encourage familiarity
The Asymmetric Market• Only 0.05% - 0.1% of population sign
• Millions of companies wish to sell to deaf people or
• Wish to be seen to be aware of deaf customers
Legislative Influences• Disability Discrimination Directive• UK Broadcasting Act 1996• UK Communications Act 2001
1999 2009
5
2
3
4
1
%Content ofDTT channel
WWW strategy• Give away basic web browser
• Sell SiGML authoring tool
• De facto standard
Face-to-face Product
• Stand-alone technology• Licensed speech recognition• Sell ViSiCAST signing software• Systems require s/w & h/w support
Broadcasting Production & Transmission Technology
• Body suits combining capture technologies• Digital compression systems• Multiplexers for combination with MPEG-2
services• Studio practices for handling signing data• Encouraging use of the system within DVB
The Set-top Box Strategy
• Develop special set-top box for deaf users by November 2001
• Work with STB manufacturer to develop ‘concept box’ with VH interface– speech synthesis, speech recognition
• Explore human factors with elderly users• Develop speech for blind users
Progress on Counter Services
TESSAVideo of Deliverable
ExampleThe weather forecast of the KNMI, drawn up on Monday Oct. 9, and valid till midnight.First a shower, in the west sometimes with thunder. Furthermore now and then sunny. In the afternoon more cloudy and in the evening rain from the west.Temperature in the afternoon around 14 degrees.
The weather forecast of the KNMI, drawn up on Monday Oct. 9, and valid till midnight. (p.1)In the morning a shower. (p.7) Sometimes in the west thunder. (p.8)Sometimes sunny. (p.7)In the afternoon increasingly cloudy.(p.7)Tonight from the west rain. (p.8)In the afternoon 14 degrees. (p.11)