AVTOdyssey:VoyagetotheFutureAnnaMatamalaUniversitatAutònomadeBarcelonaTransMediaCataloniaresearchgroupanna.matamala@uab.catIntermedia,14-16April2016FFI2015-62522-ERC,2014SGR0027,FFI2015-64038-P

Myview

•  Audiovisualtransla2on(includingmediaaccessibility):accessthroughtransfer

•  Tothelanguageoftheoriginal,toaudioandvisualelements

•  Variouscontent,modali2es/services,plaAorms,agents,andusers

•  Mul2pleapproaches

SomefirstthoughtsonAVTresearch

•  AVTmodali2es:dubbing/sub2tlingversusmediaaccessibility

•  Methodologicalapproaches:descrip2ve,movingtowardsmoreempiricalandapplied.

•  Topics:moretechnologyandrecep2onresearch.

Isthisreallyso?

ResearchonAVT:data

Jumpback10yearsto…–  InSoManyWords.London,2004.– BetweenTextandImage.Forlì,2005.– MediaforAllI.Barcelona,2005.

ResearchonAVT:data

…andbacktothepresent–  Intermedia.Lodz,2016.– Linguis2candculturalrepresenta2oninaudiovisualtransla2on.Rome,2016.

– MediaforAll.Sydney,2015.

Audiovisualtransfermodes

BTS:AVTmodes

Approach

Topics•  Manydivergingtopics•  Similarinterest– Culturalapproaches– Training,didac2cs– Specificprac2ces

•  Slightdrop– Linguis2capproaches(s2llanimportanttopic)

Topics

•  Increasein:–  recep2onofcertainfeaturesbyend-users–  fantransla2on,crowdsourcing– mul2lingualism

FocusonInterMedia16

•  Technologies:MTandTM,cleanaudio,secondscreens,SR(liveinterlingualsub2tling),computa2onalstylis2cs,collabora2ontools

•  Recep2onofAD(divergingstrategies,amountofinforma2on)

•  Processresearch(respeakingcrisispoint)•  Differentgenres(emergencynews,TVads),paratexts(taglines)andstrategies

FocusonInterMedia16

•  AVTtrainingandAVTinteachingforeignlanguages

•  Humourandinterculturalelements•  Filmfeatures:mul2lingualismandpar2alsub2tling,textonscreen

•  Sub2tlingfeatures(segmenta2on)

FocusonInterMedia16

•  Filmadapta2on•  Touchtours•  Fansub2tling•  Legalaspects

•  LessWestern-European-centricapproach:Russian,Persian,Arabic,Indian,Belarus,etc.

AVTResearch&Technology

Theroleoftechnology

•  Intheprocessofcrea2ng•  Intheprocessofdelivering•  Intheprocessofreceiving

•  Intheprocessofresearching

Speechrecogni2on

•  Respeakinginsub2tling:fromintralingualtointerlingual

•  RespeakinginotherAVT-relatedtasks– Anexample:documentarytranscrip2on(ALST,withLukaszDanilukandPabloRomero-Fresco)

•  Automa2ctranscrip2on(ALST,withHéctorDelgadoandJavierSerrano)– Domain-specificissues,speakerdiariza2ontools,languagedifferences,meaningoferrorrate

Machinetransla2on

•  Extensiveresearchonsub2tling

•  Newresearchonothertransfermodes:ADandVOinALST

•  Focus:translator/describerandend-user

Machinetransla2on:example1

12AVTMAstudentsHumantransla2onvspost-edi2ngTwoshortwildlifedocumentaryexcerpts(EN>ES):voice-overandoff-screendubbingTemporal,technical,cogni2veeffort(Inputlog)

Or2z-Boix&Matamala(forth)

Machinetransla2on:example1

•  Post-edi2ng,faster;lesstechnicalandcogni2veeffortthantransla2on

•  Outputqualityanalysisbyexperts(differentevalua2onrounds),indubbingstudio,andbyend-users

•  Transla2ons,slightlybefer(trainedtranslators)

Machinetransla2on:example2

•  Butwhatabouttranslator’sfeelings?

•  MTinAD– 12AVTMAstudents,EnglishintoCatalan– crea2on,humantransla2on,post-edi2ng– Temporal/technical/cogni2veeffort– Subjec2veopinions

Machinetransla2on:example2

•  Temporal:nosta2s2caldifferences,butADcrea2on(55.95’),postedi2ng(44.44),transla2on(48.66)

•  Technical:post-edi2ngwassta2s2callytheleastkeyboardintensivetask.

•  CogniSve:post-edi2nglessthanADcrea2on(onlysta2s2caldifference,althoughvaluesalwayslowerinpost-edi2ngintermsofeffortinvolved)

Whataboutperceivedeffort?

Machinetransla2on:example3

•  WhydoweneedMTfor?Automa2c?Partofaprocess?

– Tes2ngonautoma2csub2tlingwithinHBB4ALLproject,provedusefulonlyforcertainusers(Matamala,Oliver,Álvarez,Azpei2a2015)

Text-to-speech

•  TTSADinPoland(Szarkowskaetal),spokensub2tleswithTTS

•  TTSADinCatalan(Fernández-Torné&Matamala)

•  TTSVOinwildlifedocumentariesinCatalan(Or2z-Boix&Matamala)

Othertechnologiesandtools

•  Textsimplifica2on– Simplextproject,Able-to-include

•  Textcompression– Aziz,deSousa&Specia(2012)

•  Textsegmenta2on– Scaianoetal(2010)andÁlvarezetal(2016)

Othertechnologiesandtools

•  Automa2cimagerecogni2on– SeeingFB

hfp://newsroom.n.com/news/2016/04/using-ar2ficial-intelligence-to-help-blind-people-see-facebook/

•  Automa2clipsynch/dubbing hfp://aescripts.com/auto-lip-sync/hfp://mrl.nyu.edu/~bregler/videorewrite/hfp://videodubber.com/

Othertechnologiesandtools

•  AVTasbigdata

– Textalignedwithaudio–  Imagesalignedwiththeirdescrip2on

•  Informa2onretrieval

Edi2ngtools&thecloud

•  Professionaltechnology:– Sub2tlingsopwarevsdubbingsopware

•  Collabora2vetoolsforprofessionalsandthecloud:cohesion

Collabora2veplaAorms

•  Crowdsourcing,fan(sub/dubb)ingoruser-generatedtransla2ons(O’Hagan,Dwyer)

•  Dubbing(Dubroo,Dubjoy)•  Sub2tles(Amara,Viki)

Collabora2veplaAorms

•  Currentframework:collabora2veconsump2on(airbnb)andcrea2on(wikipedia),immediacy,personalisa2on,consciouseconomy?

•  Legalissues•  Whatisquality?Userexpecta2ons?•  Endangered/minoritylanguages•  Professionalvsuser-generatedcontent

Toolsfortheend-user

•  Consump2onacrossdevices•  Apps:– MovieReading,Whatscine,ArtAccés,etc.– MyLingoApp.

Dotheyreallywork?(Walczak’sresearch)

Toolsforresearch

•  (Mul2media)corpusanalysistools– POStagging,seman2ctagging(inAD)

•  Keyloggingandscreenrecordingsopware•  Eye-trackingandelectrophysiologicalmeasures•  Sen2mentanalysisinsocialmedia– hfp://www.laurenceanthony.net/sopware/fireant/

•  Crowdsourcing(micro)tasks(AMT)– Example:Transcriu-me(BibliotecaCatalunya)

Technologyimpactsonresearch

•  Wes2llneed:– Concepts(theory)– Stateoftheart(descrip2on)

•  Butwecanalsofocusonpeople(translators,end-users)– Processresearch– Recep2onresearch

Butneedtogoonestepfurther

Reachingout

Researchreachingpeers

•  Anexample:theVIWproject

hfp://pagines.uab.cat/viw•  Openaccess

•  publica2onsandpresenta2ons•  audiovisualcontent(copyrightissues)•  datasetsavailable

Reachingtheend-user

•  Sciencedissemina2on

– Raisingawarenessaboutourresearch– Goingbeyondthescien2ficarena

•  Asaserviceincorporatedbytheindustry

Parallelworlds

•  Virtualvoice-overtransla2onatBBC

h"ps://www.journalism.co.uk/news/how-the-bbc-uses-virtual-voiceover-transla9on-to-reach-a-mul9lingual-audience/s2/a620320/

Reachingtheindustry

•  Standardiza2onwork

•  Anexample:ISOWork–  ISO/IECJTC1/SC35Userinterfaces(WG6:accessibility)

•  Anewstandardonaudiosub2tles– Differentbackgroundsandinterests– Reachingconsensus(content,terminology,etc.)

Thefuture?

•  Evolu2onfromdescrip2vetoothertypesofresearch(notforgetngconceptualframeworks)

•  Adap2ngtoneweconomiesandtechnologieswithoutforgetngthepast

•  AlignmentwithEUpoliciesversusindependentresearch?

•  Transferingknowledgetosociety•  Coopera2on:interna2onal,interdisciplinary

AVTOdyssey:VoyagetotheFutureAnnaMatamalaUniversitatAutònomadeBarcelonaTransMediaCataloniaresearchgroupanna.matamala@uab.catIntermedia,14-16April2016FFI2015-62522-ERC,2014SGR0027,FFI2015-64038-P