MELJUN CORTES Research Seminar 1 Introductory Lectures Research Seminar 1
2015-02-12 research seminar
-
Upload
ifi8106tlu -
Category
Education
-
view
59 -
download
3
Transcript of 2015-02-12 research seminar
!"#$%&'()*+',-.*-/0-1'(23)' 3'
New applications based on implicit, physiology based interaction!!
Dr. Aleksander Väljamäe Marie Sk!odowska-Curie Fellow
Decision, Emotion and Perception Lab Department of Behavioural Sciences and Learning
Linköping University, Linköping, Sweden &
Laboratory of psychophysiology Department of Higher Nervous Activity and Psychophysiology
SPbSU, Saint Petersburg, Russia &
NeuroGenetics Media Lab, CIANT, Prague, Czech Republic
!"#$%&'()*+',-.*-/0-1'(23)' ('
!"#$%&'()*+',-.*-/0-1'(23)' 4'
Born in Tallinn, USSR, in 1978 1996-2000 First degree (B.Sc.) – Telecommunication, Tallinn TU, Tallinn, Estonia 2001-2003 Second degree (M.Sc.) – Telecommunication, Chalmers University of Technology, Gothenburg, Sweden 2003-2007 Ph.D. on multisensory virtual reality – Chalmers University of Technology, Gothenburg, Sweden 2007-2010 First postdoc – neuroscience lab SPECS, University of Pompeu Fabra, Barcelona, Spain 2011-2012 Second postdoc – Graz BCI lab, TU Graz & Neuropsychology Laboratory, University of Graz, Graz, Austria
Short CV
!"#$%&'()*+',-.*-/0-1'(23)' )'
!"#$%&'()*+',-.*-/0-1'(23)' 5'
from Wolpaw et al. 2002
!"#$%&'()*+',-.*-/0-1'(23)' 6'
from Wolpaw et al. 2002
!"#$%&'()*+',-.*-/0-1'(23)' 7'
'Implicit, physiology based interaction Physiological computing provides new channel to the conventional Human Computer Interaction paradigm. My interest is implicit interaction with multisensory media and using it for health and art applications.
1. Subtopic one: Cinema and neuroscience: recent research developments and future applications
2. Subtopic two: B-Reactable: multimodal tabletop system for collaborative physiology monitoring and training
!"#$%&'()*+',-.*-/0-1'(23)' 8'
Media as a constructed... but "real" experience
!"#$%&'()*+',-.*-/0-1'(23)' 9'
MEDIA AS CONSTRUCTED EXPERIENCES
Q1: What is neurocinema and enactive cinema?
Q2: How in can be used in art and research?
!"#$%&'()*+',-.*-/0-1'(23)' 32'
neuro cinematics
!"#$%&'()*+',-.*-/0-1'(23)' 33'
Nishimoto et al., 2011, Current Biology
Work by Gallant Lab at UC Berkeley
!"#$%&'()*+',-.*-/0-1'(23)' 3('
Measuring Inter-Subject Correlation - ISC Neurocinematics –
"The ability to measure the effect of films on viewers’ brains with high spatial and temporal precisions can provide a new analytical paradigm for assessing and analyzing different aspects of films, film genres, and cinematic styles."
Hasson et al., Projections 2008
!"#$%&'()*+',-.*-/0-1'(23)' 34'
Inter-Subject Correlations for different films (1)
Bang! You're Dead (Hitchcock, 1961)
Hasson et al., Projections, 2008
!"#$%&'()*+',-.*-/0-1'(23)' 3)'
Inter-Subject Correlations for different films (2)
Hasson et al., Projections, 2008
!"#$%&'()*+',-.*-/0-1'(23)' 35'
Interactive cinema
!"#$%&'()*+',-.*-/0-1'(23)' 36'
Creating interactive narrative
!! Script-Based Interactive Narrative: - all scenes are pre-scripted - linear or non-linear pre-dermined branchining structure
!! Emergence-Based Interactive Narrative: - agents autonomy (goal-directed and reactive actions) - story is created by agents and user interaction
!! Plan-Based Interactive Narrative: - agents autonomy - narrative manager dynamically evaluates the actions of agents and user and channels them towards a coherent story
!"#$%&'()*+',-.*-/0-1'(23)' 37'
'Affective chronometry and database cinema
!"#$%&'()*+',-.*-/0-1'(23)' 38'
Alexander Ponomarev, INFO-SHOWER, Russian Pavillon, Venice Bienalle 2007
Pulling the armatures one can regulate the speed of the pictureflood. The content itself is mixed with news, entertainment, porn, musicvideos etc.
Database media and cinema
http://tagr.tv http://www.youtube.com/watch?v=NEfVjAbMsZQ
!"#$%&'()*+',-.*-/0-1'(23)' 39'
Interactive cinema
neuro cinematics
enactive cinema
Interactive cinema based on physiological sensing
!"#$%&'()*+',-.*-/0-1'(23)' (2'
Using film for studying socio-emotional interaction
Tikka et al., 2012
!"#$%&'()*+',-.*-/0-1'(23)' (3'
Implicit interaction: wearable systems
www.neurosky.com
www.emotiv.com
www.smartex.it
www.starlab.es
!"#$%&'()*+',-.*-/0-1'(23)' (('
"Room of desires" installation (Smetana, 1996) http://www.smetana.cz
!"#$%&'()*+',-.*-/0-1'(23)' (4'
"Room of desires" installation – system architecture
!"#$%&'()*+',-.*-/0-1'(23)' ()'
Enactive cinema: "Obsession" installation, 2005
Tikka et al., 2006
!"#$%&'()*+',-.*-/0-1'(23)' (5'
"Obsession" installation (Tikka, 2005)
Tikka et al., 2006
!"#$%&'()*+',-.*-/0-1'(23)' (6'
Kaipainen et al., 2011
Enactive Cinema project: ontospace narrative engine
!"#$%&'()*+',-.*-/0-1'(23)' (7'
SPECS-UPF research lines
!"#$%&'()*+',-.*-/0-1'(23)' (8'
'Affective narratives as guided experience
!"#$%&'()*+',-.*-/0-1'(23)' (9'
EDA-based modulation of video effects
!"#$%&'()*+',-.*-/0-1'(23)' 42'
'Neurofeedback in XIM – self-motion paradigm
!"#$%&'()*+',-.*-/0-1'(23)' 43'
Controlled stimulus – affective databases
HAU
LAU
HAP
LAP
!"#$%&'()*+',-.*-/0-1'(23)' 4('
Studying transitions using subsequent picture-pairs
HAU
LAP
!"#$%&'()*+',-.*-/0-1'(23)' 44'
'(A) Low arousal pleasant -> High arousal unpleasant
3 s pause between pictures 6 s pause between pictures
Patients: 22 migraineurs; age and gender matched controls: 12
Controls show positive valence supression, but patients do not
!"#$%&'()*+',-.*-/0-1'(23)' 4)'
!"#$%&'()*+',-.*-/0-1'(23)' 45'
Can enactive cinema be used in science?
Topics that can be addressed by enactive cinema: - temporal dynamics of cognition, emotion and perception - unconscious processing - decision making - emotional synchronization, collective experience - imaging genetics
!"#$%&'()*+',-.*-/0-1'(23)' 46'
Conclusions / Topics for discussions
Enactive cinema might have a big potential as a scientific tool
!"#$%&'()*+',-.*-/0-1'(23)' 47'
Conclusions / Topics for discussions
Enactive cinema might have a big potential as a scientific tool
What are the areas of common interest: - research questions (creating a demand) - research methodologies (e.g. ERPs during film presentation) - common video databases and related infrastructure (e.g. "narratives" engine) - ...
!"#$%&'()*+',-.*-/0-1'(23)' 48'
Radically new neurocinematic experiences?
Gerson et al., IEEE Trans. Neural Syst. 2006
Rapid Serial Visual Presentation – RSVP- sequence for "human visual processors"
100 ms per image
!"#$%&'()*+',-.*-/0-1'(23)' 49'
:;<='<>'?@1A'0B'$11<C'D<11EF&'(224'
!"#$%&'()*+',-.*-/0-1'(23)' )2'
Brain to Brain communication
.-1F<G@C'@H-IJK-'LEIJ<G@1B'
.-1F<G@C'@H-IJK-'LEIJ<G@1B'
!"#$%&'()*+',-.*-/0-1'(23)' )3'
New installation in progress
!"#$%&'()*+',-.*-/0-1'(23)' )('
B-Reactable project: 2013-2015
!"#$%&'()*+',-.*-/0-1'(23)' )4'
Collaborations: - Aleksei Ossadchi, Saint Petersburg State University - Prof. Juri D. Kropotov, Natalia Shemyakina, Institute of Human Brain, Russian Academy of Sciences - Prof. Daniel Västfjäll, Linköping University, Linköping, Sweden - Prof. Sergi Jorda, Sebastian Mealla, University of Pompeu Fabra, Barcelona, Spain
“B-Reactable: multimodal tabletop system for collaborative physiology monitoring and training" https://vimeo.com/14675468
B-Reactable project: 2013-2015
!"#$%&'()*+',-.*-/0-1'(23)' ))'
'First B-Reactable experiments
!"#$%&'()*+',-.*-/0-1'(23)' )5'
'Collaborative tasks using EEG monitoring
!"#$%&'()*+',-.*-/0-1'(23)' )6'
'Testing new setup at Sankt-Petersburg
!"#$%&'()*+',-.*-/0-1'(23)' )7'
'New B-Reactable experiments: system architecture
!"#$%&'()*+',-.*-/0-1'(23)' )8'
'New B-Reactable experiments: test conditions
!"#$%&'()*+',-.*-/0-1'(23)' )9'
'B-Reactable work in progress
!"#$%&'()*+',-.*-/0-1'(23)' 52'
'New B-Reactable experiments: results
!"#$%&'()*+',-.*-/0-1'(23)' 53'
Thank you for your attention!
Web: http://www.ibl.liu.se/medarbetare/valjamae-aleksander?l=en