Introduction to Motion Tracking to Dance
-
Upload
marlon-solano -
Category
Design
-
view
115 -
download
1
description
Transcript of Introduction to Motion Tracking to Dance
![Page 1: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/1.jpg)
Motion Tracking
Robert Wechsler
Palindrome Inter.media Performance GroupStuttgart, Germany
Lausanne, June 2009
www.palindrome.de
![Page 2: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/2.jpg)
workshop concept
"dance first, think afterwards."
- Estragon (S.B.)
"to work with motion tracking as an artist, start and finish with the body, not with the technology."
- Robert (Palindrome)
![Page 3: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/3.jpg)
workshop plan
1 Training. non-dancers: 11:30-11:55 ok?2 Talk / Discussion / video clips 40 min3 Improv. Chor. programming (patch-writing). 4 Tech. Sessions5 Project work6 Stretching, etc.
![Page 4: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/4.jpg)
Talks from me (12-12:40)
Talk 1: "What is Palindrome?
A very brief history of interactive art. Motion tracking and motion capture."
Talk 2: "Sensors: body (physiological) vs. environment-based systems. mapping."
Talk 3: "Practical issues: how to get the equipment, etc. A critical discussion of video projections."
![Page 5: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/5.jpg)
Palindrome Inter.media Performance Group
Palindrome is a kind of dance company....
• new york 82-90, germany 91-09, france?? 09-...
• performances, some installations, touring, workshops
• works with interactive music, video and lights
• develops software hardware applications*
• specialized in input and mapping
* - I'm not an engineer, but work _with_ engineers, by itself a special skill...
![Page 6: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/6.jpg)
what is motion tracking?
• technologies that collect data on human movement (input)
• used to control sounds, music, recorded or projected text, video art, stage lighting (output)
• via performer actions / gestures / movements / bio-data
• used in interactive performances and installations
• other uses....
![Page 7: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/7.jpg)
what is motion tracking?
• other uses:
• animation modeling (motion capture)
• scientific research (musicology, semantics, ergonomics, medicine, sports medicine, architecture)
• therapy for physically and mentally handicapped
•
![Page 8: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/8.jpg)
motion tracking vs. motion capture
motion capture
– tracks location of fixed positions on body
– highly accurate, expensive ($200k-2m)
– generally not realtime
– used for data collection (research) and for making human or animal motion in animations (films, games, etc.) more realistic
motion tracking
– less equipment, less data, less cost (ca. $1k-2k)
– concerned with motion qualities like dynamic, direction of motion
– real time
– used for live applications: installation art, dance, theater and music
![Page 9: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/9.jpg)
motion capture
– tracks location of fixed positions on body with reflective markers– 12-24 cameras, each lens is ringed with infrared lights
![Page 10: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/10.jpg)
• Vicon is a leading company in motion capture
motion capture
![Page 11: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/11.jpg)
motion capture
![Page 12: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/12.jpg)
motion capture
![Page 13: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/13.jpg)
• typical uses
– human movement research (sports, musicology, ergonomics, medicine,...)
– Film and Animation -- often used with 3-D animation (modeling) programs to make animations
• maya (http://www.animationarena.com)
• houdini (http://www.sidefx.com)
motion capture
![Page 14: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/14.jpg)
motion capture
![Page 15: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/15.jpg)
– performances of interactive animated characters: "No DNA" (http://www.nodna.com)
– experiments with dance performances
– Motione (2003-5) Arizona State U. (http://ame.asu.edu/motione)
"how long does an object linger on the end of the volume..." (Trisha Brown, Kaiser, et.al.)
"22" (Bill T. Jones, Kaiser , et.al.)– Luc Vanier et al. at the interactive performance facility U of Wisconsin at
Milwaukee (http://www.isl.uiuc.edu/Publications/final%20dance1.pdf)
experiments in real-time mo-capture
![Page 16: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/16.jpg)
input
physical
human action
motion tracking and interactivity
media output
sounds,
music, text,
projections,
lighting
![Page 17: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/17.jpg)
input
physical
human action
motion tracking
computersensor
(e.g. video camera)
output device (e.g. loud speakers)
media output
sounds,
music, text,
projections,
lighting
![Page 18: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/18.jpg)
input
physical
human action
motion tracking
computersensor
(e.g. video camera)
output device (e.g. loud speakers)
media output
sounds,
music, text,
projections,
lighting
analogue to digital
conversion
digital to analogue
conversion
![Page 19: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/19.jpg)
digital vs. analogue analogue data
• hard to reproduce• "rich data" (infinite values)• very high resolution• more details• contaminated data (becomes noisy, but rarely fails completely)
digital data
• easy to reproduce• lower resolution, less „human-feel“• easy to store• easy to process• contaminated data remains clean (errors can be filtered) or signal fails altogether
![Page 20: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/20.jpg)
input
physical
human action
motion tracking
sensor (e.g. video camera)
output device (e.g. loud speakers)
media output
sounds,
music, text,
projections,
lighting
computer
![Page 21: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/21.jpg)
What does motion tracking have to do with interaction or art?
![Page 22: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/22.jpg)
interaction means
human being 1 human being 2
• a back and forth of energy, impulse and ideas
![Page 23: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/23.jpg)
Drawings. France and Spain ca. 3000 BC
interactive performance is nothing new
-- you don't need computers to do it!
![Page 24: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/24.jpg)
interaction
who with whom? what with what?
• between performers ?
• between performer and audience ?
• between audience members ?
• between humans and machines ?
![Page 25: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/25.jpg)
interaction vs. automation
processing
interaction
• is an integral part of human life
• is instinctive, deeply felt, highly
communicative
• can and usually does occur on
many sensory levels
• is the heart and soul of live
performance
automation
• if new, excites curiosity
• feels like "future vision"
• hugely popular in film, tv... processingaction re-action
human action human re-action
![Page 26: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/26.jpg)
a very brief history of technology-supported interaction in performance
Dionysis Thr. Athens - 3c BC
![Page 27: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/27.jpg)
Ancient greek theater employed a number of mechanical devices to extend the
power of the actor to allow them to play gods and goddesses.
![Page 28: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/28.jpg)
Louis XIV, the Sun King, was fascinated by remote-acting mechanical devices
![Page 29: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/29.jpg)
on the battle field. Cannons allowed you to take deadly action from a distance
(500 meters). "gesture amplification". (not an interactive device...)
![Page 30: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/30.jpg)
Electronics, interactive sound and lighting devices were used by New York performance artists in the 60's and 70's
Robert Rauschenberg - "Pelican" 1963
Laurie Anderson - "Walking Solo for Tape Bow Violin" 1979
![Page 31: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/31.jpg)
reasons to use motion tracking
• amplification of gesture
• makes possible new ways of associating and connecting media
• _can_ be interactive
• sense of spontaneity and engagement
• environment becomes lively and responsive
• text and subtext (body can speak, as well as mouth)
• synesthesia
• the technology itself is catchy. marketable.
![Page 32: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/32.jpg)
end of part 1.
www.palindrome.de
![Page 33: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/33.jpg)
Motion Tracking - part 2
Talk 2: sensors mapping eyecon instruction Talk 3: what you need to track motion different software and hardware available where to buy it cheaply video projection techniques artistic issues
robert wechsler www.palindrome.de
![Page 34: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/34.jpg)
motion tracking sensors
• body-oriented
• environment-oriented
![Page 35: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/35.jpg)
motion tracking sensors
• body-oriented
– body-worn electrodes
• EEG
• ECG
• EMG
• Touching
– joint bend-sensors
– accelerometers, other body-worn devices
• environment-oriented
– video camera-based (eg. eyecon)
– infrared camera-based
– floor sensors, wall sensors, etc
![Page 36: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/36.jpg)
mapping
cause and effect. what does what.
mapping has 3 parts:
• input
• output
• logical relationship
![Page 37: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/37.jpg)
input parameter relationship/processing output parameter
components of mapping
![Page 38: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/38.jpg)
relationship/processing
• logical direction• compliance• transparency (clearness)• diffusion (subtlety)
![Page 39: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/39.jpg)
mapping
mapping choices multiply:
input parameters x output parameters x logical relationships
= number of mappings
there are important human perception issues in these choices!
some can be perceived, many can not!
![Page 40: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/40.jpg)
video
1. body-part locations (presence /absence), e.g. body parts
2. movement dynamic (in defined zones)
3. movement direction (left-right, up-down)
4. size factors (height, width)
5. position in room (or stage)
bio-sensors• physiological properties (muscle contractions, joint bendign, etc.• contact with objects or other people
mapping
"useful" inputs
![Page 41: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/41.jpg)
internal • audio synthesizer (on/off, volume control, pitch bend, panning)• audio samples (on/off, volume control, pitch bend, panning)• images, videos (on/off, scrubbing)
external (secondary machines)• sound programs (soundforge, MAX/msp, ...), algorithmic composition• controlling stage lighting (DMX) • realtime graphics (Director, Flash, Isadora, Kalypso, Nato*, Jitter*, SoftVNS*,
SquishedEyeball*, ...) • mechanical devices
* objects that interact with cycling74‘s „Jitter“, the graphics processing environment for MAX/msp.
mapping
"useful" outputs
![Page 42: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/42.jpg)
Boolean (logic in on/off relationships)
• positive: movt.=media, no movt.=no media
• negative: movt.=no media, no movt.=media
• toggle: movt.=media, 2nd. movt.=no media
• iteration: movt.=media, 2nd movt.=2nd media, etc.
• random, sequential or simultaneous effects
continuous controllers
• higher/lower – as in a slider (loudness, pitch, filters, etc.)
• thresholds – beyond which a certain change occurs
•
mapping
"useful" relationships
![Page 43: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/43.jpg)
motion tracking systems
• EyeCon (http://www.eyecon.palindrome.de)
• Isadora (http://www.troika.org)
• Eyes Web (http://www.eyesweb.org)
• Very Nervous System (http://homepage.mac.com/davidrokeby/vns.html)
• Cyclops (http://www.ericsinger.com/cyclopsmax.html) *
• CV.jit (http://www.iamas.ac.jp/~jovan02/cv/) *
• MIDI Dancer (http://www.troika.org)
• Big Eye (http://www.steim.org)
* - programmed in Jitter the graphics processing environment for MAX/msp.
![Page 44: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/44.jpg)
end of part 2.
robert wechsler www.palindrome.de
![Page 45: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/45.jpg)
Motion Tracking - part 3
Talk 3: what you need to track motion different software and hardware available where to buy it cheaply video projection techniques artistic issues next steps...
Eyecon Training how to control images and video how to connection eyecon to MAX/mp
Project Work
robert wechsler www.palindrome.de
![Page 46: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/46.jpg)
what you need to track motion Of course there are different ways to do it. Different artists have different styles
of working, different needs...
Advantages of EyeCon
flexibility, sensitivity -- you can use it in many different ways
ease of use, quick start-up time, DIY (do it yourself)
relatively low cost
To use EyeCon in the "best way", requires 4 things:
computer running windows
software
camera
industrial framegrabber*
![Page 47: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/47.jpg)
what you need to track motion Computer needs video input
Digital video (Firewire, USB2)
+ digital cameras (camcorder, webcams)
+ low noise
+ works with laptops
- latency issues
- image resolution issues (smaller chip sizes)
- limited cable length
Analog video
+ "unlimited" cable length
+ lower latency
+ even digital cams usually have analog output
- cost more (although many older cameras work quite well)
- works less well with laptops... (?) i.e. need an external or internal framegrabber
![Page 48: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/48.jpg)
what you need to track motion Framegrabber
captures, or digitizes video (turns analog video into digital video)
desktop allows for industrial framegrabber:
must be: Falcon (LeoLite, et al.) www.ids-imaging.com
![Page 49: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/49.jpg)
what you need to track motion Camera
almost any camera will work, but...
chip size 1/2" allows WIDE angles and high resolution
does not need to be color
![Page 50: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/50.jpg)
what you need to track motion Lowest Costs
desktop computer
get a used one? min. 1 GHz 300 e
new, shuttle, barebones, 4 GHz 600 e
framegrabber 250 e
camera 1/2" chip 350 e
zoom lens (4-12 mm) 75 e
software licenses 100 – 500 e
Total Equipment Cost: 1000 – 2000 e
Training (w/ me :) ) includes building equipment
1 week 1000, 3 day intensive 600
![Page 51: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/51.jpg)
motion tracking systems
• EyeCon (http://www.eyecon.palindrome.de)
• Isadora (http://www.troika.org)
• Eyes Web (http://www.eyesweb.org)
• Very Nervous System (http://homepage.mac.com/davidrokeby/vns.html)
• Cyclops (http://www.ericsinger.com/cyclopsmax.html) *
• CV.jit (http://www.iamas.ac.jp/~jovan02/cv/) *
• MIDI Dancer (http://www.troika.org)
• Big Eye (http://www.steim.org)
* - programmed in Jitter the graphics processing environment for MAX/msp.
![Page 52: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/52.jpg)
motion tracking - issues for artists• Artists use technology differently than scientists and engineers! They have different needs, different criteria.
1 working with engineers
- higher tech, flashier effects
- better control, more reliable
- less problems? probably not. different problems...
- you will find yourself making work _about_ technology – whether you intended to or not!
2 working alone
- easier to focus on artistic issues
- simpler systems
- less flashy (= less attention, less money)
- requires tons of patience
![Page 53: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/53.jpg)
movies, sounds and lights
Media can compliment and support the live performer
or it can distract from her-him.
This means it depends on how you do it.
And there is a fundamental issue to think about...
![Page 54: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/54.jpg)
1. make the image smaller
2. make the image less bright, or black & white
3. lower the resolution of the film, "pixilate" it or use various image abstraction methods
4. use non-human material -- human images and faces distract most of all
5. use semi-transparent screens
6. put the screen up high, down low, in the back, or use thin materials
7. project on the side walls, on the floor or ceiling
8. project on the bodies of the performers
9. have the projection incorporate the performer's live image
10. put the image under the control of the of the performer's movement (ie. make it interactive)
movies (moving images) are extremely distracting to the live performer.
![Page 55: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/55.jpg)
APPROACH 2:
Use Stills. Low Motion Slow Motion.
![Page 56: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/56.jpg)
• Palindrome Intermedia Performance Group• Krisztina de Chatel• Igloo• Ventura Dance (Pablo Ventura)• Robert Lepage • André Werner • Marlon Barrios Solano• La la la Human Steps• Georg Hobmeier• Leine Roebana Dans Kompanie• Troika Ranch• Blue Man Group• you
who is using motion tracking?
![Page 57: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/57.jpg)
motion tracking - review
Positive
amplification of gesture
sense of spontaneity and engagement
text and subtext (words say one thing, body says another)
can make art interactive in new ways
generates genuinely new ways of associating and connecting media
helps to get grants and invitations
Negative
substitute art with special effects
complex processes -- compromise quality and subtlety of expressive media.
extremely time consuming, distracting
life-less materials and tools de-humanize art and artists
![Page 58: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/58.jpg)
Motion Tracking
End of part 3.
[email protected] www.palindrome.de
"dance first, think afterwards."
- Estragon (S.B.)
![Page 59: Introduction to Motion Tracking to Dance](https://reader035.fdocuments.in/reader035/viewer/2022081414/54c7726d4a7959b6248b4624/html5/thumbnails/59.jpg)
eyecon www.eyecon.palindrome.de