Processing, Authoring and Composing of Spatial Sounds...
Transcript of Processing, Authoring and Composing of Spatial Sounds...
![Page 1: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/1.jpg)
Thibaut Carpentier & Jérémie Garcia
UMR 9912 STMS IRCAM — CNRS — UPMC!
Processing, Authoring and Composing of Spatial Sounds (using Spat and OpenMusic)!
Workshop on Interactivity in Music Composition and Performance
May 5th 2015 CIRMMT — McGill University, Montreal
ANR-13-JS02-0004
![Page 2: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/2.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 2
Outline!
• Spat : state of the art & processing aspects
![Page 3: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/3.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 3
Outline!
• Spat : state of the art & processing aspects
• Goals & Motivations
![Page 4: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/4.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 4
Outline!
• Spat : state of the art & processing aspects
• Goals & Motivations
• Compositional tools (ongoing work)
![Page 5: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/5.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 5
Outline!
• Spat : state of the art & processing aspects
• Goals & Motivations
• Compositional tools (ongoing work)
• Mixing and post-production
![Page 6: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/6.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 6
Outline!
• Spat : state of the art & processing aspects
• Goals & Motivations
• Compositional tools (ongoing work)
• Mixing and post-production
• Limitations & Perspectives
![Page 7: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/7.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 7
Spat
![Page 8: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/8.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 8
• What? Digital processor for sound spatialization and artificial reverberation
Spat : Overview!
![Page 9: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/9.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 9
• What? Digital processor for sound spatialization and artificial reverberation
• Why? Allow composers, performers or sound engineers to control the diffusion of sounds in a real or virtual space
Spat : Overview!
![Page 10: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/10.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 10
• What? Digital processor for sound spatialization and artificial reverberation
• Why? Allow composers, performers or sound engineers to control the diffusion of sounds in a real or virtual space
• How? Room effect module (temporal aspects) ��� + Spatialization module (directional aspects) ��� + intuitive control interface��� = Spat ��� = Global description of the spatial effect, modular and scalable design
Spat : Overview!
![Page 11: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/11.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 11
Spat : Architecture!
![Page 12: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/12.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 12
DSP chain
input instrumental or synthetic sources, typically mono or stereo
Spat : Architecture!
![Page 13: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/13.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 13
pre-processing DSP chain
input
Doppler effect, Air absorption, pre-delay, EQ, etc.
Spat : Architecture!
![Page 14: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/14.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
14
room effect
pre-processing DSP chain
input
Doppler effect, Air absorption, pre-delay, EQ, etc.
Artificial reverberator
![Page 15: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/15.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
15
room effect
pre-processing DSP chain
D ER Rev
input
![Page 16: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/16.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
16
room effect
pre-processing DSP chain
D ER Rev
input
![Page 17: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/17.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
17
room effect
pre-processing DSP chain
D ER Rev
input
![Page 18: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/18.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
18
room effect
pre-processing DSP chain
D ER Rev
input
![Page 19: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/19.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
19
room effect
pre-processing DSP chain
D ER Rev
input
![Page 20: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/20.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
20
room effect
pre-processing DSP chain
D ER Rev
input
Doppler effect, Air absorption, pre-delay, EQ, etc.
Artificial reverberator Object-oriented internal representation (based on simplified space-time-frequency model)
![Page 21: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/21.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
21
room effect
panning
pre-processing DSP chain
D ER Rev
input
Doppler effect, Air absorption, pre-delay, EQ, etc.
Artificial reverberator
Configurable panning module (VBAP, HOA, WFS, stereo, etc.)
![Page 22: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/22.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
22
room effect
panning
decoding / transcoding
pre-processing DSP chain
D ER Rev
input
speakers signals
Doppler effect, Air absorption, pre-delay, EQ, etc.
Artificial reverberator
Configurable transcoding module (transaural, HOA, etc.)
Configurable panning module (VBAP, HOA, WFS, stereo, etc.)
![Page 23: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/23.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Architecture!
23
room effect
panning
decoding / transcoding
pre-processing DSP chain
D ER Rev
input
speakers signals
Perceptual control interface
— Perceptually relevant — Intuitive control, mutually independent parameters — Continuous interpolation between room acoustical qualities — Does not rely on a physical or geometrical description — Independent of the reproduction system
![Page 24: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/24.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Software engineering!
24
![Page 25: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/25.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Software engineering!
25
• Development started in the early 1990s, originally as a library of Max patches
![Page 26: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/26.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Software engineering!
26
• Development started in the early 1990s, originally as a library of Max patches
• Many limitations (number of input/output)
• lack of flexibility
• lack of efficiency
• difficult to maintain
![Page 27: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/27.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Software engineering!
27
DSP library GUI library Spat
• Completely refactored as a set of two libraries
• host-independent
![Page 28: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/28.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
28
MaxMSP / PureData objects
DSP library GUI library Spat
![Page 29: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/29.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 29
MaxMSP / PureData objects
Live / interactive
performances
DSP library GUI library Spat
Spat : Environments!
![Page 30: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/30.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
30
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library Spat
![Page 31: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/31.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 31
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library
Mixing / post-production
Spat
Spat : Environments!
![Page 32: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/32.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
32
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production
![Page 33: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/33.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
33
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production
(Computer Aided)
Composition
![Page 34: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/34.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 34
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production
matlab
(Computer Aided)
Composition
Spat : Environments!
![Page 35: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/35.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
35
MaxMSP / PureData objects
VST / AU / AAX plugins
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production research /
simulations
matlab
(Computer Aided)
Composition
![Page 36: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/36.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
36
MaxMSP / PureData objects
VST / AU / AAX plugins
Realtime rendering Offline rendering
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production research /
simulations
matlab
(Computer Aided)
Composition
![Page 37: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/37.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
37
• Main advantages & drawbacks of each environments
![Page 38: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/38.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
38
• Main advantages & drawbacks of each environments
MaxMSP / PureData objects
Live / interactive
performances
Flexible and powerful rendering
![Page 39: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/39.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
39
• Main advantages & drawbacks of each environments
MaxMSP / PureData objects
Live / interactive
performances
Flexible and powerful rendering
Lack of authoring tools (essentially mapping of controllers/sensors)
inadequate for manipulation of temporal structures
![Page 40: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/40.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
40
• Main advantages & drawbacks of each environments
VST / AU / AAX plugins
Mixing / post-production
Robust timeline/transport, automation, edition
![Page 41: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/41.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
41
• Main advantages & drawbacks of each environments
VST / AU / AAX plugins
Mixing / post-production
Robust timeline/transport, automation, edition
Inappropriate for large multichannel productions
![Page 42: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/42.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
42
• Main advantages & drawbacks of each environments
OpenMusic
(Computer Aided)
Composition
Efficient/flexible manipulation of temporal and musical structures
![Page 43: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/43.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spat : Environments!
43
• Main advantages & drawbacks of each environments
OpenMusic
(Computer Aided)
Composition
Efficient/flexible manipulation of temporal and musical structures
Need for spatial authoring tools;
Tied to an offline workflow
![Page 44: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/44.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Goals!
44
• Improve workflow • Inter-exchange / inter-operability
• Reduce the gap between realtime and offline approaches
MaxMSP / PureData objects
VST / AU / AAX plugins
Realtime rendering Offline rendering
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production research /
simulations
matlab
(Computer Aided)
Composition
![Page 45: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/45.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 45
Compositional tools!
![Page 46: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/46.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
OM-Spat!
46 J. Bresson & M. Schumacher (ICMC 2011)!
![Page 47: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/47.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Observational Study!
47
Interviews with 4 composers at Ircam (~ 1h30)!
!
Observe how composers explore and implement spatial sound during the composition process.!
!
• Observations!
• Critical incidents!
• Brainstorming!
![Page 48: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/48.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Results: trajectories & geometrical contexts!
48
![Page 49: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/49.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Results: time organization!
49
Antescofo (P1)!Max Cues (P2)!DAW (P3)!Score (p4)!
![Page 50: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/50.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Results: problems and suggestions!
50
Studio vs Stage adaptation (all)!Trajectories identification and speed visualization (all)!
!
Quick input for creating new trajectories (P1 – P2)!
Room models and space transformations (P2 – P3)!
![Page 51: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/51.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Design implications!
Support gestural/quick input!• Drawing & input devices!• Generators and transformations!
!
51
![Page 52: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/52.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Design implications!
Support gestural/quick input!• Drawing & input devices!• Generators and transformations!!
Parameters Visualization & Geometrical context!!!
52
![Page 53: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/53.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Design implications!
Support gestural/quick input!• Drawing & input devices!• Generators and transformations!!
Parameters Visualization & Geometrical context!!Timeline : !
• Focus + Context!• Time manipulations!
!
53
![Page 54: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/54.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
OpenMusic prototypes!
![Page 55: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/55.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Room and parameters visualization!
55
Speed scale! Anaglyph!
![Page 56: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/56.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Gestural Input!
56
Input from external controllers/sensors (OSC)!
![Page 57: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/57.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Spatial scenes!
57
![Page 58: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/58.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Timelines challenges!
Handle various data!• Gestures / Computed trajectories / no time defined!!
Support time operations!• Time stretch / Synchronize / Interpolate!
58
![Page 59: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/59.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Timelines challenges!
Handle various data!• Gestures / Computed trajectories / no time defined!!
Support time operations!• Time stretch / Synchronize / Interpolate!
!!Goal: provide tools for interacting with temporal data!
59
![Page 60: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/60.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
TimedObjects in OpenMusic!
Objects with times!Different time labels!
• defined!• not defined!• master !
60
![Page 61: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/61.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
TimedObjects in OpenMusic!
Objects with times!Different time labels!
• defined!• not defined!• master !
Have different strategies for!“filling” missing time values!and inserting data at time!
• Constant speed!• Constant duration!!!
61
![Page 62: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/62.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
TimedObjects in OpenMusic!
Objects with times!Different time labels!
• defined!• not defined!• master !
Have different strategies for!“filling” missing time values!and inserting data at time!
• Constant speed!• Constant duration!!
Ongoing work: specifying time operations!
62
![Page 63: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/63.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Discussion & Perspectives!
Support quick input!Ø Generators and transformations!
!
63
![Page 64: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/64.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Discussion & Perspectives!
Support quick input!Ø Generators and transformations!
Parameters visualization & geometrical context!Ø Non color based mappings, 3d models!
64
![Page 65: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/65.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Discussion & Perspectives!
Support quick input!Ø Generators and transformations!
Parameters visualization & geometrical context!Ø Non color based mappings, 3d models!
!Timelines and timed objects!Ø Rhythm!Ø Post performance alignment!
65
![Page 66: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/66.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Discussion & Perspectives!
Support quick input!Ø Generators and transformations!
Parameters visualization & geometrical context!Ø Non color based mappings, 3d models!
!Timelines and timed objects!Ø Rhythm!Ø Post performance alignment!
Full process in OpenMusic!Ø Real-time audio feedback & offline synthesis!
66
![Page 67: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/67.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: mobile application!
67
Explore how mobile devices can support quick/gestural input, edition and comparison of trajectories.
With Xavier Favory, intern at Ircam
![Page 68: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/68.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: Interviews!
68
4 composers!
• 3 students!
• 1 experienced!
!
Focus on spatialization control!
Brainstorm with their examples!
![Page 69: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/69.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 69
Trajectoires: Findings!
Composers think about spatialization early but:!• Need sounds / musicians to explore it!• Usually one of the last aspects they do!!
![Page 70: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/70.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 70
Trajectoires: Findings!
Composers think about spatialization early but:!• Need sounds / musicians to explore it!• Usually one of the last aspects they do!
!Different strategies:!!
• Recording improvisations!• Writing automations!• Movements in Max (line)!
!
![Page 71: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/71.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 71
Trajectoires: Findings!
Composers think about spatialization early but:!• Need sounds / musicians to explore it!• Usually one of the last aspects they do!
!Different strategies:!!
• Recording improvisations!• Writing automations!• Movements in Max (line)!
!Suggestions (after trying a drawing app)!
• Real time audio feedback!• Store and replay trajectories for comparison!• Edit existing gestures!
![Page 72: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/72.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires!
72
![Page 73: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/73.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: Explorations with composers!
73
Technology probes approach (Hutchinson et al CHI 2003)!
Observe & understand in real world settings!
Test & assess the technology!
Generate new ideas!
![Page 74: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/74.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: Results!
ü Create and store trajectories during rehearsals and use them during the concert.!
ü Move to test in different locations of the room.!ü Extensively use the replay to compare and identify best
trajectories.!!
74
![Page 75: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/75.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: Results!
ü Create and store trajectories during rehearsals and use them during the concert.!
ü Move to test in different locations of the room.!ü Extensively use the replay to compare and identify best
trajectories.!!Ø Segmented drawing!Ø Source selection for a curve!!
75
![Page 76: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/76.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Trajectoires: Results!
ü Create and store trajectories during rehearsals and use them during the concert.!
ü Move to test in different locations of the room.!ü Extensively use the replay to compare and identify best
trajectories.!!Ø Segmented drawing!Ø Source selection for a curve!!² Edition!² Storage (librairies) !!² Time manipulations!² Multitouch + sensors for richer input!!
76
![Page 77: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/77.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 77
Mixing & Post-production
![Page 78: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/78.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
78
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
![Page 79: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/79.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
79
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
• Scene-based (Ambisonic)
![Page 80: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/80.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
80
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
• Scene-based (Ambisonic)
• Object-based
![Page 81: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/81.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
81
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
• Scene-based (Ambisonic)
• Object-based
(also hybrid approaches e.g. Dolby Atmos)
![Page 82: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/82.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 82
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
• Scene-based (Ambisonic)
• Object-based
(also hybrid approaches e.g. Dolby Atmos)
Object-based approach has several advantages (adaptation to various setups, user interactivity, etc.)
Context & Motivations!
![Page 83: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/83.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 83
Three paradigms for multichannel productions :
• Channel-based (5.1 or alike)
• Scene-based (Ambisonic)
• Object-based
(also hybrid approaches e.g. Dolby Atmos)
Object-based approach has several advantages (adaptation to various setups, user interactivity, etc.)
Current shift of content producers towards object-based spatialization (3D movies, binaural broadcast, interactive streaming, etc.)
Context & Motivations!
![Page 84: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/84.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
84
Object-oriented spatial processors are :
• plenty
• efficient and offer great possibilities
• mostly integrated into real-time environments (Max/MSP, PureData, etc.)
![Page 85: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/85.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Context & Motivations!
85
Object-oriented spatial processors are :
• plenty
• efficient and offer great possibilities
• mostly integrated into real-time environments (Max/MSP, PureData, etc.)
Constraints :
• authoring of spatial sound scenes remains a major challenge (real-time environments are ill-suited to manipulation of temporal structures such as trajectories)
• DAWs provide an efficient timeline and transport bar, but they lack flexibility for multichannel streams
![Page 86: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/86.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
86
Proposed approach :
• Mixing, editing and automation within the DAW
• Spatial rendering delegated in a remote application
• OSC protocol for inter-applications communication (spatialization metadata)
![Page 87: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/87.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
87
![Page 88: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/88.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
88
![Page 89: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/89.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
89
![Page 90: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/90.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
90
![Page 91: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/91.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
91
![Page 92: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/92.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Workflow!
92
![Page 93: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/93.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Limitations & Perspectives!
93
• Flexible / generic plugin that eases inter-exchange
• Not (yet) a standardized format for metadata
• Still tedious to setup
![Page 94: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/94.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Limitations & Perspectives!
94
![Page 95: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/95.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015 95
Conclusion
![Page 96: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/96.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Conclusion!
96
• Improve workflow • Inter-exchange / inter-operability
• Reduce the gap between realtime and offline approaches
MaxMSP / PureData objects
VST / AU / AAX plugins
Realtime rendering Offline rendering
Live / interactive
performances
DSP library GUI library
OpenMusic
Mixing / post-production research /
simulations
matlab
(Computer Aided)
Composition
![Page 97: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/97.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Future work!
97
• Improve workflow • Inter-exchange / inter-operability
• Reduce the gap between realtime and offline approaches
Ø Only trajectories but other paradigms
Ø Interoperability with other tools
Ø Evaluations in musical context
![Page 98: Processing, Authoring and Composing of Spatial Sounds ...recherche.enac.fr/~jeremie.garcia/docs/2015_Workshop...Lack of authoring tools (essentially mapping of controllers/sensors)!](https://reader036.fdocuments.in/reader036/viewer/2022071403/60f718244887894a266f3fd5/html5/thumbnails/98.jpg)
T. Carpentier, J. Garcia — Processing, Authoring and Composing of Spatial Sounds — CIRMMT Workshop, 2015
Thibaut Carpentier & Jérémie Garcia
UMR 9912 STMS IRCAM — CNRS — UPMC!
Processing, Authoring and Composing of Spatial Sounds (using Spat and OpenMusic)!
Workshop on Interactivity in Music Composition and Performance
May 5th 2015 CIRMMT — McGill University, Montreal
ANR-13-JS02-0004