Gestures for Gestures for the Science of Collaborative Arts the Science of Collaborative Arts...

Gestures for Gestures for the Science of the Science of Collaborative ArtsCollaborative Arts

Guerino MazzolaGuerino MazzolaU & ETH ZürichU & ETH Zü[email protected]@mazzola.chwww.encyclospace.org/talkswww.encyclospace.org/talks

• FactsFacts

• ProcessesProcesses

• GesturesGestures

• CollaborationCollaboration

to refer to

to be the case

to make

to interact

• FactsFacts




Point of departure: free jazz vs. mathsPoint of departure: free jazz vs. maths

Splitting objective facts from subjective action

Mathematical Music Theory (MaMuTh)Mathematical Music Theory (MaMuTh)

Books:Books:• Gruppen und Kategorien in der Musik (1985)Gruppen und Kategorien in der Musik (1985)• Geometrie der Töne (1990)Geometrie der Töne (1990)• The Topos of Music (2002)The Topos of Music (2002)• La vérité du beau dans la musique (2006)La vérité du beau dans la musique (2006)

Software:Software:• prestopresto® ®

(Atari 1988)(Atari 1988)

• RubatoRubato® ®

(NEXTSTEP 1983, Mac OS X 1998, Java 2002)(NEXTSTEP 1983, Mac OS X 1998, Java 2002)

MaMuTh ModelsMaMuTh Models

• Tonal Modulation (Muzzulini, Straub, Noll, Radl)Tonal Modulation (Muzzulini, Straub, Noll, Radl)

• Fux Counterpoint (Muzzulini, Hichert, Wieser, Brunner)Fux Counterpoint (Muzzulini, Hichert, Wieser, Brunner)

• Riemann Function Theory (Noll)Riemann Function Theory (Noll)

• Riemann/GTTM Metric and Rhythms (Volk-Fleischer)Riemann/GTTM Metric and Rhythms (Volk-Fleischer)

• Reti Motif Theory (Straub, Buteau, Nestke)Reti Motif Theory (Straub, Buteau, Nestke)

• Ruwet-Nattiez Paradigmatic Composition (Blum, Milmeister)Ruwet-Nattiez Paradigmatic Composition (Blum, Milmeister)

• Classification and Enumeration Theory (Fripertinger, Noll)Classification and Enumeration Theory (Fripertinger, Noll)

• Performance Theory (Zahorka, Stange-Elbe, Müller)Performance Theory (Zahorka, Stange-Elbe, Müller)

• Denotator Concept Framework Denotator Concept Framework (Zahorka, Montiel, Göller, Milmeister)(Zahorka, Montiel, Göller, Milmeister)

Pitch classes inPitch classes in ŸŸ1212

chords = truth valueschords = truth values


chords = truth valueschords = truth valuesPitch classes inPitch classes in ŸŸ1212

truth values = chordstruth values = chords


truth values = chordstruth values = chords

II IVIV VVIIII IIIIII VIVI VIIVII

CC(3)(3)

I

IV

II

VIV

III

VII

Schönberg‘s „harmonic strip“ N(XSchönberg‘s „harmonic strip“ N(X(3)(3)))(Harmonielehre 1911)(Harmonielehre 1911)

„ „ harmonic strip = Möbius strip“harmonic strip = Möbius strip“is a logical fact!is a logical fact!

„ „ harmonic strip = Möbius strip“harmonic strip = Möbius strip“is a logical fact!is a logical fact!

John Keats (Ode on Grecian Urn, 1819):John Keats (Ode on Grecian Urn, 1819):

„„Beauty is truth, truth beauty,“—that is Beauty is truth, truth beauty,“—that is all Ye know on earth, and all ye need to all Ye know on earth, and all ye need to know.know.

Alexander Grothendieck (1960...):Alexander Grothendieck (1960...):

Topos TheoryTopos TheoryUnification of logic and geometryUnification of logic and geometry„„Truth values are point sets“Truth values are point sets“

n/16n/16

aa

bb

cc

dd

ee

00 22 33 44 66 88 1010 1212

b

Nerve N(XNerve N(XMAXMAX))of covering of covering MAX = {a, b, c, d, e}MAX = {a, b, c, d, e}

e

c

a

d

aabbccddee

00

be

c

a

d

N(XN(XMAXMAX))

00 22 33 44 66 1010 1212

44

22

1212 1010

33

x dominates y iff Sp(y) x dominates y iff Sp(y) Sp(x) Sp(x)x dominates y iff Sp(y) x dominates y iff Sp(y) Sp(x) Sp(x)

66

w(x) = w(x) = M M Œ Œ Sp(x), mSp(x), m £ £ length(M)length(M) length(M) length(M)ppw(x) = w(x) = M M Œ Œ Sp(x), mSp(x), m £ £ length(M)length(M) length(M) length(M)pp

m = p = 2m = p = 2

4

6

8

10

12

14

16

18

20

22

• FactsFacts




sets of notessets of notes

Transformational Theory, K-nets (Lewin et al.)Transformational Theory, K-nets (Lewin et al.)

Conference „Autour de la Set Theory“Conference „Autour de la Set Theory“IRCAM/Paris Oct. 15-16 2003IRCAM/Paris Oct. 15-16 2003

(J. Rahn, R. Morris, A. Forte,...)(J. Rahn, R. Morris, A. Forte,...)

{c, {c, (c), (c), 22(c),...} (c),...} = {c, e, g} = {c, e, g}

= major triad= major triad

ŸŸ1212

Circle ChordsCircle Chordscc

gg

ee

c = 0c = 0 (p) = 3p+7 (p) = 3p+7

Classification of circle chords (cf. Classification of circle chords (cf. Gruppen und KategorienGruppen und Kategorien))Classification of circle chords (cf. Classification of circle chords (cf. Gruppen und KategorienGruppen und Kategorien))

Diagram in a category CDiagram in a category CC = sets, groups, vector spaces, automata, etc. C = sets, groups, vector spaces, automata, etc. = digraph= digraph

DD: : CC

ii

jj

ll

mm

aaijijtt

aaililqq

aajmjmss

aalilipp

aajljlkk

aallllrr

• DDii = objects in = objects in CC• DDijij

tt = morphisms in = morphisms in CC

DDii

DD jj

DDll

DD mm

DDijijtt

DDililqq

DDjmjmss

DDlilipp

DDjljlkk

DDllllrr

DD

CC

Nets of dodecaphonic seriesNets of dodecaphonic series

Multiply iterated netsMultiply iterated nets

ssmm

ssjj

ssll

ssii

FFii

FFjj

FFll

FFmm

ffijijtt DD

ffililqq

ffjmjmss

fflilipp

ffjljlkk

ffllllrr GGii

GGjj

GGll

GGmm

ggijijtt DD**

ggililqq

ggjmjmss

gglilipp

ggjljlkk

ggllllrr

ss

FFDD FFDD**FFss

Perspectives of New Music (2006)Guerino Mazzola & Moreno Andreatta:From a Categorical Point of View: K-nets as Limit Denotators

Perspectives of New Music (2006)Guerino Mazzola & Moreno Andreatta:From a Categorical Point of View: K-nets as Limit Denotators

Theorem: Using topos theory, we can show thatnetworks of networks of... are networks: Arbitrary iterations possible without conceptual changes.

Theorem: Using topos theory, we can show thatnetworks of networks of... are networks: Arbitrary iterations possible without conceptual changes.

global networks global networks

Theorem:Theorem: Isomorphism classes of global Isomorphism classes of global compositions are described by local compositions are described by local networks of affine functions.networks of affine functions.

Theorem:Theorem: Isomorphism classes of global Isomorphism classes of global compositions are described by local compositions are described by local networks of affine functions.networks of affine functions.

COLLOQUIUM ON MATHEMATHICAL MUSIC THEORY COLLOQUIUM ON MATHEMATHICAL MUSIC THEORY H. Fripertinger, L. Reich (Eds.) H. Fripertinger, L. Reich (Eds.) Grazer Math. Ber., ISSN 1016–7692 Bericht Nr. 347 (2005), Grazer Math. Ber., ISSN 1016–7692 Bericht Nr. 347 (2005), Guerino Mazzola: Local and Global Limit Denotators and the Guerino Mazzola: Local and Global Limit Denotators and the Classification of Global CompositionsClassification of Global Compositions

COLLOQUIUM ON MATHEMATHICAL MUSIC THEORY COLLOQUIUM ON MATHEMATHICAL MUSIC THEORY H. Fripertinger, L. Reich (Eds.) H. Fripertinger, L. Reich (Eds.) Grazer Math. Ber., ISSN 1016–7692 Bericht Nr. 347 (2005), Grazer Math. Ber., ISSN 1016–7692 Bericht Nr. 347 (2005), Guerino Mazzola: Local and Global Limit Denotators and the Guerino Mazzola: Local and Global Limit Denotators and the Classification of Global CompositionsClassification of Global Compositions

Corollary:Corollary: The theory of global The theory of global compositions is absorbed by the theory of compositions is absorbed by the theory of global networks.global networks.

Corollary:Corollary: The theory of global The theory of global compositions is absorbed by the theory of compositions is absorbed by the theory of global networks.global networks.

global compositionsglobal compositions

Gérard MilmeisterGérard Milmeister(chief programmer)(chief programmer)

Julien JunodJulien Junod(counterpoint)(counterpoint)

Florian ThalmannFlorian Thalmann(morphing)(morphing)

Karim MorsyKarim Morsy(macro objects)(macro objects)

http://www.rubato.orghttp://www.rubato.org

• FactsFacts




May 18th 2002May 18th 2002IRCAM talkIRCAM talk

DigraphDigraph

NetworksNetworksTransformationsTransformations

ParadigmParadigm

??

Journal of Mathematics and Music (2007)Guerino Mazzola & Moreno Andreatta:Diagrams, Gestures, and Formulas in Music.

Journal of Mathematics and Music (2007)Guerino Mazzola & Moreno Andreatta:Diagrams, Gestures, and Formulas in Music.

ll

hh

ee

sonicsoniceventsevents

scorescore

analysisanalysis

instrumentalinstrumentalinterfaceinterface

√√thawthaw freeze (MIDI)freeze (MIDI)

instrumentalizeinstrumentalizeinstrumentalizeinstrumentalize

positionposition

pitchpitch

timetime

gesturesgestures

Folie 2

Czeslaw Marek: Czeslaw Marek: Die Lehre des KlavierspielsDie Lehre des Klavierspiels

Every Noh play is a cross sectionEvery Noh play is a cross sectionof the life of one person, the of the life of one person, the shiteshite..Shite weaves a texture of fantasy Shite weaves a texture of fantasy using using curvescurves..

The The wakiwaki is a kind of cosubject is a kind of cosubject and mirror person of the shite.and mirror person of the shite.

Waki describes reality Waki describes reality using using straight linesstraight lines..

Noh = expression of a space-timeNoh = expression of a space-time

The semiotic of gestures is reversed:The semiotic of gestures is reversed:

Gestures serve mainly the expressionGestures serve mainly the expressionof movements within this space-time,of movements within this space-time,the (usual) semantic is of second order.the (usual) semantic is of second order.

——positionposition

pitchpitch

timetime 00

11

11

22

2 2 + + 11

t.t.

22

11

1 1 22

II

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Z Y

Xxx

zz yy

bb55

bb22

dd55

dd22

11(t(t))

66(t(t))

22(t(t))

33(t(t))

44(t(t))

55(t(t))

One hand One hand product product = = 112233445566

of 6 gestural curves in space-time (x,y,z;e) of pianoof 6 gestural curves in space-time (x,y,z;e) of pianoj = 1, 2, ... 5: tips of fingers,j = 1, 2, ... 5: tips of fingers,j = 6: the carpus, j = 6: the carpus, 6 6 = = rootroot

parameter t parameter t sequence of points: sequence of points:

(t) = ((t) = (11(t),...,(t),...,66(t))(t))

two base vectorstwo base vectorsof fingersof fingersdd22, d, d55

from carpus.from carpus.

e = timee = time

Gestural interpretation of Carl Czerny‘s op. 500Gestural interpretation of Carl Czerny‘s op. 500

Zur Anzeige wird der QuickTime™ Dekompressor „H.264“

benötigt.

The universe of gesturesThe universe of gestures• Paul ValéryPaul Valéry ......les ouvrages de la main prodigieuse de l‘ariste les ouvrages de la main prodigieuse de l‘ariste

égale et rivale de sa pensée - l‘une n‘est rien sans l‘autreégale et rivale de sa pensée - l‘une n‘est rien sans l‘autre• Charles AlunniCharles Alunni Ce n‘est pas la règle qui gouverne l‘action Ce n‘est pas la règle qui gouverne l‘action

diagrammatique, mais l‘action qui fait émerger la règle.diagrammatique, mais l‘action qui fait émerger la règle.• Jean CavaillèsJean Cavaillès Comprendre est attraper le geste et pouvoir continuer.Comprendre est attraper le geste et pouvoir continuer.• Pierre BoulezPierre Boulez Mais on est passé depuis Adorno de l‘informel à l‘informulé, Mais on est passé depuis Adorno de l‘informel à l‘informulé,

de l‘objet au geste.de l‘objet au geste.• Ludwig WittgensteinLudwig Wittgenstein I think with my pen.I think with my pen.• Cecil TaylorCecil Taylor The body is in no way supposed to get involved in Western musicThe body is in no way supposed to get involved in Western music.

I try to imitate on the piano the leaps in space a dancer makes.I try to imitate on the piano the leaps in space a dancer makes.• Robert S. HattenRobert S. Hatten Given the importance of gesture to interpretation, why do we not Given the importance of gesture to interpretation, why do we not

have a comprehensive theory of gesture in music?have a comprehensive theory of gesture in music?• Francis BaconFrancis Bacon The marks are made, and you survey the thing like you would The marks are made, and you survey the thing like you would

a sort of graph. And you see within this graph the possibilities of a sort of graph. And you see within this graph the possibilities of all types of fact being planted.all types of fact being planted.

• Bertold BrechtBertold Brecht ...non verbis sed gestibus....non verbis sed gestibus.• Theodor W. AdornoTheodor W. Adorno Der Ausdruck ist bereits die Rationalisierung des Gestischen, Der Ausdruck ist bereits die Rationalisierung des Gestischen,

d.h. dessen Objektivierung durch Signifikation.d.h. dessen Objektivierung durch Signifikation.

Hugues de Saint-Victor (Parisian theologian XIIth century):Hugues de Saint-Victor (Parisian theologian XIIth century):

Gestus est motus et figuratio membrorum corporis,Gestus est motus et figuratio membrorum corporis,ad omnem agendi et habendi modum.ad omnem agendi et habendi modum.

Gesture is the movement and figuration of the body's Gesture is the movement and figuration of the body's limbs, according to whatever modality of action and attitudelimbs, according to whatever modality of action and attitude

Have category of gesturesHave category of gestures

gesturegesture = morphism g: = morphism g: of digraphs of digraphs

XX

positionposition

pitchpitch

timetime

XXgg

For a topological space X, we have For a topological space X, we have

the digraph of continuous curves the digraph of continuous curves : I : I X XXX

„„-gesture in X“-gesture in X“

TipSpaceTipSpace

positionposition

pitchpitch

timetime

What about real shapes?What about real shapes?

circlecircle

Answer: Hypergestures! Answer: Hypergestures!

knotknot

„„loop of loops“loop of loops“

Zur Anzeige wird der QuickTime™ Dekompressor „Animation“

benötigt.

spacespace

spacespace

timetime

ET-dance Gesture ET-dance Gesture

X = torusX = torus

gg

hh

Hypergesture impossible!

Fact:Fact:

For a a digraph For a a digraph and topological space X, the set and topological space X, the set

@@X of X of -gestures in X-gestures in X bears a canonical topology bears a canonical topology

(essentially the compact-open topology).(essentially the compact-open topology).

XX

@@@@XX

„„gesturesgestures of gesturesof gestures of gesturesof gestures of....“of....“

homotopichomotopiccurves:curves:homotopy classeshomotopy classesof curvesof curves

XX

Gestoids:Gestoids: Transition from gestures to algebraic Transition from gestures to algebraicstructures connecting gesturesstructures connecting gestures

0 10 1

0 1

compositioncompositionof homotopyof homotopyclasses of curvesclasses of curvesis associative!is associative!

XX

The homotopy classes of curves of a gesture g,The homotopy classes of curves of a gesture g,together with their composed curves, give risetogether with their composed curves, give riseto a category, the to a category, the groupoidgroupoid Gr Grgg of the gesture. of the gesture.

Ist objects are the Ist objects are the verticesvertices x,y,... of g, while the x,y,... of g, while themorphisms Grmorphisms Grgg(x,y) are the homotopy classes c(x,y) are the homotopy classes c

of of composed curvescomposed curves from x to y. from x to y.

The The gestoidgestoid G Ggg is the is the ¬¬-linear category -linear category ø¬ø¬GrGrgg..

The morphism GThe morphism Ggg(x,y) are the formal (x,y) are the formal ¬¬--linearlinear

combinationscombinations

nn a annccnn

of morphisms cof morphisms cnn in Gr in Grgg(x,y). (x,y). Too abstract?

Too abstract?

eei2i2pptt

——

ii——

11

ii

X = SX = S11

GGg g ¬ ¬ pp11(S(S11))

fundamental groupfundamental group

pp11(S(S11) ) ŸŸ

eei2i2ppntnt ~ n ~ n

~ Fourier representation f(t) = ~ Fourier representation f(t) = nn a ann eei2i2ppnt nt nn a ann eei2i2ppntnt

Gestures Gestures ==

Musical String TheoryMusical String Theory

Gestures Gestures ==

Musical String TheoryMusical String Theory

g:g:

• FactsFacts




• Jazz: Collaboration as a performing artJazz: Collaboration as a performing artR.A.R. MacDonald & G.B. Wilson: R.A.R. MacDonald & G.B. Wilson: Construction of jazz: how jazz musicians present their collabortive Construction of jazz: how jazz musicians present their collabortive musical practicemusical practice. Musicae Scientiae X, Nr.1, Spring 2006. Musicae Scientiae X, Nr.1, Spring 2006

• Flow (Mihaly Csikszentmihalyi)Flow (Mihaly Csikszentmihalyi)is being described as „being completely involved in is being described as „being completely involved in an activity for its own sake. The ego falls away. an activity for its own sake. The ego falls away. Time flies. Every action, movement, and thought Time flies. Every action, movement, and thought follows inevitably from the previous one, like follows inevitably from the previous one, like playing jazz.“playing jazz.“Jazz improvisation as flow:Jazz improvisation as flow:„...the music will work itself“„...the music will work itself“„...goes into autopilot“„...goes into autopilot“„...stream of consciousness“„...stream of consciousness“B..J. Kenney M. Gellrich: Improvisation. In: R. Parncutt and G.E. McPherson (eds.) B..J. Kenney M. Gellrich: Improvisation. In: R. Parncutt and G.E. McPherson (eds.) The science and The science and psychology of musical performance. Creative Strategies for teaching and psychology of musical performance. Creative Strategies for teaching and learninglearning. New York: OUP 2002. New York: OUP 2002

chick-sent-me-high-eechick-sent-me-high-ee

• Francis Bacon (relating to the diagram debate): Francis Bacon (relating to the diagram debate): In the way I work I don’t in fact know very often what In the way I work I don’t in fact know very often what the paint will do, and it does many things which are very the paint will do, and it does many things which are very much better than I could make it do. Is that an accident?much better than I could make it do. Is that an accident?

......What has never yet been analysed is why this particular What has never yet been analysed is why this particular way of painting is more poignant than illustration. I way of painting is more poignant than illustration. I suppose because it has a life completely of its own. (...) suppose because it has a life completely of its own. (...) So the artist may be able to open up or rather, should I So the artist may be able to open up or rather, should I say, unlock the valves of feeling.say, unlock the valves of feeling.

D. Sylvester: Interview with Francis Bacon: The Brutality of Fact. Thames and Hudson, New York 1975D. Sylvester: Interview with Francis Bacon: The Brutality of Fact. Thames and Hudson, New York 1975

Thesis: Gestures help Thesis: Gestures help • articulate/communicatearticulate/communicate• organize/structureorganize/structure• intensify/deepenintensify/deepenthe flow in collaborative arts the flow in collaborative arts

Thank you!Thank you!

Download this talk fromDownload this talk from

www.encyclospace.org/talkswww.encyclospace.org/talks

Gestures for Gestures for the Science of Collaborative Arts the Science of Collaborative Arts...

Documents

Transcript of Gestures for Gestures for the Science of Collaborative Arts the Science of Collaborative Arts...