Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β...
Transcript of Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β...
![Page 1: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/1.jpg)
Music and Text Generation βin the style ofβ
François Pachet
SONY CSL & LIP6, UPMC
![Page 2: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/2.jpg)
![Page 3: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/3.jpg)
![Page 4: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/4.jpg)
Constraints and composition
![Page 5: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/5.jpg)
Constraints and text writing
Palindromes:- Β« A man, a plan, a canal: Panama Β»- Georges Perecβs palindromes (1,247 words)
Lipogrames- G. Perec βLa Disparitionβ without voyel βeβ- βLes revenentes (Perec, texte)β, with only voyel βeβ
Β« Telles des chΓ¨vres en dΓ©tresse, sept MercΓ©dΓ¨s-Benz vertes, les fenΓͺtres crΓͺpΓ©es de reps grΓ¨ge, descendent lentement West end Street et prennent sΓ©nestrementTemple Street vers les vertes venelles semΓ©es de hΓͺtres et de frΓͺnes prΓ¨s desqelles se dresse, svelte et empesΓ© en mΓͺme temps, l'EvΓͺchΓ© d'Exeter.
![Page 6: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/6.jpg)
Creativity often arises from playingwith styles
Ghedini et al. The Flow Machines project. Ijcai 2013 and AAAI 2013 best video award
![Page 7: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/7.jpg)
Imitative Sequence Generation
That sounds/reads/looks like πΆAND
enforce specific properties:domain-dependentuser defined
Given a corpus, i.e. a set of finite sequences
πΆ = π1, π2, β¦ , ππ π€βπππ ππ = ππ1, ππ2, β¦ , ππ
π
Generate 1 sequence / the best
Generate all sequences
Generate a representative subset of sequences (β sampling)
![Page 8: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/8.jpg)
Imitative Sequence Generation
That sounds/reads/looks like πΆAND
enforce specific properties:domain-dependentuser defined
Given a corpus, i.e. a set of finite sequences
πΆ = π1, π2, β¦ , ππ π€βπππ ππ = ππ1, ππ2, β¦ , ππ
π
Generate 1 sequence / the best
Generate all sequences
Generate a representative subset of sequences (β sampling)
Optimization
![Page 9: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/9.jpg)
Imitative Sequence Generation
That sounds/reads/looks like πΆAND
enforce specific properties:domain-dependentuser defined
Given a corpus, i.e. a set of finite sequences
πΆ = π1, π2, β¦ , ππ π€βπππ ππ = ππ1, ππ2, β¦ , ππ
π
Generate 1 sequence / the best
Generate all sequences
Generate a representative subset of sequences (β sampling)
Statistical inference
Optimization
![Page 10: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/10.jpg)
Imitative Sequence Generation
That sounds/reads/looks like πΆAND
enforce specific properties:domain-dependentuser defined
Given a corpus, i.e. a set of finite sequences
πΆ = π1, π2, β¦ , ππ π€βπππ ππ = ππ1, ππ2, β¦ , ππ
π
Generate 1 sequence / the best
Generate all sequences
Generate a representative subset of sequences (β sampling)
Statistical inferenceCSP & Global constraints
Optimization
![Page 11: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/11.jpg)
Imitative Sequence Generation
That sounds/reads/looks like πΆAND
enforce specific properties:domain-dependentuser defined
Given a corpus, i.e. a set of finite sequences
πΆ = π1, π2, β¦ , ππ π€βπππ ππ = ππ1, ππ2, β¦ , ππ
π
Generate 1 sequence / the best
Generate all sequences
Generate a representative subset of sequences (β sampling)
Statistical inferenceCSP & Global constraints
Information geometryOptimization
![Page 12: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/12.jpg)
Style and Markov chains
Markov Hypothesis
π π π π 1, π 2, . . , π πβ1) = π π π π πβ1)
Random walk
![Page 13: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/13.jpg)
Generating Markov Sequences
π π π π 1, π 2, . . , π πβ1) = π π π π πβ1)
Random walk:generate X1 with prior P(X1),then X1with P(X2 X1then find the longest prefix π β€ πππ₯ for which
there are at least π continuations (π β₯ 1) and drawP(Xn Xnβk, Xnβk+1, β― , Xnβ1
π1 π2
π1 π2 π3 π π π π 1, π 2, . . , π πβ1) = π π π π πβ1, π πβ2)
Order 1
Order 2
Variable order with max bound
![Page 14: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/14.jpg)
Markov and Music Improvisation: Continuator
Pachet, F. The Continuator: Musical Interaction with Style J. of New Music Research, 2003, best paper awardPachet, F. Music Interaction With Style in SIGGRAPH 2003 Abstracts and Applications, San Diego, 2003
A Veenendaal
Alan Silva
Bernard Lubat
Interactive Markov chains for stylistic imitation
![Page 15: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/15.jpg)
Continuator
![Page 16: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/16.jpg)
Continuator with children
Addessi, A.-R. and Pachet, F. Experiments with a Musical Machine: Musical Style Replication in 3/5 year old Children. British J. of Music Education, 22(1):21-46 March 2005
![Page 17: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/17.jpg)
2 days later, the childinvents a new style,
which sounds like Bach arpeggios β¦
![Page 18: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/18.jpg)
![Page 19: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/19.jpg)
Composition
Arranging
Accompaniment
Style?
Improvisation
![Page 20: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/20.jpg)
Problem: control is incompatible with Markov
=
π π π π 1, π 2, . . , π πβ1) β π π π π πβ1)
Long-range correlationsGlobal constraints
Because of
![Page 21: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/21.jpg)
control ?
Unary, binary, nary constraints (music Β« rules Β»)
Max order (avoid plaggiarism)
Meter (essential !)
Cardinality (grains of salts)
Alldiff (ensure diversity)
Spreading and distributions (natural properties, e.g. 1/f)
Etc.
![Page 22: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/22.jpg)
Unary, binary, nary constraints (music Β« rules Β»)
Max order (avoid plaggiarism)
Meter (essential !)
Cardinality (grains of salts)
Alldiff (ensure diversity)
Spreading and distributions (natural properties, e.g. 1/f)
Etc.
.. impossible with random walk in Markov chains
![Page 23: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/23.jpg)
Constrained Markov sequences: a new class of problems
General Solution for optimization problems:Pachet, F. and Roy, P. Markov constraints: steerable generation of Markov sequences, Constraints, 2011.
Unary constraints solved in linear time:Pachet, Roy & Barbieri, Finite-Length Markov Processes with Constraints, IJCAI 2011
Meter in pseudo-polynomial time:Roy, P. and Pachet, F. Enforcing Meter in Finite-Length Markov Sequences. AAAI, 2013
MaxOrder (= enforcing novelty) in linear timePapadopoulos, A., Roy, P., Pachet, F. Avoiding Plagiarism in Markov Sequence Generation, AAAI, 2014 http://www.flow-machines.com/maxOrder
Distribution (= spectrum, 1/f) Pachet, F. Roy, P. Sakellariou, Generating 1/f noise sequences as a CSP, IJCAI, 2015
![Page 24: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/24.jpg)
Virtuoso phrase example:Stefano di Battista, Β« Night in Tunisia Β»
![Page 25: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/25.jpg)
The Physiological Perspective on Virtuosity
β Strong human motor and perception limits (seeextreme drumming)
β Virtuoso = 10,000 hours of practice (Sloboda et al.)
β Virtuosos dont make mistakes, because theysuppress the slow monitoring functions of the brain:
Virtuosos avoid the speed traps of their pre-frontalcortices1
In other words: they donβt think
1 Justin London, The Psychology and Neurobiology of Musical Virtuosity, Whitehead Lecture in
"Cognition, Computation, and Culture," given at Goldsmiths College, U. London, 2010.
![Page 26: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/26.jpg)
The AI View on Virtuosity
β’ Virtuoso do extraordinary things (super experts)
β’ Compilation of know-how in the body is a wayto Β« solve problems Β»
β’ From the viewpoint of Markov chains:
Well-defined, apparently hard problem
Β« Virtuosos are NP-hard problem solvers Β»
Pachet, F. Bebop Virtuosity Explained McCormack & d'Inverno, Eds. Computers and Creativity, Springer, 2012
![Page 27: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/27.jpg)
Unary constraints can be solved in real-time
If one considers only:
β Unary (1 variable) constraints at any position in the sequence, or
β n-ary between n consecutive states (n < order)
Then β a Markov chain Mβ s.t.
β Mβ generate only the sequences satisfying the constraints
β Mβ and M are statistically equivalent
=> virtuosity problem solved
Pachet, Roy & Barbieri, Finite-Length Markov Processes with Constraints, IJCAI 2011
![Page 28: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/28.jpg)
Example
![Page 29: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/29.jpg)
Generate from this example
Generate all 4-notes melodies ending with D with theircorrect probabilities (sampling)
β Reformulation as CSP with 4 variables
β Arc-consistency of the network with Markov constraint
β Retro-normalisation
(was Β½ 1/6 1/3)
Papadopoulos, Pachet, Roy, Sakellariou, Exact sampling for regular and Markov constraint with belief propagation, submitted
![Page 30: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/30.jpg)
Virtuoso, with Mark dβInverno (U. of London)
![Page 31: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/31.jpg)
Markov and meter
General Solution for optimization problems:Pachet, F. and Roy, P. Markov constraints: steerable generation of Markov sequences, Constraints, 2011.
Unary constraints solved in linear time:Pachet, Roy & Barbieri, Finite-Length Markov Processes with Constraints, IJCAI 2011
Meter in pseudo-polynomial time:Roy, P. and Pachet, F. Enforcing Meter in Finite-Length Markov Sequences. AAAI, 2013
MaxOrder (= enforcing novelty) in linear timePapadopoulos, A., Roy, P., Pachet, F. Avoiding Plagiarism in Markov Sequence Generation, AAAI, 2014, http://www.flow-machines.com/maxOrder
Distribution (= spectrum, 1/f) Pachet, F. Roy, P. Sakellariou, Generating 1/f noise sequences as a CSP, IJCAI, 2015
![Page 32: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/32.jpg)
Meter
β’ In the style of M. Legrand
β’ Total duration = 8-bars
β’ No note spanning bar lines
β’ Total duration is π=1π ππ’πππ‘πππ(πππ‘ππ)
β’ Duration until πππ‘ππΎ is π=1πΎ ππ’πππ‘πππ(πππ‘ππ)
![Page 33: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/33.jpg)
Impossible to explore all chains
β’ For n=23, there are 13 millions pathsβ’ Exponential growth with sequence length
β¦
![Page 34: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/34.jpg)
Additive number theory
β’ Counting integer points:β Polytopes
β Sumsets βπ΄ = π1 + π2 +β―+ πβ| ππ β π΄
β Naively: β(π΄) β ( π΄ /2)β
β’ Khovanskii theorem: the number of integer points in a convex polytope is a polynomial:β β(π΄) = π(β) with P polynomial (for h big enough)
β Degree of P is < π΄
Khovanskii, A. 1992. Newton polyhedron, Hilbert polynomial, and sums of finite sets.Functional Analysis and Its Applications 26(4):276β281
![Page 35: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/35.jpg)
After 4 steps
β’ N = 4 => more than 130 paths
β’ But only 3 unique path lengths: 4, 7 and 10
![Page 36: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/36.jpg)
We explore the set of all path lengths
β’ n=23, 13 millions paths
β’ Only 9 unique lengths !
![Page 37: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/37.jpg)
![Page 38: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/38.jpg)
Capturing the style of Ray dβInverno
Original recording of Rayβs comping on Girl From Ipanema
Constraint: Giant Steps score
Basic rendering
![Page 39: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/39.jpg)
![Page 40: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/40.jpg)
Rayβs reaction
Hi Francois, It is fantastic. Well done.
It is very realistic - you can even hear the wrong notes in some of the chords !
Keep up the good work.
Cheers, RealRay
Pachet, F. and Roy, P. Beyond minus ones: Virtual Band Siggraph talk and demo, Los Angeles, 2012
![Page 41: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/41.jpg)
Giant Steps by Wagner
![Page 42: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/42.jpg)
Capture the style of Take 6!
One of the best a capella jazz groups. 10 Grammy awards
Rich harmonisations, veryrecognisable
Very hard to imitate, even to transcribe!
![Page 43: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/43.jpg)
Reaction of composers: Ivan Linsβ The Island
Grammy-winning Brazilian songwriter. His hit "Love Dance" is one of the most re-recorded songs in musical history (Wikipedia)
![Page 44: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/44.jpg)
The Island harmonized by Take 6Rio, 2013
Pachet & Roy, Non-Conformant Harmonization: The Real Book in the style of Take 6, Int. Conf. on Comput. Creativity, 2014.
![Page 45: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/45.jpg)
Beyond random walk: principled generation
General Solution for optimization problems:Pachet, F. and Roy, P. Markov constraints: steerable generation of Markov sequences, Constraints, 2011.
Unary constraints in linear time:Pachet, Roy & Barbieri, Finite-Length Markov Processes with Constraints, IJCAI 2011
Meter in pseudo-polynomial time:Roy, P. and Pachet, F. Enforcing Meter in Finite-Length Markov Sequences. AAAI, 2013
MaxOrder (= enforcing novelty) in linear timePapadopoulos, A., Roy, P., Pachet, F. Avoiding Plagiarism in Markov Sequence Generation, AAAI, 2014, http://www.flow-machines.com/maxOrder
Distribution (= spectrum, 1/f) Pachet, F. Roy, P. Sakellariou, Generating 1/f noise sequences as a CSP, IJCAI, 2015
![Page 46: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/46.jpg)
The Max Order problem
![Page 47: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/47.jpg)
![Page 48: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/48.jpg)
Max OrderDef: The max order of a generated sequence is the maximum length of replication in the original corpus.
Example: an order 1 Markov sequence with max order 7
![Page 49: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/49.jpg)
The MaxOrder automaton
Villeneuve, D., and Desaulniers, G. 2005. The shortest path problem with forbidden paths. European Journal of Operational Research 165(1):97β107.Aho, A. V., and Corasick, M. J. 1975. Efficient string matching: An aid to bibliographic search. CACM 18(6):333β340.Gilles Pesant: A Regular Language Membership Constraint for Finite Sequences of Variables. CP 2004: 482-495
L πππ₯π πππ₯ = πΏ π β©
π=1, πππππ’π βπππ₯
πΏ π΄ πππππππ
Our contribution:β’ Build this automaton quicklyβ’ Naive approach not polynomial, Polynomial construction algorithm inspired by 1, 2
β’ Automaton is fed to the regular constraint to create ALL length-L sequences
![Page 50: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/50.jpg)
The MaxOrder automaton
Papadopoulos, Roy & Pachet, Avoiding Plagiarism in Markov Sequence Generation, AAAI, 2014
Papadopoulos, Roy & Pachet, Generating non-plagiaristic Markov sequences with max order Sampling, Creativity and Universality in Language, Degli Esposti, Altmann, Pachet Eds, Springer, Morphogenesis series, 2015
![Page 51: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/51.jpg)
Same with Leadsheet generation
Pachet, F. and Roy, P. Imitative Leadsheet Generation with User Constraints, ECAI 2014
![Page 52: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/52.jpg)
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Distribution of chunk size
Order min 1
Order min 2
Order min 3
Order min 4
Order min 3max 10PlagiarismJunk Sweet spot
MIN ORDER = 3 & MAX ORDER = 10
MaxOrder Constraint
![Page 53: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/53.jpg)
Leadsheet Generation
Pachet & Roy, Imitative LeadsheetGeneration with User Constraints, ECAI 2014
LSD: http://lsdb.flow-machines.com
Pachet et al, A Comprehensive Online Database of Machine-Readable Lead Sheets for Jazz Standards, ISMIR 2013
www.flow-machines.com/leadsheetgeneration
![Page 54: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/54.jpg)
Flow Composer
![Page 55: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/55.jpg)
![Page 56: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/56.jpg)
![Page 57: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/57.jpg)
![Page 58: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/58.jpg)
Without production Original (produced)
There is a huge difference between a SCORE and actual MUSIC
Without production Original (produced)
Without production Original (produced)
![Page 59: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/59.jpg)
Music Synchronization (with M. Marchini)
Revisiting Ode to Joy (Beethoven)
From Chi Mai (Ennio Morricone )
From Penny Lane (the Beatles)
From Prayer in C (Lilly Wood & The Prick)
Marchini, Pachet, Roy, Synchronization Constraints: An Audio-content Based Method for Generating Multi-Instrumental Music, submitted Ismir 2015
![Page 60: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/60.jpg)
Brazil guitar style capture
![Page 61: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/61.jpg)
Style Capture in Brazil
![Page 62: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/62.jpg)
Listening to Generation
![Page 63: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/63.jpg)
Generation examples
ReChord generation
Insensatez by Geber Ramalho Blue in Green
Arrastao by Giordano Cabral Canto Triste (Edu Lobo)
Pop chords by Mathieu Ramona Michelle
Friends to go by Benoit CarrΓ© All of me
Original played by various musicians Automatic generation on arbitrary leadsheets
Ramona, Pachet Groove preserving concatenative synthesis, submitted
![Page 64: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/64.jpg)
Benoit CarrΓ© Case StudyFrench Pop Artist
Original version by Benoit CarrΓ©
Version by Enzo Enzo, popularartist in France
Original guitar accompaniment by Benoit CarrΓ©
Michelle
Blue in GreenReChord generation
![Page 65: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/65.jpg)
Markov Constraint applied to textΒ« Yesterday Β» by the Beatles
Barbieri, Pachet, Roy & Degli Esposti, Markov Constraints for Text Generation with Style, ECAI 2012
![Page 66: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/66.jpg)
Yesterday β¦ by Bob Dylan
Innocence of a story I could leave today
When I go down in my hands and pray
She knocked upon it anyway
Paradise in the dark side of love it is a sin
And I am getting weary looking in
Their promises of paradise
Now I want to know you would be spared this day
Wind is blowing in the light in your alleyway
Innocence in the wind it whispers to the day
Out the door but I could leave today
She knocked upon it anywayKhatchatourov, Pachet et al. Action Identity In Style Replication Systems , submitted to Music Perception
![Page 67: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/67.jpg)
Yesterday by the Beach Boys
Anyone to call after night and start today
Back home let watch the long time I pray
Here we surround you anyway
Heavenly skies and rock like that in my mind hey
Walk a summer summer summer day
You realize a deity
At a time for a change of the while I say
Guys I wonder love I have took a runaway
Anyone to me yeah I over heard you say
Moved her so you will love like today
Here we surround you anyway
![Page 68: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/68.jpg)
Yesterday by ACDC
Fortunate if you wanna take you out to playStage I'm big and I want to sayI get enough and I can playSatellite blues yeah yeah yeah yeah yeah you shook meToss off buddy she's gotta seeA fireball in the back onceStage I'm in the high I said it's wayToss off buddy she's got it and I'm heyFortunate if you ain't nothin I can playPlayed all the time you just keep away
![Page 69: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/69.jpg)
Yesterday by Johnny CashTheater of your liquor I don't take pay
Gone in the clouds how to live this way
Hit him again and when my way
Theater of your plans I'm down and tell me
Played the boogie in the saddle he
A wanderer a wandering
Play for you and I know what's the way they
Played the boogie in the harp with the key the way
Honeycomb and live in the water and bread they
Play in the sun and I rode away
![Page 70: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/70.jpg)
An interesting and complex global property: Palindromes
β’ Palindromes are extreme display of human creativity / virtuosity
β’ DOC NOTE: I DISSENT. A FAST NEVER PREVENTS A FATNESS. I DIET ON COD(Peter Hilton: βone full sleepless nightβ)
![Page 71: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/71.jpg)
Palindrome Generation
β’ Goal: construct palindromic sequences from a corpus of N-grams (e.g., google n-grams, text, any sequences, etc.)
β’ Major difficulty:β Two levels are inter-related: LETTERS and WORDS
β’ No algorithm to solve this problemβ Brute forceβ Stochastic, HMM, Monte-Carlo, Metropolisβ Double recursionβ Automataβ Graphsβ Constraints
![Page 72: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/72.jpg)
The Palindrome Graph(with A. Papadopoulos and J.-C. RΓ©gin)
CONJUNCTION graph: Gf x Gb= Tensor product AND encodes the βsame characterβ relation
Papadopoulos, Roy, RΓ©gin, and Pachet, Generating all Possible Palindromes from Ngram Corpora, IJCAI 2015
![Page 73: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/73.jpg)
Palindromes: Examples
β’ βEvil on an oliveβ,
β’ βTo lay a lotβ,
β’ βBorn a man, robβ,
β’ βTill I kill itβ,
β’ βGod all I had, I hid: a hill a dogβ,
β’ βGod, a sin: a man is a dogβ,
β’ βSworn in us at a sun in rowsβ
β’ βDrawn in war, died. I set a gate side, I, drawn in wardβ
β’ βNever a way. By a war evenβ
β’ βEvil as a witness is sent, I was a dog; God as a witness is sent, I was aliveβ
β’ βEt on a lΓ , la baraba, lΓ , la note.β
β’ + very very long ones (80,000 words)
![Page 74: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/74.jpg)
Other Games on Words
β’ Ambiphrases (each direction in a different language)
β El, a Roma se dedica -> Acide de sa morale
β No delay -> Y a le don
β Γme de Roy a le don -> No delay or edema
β’ More to inventβ¦
See interactive demo at Ijcai 2015, Buenos Aires
![Page 75: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/75.jpg)
Conclusion
β’ Style as a computational object: an ingredientfor creativity enhancing tools,
β’ New research problems in sampling fromstatistical models under global constraints
β’ Fruitful combinations of ideas from discretedomain combinatorial optimization, machine-learning and statistical inference
β’ Industrial potential in entertainmenteconomy
![Page 76: Music and Text Generation ^in the style ofΒ Β· 2015-12-21Β Β· πΆ=π1,π2,β¦,π β ππ=ππ 1,π π 2,β¦,π π Generate 1 sequence / the best Generate all](https://reader033.fdocuments.in/reader033/viewer/2022041507/5e25cf27efdde10d3844f8ed/html5/thumbnails/76.jpg)