First species counterpoint generation with VNS and...
Transcript of First species counterpoint generation with VNS and...
First species counterpoint generationwith VNS and vertical viewpoints
D. Herremans1, K. Sorensen1, D. Conklin21University of Antwerp2University of the Basque Country & IKERBASQUE
DMRN+8, London
University of Antwerp
Operations Research Group
ANT/OR
Overview
Generating counterpoint music
Counterpoint
Variable Neigborhood SearchHow it worksExperiments & Results
Machine LearningMarkov Model (Vertical Viewpoints)VNS vs Random Walk & Gibbs SamplingResults
Conclusion
Acknowledgements
I Lrn2Cre8: project funded by the EuropeanCommission Framework Programme 7
I Consortium of 6 partners from 5 countries
I Understand the relation between machine learning andcreativity, applied to music→ Generate new musical structures based on learned models
⇒ New sampling method VNS applied to a controlled musicgeneration problem
Computer aided composition (CAC)
Composing music = combinatorial optimization problem
I Music → combination of notes
I “Good” music → fits a style as well as possible
I Formalized and quantified “rules” of a style → objectivefunction
5th species counterpoint
I Counterpoint & Cantus firmus
4444
I Polyphonic baroque musicI Inspired Bach, Haydn,. . .I One of the most formally defined musical styles→ Rules written by Fux in 1725
Quantifying musical quality using rules
Examples of rules:
I Each large leap should be followed by stepwise motion in theopposite direction
I Half notes should always be consonant on the first beat, unlessthey are suspended and continued stepwise and downward
I All perfect intervals should be approached by contrary oroblique motion
→ 19 vertical and 19 horizontal subscores between 0 and 1
Quantifying musical quality using rules
fcf (s) =
19∑i=0
ai.subscore cfHi (s)︸ ︷︷ ︸horizontal aspect
(1)
fcp(s) =
19∑i=0
ai.subscore cpHi (s)︸ ︷︷ ︸
horizontal aspect
+
19∑j=0
bj .subscoreVj (s)︸ ︷︷ ︸vertical aspect
(2)
f(s) = fcf (s) + fcp(s) (3)
Variable Neigborhood Search
I Local search with 3 neighborhoodsI Selection
I Steepest descentI Based on adaptive score fa(s)
Ni Name Description
Nsw Swap Swap two notesNc1 Change1 Change one noteNc2 Change2 Change two notes
Variable Neigborhood Search
I Excluded framentsI Tabu listI Infeasible
I PerturbationI Change r% of the notes randomly
I Adaptive weights mechanism
I Update best solution sbest, based on original score f(sbest)
LS Swap NH
LS Change1 NH
LS Change2 NH
Current s <
s at A?
Change r% of notes randomly
YesNo
Update s best
Generate random s
A
Max itersreached?
Update adaptive weights
Iters ++
No
Exit
Exit
Optimumfound?
yes
Yes
Machine learning
I Specifying complex objective function by hand↓
I Automatically generate objective function→ Learn from a corpus
I To evaluate this:→ First species: optimal solution known
How does VNS perform compared to Random Walk and GibbsSampling?
1st species counterpoint
44 44
→ Represented as a sequence of dyads[6048
] [6550
] [6452
] [6255
] [6057
] [6455
]. . .
Vertical viewpoints method
I Horizontal & vertical aspects→ linked
I 3 linked features per dyad:I Two pitch class intervals between
the two melodic linesI Pitch class interval within the
dyadI τ(b|a) = [5, 2, 3]
3
5
2
I Dyad sequence transformed in abstract feature sequences(Sufficiently abstract to gather statistics in a corpus)
→ First order Markov model of abstract features
Deriving dyad TM from a viewpoint model
Let v = τ(b|a) be the feature assigned by a viewpoint τ to dyad b,in the context of preceding dyad a
P (b|a) = P (b, v|a) since v is determined by b and a
= P (b, v, a)/P (a)
= P (v)× P (a|v)× P (b|a, v)/P (a) chain rule
= P (v)× P (a, v)/P (v)× P (b|a, v)/P (a)= P (b|a, v)× P (a, v)/P (a)= P (b|a, v)× P (a)× P (v)× Cab/P (a) ass. indep. of a and v
= P (b|a, v)× P (v)× Cab
Quality of a solution
Probability of a sequence with respect to the model:
P (s) =
l∏i=2
P (ei|ei−1)
Cross-entropy (to be minimised):
f(s) = −1
l
l∑i=2
log(P (ei|ei−1))
For all dyads e1, . . . el.
Experimental Setup
I 1000 pieces generated by VNS with rules → training
I Fragment with 64 dyads
I Fixed cantus firmus→ 1161 total combinations
I First dyad fixed to
[6048
]I Last two dyads fixed to
[5950
]and
[6048
]
Experimental setup
I 3 MethodsI VNSI Random WalkI Gibbs Sampling
I Complexity: number of TM lookups
I 10 runs for each method
I Stop criteria: optimum found or 30× 106 TM lookups
I VNS TM lookups = 4 * number of moves (overestimated)
Random Walk
I Start with initial fixed dyad.I Repeat for 1 to l:
I Select next dyad ei with probability p(ei|ei − 1)I If no next dyad with non-zero probability: dead end
I Several iterations
I On each iteration: solution stored if it is the best so far
Gibbs Sampling
I Repeat:I Select a non-fixed dyadI Consider all possible permitted dyads at that positionI Compute the score of each modified pieceI Construct probability distribution over these scoresI Select a new piece based on this distribution
I Several iterations
I On each iteration: solution stored if it is the best so far
VNS vs Random Walk & Gibbs Sampling
0 0.5 1 1.5 2 2.5 3 ·107
200
220
240
260
280
Number of TM lookups
Objectivefunction
RWGSVNS
Optimum
→ VNS: f(s)opt found after an average of 15.8× 106 TM lookups→ GS & RW: optimum not found in any of the iterations
VNS vs Random Walk & Gibbs Sampling
0 0.2 0.4 0.6 0.8 1 ·106
200
220
240
260
280
300
Number of TM lookups
Objectivefunction
RWGSVNS
Optimum
Conclusion
The proposed VNS is a valid and flexible sampling method thatoutperforms both Random Walk and Gibbs Sampling using anobjective function created by machine learning.
Future research:
I Multiple viewpoints
I More complex music, e.g. fifth species counterpoint usingcontrapuntal patterns approach of Conklin & Bergeron (2010).
I Learning on “real” data