LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah...

24
111111111111111111111111111111111111111111111111111111111111 0000043463 LOCATING RECURRING THEMES IN MUSICAL SEQUENCES by Shyamala Doraisamy A dissertation submitted in partial fulfilment of the requirements for the degree of Master of Information Technology Faculty of Information Technology UNIVERSITI MALAYSIA SARAW AK July 1995

Transcript of LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah...

Page 1: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

111111111111111111111111111111111111111111111111111111111111 0000043463

LOCATING RECURRING THEMES IN MUSICAL SEQUENCES

by Shyamala Doraisamy

A dissertation submitted in partial fulfilment of the requirements for the degree

of Master of Information Technology

Faculty of Information Technology UNIVERSITI MALAYSIA SARA W AK

July 1995

Administrator
Page 2: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

DECLARATION

No portion of the work referred to in this dissertation has been submitted in

support of an application for another degree or qualification of this or any other

university or institution of higher learning.

ii

Page 3: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

ACKNOWLEDGEMENTS

I would like to thank my supervisor Professor Dr. Zahran Halim for the guidance

throughout this project.

I would also like to thank my external examiner Professor Dennis Longley and

co-supervisor Dr. Abdelhamid.

I wish to thank Associate Professor Dr. Zaidah Abdul Razak, Mr. K. Narayanan,

Mr. Leng Chee Kong, Ms. Beverly La Rock and Dr. Lloyd Smith. Thanks also to

the staff of the Faculty of Information Technology for all the assistance.

Thanks ,to my friends Yong Huan Loon, Ong Chooi Sim, Sih Yiak May and

Rueben Wee. A special thanks to Wong Pek Wan for the support. Also thanks to

my friends in Sarawak , namely: Alvin, Sita, Shakila, Shela and Hartinah.

Thanks to my family for their love and understanding. Thanks to Ann Perreau

for her support.

Finally thanks to the special person in my life - Suresh Singam.

Shyamala Doraisamy.

iii

Page 4: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

~--

TABLE OF CONTENTS

Title 1

Declaration 11

Acknowledgement 111

Table Of Contents IV

List Of Figures VlV

List Of Tables Xl

Abstrak Xli

Abstract Xlll

CHAPTER 1 INTRODUCTION

1.1 Introduction 1

1.2 Computers And Music 1

1.3 Musical Sequence Comparison 3

1.4 Application Areas For Theme Location 5

1.4.1 Musical Forms Analysis 5 , 1.4.2 Index 6

1.4.3 Musicology 6

1.4.4 Copyright Issues 7

1.5 Research Objectives 7

1.6 Methodology 8

1.7 Scope Of Study 10

1.8 Outline Of Dissertation 11

IV

Page 5: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

CHAPTER 2 SEQUENCE COMPARISON

2.1 Introduction

2.2 Sequence Comparison

2.3 Sequence Comparison In Music

2.4 Sequence Comparison Techniques

2.4.1 Template Matching

2.4.2 Statistical Methods

2.4.3 Structural Methods

2.5 Examples Of Sequence Comparison Techniques

2.5.1 Matching (Template Matching)

2.5.1.1 Exact Matching

2.5.1.2 Approximate Matching

2.5.2 Cluster Analysis And Decision Method

(Statistical Approach)

2.5.3 Rhythmic Patterns In Musical Lines

(Structural Methods)

2.5.3.1 Exact Matching

2.5.3.2 Rhythmic Elaboration

2.6 Summary

13

13

14

19

19

20

22

24

24

24

24

32

36

37

37

39

CHAPTER 3 COMPUTER-ASSISTED MUSIC ANALYSIS

3.1 Introduction 40

3.2 Survey Of Comparison Methods 40

3.2.1 Template Matching 41

3.2.1.1 Automated Identification Of Melodic Variants In

Folk Music - Martin Dillon & Michael Hunter 41

3.2.1.2 A Computer-Assisted Approach To Micro-Analysis

Of Melodic Lines - David A.Stech 41

3.2.1.3 Comparison Of Musical Sequences-

Marcel Mongeau & David Sankof'f 42

3.2.2 Statistical Methods 44

Page 6: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

3.2.2.1 Cluster Analysis For The Computer-

Assisted Statistical Analysis Of Melodies - Luigi

Logrippo & Bernard Stepien 44

3.2.2.2 Minimum Message Length Comparison Of

Musical Sequences - Shane Samuel Cook 44

3.2.3 Structural & Syntactic Methods 46

3.2.3.1 On Finding Rhythmic Patterns In Musical Lines -

Bernard Mont-Reynaud & Mark Goldstein 46

3.3 Rationale For Adopting Mongeau & Sankoffs Method 47

3.4 Music Data Representation 50

3.4.1 Overview Of Repesentation Schemes

3.4.1.1 DARMS

3.4.1.2 MIDI Notelist

3.4.1.3 Digital-tradition

3.4.1.4 Humdrum

3.4.2 Review Of Representation Schemes

3.5 Conclusion

CHAPTER 4 PERFORMANCE ASSESSMENT

4.1 Introduction

4.2 Algorithm By Mongeau & Sankoff (1990)

4.3 Encoding & Parameters

4.4 Matching Experiments

4.4.1 Matching Of Sequences An Octave Apart

4.4.2 Matching Of Sequence A 5th Higher

4.4.3 Matching Of Sequence A 4th Lower

4.4.4 Tests For Passing Notes

4.4.5 Tests For Key Change

4.4.6 Tests For Rhythmic Change - Augmentation

4.4.7 Tests For Rhythmic Change - Diminution

4.5 Experiment Results

4.5.1 Sequences An Octave Apart

50

50

50

51

51

51

52

53

53

55

59

61

63

64

65

66

67

68

68

68

Page 7: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

4.5.2 Sequences A 5th Higher 68

4.5.3 Sequences A 4th Lower 68

4.5.4 Sequences With A Passing Note 68

4.5.5 Sequences With A Key Change 70

4.5.6 Sequences With Rhythmic Change - Augmentation 70

4.5.7 Sequences With Rhythmic Change - Diminution 70

4.6 Conclusion 71

CHAPTER 5 SYSTEM OVERVIEW & IMPLEMENTATION

5.1

5.2

5.3

5.4

5.5

Introduction

Theme Locater System

5.2.1 Approach

5.2.2 Requirements Of The System

Humdrum Format

Processes In TLS

5.4.1 Extraction

5.4.2 Conversion

5.4.3 Comparison

5.4.4 Analysis

Results

CHAPTER 6 ENHANCEMENT & APPLICATION

6.1

6.2

6.3

6.4

6.5

Introduction

Strengths And Weaknesses

Enhancement

6.3.1 Weight Change

6.3.2 Melodic Contour

6.3.3 Rhythmic Contour

Matching experiments

Experiment Results

6.5.1 Exact Pitch Contour

6.5.2 Rhythm Contour

72

72

73

74

74

75

77

79

83

83

84

87

87

88

88

88

91

93

97

97

97

Page 8: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

6.5.3 Passing Note

6.6 Application On Fugue

6.7 Results

6.8 Conclusion

CHAPTER 7 CONCLUSION & FUTURE WORKS

7.1 Introduction

7.2 Findings

7.2.1 Dynamic Programming

7.2.2 Data Encoding

7.2.3 Music Data Representation

7.2.4 Weight Tables ,,!

7.2.5 Pattern Recognition

7.3 Accomplishments

7.4 Limitations

7.4.1 Data Set

7.4.2 Variations

7.4.3 Threshold

7.5 Future Works

7.5.1 Variation

7.5.2 Dimensions For Comparison

7.5.3 Result Reporting Method

7.5.4 User Interface

7.5.5 Experiments

7.5.6 Other Possible Works

7.6 Concluding Remarks

BIBLIOGRAPHY

GLOSSARY

APPENDIX A - SCORES

APPENDIX B - PROGRAM LISTING

97

101

102

106

107

107

107

107

108

108

108

109

110

110

110

110

111

111

111

111

112

112

112

113

Page 9: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

LIST OF FIGURES

2.1 Modification with a passing note 16

2.2 Modification based on transposition 17

2.3 Modification based on augmentationldimunition 18

2.4 Pattern recognition using template matching 19

2.5 Pattern recognition using a statistical pattern recogniser 21

2.6 Examples of structural pattern representation 22

2.7 Pattern recognition JIBing a structural recogniser 23

2.8 Trace linking two sequences 24

2.9 Illustration of the principle of optimality 27

2.10 Dynamic programming array 28

2.11 Dynamic programming example 31

2.12 Optimal alignment with a dissimilarity measure of 2 32

2.13 Visualisation in 2-dimensional space 34

4.1 Alignment for sequences an octave apart 61

4.2 Alignment for sequence a 5th higher 62

4.3 Alignment for sequence a 4th lower 63

4.4 Alignment for sequence with a passing note 64 \

4.5 Alignment for sequence with key change 65

4.6 Alignment for sequence with rhythmic change - augmentation 66

4.7 Alignment for sequence with rhythmic change - dimunition 67

5.1 Theme locater system '76

5.2 A portion of Fugue I, Bk I in Humdrum format 78

5.3 A portion of Fugue I, Bk I after extraction 80

5.4 Output from extraction of Fugue I, Bk I 81

5.5 Data encoding 82

5.6 Dissimilarity array 85

5.7 Results of analysis 86

6.1 Weight change 89

6.2 Two significantly different sequences 92

6.3 Pitch versus duration graph 94

viv

Page 10: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

6.4 Modifications 96

6.5 Exact pitch contour for transposition 98

6.6 Exact rhythmic contour for augmentation and dimunition 99

6.7 To detect passing notes 100

6.8 Theme locater system - modified 101

6.9 Data encoding with contours 103

6.10 Dissimilarity array after change 104

6.11 Locations of theme~ 105

Page 11: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

LIST OF TABLES

2.1 Example of note occurrence frequency 33

2.2 Distance table example 34

4.1 Weight assignment base~n difference in degrees 57

4.2 Weight assignment based on difference in semitones 58

4.3 Dynamic programming match 69

xi

Page 12: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

ABSTRAK Penggunaan komputer pada masa kini dalam hampir semua subdisiplin­

subdisiplin muzik semakin bertambah. Otomasi penggubahan lagu, sistem

pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang

muzik bagi penggunaan komputer.

Kerja yang dilapurkan di sini mengkaji kaedah-kaedah yang diguna dalam

pembezaan dua jujukan muzik (musical sequence). Dalam pembezaan dua

jujukan muzik, apa yang perlu dipertimbangkan ialah sujektiviti yang terlibat

dan penganalisaan yang diperlukan. Beberapa algoritma dalam pembezaan

jujukan muzik di mana teknik-teknik yang diguna dikaji berdasarkan teknik­

teknik pengcaman corak (pattern recognition). Sistem-sistem pengwakilan notasi

muzik bagi komputer yang berlainan juga dikaji supaya muzik data boleh

diubahsuai dalam bentuk yang boleh dibaca oleh mesin.

Tujuan mengkaji algoritma-algoritma pembezaan jujukan muzik ialah untuk

memperolehi satu kaedah untuk menglokasi pengulangan tema utama dalam

sesuatu gubahan lagu walaupun tema tersebut diubahsuai oleh penggubah

untuk menjadikan pengulangan lebih menarik. Manusia boleh mengcam

pengulangan tema walaupun diubahsuai dengan senang tetapi jika kaedah

komputasi diguna, algoritma akan diperlukan.

Kaedah oleh Marcel Mongeau & David Sanko!! (1990) diguna sebagai algoritma

pembezaan jujukan muzik. Ujikaji-ujikaji dilakukan untuk menguji kebolehan

penggunaan algoritma ini untuk penglokasian tema. Satu sistem Penglokasian

Pengulangan Tema dibangunkan di mana kaedah terse but diguna untuk

mencari subjek fugue' yang berulang dalam dalam (suara-suara' lain dalam

sesutu fugue' Bach yang tertentu. Kelebihan-kelebihan dan kelemahan­

kelemahan kaedah terse but dikaji. Perubahan dicadang dan diimplimentasikan

di mana corak melodi dan ritma diguna sebagai ciri pembezaan untuk

membolehkan subjek fugue' dikesan walaupun subjek tersebut diubahsuai

dengan penambahan nota laluan, (transposisi' melodi dan ogmentasi dan

dimunasi ritma.

Page 13: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

ABSTRACT

Computers are now being used in almost all (sub)disciplines of music.

Automated composition, music theory expert systems, architectural acoustics are

just a few of these many areas.

This thesis looks into methods being used in comparing two musical sequences.

In comparing two musical sequences, the subjectivity and the reasoning required

for comparison needs to be considered. A few algorithms in musical sequence

companson and the different techniques used were studied based on pattern

recognition techniques. Apart from that, different musical representation

systems were also looked into for encoding music notation into machine-readable

form.

The purpose of surveying musical sequence comparison algorithms was to be

able to find a method to reliably locate recurring themes in a given score as the

theme can be varied by the composer to add variety to the composition. Humans

might have little difficulty identifying the theme despite it being varied whereas

algorithms will be needed if computational methods are to be used in locating

recurring themes.

The method by Marcel Mongeau & David Sankoff (1990) was selected as the

comparison algorithm. Experiments were done to test its usability in locating

recurring themes. A Theme Locater System was developed and this comparison

method was used in finding the recurring fugue subject in the different voices in

a particular Bach's fugue. Its strengths and weaknesses were studied.

Enhancements where melodic and rhythmic patterns were used as the basis for

comparison to be able to detect the fugue subject even when some basic

modifications such addition of passing notes, transposition of the melody and

augmentation and diminution of the rhythm had transformed this fugue subject.

XUl

Page 14: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

1 INTRODUCTION

1.1 Introduction

This chapter first gives a discussion of combining two contrasting fields of study:

computers and music. Next, it presents the significance of the task undertaken

for this study in comparing musical sequences to locate recurring themes. Then

it discusses why this study was undertaken by presenting the application areas

for locating recurring themes. Following that, the objectives of this study are

given before the methodology taken to achieve the objective stated is described.

The scope of the study is then outlined. Finally, an outline of this dissertation is

presented.

1.2 Computers And Music

Music intelligence is research into methods of making the computer more

cognizant of and powerful over the manipulation and organisation of musical

materials, structures and systems.

- Phil Winsor and Gene DeLisa, 1991

Computers are used in almost every field now and music is no exception. One

might ask how can computers contribute to the musical arts which involve

creative thinking?

Real creative thinking takes practice and involves carefully following

principles and effective techniques. [JF90]

By analysing these principles and techniques, computational methods can be

applied to almost every domain in music.

Page 15: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

For mUSIC researchers pursumg systematic investigations of musical

information, systems in posing and answering research questions are being

developed. This includes the posing of factual questions of use to music theorists,

musIc analysts, ethnomusicologists, historical musicologists,

psychomusicologists, music librarians and others.

Apart from the use of computers in systematic investigations of precomposed

music, computers are also being used extensively in music performance

(generating the sounds, including timbre, dynamics, synthesis/simulation of

orchestral and new instruments, etc.) and in music creation (generation of

rhythm, melody, harmony, orchestration, etc.). Music performed or created this

way is referred to as computer music.

Previously, both sound production and compositional processes were carried out

with the mainframe computer in a single environment. The scenario today is

that the personal computer often fills the role of composer/performer while the

synthesizer acts as the instrument of performance, in the same way that people

have traditionally played acoustic instruments. The development of MIDI

(Musical Instrument Digital Interface), is a primary reason for this advancement

where MIDI IS a communications protocol that allows electronic musical

instruments to interact with each other and with a general-purpose

computer[CA86]. With this, electronic musicians are now creating new and

different sounds.

Researchers in every music (sub)discipline are finding ways to use the computer

to help solve musical problems. Some of the major research areas are:[WD91]

Algorithmic and Automated Composition, Music Theory Expert Systems,

Computer-Aided Instruction / Music Education, Musicology & Ethnomusicology,

Performance, Psychoacoustic Phenomena (Cognition), Physics of Sound,

Architectural Acoustics, Performance Environment Simulation, Ecological (sonic

environment) Acoustics, Esthetic Perception Theory, General System Theory :

Information Theory, Intermedia Explorations (multidisciplinary events

comprised of multiple information media, such as choreography, video, film,

laser imagery, light sculpture, and music), Music representation systems,

2

Page 16: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

Information retrieval systems for musical scores, Music Printing and Music

Analysis.

The focus of this thesis is in the usage of computers in music research where

methods of comparing two musical sequences encoded in machine readable

format is studied.

1.3 Musical Sequence Comparison

The purpose of this study is to identify a method to locate recurring themes in a

given music score. The theme here is referring to the main melody or musical

idea that forms the basis of the composition. Composers usually repeat this

theme throughout the composition and upon repetition, this theme is usually

varied or modified to add variety to their composition. Therefore, a method is

required to identify these reoccurrences despite it being varied.

The method needed is a musical sequence comparison method where two musical

sequences encoded in machine-readable format is compared. Different musical

sequence comparison methods are studied using pattern recognition techniques

as a basis for the study since musical sequence comparison can be viewed as an

application area of pattern recognition.

Sequence comparIson IS sometimes referred to as approximate string

matching[Sl-94]. Sequences are often different lengths. In speech, for example,

the length of a word or syllable changes with context and speaker.

In music, the complexity of sequence comparison is best described by [Stech81]:

Analysing and defining the factors affecting the succession of tonal events

ut a musical composition is a complex task. It requires evaluating particular

musical events within the context of the prevailing musical style and abstracting

those elements which appear to affect significantly the aural logic of the

composition. Through such a method, the analyst hopes to define more clearly, in

his own mind, how successfully the composer develops his musical material

without violating the composer's own sense of musical logic of order.

3

Page 17: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

The difficulty lies in locating a recurring theme as the theme might have been

modified. Humans, normally, will find little difficulty recognising slightly

modified or decorated themes. Computers, on the other hand, do very well with

exact matching but struggles somewhat (i.e., it is difficult to define 'good'

algorithms) with approximate matching. Central to the problem is our ability (or

inability) to define 'similarity' - when are two sequences similar enough to be

considered variations of the same theme? There is an element of subjectivity in

such judgements which is difficult to capture algorithmically.

Apart from that, when the musical score being analysed is "full of notes", the

naked eye sometimes misses the recurrence of a melody as it is "hidden"

somewhere in the web of the notes. For large works, it can be tedious locating

the recurrences especially when modifications have happened. It is here that if

"sufficiently good" sequence comparison algorithms exist, computers can perhaps

do better - computers do not get tired or bored of tedious tasks.

Up till now, all these analysis has been based on experience and knowledge of

the listener and music has been looked upon as an art where it has been

impossible to apply computational methods. With systematic study as much as

possible of the processes possible in compositional works, it can move towards

automating analysis of music. Computers are now being introduced in assisting

or even acting as the expert in analysing this creative process. The need for

development of techniques which utilise melodic pattern analysis is necessary

towards this change.

The method used in this thesis to locate recurring themes can serve as a basis

for the future work in locating recurring themes or incorporated as modules for

automated musical forms analysis, copyright issues, recognising themes which

can prove to be an index for looking up a theme dictionary or even in expert

systems for some of the subdisciplines of music which requires comparison of

musical sequences.

4

Page 18: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

1.4 Application Areas For Theme Location

The main purpose of studying methods in musical sequence comparison in this

thesis is in locating recurring themes. In this section, some of the areas in music

where locating recurring themes can be useful are discussed.

1.4.1 Musical Forms Analysis

Musical forms analysis is an important aspect of performing, listening and

studying music. Every piece of music, from the simplest song to the most

elaborate symphony, needs to have some kind of organisation, or form, if it is to

be coherent to the listener. [SS93]

Western music incorporates some aspects of departure and return. A departure

suggests a musical idea that is different from that which has already been

presented - a point of contrast. A return is either an exact or modified

repetition of the original idea. Patterns of departures and returns, contrasts and

repetitions and key relationships are determining factors in establishing the

form of a piece of music. To identify these contrasts between departure and

returns, the different elements of music are used as the basis for comparison.

The basic elements in music are pitch, rhythm (duration), intensity (dynamics)

and timbre ( tone quality - a flute sounds different from a clarinet ).

Whether listening to music or performing a piece, analysing and understanding

the form of the music is important. This process is called musical forms analysis

and is an important activity for those doing musical studies, research,

performance and so on. Musical analysis results in a higher level of musical

understanding, and this in turn leads to improved music performance and fuller

music appreciation.

The listening process is a communication of compositional ideas between the

composer and the listener. Awareness of musical patterns and musical forms is

an important aspect in this regard. The performer on the other hand has the

duty of presenting these ideas to the listener. Thus, understanding the form is

just as important if not more so. For example, the player might need to end the

first theme softly and begin the second theme louder in order to make it clear to

5

Page 19: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

the listener that the second theme is being introduced. This process is called

musical interpretation. By knowing form, one will know where to expect

changes and the likely type of change.

The first step one might take in analysing musical forms is to look at musical

sequences. Intuitively, a phrase will be characterised by its sequence of notes, in

both pitch and rhythm. The recognition of a recurring sequence and its location

would be a starting point in determining the form. Sequence comparison is

clearly central to the detection of recurring themes.

1.4.2 Index

For a musical theme dictionary or a musical score database to be constructed, an

index will be needed for retrieval. Another application of musical sequence

comparison is the retrieval of a musical work based on themes acting as indexes

like in Barlow and Morgenstern's Dictionary of Musical Themes or a score

database. A sequence that is found to be repeating very frequently might be as

the theme which is then extracted and used as a search index.

1.4.3 Musicology

Research into melodic pattern relationships provides insights into the mental

processes of both the listener and the composer [Stech81]. Musicologists

analysing particular compositional styles of a composer or works of a particular

period will need to analyse the recurrences of musical sequences in the scores

being studied. This does not necessarily mean that most composers consciously

attempt to construct their thematic or rhythmic variants, although such an

approach may have been used by a few composers.

Musicologists and anthropologists study folk songs as an indicator of population

migration. [Sl-94] This is done principally through comparing and detecting

similarities in melody lines of songs from different regions. Methods to compare

musical sequences are therefore of direct value in such efforts.

Ethnomusicology is another area of application. Ethnomusicology includes the

study of folk music, Eastern art music and contemporary music in oral tradition

6

Page 20: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

as well as conceptual issues such as the origins of music, musical change, music

as a symbol, universals of music, the function of music in society, the comparison

of musical systems and the biological basis of music and dance. [HM92].

Ethnomusicology formerly known as comparative musicology has been described

as:

"Comparative musicology has as its task the comparison of the musical

works - especially the folksongs - of the various peoples of the earth for

ethnographical purposes, and the classification of them according to their various

forms. "[HM92]

Musical sequence comparison methods to locate themes reoccurring can be used

in helping in this classification processes.

1.4.4 Copyright Issues

In automating the process of detecting if melodies have been copied, musical

sequence comparison forms the basis. Maybe one way is setting a limit to the

degree of variance allowed - anything below that figure is considered to have

been copied.

1.5 Research Objectives

The main aim is to be able to find a method in locating repeating themes in a

given music score. In achieving this aim, the objectives set are:

• to identify a sequence comparison method which can be used to locate a

recurring theme.

• to identify a music data representation system which can act as input for

music scores in machine-readable format.

• to design experiments that would test the usability of the algorithm m

identifying recurring themes.

• to develop a system that would locate a recurring theme automatically from a

given music score.

7

Page 21: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

1.6 Methodology

The methodology used for this study to achieve the aim of developing a system

which can locate a recurring theme in a given score is described below.

• Review sequence comparison techniques

To study sequence comparison in the context of music based on the different

categories of pattern recognition techniques. This will be the first step in trying

to identify a method to compare musical sequences for locating recurring themes.

What is to be obtained is an overview of the different categories for a

comparative study of the different techniques to be done before a decision can be

made as to the choice of the algorithm required for the study.

• Review studies done on computer-assisted music analysis

To look at works done in the different categories of sequence comparison with

the aim of obtaining a method that can be used as the comparison method

required to locate recurring themes. The works reviewed are:

- "A computer-assisted approach to micro-analysis of melodic lines", David

A.Stech (1981)

- "Automated identification of melodic variants in folk music" by Martin

Dillon & Michael Hunter (1982)

- "On Finding Rhythmic Patterns in Musical Lines" ,Bernard Mont­

Reynaud& Mark Goldstein (1985)

- "Cluster analysis for the computer-assisted statistical analysis of

melodies", Luigi Logrippo & Bernard Stepien (1986)

- "Comparison of musical sequences" by Marcel Mongeau & David Sankoff

(1990)

- "Minimum Message Length comparison of musical sequences" -Shane

Samuel Cook (1994)

• Comparative study

An algorithm is to be chosen based on a comparative analysis of the different

categories of sequence comparison and the different techniques used in the

works surveyed. One of these methods would be used as the comparison method

in this study to locate recurring themes.

8

Page 22: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

• Review music data representations

Different music data representation methods used to encode mUSIC data into

machine readable format is surveyed with the aim of obtaining and using one of

these different methods. A study of the format used must be done.

• Experiment design and implementation

The algorithm to be used IS to be coded and tested on experiments set.

Experiments are designed to test its usability in comparing two musical

sequences where the sequence has been varied with some basic modifications.

Assessment is based on whether it compares two musical sequences that had

been varied with some frequently used techniques of varying a melody as similar

just as how a human might consider it as two musically similar sequences. An

assessment of the results and an analysis of strengths and weaknesses in

locating recurring themes would be done.

• Theme locater system

To locate recurring themes, a theme locater system needs to be implemented.

The main feature of the system would be that the theme and its recurrences

would be detected without any human intervention. Given a score, its theme

and its recurrences are to be identified. Bach's 48 Preludes and Fugues are the

scores studied where its subject and its recurrences in the different voices are

located. Data needs to be converted to the input format necessary for this

system. The theme and sequences from the score need to be extracted for

comparison. The algorithm tested is to be incorporated into the system as the

comparison method for comparing the theme of the score with sequences

extracted from the score to see if it is able to detect recurring themes

benchmarked on human analysis.

• Algorithm enhancements

Enhancements are to be suggested based on successes and failures of the

experiments and its performance in the Theme locater system.

9

Page 23: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

• Test of enhanced algorithms

The enhanced algorithm is tested to see if the weaknesses encountered earlier

are overcome and the enhancement is incorporated into the Theme locater

system.

1.7 Scope Of Study

• Elements in music

The different elements of music which are pitch, duration, timbre and dynamics

can act as a basis for comparison. For this thesis, only two dimensions are

considered which are pitch and duration. Timbre might need to be considered if

an analysis of orchestral scores are done where musical sequences might be

repeated by the different instruments or in the study of acoustics where musical

sequences based on sounds produced by different instruments are analysed.

Dynamics can be considered if more detailed study of a particular work where

repetition of a musical sequence with different dynamic level needs to be

considered. But for this study, we assume that pitch and duration is sufficient

basis for comparison of two musical sequences.

• Melody line

For this thesis, the sequences compared are a single melodic line, i.e. sequences

of chords or harmony are not considered.

• Variations

There are many types of variations or modifications that can happen to a theme

every time it recurs. Sequences comparison methods are applied to detect the

recurrence of a theme despite these modifications. In this thesis, the variations

of the melody that are taken into account are transpositions and changes where

passing notes are added, and for variations of rhythms that are looked into are

augmentation and diminution. Other modifications such as inversion (notes are

inverted), retrograde(upside-down) and so on are not looked into.

10

Page 24: LOCATING RECURRING THEMES IN MUSICAL SEQUENCES · pakar teori muzik, (acoustic' senibina adalah beberapa dari pelabgai bidang muzik bagi penggunaan komputer. Kerja yang dilapurkan

• Algorithm and data

Only one algorithm was chosen for detailed implementation and testing out of

several reviewed due to the lack of time. The algorithm was chosen based on the

study done$1 different comparison methods. Music data from one database only

was used also due to a lack of time to obtain and convert data from different

databases in different music data representation formats.

1.8 Outline Of Dissertation

This thesis comprises 7 chapters:

CHAPTER 1 gives an introduction to the area of research which is the usage of

computers in the music field, the scope of study, the significance of the task

undertaken and the research objectives.

CHAPTER 2 surveys techniques used in musical sequence comparison.

CHAPTER 3 presents works done on computer-assisted musical analysis where

techniques in musical sequence comparison are used. It also presents some of

the music data representations that can be used before reasons for adopting the \

algorithm by [M&S90] is given.

CHAPTER 4 provides details of experiments for testing if sequences are

detected as similar even with certain modifications done to the melody. Details

on the implementation of the algorithm, data used and the results obtained are

also given.

CHAPTER 5 provides details of the theme locater system where the algorithm

tested is incorporated into the system as the sequence algorithm needed for

locating recurring themes. The results of its performance in the system is

discussed.

CHAPTER 6 provides a discussion of strengths and weaknesses based on

experiments and analysis. Enhancement suggestions and results are given.

11