Chapter 22 - Contemporary Performing Arts Testbed

7/31/2019 Chapter 22 - Contemporary Performing Arts Testbed

1/22

Chapter 22

Contemporary Performing Arts Testbed

22.1 Historical Introduction to the Issue

Since the 1970s, the field of performance arts has quickly evolved thanks to the

development of, and innovation in computers, software and electronic devices that

have transformed stage practices. Whereas performers used hardware devices for all

signal processing required on stage, they progressively moved to software environ-

ments enabling them to develop personal interactive modules. This initially applied

to music, but quickly expanded to dance, theatre and installations.

22.1.1 The 1950s: The Pioneers

The idea of using computers in order to generate music emerged in the 50s, but

was mainly reserved to laboratories. Max Mathews, a pioneer in the domain, writes

for instance: Computer performance of music was born in 1957 when an IBM

704 in NYC played a 17s composition on the Music I program which I wrote.

The timbres and notes were not inspiring, but the technical breakthrough is still

reverberating.

22.1.2 The 1970s: The Popularization

One major step was the invention of sound synthesis by using frequency modu-

lation in the 1970s. This invention, patented in 1975, was discovered at Stanford

University by John Chowning, another pioneer. The technique which consists in

applying a frequency modulation in the audio range to a waveform also in the

audio range, results in complex sounds that cannot be generated by other means,

such as boings, clang, twang and other complex sounds that everyone can

now easily recognize as sounds from the 1970s or 1980s. This invention was

the basis of some of the early generation digital synthesizers like the famous

Yamaha DX7.

407D. Giaretta, Advanced Digital Preservation, DOI 10.1007/978-3-642-16809-3_22,C Springer-Verlag Berlin Heidelberg 2011


2/22

408 22 Contemporary Performing Arts Testbed

22.1.3 The 1980s and Later: Mixed Music and Extension

to Other Domains

The first interactive works combining performers and real-time electronic modu-

lation of their parts have appeared in the middle of the 1980s. Electronic devices,

either hardware or software, have been used with various musical configurations:

the instrument-computer duet, for instance in Philippe Manourys works (Jupiter,

for flute and computer, 19871992 ; En Echo, for voice and computer, 19931994);

the works for ensemble and live electronics, such as Fragment de lune (19851987)

by Philippe Hurel; the works for soloists, ensemble and electronics, such as Rpons

(19811988) by Pierre Boulez.

Various digital techniques have been developed since then: from the use of var-

ious forms of sound synthesis (additive, granular, by physical modelling. . .), to the

real-time analysis and recognition of inputs (audio as well as video input), based onartificial intelligence (neural networks, hidden Markov models. . .), passing through

various forms of distortion of the input sounds (reverberation, harmonization,

filtering. . .).

Meanwhile, these techniques penetrated into some neighbouring domains, such

as opera with live sound transformations (K, music and text by Philippe Manoury,

2001), theatre with live sound transformations (Le Privilge des Chemins, by

Fernando Pessoa, stage direction by Eric Gnovse, sound transformations by

Romain Kronenberg, 2004), theatre with image generation (La traverse de la nuit,

by Genevive de Gaulle-Anthonioz, stage direction by Christine Zeppenfeld, real-time neural networks and multi-agent systems by Alain Bonardi, 2003), music

and video performances (Sensors Sonic Sights), music/gestures/images with Atau

Tanaka, Laurent Dailleau and Ccile Babiole), or installations (Elle et la voix, vir-

tual reality installation by Catherine Ikam and Louis-Franois Flri, music by Pierre

Charvet, 2000).

22.1.4 And Now. . .

After nearly 25 years of interactive works, institutions have become aware that this

type of music is completely dependent on its hardware and software implemen-

tations. Should the operating system or the processor evolve, the work cannot be

performed again. This is for instance what nearly happened to Diadmes, a work

by composer Marc-Andr Dalbavie for alto solo, ensemble and electronics. First

created in 1986 and honoured by the Ars Electronica Prize, the work was last per-

formed in 1992. In December 2008, its American creation was planned, more than

22 years after its premiere in France. But the Yamaha TX 816 FM synthesizers pre-

viously used are no longer available, and the one still present at IRCAM is nearly

out of commission. Moreover, composer Dalbavie has tried several software emu-

lators, but none of them was suitable according to him to replace the old hardware

synthesizer.


3/22

22.2 An Insight into Objects 409

In April 2008, Dalbavie and his musical assistant Serge Lemouton decided to

choose another technique: they built a sampler. It is a kind of database of sounds

produced by an instrument. The sounds have been recorded from the old TX 816 at

various pitches and intensities. This solution enabled to re-perform the piece by

means of a kind of photography of the previous sounds. When no sound corre-sponding to a given pitch exists, the sampler is able to interpolate between existing

files in order to give the illusion that the missing note exists.

22.1.5 What Is to Be Preserved?

As for music from the past centuries, we need to preserve the ability to re-perform

the works, and not simply to preserve the outputs audio or video recordings

even if these recordings are clearly part of the objects to be preserved. This implies

a careful analysis of the objects that have to be preserved, including the objects that

are at the core of the digital part of the work: the process. The context of these

objects is also to be preserved, from its various dependencies hardware and soft-

ware to the knowledge that is needed in order to install it and run it correctly. The

amount of dependencies of the process is immense, from the hardware platform on

which it runs, to the almost uncontrollable amount of libraries used by the multiple

layers of software: from the underlying operating system with its device controllers,

to the libraries included in the course of the software development process. One

quickly understands that the maintenance activity needed in order to be able to re-perform a work is a never ending activity that should moreover respect a minimum

of authenticity.

Authenticity in this context means that, despite the various migrations, emula-

tions and other transformations that have to be applied to objects, one has also to

maintain the information needed in order to allow future actors to answer the very

important question: DOES IT SOUND LIKE IT WAS INTENDED TO?. This is

not the most straightforward task, since any judgment on authenticity in this context

incorporates a certain amount of fuzziness. . .

22.2 An Insight into Objects

22.2.1 Complexity

Complexity seems to be inherent to musical creation, at least in the Occidental

approach. This is not the right place to expose a musicological treatise on this impor-

tant question, but on can refer to many examples from the past, from Jean Sebastian

Bach to Mahler, Wagner, Stravinsky. . .

. Moreover, some musicologists have shown

that even in songs from Africa there is a hidden complexity, see for instance in the

CD-ROM by Simha Arom Pygmes Aka. Peuple et musique. One can also refer to

Claude Levi-Strauss for an approach of the importance of complexity in cultures. . .


4/22


Fig. 22.1 A complex patch by Olivier Pasquet, musical assistant at IRCAM

Modern music does not escape from this law, and works produced by modern

composers are very often judged as too complex for our ears. In the technical part,

complexity is inevitable, as shown in Fig. 22.1.

One can think of the hidden part of a modern piano for comparison.

22.2.2 Obsolescence and Risk

In our domain, obsolescence is very rapid, and this is a quite new experience for

musicians. We are accustomed to objects that last for centuries (scores, musical

instruments), and we also benefit from structures that are dedicated to the transmis-

sion of knowledge about music (conservatories, music schools, treatises, schools for

musical instruments manufacturing. . .). This allows for the long-term preservation

of musical works in the future, even in the case where the original instrument which

were to be used when a piece was composed have disappeared, such as Schuberts

Sonata for Arpeggione, or Mozarts works for Glass Harmonica. In this case, the

knowledge makes it possible to find a similar instrument, to adapt the musical score


5/22

22.3 Challenges of Preservation 411

or the techniques for the sake of a new performance, assuming a certain degree of

authenticity. . .

But for digital music, things are not so well organized. The dependency on indus-

try is very high, knowledge is not maintained, obsolescence of systems and software

is very rapid, thereby increasing the risk.Consider for example Emmanuel Nunes Lichtung II. This work was first cre-

ated in 1996 in Paris, using a NeXT workstation extended with a hardware DSP

platform developed by IRCAM. The work was then recreated in 2000, using then

a Silicon Graphics workstation with a specific piece of software (jMax). The work

was recreated again in Lisbon in 2008, using a Macintosh with the Max/MSP soft-

ware. For each of the subsequent re-performances, a porting of the original process

was needed, and was implemented by the original engineer who had developed the

first version.

This is only one example, but the amount of works that were originally created forthe NeXT/ISPW workstation is huge, and for all of them, the cycle of obsolescence

is similar.

The risk is then to completely lose some works: for lost compositions from the

past, the musical score is sufficient for any new performance. For the digital part,

the score does not give any information.

22.3 Challenges of Preservation

The most important challenge for preservation in performing arts is to be able tore-perform the works. It is not sufficient to preserve the recordings (audio or video),

but to preserve all the objects that allow for a new live performance of the work.

This implies not only the preservation of all objects that are part of a work, but also

preservation of the whole set of logical relationship between these elements.

Objects that have to be preserved are:

data objects (midi files, audio files)

processes (real-time processes)

documentation (images, text)

recordings (audio, video)

Data objects and processes are used during the live performance, while documen-

tation and recordings are used when preparing a new live performance. Recordings

can be considered to be a specific kind of documentation which aims at providing

a descriptive documentation of the work (how it sounds like. . .), while documen-

tation as images and text aim to provide a prescriptive documentation on the work

(what has to be done). As explained above, recordings are part of the objects to be

preserved, but are not sufficient to preserve the ability to re-perform the work.

There can be several objects that are used inside a work. For example, there

can be several hundreds of different data objects (audio files, midi files. . .), several

real-time processes (for instance, more than 50 in En Echo by Philippe Manoury),


6/22


several different documentation files (one for speakers installation, one for micro-

phones, one for general setup and installation. . .). All these documentation files can

be grouped together in a single PDF file.

22.3.1 Challenge 1: Preserving the Whole Set of Objects, with Its

Logical Meaning

The first important challenge is to preserve the logical relationship between all these

objects. That logical relationship is one of the most important part of the challenge,

since the preservation strategy applicable to each element can be dependent on the

logical relationship an element has with the whole set.

An example of this is a set of audio files used to store parameters. During our

analysis of the content of the repository, some problems occurred, one of the mostglaring being the case of audio files used to store numerical parameters: instead of

storing audio, some audio files are used to store numerical parameters that serve

as input to real-time processes in order to change their behaviour. This behaviour

seems to be very frequent, since audio files are well known to the community, and

their use is thus facilitated.

Due to this fact, migration techniques that are applicable to audio files cannot

be applied to these specific parameter files (the reason of this is very evident to any

member of the community). Thus, the logical relationship to the whole set of objects

has to be maintained in order to achieve preservation.

22.3.2 Challenge 2: Preserving the Processes, and Achieving

Authenticity Throughout Migrations

As explained in detail below, the second important challenge is to preserve the real-

time process (the so-called patch). The obsolescence of the environments able

to execute the real-time process is so rapid that processes need to be migrated

approximately every 5 years. Moreover, there is a need to achieve a certain

form of authenticity throughout the successive migrations that cannot be evi-dently based on simple provenance and Fixity Information, as defined by the OAIS

model.

22.4 Preserving the Real-Time Processes

Briefly speaking, the Representation Information for our real-time processes is:

Structure: the structure is a block-diagram flow structure.

Semantics: semantic of each element is (most of the time) already existing indocumentation.

A simple example, showing the structure of a (very simple) process, that provides

the structure and the semantics of one of these elements is shown in Fig. 22.2.


7/22

22.4 Preserving the Real-Time Processes 413

Fig. 22.2 Splitting a process into structure and semantics

Moreover, existing documentation is written according to a template (which can

be expressed either in LaTEX or PDF).

The methodology was defined as follows:

Reduce the block-diagram flow to an algebra (choice of existing FAUST lan-

guage, concise and sufficiently expressive developed by Grame, Lyon, France).

Store the semantics of the elements, by extracting them from existing documen-

tationTo this end, several tools have been developed, according to the architecture

shown in Fig. 22.3.

Reference

Manual (PDF)

Ontology

template (RDF)

RepInfo (XML)

RepInfo (XML)

PDI (RDF)

DATA (XML)Patch files (MAX)

IRCAM DOCtool

IRCAM

FUNC tool

IRCAM FILE

tool

IRCAM LANG tool

MustiCASPAR

- Ingest

WHUNESCO

committee

Fig. 22.3 The process for generation of RepInfo and PDI


8/22


The role of each of these tools is defined as follows:

DOC tool: extracts from existing documentation the semantics of elements

FUNC tool: parses the code of each single process in order to identify elements,

verify existence of RepInfo (if RepInfo missing, a warning is generated, see demobelow), and provides PDI for the process according to the PATCH ontology

template

FILE tool: analyze the global structure of all provided files, encodes PDI of work

according to provided WORK ontology template

LANG tool: re-encodes the syntax of the original process according to the chosen

language (FAUST)

The results of these extractions are stored in the archive, according to the OAIS

methodology, during the Ingest phase.

22.4.1 Preserving Logical Relationships

In order to preserve logical relationship, we developed several ontology templates

for specific elements of the objects we have to preserve. These ontologies have

been expressed using CIDOC-CRM (Conceptual Reference Model), that provides

definitions and a formal structure for describing the implicit and explicit concepts

and relationships used in cultural heritage documentation. CIDOC CRM is an ISO

standard (ISO 21127:2006).

Ontologies have been defined, and expressed in RDF for each element we haveto preserve:

Work

Real-time process (with subclasses, patch libraries, functions)

Documents: program notes, Hall program, Biography, Interview, audio sample,

Video sample, Score, Recording. . .

These ontologies provide a template for relationship with other elements of the work

to be preserved.Examples: Ontology for WORK provides a template aimed to express the rela-

tionship between all elements composing the work, and particularly documentation

files is shown in Fig. 22.4.

The ontology for patch (real-time process) is shown in Fig. 22.5.

22.4.2 IRCAM Scenario

In order to test the validity of the provided information (PDI as well asRepresentation Information), we developed specifically an accelerated lifetime

test, by assuming that one of the main elements in use today (the Max/MSP

software) is not available anymore.


9/22


E35_TitleC

LiteralC

work titleL

work conception labelL

composer nameL

work conception beginning dateL

end of work conception detai lL

I

I

I

I

I

I

I

I

I

C F46_Individual_Work

C F30_Work_Conception

C E35_TitleC E52_Time_Span

C E21_Person

C E82_Actor_Appelation

C Literal

C Property

P14_1F_WriterP

P14F_carried_out_byP

P14_1F_ComposerP

R58F_is_derivative_ofP

R58_1F_MigrationP

P14_1F_VideastP Migration DerivativeL

Fig. 22.4 Ontology for work

22.4.3 Scenario Summary

We (as archivists) ingest of a new WORK into MustiCASPAR,

We (as registered experts) receive an asynchronous notification of loss of

availability for a COMPONENT,

We (as registered experts) search for equivalent COMPONENTS:

A new version of the COMPONENT becomes available, or: We apply a migration of the component, on the basis of RepInfo

We (as archivists) ingest of the new version of the COMPONENT.

Used Packages

Representation Information Toolbox

Registry

Authenticity manager

Preservation Orchestration Manager Packaging

Knowledge Manager


10/22


11/22


22.4.4 Validation of Representation Information

The purpose of this checking is to validate the Representation Information extracted

from the real-time process (see Fig. 22.6).

Fig. 22.6 Checking completeness of RepInfo


12/22


To this end, Representation Information checking is performed in three steps:

1. Checking completeness of information: reconstruction of original process from

extracted Representation Information

2. Checking usefulness: construction of an equivalent process, from extracted

RepInfo , but executed from PureData (equivalent to a migration)

3. Authenticity: comparison of audio outputs, according to defined Authenticity

protocol

22.4.4.1 Checking Completeness of Information

The purpose of this checking is to check the completeness of Information

Representation. To this end, we apply a transformation using the Language Tool

(described above) to the original object (process). We then apply the reverse trans-

formation in order to obtain a new process. This process is supposed to be the same

as the original one. We can apply a bit-to-bit comparison method to the objects in

order to detect any loss of information illustrated in Fig. 22.7.

22.4.4.2 Checking Usefulness of Information

The purpose of this check is to show that a new process, different from the orig-

inal one, but functional, can be reconstructed from the provided Representation

Information as illustrated in Fig. 22.7.

In order to show this, an automatic translation tool is used (based on theLanguage Tool already described), replacing the original Max/MSP environment

by the PureData environment.

It should be noticed that some manual adjustments have to be made in the cur-

rent version of the tools (due to incompleteness of Representation Information with

PureData).

22.4.4.3 Checking Authenticity

In order to check Authenticity, we apply an Authenticity protocol.

Here is a slightly simplified version of the AP:

At Ingest phase, 3-steps Authenticity Protocol:

Choose an input audio file (inputFile1)

Apply audio effect on it

Record output audio file (outputFile1)

At Migration phase, 3-steps Authenticity Protocol :

After migration, apply new audio effect on inputFile1

Record output audio file (outputFile2)

Compare outputFile1 and outputFile2 (by ear audio engineer, or any other

method of comparison, for example comparing spectrograms), illustrated in

Fig. 22.8


13/22

22.5 Interactive Multimedia Performance 419

Fig. 22.7 Checking usefulness of RepInfo

As an important remark, it has to be noticed that, when comparing output files, some

adjustments have to be made on the object itself in order to achieve authenticity.

22.5 Interactive Multimedia Performance

In this Section, we discuss the motivation, considerations, approaches and results

of the CASPAR contemporary arts testbed with a particular attention on Interactive


14/22


Fig. 22.8 Checking authenticity

Multimedia Performances (IMP) [221]. The section describes several different IMP

systems and presents an archival system, which has been designed and implemented

based on the CASPAR framework and components for preserving Interactive

Multimedia Performances.

22.5.1 Introduction

IMP is chosen as part of the testbeds for its challenges due to the complexity and

multiple dependencies and typically involves several difference categories of digitalmedia data. Generally, an IMP involves one or more performers who interact with a

computer based multimedia system making use of multimedia contents that may be

prepared as well as generated in real-time including music, audio, video, animation,

graphics, and many others [222, 223].

The interactions between the performer(s) and the multimedia system [224226]

can be done in a wide range of different approaches, such as body motions (for

example, see Music via Motion (MvM) [227, 228]), movements of traditional musi-

cal instruments or other interfaces, sounds generated by these instruments, tension

of body muscle using bio-feedback [229], heart beats, sensors systems, and many

others. These signals from performers are captured and processed by multimedia

systems. Depending on specific performances, the input can be mapped onto mul-

timedia contents and/or as control parameters to generate live contents/feedback

using a mapping strategy.
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


15/22


Traditional music notation as an abstract representation of a performance it is

not sufficient to store all the information and data required to reconstruct the per-

formance with all the specific details. In order to keep an IMP performance alive

through time, not only its output, but also the whole production process to create the

output needs to be preserved.

22.5.2 Interactive Multimedia Performance (IMP) Systems

In this section we describe several different IMP systems and software with different

types of interaction and different types of data while the following section explains

how the CASPAR framework is used for their preservation.

22.5.3 The 3D Augmented Mirror (AMIR) System

The 3D Augmented Mirror (AMIR) [230, 231] is an example IMP system which

has been developed in the context of the i-Maestro project (www.i-maestro.org)

[232], for the analysis of gesture and posture in string practice training. Similar to

many other performing arts, string players (e.g. violinist, cellists) often use mirrors

to observe themselves practicing to understand and improve awareness of their play-

ing gesture and posture. More recently, video has also been used. However, this is

generally not effective due to the inherent limitations of 2D perspective views of the

media.

The i-Maestro 3D Augmented Mirror is designed to support the teaching and

learning of bowing technique, by providing multimodal feedback based on real-time

analysis of 3D motion capture data. Figures 22.9 and 22.10 show screenshots of the

i-Maestro 3D Augmented Mirror interface which explore visualization and sonifica-

tion (e.g. 3D bow motion pathway trajectories and patterns) to provide gesture and

posture support. It uses many different types of data including 3D motion data (from

a 12-camera motion capture system), pressure sensor, audio, video and balance.

The i-Maestro AMIR multimodal recording, which includes 3D motion data,

audio, video and other optional sensor data (e.g. balance, etc) can be very useful

to provide in-depth information beyond the classical audio visual recording many

different purposes including technology-enhanced learning, and in this context for

the preservation of playing gesture and style for detailed musicological analysis

(now and in the future).

22.5.4 ICSRiM Conducting Interface

The ICSRiM Conducting System is another IMP system example. It has been devel-

oped for the tracking and analysis of a conductors hand movements [233, 234]. The

system is aiming at supporting students learning and practicing conducting, and
http://-/?-http://-/?-http://www.i-maestro.org/http://www.i-maestro.org/http://www.i-maestro.org/http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://www.i-maestro.org/http://-/?-http://-/?-


16/22


Fig. 22.9 The i-Maestro 3D augmented mirror system showing the motion path visualisation

Fig. 22.10 AMIR interface showing 3D motion data, additional visualizations and analysis


17/22


18/22


inter-relationships and additional information considering the reconstruction issues.

It is a challenging issue since it is difficult to preserve the knowledge about the log-

ical and temporal components, and all the objects such as the captured 3D motion

data, Max/MSP patches, configuration files, etc, in order to be properly connected

for the reproduction of a performance [235].Due to these multiple dependencies, the preservation of an IMP requires robust

representation and association of the digital resources. This can be performed

using entities and properties defined for CIDOC-CRM and FRBRoo. The CIDOC

Conceptual Reference Model (CRM) is being proposed as a standard ontology for

enabling interoperability amongst digital archives [236].

CIDOC-CRM defines a core set concepts for physical as well as temporal entities

[237, 238]. CIDOC-CRM was originally designed for describing cultural heritage

collections in museum archives. A harmonisation effort has also been carried out

to align the Functional Requirements for Bibliographic Records (FRBR) [239] toCIDOC-CRM for describing artistic contents. The result is an object oriented ver-

sion of FRBR, called FRBRoo [240]. The concepts and relations of the FRBRoo are

directly mapped to CIDOC-CRM.

Figure 22.12 demonstrates how the CIDOC-CRM and FRBR ontologies are used

for the modelling of an IMP.

22.5.6 ICSRiM IMP Archival System

The CASPAR project evaluated a set of preservation scenarios and strategies in

order to validate its conceptual model and architectural solutions within the different

F52.Performance

IMP

F8.PersonKia (Director)

Frank

(Performer)

E53.PlaceLeeds -UK

E52.Time-Span2hours:5PM-12/02/07

E22.Man-MadeObject

Cello

Sound MixerComputer System

E73.InformationObjectMusic

Music Score

P16F use specific

object

P16F use specific

object

P7F took place atP4F has time

span

P14F carried out by

Fig. 22.12 Modelling an IMP with the use of the CIDOC-CRM and FRBR ontologies
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


19/22


Fig. 22.13 The interface of the Web archival system

testbed domains. In this case, our scenarios are related with the ingestion, retrieval

and preservation of IMPs.

The ICSRiM IMP Archival System has been designed and developed with theCASPAR framework integrating a number of selected CASPAR components via

web services. The system has been used to implement and validate the preservation

scenarios.

The archival system is a web interface, shown in Fig. 22.13, which communicates

with a Repository containing the IMPs and the necessary metadata for preserving

the IMPs. The first step for preserving an IMP is to create its description based on the

CIDOC-CRM and FRBRoo ontology. This information is generated in RDF/XML

format with the use of the CASPAR Cyclops tool. The Cyclops tool [241] is used to

capture appropriate Representation Information to enhance virtualisation and futurere-use of the IMP. In particular, this web tool is integrated into the Archival System

and it used in order to model various IMPs.

During ingestion, the IMP files and the metadata are uploaded and stored in the

Repository with the use of the web-based IMP Archival System. For the retrieval

of an IMP, queries are performed on the metadata and the related objects are

returned to the user. The following Figure shows the web interface of the ICSRiM

IMP Archival system.

In case a change occurs in the dataset of an IMP, such as the release of a

new version of the software, the user has the ability to update the Representation

Information and the dataset of the IMP with the new modules (e.g. the version

of new software). A future user will be able to understand which one is the lat-

est version of a component and how these components can be reassembled for the

reproduction of the Performance by retrieving the Representation Information of

the IMP.
http://-/?-http://-/?-


20/22


22.5.7 Conclusion

This section of the chapter introduces the usages and applications of interactive

multimedia for contemporary performing arts as well as its usefulness for captur-

ing/measuring multimedia and multimodal data that are able to better represent theplaying gesture and/or interactions. With two example IMP systems, it discusses

key requirements and complexities of the preservation considerations and presents

a digital preservation framework based on ontologies for Interactive Multimedia

Performances.

With the CASPAR framework, standard ontology models were adopted in order

to define the relations between the individual components that are used for the re-

performance. We also described the development and implementation of a web-

based archival system using the CASPAR framework and components.

The ICSRiM IMP Archival System has been successfully validated by userswho have created their own IMP systems using their own work for ingestion and

using ingested works from others (without any prior knowledge) to reconstruct a

performance with only the instruction and information provided by the archival

system.

22.6 CIANT Testbed

22.6.1 RepInfo Validation

It is quite difficult to properly demonstrate that all the RepInfo necessary to re-

perform the performance has been collected making the information in the archive

Independently Understandable. The only ultimate proof would be to grant access to

the archive to a group of artists (Designated Community) that would hire a theatre

and attempt to re-perform the piece. Since this is not a convenient solution from

obvious reasons, we decided to implement a Performance Viewer tool that would

facilitate the process of RepInfo validation by providing immediate visual and audio

feedback.

The architecture of Performance Viewer tool consists of the following compo-

nents:

Ontology loader

Timeline controller

Different visualisation profiles

The Ontology loader component serves as a bridge between the ontology which

is stored in the repository and the rest of the application. It understands the pecu-liarities of the CIDOC-CRM and the semantics of our CIDOC extensions. It also

provides a modular architecture where other components, so called Visualisation

profiles can register their event handlers. When the loading procedure is initiated,

all registered observers would receive data depending on their focus. For instance:


21/22


22/22


Fig. 22.15 Screenshot of the preview of the GOLEM performance in the performance viewer tool

Fig. 22.16 Performance viewer: from left to right, model of the GOLEM performance, timeline

slider, three different video recordings of the performance, 3D model of the stage including the

virtual dancer, 3D model used for the video projection, audio patch in Max/MSP and pure data

22.7 Summary

This chapter has shown some of the results from the Contemporary Performing Arts

testbeds which apply the techniques described in this book to the multitude of digital

objects used in this area. The view of preservation as involving re-performance, and

the ideas of authenticity, introduce a number of new ways to test our concepts, tools

and techniques.

Chapter 22 - Contemporary Performing Arts Testbed

Documents

Transcript of Chapter 22 - Contemporary Performing Arts Testbed