ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris...

17
ALGORITHMICALLY GENERATED MUSIC ENHANCES VR DECISION SUPPORT TOOL Dr. Panaiotis Department of Music & Department of Electrical and Computer Engineering The University of New Mexico Steven A. Smith Los Alamos National Laboratory Victor M. Vergara and Shan Xia Department of Electrical and Computer Engineering The University of New Mexico Dr. Thomas P. Caudell Department of Electrical and Computer Engineering & Center for High Performance Computing The University of New Mexico

Transcript of ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris...

Page 1: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

ALGORITHMICALLY GENERATED MUSICENHANCES VR DECISION SUPPORT TOOL

Dr. PanaiotisDepartment of Music &

Department of Electrical and Computer EngineeringThe University of New Mexico

Steven A. SmithLos Alamos National Laboratory

Victor M. Vergara and Shan XiaDepartment of Electrical and Computer Engineering

The University of New Mexico

Dr. Thomas P. CaudellDepartment of Electrical and Computer Engineering &

Center for High Performance ComputingThe University of New Mexico

Page 2: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

CB Defense Decision Support Tool

University Partnership Team UNM – Frank Gilfeather, Thomas Caudell,

Panaiotis, Tim Ross, Mahmoud TahaNMSU – Jim Cowie, Chris Fields,

Hung Nguyen , Bill Ogden, Ram PrasadMIIS – Gary Ackerman, Markus Binder,

Sundara Vadlamudi

Engages a broad-based team of creative professionals

BiochemistsKnowledge engineers

Computer Engineers

Mathematicians

Scientists

Engineers

Cognitive Psychologists Economists

Political ScientistsHistorians

Artists

Page 3: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Science Serving

Art

Page 4: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Art Serving Science

Page 5: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Scientist Serving Pasta

Page 6: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Visualization of Mockup System (1st Generation)

Page 7: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Flatland Sound Services

Page 8: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Max/MSP

Page 9: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

SoundCycler Architecture

Page 10: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Six Music Functions in DST

1. The music provides an appropriate ambience to the task at hand

2. The music displays user adjustable input values

3. The music provides user activity feedback

4. The music provides cues for orientation in the virtual environment

5. The music characterizes the data

6. The music provides a means to analyze the data

Page 11: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Musically Represented Data

The music represents five classes of data in the DST:

1. A twenty-two-element scenario likelihood vector

2. A twenty-two-element differential between the likelihood-consequence vector (need for mitigation) and the after-mitigation consequence

3. Eight funding levels

4. One total funding level

5. Final expected consequence value.

Page 12: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Likelihoods

Each horizontal bar represents a likelihood level of a scenario attack within a vector.

Page 13: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Ratios

Each horizontal bar represents a ratio difference between consequence-likelihood level and the projected

consequence-likelihood after remediation.

Page 14: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Funding Portfolios

Each horizontal bar represents a funding portfolio amount.

Page 15: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Total Funding Limit

The total funding limit affects tempo. More money is livelier.

Page 16: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

Aggregated Final Expected Consequence

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Greater adverse consequence has more persistent bass line.

Page 17: ALGORITHMICALLY GENERATED MUSIC …...Panaiotis, Tim Ross, Mahmoud Taha NMSU – Jim Cowie, Chris Fields, Hung Nguyen , Bill Ogden, Ram Prasad MIIS – Gary Ackerman, Markus Binder,

??

??

????

????

????

??

??