Interesting Sounds & Musical Acoustics & Psychoacoustics Donald Byrd 29 Feb. 2008 Copyright ©...
-
Upload
dinah-kelly -
Category
Documents
-
view
221 -
download
3
Transcript of Interesting Sounds & Musical Acoustics & Psychoacoustics Donald Byrd 29 Feb. 2008 Copyright ©...
Interesting Sounds & Musical Acoustics & Psychoacoustics
Donald Byrd29 Feb. 2008
Copyright © 2006-08, Donald Byrd
rev. 20 Feb. 08 2
Rudiments of Musical Acoustics
• Need some musical acoustics for almost anything in digital audio
• Acoustics: branch of physics that studies sound (of any kind)– Concepts like frequency & amplitude– Besides musical, architectural, ultrasonic (for
medicine, underwater, etc.), other
• Psychoacoustics: study of how sound is perceived; mostly psychology– Concepts like pitch, loudness, timbre
• Relationship to physical concepts often roughly logarithmic
• …but only roughly: always more complex than that
rev. 20 Feb. 08 3
Materials for Studying Audio
• What are interesting sounds really like?– Sine waves, etc. are boring (cf. addsynenv)– Sounds of acoustic instruments are “rich”– Vary in every way: with pitch, loudness, time
• Musical instrument samples• Audacity audio editor
– For Windows, Mac OS 9 and X, Linux– Download from
http://audacity.sourceforge.net/
• Programs in (e.g.) R• addsynenv (additive synthesis demo)
10 Sept. 2006 4
Review: Parameters of Musical Sound
• Four basic parameters of a definite-pitched musical note1. pitch: how high or low the sound is: perceptual
analog of frequency
2. duration: how long the note lasts
3. loudness: perceptual analog of amplitude
4. timbre or tone quality
• Above is decreasing order of importance for most Western music
• …and decreasing order of explicitness in CMN (Conventional Music Notation)!
rev. 11 Sept. 06 5
Musical Acoustics (1)• Acoustics involves physics• Musical (opposed to architectural, etc.) acoustics
– Frequency (=> pitch)– Amplitude (=> loudness)– Spectrum, envelope, & “other” characteristics => timbre– Partials vs. harmonics
• Psychoacoustics involves psychology/perception– Perceptual coding (for “lossy” compression: MP3, etc.)
• Timbre– Old idea (thru ca. 1960’s?): timbre produced by static
relationships of partials, plus envelope– …but attack often more distinctive than “steady state”!– Rich (interesting) sounds are complex; nothing is static– Time domain (waveform) vs. frequency domain (spectrum,
spectrogram) views
rev. 20 Feb. 08 6
Materials for Studying Acoustics
• Most useful– Piano (preferably a grand)– String instruments (preferably large &
bowed)– Strobe light– Drinking glasses filled to different levels– Simple percussion: sleighbell, “rainegg”, etc.– Recording equipment– Audio editor, spectral-analysis program
• Also helpful– Other musical instruments– Tuning fork– Echoic & anechoic chambers
20 Feb. 08 7
Acoustic Phenomena
• Resonance & sympathetic vibration• The harmonic series• Standing waves in a solid (e.g., string)• Interference & beats
– Chorus effect
• Harmonics (on bowed strings) show:– Modes of vibration– Nodes– Relationship to sul ponticello?
• Architectural acoustics– Resonance (as in I Am Sitting in a Room)– Standing waves in air
4 April 07 8
Modes of Vibration & Simple Vibrating Systems
• Waves can be transverse or longitudinal– Which is water? Which is sound?
• Simple vibrating systems & waveforms– Tuning fork
• Not many other examples!
– What’s the waveform?
20 Feb. 08 9
Complex Vibrations (1)
• Modes of vibration– String clamped at both ends vibrates at f, 2f, 3f…– Air column closed at both ends: likewise– …where f = frequency determined by length– Air column closed at one end, open at other: f, 3f,
5f…– …where f = half the frequency of previous case
20 Feb. 08 10
Complex Vibrations (2)
• Resonance– “When a system that can vibrate at a certain frequency is
acted on by periodic disturbance at same frequency, vibrations of large amplitude can be produced.” —Backus
– Ex: swing in playground– Ex: string & body of musical instrument– Ex: sympathetic vibration– Def. of resonance: the tendency of a system to oscillate at
maximum amplitude at a certain frequency.
rev. 20 Feb. 08 11
How String Instruments Make Sounds
• Vibrating strings• Bowing
– Rosin increases friction: bow pulls string, snaps back
– …so motion of string is almost a sawtooth
– Waveform is somewhat like sawtooth => lots of harmonics!
– Special effects involving bow: sul ponticello, col legno, etc.
• Plucking• Special effects not involving bow
– Harmonics, “natural” & “artificial”
• Player touches string lightly => forces node at that point
• NB: not the usual acoustics sense of harmonics
– Muting
20 Feb. 08 12
How Woodwinds Make Sounds
• Vibrating air columns• Without reeds: flute
– Acts like open column
• With reeds: oboe, clarinet, bassoon– Clarinet acts like half-open column; others like open
column– Cause: impedance mismatch w/ outside air(?)– Result: tube is about same length as flute & oboe,
but it sounds an octave lower
• Overblowing• What about brass instruments?
4 April 07 13
Psychocoustic Phenomena
• A matter of psychology (and, to some extent, music theory)
• Basis for auditory illusions– Shepard tones (a.k.a. endless glissando)– “Mystery melody”
• Practical application: perceptual coding used in lossy compression (MP3, WMA, etc.)– Masking– Threshhold of audibility
rev. 29 Feb. 08 14
Scholars (and others) Beware! (1)
• Plausible (at the time!) assumptions– Men have more teeth than women (ancient)
• Aristotle, The History of Animals, Book II– Diseases can’t be transmitted by invisible
organisms (19th century)– Stomach ulcers can’t be caused by
organisms (20th century)
• What you expect vs. what you see/hear– Standard use of sponges on kitchen
counters increases harmful bacteria (early 21st cent.)
– New York Times headline: “Dinosaurs Cavort in Film for Doyle” (1923)
rev. 29 Feb. 08 15
Scholars (and others) Beware! (2)• What you expect vs. what you hear
– Don & the Kurzweil 250 flute sound• “Obvious” something was wrong w/ the
sound—until compared to a real flute– Hammond organ vs. a real pipe organ
• Experts & laypeople couldn’t distinguish– Don, a famous musician, & K250 handclaps– Huron on what he “knew” & learned
• R. Moog at Kurzweil & piano touch
11 Sept. 2006 16
Creating Interesting Sounds (1)
• Periodic, nearly-, & non-periodic signals• Approaches to creating sounds
– Additive synthesis: like painting– Subtractive synthesis: like sculpture– Sampling: like collage– Others: modulation, physical modeling
• Most possible with analog or digital hardware, but analog is limited
rev. 13 Sept. 2006 17
Creating Interesting Sounds (2)• Cf. “Electronic Music Tutorial” (Ishkur)
– In Ishkur’s Electronic Music Guide website
• Additive Synthesis• Fourier's Theorem
– Any periodic signal => sum of harmonically-related sine waves
• Envelopes: continuous & piecewise linear– Important special case: “ADSR”– Attack, Decay, Sustain, Release
• Phase, interference, & beats– Phase by itself rarely important, but relationships are– Diagrams & demo in CECM Acoustics Primer, Sec. 8– Interference between channels or very close partials
rev. 29 Feb. 08 18
Additive Synthesis & Envelopes• addsynenv does additive synthesis of up to six partials
– Each has arbitrary partial no., starting phase, "ADSR" envelope– Partial no. can be non-integer => not harmonic– ADSR = Attack/Decay/Sustain/Release (3 breakpoints)– …but addsynenv allows much more complex envelopes– Plays one note with waveform specified by partial nos. & their
envelopes (maybe also phases)– Simultaneously displays “spectrogram” or “sonogram”– …but not waveform– Phase in real world normally has little effect, but can be critical in
recording & digital worlds (e.g., cancellation)• Cf. Roads, Computer Music Tutorial• Really additive Fourier synthesis: alternatives are grains• (Wavelet Transform) & Walsh functions• Additive synthesis can’t create aperiodic (non-definite pitch)
sounds• ...or many realistic attacks
11 Sept. 2006 19
Creating Interesting Sounds (3)
• Early CCRMA (Stanford) studies of acoustic instrument sounds– Envelope for each partial with a few segments– Similar to addsynenv
• Subtractive Synthesis• Early (analog: Moog, etc.) synthesizer model• Signal source: sine/square/sawtooth (for
additive)• …or noise & filters (for subtractive)• LFO to add vibrato, tremolo, glissando, etc.• ADSR envelope for entire sound
31 Oct. 07 20
Periodicity & Definite PitchPeriodic waveform: clearcut definite pitch
With sharp corners: much high-freq. energy
square wave; loud trumpet
Without sharp corners: little high-freq. energy
sine wave; low, soft flute
Almost-periodic waveform: definite pitch
piano
Somewhat periodic waveform: complex or multiple pitches
bells
Aperiodic waveform: noisy, no definite pitch
cymbal, bass drum
12 Feb. 07 21
Real-World Musical Sounds (1)• The “Attack/Sustain/Release” (ASR) model for notes
– Attack, Sustain, Release modified from recordings– Attack includes Decay from ADSR model
• Used in the Kurzweil 250 (1984), etc.– Original version had only 2 MB for all samples– Piano had diff. samples for 2 loudness levels– …and diff. sound for every 4-6 semitones– 1-2 sec. per sample for A+S+R
• How good did the K250 really sound?– COUNTDOWN, by Christopher Yavelow– “An opera for the nuclear age”
• “the ‘orchestral accompaniment’ is in reality a Kurzweil-250 digital sampler, synchronized to the baton of the conductor…”
– http://www.yavelow.com/docs/countdown.html
19 Feb. 07 22
Real-World Musical Sounds (2)
• Nowadays, can afford “unlimited” sustain• …but also need diff. sounds for many (8?) diff.
loudness levels (multisampling)– “All Together Now”, Electronic Musician, Jan. 2007
• …and diff. sound for every semitone or two• W/ unlimited sustain, takes gigabytes just for
piano!
29 Oct. 07 23
Timbre Space & Envelopes (1)• Timbre space
– Includes all possible timbres– Not well understood; may be 3-dimensional
• “FourierRandomTimbre” covers only a tiny part
• Some ways to enlarge it:– Overall envelopes– Allow (inharmonic) partials– Envelopes for partials
• Formants show timbre involves context: not really a property of a note!
• Timbre as "the psychoacoustician's multidimensional wastebasket category"
29 Oct. 07 24
Timbre Space & Envelopes (2)• Envelope
– Cf. Wikipedia article on timbre– Amplitude structure of a sound, so called because sound
just "fits" inside its envelope• Piecewise-linear envelope
– Envelope that can be drawn with small number of straight lines & no curves
– Important in music synthesis because can approximate many complex real-world sounds well with very little data
– Special case: "ADSR" (Attack - Decay - Sustain - Release) envelopes; common on synthesizers, etc.
– Samplers may combine A & D
22 Sep. 2006 25
Uncompressed Audio Files are Big
• 1 byte = 8 bits (nearly always)• How much data on a CD?
– CD audio is 44,100 samples/channel/sec. * 2 bytes/sample * 2 channels = 176,400 bytes/sec., or 10.5 MByte/min.
– CD can store up to 74 min. (or 80) of music– 10.5 MByte/min. * 74 min. = 777 MBytes– Actually more: also index, error correction
data, etc.
16 Feb. 06 26
Compressed Audio: Lossless & Lossy
• Don’t confuse data compression and dynamic-range compression (a.k.a. audio level compression, limiting)
• Codec = COmpressor/DECompressor• Lossless compression
– Standard methods (LZW: .zip, etc.) don’t do much for audio
– Audio specific methods
• MLP used for DVD-Audio• Apple & Microsoft Lossless
• Lossy compression– Depends on psychoacoustics (“perceptual coding”)
13 Feb. 06 27
Specs for Some Common Audio Formats
22 Sep. 2006 28
Psychoacoustics & Perceptual Coding
• Pohlmann, Ken (2005). Principles of Digital Audio, 5th ed., Chapter 10: Perceptual Coding
• Rationale: much better data compression• Physiology of ear and critical bands
– Not fixed frequency: any sound creates one or more critical bands
• Masking– Depends on relative loudness & frequency– Noise is much better than pitched sounds
• Threshhold of hearing– Depends greatly on frequency
22 Feb. 06 29
Compressed Audio: Lossy Compression
• General method1. Divide signal into sub-bands by frequency2. Take advantage of:
• Masking (“shadows”), via amplitude within critical bands
• Threshhold of audibility (varies w/ frequency)• Redundancy among channels
• MPEG-1 layers I thru III (MP-1, 2, 3), AAC get better & better compression via more & more complex techniques– “There is probably no limit to the complexity of
psychoacoustics.” --Pohlmann, 5th ed.– However, there probably is an “asymptotic” limit to
compression!• Implemented in hardware or software codecs
rev. 25 Sep. 2006 30
Another “Analog vs. Digital” Battle
• Merton, Orren (2006, February). The Sum of All Tracks. Electronic Musician 22,2, pp. 57-63
• About summing (mixing) with analog vs. digital hardware
• Tested with excellent hardware, panel of experts
• Methodology not really scientific• But is this fair? Does it really matter? How can
you do better with subjective phenomena?– Electronic Musician isn’t a scholarly journal– Critical: blind study => results probably meaningful– Cf. http://createdigitalmusic.com/2006/09/06/analog-
summing-pm8-for-people-who-dont-trust-software-mixing/
25 Sep. 2006 31
Scientific Evaluation of Subjective Phenomena: Lossy Compression (1)
• Pohlmann, Ken (2005). Principles of Digital Audio, 5th ed., pp. 403-413
• Discusses perceptual coding performance evaluation
• Want to evaluate as objectively as possible!• One idea: measure artifacts
– Difference between original & compressed => distortion
– Psychoacoustic model can estimate NMR (Noise-to-Masking Ratio)
25 Sep. 2006 32
Scientific Evaluation of Subjective Phenomena: Lossy Compression (2)
• Pohlmann: “best way to evaluate is to exhaustively listen”
• Tests with excellent hardware, panel of experts
• …but serious scientific methodology• Double-blind, ITU-R methods, CCIR 5-pt scale,
>=50 subjects, etc.• Similar methods used in validation of MP3,
AAC, etc.• …and ff123 et al’s evaluation of Ogg Vorbis• Our listening test: at http://www.informatics.
indiana.edu/donbyrd/N560Site-Fall06/Worst/
25 Sep. 2006 33
Auditory Illusions 1
• Wikipedia article “auditory illusion”• Deutsch's Musical Illusions
– E.g., Mysterious melody, Chromatic illusion– http://psy.ucsd
.edu/~ddeutsch/psychology/deutsch_research1.html
• Visual and Auditory Illusions (from UBC)– Often analogous illusions– E.g., Shepard's Tones, Tritone Paradox– http://www.cs.ubc.
ca/nest/imager/contributions/flinn/Illusions/Illusions.html
rev. 29 Feb. 08 34
Auditory Illusions 2: Note, Chord, or Scale?
• addsynenv whole-tone preset• Issue is partials each with its own envelope
– Clue: relative frequencies of partials– Clue: shapes of envelopes
• Try to simplify the sound…• Do you experience fusion?
– Probably not, but…
rev. 5 Nov. 2007 35
Auditory Illusions: Shepard’s Tones
• Also called (less accurately) endless glissando• Demo on Web
– http://www.cs.ubc.ca/nest/imager/contributions/flinn/Illusions/Illusions.html
– Wikipedia version has a more interesting timbre
• Again, it’s partials & envelopes• Clue: relative freqs. of partials are important,
unusual• Clue: envelopes of partials are important• Used by J.C. Risset in computer-synthesized
music (Little Boy, etc.)– in one case, with rhythm equivalent
rev. 25 Jan. 2007 36
Visual Analogue of Shepard’s Tones
rev. 5 Nov. 2007 37
Auditory Illusions: Mysterious Melody
• Hard to recognize, unless you already know what it is!
• Clue: probably easier for Americans