Audio EffectsProcessing · 2019. 2. 15. · Impression of space (echo, reverb) Increasing perceived...
Transcript of Audio EffectsProcessing · 2019. 2. 15. · Impression of space (echo, reverb) Increasing perceived...
-
1
Audio Effects ProcessingVesa Välimäki, Fabián Esqueda & Benoit Alary
ELEC-E5620 Audio Signal Processing
15.02.2019
DEMO:Pink Noise GeneratorsEero Lehtimäki & Uljas Pulkkis
2
-
2
Course Schedule in 2019 (Periods III, IV)
Välimäki, Parker, Esqueda & Alary15.2.2019
0. General issues (Vesa & Benoit) 11.1.20191. History and future of audio DSP (Vesa) 18.1.20192. Digital filters in audio (Vesa) 25.1.20193. Audio filter design (Vesa) 1.2.20194. Analysis of audio signals (Vesa) 8.2.20195. Audio effects processing (Vesa) 15.2.2019* No lecture (Evaluation week for Period III) 22.2.20196. Synthesis of audio signals (Fabian) 1.3.20197. Reverberation and 3-D sound (Benoit) 8.3.20198. Physics-based sound synthesis (Vesa) 15.3.20199. Sampling rate conversion (Vesa) 22.3.201910. Audio coding (Vesa) 29.3.2019
3
Outline• Echo / Delay• Flanging and Phasing• Chorus • Pitch shifting / Time stretching• Dynamic processing (compression & expansion)• Other effects
Demo• Beat-Aligning Looper• Pink Noise Generators
Välimäki, Parker, Esqueda & Alary15.2.2019
4
-
3
What is an audio effect? Any kind of audio signal processing applied to a
captured or synthesized sound for creative purposes
Possible purposes: Impression of space (echo, reverb) Increasing perceived size of a sound (chorus) Introducing movement into a static sound (flanging, phasing) Altering timbre (distortion) Altering dynamics (compression, limiting)
5
Echo/Delay One of the simplest and earliest audio effects
Initially they were made using tape loops Digital version very simple
Delay line with feedback Filtering or distortion can be added to the feedback loop Extra taps can be added for more complex pattern Real-time implementation using “circular buffer”.*
Sound example taken from: http://en.wikipedia.org/wiki/Delay_%28audio_effect%29*Good reference: The Audio Programming Book by R. Boulanger & V. Lazzarini
Välimäki, Parker, Esqueda & Alary15.2.20196
-
4
Bucket-Brigade Devices (BBDs)• Tape machines are expensive! • BBDs are discrete-time analog delay lines• Invented by F. Sangster and K. Teer at the Phillips Research Labs in 1968.• Input signal is sampled in time and passed through a series of capacitors
and switches.• Charge in each capacitor is passed to subsequent stage at a rate determined
by clock.
http://www.electrosmash.com/mn3007-bucket-brigade-devices
Välimäki, Parker, Esqueda & Alary15.2.2019
7
BBDs (cont’d)• CLK 1 and 2 in anti-phase configuration.• BBD flangers typically have a single 1024-stage unit.• Number of stages fixed. Delay length determined by clock’s rate.
Välimäki, Parker, Esqueda & Alary15.2.2019
8
-
5
BBDs (cont’d)• Although analog, BBD delay samples input signal and we must
adhere to Sampling Theorem• Appropriate anti-aliasing and anti-imaging filters required at input
and output, respectively.• Signal-to-noise ratio (SNR) is typically poor.• To ameliorate this, BBD is preceded by compressor and succeeded
by expander (compander).• Very smart but not so intuitive design!
http://ant-s4.unibw-hamburg.de/dafx/paper-archive/2005/P_155.pdf
BBD Flanger
Välimäki, Parker, Esqueda & Alary15.2.2019
9
Flanging Invented by Les Paul (1915-2009) in 1945, but the name
came from John Lennon in 1966 (http://en.wikipedia.org/wiki/Flanging) Original analog method for flanging
Copy the same sound on two open-reel tapes Play the 2 tapes on 2 synchronized tape machines Touch the flange of one tape reel to slow it down Get a nice “wooshing” phase-cancellation effect
Välimäki, Parker, Esqueda & Alary15.2.2019
10
-
6
Flanging: Analog-Era Realization
http://www.audiotechnology.com/tape-flanging-in-the-new-world/
Välimäki, Parker, Esqueda & Alary15.2.2019
11
Flanging Sounds Familiar
Many everyday cases C. Huygens (1693): the
sound of a fountain has a pitch when it reflects from a staircase
Moving and hissing sound source (or listener moving)
Jet airplane flying over a city Direct sound and its echo
Time-varying delay
Välimäki, Parker, Esqueda & Alary15.2.2019
12
-
7
Digital Flanger – Naive Version
A copy of the signal is fed through a variable digital delay line and added to the original
Produces a time-varying comb filter Magnitude response contains many uniformly spaced, moving
notches
Välimäki, Parker, Esqueda & Alary15.2.2019
13
Digital Flanger – Naive Version with LFO
Delay-line length is modulated with a Low Frequency Oscillator (LFO) Slow modulation frequency, approx. 0.1 Hz – 10 Hz
Pink noise Pink noise
E-gtr examples by Timo Hiekkanen and Tuukka Lyly, TKK, 2007
E-Guitar
Drums Drums
E-Guitar
Välimäki, Parker, Esqueda & Alary15.2.2019
14
-
8
Digital Flanger – Thru Zero
Problem with naive implementation Dry and delayed signal never coincide exactly
Solution: Add a static delay l to the ‘undelayed’ path, which is about half of the max value of m
Välimäki, Parker, Esqueda & Alary15.2.2019
15
Interpolated Variable Delay Line
In flanging, the delay-line length must vary smoothly to avoid discontinuities and clicks Otherwise “zipper noise” is produced
A fractional delay is needed Usually an FIR interpolation filter
z-1x(n)
h(0) h(1)
y(n)
z-1 z-1
h(N)h(2) ...
Välimäki, Parker, Esqueda & Alary15.2.2019
16
-
9
Delay Line with Linear Interpolation
• For digital audio effects, linear interpolation may be sufficiently good– The two-tap FIR is a mild lowpass filter, which varies with d
dd1
)(nx
)(ˆ)( dMnxny
1zMz
Välimäki, Parker, Esqueda & Alary15.2.2019
17
Flanging – The Movie
Flanging - No Interpolation(notches move in steps)
Flanging - Linear Interpolation(smooth sliding with filtering)
Välimäki, Parker, Esqueda & Alary15.2.2019
18
-
10
Phasing Allpass-filtered signal is added to the original signal
Originally an analog electronic version of flanging (“a poor man’s flanger”)
Usually a series of allpass filters Each allpass filter is of low order, e.g., first or second order Phase shift of each allpass is modulated with LFO
Välimäki, Parker, Esqueda & Alary15.2.2019
19
Phasing - Notches Notches are generated at frequencies where the phase
response of filter chain is multiple of –π (or –180˚) For example, four 2nd-order allpass filters in cascade → 4 notches Change of coefficients moves the notches
Magnitude response of the overall system
Frequency
Phase response of the allpass chain
Frequency
Välimäki, Parker, Esqueda & Alary15.2.2019
20
-
11
The Allpass Filter RevisitedFilters• The z-domain transfer function of a digital allpass filter is given by
• Parameter a1 determines break frequency.
• The phase response of a single allpass and several cascaded units is then:
Välimäki, Parker, Esqueda & Alary15.2.2019
One allpass filter Several allpass filters
21
Phaser with 10 Allpass Filters• Topology similar to that of MXR Phase 100 Pedal (5 notches, i.e. 10 filters).
• DC Blocker at the input.
• Naive approach; parameter a1 fully modulated. Not so useful!
a1 = 0.1 a1 = 0.9
100 1k 10k 20kFrequency (Hz)
-40
-35
-30
-25
-20
-15
-10
-5
0
5
10
Mag
nitu
de (d
B)
100 1k 10k 20kFrequency (Hz)
-40
-35
-30
-25
-20
-15
-10
-5
0
5
10
Mag
nitu
de (d
B)
Välimäki, Parker, Esqueda & Alary15.2.2019
22
-
12
Phaser Measurements• Time-varying behavior of pedal can be measured using an allpass chirp train
• Modulation patterns and notch locations can be extracted from measurements
• Case Study: Fame Sweet Tone Phaser (MXR 100 clone)
R. Kiiski, F. Esqueda and V. Välimäki, “Time-variantgray-box modeling of a phaser pedal”, DAFx-16.
Välimäki, Parker, Esqueda & Alary15.2.2019
23
Gray-Box Phaser Model• Same topology as previous example.
• Measurement-based, modulation of a1 restricted to frequencies of interest.
• Measurement also exhibited LFO waveform.
Sound examples by Ricardo Falcón and Aleksi Myöhänen, ASP 2017
Original100 1k 10k 20k
Frequency (Hz)
-40
-35
-30
-25
-20
-15
-10
-5
0
5
10
Mag
nitu
de (d
B)
100 1k 10k 20kFrequency (Hz)
-40
-35
-30
-25
-20
-15
-10
-5
0
5
10
Mag
nitu
de (d
B)
a1 = –0.8 a1 = –0.4
Välimäki, Parker, Esqueda & Alary15.2.2019
24
-
13
Gray-Box Phaser Model
Välimäki, Parker, Esqueda & Alary15.2.2019
R. Kiiski, F. Esqueda and V. Välimäki, “Time-variantgray-box modeling of a phaser pedal”, DAFx-16.
25
Flanging vs. Phasing
Flanging Variable time-delay Short delay (< 10ms) Hundreds of notches Notches harmonically related Number of notches is time-
varying
Phasing Variable phase shift Very short delay Few notches (1-10) Notches not harmonic Notches can be individually
modulated Number of notches is fixed
Välimäki, Parker, Esqueda & Alary15.2.2019
26
-
14
Flanging or Phasing?
Flanger Phaser Phaser
Flanger Phaser Flanger
Välimäki, Parker, Esqueda & Alary15.2.2019
27
DEMO:Beat-Aligning LooperJon & Petteri
28
-
15
Chorus The goal: make one source sound like many sources
Useful for vocals and electrical instrument sounds Very similar structure to flanger and echo effects
Välimäki, Parker, Esqueda & Alary15.2.2019
29
Famous Chorus Examples
Chorus Unit: EHX Small Clone
Chorus Unit: Boss Chorus CE-2
Välimäki, Parker, Esqueda & Alary15.2.2019
30
-
16
Chorus Implementations (1) One choice: multiple feedforward paths with modulated
delay-lines (Orfanidis, 1996) Modulation waveforms may be sinewaves or lowpass-filtered
noise (“random walk” signal)
Välimäki, Parker, Esqueda & Alary15.2.2019
31
Chorus Implementations (2) “Industry standard” (Dattorro, 1997)
Välimäki, Parker, Esqueda & Alary15.2.2019
32
-
17
Chorus Implementations (3) The “industry standard” structure is cheap to implement
Use one for each stereo channel, or more Generalized allpass-comb filter
Becomes an allpass filter, when delays and coefficients are equal
Negative feedback is used for flattening the spectrum (“white chorus”)
For clean effect, allpass fractional delay filter must be used for the variable delay, not linear interpolation
Välimäki, Parker, Esqueda & Alary15.2.2019
33
Chorus Implementations (4) The industry standard structure can produce many effects
Vibrato: blend = 0, feedback gain = 0, large modulation depth Flanger: small delay (< 10 ms) Doubling (double tracking) when blend = feedforward gain, feedback
gain = 0, large delay (> 10 ms), with random modulation Echo: feedback or feedforward gain is zero; a lowpass filter is inserted
in the non-zero path; delay is large (> 50 ms) Stereo effects: modulating sine waves out of phase or in quadrature
for the 2 channels
Crazy cartoon-like effects!Great reference for this:
Udo Zölzer’s DAFX Book.
Välimäki, Parker, Esqueda & Alary15.2.2019
34
-
18
Chorus vs. Flanging Flanging
Small time delays (5ms) Separate signals perceived Min. delay approx 5 ms Notches usually
undesirable
Välimäki, Parker, Esqueda & Alary15.2.2019
35
Pitch Shifting
• Multiply all the frequencies with a constant value• Can be done with resampling (reading a buffer faster),
but it changes the playback speed as well– 1.5 x
36
-
19
Time Stretching using Phase Vocoder• Use the short-time Fourier transform• Reconstruct the signal with a larger
hop size• Makes a good pitch shifter
– Pitch up and time stretch to preserve time
• What happens to the phase?
https://cycling74.com/tutorials/the-phase-vocoder-%E2%80%93-part-i
38
Time Stretching
• Stretching the duration of a signal while retaining the pitch
39
-
20
Sound microscope
Damskägg, E.-P.; Välimäki, V. Audio Time Stretching Using FuzzyClassification of Spectral Bins. Appl. Sci.2017, 7, 1293.
• Decomposes sound into threeclasses:
1. Tonal
2. Transients
3. Noise
40
Sound Microscope
• Tonal layer:– Must restore
phase continuity
41
-
21
Sound Microscope
• Transient layer:– Must avoid repeating
or smearing
42
Sound Microscope
• Noise layer:– Must avoid
windowing artefacts
43
-
22
Dynamic range processing
44
Dynamics Processing Compressor reduces the dynamic range of an audio signal
Pop music, radio and TV broadcasting, live PA systems Special case: Limiter, which saturates at a certain max. level
Expander increases the dynamic range For example to reduce background noise in silent passages Special case: Gate, which mutes the signal below a threshold
Compressor
x0 x (dB)
Limitery0
y (dB)
Expander
x (dB)
Gate
y0
y (dB)
x0
Välimäki, Parker, Esqueda & Alary15.2.2019
45
-
23
Time-Domain View of Compression When signal level gets high, the gain is reduced
automatically 2 parameters: attack time TA and release time TR
Välimäki, Parker, Esqueda & Alary15.2.2019
46
Digital Feed-forward Compressor Signal level detector
Temporal envelope: average power over a short time interval Gain must not be changed instantaneously (aliasing can occur)
Gain computer Gain G is adjusted based on signal level (power)
Level detector
Välimäki, Parker, Esqueda & Alary15.2.2019
47
-
24
Level Detector Full-wave rectification (abs) and temporal averaging
For example, a leaky integratory(n) = (1 – a1) |x(n)| + a1 y(n – 1)where a1 = 1 – ε (such as a1 = 0.99)
Välimäki, Parker, Esqueda & Alary15.2.2019
48
Compression of Musical Signals After compressing the signal, the overall gain is
increased by applying make-up gain Many potential uses
Maximizing loudness, like in the recent “loudness war” Controlling transients/emphasizing decay
Välimäki, Parker, Esqueda & Alary15.2.2019
49
-
25
Multiband Compression• Split audio signal into sub-bands for improved performance.• Avoid “pumping” effect.
Demo by Tae Ho Kim and Elias Raninen, ASP-2016
comp
comp
comp
comp
Välimäki, Parker, Esqueda & Alary15.2.2019
50
Gating and Limiting of Musical Signals Limiting is an extreme version of a compressor
Signal value is not allowed to exceed a certain level Mainly used to maximize loudness
Gating mutes audio when below a certain threshold amplitude Used to e.g. remove background noise between notes Can also be used creatively (e.g. gate chord sound based on
amplitude of high-hats)
Välimäki, Parker, Esqueda & Alary15.2.2019
51
-
26
Gating and Limiting Examples
The outcome can sound very loud but lacking natural dynamics Original music file Gated (< -20 dB) Hard limited (> -10 dB) Gated (< -40 dB) & limited (> -20dB)
Audio examples by Mika Luukkanen, TKK, 2004
Välimäki, Parker, Esqueda & Alary15.2.2019
52
Audio Antiquing
• Render a new recording to sound aged For example, imitate the lo-fi sound of LP,
gramophone, or phonograph recordings• Simulate degradations with signal processing techniques
(González, thesis 2007; Välimäki et al., JAES 2008) Local degradations: clicks and thumps (low-frequency pulses) Global degradations: hiss, wow, distortion, limited dynamic range, frequency band limitations, resonances
© 2013 Vesa Välimäki
53
-
27
Audio Antiquing Example
1. CD (original)2. Phonograph cylinder (new – best quality)3. Phonograph cylinder (worn)
© 2013 Vesa Välimäki
54
Phonograph and Gramophone Simulation
© 2013 Vesa Välimäki
55
• Insert local and global disturbances Local disturbances: clicks, thumps, tracking errors (editing) Global (all samples affected): background noise, pitch variation,
distortion, coloration (e.g. bandlimiting, acoustic horn modeling), dynamic range limitation
-
28
Other Effects (not exhaustive) Vocoder
Technique from speech coding misused for effect. Uses large banks of bandpass filters to analyse the spectral envelope of a
sound and apply it to another sound Sounds like robot voice
Frequency shifting (do not confuse with pitch-shifting) Shifts all frequencies by a fixed amount additively, hence ruining harmonic
relationships Can also be used to produce very rich chorus sounds
Autotune (1998-) Highly popular pitch corrector with quantization (“Cher”) Uses LPC and interpolation, or spectral techniques
Välimäki, Parker, Esqueda & Alary15.2.2019
56
Other Effects (not exhaustive) Wah-wah
Center frequency of a resonator is modulated with envelope follower, control pedal, or LFO
Filtering (telephone sound, resonances etc.) Enhancer
Harmonic distortion of only high frequencies to increase brightness
Distortion and tube-amp modeling effects Reverberations effects Spatial audio effects Stereo expansion, 3-D sound etc. More?
Välimäki, Parker, Esqueda & Alary15.2.2019
57
-
29
Välimäki, Parker, Esqueda & Alary15.2.2019
Resources
58
Literature – Effects E.-P.Damskägg and V. Välimäki, ”Audio time stretching using fuzzy classification
of spectral bins,” Applied Sciences, 2017, 7, 1293. J. Dattorro, “Effect design—part 2: Delay line modulation and chorus,” J. Audio
Eng. Soc., vol. 45, no. 10, pp. 764–788, Oct. 1997. Available online at:http://www.stanford.edu/~dattorro/research.html
R. Dobson, A Dictionary of Electronic & Computer Music Technology. OxfordUniversity Press, 1992.
W. M. Hartmann, “Flanging and phasing,” J. Audio Eng. Soc., vol. 26, no. 6, pp.439–443, June 1978.
S. J. Orfanidis, Introduction to Signal Processing. Prentice-Hall, 1996. Section8.2 (“Digital Audio Effects”), pp. 355-388.
G. D. White, The Audio Dictionary (2nd ed.). University of Washington Press,1991.
D. Giannoulis, M. Massberg and J. D. Reiss, ”Digital dynamic range compressordesign – A tutorial and analysis”, J. Audio Eng. Soc., vol. 60, no. 6, June 2012,pp. 399–408
Välimäki, Parker, Esqueda & Alary15.2.2019
59
-
30
Literature – Effects (2) • R. Kiiski, F. Esqueda & V. Välimäki, “Time-variant gray-box modeling of a phaser
pedal,” in Proc. 19th Int. Conf. Digital Audio Effects (DAFx-16), pp. 31–38, Brno,Czech Republic, Sept. 2016.
• V. Välimäki, S. González, J. Parviainen, and O. Kimmelma, ”Digital audioantiquing – Signal processing methods for imitating the sound quality ofhistorical recordings,” Journal of the Audio Engineering Society, vol. 56, no. 3,pp. 115–139, March 2008.
Välimäki, Parker, Esqueda & Alary15.2.2019
60