Digital Sound and Video Chapter 10, Exploring the Digital Domain.
-
Upload
roger-price -
Category
Documents
-
view
225 -
download
0
Transcript of Digital Sound and Video Chapter 10, Exploring the Digital Domain.
![Page 1: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/1.jpg)
Digital Sound and Video
Chapter 10,Exploring the Digital Domain
![Page 2: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/2.jpg)
Digital Sampling of Sound
Digital sound is sound that has been converted to, or created in, a discrete form (namely a set of numeric values)
Natural sound is a continuous phenomena and is converted to digital form by sampling techniques
Properties of sound waves amplitude (measure of loudness) frequency (measure of pitch)
![Page 3: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/3.jpg)
Continuous Sound Wave
![Page 4: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/4.jpg)
Sampling Sound
Temporal sampling sample rate resolution
Analog-to-digital converters (ADCs) Digital-to-analog converters (DACs)
![Page 5: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/5.jpg)
Sampling of a Sound Wave Illustrated
![Page 6: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/6.jpg)
Sampling of a Sound Wave Illustrated (cont’d)
![Page 7: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/7.jpg)
Sampling of a Sound Wave Illustrated (cont’d)
![Page 8: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/8.jpg)
Resolution and Dynamic Range
Resolution depends on how much memory is devoted to storing individual sample amplitudes 8-bit sound 16-bit sound
Dynamic range and clipping
![Page 9: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/9.jpg)
Resolution Illustrated
![Page 10: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/10.jpg)
Storing Digital Sound
Sample rate and resolution -- tradeoffs
Voice and speech: 8-bit resolution and 5-10 KHz sample rate
CD-quality music: 16-bit or higher resolution and 44 KHz
Nyquist’s “Theorem” and aliasing Audio file compression Sound on the Web -- streaming audio
![Page 11: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/11.jpg)
Synthesizing Music
Uses simple waveforms and oscillators to “build” more complex sound waves
Various techniques are employed to do this “build” Subtractive synthesis Additive synthesis FM synthesis Phase distortion synthesis Integrated synthesis
Impose an ADSR (attack, decay, sustain, release) envelope to simulate instruments
![Page 12: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/12.jpg)
Oscillators
![Page 13: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/13.jpg)
ADSR Envelope
![Page 14: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/14.jpg)
MIDI Instruments and Devices
MIDI is a standard interface between electronic musical instruments and synthesizers
Devices which are MIDI-compatible can communicate with each other
Advantages to MIDI files are encoded and are much smaller than
digitized sound files files can be easily edited and mixed for
multiple tracks
![Page 15: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/15.jpg)
Speech Synthesis
Speech synthesis involves creating speech from written text (or other encodings)
Two approaches Store digitized recordings of words Analysis of written text focuses on
breaking the text into phonemes
![Page 16: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/16.jpg)
Digitized Word Approach Parser separates text into words Uses a binary search tree to look up words Similar to a spelling dictionary – instead an
enunciation dictionary Advantage is very realistic sound Disadvantage is lack of speed
Works well if the number of words are limited Grocery checkout Alert systems
Not very useful for real-time “reading” where the words in the text is not know in advance
![Page 17: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/17.jpg)
Rule-Based Phoneme Approach
Analysis of written text focuses on breaking the text into phonemes rather than words Text is parsed for phonemes Phonemes are identified Enunciation is looked up in a small
indexed file English employs approximately 50
basic phonemes
![Page 18: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/18.jpg)
Phoneme Approach (cont’d)
Rules allow a speech synthesis program to evaluate alternate pronunciations of phonemes appropriate for the context
Such rule-based phoneme analysis produces excellent speech synthesis results
![Page 19: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/19.jpg)
Speech Recognition
Speech recognition attempts to interpret digitized speech for meaning
The task is complicated by the differences among speakers and even the different ways a given speaker might pronounce the same word depending on mood, context, etc.
![Page 20: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/20.jpg)
Speech Recognition:Illustrating the Difficulties
![Page 21: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/21.jpg)
Speech Recognition (cont’d)
Some success has been achieved by tailoring/training a program to recognize a particular speaker
Some reasonably successful voice activation systems have been produced where vocabulary is limited to small number of words
Speech recognition remains a very challenging problem
![Page 22: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/22.jpg)
Editing Digitized Sound
Digital sound editing is part of a larger field called digital signal processing
Once sound is digitized, it is in discrete numerical format
Numerical transformations on this data can be used to: change the sound’s pitch change the sound’s amplitude add echoes and other special effects
![Page 23: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/23.jpg)
Summary -- Sound
Digital sound is produced by sampling sound waves over time
A digital sound file consists of sampled amplitudes at a number of discrete times within a given time interval
The number of samples per second is called the sample rate
The number of bits devoted to storing individual sampled amplitudes is called the resolution of the digitized sound: 8-bit, 16-bit and higher resolutions are used depending on the kind of sound being digitized
Fidelity will be largely determined by the sample rate and resolution
![Page 24: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/24.jpg)
Producing Digital Video
Video capture Editing Playback
![Page 25: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/25.jpg)
Digital Video The Process Illustrated
![Page 26: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/26.jpg)
Digital Video:Advantages and Disadvantages
Advantages Scalable to different playback systems Random access to frames Nonlinear editing More playback options Potential for interactivity
Disadvantages Special hardware/software needed for
production High playback and storage requirements
![Page 27: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/27.jpg)
Compression: Coping with Large Files
Compression is an encoding process that filters the original file in several successive stages
![Page 28: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/28.jpg)
Other Methods for Reducing Demands Frame rate adjustment
adjusts for with slower CPUs helps keep video and audio synchronized
Lower resolution on individual frames sometimes used in conjunction with
smaller display window
![Page 29: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/29.jpg)
The Desktop Video SystemBasic Components
Analog Source Video Capture Card CPU Secondary Storage Monitor Edit and Playback
Control
![Page 30: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/30.jpg)
Editing Digital Video
Clip Logging Assembling Transitions
dissolves wipes, etc.
Rotoscoping Compositing
keying titling
![Page 31: Digital Sound and Video Chapter 10, Exploring the Digital Domain.](https://reader035.fdocuments.in/reader035/viewer/2022081512/56649de55503460f94addd0b/html5/thumbnails/31.jpg)
Summary --Video
Digital video is: scalable allows nonlinear editing has interactive potential
Digital video can be produced with desktop systems
Flexible editing and playback options are major advantages
Storage requirement is biggest disadvantage