December 2006 Cairo University Faculty of Computers and Information HMM Based Speech Synthesis...

December 2006

Cairo UniversityFaculty of Computers and Information

HMM Based Speech Synthesis

Presented byOssama Abdel-Hamid Mohamed

2 HMM Based Speech Synthesis

Agenda

Speech SynthesisHMM Based Speech SynthesisProposed SystemChallenges


Speech Synthesis

What is speech synthesis?– Generating human like speech using computers.

Applications– Text To Speech.

– Conversation systems.

– Speech to speech translation.

– Concept to speech.

Systems built since late 1970s.– MITTALK 1979

– Klattalk 1980


Speech Synthesis, Cont.

Challenges:– Intelligibility.

– Naturalness.

– Pleasantness.

– Emotions.


Speech Synthesis, Techniques

•Techniques

•Formant Based •Concatenative HMM Based

•Rule Based

•Difficult to make

•Machine Like

•Instance Based

•Based on corpus

•Better quality

•Not flexible

•Statistical Based

•Based on corpus

•Newest technique

•More flexible


Agenda



HMM Based Speech Synthesis Overview

HMM has been used successfully in speech recognition.

In Recogntion

In Speech Synthesis:

)|(maxarg* OPOO

)|(maxarg*

OP


HMM Based Speech Synthesis Overview, Cont. Include delta and acceleration to get smooth

output


The Overall System

Synthesis Part

Speech Database F0

ExtractionMel-Cepstral

Analysis

HMM Training

Models

Labels and context features

Text Analysis

Text

Text Analysis Parameters Generation


Pulse or Noise Excitation

f0

MLSA filter Speech

Mel-cepstrum

Excitation

Mel-cepstrum

f0

Training Part


The Overall System

Synthesis Part

Speech Database F0


Analysis

HMM Training

Models


Text Analysis

Text




f0

MLSA filter Speech

Mel-cepstrum

Excitation

Mel-cepstrum

f0

Training Part

Modeled using MSD-HMM 25 Mel-Cepstral


The Overall System

Synthesis Part

Speech Database F0


Analysis

HMM Training

Models


Text Analysis

Text




f0

MLSA filter Speech

Mel-cepstrum

Excitation

Mel-cepstrum

f0

Training Part

Context Dependant Models

Each model 5 States


The Overall System

Synthesis Part

Speech Database F0


Analysis

HMM Training

Models


Text Analysis

Text




f0

MLSA filter Speech

Mel-cepstrum

Excitation

Mel-cepstrum

f0

Training Part


The Overall System

Synthesis Part

Speech Database F0


Analysis

HMM Training

Models


Text Analysis

Text




f0

MLSA filter Speech

Mel-cepstrum

Excitation

Mel-cepstrum

f0

Training Part

Each Frame is either voicedor unvoiced


The Overall System

Synthesis Part

Speech Database F0


Analysis

HMM Training

Models


Text Analysis

Text




f0

MLSA filter Speech

Mel-cepstrum

Excitation

Mel-cepstrum

f0

Training Part


Advantages

1. Its voice characteristics can be easily modified,

2. It can be applied to various languages with little modification,

3. A variety of speaking styles or emotional speech can be synthesized using the small amount of speech data,

4. Techniques developed in ASR can be easily applied,

5. Its footprint is relatively small. An HMM based TTS system produced best

results in Blizzard challenge.


Agenda



Problems we tried to solve

1. Marking each frame as either voiced or unvoiced degrades quality, because there are some unvoiced components on most voiced speech parts, and there are mixed-excitation phonemes.

2. Used speech signal analysis / synthesis techniques and parameters degrades quality.


Multi-Band Excitation

In MBE (Multi-Band Excitation) speech is divided into a number of frequency bands, and voicing is estimated in each band (used 17 bands).


Mixed Excitation

In synthesis periodic and noise excitations are mixed according to voicing parameters


Spectral Envelop Estimation

Find values for a fixed number of samples

Use sinusoidal model for synthesis


Modified System

Synthesis Part

Speech Database F0

ExtractionSpectral Envelop

Analysis

HMM Training

Models


Text Analysis

Text



Spectral Envelop Samples

f0

Training PartBands Voicing

detectionBands Voicing

Noise + STFT filter

HarmonicsSynthesis

Bands Mixing

Spec. Env. Samples+ f0

Bands Voicing

Voiced Speech

Unvoiced Speech Speech


Result

MOS scores

1

1.5

2

2.5

3

3.5

4

4.5

5

BaselineSystem

Baseline +MBE

ProposedSystem

Sc

ore


Agenda



Other Challenges

Speech is overly smoothed– Use global variance.

Modeling accuracy, the system uses same modeling as recognition.

– Hidden semi markov models (duration).

– Trajectory HMMs,

– Minimum Generation error training

– More states clusters and use acoustic context (under research).


More States Clusters

Instead of computing one Gaussian per state, we store all occurrences. And record the context of each occurrence.

At synthesis we get the best sequence using dynamic programming.

Previous NextCurrent

…


Thank You

December 2006 Cairo University Faculty of Computers and Information HMM Based Speech Synthesis...

Documents

Transcript of December 2006 Cairo University Faculty of Computers and Information HMM Based Speech Synthesis...