WP4 – Sound Object Identification

WP5 – Enriched Access Tools

18 Month Biannual

Technical Meeting

Workpackage 5 - Tasks

WP5 Partner Role

DIT (Lead) - Pitch / timescale modification, - Source Separation**, - EQ, - Noise Reduction.

QMUL - Looping and marking, - Video TSM,

NICE - Speech enhancement tools, - subjective quality tools

Total 80 PM **developed in WP4 but ultimately an access tool also.

18 Month Biannual

Technical Meeting

Deliverables and MilestonesMonth 1 5 10 15 20 25 30

T5.3 Multimedia AlignmentVideo + Audio TSM 90% Complete

T5.1 Looping and MarkingD5.1 M9100% Complete

T5.2 Time and Pitch Scale ModificationD5.2 M2790% Complete

D1 D2 D3

T5.4 Sound Enhancement & NRD5.3 M3060% Complete

1 5 10 15 20 25 30

M5.2 Initial prototype integrated with interfaces (Month 18)

You AreHere

i

i

i

x

T4.2 Sound Source SeparationD4.2 M1490% Complete x

i = integrated

18 Month Biannual

Technical Meeting

Real-time engine integration

Load Frame

Interpolate

Window + FFT

Is Transient ?

Update Phases

IFFT + window

75% RT Overlap

++ SynHop x TSM ++ SynHop x 4

75% RT Overlap

IFFT + window

Yes

Real-time Access Tools

Source Separation

Equalisation

Noise Reduction

PSM & TSM factor

Real-time Engine

EASAIER HOST APP.

Audio

UI control

Clo

ck F

eedb

acki

18 Month Biannual

Technical Meeting

Workpackage Progress

TSM/PSM – successfully integrated and operational Video synchronised TSM integrated and operational

i

TSM

VTSM

18 Month Biannual

Technical Meeting

Further Modification

Link option for time and pitch scaling

Pitch change via transposition (requires Key sig estimate)

Time change via tempo change (requires tempo estimate)

18 Month Biannual

Technical Meeting

Workpackage Progress

Sound Source Separation

Source separation tool is fully developed and ready for integration

Problem: DIT IP regulations will not allow code to be open source

Solution: DLL for windows release and shared library for Linux release

18 Month Biannual

Technical Meeting

Workpackage Progress

Equalisation / Enhancement

Equalisation tool is fully developed and ready for integration

The equalisation tool can be used to filter out tones and hums which are often present in archival formats such as magnetic tape.

Alternately, the tool can be simply used to tailor the sound to a listeners taste and playback system

18 Month Biannual

Technical Meeting

Workpackage Progress

Equalisation / Enhancement

• Simple mode interaction uses 5 controls to increase or decrease the level of 5 fixed bands

• Advanced mode allows user to view the spectrum and draw the desired curve onto spectrum

• Demo to follow

• Integration – Mid December 07’

18 Month Biannual

Technical Meeting

Upcoming Work Plan Months 19-30

• Integrate EQ module

• Integrate Source Separation

• Work is ongoing w.r.t. “intelligent noise reduction” algorithms

• Evaluation / Benchmarking / User testing.

• Journal Publication planned for VTSM

18 Month Biannual

Technical Meeting

Demonstrations

1. Integrated pitch and timescale modification

2. Integrated video/audio time and pitch scale modification

3. Prototype restoration EQ and simple EQ

18 Month Biannual

Technical Meeting

Questions ?

Discussion