Pal - COnnecting REpositories · is close interaction with MIT Lincoln Labora-tory and Woods Hole...

Pal

~d~A

217

I

a i

"s i

218 RLE Progress Report Number 131

Section 1

Chapter

Chapter

Chapter

Chapter

Chapter

Digital Signal Processing

Digital Signal Processing.

Cognitive Information Processing.

Optical Propagation and Communication.

Custom Integrated Circuits.

Neurophysiology and Neural Computation.

219

Chapter 1. Digital Signal Processing


Academic and Research StaffProfessor Alan V. Oppenheim, Professor Jae S. Lim, Professor Bruce R. Musicus, ProfessorArthur B. Baggeroer, Dr. Meir Feder, Giovanni Aliberti

Visiting ScientistDr. Ehud Weinstein'

Collaborating ScientistProfessor Ramesh Patil2

Graduate StudentsMatthew M. Bace, Joseph E. Bondaryk, Michael S. Brandstein, Lawrence M. Candell, DanielT. Cobra, Michele M. Covell, Dennis C.Y. Fogg, John C. Hardwick, Steven H. Isabelle, JacekJachner, Tae H. Joo, Rosalind W. Picard, G.N. Srinivasa Prasanna, James C. Preisig, MichaelRichard, Gregory W. Wornell

Support StaffDeborah A. Gage, Cynthia LeBlanc

Part-Time Assistants/Special ProjectsAlfredo Donato, Hassan Gharavy

1.1 Introduction

The Digital Signal Processing Group is car-rying out research in the general area ofsignal processing. In addition to specificprojects being carried out on campus, thereis close interaction with MIT Lincoln Labora-tory and Woods Hole Oceanographic Institu-tion. While a major part of our activitiesfocus on the development of new algorithms,there is a strong conviction that theoreticaldevelopments must be closely tied to appli-cations. The application areas that we cur-rently are involved with are speech, image,video and geophysical signal processing. Wealso believe that algorithm developmentshould be closely tied to issues of imple-mentation because the efficiency of an algo-rithm depends not only on how manyoperations it requires, but also on how suit-

able it is for the computer architecture onwhich it runs. Also strongly affecting ourresearch directions is the sense that while,historically, signal processing has principallyemphasized numerical techniques, it willincreasingly exploit a combination of numer-ical and symbolic processing, a direction thatwe refer to as knowledge-based signal pro-cessing.

In the area of knowledge-based signal pro-cessing, there are currently two researchprojects. One involves the concept of sym-bolic correlation, which is concerned withthe problem of signal matching using mul-tiple levels of description. This idea is beinginvestigated in the context of vector codingof speech signals. Symbolic correlation will

1 Tel Aviv University, Faculty of Engineering, Tel Aviv, Israel.

2 MIT Laboratory for Computer Science.

221


entail the use of both symbolic and numericinformation to efficiently match a speechsignal with stored code vectors. The secondproject in this area deals with the represen-tation and manipulation of knowledge andexpressions in the context of signal pro-cessing. This work examines issues such asthe representation of knowledge, derivationof new knowledge from that which is given,and strategies for controlling the use of thisknowledge.

In the area of speech processing, we have,over the past several years, worked on thedevelopment of systems for bandwidth com-pression of speech, parametric speech mod-eling, time-scale modification of speech andenhancement of degraded speech. A newmodel-based speech analysis/synthesissystem has been developed and shown to becapable of high-quality speech production. Itis currently being used in several mid- andlow-rate speech coding systems. A codingsystem based on this model has achieved abit rate of 4.8 kbps while still maintaininghigh speech quality. Current work seeks toreduce the bit rate to 2.4 kbps. A differentmodel has been proposed with dualexcitation and filter structure to improvespeech processing performance. Researchon adaptive noise cancellation techniques ina multiple microphone environment has ledto an approach based on maximum likelihoodestimation that has shown substantialimprovements over previous techniques.

A number of interesting image processingprojects are currently being conducted. Onesuch project is examining texture regions inimages. The goal of this research is todevelop a model which is capable of synthe-sizing a continuum of structural andstochastic textures. Another project isfocusing on a new technique of motion esti-mation with improved performance at motionboundaries. Fuzzy segmentation informationis used to iteratively refine the estimates ofthe motion field. A third project seeks todevelop a receiver-compatible scheme toreduce the effects of channel imperfectionson the received television picture. Adaptivemodulation is being investigated in order tomake more efficient use of the currentlyunderutilized bandwidth and dynamic rangeof the NTSC signal. Several other image

processing projects have recently been com-pleted. These include the estimation ofcoronary artery boundaries in angiograms,motion compensation for moving pictures,and magnitude only reconstruction ofimages.

In the area of geophysical signal processing,one research project involves the transforma-tion of side-scan sonar data. In practice thisdata is corrupted by a number of factorsrelated to the underwater environment. Inthis project, the goal is to explore digitalsignal processing techniques for extractingthe topographic information from the actualsonographs. Concepts under study includethe removal of distortions caused by towfishinstability and reconstruction based on mul-tiple sonographs taken from different angles.A second project involves near-fieldbeamforming for underwater acoustics.

There are also a number of projects directedtoward the development of new algorithmswith broad potential applications. For sometime we have had considerable interest in thebroad question of signal reconstruction frompartial information, such as Fourier transformphase or magnitude. We have shown the-oretically how, under very mild conditions,signals can be reconstructed from Fouriertransform phase information alone. Thiswork has also been extended to the recon-struction of multidimensional signals fromone bit of phase and exploiting duality, zerocrossing and threshold crossing information.Reconstruction from distorted zero crossingshas been studied as well as reconstructionfrom multiple threshold crossings which hasbeen shown to be a better-conditionedproblem than reconstruction using only asingle crossing.

We are also examining the problem ofnarrowband signal detection in widebandnoise, in particular in the presence of relativemotion between transmitter and receiver.New algorithms are applied to the detectionof gravitational waves generated by theorbiting motion of binary star systems.Another area of research is the estimation ofthe angle of arrival of multiple sources to anantenna array which is in motion. Algo-rithms are pursued which are computationallyefficient for general antenna geometries, withapplication in radar and sonar systems.



Research also continues on the relationshipbetween digital signal processing andstochastic estimation. We are exploringalgorithms which can be applied to statisticalproblems, iterative signal reconstruction,short-time analysis/synthesis, and parameterestimation. The theory of functional approxi-mation using generalized polynomials isbeing studied for application to windowingof arrays for near-field beamforming, win-dowing of time series with non-uniformsamples, windowing of non-uniform andmultidimensional arrays, and mask design forx-ray lithography.

A new area of interest is chaotic dynamicalsystems. Present work is aimed at applyingnon-linear dynamical system theory andfractal process theory to digital signal pro-cessing scenarios. Self-similar or fractalsignal models are being investigated andnovel spread spectrum techniques in the useof iterated cellular autonoma for modelingsignals. The chaotic dynamics at work inquantization and overflow problems are alsobeing explored.

With the advent of VLSI technology, it isnow possible to build customized computersystems of astonishing complexity for verylow cost. Exploiting this capability, however,requires designing algorithms which not onlyuse few operations, but also have a highdegree of regularity and parallelism, or canbe easily pipelined. The directions we areexploring include systematic methods fordesigning multi-processor arrays for signalprocessing, isolating signal processing primi-tives for hardware implementation, andsearching for algorithms for multidimensionalprocessing that exhibit a high degree ofparallelism. We are also investigating highly-parallel computer architectures for signalunderstanding, in which a mixture of inten-sive computation and symbolic reasoningmust be executed in an integrated environ-ment. Another research direction involvescomputer-assisted design of signal pro-cessing chips. The approach taken is to usedesign specifications that include multiple,interacting performance specifications andgenerate designs at a multitude of perform-ance levels to present possible trade-offsamong the designs.

1.2 Receiver-CompatibleAdaptive Modulation forTelevision

SponsorsNational Science Foundation FellowshipNational Science Foundation

(Grants ECS 84-07285and MIP 87-14969)

U.S. Navy - Office of Naval Research(Contract N00014-81 -K-0742)

Project Staff

Professor Jae S. Lim, Matthew M. Bace

There have been numerous proposals formethods to improve upon the quality of thecurrent NTSC television picture. Most ofthese proposals have concentrated onmethods for increasing either the spatial orthe temporal resolution of the televisionpicture. While these proposals promise sig-nificant improvements in picture quality, thefact remains that until an effective scheme tocombat channel noise has been introduced,these improvements will never be fully real-ized. Degradations such as random noise("snow"), echo, and intersymbol interference(channel crosstalk) are still the greatestbarrier to high-quality television.

This research will attempt to develop areceiver-compatible scheme to reduce theeffects of channel imperfections on thereceived television picture. In particular, themethod of adaptive modulation will beemployed in an attempt to make more effi-cient use of the currently underutilized band-width and dynamic range of the NTSCsignal. By concentrating more power in thehigher spatial frequencies and using digitalmodulation to send additional information inthe vertical and horizontal blanking periods,the existing television signal can be alteredso that it is more robust in the presence ofhigh-frequency disturbances. Furthermore, itis possible to adjust the parameters of thisscheme so that the modulated signal may bereceived intelligibly even on a standardreceiver (although an improved receiver willbe required to realize the full benefits ofadaptive modulation).

223


Before a conclusion can be reached as towhich adaptive modulation scheme isoptimal, many details must be considered.Among the parameters which may be variedare: the control over the adaptation and com-pression factors, the form of the input low-pass filters, the interpolation scheme to beused at both the transmitter and the receiver,and the encoding of the digital data. Theseparameters will be adjusted to optimize theperformance of the modulation scheme withrespect to two fundamental performance cri-teria: the degree to which the channel degra-dations are removed when the signal isreceived on an improved receiver and thedegree to which the signal is distorted whenreceived on a standard receiver.

1.3 Reconstruction OfNonlinearly Distorted ImagesFrom Zero Crossings

SponsorsNational Science Foundation

(Grant ECS 84-07285)U.S. Navy - Office of Naval Research

(Contract N00014-81 -K-0742)

Project Staff

Professor Alan V. Oppenheim, Joseph E.Bondaryk

It has been shown theoretically that band-limited, multidimensional signals can bespecified to within a constant factor by theinformation contained in the locations oftheir zero crossings. It has been shownexperimentally that two-dimensional, band-limited signals can be reconstructed to withina constant factor from zero crossing informa-tion alone. The two-dimensional signals usedwere derived from images and their zerocrossings corresponded to the thresholdcrossings of the images.

In this research, the problem considered isthat of two-dimensional, bandlimited signalswhich have been affected by memoryless,nonlinear distortions. It is shown that suchdistortions retain the information required bythe above theory, if they contain amonotonic region. Therefore, reconstructionto within a scale factor of an original, band-

limited image from the threshold crossinginformation of a distorted image is made pos-sible. The zero crossing coordinates of thetwo-dimensional signal derived from a dis-torted image are substituted into a FourierSeries representation of the original signal toform a set of homogeneous, linear equationswith the Fourier coefficients of the originalsignal as unknowns.

The least squares solution to this set ofequations is used to find the Fourier coeffi-cients of the original signal, which areinverse Fourier transformed to recover theoriginal image. By comparison of the dis-torted and reconstructed images, the natureof the distortion can be described. Some ofthe numerical problems associated with thereconstruction algorithm are also considered.

The reconstruction process is particularlystable for images which have bandedgeFourier components of high magnitude. It isshown that reconstruction from the zerocrossing information of these images issimilar to reconstruction from halftones,binary-valued images used to representimages which contain a continuous range oftones. This theory is finally extended toinclude reconstruction of bandlimited imagesfrom the zero crossings of distortedhalftones.

This research was completed in May 1988.

1.4Kbps

Development of a 2.4Speech Vocoder

Sponsors

National Science Foundation FellowshipNational Science Foundation

(Grant MIP 87-14969)U.S. Navy - Office of Naval Research

(Contract N00014-81 -K-0742)

Project Staff

Professor Jae S. Lim, Michael S. Brandstein

The recently developed Multi-BandExcitation Speech Model has been shown toaccurately reproduce a wide range of speechsignals without many of the limitationsinherent in existing speech model based



systems.3 The robustness of this model makesit particularly applicable to low bit rate, high-quality speech vocoders. Griffin and Lim 4

first described a 9.6 Kbps speech coderbased on this model. Later work resulted ina 4.8 Kbps speech coding system.5

Both of these systems have been shown tobe capable of high-quality speech reprod-uction in both low and high SNR conditions.

The purpose of this research is to exploremethods of using the new speech model atthe 2.4 Kbps rate. Results indicate that asubstantial amount of redundancy existsbetween the model parameters. Currentresearch is focused on exploiting theseredundancies so as to quantize these param-eters more efficiently. Attempts are alsounderway to simplify the existing modelwithout significant reduction in speechquality.

1.5 Correction of GeometricDistortions in Side-ScanSonographs Caused by SonarMotion Instabilities

Sponsors

Scholarship from the Federative Republicof Brazil

National Science Foundation(Grants ECS 84-07285and MIP 87-14969)

U.S. Navy - Office of Naval Research(Contract N0001-81-K-0742)

Project Staff

Professor Alan V. Oppenheim, Daniel T.Cobra

Since its introduction in the early sixties,side-scan sonar has proved to be a very

important tool for underwater exploration,and in particular for marine geology. Itsapplications include surveying the seafloor,search and location of objects on the bottomof the sea, and prospection of mineraldeposits.

The information contained in reflected soundwaves is used by side-scan sonar to producean acoustic image, called a sonograph, whichconstitutes a composite representation of thetopographic features and the relativereflectivity of the various materials on theseabed. Due to several factors, however,sonographs do not provide a precisedepiction of the topology. The image cansuffer from radiometric interferences such asthose caused by dense particle suspension inthe water, shoals of fish, or by ultrasonicwaves generated by passing ships. Large-scale geometric distortions can be caused bydeviations in the ship's trajectory from theideal straight path. Small-scale distortionscan be caused by motion instability of thetowing body on which the transducers aremounted, due to underwater currents or tothe ship's sway being communicated to thetowed body. As a result, the interpretation ofsonographs often requires extensive practiceand can be a tedious and time-consumingtask.

Radiometric distortions and large-scalegeometric distortions have been successfullycorrected through standard image-processingtechniques. Our goal is to develop tech-niques to detect and correct small-scalegeometric distortions due to sonar motioninstabilities by digital post-processing ofsonographs. This is a problem which hithertohad not been successfully addressed in theliterature. This project is being conductedunder MIT's joint program with the WoodsHole Oceanographic Institution, with thecooperation of the U.S. Geological Survey.

3 D.W. Griffin and J.S. Lim, "A New Model-Based Speech Analysis/Synthesis System," Proceedings of the IEEEInternational Conference on Acoustics, Speech, and Signal Processing, Tampa, Florida, March 26-29, 1985, pp.513-516.

4 D.W. Griffin and J.S. Lim, "A High Quality 9.6 kbps Speech Coding System," IEEE International Conference onAcoustics, Speech, and Signal Processing, Tokyo, Japan, April 8-11, 1986.

5 J.C. Hardwick, A 4.8 Kbps Multi-Band Excitation Speech Coder. S.M. thesis, MIT, 1988.

225


1.6 Representation andManipulation of SignalProcessing Knowledge andExpressions

Sponsors



Project Staff

Professor Alan V. Oppenheim, Michele M.Covell

The phrase "signal processing" is used torefer to both "symbolic" and "numeric"manipulation of signals. "Symbolic" signalprocessing manipulates the signal descriptionas opposed to the signal values with which"numeric" signal processing is primarily con-cerned. Efforts have been made to createcomputer environments for both types ofsignal processing.' Some issues that arise asa result of this work concern uniform repre-sentation of knowledge, derivation of newknowledge from that which is given, andstrategies for controlling the use of thisknowledge. This research is concerned withthese areas and how they apply to digitalsignal processing.

Representations that have been used in sym-bolic signal processing7 have been largelydistinct from those used in numeric signalprocessing.8 The types of representationsused are further separated by the controlstructures that the numeric and symbolicinformation commonly assume, the dis-

tinction essentially being the same as the dis-tinction between Algol-like languages andlogic programming languages. Thisdichotomy results from the differing amountsof available knowledge about appropriateapproaches to the problems being addressed.By separating the control structure fromapplication knowledge, this dichotomy canbe avoided.

Strategies for controlling when knowledgeabout a signal is used should be providedand new strategies should be definable, sincethese control structures provide additionalinformation about the problem space,namely, approaches that are expected to beprofitable. Control strategies can also beused to outline new approaches to a problemwhich would not be considered by simpletrigger-activated reasoning.

Finally, the ability to derive new knowledgefrom that which is given is desirable. Thisability would allow the amount of informa-tion initially provided by the user to be mini-mized. The environment could increase itsdata base with new conclusions and theirsufficient pre-conditions. Two immediateadvantages of providing the environmentwith this ability are the reduction in the pro-gramming requirements and the possible"personalization" of the data base. A reduc-tion in programming requirements is availablesince information that is derivable from giveninformation need not be explicitly encoded.Commonly, this type of information is pro-vided to improve the performance of the deri-vation process. Secondly, since theenvironment would add information to thedata set according to conclusions promptedby the user's queries, the data set wouldexpand in those areas which the user hadactively explored.

6 C. Myers, Signal Representation for Symbolic and Numeric Processing. Ph.D. diss., Dept. of Electr. Eng. andComp. Sci., MIT, 1986; W. Dove, Knowledge-Based Pitch Detection. Ph.D. diss., Dept. of Electr. Eng. and Comp.Sci., MIT, 1986.

7 Ibid.; E. Milios, Signal Processing and Interpretation Using Multilevel Signal Abstractions. Ph.D. diss., Dept. ofElectr. Eng. and Comp. Sci., MIT, 1986.

8 C. Myers, Signal Representation for Symbolic and Numeric Processing. Ph.D. diss., Dept. of Electr. Eng. andComp. Sci., MIT, 1986; G. Kopec, The Representation of Discrete-Time Signals and Systems in Programs. Ph.D.diss., Dept. of Electr. Eng. and Comp. Sci., MIT, 1980.



1.7 Iterative Algorithms forParameter Estimation fromIncomplete Data and TheirApplications to SignalProcessing

Sponsors



Project Staff

Professor Alan V. Oppenheim, Dr. Meir Feder

Many signal processing problems may beposed as statistical parameter estimationproblems. A desired solution for the statis-tical problem is obtained by maximizing theLikelihood (ML), the A-Posteriori probability(MAP) or some other criterion, depending onthe a-priori knowledge. However, in manypractical situations the original signal pro-cessing problem may generate a complicatedoptimization problem, e.g., when theobserved signals are noisy and "incomplete."

An iterative framework for maximizing thelikelihood, the EM algorithm, is widely usedin statistics. In the EM algorithm, the obser-vations are considered "incomplete" and thealgorithm iterates between estimating thesufficient statistics of the "complete data"given the observations and a current estimateof the parameters (the E step), and maxi-mizing the likelihood of the complete data,using the estimated sufficient statistics (theM step). When this algorithm is applied tosignal processing problems, it yields, in manycases, an intuitively appealing processingscheme.

In the first part of this research, we investi-gate and extend the EM framework. Bychanging the "complete data" in each step ofthe algorithm, we can achieve algorithmswith better convergence properties. In addi-tion, we suggest EM type algorithms to opti-mize other (non ML) criteria. We alsodevelop sequential and adaptive versions ofthe EM algorithm.

In the second part of this research, weexamine some applications of this extendedframework of algorithms. In particular, weconsider

* Parameter estimation of compositesignals, i.e., signals that can be repres-ented as a decomposition of simplersignals. Examples include:

- Multiple source location (or bearing)estimation

- Multipath or multi-echo time delayestimation

* Noise cancellation in a multiple micro-phone environment (speech enhance-ment)

* Signal reconstruction from partial infor-mation (e.g., Fourier transform magni-tude).

The EM-type algorithms suggested forsolving the above "real" problems providenew and promising procedures, and theythus establish the EM framework as animportant tool to be used by a signal pro-cessing algorithm designer.

Portions of this work were supported in partby the MIT - Woods Hole OceanographicInstitution Joint Program. This research con-cluded in August 1988.

1.8 Assisting Design GivenMultiple Performance Criteria

Project Staff

Professor Bruce A. Musicus,Ramesh Patil, Dennis C.Y. Fogg

Professor

In this thesis, we explore ways for computersto assist designers of signal processing chips.Our system's novelty is that the design spec-ifications include multiple, interacting per-formance specifications in addition to thetraditional functional specification. The per-formance factors that can affect a designinclude: throughput rate, latency, chip area,power usage, technology, and pin count.The ability to explicitly consider tradeoffsbetween various performance factorschanges both the designed object and the

227


process by which the object is designed.The system's output is a set of possibledesigns. We limit the system's expertise to aniche of signal processing where computa-tion is not extremely regular.

The design process is divided into twophases. The first transforms the functionalspecification into various architecturesdescribed at the register transfer level. Thesecond phase chooses implementations forthe various functional units. We have devel-oped two techniques for the second phase.One does a heuristically limited enumerationof possible combinations. The other formu-lates the task as a linear programmingproblem to find optimal assignments. Bothmethods generate designs at a multitude ofperformance levels thereby giving the user asense of the possible tradeoffs amongdesigns.

1.9 A Dual Excitation SpeechModel

Sponsors



U.S. Air Force - Electronic Systems Division(Contract F19628-85-K-0028)

Project Staff

Professor Jae S. Lim, John C. Hardwick

One class of speech analysis/synthesissystems (vocoders) which has been exten-sively studied and used in practice is basedon an underlying model of speech. Eventhough traditional vocoders have been quitesuccessful in synthesizing intelligible speech,they have not been successful in synthesizinghigh-quality speech. The MultiBandExcitation (MBE) speech model, introducedby Griffin, improves the quality of vocoderspeech through the use of a series of fre-quency dependent voiced/unvoiced deci-sions. The MBE speech model, however, stillresults in a loss of quality as compared to theoriginal speech. This degradation is causedin part by the voiced/unvoiced decision

process. A large number of frequencyregions contains a substantial amount ofboth voiced and unvoiced energy. If aregion of this type is declared voiced, then atonal or hollow quality is added to the syn-thesized speech. Similarly, if the region isdeclared unvoiced, then additional noiseoccurs in the synthesized speech. As thesignal-to-noise ratio decreases, the classifica-tion of speech as either voiced or unvoicedbecomes more difficult and, consequently,the degradation is increased.

A new speech model has been proposed inresponse to the aforementioned problems.This model is referred to as the DualExcitation (DE) speech model, due to its dualexcitation and filter structure. The DEspeech model is a generalization of most pre-vious speech models, and with the properselection of the model parameters it reducesto either the MBE speech model or to avariety of more traditional speech models.

Current research is examining the use of thisspeech model for speech enhancement, timescale modification and bandwidth com-pression. Additional areas of study includefurther refinements to the model andimprovements in the estimation algorithms.

1.10 Motion Estimation fromImage Sequences

Sponsors

ATET Bell LaboratoriesDoctoral Support Program



Project Staff

Professor Jae S. Lim, Steven H. Isabelle

Most biological systems possess some typeof mechanism for sensing motion in theworld around them and use this informationto assist them in many tasks. For example,visual motion information is used for objecttracking, feedback for physical movement,and forming inferences about the sur-



rounding environment. Research in the areaof motion estimation from image sequencesis aimed at equipping computers with someof these capabilities. Many motion esti-mation techniques rely on simple models forobject motion to form local estimates ofmotion directly from image luminance atmany points in an image, then combine theselocal estimates in accordance with sameglobal model of the motion field. Thesealgorithms tend to perform poorly in regionsof motion field discontinuities such asforeground/background boundaries.

Improved performance could be achieved ifthe frames of the image sequence could besegmented into regions containing similarmotion. As posed, this is a circular proposi-tion, since an image frame must be seg-mented based on motion to estimate motion.This circularity may be resolved by using atechnique of iterative refinement which,starting with a crude estimate of the motionfield, produces a crude segmentation whichis then used to refine the motion estimate.

This research is aimed at investigating tech-niques of image motion estimation withimproved performance at motion boundariesby incorporating fuzzy segmentation informa-tion. As a by-product, the segmentationproduced is itself useful for applications suchas object tracking and object detection.

1.11 MultiLevel SignalMatching for VectorQuantization

Sponsors

Canada, Bell Northern Research ScholarshipCanada, Fonds pour la Formation de

Chercheurs et I'Aide a la RecherchePostgraduate Fellowship


9 E. Milios, Signal Processing and Interpretation UsingEng. and Comp. Sci., MIT, 1986.


Project Staff

Professor Alan V. Oppenheim, Jacek Jachner

This thesis investigates the use of multilevelsignal representations, or hierarchies of signalrepresentations with increasing levels ofabstraction of detail, to perform efficientsignal matching for Vector Quantization in aspeech coder. The signal representationsrange from the numeric sequence that com-pletely characterizes a signal to high-levelrepresentations that abstract detail and groupsignals into broad classes.

The use of abstraction of detail in signalmatching has been proposed in the contextof a helicopter sound signature detectionproblem. 9 The current work focuses on low-bit-rate speech coding, for which recentlyproposed approaches, such as the CodeExcited Linear Predictor (CELP) structure,10

rely on a Vector Quantizer to code the resi-dual signal after linear prediction stages. Theresidual is matched with one codewordsignal out of a codebook of such signals,such that an error criterion between inputand codeword signals is minimized. In theCELP coder, the error criterion is a time-varying linearly weighted mean-square error.The signal matching operations required forVector Quantization (VQ) represent the majorsource of complexity in the CELP structure,and limit the practical size, hence perform-ance, of VQ codebooks.

This work seeks to reduce matching com-plexity by using multilevel signal represent-ations simplified by abstraction of detail. Thecomputational benefit of such signal repres-entations is twofold. First, the matchingoperations on simplified or partial signal rep-resentations, termed partial errors, are sub-stantially simpler to perform than applyingthe error criterion to the complete signal rep-resentation. Next, the simplified signal rep-

Multilevel Signal Abstractions. Ph.D. diss., Dept. of Electr.

10 M. Schroeder and B. Atal, "Code-Excited Linear Prediction (CELP): High-Quality Speech at Very Low Bit Rates,"International Conference on Acoustics, Speech and Signal Processing, 1985.

229


resentations partition the codebook intoequivalence classes of codewords that sharea partial signal representation value. Thepartial error is the same for all codewords ina class and needs to be computed only once.

When seeking the optimal codeword thatminimizes the error criterion, the partial errorsprovide sufficient information to eliminatesome codewords from further consideration.Multiple signal representations with progres-sively greater representation detail are usedfor matching, and a branch-and-bound pro-cedure" prunes the set of eligible code-words.

In situations where a non-optimal codewordis still acceptable, the "best" codeword isselected using only the simple partial errors.This yields significant reductions in com-plexity. The cost in performance is mini-mized by the adaptive selection of the partialrepresentations used to match each inputsignal.

The formulation of suitable multilevel repres-entations and the efficient evaluation ofpartial error functions for VQ in the CELPcoder are the main directions of currentwork. Structured codebooks which simplifythe adaptive multilevel matching procedureare being investigated. Performance simu-lations are conducted using the SPLICE12

signal processing environment on LISPmachines.

1.12 Application ofNonparametric Statistics inNarrowband Signal Detection

Sponsors


Sanders Associates, Inc.U.S. Navy - Office of Naval Research

(Contract N00014-81 -K-0742)

Project StaffProfessor Alan V. Oppenheim, Tae H. Joo

The problem of detecting a narrowbandsignal in wideband noise occurs in many dif-ferent contexts. This research is motivated bythe problem of detecting a periodic gravita-tional wave signal which is generated by theorbiting motion of binary stars, for example.Because of the relative motion between thegravitational wave source and the receiver,the received data are modeled as a non-linear, frequency-modulated signal in addi-tive noise.

When the received data are processed asmultidimensional data, the periodic gravita-tional wave detection problem becomes theproblem of detecting a sinusoidal signal inwideband noise using multiple observations.Using the time-frequency relationship of thesignal specified by the frequency-modulationstructure, the spectra can combined to detectthe signal. A critical component of thisdetection problem is the requirement that thesinusoidal signal must exist in all channels.Because local disturbances can exist in real-world channels, the detector should decidethat the received data contain a gravitationalwave only when the sinusoidal signal existsin all channels.

If the sinusoidal signal exists in all channels,the optimal detector computes the average ofthe periodograms then threshold-tests theaverage. However, a shortcoming in usingthe average as the detection statistic is thatonly the total energy is tested. Hence, astrong signal which is present in one channelcan exceed the threshold to result in a falsealarm. To rectify this incorrect decision, theexistence of the signal in all channels mustbe explicitly incorporated into a detectionstatistic. Using nonparametric statistics, anew detection statistic is developed whichmeasures the coherence between channels toenforce the signal existence requirement.This detection statistic will be tested using

11 E. Lawler and D. Wood, "Branch-and-Bound Methods: A Survey," Oper. Res., 14(4):699-719, 1966.

12 C. Meyers, Signal Representation for Symbolic and Numeric Processing. Ph.D. diss., Dept. of Electr. Eng. andComp. Sci., MIT, 1986.



the gravitational wave data which are meas-ured by the MIT 1.5 meter prototypeinterferometric gravitational wave detector.

1.13 Image Texture Modelling

Sponsors



Project Staff

Professor Jae S. Lim, Rosalind W. Picard

Textured regions are the nemesis of manyimage processing algorithms. For example,algorithms for image segmentation or imagecompression often rely on assumptions ofstationarity or high correlation of the imagegray-level data. These assumptions generallyfail in textured regions. In this research, wewill look for a model for the textures whichallows them to be synthesized with only asmall number of parameters. Such a modelwould have applications for image coding,segmentation, and classification problems.

One desirable property of a texture model isan ability to synthesize syntactically regular"structural" textures, e.g., a tiled wall, as wellas the more random "stochastic" textures,e.g., shrubbery. Most models assume onlyone of these cases. In this research, weexamine the abilities of the Gibbs distribution(used in statistical mechanics to model inter-molecular interactions on a lattice) to modelboth structural and stochastic textures. Inparticular, we will analyze the effects of dif-ferent synthesis methods on the texturesgenerated. The purpose of this analysis is totry to understand how structure can beincorporated into the random model. Inaddition to an analysis of the Gibbs distribu-

tion, we will explore some ideas for a newmodel.

The goal of this research is to develop amodel which is capable of synthesizing acontinuum of structural and stochastic tex-tures. Although analysis is a major part ofthe modeling problem, the focus is on syn-thesis to see if one can characterize thetexture completely.

1.14 Structure DrivenScheduling of DSP and LinearAlgebra Problems

Sponsors


OKI Semiconductor, Inc.Sanders Associates, Inc.

Project Staff

Professor Bruce R. Musicus, G.N. SrinivasaPrasanna

In this thesis, 13 we explore issues related tothe mapping of linear algebra and digitalsignal processing problems onto multi-processors. The highly regular structure, highcomputation, and data-independent controlof these problems makes them ideally suitedfor semi-automatic compilation onto multi-processors. The large number of known,well-structured algorithms provides a readyknowledge base for the compiler.

The system to be developed, KALPANA(imagination), has knowledge of a variety ofdifferent algorithms, and maps (schedules)these different algorithms to the given archi-tectures. For example, several good algo-rithms are known for matrix products. Aproblem involving a matrix product wouldexploit one of these. Solutions with optimalspeed and efficiency are sought. Searchcomplexity will be minimized by exploiting

13 C. Myers, Signal Representation for Symbolic and Numeric Processing. Ph.D. diss., Dept. of Electr. Eng. andComp. Sci., MIT, 1986.; G.N.S. Prasanna, Structure Driven Scheduling of Linear Algebra and Digital Signal Pro-cessing Problems Ph.D. diss. proposal, Dept. of Electr. Eng. and Comp. Sci., MIT, 1988.; M.M. Covell, Represen-tation and Manipulation of Signal Processing Knowledge and Expressions, Ph.D. diss. proposal, Dept. of Electr.Eng. and Comp. Sci., MIT, 1987.

231


the problem structure, and choosing the sim-plest architectures - fully-connected multi-processors. Low-level details like processortiming are also targeted to be handled. Notethat the algorithm can change drastically asprocessor timing changes.

Exploiting the structure of the problem toderive a good algorithm and its implementa-tion (Structure Driven Scheduling) is thecommon thread throughout this thesis. Forexample, an N by N matrix-matrix multiplycan be broken up into N instances of matrix-vector multiplications. If schedules formatrix-vector multiplication are known, thenwe obtain an algorithm for the matrixproduct. If not, we can further split eachmatrix-vector multiply into N dot-products. Ifthe schedules for dot-products on the speci-fied architecture are known, then we obtainanother algorithm for computing the matrix-matrix multiply. Note that the structure ofthe problem has enabled us to derive optimalsolutions from the vast number of possibil-ities, without encountering NP-completenessor worse. Currently, the problem breakup isprespecified.

Note that the layout of the two matricesacross the processors has to appropriate for agood schedule. One matrix has to be trans-posed relative to the other, to minimize com-munication. We have derived a generaltechnique for optimal data layout for matrixexpressions involving additions and multipli-cations. A dynamic programming approach tohandle non-square matrices has also beenexplored.

The problems being targeted include matrixproducts and the Fourier Transform. Both arevery important basic non-recursive numericalgorithms. Extensions to optimally-scheduled block diagrams containing thesealgorithms as basic operators are also beingdone.

1.15 The Application ofFunction Approximation UsingGeneralized Polynomials to theDevelopment of OptimumWindows

Sponsors

National Science Foundation FellowshipNational Science Foundation

(Grant MIP 87-14969)U.S. Navy - Office of Naval Research

(Contract N00014-81 -K-0742)

Project Staff

Professor Alan V. Oppenheim, James C.Preisig

The windowing of a time series of uniformlyspaced samples to reduce spectral leakage,the spatial domain equivalent of windowinga uniform linear array to reduce leakagebetween far-field sources at differentbearings, and the design of LTI FIR linearphase filters are problems whose optimum(in a min-max sense) solutions are well-understood and are based upon the approxi-mation of functions on intervals (possiblydisjoint) of the real line using trigonometricpolynomials. The purpose of this work is toextend and apply the theory of functionapproximation using generalized polynomialsto handle approximation on closed sets ofmultidimensional space using polynomialcoefficients which may be real, complex orrestricted to a finite set. Applications includewindowing of arrays for near-fieldbeamforming, windowing of time series withnon-uniform samples, windowing of non-uniform and multidimensional arrays, andmask design for x-ray lithography.

232 RLE Progress Report Number 1 31


1.16 Iterative Algorithms forParameter Estimation fromIncomplete Data and TheirApplications to SignalProcessing

Sponsors

Tel Aviv University, Department ofElectronic Systems

Sanders Associates, Inc.U.S. Navy - Office of Naval Research

(Contract N00014-85-K-0272) 14


Project Staff

Dr. Ehud Weinstein

The Estimate-Maximize (EM) algorithm is aniterative method for finding the MaximumLikelihood (ML) or the Maximum-A-Posteriori (MAP) parameter estimates givenincomplete data. The method has beenfound useful in a variety of signal processingproblems including medical imaging, param-eter estimation of superimposed signals,speech enhancement in multiple microphoneenvironment, and signal reconstruction fromspatial information.

We have developed an extension of the EMalgorithm, termed the Cascade EM (CEM)algorithm, that is found useful in acceleratingthe rate of convergence of the algorithm, andin simplifying the computations involved.15

Using the EM and the CEM algorithms, wehave considered the problem of simultaneousstate estimation and parameter identificationin linear dynamical continuous/discretesystems. The main result here is that one can

use the Kalman smoothing equations for MLidentification of the system parameters. Wealso develop a new method for calculatingthe log-likelihood gradient (score), theHessian, and the Fisher's Information Matrix(FIM), that can be used for efficient imple-mentation of gradient-search algorithms, andfor assessing the mean square accuracy ofthe ML parameter estimates. 16

We have also developed a computationallyefficient algorithm for estimating the spatialand spectral parameters of multiple sourcesignals using radar/sonar arrays. The mostattractive feature of the proposed algorithm isthat it decouples the spatial parameter opti-mization from the spectral optimization,leading to a significant reduction in the com-putations involved."7

1.17 Equalization(Identification) ofNon-Minimum Phase Systems

Sponsors

Tel Aviv University, Department ofElectronic Systems,

U.S. Navy - Office of Naval Research(Contract N0001 4-85-K-0272)' 8


Project Staff

Dr. Ehud Weinstein 19

The problem considered here is the fol-lowing: we observe the output of a discretetime-invariant linear possibly non-minimumphase system H with input being a realization(sample function) from a stochastic process.We want to recover the input signal or equiv-

14 Administered through the Woods Hole Oceanographic Institution.

15 Ibid.

16 Research developed with Mordechai Segal, Tel Aviv University.

17 Research developed with Mordechai Segal and Bruce Musicus.

18 Administered through the Woods Hole Oceanographic Institution.

19 Research developed with Ofir Shalvi, Tel Aviv University

233


alently to identify the magnitude and phaseof the inverse of H using a tap-delay line(Equalizer). This problem referred to as self-recovering or blind equalization is of primeinterest in data communication, acousticaland geophysical signal processing.

We have developed necessary and sufficientconditions for equalization. Based on that,we propose several equalization criteria, andprove that their solution corresponds to thedesired response. These criteria are universalin the sense that they do not impose anyrestrictions on the probability law of theinput (unobserved) process. The resultingalgorithms only involve the computation of afew moments of the system output, implyinga simple tap update procedure.

1.18 Fractals and Chaos forDigital Signal Processing

Sponsors

Natural Sciences and EngineeringResearch Council of Canada,Science and Engineering Scholarship



Project Staff

Professor AlanWornell

V. Oppenheim, Gregory W.

Generally, the objectives of this research areto apply non-linear dynamical system theoryand fractal process theory to problems ofinterest to the signal processing community.

The first aspect of this research involves theuse of self-similar or fractal signal models(e.g., 1/f) in solving signal processing prob-lems. The ubiquity of self-similar phenomenaand structures in nature suggests that thesemodels may arise rather naturally in the effi-cient representation of information in variouscontexts. Novel spread spectrum techniquesbased on similar formulations are also beingconsidered, as are techniques for modelingsignals through the use of iterated cellularautonoma.

A second aspect of this research is aimed atexploiting the emerging theory of chaoticdynamical systems in problems pertaining todigital signal processing. Chaotic systemsare characterized by exponential sensitivity toinitial conditions, so that the long-term stateevolution of such systems is essentiallyunpredictable. Consequently, chaoticdynamical systems are deterministic systemsof arbitrarily low order capable of exhibitinghighly complex and stochastic behavior. Insome implementation scenarios, accumulatoroverflow effects have been shown to causedigital filters to exhibit such chaotic behavior.Present work is aimed at identifying andcharacterizing chaotic dynamics at work inother digital filtering scenarios involvingquantization and overflow effects.


Pal - COnnecting REpositories · is close interaction with MIT Lincoln Labora-tory and Woods Hole...

Documents

Transcript of Pal - COnnecting REpositories · is close interaction with MIT Lincoln Labora-tory and Woods Hole...