Modern Digital Signal ProcessingECE 5650/4650 Lecture Notes

© 1999–2019 Mark A. Wickert

n

n

y n

x n h n H e j

Frequency

z-plane

0

-45G

ain

- dB

0 0.5

0

0Re

Im

Chapter 1Course Introduction/Overview

Contents

1.1 Introduction to Modern Digital Signal Processing . . 1-31.2 Contemporary DSP . . . . . . . . . . . . . . . . . . 1-4

1.2.1 The Technology . . . . . . . . . . . . . . . 1-41.2.2 Key Applications Areas . . . . . . . . . . . 1-5

1.3 Course Perspective in Comm/DSP Area ECE . . . . 1-71.4 What is this course about? . . . . . . . . . . . . . . 1-81.5 Software Analysis/Simulation Tools . . . . . . . . . 1-91.6 The Computer Projects . . . . . . . . . . . . . . . . 1-101.7 Course Syllabus . . . . . . . . . . . . . . . . . . . . 1-121.8 Instructor Policies . . . . . . . . . . . . . . . . . . . 1-14

1-1

1.1. INTRODUCTION TO MODERN DIGITAL SIGNAL PROCESSING

1.1 Introduction to Modern Digital Sig-nal Processing

� Contemporary DSP

– Theory

– Technology

– Applications

� Course perspective

– Expected background

– Where to go from here

� What is this course about?

� The role of computer analysis/simulation tools in and outsidethis course

� The computer simulation project

� Instructor policies

ECE 5650/4650 Modern DSP 1-3

CONTENTS

1.2 Contemporary DSP

� The theoretical foundation for Modern signal processing gotit’s start back in the late 1960s and early 1970s

� Two key texts which started this era are Digital Signal Process-ing by Oppenheim and Schafer and Theory and Application ofDigital Signal Processing by Rabiner and Gold, both publishedin 1975

� The Oppenheim and Schafer Discrete-Time Signal Processingtext (1989, 1999, 2010) is now in its third edition

� A good source of recent theoretical developments is the IEEEJournal on Signal Processing (formerly Acoustics Speech andSignal Processing) and the annual Proceedings of the Interna-tional Conference on Acoustics Speech and Signal Processing(ICASSP)

� Other applied digital signal processing conferences exist andare usually associated with trade shows

1.2.1 The Technology

� DSP chips/architectures

– Integer/Fixed point implementations

– Floating point implementations

– FPGA based implementations

� DSP software and tools

1-4 ECE 5650/4650 Modern DSP

1.2. CONTEMPORARY DSP

1.2.2 Key Applications Areas

� Computers/Internet

– Broadband access, e.g., cable modem

– Voice over IP

– MP3 and related high quality audio formats

– Multimedia in general: data, voice, music, & video

� Wireless Communications

– Mobile communications (over 200 million phones/yr overthe world)

– xDSL

– Real-time data compression for voice and video

– Wireless and telecommunication infrastructure

� Industrial

– Real-time processing of industrial and medical signals

– PC with DSP for virtual instruments for test and analysis

� Digital Control Systems

– DSP vs microprocessors and microcontrollers

� Audio

– Internet audio streaming

– Home audio and theatre systems

– Noise cancelling, e.g., quieting a car interior using adap-tive noise cancellation or headphones

ECE 5650/4650 Modern DSP 1-5

CONTENTS

� Video Imaging

– HDTV in its various forms

– Special purpose image processing in instrumentation andmedical

� Biomedical

– Many possibilities

– Hearing aids

– Diagnostic imaging

� Military/Aerospace

– An active DSP area for over 40 years

– Many consumer DSP applications had their start here

– Sophisticated surveillance systems and smart weapons

– Specialty technologies include:

� Frequency domain processing� Parallel processing� Radar signal processing� Software defined radio (SDR) is very big

1-6 ECE 5650/4650 Modern DSP

1.3. COURSE PERSPECTIVE IN COMM/DSP AREA ECE

1.3 Course Perspective in Comm/DSPArea ECE

Sign

als

&

Syst

ems

Mod

ern

DSP

Com

mSy

s I

Rea

l-Tim

eD

SP

Com

mSy

s II W

irele

ss

Net

wor

king

Prob

&

Stat

istic

sC

omm

Lab

Sign

alPr

oces

s La

b

Stat

istic

alSi

gnal

Proc

ess

Und

ergr

adua

te

Engi

neer

ing

Cur

ricul

um

Seni

or/1

st Y

ear

Gra

duat

e Si

gnal

s &

Syst

ems

Cou

rses

Ran

dom

Sign

als

Spre

adSp

ectr

um

Wire

less

&M

obil

Com

Opt

ical

Com

m

PLL

&Sy

nch

Rad

arSy

stem

s

Com

mN

etw

orks

Sate

llite

Com

m

Det

ect/

Estim

atio

n

Estim

&A

dap

Filt

Info

rm/

Cod

ing

Spec

tral

Estim

atio

nIm

age

Proc

essi

ng

Oth

er G

radu

ate

Sign

als

& Sy

stem

s C

ours

es O

ffere

d on

D

eman

d/In

dep.

Stu

dy

Sp SpSpFa

Fa

Sp

? (odd

)

Sp (as

early

as

sp20

20)

(odd

)

(pla

nned

for

sp

ring

2020

)

?(e

ven)

ECE 5650/4650 Modern DSP 1-7

CONTENTS

1.4 What is this course about?

� This course has as its focus the nuts and bolts of one-dimensionaldiscrete-time signals and systems analysis

– Developing analytical skills is of primary importance

– Introducing specialized applications is secondary

� Most all of the theoretical developments will parallel those ofa traditional continuous-time signals and systems course

– Here at UCCS, we now introduce discrete-time signal andsystems in ECE 2610 (Intro to Sig & Sys), and more DSPtopics are covered in ECE 3205 (Ckts & Sys II)

– This course pushes the math well beyond the introductionin those courses

� Unique aspects of discrete-time signals and systems include:

– Analog-to-digital and digital-to-analog interfaces

– Multi-rate processing systems, i.e., more than one sam-pling rate in the signal input/output path

– Software reconfigurability of systems

� Discrete random signals will be introduced

– This will allow analysis of quantization affects

– Key to follow-on courses involving statistical signal pro-cessing

– Building simulation models that include noise

1-8 ECE 5650/4650 Modern DSP

1.5. SOFTWARE ANALYSIS/SIMULATION TOOLS

1.5 Software Analysis/Simulation Tools

� The experimental investigation of discrete-time signals and sys-tems does not require a lab full of test equipment

� What is available from the academic perspective:

– Basic mathematical analysis tools include

� Python (Scipy stack), MATLAB, & Mathematica

– DSP specific tools include

� Python using scikit-dsp-comm1 along with the SciPymodule scipy.signal

� MATLAB using the signal processing toolbox

– Starting from scratch

� The C/C++ programming language

– Real-time DSP Development

� For ARM Cortex M – MDK Keil� Verilog/VHDL hardware description languages� For DSP TI devices Code Composer Studio� Other tools that have code generation capability

� Inexpensive hardware evaluation is also a reality, e.g.,

– ARM Cortex M4 and M7 with CMSIS DSP library, e.g.,Cypress Pioneer Kit $50 (ECE 4680 & ECE 4655/5655)

– TI LCDK at $195 using the OMAP L138

– Others from Analog Devices and Motorola

1http://scikit-dsp-comm.readthedocs.io/en/latest/?badge=latest

ECE 5650/4650 Modern DSP 1-9

CONTENTS

1.6 The Computer Projects

� Computer based exercises using Python will be assigned dur-ing the semester (Jupyter Notebook or Lab with Python 3.6+)

– These projects first familiarize you with Python, then workthrough the many details of DSP in the time and fre-quency domains

– Python examples are also embedded in the notes and willbe used for in-class examples and demos

– The module pyaudio_helper of scikit-dsp-comm al-lows real-time DSP to be implemented inside the Jupyternotebook, including GUI widgets2

– For Fall 2018 at lease one project problem will introduceyou to pyaudio_helper and your PCs audio subsystem

� A larger computer simulation project, a capstone project ofsorts, will be assigned during the second-half of the semester

– Past simulation projects have focused on software definedradio concepts, e.g., IF sampling, adaptive filters, or aDSP based modem

� In the Spring Semester ECE 5655, Real-Time DSP, is offeredas a complement to Modern DSP

– In this course we currently use the ARM Cortex M4 viathe Cypress Pioneer Kit and the Keil MDK IDE

2http://conference.scipy.org/proceedings/scipy2018/mark_wickert_250.html

1-10 ECE 5650/4650 Modern DSP

1.6. THE COMPUTER PROJECTS

� Another follow-on path is ECE 5615, Statistical Signal Pro-cessing, which builds upon the theory side of DSP and intro-duces random signals

– Beyond ECE 5650, this course requires an undergradu-ate background in probability and random variables, e.g.,ECE 3610

ECE 5650/4650 Modern DSP 1-11

CONTENTS

1.7 Course SyllabusECE 5650/4650

Modern Digital Signal ProcessingFall Semester 2019: Room UH317, 8:00–10:40 AM

Dr. Mark Wickert Office: EB-292 Phone: 255-3500 mwickert@uccs.edu Fax: 255-3589 http://www.eas.uccs.edu/~mwickert/ece5650/

Wed. 10:40–11:15 am & Mon/Wed 1:30–2:15 pm, others by appointment.

Alan V. Oppenhiem and Ronald V. Schafer, Discrete-Time Signal Processing, third edition, Prentice-Hall, Englewood Cliffs, New Jersey, 2009.

Course lecture notes will be posted on the course Web Site as password required PDF files. Students are encouraged to download and print them.

Open source Python 3.7x using the Jupyter Notebook & Lab. I suggest Ana-conda then install the package scikit-dsp-comm. See course Web Site too.

1.) Graded homework assignments, including use of Python with the scipy-stack + Python project 1, assignments 25%

2.) Final Python computer project worth 20%/15%. Grade option with final.3.) Two “Hour” exams at 15% each, 30% total.4.) Final exam worth 25%/30%.

Instructor:

Office Hrs:

Required Texts:Notes:

Grading:

Topics Text Sections

1. Introduction and course overview 1

2. Discrete-time signals and systems 2.0–2.9

3. The z-transform 3.0–3.4

4. Sampling of continuous-time signals and discrete-time ran-dom signals

4.0–4.6 2.10, App. A

4.7–4.9

5. Transform analysis of linear time-invariant systems 5.0–5.7

6. Structures for discrete-time systems and finite precision issues

6.0–6.9

7. The discrete Fourier transform 8.0–8.7, 8.9?

8. Computation of the discrete Fourier transform 9.0–9.6

9. Applications of the DFT Portions of 10.0–10.6

Learning Outcomes

The expected learning outcomes of this course are: An introduction to Python(via the Scipy stack) for digital signal processing modeling and filter design,with emphasis on the Jupyter Notebook interface; audio signal processing sim-ulation; real-time signal processing using the Cortex M4 micro-controller;ANSI C for developing real-time signal processing; the ARM CMSIS-DSPlibrary and the associated application programming interface (API); Hardwareinterfacing to ADC and DAC as well as GPIO for code timing analysis; imple-mentation of real-time FIR and IIR filters and signal generation using directdigital synthesis (DDS).

Installing Python & scikit-dsp-comm

Download and Install the Anaconda Python 3.6 Distribution: https://ana-conda.org/anaconda/python. Do not choose the defaults! I recom-mend you have Anaconda configure paths to the tools. This is not the default.Jupyter Notebook, QT Console, and Jupyter Lab will be installed with theAnaconda distribution, but more formation can be found at: http://ipy-thon.org/. Optionally Download and Install a code development environment that alsointegrates git version control such as Visual Studio Code (VS Code): https://code.visualstudio.com/ or PyCharm Community Edition IDE for Python:https://www.jetbrains.com/pycharm/.Download and Install the distributed version control application git: https://git-scm.com/ on Windows systems (included on macOS and Linux). Clone and Install the package scikit-dsp-comm by following the GitHubREADME page at: https://github.com/mwickert/scikit-dsp-comm. Maintain scikit-dsp-comm using git pull origin master (see the informa-tion in the GitHub link).

Optional Jupyter Notebook to PDF Conversion

The Jupyter notebook is the perfect place to write code, document code, writetext using markdown, import figures, and typeset math equations usingLaTeX syntax. To render a Jupyter notebook as a PDF document a few moreopen source software components are needed:

• Install Pandoc for file conversion to LaTeX and other formats: https://pandoc.org/index.html

• Install MikTeX for converting LaTeX documents to PDF on Windows:https://miktex.org/. When installing MikTeX be sure to choose theoption to automatically download needed LaTeX packages on-the-fly. OrInstall TeXLive for converting LaTeX documents to PDF on macOS andLinux: https://www.tug.org/texlive/

• Install Typora: As an alternative to installing LaTeX (MikTeX or TeX-Live), install the markdown editor Typora: https://typora.io/. Now youcan export as *.md and then open the file in Typora and save to PDFdirectly. You can also do some nice file editing if need be.

Important Deadlines: Review the Fall 2019 deadlines: https://www.uccs.edu/registrar/course-deadlines/fall-2019. Performance histograms (HW, Quiz, & Exams) will be discussed in class prior to the last day to drop, Friday November 1. Use this to decide on continuing or dropping the course – the deadline for dropping without ECE Chair signature (NOT the Dean as stated in the link above) is November 1. Only under extenuating circumstances will a late drop be considered.

1-12 ECE 5650/4650 Modern DSP

1.7. COURSE SYLLABUS

The expected learning outcomes of this course are a more in-depth treatment of discrete-time signals and systems as first started in the UCCS course ECE2610 or similar from another university. As a discipline within electrical engineering this known as digital signal processing (DSP). Specifically the student will learn how to model discrete-time signals and systems in the time domain; extend the time domain modeling to the frequency domain using the discrete-time Fourier transform (DTFT); working signals and linear time invariant (LTI) systems us-ing z-transform (ZT) techniques; sampling theory and multirate sampling theory as found in modern DSP; discrete-time random processes and modeling quanti-zation/fixed-point arithmetic; DSP problem solving using time, frequency, and z-domains effectively; properties of LTI systems having minimum phase and linear phase; choice of various DSP implementation topologies; the value and power of the discrete Fourier transform (DFT) and its efficient implemetation via fast Fourier transform (FFT) algorithms; Simulation of DSP algorithms and subsystems using Python with the Scipy stack.

Download and Install the Anaconda Python 3.7+ Distribution: https://ana-conda.org/anaconda/python. I recommend you have Anaconda configure paths to the tools. This is not the default. Jupyter Notebook, QT Console, and Jupy-ter Lab will be installed with the Anaconda distribution. Finish installation on Windows in Powershell by typing: conda init powershell. Follow the Jupy-ter Lab link for more steps on getting this up. Finally, a very good alternative to the full Anaconda is miniconda and the use virtual environments. We will talk about this in class, as it can save disk space and make reconfiguration easier. Optionally Download and Install a code development environment that also integrates git version control such as Visual Studio Code (VS Code): https://code.visualstudio.com/ or PyCharm Community Edition IDE for Python: https://www.jetbrains.com/pycharm/.Download and Install the distributed version control application git: https://git-scm.com/ on Windows systems (included on macOS and Linux). Clone and Install the package scikit-dsp-comm by following the GitHub RE-ADME page at: https://github.com/mwickert/scikit-dsp-comm. Maintain scikit-dsp-comm using git pull origin master (see the information in the GitHub link).Install Pandoc for file conversion to LaTeX and other formats: https://pandoc.org/index.html. I think Anaconda now installs this by default.Install MikTeX for converting LaTeX documents to PDF on Windows: https://miktex.org/. When installing MikTeX be sure to choose the option to automati-cally download needed LaTeX packages on-the-fly. Or Install TeXLive for converting LaTeX documents to PDF on macOS and Linux: https://www.tug.org/texlive/Install Typora: As an alternative to installing LaTeX (MikTeX or TeXLive), install the markdown editor Typora: https://typora.io/. Now you can export as *.md and then open the file in Typora and save to PDF directly. You can also do some nice file editing, including LaTeX equations, if need be.Install Inkscape for converting embedded SVG graphics in Jupyter notebooks via Pandoc to LaTeX and then PDF: https://inkscape.org/en/release/0.92.2/.

Learning Outcomes

Installing Python & scikit-dsp-comm

Optional Jupyter Notebook to PDF Conver-sion

ECE 5650/4650 Modern DSP 1-13

CONTENTS

1.8 Instructor Policies

� Homework papers are due at the start of class

� If business travel or similar activities prevent you from attend-ing class and turning in your homework, please inform me be-forehand

� Grading is done on a straight 90, 80, 70, ... scale with curvingbelow these thresholds if needed

� Screencasts of the lectures will be made available as soon aspossible after each lecture; this may be of help to those of youthat travel and to others for review purposes

� Homework solutions will be posted on the course Web site asPDF documents with password protection

� Old exams will be posted on the Web site prior to the hourexams

1-14 ECE 5650/4650 Modern DSP