SOFTWARE AND HARDWARE CO-SIMULATION PLATFORM FOR IMAGE PROCESSING

JOHNNY KONG JAK KAN

UNIVERSITI TEKNOLOGI MALAYSIA

i

“I hereby declare that I have read this project report and in my

opinion this project report is sufficient in terms of scope

and quality for the award of the degree of

Master of Engineering (Electrical - Computer and Microelectronics System)”

Signature : ...............................................

Name of supervisor : PROF. DR. MOHAMED KHALIL HANI

Date : 15 JANUARY 2014

i

SOFTWARE AND HARDWARE CO-SIMULATION PLATFORM

FOR IMAGE PROCESSING

JOHNNY KONG JAK KAN

A project report submitted in partial fulfilment of the

requirements for the award of the degree of

Master of Engineering (Electrical - Computer and Microelectronics System)

Faculty of Electrical Engineering

Universiti Teknologi Malaysia

JANUARY 2014

iii

My genuine dedications to,

My beloved daddy, mummy and sister;

My enduring supervisor;

My compassionate friends;

And the love ones;

Who are always there for me,

Every step of the way,

Thanks!

iv

ACKNOWLEDGEMENTS

First and foremost I would like to convey my deepest gratitude to my supervisor,

Prof. Dr. Mohamed Khalil who has been the most influential person who always guide,

motivate, support, and advise me throughout my thesis completion with his inspiring and

constructive comments and valuable feedbacks.

Furthermore, I would like to pay special tribute to my seniors, Miss Lee, Mr. Sia,

Mr. Liew and my cherished friends, who are willing to guide me with patience and

keenness. Thank you so much for deserving a lot of time and passion to guide, teach,

motivate and share their experiences and knowledge with me.

In short, I would like to extend my deepest and sincere appreciation to my beloved

family, especially to my mother, who are always support me, encourage me, and listen to

me along my research. They are my source of inspiration and my greatest refuge. And

also for my special person, Miss Hii, thank you for understand me and support me.

v

ABSTRACT

Nowadays, modern SoCs have larger scale and complexity. Modelling hardware design archicteture of SoCs normally required RTL design. Since, the system design is getting larger, it is usually partitioned into two parts: software and hardware. For the sack of achieving real time performance, it is essential to map some of the algorithms into hardware. Previously, the hardware and software design of a system are done by different people. Therefore, the design lifecycle is longer. Now, market pressures on short design cycle, maintainability and reusability of system design. To reduce design cycle, a new method must be applied. SystemVerilog is the extension of Verilog HDL that improved with a lot of new features added. Direct programming interface (DPI) is one of the new features whereas it allow SV call C function and vice versa. This work proposed to use Altera-Modelsim simulator to run co-simulation on several test cases: 16-bit unsigned adder, greater common divisor (GCD), 9-tap FIR filter and binary median filter. All the hardware design modelling are using SystemVerilog due to DPI technique can only work with SV. 16-bit unsigned adder and GCD test cases are the startup work before a more complex and real world case studies are applied. 9-tap FIR filter is designed and the input data are passed from random number generator from C function. The output of FIR fitler are verified with C function. Due to window size of median filter is small, line buffer technique is applied in this work. It is a straight forward method and suit for design processing window in digital hardware. The proposed work has overcome the problems faced when running co-simulation based on Modelsim simulator using DPI technique.

vi

ABSTRAK SoCs kini mempunyai skala yang lebih besar dan rumit. Kebanyakan reka bentuk untuk perkakasan adalah menggunakan teknik RTL. Sistem reka bentuk telah dipechkan kepada dua jenis, iaitu perisian dan perkakasan. Untuk mencapai prestasi masa nyata, sesetengah algoritma perlu dibentukkan dalam perkakasan. Sebelum ini, reka bentuk perkaksan dan perisian adalah dibuat oleh orang yang berlainan. Oleh itu, kitaran hayat reka bentuk akan menjadi lebih panjang. Buat masa ini, pasaran teknologi mempunyai satu trend, iaitu kitaran hayat reka bentuk yang singkat, penyelenggaran dan pengunaan semlua reka bentuk. Demi mengurangkan kitran hayat reka bentuk, kaedah baru haruslah diaplikasi. SystemVerilog merupakan lanjuatan daripada Verilog HDL telah diubahsuaikan dan penambahan fungsi yang banayak. Antaramuka pemprograman terus ialah salah satu fungis baru yang boleh memanggil fungsi dalam bentuk C ke dalam SV atau sebaliknya. Dalam tesis ini, perisian Altera-Modelsim digunakan untuk menjalankan co-simulation terhadap kes-kes ujian yang dipilih. Antara kes ujian yang dipilih ialah 16-bit unsigned adder, pembahagi sepunya yang lebih besar (GCD), penapis FIR dan penapis median binari. Semua reka bentuk perkakasan dimodelkan dengan menggunakan SV. 16-bit unsigned adder dan GCD merupakan kes-kes ujian asas sebelum kes-kes ujian yang lebih rumit dan sebenar diaplikasikan. Data input penapis FIR dipindahkan dari penjana nombor rawak daripada fungsi C. Output penapis FIR akan disahkan dengan fungsi C. Disebabkkan saiz tetingkap penapis median adalah kecil, teknik line buffer digunakan dalam kes ujian ini. Segala masalah yang akan dihadapi semasa simulasi berdasarkan perisian Modelsim mengguanakan teknik DPI telah diatasi dengan mengaplikasikan kes-kes ujian yang dipilih.

vii

TABLE OF CONTENTS

CHAPTER TITLE PAGE

DECLARATION ............................................. ii

DEDICATION ...................................................................................... iii

ACKNOWLEDGEMENTS .................................................................. iv

ABSTRACT ........................................................................................... v

ABSTRAK............................................................................................. vi

TABLE OF CONTENTS ..................................................................... vii

LIST OF TABLES ................................................................................. x

LIST OF FIGURES .............................................................................. xi

LIST OF ABBREVIATIONS AND SYMBOLS ................................ xiii

LIST OF APPENDICES...................................................................... xv

1 INTRODUCTION ............................................................................................ 1

1.1 Background of Study .............................. Error! Bookmark not defined.

1.2 Problem Statement .................................................................................. 3

1.3 Objectives ............................................................................................... 4

1.4 Scope of the Research ............................................................................. 4

1.5 Expected Contribution ............................................................................ 5

1.6 Thesis Outline ......................................................................................... 5

2 BACKGROUND AND LITERATURE REVIEW .......................................... 7

2.1 Median Filters ......................................................................................... 7

2.2 Design of Binary Median Filter in Digital Hardware ............................. 11

1

viii

2.3 SystemVerilog ...................................................................................... 13

2.4 Direct Programming Interface ............................................................. 175

2.5 Previous Work on Co-simulation using SystemVerilog

3 METHODOLOGY ......................................................................................... 19

3.1 Overall Workflow ................................................................................. 19

3.2 Software Tools and Design Environment .............................................. 21

3.2.1 Mentor Graphic Modelsim ......................................................... 21

3.2.2 Using GUI and Command-line Version of Modelsim ................. 22

3.3 Direct Programming Interface ............................................................... 26

3.3.1 Import Function ......................................................................... 27

3.3.2 Export Function ......................................................................... 27

3.4 Test Cases for Verifying Usability of DPI-C ......................................... 28

3.4.1 Startup Test Cases ...................................................................... 28

3.4.2 9-tap FIR Filter .......................................................................... 29

4 SOFTWARE AND HARDWARE PLATFORM DEVELOPMENT ........... 30

4.1 9-tap FIR Filter ..................................................................................... 30

4.1.1 Datapath Unit ............................................................................. 31

4.1.2 Control Unit ............................................................................... 34

4.1.3 Interfacing between C and SV .................................................... 36

4.2 Noise Removal with Binary Median Filter ............................................ 36

4.2.1 Conversion of RGB Image to Grayscale Image .......................... 38

4.2.2 Datapath Unit Design ............................................................... 399

4.2.3 Control Unit Design ................................................................... 42

4.2.4 Interfacing between C and SV .................................................... 44

4.2.5 Miscellaneous ............................................................................ 45

17

15

39

ix

5 RESULTS AND DISCUSSION ..................................................................... 46

5.1 16 bit Unsigned Adder and GCD .......................................................... 46

5.2 9-tap FIR filter ...................................................................................... 49

5.3 Binary Median Filter ............................................................................. 50

5.4 Limitations ........................................................................................... 54

5.4.1 Software Tools of Modelsim ...................................................... 54

5.4.2 Hardware Design of Binary Median Filter .................................. 54

6 CONCLUSION AND RECOMMENDATION ............................................. 55

6.1 Conclusion ............................................................................................ 55

6.2 Recommendation for Future Works ...................................................... 56

REFERENCES .......................................................................................................... 57

Appendices A-C ..................................................................................................... 60-92

x

LIST OF TABLES

TABLE NO. TITLE PAGE

2.1 Partial of difference between SV and Verilog 14

2.2 SystemVerilog DPI capabilities versus Verilog PLI 16

2.3 Data types mapping between C and SystemVerilog 17

2.4 Previous work on co-simulation using SystemVerilog 18

xi

LIST OF FIGURES

FIGURE NO. TITLE PAGE

2.1 An example of 3x3 window mask for median filter. 8

2.2 An image corrupted with impulse noise [4]. 8

2.3 An image after standard median filtering using 3x3

window mask [4].

9

2.4 Generic window mask for CWM filter [11]. 9

2.5 Block diagram of tri-state median filtering scheme. 10

2.6 Searching block for median filter. 11

2.7 Pipelined of multilayer architecture for median filter. 12

2.8 Implementation of 3x3 filter window. Row buffers are

used to reduce the memory access to one pixel per clock

cycle [12].

13

3.1 Overall workflow 20

3.2 Generic block diagram of interface between

SystemVerilog and C.

21

3.3 The command-line and GUI based of Modelsim. 23

3.4 A successful output display when running DPI-C on

hello module.

25

4.1 Data flow graph (DFG) of 9-tap FIR filter. 32

4.2 Signal data flow graph (SDFG). 32

4.3 Signal flow graph. 33

xii

4.4 Functional block diagram of FIR filter. 33

4.5 ASM chart for FIR filter. 35

4.6 5x5 processing window for binary median filter. 37

4.7 Interfacing between binary median filter and conversion

of RGB image to grayscale.

38

4.8 Generic design of line buffers for window-based

computation [7].

40

4.9 Pixel representation diagram for a 6x6 image. 41

4.10 Pixel shifting in line buffers. 42

4.11 ASM chart for binary median filter. 43

4.12 Portion of C code to create a readable mif file by SV. 44

4.13 Portion of SystemVerilog code from testbench. 44

5.1 Verification of 16-bit unsigned adder using DPI-C. 47

5.2 Functional block diagram of top level entity of GCD. 47

5.3 Random numbers set generated by C code as input for

FIR filter.

49

5.4 A portion of output data from SV side of FIR filter. 50

5.5 Image of finger-vein before filtered. 53

5.6 Image of finger-vein after filtered. 53

xiii

LIST OF ABBREVIATIONS AND SYMBOLS

ALU - Arithmetic logic unit

API - Application programming interface

ASM - Algorithmic sate machine

CWM - Center weighted median

DFG - Data flow graph

DPI - Direct programming interface

FIR - Finite impulse response

FPGA - Field programmable gate array

GUI - Graphic User Interface

HDL - Hardware description language

HDVL - Hardware decription and verification language

MATLAB - Matrix laboratory

MRI - Magnetic resonance imaging

PLI - Programming language interface

RAM - Random access memory

RGB - Red Green Blue

RTL - Register transfer level

SDFG - Signal data flow graph

SFG - Signal flow graph

SM - Standard median

SoC - System-on-chip

SV - SystemVerilog

UCDB - Unified Coverage Database

xiv

VLI - Verilog Procedural Interface

2D - 2-dimensional

xv

LIST OF APPENDICES

APPENDIX TITLE PAGE

A Tutorial on how to use modelsim to run DPI 60

B Sources code for 9-tap FIR filter 68

C Sources code for binary median filter 79

1

CHAPTER 1

INTRODUCTION

This thesis proposes a co-design simulation model that utilizing the state-of-art

SystemVerilog HDVL via direct programming interface (DPI) technique on subsystems

of digital signal processing and image processing. This chapter gives an overview of the

proposed title and, problem statement, objectives, scope of work, expected contribution

and thesis outline.

1.1 Background of Study

Nowadays, modern system-on-chip (SoCs) design is getting more complex and

sophisticated. Hardware modeling or design at register transfer level (RTL) will consume

a lot of time and effort. It is impossible to integrate every algorithms or problems into a

single piece of hardware. Even it is done so, the end product of hardware will be costly

and not suitable for the market right now. The market now pressures on short design cycle,

hight fault coverage, high speed and low cost. Since the register transfer level design now

is getting more complicated, it is advisable that some of the algorithm remain in the

2

software. Hardware design is done only when performance is taken into concern. This is

due to hardware design will speed up the whole system or an application.

There is a report show that functional verification has become the biggest

bottleneck as it consumes roughly 70% of chip development time and efforts. RTL

testbenches have become more complex and difficult to manage [1]. New method has to

be introduced to reduce the verification cycle. Therefore, SystemVerilog (SV) can be used

to overcome drawbacks as aforesaid. SV is the extension of Verilog 2001 HDL, contains

both hardware description language and hardware verification language [2]. It enhances

the design specification method, testbenches language including coverage and assertions

application programming interface (API) and direction programming interface (DPI).

Direct programming interface is an interfacing between SystemVerilog and

foreign programming language. Usually, the foreign programming language is referred as

C or C++ programming language. Designers can call C or C++ code into SystemVerilog

or vice versa by using correct protocol and linking model. Hence by using SV and DPI,

designers are able to develop a fast co-simulation model to their desired applications.

Other than that, designers can use the C/C++ code output result as a golden reference to

verify the hardware model designed in HDVL.

Image processing can be defined as 2D function, f(x,y) where x and y are spatial

coordinates. Amplitude f is called intensity or gray level of the image [4]. Digital image

is a 2D rectangular array of quantized sample values of an image. These quantized values

are known as picture elements, image elements and pixels. Basically, digital image

processing is a processing of an input image to obtain desired output image. Image

analysis is understanding of an image as input to extract some useful information of image

output such as features, texture or shape. Computer vision is known as emulating human

vision such as face recognition, fingerprint recognition, tracking moving object and etc.

Image processing can be applied in many area especially, medical (MRI, X-ray,

Ultrasound), military, untouchable territorial explorations (deep space and deep sea),

security and surveillance (biometric, border control) and entertainment.

3

1.2 Problem Statement

With the massive development of electronic information system in today’s society,

SoCs have bigger and more complicated design. Previously, the hardware and software

design are done separately by different designers. The only way to verify the system

design is using field programmable gate array (FPGA). However, verification through

FPGA will take longer time and more effort due to technical details for FPGA board must

be well-equipped. As aforesaid when the design on any applications is getting bigger and

complex, the design has to be done by the same person. If the conventional method is used,

it will cause more debugging problems and interfacing between hardware and software

design. Therefore, it is necessary to ultilize the SystemVerilog HDVL which provides

interfacing between SV and C. It enables the same designers to do both software and

hardware system design on a co-simulation platform. Hence, the design lifecycle of a

system can be greatly reduced.

To achieve real time performance in image processing’s application, it is advisable

to map some systems design into hardware. It is not a good method to keep every modules

of design in software design because a high power computer is needed in order to achieve

real time performance. Thus, it is better to design an embedded system for a particular

applications. Most of the algorithms of image processing compute intensively such as

multiplication and addition. To impove the performance, it is necessary to partition the

design into hardware and software instead of keeping every algorithms in software.

4

1.3 Objectives

The project aims to develop a co-simulation model based on open source software

tool, Modelsim simulator by using SystemVerilog HDVL via direct programming

interface technique. The main objective are supported by the sub-objectives as follow:

(i) To overcome the problems and discover the limitation of Modelsim simulator

when interfacing between SystemVerilog and C functions.

(ii) To propose an efficient design of background noise removal to achieve the

real time performance.

1.4 Scope of the Research

The scopes of the work for this thesis are:

(i) SystemVerilog HDVL is used throughout the all the hardware modeling and C

programming language is applied for the foreign programming language

(Currently DPI only support C but it is designed to be extensible interface that

can support other languages [25]).

(ii) Open source software tools such as Quartus II 13.0 and Altera-Modelsim 10.1d

are used to ensure this work can be repeated and extended in future.

(iii) Test cases that are applied in this work are limited to these problems: a 16-bit

unsigned adder, greater common divisor, 9-tap FIR filter and binary median

filter.

5

1.5 Expected Contribution

The expected contribution of this work is creation a co-simulation environment

based on a free software tool, Modelsim simulator with limited features. Most of the

technical details as well as limitations about the Modelsim are overcome and discovered

by applying several test cases. Moreover, this work will apply the state-of-art hardware

description and verification language (HDVL), SystemVerilog. SV is the extension of

Verilog HDL with more advanced features such as direct programming interface (DPI).

Due to the new release of SV, not many research or papers are published on hardware

design using SystemVerilog. In SV, it allows the interfacing between hardware design

with software design. In another words, a co-simulation model for software and hardware

designs can be developed by ultilizing this state-of-art HDVL. The software designs here

are meant to C.

1.6 Thesis Outline

This thesis is organised into six chapters. The first chapter presents the

introduction of this work, problem statement, objectives, scope of work and expected

contribution of this thesis.

Chapter 2 is the background and literature review chapter. It gives some

information of subsystem of image processing, hardware design implementation of

median filter. Previous related work on state-of-the-art co-simulation using

SystemVerilog DPI is presented as well.

Chapter 3 decribes the methodology of this work. The commands and workflow

on how to use DPI in Modelsim simulator are illustrated. A startup example using DPI

that printf(“Hello World”) is presented. This is a key point before a more complex test

6

case is applied. The import and export are the main keyword to be used to call C function

or SV are explained.

Chapter 4 mainly exaplains on the algorithm of the applied in the proposed test

cases especially FIR filter and binary median filter. The method being used such as

circular buffers and line buffers for data aligning are discussed. The chapter continues

with more explanation on the hardware design from algorithms (9-tap FIR filter and binary

median filter).

Chapter 5 is the design verification and testing chapter. Results and discussion are

given in this chapter. Chapter 6 concludes the limitation of Modelsim simulator and future

work to further enhance the co-simulation using SystemVerilog DPI.

57

REFERENCES

1. Modi D., Sitapara H., Shah R., Mehul, E., (2011). Integrating MATLAB

with Verification HDLs for Functional Verification of Image and Video

Processing ASIC, 2(2), 258–265.

2. SystemVerilog 3.1a Language Reference Manual Accellera’s Extensions

to Verilog.

3. Eng P. C. (2011), "Finger-Vein Biometric Authenticatin in a System-On-

Chip Based on Field Programmable Gate Arrays". Master, Universiti

Teknologi Malaysia, Skudai.

4. Rafael C. Gonzalez, Richard E. Woods (2002). Digital Image Processing.

(Second Edition) Upper Saddle River, New Jersey: Prentice-Hall.

5. Chin C.C., Ju Y. H., Chih P. H. (2008), "An Adaptive Median Filter for

Image Denoising," Intelligent Information Technology Application. IITA

'08. Second International Symposium on , vol.2, no., pp.346,350, 20-22

Dec. 2008

6. J. Astola and P. Kuosmanen. (1997) “Fundamentals of nonlinear digital

filtering,” CRC Press, Boca Raton, FL.

7. Lee Y. H. (2013), "Finger-Vein Biometric System Implemented in an

Embedded Hardware Base on Field Programmable Gate Arrays". Master,

Universiti Teknologi Malaysia, Skudai.

8. Chen T., Ma K. K., Li H. C., (1999). "Tri-state median filter for image

denoising," Image Processing, IEEE Transactions on , vol.8, no.12,

pp.1834,1838.

9. Ko S. J., Lee Y. H., (1991) "Center weighted median filters and their

applications to image enhancement," Circuits and Systems, IEEE

Transactions on , vol.38, no.9, pp.984,993.

58

10. I. Pitas and A. N. Venetsanopoulos (1990), “Nonlinear digital filters:

principles and applications,” Kluwer, Boston, MA.

11. Sun T., Gabbouj M., Neuvo Y. (1995), "Anaylsis of two-dimensional

center weighted median filters", Multidimensioanl Systems and Signal

Processing, Vol 6, Number 2, Page 159-172

12. Vasicek Z., Sekanina L. (2008),"Novel Hardware Implementation of

Adaptive Median Filters," Design and Diagnostics of Electronic Circuits

and Systems, 2008. DDECS 2008. 11th IEEE Workshop on , vol., no.,

pp.1,6, 16-18.

13. Fahmy S.A., Cheung P. Y K, Luk W., (2005) "Novel FPGA-based

implementation of median and weighted median filters for image

processing," Field Programmable Logic and Applications, 2005.

International Conference on , vol., no., pp.142,147, 24-26.

14. Abadi H.Z.H., Samavi S., Karimi N. (2013), "Image noise reduction by

low complexity hardware median filter," Electrical Engineering (ICEE),

2013 21st Iranian Conference on , vol., no., pp.1,5, 14-16.

15. Leeb S.B., Ortiz A., Kirtley J.L.(1991), "Real-time median filtering with

a fast hardware sorter," Applied Power Electronics Conference and

Exposition, 1991. APEC '91. Conference Proceedings, 1991., Sixth

Annual , vol., no., pp.254,260, 10-15.

16. Hiasat A.A., Al-Ibrahim M.M., Gharaibeh K. M., (1999) "Design and

implementation of a new efficient median filtering algorithm," Vision,

Image and Signal Processing, IEE Proceedings - , vol.146, no.5,

pp.273,278.

17. Chatterjee S., Ray A.K., Karim, R., Biswas A.,(2011) "Architecture

Design for Median Filter," Computer Vision, Pattern Recognition, Image

Processing and Graphics (NCVPRIPG), 2011 Third National Conference

on , vol., no., pp.247,250, 15-17.

18. Modelsim Manual 10.1d.

19. Khalil-Hani, M. Digital Systems: VHDL & Verilog Design. UTM

Skudai: Prentice Hall. 2012.

59

20. Mohamed Khalil Hani, PhD (2013) RTL Design for FPGA with

SystemVerilog, 1st Edition. UTM Skudai. Desktop Publisher.

21. Delgado, M. and Pegalajar, M. A multiobjective genetic algorithm for

obtaining the optimal size of a recurrent neural network for grammatical

inference. Pattern Recognition, 2005. 38(9): 1444–1456. ISSN 00313203.

doi:10.1016/j.patcog.2004.03.026.

22. Cochran, J., Horng, S. and Fowler, J. A multi-population genetic

algorithm tosolve multi-objective scheduling problems for parallel

machines. Computers& Operations Research, 2003. 30(7): 1087–1102.

23. Thompson C. D. (1983), "The VLSI complexity of sorting", IEEE Trans.

Computers, vol. 32, pp. 1171-1184.

24. Karaman M., Onural L., Atalar A. (1990), "Design and implementation

of a general-purpose median filter unit in CMOS VLSI", IEEE J. Solid-

State Circuits, vol. 25, pp. 505-512.

25. S. Sutherland, The Verilog PLI is Dead (maybe) Long Live the System

Verilog DPI! (2004).

26. Ruan, A. W., Liao, Y. B., Li, P., Huang, H. C., & Devices, I. (2010). A

Run- Time RTL Debugging Methodology for FPGA-based Co-

Simulation, 891–895

27. You, M.-K., & Song, G.-Y. (2009). Case study : Co-simulation and co-

emulation environments based on SystemC & SystemVerilog. TENCON

2009 - 2009 IEEE Region 10 Conference, 1–4.

28. Li, W. (2009). Throughput Estimation for ModelSim Simulator Tool

Based I ~ j ~, 1014–1018.

29. Chris S., (2008). SystemVerilog for Verification A Guide to Learning the

Testbench Language Features, 2nd edition. Spring Street, New York:

Springer.