TEXTURE ANALYSIS OF ULTRASOUND IMAGES TO

ASSESS MEAT QUALITY IN BEEF CATTLE

A Thesis

Presented to

The Faculty of Graduate Studies

of

The University of Guelph

by

JABER JUNTU

In partial fulfillment of requirments

for the degree of

Master of Science

Decernber: 1998

National Library 1+1 d,", Bibliothèque nationale du Canada

Acquisitions and Acquisitions et Bibliographie Services services bibliographiques

395 Wellington Street 395, rue Wellington OttawaON K I A O N 4 Ottawa ON K1 A ON4 Canada Canada

The author has granted a non- exclusive licence allowing the National Library of Canada to reproduce, loan, distribute or sell copies of this thesis in microform, paper or electronic formats.

The author retains ownership o f the copyright in this tbesis. Neither the thesis nor substantid extracts fiom it may be printed or otherwise reproduced without the author's permission.

L'auteur a accordé une licence ncn exclusive permettant a la Bibliothèque nationale du Canada de reproduire, prêter, distribuer ou vendre des copies de cette thèse sous la forme de microfiche/nIm, de reproduction sur papier ou sur format électronique.

L'auteur conserve la propriété du droit d'auteur qui protège cette thèse. Ni la thèse ni des extraits substantiels de celle-ci ne doivent être imprimés ou autrement reproduits sans son autorisation.

ABSTRACT

TEXTURE ANALYSIS OF ULTRASOUND IMAGES TO

ASSESS MEAT QUALITY lN BEEF CATTLE

Jaber Guma Juntu

University of Guelph, 1998

Advisors:

Professor O.A. Basir

Professor Valerie J . Davisdon

Meat Grading represents a major concern for the meat industry, as it sets the basis for

pricing meat based on its quality. The current practice to assess marbling garde is done by

human experts by loolgng at the ribeye muscle between the and 1 3 ' ~ rib bones, which is

a very subjective approach and prone to high error rates. The main objective of the thesis

is to grade the meat of live beef animals using ultrasound images of the ribeye muscle. Two

classification algorithrns are employed, namely, the Minimum Euclidean Distance classifiers and

the k-means clustering algorithm. The input of the classifiers is the texture features vectors

e~ t rac ted hom ultrasound images using the Co-occurrence matrix and the Laws masks. To

irnyrove the classification results, the texture features are transformed using the whitening

transformation process to make them uncorrelated and to normalize their scale. A feature

selection process is ais0 used to select the best features that are more correlated to meat grade.

To furtber improve the results and to reduce the uncertainty associated with the classification

results, the classifiers are fused using the Generalized Bayesian Fusion approach. The GBF

approach uses Bayes' theorem to combine the results of two or several classifiers.

ACKNOWLEDGMENTS

It is of my great pleasure to thaak my advisors Prof. O. A. Basir and Prof. V. Davidson for

their guidance, continues support, and helpful suggestion and comments. They provided much

of the expertise needed to complete this study. Moreover, their different backgrounds, one in

food science and the other in engineering added great value to the thesis.

1 would like to thank Prof. R. Brown for providing the ultrasound images that were used in

this research. His previous research in meat quality assessrnent was the starting point of the

curent work.

1 wish to extend my appreciation to the examining community, namely, Prof. R. Dony and

Prof. W. R. Smith for their work and helpful suggestions.

I would like to acknowledge the assistant given by other faculty members and colleagues in

SchooI of Engineering.

Finally, 1 dedicate this work to my parents in rny home country, and to my wife whom

without her encouragement, support and patience, this work would not have been realized.

Contents

1 Introduction 1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Meat Grading 1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Marbling Approaches 3

. . . . . . . . . . . 1.3 Modeling of the Interaction Between Ultrasound and Tissues 3

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Review of Relevant Literature 5

. . . . . . . . . . . . 1.4.1 Tissues Characterization Based on Tissues Modeling 6

. . . . . . . . . . . . . . . . . . . . 1.4.2 Marbling Estimation Based on Speckle 8

. . . . . . . . . . . . . . . . . . . . . . . 1.4.3 S t at ist ical Modeling Techniques 9

. . . . . . . . . . 1.4.4 Texture Extraction and Pattern Recognition Techniques 12

. . . . . . 1.4.5 Commercial Systems for Marbling and Fat Content Assessrnent 14

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4.6 Classifiers h i o n 15

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 The Thesis Objective 17

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Organization of the Thesis 18

2 Texture Analysis and Pattern Recognition Systems 20

. . . . . . . . . . . . . . . . . . . . . . . . 2.1 Meat Grading As a Texture Analysis 20

. . . . . . . . . . . . . . . . . . . 2.2 Pattern Recognition Systems for Meat Grading 22

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Designing a Classifier 24

. . . . . . . . . . . . . . . 2.3 -1 Minimum Euclidean Distance Classifier (MED) 25

. . . . . . . . . . . . 2.3.2 Maximum A Posterior Probability CIassifier (MAP) 26

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.3 k-means Clustering 27

3 Probabilistic Approach for Meat Grading 28

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction 28

. . . . . . . 3.1.1 Texture Features Based on First-Order Probability Function 28

. . . . . . . . . . . . . . 3.1.2 Second-Order Joint Probability Texture Features 31

. . . . . . . . . . . 3.1.3 Texture Features Based on the Co-occurrence Matrix 33

. . . . . . . . . . . . Designing Classification Algorithms to Predict Fat Contents 36

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Data Description 36

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Preprocessing 37

. . . . . . . . . . . . . . . . . . . . . . 3.2.3 Image Filtering and Xoise Removal 38

. . . . . . . . . . . . . . . . . . . 3.2.4 Identifying The Area Of Interest (AOI) 40

. . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.5 Selecting the Window Size 41

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Experirnental Setup 41

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Features Extraction 42

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Designing a Classifier 44

. . . . . . . . . . . . . . . . . 3.3.3 Classifier Evduation and Error Estimation 45

3.3.4 Experiment (I3gD) : Designing MED Classifier Using Haralick's Fea-

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . tures to Estimate Fat 47

3.3.5 Experiment ( H z m S ) : k-means Clustering Using Haralick's Features

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . to Estimate Fat 48

. . . . . . . . . . 3.3.6 Comments About Experirnents HE^) and ( H g m s ) 49

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Whitening of the Feature Space 50

3.4.1 Experiment (WED): Designing LMED Classifier Using Whit ened Har-

. . . . . . . . . . . . . . . . . . . . . . . . alick's Featues to Estimate Fat 52

3.4.2 Experirnent (w13Ems): k-means Clustering Using Whjtened Haral-

. . . . . . . . . . . . . . . . . . . . . . . . . ick's Features to Estimate Fat 53

. . . . . . . 3.4.3 Comments about Ekperirnents (WHED) and ( W H E m S ) 54

. . . . . . . . . . . . . . 3.5 Adding Random Data and Selecting the Best Features 54

3.5.1 Experiment (R+HKD) : MED Classifier Using Larger Data Sample and

. . . . . . . . . . . Employing Feature Selection Process to Estimate Fat 55

3.6 Designing Classifiers Based on Marbling Score Using Haralick's Features . 57

iii

3.6.1 Experirnent (H~~Z-): k-means Clustering Using Harakk's Features

. . . . . . . . . . . . . . . . . . . . . . . . . . . to Assess Marbling Grade 58

3.6.2 Experirnent (wHZans): k-means Clustering Using Whitened Haral-

ick% Features to Assess Marbling Grade . . . . . . . . . . . . . . . . . . . 59

3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

4 Meat Grading Based on Image Tkansforms 60

4.1 Texture Features Extraction by Laws7 Masks . . . . . . . . . . . . . . . . . . . . 61

4.2 Designing Classification Algorithms Based on Laws' Masks Features . . . . . . . 64

. . . . . . . . . . . . . . . . . . . . . . . . . . 4.2. 1 Texture Features Extraction 64

4.2.2 Experiment (L"~) : Designing MED Classifier Using Laws' Features to

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Estimate Fat 66

4.2.3 Experiment (L"~"~) : k-means Clustering Using Laws' Features to Es-

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . timate Fat 67

. . . . . . . . . . . . . . . . . . . . . . . . 4.3 Whitening of the Laws' Feature Space 68

4.3.1 Experiment (WLED) : Designing MED Classifier Using Whitened Laws'

. . . . . . . . . . . . . . . . . . . . . . . Texture Features to Estirnate Fat 68

4.3.2 Experiment (WL$s~) : k-means Clustering Using Whitened Laws' Fea-

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . t u e s to Estimate Fat 70

. . . . . . . . . . . . . . . 4.4 Adding Random Data and Selecting the Best Features 70

4.4.1 Experiment (R + L"~): MED Classifier Using Larger Data Sample and

. . . . . . . . . . . Employing Feature Selection Process to Estimate Fat 71

. . . . . . . . . . . 4.5 Classification Using Laws' Features to Assess Marbling Grade 72

4.5.1 Experirnent (TLZanS): k-means Clustering Using L a d Features t o

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assess ~Marbling Grade 72

4.5.2 Experirnent (WMLmzms): k-means Clustering Using Whitened Laws'

. . . . . . . . . . . . . . . . . . . . . . Features to Assess Marbling Grade 72

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Summary 72

5 Fusion of Multiple CIassifiers 74

5.1 Methodologies for Combining Multiple Classifiers . . . . . . . . . . . . . . . . . . 75

. . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Classifiers Fusion by Voting 75

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.2 The Averaged-Bayesian 76

. . . . . . . . . . . . 5.1.3 The Generalized Bayesian F'usion Approach (GBF) 79

. . . . . . . . . . . . . . . . 5.2 Fusion of k-means Classifiers by the GBF Approach 82

5.2.1 Experiment (HL~:~): Fusion of k-means Classifiers to Estimate Fat . . 82

5.2.2 Experiment ( w H L ~ ~ ) : Fusion of k-means Classifiers Using Whit ened

. . . . . . . . . . . . . . . . . . . . . . . . . . . Features to Estimate Fat 88

. . . . . . . . . . . . . . . . . . . 5.3 Fusion of Classifiers Based on Marbling Grade 91

5.3.1 Experiment (HL, 1: Fusion of k-means Classifiers to Assess Marbling

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Grade 91

5.3.2 Experiment ( W H L ~ ~ ~ ) : Fusion of k-means Classifiers Using Whitened

. . . . . . . . . . . . . . . . . . . . . . Features to Assess Marbling Grade 94

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Conclusion 97

6 Discussion and Suggestions 98

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Results Discussion 98

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Future Developments 103

A Glossary 113

B Image Filtering 115

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B . 1 Speckle Noise 115

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.2 Spatial Averaging of SpecHe 116

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3 Wavelet Denoising Algorithm 116

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.4 Conclusion 119

C Sammon's Mapping Algorit hm 120

List of Figures

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 The human preception process

. . . . . . . . . . . . . . . . . . . . . . 2.2 Stages of typical pattern recogntion systern

. . . . . . . . . . . . . . . . . . . . . . 3.1 Digital image has a resolution of 512 x 512

. . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Histogram of the image in Figure 3.1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Video image of a ribeye area.

. . . . . . . . . 3.4 An ultrasound image of the same ribeye area shown in Figure 3-3

3.5 The selected Area Of Interest (AOI) used t o develop the classification algorithm .

. . . . . . . . . . . . . . . . . . . . . 3.6 Block diagram shows the experimental setup

. . . . . . . . . . . . . . . . . . . . 3.7 The different segments of the Area Of Interest

. . . . . . . . . . 3.8 A hypothetical 2-dimensional feature space of ultrasound images

. . . . . . . . . . . 3.9 Experiment (H"~) : MED classifier using Haralick's feat ures

3.10 Experiment (HE-') : k-means classifier using Haralick's features to estimate

fat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.11 Experiment WH"^) : MED classifier using whitened Haralick's features to

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . estimate fat

3.12 Experiment WH"^^) : k-means clustering using whitened Haraück's features

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . to estimate fat

. . . . . . . . . . . . 4.1 The process of extraction of texture features by Laws' masks

. . . . . 4.2 Experiment (L"~) : MED classifier using Laws' features to estimate fat

4.3 Experiment ( L E a n S ) : k-means clustering using Laws' texture features to esti-

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . mate fat

4.4 Experiment (mgD): MED classifier using Laws' texture features to estimate

fat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

4.5 E ~ ~ e r i r n e n t ( W L ~ ~ ~ ~ ) : k-means clustering using whitened Laws' features to

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . estimate fat 70

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 The voting approach 76

. . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 The Averaged Bayesian Approach 78

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 -3 The Bayes' theorem 79

5.4 Block diagram of Experiment (HL~:~): fusion of two k-means classifiers by the

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . GBF approach 83

5.5 Block diagram of Experiment ( u ~ H L ~ : ~ ) : fusion of two k-means classifiers by

. . . . . . . . . . . . . . . the GBF approach.The classifiers use whitened features 88

. . . . . . . . . . . . . . . . . . B . 1 Xon.filtered, contrast enhanced ultrasound image 117

B.2 The image in Figure B-1 Mtered by a n averaging filter of size 3x3 . . . . . . . . . 117

B.3 Ultarsound image in Figure B-1 as filtered by the wavelet denoising algorithm . . 118

vii

List of Tables

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 The 13 Canadian beef grades 1

. . . . . . . . . . . . . . . . . . . . 3.1 The eight neighbors and angles of a pixel (*) 32

. . . . . . . . . . . . . . . . . . . . . . . . . 3.2 7x7 image has four graylevels 0.1,2,3 32

3.3 The Ceoccurrence matrices for the above image for different angles and dis-

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . tances 32

3.4 The data samples divided into three groups based on fat chernicd andysis . . . . 37

3.5 Confusion matrix of Experiment ( H " ~ ); MED classifier using Haralick fea-

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . tures to estimate fat 48

3.6 Confusion matrix of Experiment (H:Z~~*); krneans using Haralick's features

to estimate fat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.7 Conhision matrix of Experiment (WHED) ; MED cIassifier using Cniitened

. . . . . . . . . . . . . . . . . . . . . . . . . . . Haralick features to estimate fat 52

3.8 Confusion matrix of Experiment (WHZmS); krneans using Haralick's features

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . to estimate fat 53

. . . . . . . . . . . . . . . . . . . . . . . . . . 3.9 Computing the intra-class distances 55

3.10 Results of Experiment (R+H"D); MED classifictaion after adding random

. . . . . . . . . . . . . . . data and using feature selection process to estirnate fat 56

. . . . . . . 3.11 Data samples divided into three groups based on the marbling scores 58

3.12 Conhision mat rix of Experiment (H""s) ;bmeans using Haraiick's features to

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . assess marbling garde 59

3.13 Confusion matrix of Experiment (WH"-); krneans using Whitened Haral-

. . . . . . . . . . . . . . . . . . . . . . . . . ick's features to assess rnarbling grade 59

viii

4.1 One-dimensional Laws' masks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 The 5x5 Windows obtained by convoluting the one-dimensional filters given in

Table 4.1. . . . . . . . . . . . . . . . . . . . . . . . . . - . . . . . . . . . . . . . . 4.3 The set of texture images (TI) of the image X(i j). . . . . . . . . - . . . . . . . . 4.4 The 25 Density Texture Images (TDI) of image X(i j). . . . . . . . . . . . . . . . 4.5 The fifteen Laws' features . . . . . . . . . . . . . . - . . . . . . . . . . . . . . . . 4.6 The Conhion matrix of E-xperiment (L"~); MED classifier using Laws' fea-

tures to estimate fat. . . . . . . . . . . . . . . . . . . . - . . - . . . . . . . . . . . 4.7 The confusion rnatrix of Experiment (L""s) ;k-means classifier using Laws'

features to estimate fat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . - - -

4.8 The confusion matrix of Experiment (mgD); MED classifier using whitened

Lam' features to estimate fat. . . . . . . . . . . . . . . . . . . . . . . . . - . - .

4.9 Confusion matrix of Experiment (WL"~"~); k-means classifier using whitened

Laws' features to estimate fat. . . . . . . . . . . . . . . . . . . - . . . . . . . . . .

4.10 Results of Experiment (R + L"~); using Laws' features and applying feature

selection process to estirnate fat. . . . . . . . . . . . . - . . . . . . . . . . . .

4.11 Confusion matrix of Experiment ( L ~ F - ~ ) ; k-means classifier using Laws7 fea-

t u e s to assess marbiing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -

4.12 Confusion matrix of Experiment (L$Z~~); k-means classifier using whitened

Laws7 features to assess rnarbling. . . . . . . . . . . . . . . . - . . . . . . . . . . .

5.1 The results of Experiments ( H ~ Z - ~ ) , ( L F F ~ " ~ ) and Experiment ( H L ~ : ~ ) ;

fusion of two k-means classifiers using the GBF approach to estimate fat. . . . . 5.2 The confusion matrix of Experiment ( H L ~ ~ ) ; fusion of two k-means classifiers

to estirnate fat. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . kmeans 5.3 The results of Experiments ( H " ~ ~ ) , (LFat ) and ( w H L ~ ' ~ ; fusion of

k-means classifiers using whitened features to estimate fat. . . . . . . . . . . . . 5.4 The confusion rnatrix of Experiment ( w H L ~ ~ ) ; h i o n of two k-means clas-

sifers with whitened features to estimate fat. . . . . . . . . . . . . . . . . - . . . 5.5 Results of Experiment (HL&%~); fusion of k-means classifiers to assess rnarbling.

The confusion matrix of Ekperiment (HL$EF); hision of k-means classifiers to

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . assess rnarbling grade 93

The results of Experiement (wHL~:~); fusion of k-means classifier to assess

. . . . . . . . . . . . . . . . . . . marbling . The classifiers used whitened features 96

Confusion rnatrix of Ekperiment (wHL&B,F ). h i o n of two k-means classifiers

. . . . . . . . . . features to assess marbling . The classifiers use whitened features 96

. . . . . . . . . . . . . . . . . . . . . . . . Results based on chernical fat analysis 100

Results based on animal marbling grade . The (*) indicates a nearly singular

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . covariance matrix 100

. . . . . . . . . . F'usion of the best supeMsed classifiers to estirnate fat contents 101

. . . . . . . . . . Fusion of the best s u p e ~ s e d classifiers to assess marbling grade 101

. . . . . . . . . Fusion of the best unsupervised classifier to estimate fat contents 101

. . . . . . . . Fusion of the best u n s u p e ~ s e d classifiers to assess marbling grade 101

Chapter 1

Introduction

1.1 Meat Grading

In 1929, Canada initiated a program for grading the qualiv of beef meat under the name

'Canadian Beef Grading Program' to cornplement its existing meat inspection program. Since

then, millions of beef carcasses marketed in Canada were inspected and graded according to

both programs. Their haI marketing value was largely depends on the estirnated grade. Grade

standards and regdations are enforced by the Government of Canada through Agriculture and

Agri-Food Canada.

The Canadian Beef Grading Program defines four classes of meat quality. Each class is

divided into grades as shown in Table 1.1:

Table 1.1: The 13 Canadian beef grades.

The key grading criteria for the quality are:

Percentage (1996) 89.0% 3.1% 7.6% 0.3%

Class Name A B D E

Grades A,AA,AAA,Prime

B1 ,B2,133,B4 Dl,D2,D3,D4

E

muscling,

meat quality (such as texture, color),

external fat covering, and,

marbling.

Class A is the top quality meat that represents almost 90% of all the graded Canadian beef

carcasses. AU the gades within Class A have the same grading criteria in terms of age, size of

external fat, and meat quality, but differ in terms of the degree of marbhg. Marbling refers to

the s m d white flakes of fat deposited through lean meat and represents an important factor for

marketing because it adds tendemess, taste, juiciness and results in a more consistent product.

For class A grades, there is no quantitative measure of the amount of fat that can be used to

differentiate between the different grades of meat in this class. However, the official grading

specification designates the following subjective measures to assign a marbling grade to a given

animal based on the amount of fat. These measures are:

Trace marbling is designated as grade A

Slight marbling is designated as grade AA

S m d marbling is designated as grade AAA

Slightly abundant marbling is designated as Prime grade

At the present moment, the practice to assess the marbling grade is a subjective evaluation

of carcasses by skilled human graders by looking at a cross-sectional area of the ribeye or

longissimus dorsi (LD) muscle at the interface between the 12'~ and 1 3 ~ ~ ribs. Since human

perception error to estimate the abundance and distribution of fat can be as high as 20% [Il, and

the marbling grades as described by the Canadian Beef Grading Program are highly subjective,

an objective measure is needed to supplernent the current grading system.

1.2 Marbling Approaches

Different technologies demonstrated themselves as a potentid t ethniques t O determine, marbling

grades and to estimate meat quality. These techniques include X-ray, LNM.R, EIectronic Meat

Measuring Equipment (EMM), infrared rdectance and ultrasound. Each technique is based

on a different physical concept. For example, X-ray maps the density difference in tissues as

a diffaence in gray-level intensity in the X-ray image. EMM uses the electrical conductivity

to measure the difference between tissues. Infrard reflectance relies on the diEerence of the

absorption of tissues to infrared waves. Findy, ultrasound imaging measures the parameters

of ultrasound wave when passed through tissues,

In a meeting that was sponsored by the USDA' in 1984 to identiS the most encouraging

technology to grade beef, the decision was clearly in favor of ultrasound technology because

of cost, safety, and ease of use considerations [2]. In fact, since the early 1970s it has been

recognized that measurement of dtrasonic attenuation and sound wave scatter characteristics

would be useful for noninvasive tissue characterization in many applications. Unfortunately,

no estimation dgorithm or procedure has proved to be useful for practical applications due to

the lack of complete understanding of the interaction between the acoustic field and biological

tissues.

1.3 Modeling of the Interaction Between Ultrasound and Tissues

A wave can be considered as a physical phenomenon that repeats itself regularly in space or

time. It could be a variation in the electrical and magnetic fields of the medium as it is the

case of an electromagnetic wave, or it could be a movement of the medium particles as in the

mechanical waves, typicdy known as sound waves. There are two distinct types of sound waves:

compression and shear sound waves. The primary difference between them is the direction of

the medium particles' movement with respect to the direction of sound wave propagation. In

the case of compression waves, the direction of the medium particles' movement is parallel

to the wave propagation direction, while in the case of shear waves, the particles move in a

perpendicular direction with respect to the wave propagation. Both waves need a medium

- - -

' ~ h e United States Department of Agriculture

, such as solids, fluids or gases, to propagate. Sound shear waves are more evident when

sound propagates in solid materials. Its frequency is twice the fiequency of compression waves.

Medical imaging instruments use compression waves as they penetrate more deeply inside Iive

tissues than shear waves. Also shear waves attenuate and scatter rapidly which limit their usage

for medical imaging.

Sound waves span the frequency range from a few Hertz to several Mega Hertz. Perceived

audible sound spans the range 20Hz up to 20kHz. Ultrasound waves fa11 above that range. In

medical applications, the typical kequency lies between lMHz and IOMHz. Higher Erequencies

can be used at the expense of the depth of penetration inside biological tissues. Sound waves is

describecl by a set of different parameters such as pressure, densiw temperature and particle

displacement. These parameters are the same parameters used to mode1 the mechanical behav-

ior of materi*, which shows that the interaction between sound wave and tissues is defkitely

a mechanical interaction.

The speed of a sound wave varies considerably when it passes through different media.

Furthermore, its physical properties such as frequency, phase, and wave intensity change. U-

trasonic imaging uses these facts to get a picture of the tissues inside of the human or animal

body that is opaque to light waves.

Images obtained by acoustic radiation are clearly inferior to images obtained by opticai

radiation. The primary reason is the longer wavelength of sound waves compared to electro-

magnetic wavelength which makes the resolution of ultrasouad images very low. Moreover, the

coherent propagation nature of sound waves adds undesirable noise in ultrasound images that

make them difEcult to perceive and interpret.

Tissue texture, as imaged by ultrasound, is a pattern comprised of dots of various size,

shape, position, and brightness. The texture varies with the type of tissues that reflect the

sound wave back to the ultrasound machine transducer. The primary rnechanism for this

interaction is cornplex. However, two major processes contribute to the appearance of such a

texture, namely, the structure of the tissues and the interference of the echoed ultrasound waves.

The soft tissues such as fat aad muscle are postulated to be a tiny scatterers and reflectors that

are randomly distributeci in size and position. Such scatterers reflect the sound wave back to

the transducer at random times and with different wave intensities. The echoes interfere at the

transducer in a constructive or a destructive way depending on their phase. For example, if two

echoes having the same phase angle interfere at the transducer, they reinforce. Conversely, if

the echoes arrive out of phase, they cancel each other- The net result of the interaction appears

in dtrasound images as a complex pattern known as speclde.

In order for ultrasound imaging to be widely acceptable for meat grading and tissue char-

acterization, it must be developed so that it satisfies the following conditions:

0 Price: The price tag of the dtrasound unit and the supporting equipment should be low,

0 Simplicity of usage: The unit should be simpIe to operate and maintain,

Portability: The unit size should be small so that it can be moved e a d y to the field

where it can be used,

Robustness: The unit should be robust to environment conditions such as the variation

in temperature and humidity.

1.4 Review of Relevant Literature

Great demands arise in the meat industry to evaluate animal characteristics in terms of carcass

weight, fat cover, meat yield and marbling grade, mostly to be used for pricing. Animal breeders

also use such animal characteristics to monitor the developrnents and growth of their breeds.

Major beef production countries established different specification for evaluating animal grade

and motivated research in meat grading. Most of the research in that area has been done in the

United States. Other research is conducted in Australia, Japan, and England. Recently there

is a growing interest in meat grading in Canada [3, 4, 51.

The techniques that are based on ultrasound are always preferred due to ease of use, cost and

safety. Moreover, ultrasound can be used on live animals, which helps breeders and produces

to monitor and predict the value of animals pnor to marketing.

There are two different ultrasound techniques used in the assessrnent of fat contents and

marbling grades. A-mode (Amplitude mode) ultrasound measures the strength or amplitude

of the reflected signal recorded over time. It is basically a on4imensional signal, where the

horizont al axis represents the traveling time and the vertical axis represents the amplitude

of the echoes reflected back fkom the tissues to the transducer. An ultrasound transducer

that generates ultrasound waves is directed into a single path inside the tissue. The reflected

echoes from the different tissue interfaces dong this path are displayed as a series of spikes.

The position of the spikes denotes the depth of the interface and the amplitude of the spikes

represents the strength of the reflected echoes.

In B-mode (Brightness mode) ultrasound imaging, the amplitude of the sound echoes is

represented by a graylevel or a brightness value in m ultrasound image. Each ultrasound image

represents a full cross section inside tissues. There are several methods for constructing B-mode

ultrasound image, but in all cases the image represents a plane away kom the transducer, as

opposed to a ray, as in the case of A-mode 16, 7, 8, 9, 10, 111.

1.4.1 Tissues Characterization Based on Tissues Modeling

Ultrasound images are visually inferior compared to photographie or X-rays images as they need

several years of Iearning and experience to interpret and extract usefd information from them.

For example, in X-ray images of the chest, the rib bones and the lungs can be identified easily.

On the other band, in ultrasound images of the human fetus, the main features such as the

kidneys and heart of the fetus are not a predominant features. ~1trasonogra~heP identih such

features £rom the texture pattern , and the shadow they create in ultrasound image compared

to other tissues. DifTerent tissues produce different texture patterns because they interact

differently with the ultrasound field.

Two cornmon models are used to model the interaction between ultrasound and tissues, the

layered homogeneous media model and inhomogeneous random scatterers model,

In the iayered homogeneous mode1 the tissue is postulated to have different layers with

different acoustic impedances.. This mode1 results in a straightforward determination of tissue

reflectively and hence tissue density. The model seems well suited to tissues and organs which

are intrimicaDy layered in nature [12].

The inhomogeneous random scatterers model represents the tissue by an absorbing matrix

material with random point scatterers randomly distributed in size and position. When an

'~ltrasound images interpreter

incident acoustic wave interacts with this medium, reflections occurring at each scatterer result

in a complicated signal received by the transducer which appear in ultrasound images as a

random pattern of spots of different sizes and different graylevels. This pattern, known as

speckle noise, is not unique to ultrasound images. It is also common in imaging techniques

that use coherent sources such as laser, and radar signals. The mathematical representation

of this mode1 is very cornplex, however it can be simplified by assuming that the scatterers

contribution to the backscattered signal are the same and they act independently 113, 14, 151.

Speckle noise is not strictly random in the same sense as electrical noise. If the same object is

scanned twice under the same conditions, the statistical properties of the speckie noise remain

the sarne which suggests that speckle noise reveals the media properties. The size of the

scatterers and the fkequency of the incident sound wave change the statistical properties of the

backscattered signal received by the transducer. Some researchers suggested that speckle noise

has a Rayleigh probability distribution which is different fcom the electrical white noise which

has the Gaussian distribution [16, 17, 181. The Rayleigh distribution is more skewed toward

the mean of the signal. Narayanan in [19] modeled the speckle noise by the k-distribution to

encompass a wide range of distributions such as Rayleigh, Lognormal, and Rician, which might

help to include the non-Rayleigh behavior of the speckle noise. What is more important about

speckle noise that concems this thesis is that speckle noise is correlated to tissues structures, and

also speckle noise is signal dependent and should not be filtered. Part of this thesis investigated

the dependency between speckle noise and the ultrasound signal. For a more comprehensive

treatment of the subject of ultrasound and tissue interaction rnodeling, the reader can refer to

Pol. Several investigators such as Burchardt in [16] and Wagner in [18] have used stochastic

signal theory to analyze speckle noise. Speckie noise is a coherent noise that is very similar

to the noise in laser images. They showed that the speckle noise is autocorrelated and it is a

complicated function of the roughness of the surface that reflects or scatters ultrasound waves.

Much information can be deduced about the tissue structure from the second-order statistics

(e.g., variance and autocorrelation). For example, Y. Lui in [21] found that the neighbor

pixels (in ultrasound &mode image) are correlated in the direction normal to ultrasonic wave

propagation (T > 0.95) with s rnd amount of variation (02 < 0.00017). We used this direction

in this thesis to cornpute the coocurrence matrix ( s e Section 3.3.1). This autocorrelation can

be used to build a model t o predict the marbling garde of live beef animds.

Lizzi et al. derived andytical models that described the interaction between dtrasound

signal and tissues [22]. The models had proven to be useful to understand how tissues' fea-

tures were related to rneasured ultrasonic parameters. For example, the study showed that the

effective scatterer size, which was related directly to the type of the reflecting tissue, was a

predominant factor in detennining the slope and intercept values of ultrasonic spectra. The

acoustical impedance and concentration of the scatterers iduenced the spectral intercept only.

In ophtalmology the analytical results are of direct use in interpreting clinicd spectra in te-

of morphological features. The study also investigated the importance of specific tissue fea-

tures in determinhg ultrasonic spectral signatures that have proven to be diagnosticdy useful.

Three models of tissue microstructure were considered and the results were compared to clinical

database information and excellent correspondence was found.

Kuc 1986 et al. [23] computed the Kurtosis fiom the ultrasound radio-£requency' signal

to distinguish between normal and fatty-iatrated human Liver. Kurtosis (k) is a statistical

parameter represents the fourth nonnalized moment. The k-value is a measure of the peakiness

of random signals and normally used as an indicator of the distribution of a random signal.

For example, the k-value of the Laplacian distribution is 6, for the Gaussian distribution is 3,

and for the uniforrn distribution is 1.8. The results of the study suggests that the reflectors in

normal liver approximates a Gaussian distribution while those in the fatty liver are uniformly

distributed. The reflectors reflect the ultrasound signal such that it has the same distribution

as the reflectors.. The motivation of the study was that normal liver and fatty-infiltrated liver

produce signals that has different k-values.

1.4.2 Marbling Estimation Based on Speckle

Some authors used speckle noise pattern in ultrasound images to assess meat quality.

In a study made by Brethour [24], a subjective scoring system was devised to grade ani-

mal marbling based on speckle on ultrasound images. The speckle-rnarbling scores that were

estimated visually £rom speckle pattern in ultrasound imaging system rnonitor and the act ual

3 raw ultrasound signal without any processing.

marbling scores were correlated. The accuracy of the marbling scores as estimated £rom ultra-

sound images was 77%. For an ultrasound image of a given animal, specklemarbling scores

were estirnateci by looking at texture features at three different regions of interest. The kst

feature was the density of the coherent speckle density over the longissimus muscle. The second

was the echogenicity of the rib bone. The h a 1 feature was the speckle that registered in the

nether region below the rib bone. The study used an Aloka 210 Bmode ultrasound system

with 14 gray-shades which did not give enough contrast to differentiate between fat and meat.

In another study that was made by Liu et al. [21], the main objective was to develop a

statistical model based on the autocorrelation of ultrasound speckle in &mode image for the

evaluation of beef marbling. A total of 60 ultrasound images were obtained fiom two sets of beef

populations using an Aloka Mode1 500Mhz ultrasound unit. The marbling scores were obtained

subjectively by experienced meat experts based on the USDA designation. A statistical model

that relates the marbling score (intramuscular fat) to speckle noise was built. The relationship

between the overall variance intensity of a selected area fiom an ultrasound image and the noise

variance had been theoretically derived through the analysis of an autoregression model. The

conclusion of the study showed that using the autocorrelation to study marbling in ultrasonic

images was superior or at least equivalent, in terms of residual standard error and coefficients

of determination, to other methods seen in other literatures.

1.4.3 Statist i d ModeIing Techniques

Eventhough there are a large amount of research on meat quality assessrnent, most of it was

conducted by researchers of animd science background. Their approach to predict marbling

grade or fat content was building a statistical mode1 such as, linear regresçion or correlation,

£rom a rneasurements taken from live or slaughtered beef anirnals using A-mode or B-mode

ult r asound syst ems.

In Japan, a research was conducted by Qzutsumi [25] to predict th4,fat contents of Black

steers. Seventeen Japmese Black steers were scanned by the color sc;Lnning scope SR200 nearly

one week before slaughter. The animals were then slaughtered and graded. The tissues at the

7th vertebra rïb was chemically analyzed for fat composition. A high correlation coefficient

(r=0.90) was obtained between actual fat percentage in the M. longissimus muscle and the

colow-scanning scope ( the SR200) estimates. The color-scanning scope displayed the fat as

weak blue dots in the echo signal. Also estimates of the subcutaneous fat thickness and the

cross-sectional area of M. longissimus muscle from the scans were in good agreement with the

actual carcass measurements (r-0.69 and r=0.81 respectively) . From ultrasonic A-mode signals, Park et al. [26] studied the effect of fat concentration

within cubes of meat sampIes on the speed of sound. The authors noticed that the ultrasonic

wave speed decreases with increasing fat concentration. The correlation coefficient between

sound speed and fat concentration was -0.82. Another correlation coefficient that related the

visual marbling score and the fat concentration was also calcdated and found to be 0.7. Fi-

nally, a nonlinear model was built to predict the intramuscular fat concentration £rom speed

of sound with varïed accuracy. For large fat contents (> 8% fat ) the accuracy was 90%, and

for low fat contents ( <8% fat) the accuracy was 76.4%. The authors concluded that the cor-

relation coefficient between fat concentration and ultrasonic longitudinal speed (r=0.82) was

significantly higher than the correlation between the visuai marbling score and the longitudinal

speed (r=O.ï2). In conclusion, the results of the study were satisfactory given the fact that the

marbling scores are very difficult to estimate because they are an outcome of many factors such

as fat contents, animal age, meat coIor and texture.

Another experiment by Park et al. 1271 was made as a continuation of the previous study.

This experiment transformed the time domain signal to frequency signal which had then been

used to predict the intramuscular fat contents of beef tissues. The most signifxcant parameter

in the fkequency domain was the nuniber of local maxima that represents the discontinuity of

the Fourier spectrum caused by inhomogeneous fat concentrations in the longissimus muscle.

A multivariate regression model was developed to predict fat concentration with an accuracy

of 79% for fat concentration below 4%.

To evaluate the accuracy of dtrasound measurement of fat thickness and the area of Longis-

simus muscle, two different sets of beeh consisted of 495 steers and 151 heifers were isonified

with an Aloka 210DX ultrasound unit equipped with a 12.5 cm, 3.0MHz linear m a y transducer

[28]. The measurements based on ultrasound and the actual measurement taken 48 hour post-

mortem were compared. The difference between ultrasonic and actual carcass measurements

indicated error rates of 20.6% for backfat and 9.4% for longissirnus muscle area. Measurements

based on ultrasound more accurately predicted backfat thickness and longissimus muscle in

thinner and more lightly muscled cattle than fatty ones. The ultrasound machine setup para-

meters such as the near and the far gain, and transducer placement had pronounced effects on

the measurements . More than five hundred beef carcasses were used to test the ability of H e ~ e s s y Grading

Probe (HGP) to give better estimation of carcass lean measurements than the subjective visual

estimation or linear carcass measurements [29] . The combined visual score for overd fat cover

and muscle thickness was more precise than probe measurements for predicting iean yield in

warm and cold carcass. The generd equation that gave a better estimation was:

Perecentage of lean meut = 54.45 + 0- 16 * (1 ain eye area) - 0 -74 * (average f a t )

The above equation was adopted in Canada for prediction of lean percentage based in

measurements taken from carcass.

Another study in [5] was conducted to determine the repeatability and accuracy of rneasure-

ments taken by ultrasound of live animals to predict lean yield fat depth and the longissimus

dorsi area. Measurernents for more than six hundred yearling b d s was taken before slaugh-

ter. A statistical multiple regression mode1 was used to predict the yield based on ultrasound

rneasurements. The predicted yield and the actual measurements, t hat were obtained t hrough

Agriculture Canada Blue Tag Program, had been compared. The results showed that pre-

diction based on dtrasound measurements was as accurate as the yield estimated by direct

measurements ( r=0.81; P<O.O5).

In other two experiments in [3O], more than four hundred yearling were measured for sub-

cutaneous fat thickness and longissimus muscle area between 12'~ and 13~" ribs using real time

linear array uitrasound equipment. Ultrasonic rneasurements were not accurate for fatty and

heavier animals. Also the measurements of fat thickness were more accurate than estimating

the muscle area.

1.4.4 Texture Extraction and Pattern Recognition Techniques

Some researchers selected to extract texture features from ultrasound images using conventional

techniques, such as Haralick's Ceoccurrence matrix. Other researchers designed their own

texture features. The extracted features are used to clasdy ultrasound images using pattern

recognition classifiers or used as an input to algorithms that segment ultrsound images into

different regions.

A project by Muzzolini et al. [31] aimed to h d a set of features that can be used to

segment reai ultrasound images of an ovary. A set of 21 spatial domain and 11 frequency

domain features were designed and tested for their effectiveness to segment ultrasound images.

A new Multiresolution Texture Segmentation algorithm (MTS) was derived and used to test

the extracted features. The experiment al results suggested t bat the MTS algorit hm combined

with the use of robust statistics were very useM to select the best features that can be used for

segmentation. Also texture in ultrasound images can be identified regardless of its resolution

and orientation.

A set of 71 live beef animals and 88 slaughtered beef carcass had been used in another study

by Muzzolini et al. [32]. He developed an algorithm that partition an ultrasound image into

different areas based on texture. The dgorithm generalized the simulated annealing method

to a multiresolution fkamework. The algorithm treated each image as a tree. The root of the

tree was the image itself. Each level of the tree represented the image with a lower resolution.

For each block of pixels at a certain resolution, the texture was tested. If the texture was

not homogeneous, it was divided further into four smaller blocks. The process of dividing the

blocks continued until a homogeneous block of texture was found. The algorithm then merged

the neighbors blocks of similar texture. To compensate for the effect of selecting a fked block

size, the algorithm extracted texture using different block sizes from 4x4 up to 64x64 pixels.

The algorithm achieved 98.84% accuracy rate. The dgorithm suEered h m two drawbacks.

First it used large set of control parameters for the simulated annealing algorithm that must be

cdculated experiment d y - Second, the init i d level at which the algorithm st arted segment ing

the images effected the final results.

Whittaker in [33] conducted an experiment using 57 slaughtered test animais and a set

of 35 live test animals. The experimental process was categorized into the following phases:

Prescanning, scanning, acquisition, preprocessing, processing, quality grading and data analy-

sis. The acquired images were enhanced using the sequential averaging to remove speckle

noise- The sequential averaging is the simplest form of temporal filtering where multiple noisy

images scanned at the same area are averaged to produce single image that has lower noise

[34, 35, 17, 36, 371. A feature extraction techniques were employed to compute the marbling

score hom texture present in ultrasouid images included Fourier transform, hactd dimension,

attenuation methods. The marbling scores of the animal carcasses were assigned by a panel of

three trained expert graders based on the USDA quality grading standards. In the last phase

of the study, the correlations between the test data and the marbling scores were determined.

Most of the extracted features of dtrasonic images from live beef animds correlated better to

visual marbling scores than the ultrasonic images £rom slaughtered beef animals. Also the pre-

diction models developed £rom the live animals scans proved to be better predictors of marbling

scores (R2 = 0.66 for nonenhanced images, R~ = 0.46 for enhanced images) than the models

developed £rom the slaughtered animal data (R* = 0.45 for nonenhanced images, R~ = 0.23 for

enhanced images). The outcome favoring the live animal prediction models may be a result of

capillary blood loss and tissue change due to lack of oxygen during slaughter. Also nonenhanced

images produced bet t er resdts t han enhanced images because speckle noise ca ry important - information related to the marbling score. Enhancing the images by removing speckle noise

degraded the results.

R. Brown et al. [4] conducted a research at the University of Guelph to evaluate ultra-

sound and photographie image aoalysis for determination of the marbling fat content and other

grading quality factors in Iive animais and beef carcasses. An image andysis software package

EASI/PACE was used. The image database comprised ultrasound images taken at the 12th/13th

rib location on each animal just prior to slaughter, a color video image of a section made at

the 1 3 ' ~ rib, and a chernical fat analysis for the ribeye. The officiai grade designation for each

animal was dso recorded at the tirne of slaughter. Ultrasound images were filtered using Fourier

transforrn filtering technique and enhanced to improve their contrat. Two different classifiers

were used from the EASI/PACE package to classi@ a selected areas kom video images and

ultrasound images, a supervised classifier, namely, the Maximum Likelihood Classifier (MLC)

and unsupervised classifier (ISOCLUS). The MLC classinle used texture features extracted

fiom ultrasound images using HaraIick's technique. The ISOCLUS used the graylevel values of

ultrasound image pixels. The ISOCLUS classifier output images that have two graylevels, one

representing fat and the other meat tissue. The percent of marbling fat was computed as the

number of fat pixels divided by the total number of pixels in a selected area from each image.

The correlation coefficient between video images analysis results and chemical fat measurements

was 0.84. Ultrasound images produced a satisfactory results to estimate marbling fat abun-

dance in the LD muscle but they were less accurate than analysis of photographic images. The

standard error of estirnate for predicting chemical fat fiom ultrasound images was between 1

and 1.5%. The unsupervised classifier (Le., ISOCLUS, a modified version of k-means classifier)

produced a better results then the s u p e ~ s e d classifier (Le., MLC).

McCauley et al. (381 used B-ultrasound images to estimate fat contents. A set of cross-

sectional and longitudinal ultrasound images were taken from 71 live and 88 slaughtered beef

animds. Rom each image, an Area Of Interest (AOI) of size 80 x 80 was selected within the

ribeye area for the analysis. The thirteen texture features as described by Haralick e t al.

[39] were extracted hom the A01 of each image. The images were divided into an equal size

training set and testing set. The training set fed to an Adaptive Logic Neural Network (ALN).

The chemical fat analysis results were used as endpoint measures to &date the capability

of the ALN. The mean error for the ALN was between 0.83 and 0. 94% and the classification

accuracy to major grade divisions ranged from 73% to 79% which is better than the multivariate

statistical regression analysis in [2]. The ALK over-predicted low percent fat samples (<3%)

and under-predicted high percent fat samples (>7%). The authors felt that this may be due

to the fact that the majority of the training saaples fell in the mid-range of percent fat which

might be the normal case. A larger, more diverse training sample set might give better result.

1.4.5 Commercial Systems for Marbling and Fat Content Assessrnent

Today, there are several commercially available ultrasound systems that claim to predict the

marbling grade of a live animal. Such systems use a conventional ultrasound imaging device

accompanied with cornputer software to assess meat quality. A study by MacNeil et al?

'lit ig not publàshed, however it is availabk in the internet at: http://ugadsro. msu.montana. edu/extension/Beef/Gov. Conf

compared four ultrasound systems that estimate rnarbling by applying image analysis proce-

dures to a region of interest within the ribeye muscle. The systerns represented in the study

were: Animal Ultrasound Services Inc. Ithaca, N Y ; CPEC, Oakley, KS; Critical vision Inc.

Atlanta, GA; and Classic Ultrasound Equipment , Tequesta, FL. The CPEC systems predicted

marbling score directly. The other systems predicted percent fat within the muscle which was

converted to marbling score. A set of 80 cattle were used in the study. The marbling scores were

deterrnined twice, first using each ultrasound machine for the live animals and then by an expe-

rienced grader after slaughter. The correlation between ultrasound predictions and the human

grader evaluation were between (-0.04) to (0.54). The best machine was the CPEC because it

predicted the marbling score directly. The other machine predicted fat content which converted

to marbling score. But since the correlation between rnarbling scores (by human grader) and

percent fat was 0.74 the results of these machines was lower than the CPEC machine.

1.4.6 Classifiers Fusion

The experirnental work of this thesis has two parts. In the first part, Laws' and Haralick's

texture feature extraction techniques are used to quantiS. the texture of ultrasound images.

The extracted texture features are used to design a classifier, namely, the MED classifier or

the k-means classifier. The individual classifiers are optimized with respect to the available

feature data. Since the data set (ultrasound images of beef animals, their fat chernical analysis

and marbling grades) that is available to c a r y out the experiments is small, the accuracy of

individual classifiers can only be improved to certain point. Hence, another strategy to improve

the result is investigated.

In the second part of the thesis, two classifiers are combined to improve the success rate of

the classification. Two basic approaches are commonly used for combining multiple classifiers:

the dynamic classifier selection and the classifiers fusion [40]. In dynamic classifier selection,

an attempt is made to predict which single classifier gives the best result for a given problern.

The output of the successful classifier is considered in the h a l decision. In classifier fusion,

individual classifiers are applied in pardel and their outputs are combined in some manner to

achieve a group consensus. DifFerent approaches for classifier fusion have been appIied in other

studies such as : the Classifiers Voting approach [4l], the Averaged-Bayesian [42], the Behavior-

Knowledge Space (BKS) approach [43]. The General Bayesian F'usion approach (GBF) , which

is derived from the Bayesian technique in statisticd decision theory, is used in this thesis to fuse

two classifiers. The GBF approach is developed specifically for this research which is explained

in details in Chapter 6. No research was found in the area of meat grading used the classifier

fusion approach.

Using a heuristical voting approach, Kimura and Shridhar in (441 combined two algorithms

for the recognition of unconstrained isolated handwrit ten numerals. The fht algorit hm used

a modified quadratic discriminant function utilizing sensitive spatial features. The second

algonthm was a structural one that utilized features derived from the alphabet character le$

and right profiles. Each algorithm gave very low recognition rate. By combining both algorithms

the h a 1 result was improved si@cantly. Low error rates (0.2% or less) with rejection rates

below 4% was realized. Different combinations strategies to achieve such a result were tried.

For example, the algorithms were treated as they work in a parallel fashion or in a serial fashion.

Parallel combinations algorithms usually achieved better results than serial algorithms.

Kam e t al. in [45] used multiple classifiers to recognize handwritten digits and degraded

multifont machine-printed characters and words from large lexicons. In a word recognition

expriment, they used four classifiers to discriminate between 1365 classes and they got an

improvement of 7.8% at the top of the best individual classifier results. The combination

was made on a r d lists that was produced by each classser. The authors concluded that

combination methods based on ranking were more suitable and gave better results than other

methods that were based on distance measure or posterior probabilities estimations. Xn the

ranking approach each classifier arranged the unknown input patterns into a r d n g Est such

that the class candidates at the top of the list. The combinations of the ranking lists was

done in two difEerent ways: class set reduction and class set reordering approaches. In class

set reduction approach, the objective was to extract the smallest subset that contained the

true class from the ranking lists. In class set reordering, the objective was to derive consensus

ranking of the given classes, such that the true class is ranked as close to the top as possible.

The two approach were different because they can be achieved by different means depends on

the type of classifier and features used. Two approaches for set reduction and three methods

for class set reordering had been considered.

Huang e t al. 1431, used the Behavior-Knowledge Space method (BKS) to combine the

decision of three classifiers- A data sarnple consisted of 46,451 nunierals collected hom 1000

people was used. They dividecl the data set into 5,075 samples for training and 41,377 samples

for testing the performance of the BKS approach. The three classifiers performances before

applying the BKS was 90.37%, 90.93% and 92.14%. Applying the BKS method raised the

result to 95.31% with lower rejection rates and higher reliability. They cornpared this approach

to Voting approach and Bayesian approach. Voting approach gave lower resdts (93.92%). The

Bayesian approach achieved comparable results as high as 95.31%. The authors concluded that

fusion of multiple classifiers by any one of the above mentioned techniques always produced

higher results then that produced by individual classifiers. For practical applications, the key

issue to apply the BKS approach successMIy (and other fusion techniques as well) is to constnict

a representative training data set. The types of the classifiers that were combined by the BKS

method were not mentioned.

1.5 The Thesis Objective

To grade beef, the decision shodd be made according to certain properties such as animal age,

meat tenderness, meat color, meat texture, external fat cover, and intramuscular fat distrib-

ution. Properties, such as age and animal size, are easily identified while otbers need to be

inferred after slaughtering the animal. Far example, the abundance and distribution of mar-

bling fat are estimated by skilled human graders by Iooking at cross-sectional area of the ribeye

or the longissimus dorsi (LD) muscle a t the interface between the 12'~ and the 1 3 ' ~ ribs bories.

Stuclies showed that grades that are assessd by humans are subject to an error as high as 20%

[l] . An objective technique is needed that gives more accurate quantitative rneasures that can

be deterrnined without slaughtering the animal.

The main objective of this thesis is to use B-mode ultrasound images of cross-sectional

area of the ribeye at the interface between 1 2 ~ ~ and 1 3 ~ ~ ribs for the assessrnent of marbling

and fat contents of lîve beef cattle. A set of 59 ultrasound images are used for developing

grading algorithms. This set was assembled from a pool of approximately 450 animais. The

beef cattle were hom projects sponsored by OMAFRA-University of Guelph contract which

was used in a previous study [4]. A fat chernical analysis of the percentage of fat in the ribeye

area and the marbling grades for each animal were recorded. From each ultrasound image,

a set of characterizing features will be extracted, namely the Haralick's Co-occurrence and

Lam' rnasks features. Each ultrasound image is assumed to have distinctive features that

depend mainly on the animal grade or fat contents. Supervisecl and u n s u p e ~ s e d pattern

recognition cIassifiers will be designed to iden te the animal grade based on such features.

To improve the algorithms grading performance, the extracted features will be transformed

using a whitening transformation process. To optimize the grading results and to reduce the

uncertainty associated with the output of individual classifier, the results of two classifiers will

be combined using Bayes' theorem.

1.6 Organization of the Thesis

Chapter one, provides an overview of the meat grading problem. A brief introduction to the

Canadian beef grading program is given. In this chapter we point out to the need for an

objective procedure for animal grading as opposed to the subjective techniques currently been

in use. We also present a review of relevant research in the animal grading problem, and

introduce the thesis objective.

Chapter two, Iays out the theoretical foundationç. It explains the concept of texture in

ultrasound images and its role for identifyïng animal grade. This chapter dso discusses few

topics related to statistical pattern recognition systems.

Chapter three, discusses probabilistic aspects of the texture present in ultrasound images.

Each pixel in ultrasound images is assumed to be a random variable. A second joint probability

function, ( known as the Co-occurrence m a t h ), among image pixels having the same graylevel

is computed. From the Co-occurrence matrix, some texture characterizing features, specificdly)

the Haralick's C-occurrence features, are computed and used to identiQ meat gade5. Two

Qifferent grading algo~thms based on such features, namely, supervised and u n s u p e ~ s e d grad-

ing algorithms, are designed and tested. To improve the grading results, the Co-occurrence

features are transformed using the whitening transformation to make them uncorrelated and

5 fat percentage or marbiing grade.

to normalize their scaie.

Chapter four, considers the texture present in ultrasound images to be a deterministic by

assuming that the neighborhood pixels fonn a macro texture elements such as lines, spots,

waves, and edges. To characterize deterministic texture in an image, we first isolate its macro

texture elements features and then compute the density of such features. Using Laws' masks,

the texture is extracted from ultrasound images and two algorithms similar to the algorithms

designed in Chapter three are developed and tested. Again the Laws' texture features are

whitened and used to develop a grading aigoriths.

Chapter five uses Bayes' theorem to improve the classification results. In Chapter 3 and

Chapter 4, individual classifiers and a single feature set were used to determine the true class

of a given pattern. It is well known that different classifiers and different texture features

complement one another in classification performance [42]. This has led to a belief that by

using features and classifiers of different types simultaneously, the classification accuracy could

be improved and the uncertainty associated with the results of a single classifier will be reduced.

Each classifier is tread as an expert that bas a certain decision. The Bayes' theorem combines

the results of two classifiers to make a k a 1 decision.

Chapter six siimmarize the experimental results, discusses the results and gives a recom-

mendation for further work in the problem of meat grading.

The thesis is supplemented with three appendices. Appendix A defines the technical terms

used through the thesis. If the reader gets confused by a technical term while reading the thesis,

a look at this appendix is highly recommended. Appendix B ta.lks about Mage filtering and

specl.de noise reduction. It is part of the experimental work but since it does not fit very weU in

any chapter it is moved to the appendices. Finally, Appendix C explains a nonlinear mapping

algorithm (the Sammon's mapping algorithrn) which is used to map a high dimensional feature

space into lower dimensional space. The texture features as extracted by the Co-occurrence

matrix has 14-dimensions. The Sammon's rnapping algorithm maps the feature vectors to 2-

dimensions space so it can be visualized. In the this thesis, the SaInmon's algorithm is used to

study the effect of enhancing and filtering ultrasound images on the classification results and

to compute the optimum parameters that are used to extract Haralick and Laws features.

Chapter 2

Texture Analysis and Pattern

Recognition Syst ems

2.1 Meat Grading As a Texture Analysis

Texture represents an important clue for h u m a vision perception to analyze and recognize

scenes in the surroundhg environment. The smdes t distinguishable element in a digital image

called a pixel. m e n a group of pixels create a repeatable pattern that occurs across the whole

image it forms what we perceive as texture. Hence, textures portray repetition of a macro

element called a texel, a word derived from tercture element . Artificial texture is deterministic

or periodic, whereas naturd texture is generally stochastic. Textures that are perceivable by

humans are normally described by terrns such as coarse, fine, srnooth, granulated, rippled,

regular, irregular, or homogenous [46]. In an image processing context, deterministic texture is

represented by a set of texels and a structural rule or gr;tmrnar that describes their placements.

On the other hand, random texture is described by a statistical parameters. Based on human

visual perception research, it has been suggested that human visual perception of random

texture fields may be unique ody up to the second-order densities [47]. That means a broad

spectrum of texture can be represent ed efficiently by second-order st atistics.

Intramuscular fat distribution in ultrasound images shows as a texture. The fibers of soft

tissues have smaller dimensions than the wavelength of an ultrasound pulse. The tissues cause

the ultrasound wave to scatter in different directions. Constructive and destructive interfaces

of sound waves £corn various scatterers produce a random graininess pattern (speckle noise) in

an ultrasound B-mode image. The speckle noise texture is afIected by media properties such as

the size, the density, the distribution of scatterers and the kequency of ultrasound. Hence fat

and different meat tissues give a distinctive texture pattern. Images of low intramuscula. fat

distribution have large grainy texture. In contrast, images of high intramuscular fat have fine

textured appearance [48]. In fact the notable feature that can differentiate between two images

of different marbling grades or fat contents is their texture aspect 138, 48, 491.

The first step to use texture properties to classi& images is to transfonn a given texture

into measurements or features that uniquely ident* it £rom other textures. Such measurernents

may or may not have any physical meaning. Texture analysis research focuses on three main

issues:

a Classification: given unknown texture, classiS. it to a predehned texture class,

0 Modeling: describe or mode1 a given texture by mathematical, statistical or grammatical

formula,

0 Segmentation: partition an image into different regions based on texture characteristics

The research in this thesis deals with the first issue. Given an ultrasound image, use texture

analysis to classify fat distribution and marbling grade in the ribeye muscle tissue.

Different techniques can be used to describe texture. If we consider texture as a random

pattern then a probabilistic or statistical approach should be used, which is the subject of

Chapter 3. If we assume the texture consists of smailer texture primitives, not necessarily

having a deterministic placement, such as Iines, points, spots and ripples, then another approach

that isolates such texture primitives and then uses them to quanti& the texture should be used.

The subject of Chapter 4 de& with such an approach.

After transfonning the texture of an ultrasound image to a set of values or features that

uniquely describe it, such features c m be used to identify the animal grade [24, 211. In this

research the parameters or values extracted kom ultrasound images texture are used as an

input to pattern recognition systems.

2.2 Pattern Recognition Systems for Meat Grading

In real life situations we, as humans, perceive, organize and interpret information as a form of

complex patterns. The processes which enable us to do so are still largely unknown. However,

psychologists presume perception occurs according to the scheme shown in Figure 2.1. First,

patterns, such as sounds or images, must be perceived by our sensing organs. The sensing

operation is a transformation of the patterns to certain measwements which can be a sort

of chernical reactions or electrical signals in the neural cell. These measurements are sent to

the brain for further processing. The same pattern or patterns in the same class must have

been previously perceived and stored in our memory. Fhdy, based on some correspondence

or equivalence, the brain compares the perceived pattern with the stored ones. The result of

this cornparison is a symbolic representation of that object [50, 511.

Unknown pattern

1

Memory "stored patterns"

Symbolic description !I or action ! r\ 1 Brain 1 n

"comparison and decision making"

Figure 2.1: The human preception process.

Since the advent of digital computers, scientists have tried hard to make computers act like

humans and since then pattern recognition systems, as a discipline, has emerged to simulate the

human perception experience.

The input to a typical pattern recognition systems is patterns and the output of the system

is the class label of each pattern. For example, suppose that someone is trying to distinguish

normal heart fiom abnormal heart by analysis of electrocardiogram (EKG) signal [52]. The

input patterns to the pattern recognition system is a EKG chart of the patient. The output of

the pattern recognition system is a label indicating whether the patient has normal or abnormal

hem.

Whenever implementing a pattern recognition system we normdy face two difFerent sce-

narios. Zn the hst one, a set of patterns exists and we know the label of each pattern, so a

supemked pattern recognition system cari be designed. Patterns in a supenrised system are

divided into training set and testing set. The training set is used to train the classification

algorithm and the testing set is used to evaluate the classification performance. In the second

scenario, the pattern labels are not known priori and the only way to classify such patterns is

by using unsupenuised pattern recognition techniques. An u n s u p e ~ s e d system uses the simiIar-

ity between input patterns to classify them into difFerent groups where each group represents

certain claçs. This step is referred to as clmf enng.

Stazting fiom the h t block in Figure 2.2, which shows a typical pattern recognition

system, a transducer is used to make measurements. For example, in our case a transducer

of an ultrasound system is used to take ultrasound images of a live beef cattle. Each image

constitute a pattern that c m be classified by the pattern recognition system. Images obtained

by the fist process, the transducer, f o m the pattern space. Each one of the ultrasound images

used in this research has a size of 618x480 pixels, "ie., picture elements", which means that

each image has 296,640 graylevel values. Each gra~level value refers to grade of shade between

black and white. Since each image contains huge amount of redundant data, we need to select

the relevant information or features that are necessary for classification. This feature extraction

process has two purposes. First it reduces the computation burden needed for classification by

reducing the amount of data used to describe the image. Second it simplifies the process of

designhg a proper classifier. A feature space normally h a . high dimension space, typically

between 10 and 100 dimensions. However, it is still much srnaller than the original pattern

space. If 10 features are extracted from an image, then it can be represented by a vector in 10-

dimensional feature space. Referring again t o Figure 2.2, the decision algori t h transforms the

feature space into the classification space. This classification space stiU has fewer dimensions

than the feature space. In this research each classification space has 3 dimensions (the fat

contents: Fat 1, Fat II, and Fat Zn) or (the marbling grade: grade A, grade AA, grade AAA)

which represent the result of the decision algorithm.

Pattern recognition problems are difficult to deal with for the foUowing reasons:

difEculties in defiriing significant features that distinguish a given pattern £rom another,

patterns are normally distorted by noise,

variability arnong patterns belong to the same chss or category. The different classes may

overlap or have large variances.

physical world pattern space feature space decision

Figure 2.2: Stages of a typical pattern recogntion system.

I Transducers or Feature extractor

sensors "Texture analysis"

2.3 Designing a Classifier

Classifier "Decision algorith"

There are two different approaches for designing a classser, supervised and u n s u p e ~ s e d learn-

h g approaches. In a supervised learning approach, the training samples are labeled by their

actual class. In this research, labels are the classes as assigned according to the results of a

chemical analysis of fat of the ribeye area or the marbling grades as assigned by human expert

graders. The labels are then used to guide , "Le. supervise", the classifier during the learning

process. In the unsupervised learning approach, we clrop the labels of the training samples

and let the classifier induce the labels by clustering the samples into different groups based

on a s i m i l a r i ~ measure. Both supervised and unsupe~sed approaches have been adopted in

this research. The foIlowing two sections discuss two s u p e ~ s e d classifiers and an unsupervised

classifier, namely, MED, MAP and k-means respectively.

2.3.1 Minimum Euclidean Distance Classifier (1MED)

This classifier is very simple to impiement because it is based on a simple intuitive idea. In the

training process, each image is represented by single point or vector in the feature space. Images

that are in the same class are close to each other in the feature space. So if we have three classes,

we get three different clusters in the feature space. Each cluster then can be represented by

single point that characterize the whole class, namely the prototype. The prototype represents

the typicd or ided fonn of a given class. NaturaIly the prototype (a) is the statistical mean

of all points belong to the same class (Nk), i.e. :

By applying Equation (2.1) we obtain dzerent prototypes Zi, &,Zs, -....Zn. Each Zi belongs to

single class.

If we are given an unlaiown pattern 5, and we are asked to determine its class. One simple

approach is to assign 5 to the closest prototype. The closeness could be a- measure, but in a

Euclidean space, the distance measure must satis@ the folIowing conditions:

; the distance should be zero or positive number only

d(x, y) + d(y, z ) 3 d(x, z ) ; the triangular inequality

Cornmonly functions t hat sat i sh the above conditions are:

Maximum value distance: dnr (x, y) = ma% (xi - yj (

n Absolute value or "city block" distance: da (x, y) = 1x.i - Yj 1

i=l

n 1/2 Euclidean distance: dE (z, y) = [x (zi - 2]

i=l

So if we select Euclidean distance as a measure of similarity between iirzknown sample Z

and the prototypes of the classes Zl,&,&, . . . , & then the distance between the pattern Z and

(the protome of class Ck), is given by,

where xi is the ith component of vector 2. zk is the ithcomponent of the prototype %.

The decision rule based on Euclidean distance can be formulated as: assign the iinknown

sample to the nearest prototype. This can be written mathematically as:

Le., assign 3 to class Ck if the distance between 5 and the prototype of 4 is less than the

distance between Z and any other classes prototypes. For the case of three classes we use the

following formula:

and we assign a to the class which has the minimum distance.

MED is a supeMsed learning classifier because it requires the labels of the pattern samples

to create the classes prototypes. It is very easy to implement, but i t requires that the classes

have normal distributions.

2.3.2 Maximum A Posterior Probability Classifier (MAP)

In some cases MED classifier does not work very well because Euclidean distance is not the

correct metric that describes the closeness betnreen patterns of the same class. Also variability

between patterns rom the same class and noise added during the measurement process create

additional difficdties. The natural approach is to treat patterns as random vectors that can be

assigned to hown classes which are represented by their prior conditiond probability density

functions P(x/wi) . For notational simplicity we drop the dash £rom the vector 5 (Le., we write

x). Hence, the decision d e becomes: assign an unknown vector x to class wi that has the

largest probability value. Mathematically this can be written as:

where P(wi/x) and P ( w ~ / x ) are the posterior probabilities of classes wi and wj given the

observed pattern x. The d e in Equation (2.2) is optimum in the sense that it minimizes

the probability of misclassification. Since class density functions are normdy given in terms of

priori conditional probability functions, we have to &ange Equation (2.2) to use such functions.

Using Bayes' theorem:

equation (2.2) c m be changed to:

where P ( w i ) and P(wj) are the priori class probabilities.

2.3.3 k-means Clustering

Whenever the training samples are given without labels or the labels that we use for classification

are not accurate, another approach should be used for classification. UnsupeMsed approach

attempts to find structures within the training data by using certain similarity measures. So,

for two patterns that are coming hom the same class, we expect them to be more similar than

other patterns that are corning kom different classes. The unsupervised leaming algorithm that

we implement in this research is the kmeans algorithm. It is an iterative algorithm that starts

by arbitrary assignment of initial class prototypes. AU the data samples are classified using the

initial class prototypes. The prototypes are recomputed and the same samples are reclassified

to the new prototypes. The same procedure is repeated until the prototypes do not change (521.

The k-means approach even applicable for a data that has labels. For example, we are given a

labeled set that have multimodal densities and seek to determine regions in the feature space

where unimodal densities are applicable. Which in other words means, we try to approximate

mult imodd densities by unirnodal one.

Chapter 3

Probabilist ic Approach for Meat

Grading

3.1 Introduction

In the statistical representation of images, each pixel is considered as a random variable. The

entire image represents a discrete random field. Looking at individual pixels, the first-order

distribution function can be formed to describe the distribution of graylevel d u e s (tones)

across the image. More complex model, the nth-order joint probability distribution function,

c m be built by finding the relations between two or more pixels. The most commonly used

model is based on a relationship between two adjacent pixels in the image.

Marbhg in ultrasound images appears as a random texture. The statistical properties of

such texture depend on the marbling grade of the animal that has been imaged. Marbling and

fat contents are correlated, hence texture aIso reflects the fat contents. The computation of the

statistical properties of a given texture form what is known as the feature extraction process,

an important step to implement pattern recognition system.

3.1.1 Texture Features Based on First-Order Probability F'unction

To compute the £kt-order probability distribution function of a digital image, a histogram

of the graylevels in the image is constructed. AU pixels in the image having the graylevel gi

are counted and the corresponding bin in the histogram is updated. A digital image and its

histogram are shown in Figure 3.1 and Figure 3.2. Let u be a random variable representing a

graylevel in a digital image that has L graylevels. The probability distribution of any graylevel

x is given by the formula:

number of pixels with grayleuel x &(x) = Prob[u < x] =

total number of pixels in the image

From Equation (3.1), a nwnber of general features c m be computed,

Moments: L-1

mi = ~ [ u ' ] = ~ z i P , ( z ) , i = 1,2, ... etc. 1-0

Absolute moments:

Central moments:

Absolute central moments:

Entropy:

Specific features such mean = mi, variance = p,, average energy = rn2 , skewness = p3 and

kurtosis = p4 - 3 , can be cornputed £rom the general equations above. These features are

known as histogram features. For an image, these features can be computed globally for the

whole image or locally over s m d region. An interpretation of the significance of these features

is given in [46]. For example, the standard deviation feature (fi) emphasizes the strong edges

in the image, the dispersion feature (Fi) quantifies the fine edge structure, and the meam (mi)

Figure 3.1: Digital image has a resolution of 512 x 512.

Gray level value

Figure 3.2: Histogram of the image in Figure 3-1.

extracts Iow spatial-kequency image cornponents. Such features are not enough to characterize

wide range of textures because they do not capture the relation between adjacent pixels. In

textured images, whether it is d e t e d i s t i c or stochastic, the adjacent pixels are correlated.

The extracted texture properties should be cornputed based on higher order probabilities.

3.1.2 Second-Order Joint Probability Texture Features

Many types of textures can be distinguished by cornparing their features using second-order

probability distribution functions. There are two commonly used approaches that exploit this

fact. In the first approach, texture is modeled by two-dimensional random field model, the

coefficient of which is used directly to identify the texture [53, 541. In other approach, second-

order joint probability function is used to compute texture features.

The second-order joint probability function of the distribution of graylevek of digital image

is given by,

Pu(z1,xz) -= Pu, ,u2(~l ,x2)= Prob[w 5 x i , u 2 5x21 = number of two pixels ui, u2 having graylevels XI, x2

total number of pixels

For digital image, each pixel has eight neighbors as shown in Table 3.1, hence, Equation (3.2)

has more general formula because it can be computed for pixels having different distances and

oriented at different angles which could be one of these angles, 0°,4S',900, l3s0, 180°,Z50 ,270°

or 315O. EQuation (3.2) should be written as,

Pu(xli~2) = f ( d , 0 ; ~ 1 , ~ 2 ) = (3-3) number of pixels ul, u2 at distance d and angle 8 having graylevels xi, x2

total number of pixels

Pu (xl, x2) can be written as n x n matrix, where n is nurnber of grayleveIs in the image.

This matrix, known as the Co-occurence m a t n i and written as P(i. j ) , was first used by P.

Juleçz [55] in visual hurnan texture discrimination experiments. The Co-occurrence matrix can

be computed for the whole digital image or small region of it. To conipute the Co-occurrence

Table 3.1: The eight neighbors and angles of a pixel (*) .

matrix, we count the number of pair pixels having graylevels i and j , respectively, and which

are in a fixed spatial relationship, such as a k e d distance apart or fked distance and Gxed

angle. To mahe the Ceoccurrence Iike probability rnatrix, the Co-occurrence matrix c m be

normalized by dividing each entry by the sum of all of the entries in the matrix [56].

Example:

Table 3.2 represents the graylevels of a hypothetical digital image. The Co-occurrence matrices

computed for different angles and different distances are shown in Table 3.3.

Table 3.2: 7x7 image has four graylevels 0,1,2,3.

a) 0 = OO,d= 1 b) O=27O0,d=2 c ) 8 = 4 5 O , d = l

Table 3.3: The Co-occurrence matrices of the above image for different angles and distances.

3.1.3 Texture Features Based on the Co-occurrence Matrix

By applying Equation (3.3) to digital image we get P ( i , j ) . The Co-occurrence matrix c m be

thought of as an estimate of the joint PDF of graylevel pairs in an image1. According to the

axioms of the theory of probability huictions, the sum of P( i , j ) should be one, Le.,

Haralick used the Co-occurrence matrix to extract texture features as a measure of texture

in satellite images [39, 571. The method he used involves applying weighting function to each

element of P(i, j ) and surnmnig these weighted element values. Varying the weighting function

m i e s the type of texture informatisn extracted £rom the image. Haralick defined the following

14 features,

1- Energy (Angular Second Moment) :

2- Contrast:

3- Correlation:

where M , y, cz and u, are the standard deviation of Pz and Py and are defined as follows:

J

u = (2 - ) Pz (i) and a

As a cornparsion to the matrix given in section 3.1.1, it can also be thought as a two-dimensional histogram.

33

J

4 Variance (sum of squares):

5- Inverse Difference Moment (Homogeneity) :

6- Surn Average:

N9 *, where P=+,(k) = C C P(i,j), and k = i f j for k = 2,3 ,..., 2Ng.

i=l i=l

7- Sum Variance: 2h'g

where f8 is defined as,

8- Sum Entropy:

9- Entropy:

fs = - C C P(i, j ) log P(i, j )

10- Difference Variance:

f i 0 = Variance of

where Pi-j (k) = ~2~ ~2~ P(i, j) , and k = [i - j 1 for k = O, 1, ... Ng - 1

11- Difference Entropy:

12- Information Measure of Correlation:

H X Y - H X Y l fl2 = max {HX, W )

and

fi3 = (1 - exp[-2.O(HXY2 - H m ) ] )'/*

where HX, W are entropies of P,(i), PY(j) and

HXY = -CC P(i, j) log P(i, j ) , 3

H X Y l = - C c P(i, j) log{Pz(i)Py(j)) and

i j

13- Maximal Correlation Coefficient:

f14 = (Second largest eigenvalue of Q) ' /~

where

For this thesis we defked fia as the counterpart of feature h, as follows,

Feature f4 is &O redefined. It is divided into two features to quanti& variances in x and y

directions as follows:

and

The foiiowinp C~-occurrence featllr- ( fi, f2, f3, fk, f4w f5, f6, f77 f8, f9, f i o . f i l . f i21 fis)

are used to capture the texture properties of ultrasound images. Feature f14 as defined by War-

alick is not used because it needs the computation of the eigenvalues of matrix Q which is very

computationally intensive process. Features f3 > f i 2 r f 13 , and f i4 carry texutre information

related to correlation. Hence, feature fie does not add a significant discrimination power to the

feature set.

3.2 Designing Classification Algorit hms to Predict Fat Contents

3.2.1 Data Description

Fifty-nine images were used to design different classification algorithms. These images were

taken between the 1 2 ' ~ and 1 3 ~ ~ ribeye area of live beef animal just before they weie slaughtered.

These images were used in a previous study by Brown et a1.[4] to investigate the possibility

of using ultrasound and vidso images to predict fat contents of live beefs before and after

slaughter. The images were captured using Aloka SSD-500 ultrasound imaging system equipped

with 3.5MHz linear array transducer that has 172mm depth of view. The machine was capable

of producing images of 64 gray levels. All images were recorded on 8-mm NTSC standard

videotape, captured using video grabber and stored as TIF graphic file format £ile with a

resolution of 618x480. Fat content of each image was determined by chemical analysis of the

ribeye tissue after the animal was slaughtered. The analytical results are shown in Table 3.4.

The first column shows the image file name. The second column is the name of data sample as

used in the experiments. The third column is the percentage of fat in the ribeye area muscle on

the chemical andysis of meat samples after slaughtering the beef. According to the chernical

analysis results, the images were divided into three equal size groups, namely, lov~ fat (1.99%

- 4.18%), medium fat (4.24% - 5.72%) and high fat (5.83% - 9.32%). These images can be

divided into different number of groups, but we divided them into three groups so they can be,

1 Fat 1 1 Fat II II Fat III 1 II II

image. ( sample 1 fat II image 1 sampie ( fat (1 image ( sample 1 fat I

Table 3.4: The data samples divided into three groups based on fat chernical andysis.

id 6874 6925 6837 6901 6924 6843 6953 6961

if needed, mapped into marbling grades (see Section 3.6). Also the same algorithms used to

estimate fat content c m be used to assess the marbiing grade. The three groups were referred

to as: Fat 1, Fat II and Fat III, as shown in Table 3.4.

3.2.2 Preprocessing

name FI0 FI1 FI2 F13 F14 F15 F16 F17

Fitst, the areas which contained image identification and ultrasound machine settings were cut

fiom al1 images. A histogram equalization was performed to adjust the contrast of all images

to similar level. Histogram equalization non-linearly redistributed the g a y level of images such

that all the 64gray levels were utilized evenly. Figures 3.3 and 3.4 show a video image and

its corresponding ultrasound image after a histogram equalization. Some anatomical structures

within the ultrasound image can be identified such as: the backfat ( lmown as the subcutaneous

fat) and the connective tissues. Others such as the rib bones and the lower boundaq of the

percentage 1.99 3.07 3.205 3.25 3.28 3.33 3.42 3.47

id. 6844 6840 6945 6908 6893 6906 6849 6902

name F20 F21 F22 F23 F24 F25 F26 F27

percentage 4.24 4.43 4.458 4.53

4.532

id. 6921 6868 6861 6846 6848

4.65 4.719 4.72

name F30 F31 F32 F33 F34

6883 6831 6960

percent age 5.83 6.1

6.188 6.29 6.42

F35 F36 F37

6.51 6.55 6.574

nbeye area which resides in the lower part of the image c m not be seen at all. Ultrasound

images are difEcult to perceive and interpret compared to other images such as X-ray images

and NMR images. Part of this difficulty is due to the presence of speckle noise. Removal of

speckle noise may irnprove the visual quality of images but it also smooths the texture and hence

reduces the information content of the image. A critical investigation showed that speckle noise

was the main contributor for forming texture in ultrasound image. The different tissues such as

fat and meat and fibers have different macro-structures and hence they reflect ultrasound wave

differently and appear as areas of different graylevel and texture . This observation suggests

applying texture analysis combined with pattern recognition techniques for classification.

Figure 3.3: Video image of a ribeye area.

3.2.3 Image Filtering and Noise Removal

DifEerent filtering techniques were tried such as, the Spatial Averaging of Speckle and the Wauelef

Denoising Algorithm, to reduce speckle noise in ultrasound images. The detailed exphnation of

these techniques is in Appendix B. Filtering of ultrasound images increased the computational

burden and degraded the classification result. This was c o h e d by a set of experiments. In

Figure 3.4: An ultrasound Mage of the same ribeye area shcwn in Figure 3-3.

one experiment, the images were atered by the Spatial avemgÊng of speckle to remove speckle

noise. Texture features using the Ceoccurrence matrix were extracted £rom the filtered images.

Each image was represented by 14-dimensional feature vector. The vectors of dl ultrasound

images constitute what is known as the feature space. Samrnon's algorithm, which is explained

in Appendix C, was used to map the 14dimensional feature space into Zdimensional space

so that the feature space can be visualized. The separation between classes bef'ore and after

filtering ultrasound images was observed. Since ultrasound images were fiom three different

classes, we expect to see three different clusters represent the three classes. The more far away

the classes fiom each other are, the more easier was to classiSr images and consequently easier

to design the classification algorithm. Removing speckle noise form uitrasound images reduced

the separation between classes. Moreover, speckle noise is signal dependent and any attempt to

rernove it degraded the quality of the signal. This result confirmed the conclusion that was made

by Whittaker in [33] and also in Liu [21j who found strong correlation (r = 0.822) between

speckle noise and marbling score. Hence speckle noise reduction was not pursued further.

3.2.4 Identifying the Area Of Interest (AOI)

The objective of the classification a l g o r i t h was to classiS. ultrasound images in terms of three

difFerent classes based on intramuscular fat contents or marbling grades. An area of interest

(AOI) should be selected that did not contain other fat types such as the subcutaneous fat.

To find that area, the procedure below was ernployed:

a- first, an area was selected from each ultrasound image,

b- the 14 Haralick's features, known as the feature vectar, were computed for the A01 of

each ultrasound image (see section 3.3.1). The vectors of all ultrasound images form a matrix,

the Co-occumnce Texture Mat* CTM that had a size of 59x14, where each row represents a

vector of a single image,

c- Since we can not visualize spaces higher than three dimensions, an algorithm, that is

commonly used to map high dimensional space to a lower dimensional space2, was used. The

algorithm (Snmmon's dgorithm) is very useful to gain more insight about the distribution of

the data by giving us a mean of visualizing and examining high dimensional spaces (n > 3).

It is explained in more details in Appendix A. It was applied to the CTM to map the 14-

dimensional space into tw*dirnensional space. We refer to the CTM after transformation the

"Reduced CTM" or simply R-CTM, and

d- The R-CTM was displayed as a graph. The three classes showed as three different

clusters. Different ultrasound images of different fat content or marbling grades occupied dif-

ferent regions in the reduced space which supposedly refiected their separation in the original

14dimensional space.

Steps a-d were repeated for different image areas. The area that gave the best separation

between ultrasound images of different classes was selected. As seen in Figure 3.5, this area

lies in the lower part of each dtrasound image which contains most of the ribeye area. Other

researchers implemented tracing rules based on fuzzy logic to trace the ribeye area [4]. This

procedure was not followed in this thesis because the Co-occurrence matrix must be computed

for a rectangular image area. Also the extracted features from an A01 of the ribeye area can

fairly characterize the texture of the whole ribeye area.

"ammon's algorithm reduces the data dimensionaiity. The main objective is to keep the distaces between dl the data points the same in both spaces so that the data clustering preserved.

3.2.5 Selecting the Window Size

To compute the Haralick's features, a window of certain size was used to compute the Co-

occurrence matrix. Using window of small size extracts localized features which might not

represent the marbling grades or fat content of the ultrasound images. Quivalently, larger

window sizes extracts global features. There is an optimum window size which should be used

to extract features that produce a satisfactory results. Moreover, there is E mathematical

mode1 that c m be used to estimate it. An alternative is to examine different window sizes

empirically, i.e.:

a- a window of certain size was selected, say for example 25 x 25, and used to extract the

14 Baralick2s features ftom ultrasound images (see Section 3.3.1) , b- the Haralick's features were mapped by the Sammon's algon thm into two-dimensional

space to examine the class separation,

c- the window size was changed to different sizes and the same steps a and b were repeated.

The window that gave the best sepmation between the different classes was selected to extract

texture features £rom ultrasound images. A window of size 45 x 45 produced the best separation

between the classes. This window size is optimum for the images used in this project. Differ-

ent window size might be obtained for different images that would be taken under different

ultrasound machine settings.

3.3 Experimental Setup

Different experirnents were conducted. Each experiment consisted of three major steps as shown

in Figure 3.6. In the first step, the 14 Hmalick's texture features, which introduced in Section

3.1.3, were extracted £rom all ultrasound images and stored as a matrix. In the second step, a

statistical pattern recognition systern (classifier) was designed to classify each image into one of

the three classes: low fat contents, medium fat contents, and hzgh fat contents. Those classes

were represented by Fat 1, Fat II and Fat III respectively. Finally, the performance of each

classifier was evaluated in terms of accuracy of classification. The experiments steps in Figure

3.6 are explained in the following sections.

Figure 3.5: The selected Area Of Interest (AOI) used to develop the classification algorithms.

3.3.1 Features Extraction

For each ultrasound image used in the experiment, the 14 Haralick's features vector was com-

puted in two steps. In the first step, an A01 of size 360 x 180 was divided into 32 s m d e r

sub-segments of sizes 45 x 45 as shown in Figure 3.7. For each sub-segment window, the 14

Haralick's features in the dierction of ultrasound wave propagation (i-e., the vertical direction

across the image) using a distance of one pixel were computed. In the second step, the mean

(the average) of all the extracted features was computed to form a single 14features vector.

Other techniques to combine the features were used such as taking the median, the maximum,

or the minimum of the features. The feature vectors were combined in different ways and

mapped in two dimensional space using the Sammon's algorithm. It was found that, taking the

mean of the features (of the subsegments) gave the best separation in the feature space. Hence,

in all the experiments that used the Haralick's features, the feature vector of each ultrasound

image was computed by taking the mean of the feature vectors of the subsegments..

Figure 3.6: Block diagram shows the experirnental setup.

Figure 3.7: The different segments of the Area Of Interest.

The Haralick's texture features for all 59 dtrasound images can be written as a mat& of

size 59x 14. Each row represents the Haralick's features vector for a singIe image. Let us c d

this matrix CFM as an abbreviation that stands for the Co-occurrence Features Matrix. Each

image can be viewed as a point in 14 dimensional space (i.e., the feature space). The concept

behind classification is based on the assumption that images of the same dass , i.e., the same

marbling grade or the same fat content, have their corresponding points occupy the same region

in the feature space. Hence, the different classes reveal themselves as clusters occupy different

regions in the featute space. The role of the classifier is to define hyperplanes that separate

the different classes. Since we can not visualize spaces with dimensions higher than three, we

use computer algorithms to establish boudaries between the different classes. The example in

Figure 3.8 shows a 2-dimensional feature space of ultrasound images. The three classes can be

separated by h o lines. Note that there is an overlap between the classes which may introduce an

error when c1assZyïng new ultrasound images. This overlap rnight be caused by the variation

between samples that belong to the same class. Careful selection of the texture feature set

reduces the overlap between classes. The correlation between the features and the different

scale among the features are another reasons for the class overIap in the feature space. The

correlation and scale difference between features can be removed by a Iinear transformation of

the feature space into another space where the features are uncorrelated and having normalized

scale. The transformation is called the whitening transformation (see Appendix C). The new

space obtained after transformation is called the whitened features space. FoUowing the same

convention, we refer to this space as the Whitened Co-occurrence Features Matrix as WCFM.

3.3.2 Designing a Classifier

Two classifiers were designed, the MED classifier ( s u p e ~ s e d ) and the k-means classifier (un-

supervised) . The MED classifier required the data samples and their chernical andysis labels.

The classifier was first trained witb the labeled samples then tested ming unknown samples.

The k-means classifier did not need the data samples labels. It used the Haralick's features to

cluster the data samples into three classes based on the similarity (i.e., the Euclidean distance)

between the features.

O O

O

O O

Fat 1

Fat II

- First feature

Figure 3.8: A hypothetical 2-dimensional feature space of uitrasound images.

3.3.3 Classifier Evaluation and Error Estimation

After designing a classifier, the next logical step is to evaluate its performance. Three methods

are commonly used: Resubstitution, Hold-out and Leave-one out methods. The methods are

different in many aspects in terms of the computational difficulty of calculating the error, the

way the data samples are used to estimate the error and the robustness of the estimated error.

It rnust be noted that the actual error is difficult to estimate because it involves many restrictive

conditions such as the normality of the data, Le., the data is coming from Gaussian Normal

distribution.

Resubstitution technique: The data samples are first used to design the classifier.

The same data set are used to evaluate the classifier performance. The estimated error is

biased since the same samples are used to design the classifier and to evaluate it.

Hold out method: In this method the data samples are divided into two sets: training

set and testing set. The training set is used to design the classifier and the testing set

is used to evaluate the classifier performance. This method produces better estimate of

the classifier performance than the resubstitution technique because it uses two different

independent sets, one for training and the other for testing. Its disadvantage is how to

divide the data samples into training set and testing set which may create a difEculty,

especidy if the data sample is small. The estimated error is not biased but it has a large

variance.

0 Leave-one out: This technique overcomes some of the probIems associated with the

previous two techniques. It yields an error estimate that is almost unbiased and has a

narrow variance. The technique works as follow:

1. remove one sample from the data set and use it as a test sample,

2. use the remaining data set to design the classifier,

3. use the sample in step 1 to test the classifier,

4. exchange the test sample with another sample and repeat the same procedure until all

the data set is used,

5. the error rate is the ratio of the number of misclassïfied samples divided by the number

of data samples.

As mentioned earlier, this technique gives better estirnate of the classifier performance since

it uses the data more efficiently. Notice that when we remove a sample to use it as test sample

and design the classification hinction using the ot her samples, we ensure the independency

between training and testing samples. The hold-one out technique needs large amount of

computation because we have to design the classifier ( n - 1) times where n is the total number

of samples.

Because the data set used in this project is small ( 59 ultrasound images), we use the

leave-one out method to estimate the success rate of the designed classifier.

3.3.4 Experiment ( H F ) : Designing MED Classifier Using Haralick's

Features to Estirnate &t

A block diagram of the Experiment ( H " ~ ) ~ is s h o w in Figure 3.9. Ali ultrasound images were

preprocessed and the 14 Haralick's features were computed. A Minimum Distance Classifier

(MED) was designed and a leave-one out method was used for the evaluation of the classifier

performance.

Features cornputation

Preprocessed ultrasound

images L

Error Estimation

Figure 3.9: Experiment (H"): MED classifier using Haralick's features.

A -

Results:

The evaluation of the classifier is shown in Table 3.5. This table is known as the confu-

sion matrix. It shows the performance of the classifier by displaying the actual classes versus

the classifier output. Each row shows each class samples and shows how many samples were

classified correctly to that cIass and incorrectly to other classes. For example, the first row

shows that out of 20 samples belong to class Fat 1, 12 of them were classified correctly as Fat

1, 6 samples were classified as Fat II, and 2 samples were classified as Fat III. To estimate

the accuracy of the classifier, take the summation of the diagonal elements and divide them

by the total number of samples. The diagonal elements represent the data which was correctly

classified to their actual classes. In this case:

Success rate = 12+14+5

59 = 52,54%

Computation of the 14-Haralick's

features

3 ~ h e label means MED classifier uses Haralick featuers to estimate fat content.

Designing MED

classifier

Leave-one-out method

Table 3.5: The confusion m a t h of Experiment ( H " ~ ); MED classifier using Haralick features to estimate fat.

3.3.5 Experiment (HEt-) : k-means Clust ering Using Haralick's Features

to Estimate Fat

The previous Experiment (HED) was based on the assumption that all the samples were

labeled using their fat chemical analysis resdts. The labeled samples were used to train the

MED classifier. In this experiment, the samples were clustered into three groups based on the

visual features (the Haralick's features) without using the chernicd analysis as a reference. That

means the k-means classifier will divide the images into three classes based on the values of the

Haralick's features ody. Some of the Haralick's features can be linked to certain visual features,

for example, f 2 to texture contrast, f 3 to correlation, f 4 to coarseness and f5 to homogeneity.

Ot her features do not have visual interpret ations such as: f 1 (energy) , f 9 (entropy) , f 12 and

f 13 (information measures of correlation). A block diagram of the experiment is shown in

Figure 3.10.

Preprocessed ultrasound

images

Computing the 14-Haralick's

features

Clustering

k-means

Leave one- out

method

Feature extraction Error estimation

Figure 3.10: Experiment (HZ-S): k-means classifier using Haralick's features to estimate fat.

Resdts:

The three clusters (Hf1,Hf2 and Ht3) that were generated by the k-means based on Har-

1 1 k-means clustering results 1

I classes t ~ a t II i 2 I 1

1 il 17 1 1 Fat III 2

9 I 1

1 7 110 1 Table 3.6: The confusion matrix of Experiment (HP"); kmeans clustering using Haralick's features to estimate fat.

alick features had the sizes (12,24,23) respectively. There was an overlap of about 2% between

the three clusters. The three clusters were mapped into their equident chernical groups based

on the majority of samples in each cluster. For examples, one cluster produced by the k-means

classifier had 2 samples from Fat 1, 11 samples £rom Fat II, and 7 samples Erom Fat III, hence

it was mapped into the cluster Hf 2 that is equivalent to Fat II group .

The confusion matrix in Table 3.6 shows the result of the clustering a lgor i th- For example,

the first row shows that 8 samples kom Fat 1 are assigned to group H f l , 6 samples are assigned

to Hf2 and 6 samples are assigned to Hf3.

The success rate of the k-means clustering can be computed from the confusion mat* by

taking the summation of the diagonal elements and divide them by the total number of samples,

1-e.,

Success rate = + + l0 = 49.15% 59

3.3.6 Comments About Experiments (HP) and (HkTms)

Experiment (H"~) and Experiment (HEens) did not produce satisfactory r d t s because

the extracted features are correlated and they have different scales which is treated in the next

section. The data sample is also so s m d so that it is not enough to represent the real data.

This is delt with in Section 3.5.

3.4 Whitening of the Feature Space

As mentioned in Section 3.3.1, patterns can be treated as points laying in high dimensional

space, namely, the feature space. Each class in the feature space bas certain probability distrib-

ution function which is commody assumed to be a normal distribution function. The features,

as extracted during the feature extraction phase, are correlated and have different scales, t hus

cause an overlap between classes in the feature space. A whitening process can be applied to

transform the feature space into another space where the features are uncorrelated, independent

and properly scaled.

For feature vectors that are normally distributed, the covariance m a t e and the mean vector

are the only parameters that are needed to characterize the probability distribution function.

Let C, be the covariance matrix of the feature space. The transformation that makes the

features uncorrelated and independent is the transformation that diagonalizes the covariance

matrix :

The transformation A takes vector x and transforms it to another vector y in another space

where the features are uncorrelated and independent. Mathematically speaking this c m be

mitten as:

Let the covariance matrix of x be x, and the covariance matrix of y be x, then the

transformation has the form:

Rom Equation (3.4) the transformation transforms C, to a diagonal matrix , i.e.,

50

x = A = diag(Xi, ..., A,) (3-7) IJ

It c m be shown that the transformation A in Equation (3.6) is simply the eigenvectors of

the matrix x, [58]. The eigenvectors for the matrix x can be found from its characteristic

equation: 1 C, - X 1 1 = O

x, is the covaziance matrix of the the feature vecton (CFM). It is always symmetric and

square n x n matrix. By solving Equation (3.7) we get n real eigenvalues (Xi, . .., A,) and n real

eigenvectors (a1, ..., a,). The eigenvectors corresponding to different eigenvalues are orthogonal

hence the transformation in Equation (3.6) can be rewritten as: C, = aT x, O = A. The

transformation in Equation (3.5) becomes :

Applying the transformation in Equation (3.8) to the original feature space x produces

new feature space y where the features are uncorrelated and independent. Another scaling

tramformation takes care of the scaling problem by making the eigenvalues equal to unity. In

other words the new transformation changes the original feature space x to another feature

space y such that the featurg have a Gaussian Normal distribution with unit variance and

uncorrelated features. The equation that takes in consideration the scaling transformation is :

The transformation

this research, Equation

W = h-lI2 @T is knovni as the Whitening Bansfomotion. Throughout

(3.9) is applied as follows:

let the feature space (i.e., the matrix CMF) be the rnatrix x which can be viewed as

row vectors xi, x2, . . . . . . , x,, where Xi represents the feature vector corresponds to single

ultrasound image. First the cova-riance matrix, C,, of the matrix x is computed,

compute A (the eigen values matrix) and Q (the eigen vecton) of the matrix x,. Note

that mat& Ais a diagonal matrix whose elements are the eigenvalues of C, as shown in

Equation (3.4).

Use Equation (3.9) to obtain the new feature space that is uncorrelated and has indepen-

dent and normalized scaied features-

3.4.1 Experiment (WHED): Designing MED Classifier Using Whitened

Waralick's Feat ues to Est imate Fat

This Experiment has the same setup as Experiment except that the

features are whitened. A block diagram of the experiment is shown in Figure 3.11.

Features cornputation

Preprocessed ultrasound

images

Error estimation

Figure 3.1 1: Experirnent (mgD) : MED classifier using whitened Haralick% features to estimate fat.

+

The result of the experiment is shown in the confusion mat* shown in Table 3.7.

Table 3.7: The confusion matrix of Experiment (WHSD); MED classifier using Whitened Haralick features to estimate fat.

Computing the Whitening 14-Haralick's -

transformation features

The accuracy of the classifier becornes:

Success r a t e = + = 40.68% 59

4

"This label reads as: MED classifier uses whithened Haralick features to estimate fat

52

t

MED classifier

Leave one- out

method

3.4.2 Experiment (WH"-) : k-means Clustering Using Whit ened

Haralick's Feat ures to Est imate Fat

This expriment has the same setup as Experirnent (WH"-) except that the Haraiick's

features were whitened to remove the correlation between them and normalize their scale.

Figure 3.12 shows the block diagram of the experiment.

Features computation Error estimation

Figure 3.12: Experiment (WHEmS): k-rneans clustering using whitened Haralick's features to estirnate fat.

P reprocessed ultrasound

images

The confusion matrix is shown in Table 3.8.

- -

Computation of

the 1 4-Haralickrs

feature

A Y

r - -- ( k-means clustering results j

1 Actual t Fat 1 i 15 1 4 f i 1

Whitening 3

transformation

1 classes / Fat II i 4 1

1 13 1 3 1

Leave- one-out rnethod

I 1 ~ a t III 1 2 1 1 1

1 4 1 13 / -.

Table 3.8: Confusion matrix of Experiment (wE3EanS); kmeans using Haralick's features to estimate fat.

The accuracy is:

Success rate = l5 + l3 + l3 = 69.49% 59

3 -4.3 Comments about Experiments (WHED ) and (WH"-)

The result of Experiment (mgteans) improved compared to the result of Experiment ( ~ $ 2 ~ ~ 1 while the result of Experiment (WH") became worst than Experiment (H"~). The

whitening transformation matrices for Experiment (mgD) were singular because they were

computed for each class (20 samples for each class). The number of features (Le., 14) is large

compared to the size of data sample (i.e., 59), hence, the covariance matrix was nearly sin-

gular. Experiment WHEt- did not have the singularity problem because the whitening

transformation matrix was computed using all the data sampIes. This problem occurs when

the number of features and number of data sarnples are difTerent which is known as the curse

of dimensionality problem. It is very similar to the problem of solving m linear equations in n

variables when m < n. The next section deals with the problem.

3.5 Adding Random Data and Selecting the Best Features

Experiments ( H E D ) and WH"^) use the fourteen Haralick's features. I t is well known

that using large numbers of features does not always improve the classification performance. By

adding more features the dimensionality of the feature space increases which makes it difficult to

estimate the probability distribution of the data. To better estimate the classifier performance

the data must be increased. Adding more data for this thesis was diEcult because we have

to get extra ultrasound images of live beef animais.. The animals should be slaughtered and

then a chernical analysis of the ribeye data of each animal should be determined. Instead, the

following procedure was followed:

1. Artificial samples were added to the datas. Normal distribution random generator was

used to generate data samples that had the same mean and vanance as the mean and

variance of each class. Each class was added twice its size of rudom data. For example,

class Fat 1 has 20 samples, so 40 random samples were added to it,

2. The features were ranked in ascending order based on the intra-class distance criteria,

3. The success rate of the MED classifier was computed as a Eunction of the ranked features.

"The method is similar to the bootstrap technique.

feature vector fl f2 o a o f 3 . 4

Fat 1 pi 4 pi4 mean nu, vnriance 4 a O O 0t4

Fat II P? i-4 ~ f 4

- - - - - - -

Table 3 -9: Cornputing the intra-class distances.

Calculating the Intra-Class Distances and Ranking the Features

The values shown in Table 3.9 are cornputed for each ctass.

The first row in Table 3.9 shows the individual components of Haralick's feature vector.

The mean and variance of each cornponent are cornputed for each class. For example p:4 is the

mean of the 1 4 ~ ~ component of the vectors of class Fat II and of4 its variance. The distances

between classes (dl*, d23, and dis ) as a h c t i o n of individual features are coniputed using the

formula shown in the table. Since the components of the feature vector have different scales,

we divide by the summation of the variances to normalize the intra-class distances . The total

intra-class distances (dtotoi) are the mean of the intra-class distances which are shown in the

last row of Table 3.9. The larger this value is, the more important the corresponding feature

component. If this value is large, it indicates that this feature component is important because

it causes a large separation between the classes.

3.5.1 Experiment ( R + H ~ ~ ) : MED Classifier Using Larger Data Sample

and Employing Feature Selection Process to Estimate Fat

In this thesis the intra-class distances based on Haralick's features were calculated and ranked

in ascending order. An MED classifier was designed using the first best feature then designed

again using the best fi& and second features and so on until al1 the feature were used. The

same procedure was repeated for the Whitened Haralick's features. A leave-one-out met hod

was used to evaluate the classifier performance. Table 3.10 shows the result. The best results

are emphasized.

1 Number of selected 1 Success rate 1 1 features 1 No Whztenzng 1 Whztenzng 1

Table 3.10: Results of Experiment (R+H=*); MED classifictaion after adding random data and using feature selection process to estimate fat.

The results of Experiment (R+H"~)~ in this table shows a significant improvement over

the pervious Experiments (EIED) and ( w H E D ) . The results indicate that adding more

features increased the classifier performance to certain point t hen its performance degr aded.

Notice that the distribution of the original data was assumed to be normally distributed which

is not a definite assumption. The distribution of the data after adding random samples certainly

has a normal distribution because 75% of that data was generated by a normal random genera-

tor. That means adding random samples changes the distribution of the data and consequently

changes the classifiers performance. Hence, the result of the MED classser in this experiment

using the 14 features which is 40.22% is not the same like the results obtained in Experiment

( H g D ) which was 52.54%.

" ~ h e label reads as: MED classifier using Haraiick features and randorn data to estimate fat.

3.6 Designing Classifiers Based on Marbling Grades Using

Haralick's Feat mes

Since the chemical analysis is not the only factor that is normally used to grade the animal,

an alternative is needed to classi& the samples based on a different criteria or mesure. The

experiments that have been done so far are all based on chemical analysis. The three classes

(Fat 1, Fat I I , Fat I I I ) were established based on the percentage of fat of meat samples taken

hom the ribeye area of each animal. In Chapter 1, we talked about animal grading. The top

Canadian class A animal grades account for about 90% of the total graded animals. Class A

has Four subclasses which are designated by A, AA, AAA and Prime. Those classes are

practicdy identified by human grading experts based on some rules and experience. There

is no quantitative nile that can be used to grade the animal. Studies showed that chemical

analysis of the ribeye area of a beef and its grade are not weU correlated. That means fat

chernicd analysis and marbling grade did not exactly quanti@ the same thing. For marketing

purposes rnarbling grade is more important. When this research was started, the chemical

andysis of the ribeye of each animals and its ultrasound image were available but the animal

grades were not. Therefore al1 the experiments were carried out using the fat chemical aaalysis.

As the research progressed, animal grades were obtained h m the Animai and Poultry Science

Depart ment. Therefore the following experirnents were conducted t O show the capability of the

designed algorithms to evaluate the marbling grade.

First the images were divided into t h e e groups based on the marbling grades. Table 3.11

shows the new groups which were assigned the names A , AA and AAA that corresponds

reqectively to Race marblz'ng, Slzght marbling, and Srnall rnarblzng. The fourth class Prime

was not considered for three reasons:

0 the algorithms were designed to handle problems of three classes. To handle four classes

problems all classifications and error evaluation progrwns should be re-writ ten.

Prime group as dehed by the Canadian beef grading program as a meat with Slzghtly

Abundant marbling is an extreme case. htroducing such class disturbs the classifiers

current performance which dictates fine tuning them again to handle such a case .

Table 3.11: Data samples divided into three groups based on the marbling scores.

consumer studies showed that meat with small amount of evenly distributed intramuscdar

Small marbling

fat is more desirable than the void one [3]. That means meat without fat has a narrower

7 1 36 1 16

image id.

6856 6921 6868 6848 6960 6934 6836

consumer bais than a meat with s m d amount of marbling.

AAA sanaple name m30 m31 m32 m33 m34 m35 m36

Slight marbling AA

3 -6.1 Experiment (HE:-) : k-means Clustering Using Haralick's Feat ures

to Assess Marbling Grades

àmade id.

6911 6842 6955 6862

mace marbling

The results are s h o w in Table 3.12. The success rate is 37.28%.

zmage id.

6847 6925 6837 6901 6953 6961 6858 6882 6845 6844 6906 6902 6852 6917 6861 6834

sample name m20 m21 m22 m23

A sample narne ml0 ml1 ml2 ml3 ml4 ml5 ml6 ml7 ml8 ml9 ml10 ml11 ml12 ml13 ml14 ml15

image id.

6924 6843 6833 6875

6933 6866 6959 6846 6883

sample name m218 m219 m220 m221

6946 6894 6835 6896 - 6841

m24 m25 m26 m27 m28

m222 m223 m224 m225 m226 m227 m228 m229 m230 m231 m232 m233 m234 m235

6831 6830 6873 6923 6877 6954 6857 6876 6932

Number of samples

m29 1 6 8 7 2 m210 m211 m212 m213 m214 m215 m216 m217

6 8 4 0 6945 6908 6893 6849 6851 6938 6964

I I k-means clusters I 1 Actual 1

I

Table 3.12: The confusion matrix of Experiment (HE;-); krneans clustering wing Haralick's features to assess marbling gardes.

L

AA AAA

3 -6.2 Experiment (WH&?!?-) : k-means Clustering Using Whitened

Haralick's Features to Assess Marbling Grades

13 3

The results are shown in Table 3.13. The success rate iç 77.96%

13 2

J

10 2

Actual classes

Table 3.13: Confusion rnatrix of Experiment (WH"-') ; kmeans clustering using Whitened Haralick's features to assess marbling grades.

1

AA 1 5 AAA 1 O

All the classifiers described in this chapter used Haraück's features. Two different approaches

were implemented, supenrjsed and u n s u p e ~ s e d classification algorithms. The unsupervised

classifiers gave the best success rates because they clustered the samples based on the natural

cluste~ng of the Haralick's features while the supervised classifiers were forced to classiSr data

samples based on a predeôned classes based on fat chernical analysis. The whitening process

had the tendency to improve the success rates of t he classifiers but because the nurnber of

Haralick's features were large compared to the size of da ta sample, the results were not as good

as expected. To solve such a problem, the samples were increased by adding artificial samples

and a feature selection process was employed. The second part of the experimental work was

using the k-means classifier to assess the marbluig grade. Other experiments that used the

MED classiner to assess the mazbling grade was also conducted and their results are shown in

Chapter 6 in Table 6.1.

k-means clusters

2 7 1

Hm3 1

1 Hml A 13

4 6

Hm2 2

Chapter 4

Meat Grading Based on Image

Transforrns

In Chapter 3, we explained how uitrasound image can be considered a stochastic signal and

therefore its texture information c m be computed fkom the probability distribution function of

a graylevel values. In this chapter, we show how the texture of the image is estimated through

a linear filtering transformation of the image. Let the image be denoted by x(i, j) and its

transformed version be y(i, j), the kth Iinear transformation or filtering of the image can be

written as:

where 8 represents the convolution operator and gk( i , j ) is a filter or a mask.

Filtering the image by a mask gk(i, j ) produces a transformed image y&, j). The energy

(or density) of the transformed image, or an area of it, represents a feature. The essence of

such approach is to use different f3ters to extract some featureç kom the image including edges,

spots, ripples, and cornpute their dençity. This approach characterizes texture at higher level

than the one presented in Chapter 3. Experimentally, it was shown that for certain type of

textures, the high level approach gives better results than the lower level approach [59]. Next

section discusses Laws' masks technique for texture features extraction as suggested by Lam

in his Ph.D. dissertation [60].

4.1 Texture Features Extraction by Laws' Masks

The main idea of this technique is to convolute an image by a set of 5x5 windows then using an

averzging operation, the energies (or the densities), of the convoluted images are cornputed. A

set of onedimensional filters ( L5, E5, S5, W5, R.5 ), each bas a length of five pixeIs, are defined

and used to detect : level, edge, spot, wave and ripple. The 1-D Uters are d e h e d in Table 4.1.

By convoluting any two of these 1-D filters, different windows of size 5x5 are generated. For

example, by convoluting L5 and E5 we get 5x5 window (Le., L5E5). Taking al1 the 1-D filters

combinations, 25 different wiadows can be generated. These windows are listed in Table 4.2.

1 E5 = [ -1 -2 O 2 1 111 edge 1 I S5 = ( -1 O 2 O -1 ] 1) spot 1

Table 4.1: One-dimensional Laws' rnasks.

W5 = [ -1 2 O -2 1 ] R5 = [ 1 -4 6 -4 1 ]

Table 4.2: The 5x5 Windows obtained by convoIutiog the one-dimensional filters given in Table 4.1.

wave ripple

The process of extracting texture features can be performed as follows:

1. To extract Laws' texture for an image X( i , j) of size N x M, the image is first convoluted

with the 25 windows given in Table 4.2. This operation produces 25 (N x M) texture

images. We refer to each one of these images by the name of the window used to praduce

it. For example, if we use W5R5 to tilter the image X( i , j ) , the result is a texture image

that has the name Xw5=. Convoluting the image X( i , j) with the 25 windows yieIds the

set of T&um Images (TI) in Table 4.3.

2. All the kernels, except kernel L5L5, in Table 4.2 have zero mean, i.e., the summation

of the elements of the kernel is qua1 to zero. That means the texture images in Table

4.3 have the same energy as the original image X ( i , j) except the texture image X L 5 ~ 5 .

According to Laws [60], texture image XLjL5 can be used to normalize the contrast of

all the texture images in Table 4.3, by dividing them by the image XLSL5 pixel by pixel.

Afterward the image texture XLSL5 c m be discarded unless the contrast is deemed an

important feature.

3. For each one of the 25 texture images, a Tdure Density Image is computed by replacing

each pixel in the texture image (TI) by the average of the neighborhood pixels within

a window using Equation (4.1). By applying Equation (4.1) to the 25 texture images in

Table 4.3, a set of 25 Texture Density Images are obtained. These images are listed in

Table 4.4, where the dash signifies an averaged texture image or equivalently a T&UR

Density Image TDI. 7 7

4. To reduce the number of Texture Density Images (TDI) in Table 4.4, some of them can

be combined together. For example, images XE and X m c m be combined together

by adding them pixel by pixel. Image X s signifies the density of vertical levels and

horizontal edges of the original image X ( i , j), while image XE signifies the density of

horizontal levels and vertical edges of image X( i , j). By adding the X m and X m

we obtain a Rotatzonally Invariant Texture Densitg Image that represents the density of

edges and levels regardless of their orientations. This step is important if the directionality

of the texture is not significant. Table 4.5 shows the result of combining aU the TDI in

Table 4.4 to obtain the Rotationdy Invariant Texture Density Images RI-TD 1.

5. Note that, the last five rows of Table 4.5 are the Texture Density Images that origi-

nally have no orientation. These images are multiplied by twa to keep all the RI-TDI

consistent .

6. For this thesis, the average of the AOI, such as the one used to extract Haralick's features

kom each RI-TDI, was computed which represents a texture feature. The last column in

Table 4.5 shows the texture feature name extracted £rom each RI-TDI.

Table 4.3: The set of texture images (TI) of the image X(i j) .

Table 4.4: The 25 Density Texture Images (TDI) of the image X(i j).

1 RI-TDI 1 the combined TDI 1 extracted feature 1

Table 4.5: The fifteen Laws' features

4.2 Designing Classification Algorit hrns Based on Laws' Masks

Feat ures

4.2.1 Texture Features Extraction

For each ultrasound image X ( i , j), the 15- Laws' texture images given in Table 4.5 were ex-

tracted and the set of the fifteen Rotational Invariant Density Texture Density Images (RI-

TDI) in Table 4.5 werecalculated. The size of each RI-TDI was the same as the size of the

original ultrasound image X ( i , j). The mean value of an Area Of Interest 'A01 ', sirnilar to

the one used for the extraction of Haralick's features which was a window of size 360x 175, was

computed for each RT-TDI. The diagram in Figure 4.1 shows the process of Laws' texture

feature extraction.

The same 59 ultrasound images that were used to design classification algorithms using

the Haralick's features were also used to design and evaluate classification algorithms based

on texture features extracted by Laws' masks. The images were preprocessed to adjust their

contrast . No filtering operation was performed to preserve image texture.

An ultrasaund image 25 Texture Images 7'

!

l ~ n g t t e m l ~ O w Rotationally

invariant

25 Densiiy Tenure The 15 RI-TDI

images 7DI' Images

1 \

- 8 the AOI from each + (f , . f z . f ,..... f ,,) RI-TOI

\ J

Texture fearure vector for X(i.j)

Figure 4.1: The process of extraction of texture features by Laws' masks.

4.2.2 Experiment (LZ~) : Designing MED Classifier Using Laws y

Features to Estimate Fàt

A block diagram of the experiment is shown in Figure 4.2. Ali ultrasound images were processed

and the 15 Law' features were computed. A Minimum Distance Classifier ( M m ) was designed.

For error estimation, the leave-one out method, which was explained in Chapter 3, was used.

Figure 4.2: Expriment (L"~): MED classifier using Lam' features to estimate fat.

Preprocessed Computation of A the1 5-Laws' ultrasound

images features I

Results:

The resdts of the classification are shown in Table 4.6 (the confusion matrix). The perfor-

mance of the classifier is displayed in terms of the actual samples classes versus the classifier

output. Each row represents a fat class and shows how many samples are classified correctly

to that cIass and incorrectly to other classes. For example, the first row shows that out of 20

smples £rom class Fat 1, 12 were classified correctly as Fat 1 class, 7 samples were classified

as Fat II class, and one sarnple was classified as Fat III. The success rate of the classifier is

the summation of the diagonal elements divided by the total number of samples used in this

experiment,

"

Succes rate = l2 -t l1 + = 49.15% 59

Desig ning the Leave-one-out MED classifier rnethod

1 Classificatzon result 1 1

1 1 Fat 1 1 Fat II 1 Fat III 1

--

Table 4.6: The confusion matrix of Experiment (LE~); lVIED classifier using Laws' features to estimate fat.

Actual classes

4.2.3 Experiment (~2;~): k-means Clustering Using Laws9 Features to

Estimate Fat

Ekperiment (lLZD) used a supervised classifier trained with sarnples that were Labeled using

their fat chemical analysis. In this experiment an unsupervised classifier used to cluster the

data samples into three classes based on Laws' features only, Le., there was no training process

involved in designing the grading algorithm. A block diagram of the experiment is shown in

Figure 4.3,

Fat1 Fat II Fat III

Preprocessed Computation of ultrasound the 15-Laws'

images features

Leave-one- k-means out

clustering method

12 7 6

Figure 4.3: ~ x ~ e r i m e n t (LEcms): k-means clustering using Laws' texture features to estimate fat.

Results:

The three clusters (labeled as Lfl, Lr2 and Lr3) that are generated by the k-means classifier

based on Laws' features have the sizes (10, 22, and 27) respectively. Each cluster had samples

kom the chemical fat groups (Fat 1, Fat II, and Fat III). These clusters were mapped into

chemical groups based on the majority of chemical classes samples in each cluster. For example,

Lf 1 had the majority of samples £rom Fat 1, hence it was mapped to Fat 1. There was an overlap

of about 3.38% between the three clusters. The confusion matrix s h o w in Table 4.7 shows the

result of the k-means classifier. It can be seen £rom the f is t row that, 8 samples EIom Fat 1

7 11 7

1 2

6 A

were assigned to group Lfl , 5 samples were assigned to Lr2 and 7 samples were assigned to

Lf3-

1 1 k-means clustering results 1 -

I I Lfl I Lf2 I Lf3 I

1 1 1

Fat III 1 1 17 1 11 classes

I A c t u a l Table 4.7: The confusion rnatrix of Experiment (L~F-); k-means classifier using Lam' fea- tures to estimate fat.

The consistenq of the three clusters Lrl, Lf2 and Lf3 with respect to the chemical analysis

can be computed &om the above rnatrix by summing the diagonal elements divided by the total

number of samples, i.e.,

Fht II

Success rate = + l1 + l" = 49.15% 59

The resdt was low because the Laws' features were not independent which will be dealt

with in the next section.

4.3 Whitening of the Laws' Feature Space

7 1

Experiments (L"~) and ( L e S ) were performed without considering the correlation and

the scale ciifference among the Laws' features. The following two experiments deait with that

problem by whitening the feature space. Similar to Experiments ( H E D ) and (Hgt-),

two dinerent classifications slgonthm~ are used, namely, the MED classifier and the k-means

classifier.

8 - ' 10 9 5

4.3.1 Experiment (WL") : Designing MED Classifier Using Whitened

Law& Texture Feat ures to Est imate F'at

This experiment had the same setup as Experiment (L"~) except that the features were

whitened. A block diagram of the experiment is shown in Figure 4.4. As mentioned before,

the whitening transformation has two purposes. First i t rnakes all the features uncorrelated.

Second it normaiizes their scale so that there is no scale dominate among Laws' features.

Figure 4.4: Experiment (wL"~): MED classser using Laws' texture features to estimate fat.

Results :

Leave- one-out method

The results of the experiinent are shown in the confusion matrix is in Table 4.8.

Preprocessed ultrasound

images :

I . 1 Fat 1 1 Fat II 1 Fat III (

Ir

Designing the MED classifier

Computation 1

1 , I

I - Fat III 1 7 19 1 3 1

of the 1 5-Laws' features

Actual classes

Table 4.8: The confusion matrix of Experiment (WLK*); MED classifier using whitened Laws' features to estimate fat.

The success rate of the classifier is,

*

Fat 1 Fat II

Sucess rate = 6'9+3 =30.50% 59

Whitening transformation

6 1

I l 9

3 10

4.3.2 Experiment (WLzaLIS): k-means Clustering Using Whitened Laws7

Features to Estimate Fat

This experiment had the same setup as mer imen t (L"-) except that the Laws' features

were whitened. Figure 4.5 shows a block diagram of the experiment.

Computation Preprocessed Leave-one- of the Whitening

ultrasound out 1 5-Laws' r transformation

images method features

Figure 4.5: ~ x ~ e r i r n e n t ( ~ ~ ~ ~ ~ ~ ) : k-means clustering using whitened Laws' features to estimate fat.

Results: The confusion matrix is shown Table 4.9

k-means clustering results

Table 4.9: The confusion rnatrix of Experiment (WLzteans); k-means classifier wing whitened Laws' features to estimate fat.

Success rate = l9 + l3 + l3 = 76.27% 59

The result of this experiment is better than the result of Experiment (ILkee").

4.4 Adding Random Data and Selecting the Best Features

AU the previous experiments used the fifteen Laws' features. As mentioned in Experirnent

(R+H=~), it is well known that using large number of features does not guarantee an optimal

classifier performance. By including many features, the dimensionality of the feature space

increases which makes it difEcult to estimate the probability distribution of the data. To

better estimate the classifier performance, the data sample must be also increased. Hence, this

experiment consisted of three steps,

1. artScid samples were added to the data. A normal distribution randorn generator was

used to generate data samples that had the sarne mean and variance of each cIass,

2. the features were ranked in ascending order based on the intra-class distance criteria,

3. calculating the MED classifier success rate (accuracy) as a function of the ranked Laws'

features

4.4.1 Experiment (R + L"~): M E D Classifier Using Larger Data Sample

and Employing Feature Selection Process to Estimate Fat

Table 4.10 shows the result. The best results are emphasized.

Table 4.10: Results of Expe~ment (R + L"~) ; using Lam7 features and applying feature selection process to estimate fat.

Number of seiected kat ures (1 /15) .

Success rate No Whitenzng

45.25 Whitening 45.81

4.5 CIassificat ion Using Laws' Features to Assess Marbling Grades

Section 3.6 explained the justification behind carrying experiments based on Marbling grades.

4.5.1 Experiment ( L E - ) : k-means Clustering Using Laws' Features to

Assess Marbling Grades

The results are shown in Table 4.11. The success rate equal44.06% .

Actual classes

Table 4.11: The confusion matrix of Experiment (L"~"s); k-means classifier using Law' features to assess marbling.

AA AAA

4.5.2 Experiment ( W M ~ ~ - ) : k-means Clustering Using Whitened Laws'

k-means clwters

Features to Assess Marbling Grades

13 4

The results are shown in Table 4.12. The success rate equal 64.40%

L,3 1 A

15 2

L,1 10

8 1

Actual classes

Table 4.12: The confusion matrix of Experiment (LZ-~); k-means classifier using whitened Laws' features to assess mar bling.

L,2 5

AA AAA

This chapter de& with Lam' features. We have also seen how the whitening transformation

improved the success rate of the designed classifiers, specScaUy, the unsupervised one. This

shows that training the grading algorithm using the classes as established by the chernical

k-means clwters

A 6 O

L m l 11

21 1

9 6

Lm2 2

Lm3 3

analysis does not produce satisfactory results. The best approach is to use unsupervised classi-

fier to classi@ the data samples into three groups based on the Laws' features and then compute

the consistency between the visual classes1 and chemical andysis classes. Adding random sam-

ples and selecting the best features Unproved the resdts. It is a similar approach to the well

known procedure referred to as bootstmp, cross-validation, and jackknifing [61, 621. The ran-

dom samples were added based on the assumption that the data was norrnally distributed.

The results of the original data and the data with the random samples were not the same as

their distributions were not the sarne. The second part used Laws' features to assess marbling

grades. Marbling assessrnent was better than estimating fat contents. This may be because

Laws' features are more linked to marbüng than fat contents. Marbling as a property is more

evident as texture more than fat contents.

This chapter and the peMous one use two different classification techniques (supeMsed

versus unsupe~sed) and two different feature sets ( Haralick versus Lam) to seek the best

grading algorithm. The next chapter deals with classifier fusion method based on Bayes' theo-

rem as a way of improving the grading results. It is a way of making a statistical decision using

the results of two independent grading algori t hms.

'Haralick's and Laws' features.

Chapter 5

Fusion of Multiple Classifiers

Traditionally, the trend in designing a pattern recognition system to achieve high performance is

to fine tune single classifier until the required or the best result is achieved [43]. If the classifier

does not provide good results, it is replaced by a different classifier and the same tuning process

is repeated. Mostly the case is that different classifiers produce comparable performance but

at the same tirne their performance is not the same. For example, in one case two classifiers

are given certain problem, the hrst classifier produces good results and the second classifier

produces bad results, and in another different probIem their performance interchange. That

means the classifiers complement each other.

A new approach has recently emerged that combines different classifiers to achieve high

performance. Each classifier is viewed as an expert that has certain point of view in a given

problem. The decision that is made by all experts together is better than the decision that

would be made by an individuai expert alone. A combination methodology that takes advantage

of each classifier and avoid its weakness is investigated and cautiously used so that the fusion

process does not lead to wrong decisions.

To combine different classsers, they should be independent in their decisions to minimize

errors. This condition is satisfied in two different cases. In the first case multiple classifiers that

have different decisions h c t i o n s are used. For example, classifier uses MED decision function

c m be fused with another classifier that uses k-means clustering technique. In the second

case, the independency condition holds true for fusion of multiple classifiers that have the same

decision function but each classifier uses different feature set. For example two MED classifiers

c m be fused if one uses the Co-occurrence features and the other uses Laws' features. The

latter approach has been selected in this thesis because the objective is mainly to investigate

the use of texture in ültrasound images to iden te marbling grades and fat content. It is also

more practical because if we know, for example, that the Haralick's and Laws' features are

adequate for identwng the marbling grades and fat content, we just need to combine multiple

classifiers having the same decision function such that each classifier uses subset of Haralick's

or Laws' features.

5.1 Met hodologies for Combining Multiple Classifiers

Generdy speaking, classifiers can be divided into two types based on their output. m e - I

classifiers output a label of the class with the highest probability. Qpe-II classifier output a

ranked list of d possible classes such that each eIement in the list is given probability value

that shows how much that element is to be the correct one. This probability is estimated using

the observed features. Examples of ripe-Il classifiers are, the Bayes' classifier, the MAP'

and the Minimum Euclidean distance classifier. For example, Bayes a d MAP classifiers assign

posterior-probability value to the possible classes such that the correct class receives the highest

probability value. Euclidean classifiers assign distance value to all possible classes such that

the correct class is the one with the smallest distance. Type-1 classifiers are more generd than

ripe-11 classifiers as many classifiers such as the syntactic classifiers produce single class. Also

any Qpe-II classifier can be easily transformed into Type-1 classifier by selecting the class

that has the highest value in the list and ignoring the other classes. In the next sections,

three different approaches for classifiers fusion are discussed namely, the Voting approach, the

Averaged Bayesian approach and the Generalized Bayesian Approach (GBA). The GBA, which

has not been used before for fusion of multiple classifiers, is proposed and used in this thesis.

5.1.1 Classifiers Fusion by Voting

This approach is used to combine Type-I classifiers. All classifiers are treated equally by giving

each classifier single vote. The final decision is the class that receives the majority of votes. This

'Maximum A posterior Probability classifier.

technique does not consider the relative performance of the classifiers. For example, if a classifier

votes for certain class with confidence value of 95% while another classifier votes for another

class with confidence value of 51% both contribute the same to the final decision. Intuitively,

the first classifier should be given more weight than the second. Moreover, the classifiers

shodd be completely independent in their decisions otherwise the final decision is biased. For

example, three classifiers are combined, two are dependent but the third is independent. The

h a I decision is aIways biased towards the two dependent classifiers regardless of the confidence

in their decisions. FiOrne 5.1 shows block diagram of the voting approach. The thresholding

operation outputs the final decision once the voting poll receives certain number of votes.

f hresholding Operation

e The final decision

Figure 5.1: The voting approach.

5.1.2 The Averaged-Bayesian

The Bayesian mode1 uses statistical and probabilistic decision approaches based on Bayesy

theorem to fuse the decision of multiple classifiers. The Averaged Bayesian Approach fuses

Qpe- I I classifiers where each classifier outputs a list of posterior-probabilities values that reflect

the likelîhood of each class. Using this posterior-probabilities allows each classifier to be given

different weight in the &al decision. As mentioned before, the classifiers should be independent

in their decisions in order for the computational problem of this mode1 to be tractable.

Consider classification system to deal with M classes. Without loss of generaiity, suppose

two Bayes' classifiers are ernployed. For every given unknown sample x to be classified, each

clas&er j produces a list of probabilities {Pj(Ci/x)) for each class i. The h t classifier

produces the following Est for a n unknown sample x:

and the second classifier produces the list:

Normally, to h d the decision of the f i t classifier to an unknown sample x we simply select

the maximum of the probabilities in Equation (5.1), i.e.,

where dl (x) represents the decision of the first classifier.

In the Averaged Bayesian approach, the average of the probabilities of Equation (5.1) and

Equation (5.2) is computed. Shen the maximum probability is selected that represents the final

decision of both classifiers, Le.,

The maximum of the list O ( x ) represents the final decision..

To fuse of n classifiers that harde classification problem of M classes where each classifier

produces a list {Pk(Ci/x) }, Equation (5.4) can be written as:

where

The final decision is the maximum of the List (5.5):

The process is shown in Figure 5.2.

Classifier II Classifier n yJ p 2 ( c , F 1 :3(:jxF1 P (C 4x1

Averaging process a i

U The final decision

Figure 5.2: The Averaged Bayesian Approach.

The Voting approach, that was explained in the previous section, can be derived £rom the

Averaged Bayesian as follows: each classifier k outputs a list of probabilities for given unknown

sample x as follows

For single classifier k the decision is the maximum probability in list (5.6)' Le.,

The probability Pk(Cj/x) that is produced by the decision function dk(x) is considered as a

vote for class j by the classifier k. The class that gets the majority of votes is the final decision.

5.1.3 The Generalized Bayesian Fusion Approach (GBF)

The GBF approach is used in this thesis to aggregate the decision of multiple classifiers by

using Bayes' theorem without introducing other hc t ions such as the summation function used

in the Averaged-Bayesian approach.

Bayes' theorem uses some prior-probabilities combined with some sort of information such

as a test or a sample to compute the posterior-probabilities as shown in Figure 5.3. The injtial

probabilities before conducting the test or the experiment are called the prior-probabilities.

The probabilities after doing the experiment are called the posterior-probabilities.

probability or sample

Bayes' theorem

Posterior 1 probability

Figure 5.3: The Bayes' theorem,

To derive Bayes' theorem we start by the dennition of the conditional probability of event

A given B whicb is,

Any two events A and B are mutually independent if

P { A / B ) = P { A )

P {B /A) = P {B)

and

P {AB} = P { A ) *P {B)

By the same definition as in Equation (5.7), the conditional probability of event B given A

is,

Cornbining Equations (5.7) and (5.8), yields,

which is the Bayes' theorem, where P { B / A ) is the pnor-probability and P {AIB) is the

posterior-probability.

The generd definition of the Bayes' theorem is given below.

Bayes' Theorem[63]

Let { H j ) be mutually exclusive events such that P ( H j ) > O. Let A be an event with

P(A) > O, then for j = 1,2, ...

Bayes' theorem is used in various applications. The one that we interested in here is the

application of Bayes' theorem in making statistical inferences and decisions.

Hypothesis Test h g by Bayes' Theorem [64]

We sornetimes concerned with the problem of testing a hypothesis A versus an alternative

hypothesis B. We assume that A and B are m u t u d y exclusive and exhaustive hypotheses- Let

T = T(xi, ..., xN) denote an appropriate test statistics based upon a sample of N observations.

Then by Bayes' theorem, the posterior probability of A, given the observed data T, is

where P(A) and P(B) denote the pnor probabilities of A and B.

Similady, for the hypot hesis B,

Note that since A and B are exhaustive hypotheses then,

combining Equations (5.11) and (5.12) yield:

that is, the posterior ratio in favor of A is equal to the product of the prior ratio and the

likelihood ratio. If the ratio in Equation (5.13) exceeds unity, then we accept the hypothesis A

and reject B, else we accept B and reject A.

If there are several possible hypothesis, Say for example Ai, A*, .....A, the GBF approach

c m be extended by first computing the posterior probabilities as in Equations (5.11) and (5.12)

the hypothesis with the largest posterior probability is accepted as the ha1 decision, Le.,

first compute the posterior probabilities for each hypothesis Ak:

where k = 1,2,,.n

r accept the hypothesis with the maximum posterior probability value, Le.,

Interpretation of the Hypothesis Testing Based on Bayes' Theorem

If we have no current observations to make a decision about certain event A, then the

decision should be made based on our previous experience about the event, Le., using P(A)

ody. In other words, we just use the prior probability density function. If we have both

the previous experience and the current observations T, then we revise P(AJ according to the

Bayes' theorem and base our inferences about P(A/T) . Even though there are different ways

to make statistical inference, Bayes' theorem gets great attention due to its simplicity. Some of

its advantages are [64]:

1. we t ake previous experience int O account explicitly,

2. previous experience makes us more confident about our decision by making the confidence

interval smaller, and the result estimator has smaller variance with smaller mean-squared

error,

3. d o w us t o use objective probabilities instead of prior probabifities,

4. making a hypothesis testing without specifying an arbitrary level of significance to carry

our decision.

5.2 Fusion of k-means Classifiers by the GBF Approach

5.2.1 Experiment (HL$ZF): Fusion of k-means Classifiers to Estimate FBt

The results that were obtained in Experiment (Hete8ns"s)d Experiment ( L g a M ) were fused

together by the GBF approach. Haralick's and Laws' features were not whitened. A block

diagram of the experirnent is shown in Figure 5.4

Figure 5.4: Block diagram of Experiment (HLgzF): fusion of two k-means classifiers by the GBF approach.

Image data

To implernent the GBF approach, two probabilities are needed, the prior probabilities and

classes probabilities to cornpute the posterior probability. The pnor probabilities are the results

of two classiEers. The classes probabilities describes the probabiliw distribution of the original

data. The posterior probabili% is computed by combining the prior probabilities of the two

classifiers and the classes distribution using the Bayes' theorem.. The posterior probability

Computation of the 14-Haralick's

features

represent the combined decision of the two classifiers according to Bayes' theorem.

Experiment (see page 48), which used Haralick's features produced three clus

ters, namely, Hfl, Hf2 and Hp3 and Experiment E LE^-^), (see page 67), which used Laws7

features produced the clusters Lfl, Lf2 and Lr3, were fused together by the GBF approach.

For the sirnplicity of notation, let us denote the three groups (Le. Fat 1, Fat II, and Fat III

) by the symbols F I , F 2 and F3. For each sample used in the experiment, we find where it

has been assigned by the two clustering a l g o r i t h , Say for example, a sample x is c1usi;ered

to Hf 1 by the Ç s t algorithm (i.o., Experiment (HZ-) ) and Lf3 by the second algorithm

(i.e., Experiment (LE-) ). Let us drop the subscript f and denote Hf 1 by H l and Lr3 by

L3. We use the GBF approach to find the actual class of this iinknown sample by the following

procedure:

k-rneans clustering

Image

calculate the three posterior probabilities of the unknown sample x as follows:

w Decision

-

making using the GBF

data

Computation of

-

u I features I the 15-Laws'

k-rneans clustering

-

End the largest of the three posterior probabilities and assign the unknown sample x to

the class label of this posterior probability, Le.,

the label of x max{P(Fl/HlL3) , P(F2/WlL3) , P(F3IHlL3))

for example, if P(F3/HlL3) is the largest one, then assign x to class F3.

In general for an unknown sample x that is clustered into Hi and Lj, we calcul

posterior probabilities,

and then assign the x to the largest posterior probability value,

the label of x max{P(Fl/HiLj),P(F2/HiLj), P ( F 3 / H i L j ) }

e the three

We have some notes about Equation (5.14),

1- The sum of the three posterior probabilities always equals one , Le.

P ( F l / H i L j ) + P(F2/HiLj) + P(F3IHiLj) = 1

2- If for an unknown sample x, the three posterior probabiLities are equal, then we can not

assign a label to the sample x. In what follows, such sample will be referred to as an undecided

sample.

' ~ h e probability of sample x which belongs to group Fut1 gïven that it is clustered to Hl by the first algorithm and to L3 by the second algorithm.

3- The d e in Equation (5.14) is very similar to the MaxLmum A Posterion Probability

Classifier ( M M ) which was explained in Chapter 2.

Results:

The results of the fusion are shown in Table 5.1 in details. The first column is the name of

the sample. The second column shows where that sample belongs to based on the fat chernical

analysis results. The third column shows where the k-means clustered this sample based on

Haralick's features. The fourth column shows where the k-means clustered this sample based on

Laws' features. The fifth coliimns shows the results of fusion of both k-means classifiers using

the GBF approach. If a given sarnple did not have a unique maximum posterior probability (

For example two samples had the same probability), the sample was counted as undecided and

represented in the fifth column by a question mark (?).

These three tables show the results of Experiment (HZ~-~), Experiment LE^-) and

the result of Experiment (E3I$ZF). The confusion matrix of the GBF approach is shown in

Table 5.2,

The accuracy of the GBF is 57%, note that we do not count the undecided samples when

we divide by the total number of samples. About 5% of the 59 samples were not classified.

1 1 I 1 FllO 1 Fat 1 1 H; 1 h l 1 Fat 1

Sample name 1 Fat cl- Fi0 1 Fat 1

t FI1 F12 F13 F14 FI5 F16 F17 FI8 FI9

1 1 I 1 Flll 1 Fat 1 1 ~ r l 1 Ld 1 Fat 1

k-means using Laws

LF 2 k-m~iias using Haraiïck

Hf2 Final decision by GBF

Fat 11 1

Fat 1 1 ~ i 3

FI12 FI13 F114 F115 FI16 FI17 Fi18 FI19

~ i 3 Lr3 Lf 2 Lf2 L d Lf3 Lfl Lf2 h l

Fat 1 Fat 1 Fat 1 Fat 1 Fat 1 Fat 1 Fat 1 Fat 1

Fat III Fat III Fat II Fat II Fat 1

? Fat 1 Fat II Fat 1

Hf3 Hf 2 Hf 2 Hf 1 Hf 1 Hf 1 Hf 2 Hf 1

Fat 1 Fat 1 Fat 1 Fat 1 Fat I Fat 1 Fat 1 Fat 1

k-means using Laws

Lf 3 Lf 2

k-means using Haralick

Hf2 Smple name

5'20

Hf1 Hf 3 Hf 3 Hf 2 Hf3 Hf 3 Hf 1 Hf 2

Final decision by GBF Fat II Fat II

Fat class Fat II

F23 Lf2 1 Fat II

Fat II 1 Hf3 1 Lf 3 Fat III

F21 F22

Lf 1 Lf3 Lf 3 Lf 2 L F ~ L d Lf 1 Lf3

Fat II 1 Hf 2 Fat II 1 Hf 3

Fat 1 Fat III Fat III Fat II Fat III Fat 1 Fat 1 Fat II

Fat II Fat II Fat III Fat II Fat II Fat II Fat III Fat II Fat II Fat III

? Fat III Fat II Fat III Fat 1 Fat II

F24 1 Fat II 1 Hf2 Lf 2 F25 F26 F27 F28 F29

Fat II 1 Hf 2 1 Lf2 Fat II 1 Hf3 I Lf 3 Fat II 1 Hf2 Lf 2 Fat II 1 Hf 2 1 Lf 2 Fat II 1 Hf 2 Lf 2

Hf 3 1 LP 3 Hf 2 Lf 2 Hf2 1 Lf 3 Hf 3 Lf 3

F210 F211 F212 F213

Fat II Fat II Fat II Fat II

Hf 3 1 Lf3 Hf 3 1 LE 3

F214 t Fat II F215 Fat II F216 F217 F218 F219

Fat II 1 Hf 2 Lf 2 Fat II Fat II Fat II ,

Hf 3 1 Lf 3 Hf 3 Lfl Hf2 1 Lf2

Table 5.1: The results of Experiments ( ~ 2 ; ~ ~ ), (L~z-) and Ekperiment ( H L F ~ ~ ) ; fusion of two k-means classifiers using the GBF approach to estimate fat.

1 1 The reszllts of the GBF approach 1 1 The

I

1 1 FA I 1 Fat II-[ Fat III 1 undecided 1

k-means using Haralick Hf3 Hf3 Hf3 Hf3 Hf 3 Hf 2 Hf1 Hf2 Hf3 Hi2 Hf 3 Hf 1 Hf 2 Hf 3 Hf 2 Hf2 Hf 3 Hf 3 Hf 2

Sample name F30 F31 F32 F33 F34 F35 F36 F37 F38 F39

F310 F311 F312 F313 F314 F315 F316 F317 F318

k-means using Laws 1 Final decision by GBF Fat class Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III

Lf3 Lf3 Lf3 Lf3 Lf3 Lf 2 Lfl Lf 2 Lf3 Ef2 Lf3 Li3 Lf2 Lf3 Lf2 Lf2 Lf3 Lf3 Lf 2

Table 5.2: The confusion rnatrix of Experiment fusion of two k-means classifiers to estimate fat.

Fat III Fat III Fat III Fat III Fat III Fat II Fat 1 Fat II Fat III Fat Il Fat III

? Fat II Fat III Fat II Fat II Fat III Fat III Fat 11

actuab classes

Fat1 Fat II Fat III

8 1 1

6 12 7

5 6 10

1 1 1

5.2.2 Experiment (WHLgzF) : Fusion of k-means Classifiers Using Whitened

Features to Estimate Fat

This experiment had the same setup as Experiment (WLgzF) except that the features were

whitened. Experiment WH^^-) (in page 53), and Experùnent ('WLEaM) (in page 70 ),

were fused with the GBF approach. A block diagram of the experiment is shown in Figure 5.5.

Computation of Image Whitening

the1 4-Haralick's tranformation

features H k lmage

Computation of the1 4-Law's

features H Whitening tranformation t

k-means clustering

W making using the

k-means clustering

Figure 5.5: Block diagram of Experiment ( w L I L ~ ~ ) : fusion of two k-means classifiers by the GBF approach. The classifiers use whitened features.

Results:

The results of the fusion are shown in Table 5.3 in details. The confusion m a t h of this

experiment is shown in Table 5.4. The accuracy of the GBF is,

6.7% of the samples were not classified.

I I 1 1

F13 1 Fat 1 1 1 Lf 1 1 Fat 1

Final decision by GBF Fat 1 Fat 1

l

Fat 1

Sarnple name FI0 FI1 F12

F14 FI5 F16 F17 F18 FI9 FllO

, F l l l ,

Fat class Fat 1 Fat 1 Fat 1

Fat 1 Fat 1 Fat I Fat I Fat 1 Fat 1 Fat I Fat 1

I

FI15 FI16 FI17 FI18 FI19

k-means using Haralick 1 k-means using Laws

Hf 1 ~ f 3 ? F112 FI13 FI14

Hf1

Hf1 Hf 1 Hf 1 Hf 1 Hf 2 Hf 1 Hf 1

. Hf 1 Fat 1 Fat I Fat 1 Fat 1 Fat I Fat 1 Fat 1 Fat 1

Final decision by GBF Fat II

? Fat II Fat II Fat 1 Fat II Fat II Fat II Fat II Fat III Fat II Fat II Fat 1 Fat II Fat III Fat Il Fat II Fat II Fat II Fat II

Lf 1

Fat 1 Fat 1

Hf 2 Hf1

k-means uçing Laws

Lf 2 Lf 1 Lf 2 Lf 2 Lf 1 Lf 2 Lf 2 Lf 2 Lf2 Lf 2 Lf3 Lf2 Lf 1 Lf3 Lf2 Ef 2 Lf 2 Lf 3 Lf 3 Lf2

I

Lf 1 Lf 1

Hf 1 Hf 2 Hf 1 Hf 3 H F ~

k-meam using Haralick

Hf 2 Hf 3 Hf 1 Hf 1 Hf1 Hf 2 Hf2 Hf1 Hf2 Hf3 Hf2 Hf 2 Hf 2 Hf 2 Hf 3 Hf2 Hf2 Hf2 Hf2 Hf 2

Sample name F20 F21 F22 F23 F24 F25 F26

Hf 1

Lf 1 Lf 1 Lf 1 Lf 1 Lf 1 Lf 1 Lf 1 Lf 1

Fat class Fat II Fat II Fat II Fat II Fat II Fat II Fat TI

Lfl

I Fat 1 Fat 1 Fat 1 Fat I Fat 1 Fat I Fat 1 Fat 1

Lf 1 Lf 1 Lf 1 Lf 1 LF 1

Fat 1 Fat 1 Fat 1

? Fat 1

F27 F28 F29 F210 F211 F212 F213 F214 F215 F216 F217 F218 F219

Fat II Fat II Fat II Fat II Fat II Fat II Fat II Fat II Fat II Fat II Fat II Fat II Fat II

1 1 1

F318 1 Fat III 1 Hf 3 1 Lf 3 1 Fat III 1 Table 5.3: The results of Experiments (EX:?-), LE^-) m d (VVHLS~); fusion of k- means classifiers using whitened features to estimate fat.

Sample name F30 F31 F32 F33 F34 F35 F36 F37 F38 F39 F310 F311 F312 F313 F314 F315 F316 F317

k-means using Laws

Lf3 Lf3 Lf3 h 3 Lf3 Lf3 h l Lf2 Lf2

Final decision by GBF Fat III

? Fat II Fat III Fat III Fat III Fat 1

Fat III Fat II

Fat class

Fat III Fat III Fat III Fat III

The results of the GBF approach The Fat 1 Fat II Fat III undecided '

with whitened features to estimate fat.

k-means using Haralick

Hf3 Hf1 Hf 2 Hf3

/ classes

Lf3 Lf2 Lf3 Li2 Lf3 Lf3 Lf3 Lf3

actual

Fat III 1 Hf3

Fat III Fat III Fat III Fat III Fat II Fat III Fat II Fat III

Fat 1 ~ a t II Fat III

Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Fat III Lf 2 I Fat III

18

Hf3 H d Hf3 Hf2 Hf3 Hf3 Hf3 Hf 3 Hf2 Hf 3 Hf 2 Hî 3 Hf 3

O O 2 1

2 15 4

2 / 1 13 11

5.3 Fusion of Classifiers to Assess Marbling Grades

Table 3.11 (page 58) showed the marbling groups that were used to conduct the following

experiments. Two k-means ~Iassifiers, one used whitened Haralick's feat ures and the ot her

used whitened Laws' features, were fused using the GBF approach.

5.3.1 Experiment (HL&-EF): Fiision of k-means Classifiers to Assess Marbling

Grades

Experiment (Hg--) (see page 58) , and Experïment ( ~ 2 ~ ~ ) (see page 72) were fused by

the GBF approach. The results are shown in details in 5.5. The success rate was 64.4%. The

cordusion matrix of the GBF is shown in Table 5.6.

Final decision by GBF AA AA AA AA A AA AA AA A.A A AA AA AA A GA A

k-meam using Laws

Lm2 L m 1 Lm1 Lm2 Lm1 Lm3 Lm2 L m 1 Lm1 Lm1 Lm2 Lm2 L m 1 Lm1 Lm1 Lm1

k-rneans using Haralick Hm2 Hm1 Hm1 Hm1 Hm3 Hm3 Hm2 Hm1 Hm1 Hm2 Hm2 Hm2 Hm1 Hm3 Hm1 Hm3

Sample name ml0 ml1 m12 ml3 ml4 ml5 ml6 ml7 ml8 ml9 ml 10 ml11 ml 12 ml13 ml14 ml15

Marbhg dass A A A A A A A A A A A A A A A A

I m22 I A A

Sample name m20 m21

Marbling class A A AA

k-meam usiag Haralick Hm2 Hm3 Hm3 H-3 L - 3 AA

k-means using Laws

L m 2 L m 3 L m 3

m228 111229 m230 m231 111232 m233 m234 m235

Final decision by GBF AA AA AA

---

Hm1 Hm2 Hm1 Hm3 Hm2 Hm2 Hm2 Hm1

r

AA i\A AA AA AA AA AA AA

. -

L m 1 L m 2 L m 1 L m 1

AA AA AA A

L m 2 1 AA L m 2 1 AA L m 2 1 AA L m 1 ( A A

Sample name m30 m31 m32 m33

- -

Marbling dass f k-means Ging Haralick 1 k-means ming Laws 1 Final decision by GBF AAA - I H i 3 l Lm3

-- - --- AAA 1 H-2 1 Lm2

AAA AAA AAA AAA

Table 5.5: The results of Ekperiment ( H L & ~ ~ ) ; fusion of k-means classifiers to assess macbling.

1 1 The results of the GBF appmoch 1

Hm1 Hm3 Hm2 H A

1 Actual 1 1

1 A 1 AA 1 AL i Undecided

Lm1 Lm1 Lm2 Lm 1

1 classes 1 A I 1 1 I

1 4 112 1 0 1 O

Table 5 -6: The confusion matrix of Experiment (HL%",*) ; fusion of k-means classifiers to assess the marbling grades.

5.3.2 Experiment (VVHL&ZF) : Fusion of k-means Classifiers Using Whitened

Features to Assess Marbling Grades

Experiments (WHg2"9) (see page 59) and (7NLgaLY) (see page 72). The resdts are shown

in details in Table 5.7. The confusion matrix is in Table 5.8. The success rate was 86.40%.

name , ml0 ' ml1

ml2 ml3 ml4 ml5

ml13 ml14 ml15

Marbling class A A A A A A

A A A

k-means using Haralick HmJ- Hm1 Hm1 Hm1 H m 1 Wrn1

m16 1 A Hm1 HCl Hm1 Hm1

ml7 ml8 ml9

Hm1 Hm3 Hm1

k-means using Laws

Lm1 L 1 L m 1 Lm1 L m 1 Lm1

A A A

Final decision by GBF A A A A A A

Lm1 L m 1

Lm1

Lm3 Lm3 Lm1

A A - AA A

A AAA A

Sarnple name m20 m21 m22 m23 m24 m25 m26 m27

Marbling class AA AA AA AA AA AA AA AA

k-means using Haralick

Hm2 H m 2 H m 1 H m 1 H m 2 Hm2 Hm2 Hm2

m28 1 AA H m 2 H m 1 Hm2 H m 1 Hm2 H m 2 Hm2 Hm2 H m 2 Hm2 Hm2 Hm2 H m 2 H m 1 Hm2 H m 2 Hm2 H m 3 Hm2 Hm2 H m 3 Hm2 Hm2 H m 3 Hm2 H m 3 Hm2 H m 2

m29 m210 m211 m212

k-means using Laws

Lm2 Lm3 Lm2 Lm2 L m 2 Lm2 L m 2 Lm1

AA AA AA AA

Final decision by GBF AA AA AA AA AA AA AA AA

L m 2 L m 2 L m 1 L m 1 L m 2 Lm2 L m 2 L m 3 L m 3 L m 2 Lm2 Lm2 L m 3 L m 2 Lm3 Lm3 L m 2 L m 2 Lm3 Lm1 Lm2 L m 3 Lm1 Lm3 Lm1 L m 2 Lm2 Lm2

AA AA AA A AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA AAA AA AA AA AA

m213 1 AA m214 m215 111216 m217 m218 m219 m220 m221 m222

A A AA AA AA AA AA AA AA AA

m223 m224 m225 m226 m227 m228 m229 m230 m231 m232 m233 m234 m235

AA AA AA A A AA AA AA AA AA AA AA AA AA

1 Sample lename 1 Marblinn class 1 k-means using Haralidc 1 k-means using Laws 1 Final decision by GBF 1 m30 m31 m32

Table 5.7: The resdts of Experiement (WHLgEF); fusion of k-means classifier to assess mar-

m35 m36

bling grades. The classifiers used whitened features.

- AAA AAA AAA

I - - --

1 The results of the GBF appmach 1

-

AAA 1 Hm3 AAA 1 Hm3

1 Actual 1 I

1 A 1 AA 1 AAA -1 Undecided 1

-

Hm3 Hm3 H-2

Lm3 Lm3

Table 5.8: The confusion rnatrix of Experiment (WHLgzF) ; fusion of two k-means classifiers

- Lm3 Lm2 L-3

hL4 AAA

classes

to assess the marbling grades. The classifiers used whitened features.

M A AA AA

1 1 5

3 34 2

A AA AAA

O O O

12 1 O

5.4 Conclusion

This chapter discusses the GBF technique to improve the grading success rate by fusing two

different k-mean classifiers one uses Haralick's features and the other uses Laws' features. The

results were improved significantly even without the whitening process. Fusion of s u p e ~ s e d

classifiers (i.e., MED classifiers) was also carried out and the results are listed in the sumrnary

table of the results in the next chapter.

Chapter 6

Discussion and Suggestions

6.1 Results Discussion

Table 6.1 and Table 6.2 list all the experiments results. Some experiments were carried out

but not listed through the previous chapters, however, their results are shown below. Two

different sets of experiments were implemented, one set uses the fat content classes and the

other set uses the marbling grade classes. Since marbling grade and fat contents show as a

difference in texture, two different texture techniques were used to extract texture features,

namely, Haralick and Laws so as to extract as much texture information as possible . Also two

different algorithms were used to classify the texture, supervised which was the MED classifier

and unsupervised which was the k-means algorithm. The MED classifiers were trained with

labeled samples. The k-means classifier used the visual features (Haralick or Laws) to cluster

data into three groups. The clusters were mapped into equivalent fat or marbhg classes based

on the majority of samples in each cluster. For examples, a k-means classifier produced three

clusters. The number of sarnples based on the marbling grades were counted in each cluster.

If a cluster had the majority of samples from class AAA , it would have been mapped to that

class. The same also applies to other clusters.

Two assumptions were made about the data'. First, it had a Gaussian normal distribution.

The seconed assumption was that the characteristics of the data fairly represents the charac-

'Data here refers to: ultrasouad images, the results of fat analysis and the marbling grade.

teristics of the actual population. k-means classifier was more robust than MED classifier to

the normality assumption which might explains why k-means produced better results than the

MED classifier.

The data used to carry out the experiments was s m d and its quality was bad. The ul-

trasound machine used to capture the images had 64 graylevels and a resolution of 618 x 480

pixels. Recent machines can produce Mages with 256 graylevels and a resolution of 512 x 512

pixels or more. A visual cornparison between the Mages used in this research and other im-

ages seen in other research literature showed a big clifference in terms of the general quality of

our images. For examples, there was a big clifFerence between ultrasound images of difFerent

marbling grades and fat contents in terms of visual quality. The texture of low fat images

was coarser than the texture of high fat images wbich is not evident in the images used in

this research. Collecting new data was not an easy job to do as it involves many tasks such

as Maging beeh by ultrasound machine, slaughtering them within short time period to assess

their marbling grades and estimate their fat contents. The marbling grades must be assessed

by an animal expert and the fat contents should be estimated by a chexnical analysis of meat

samples. The marbling grades, as mentioned before, are prone to an error as high as 20% and

the chernical analysis results are largely depends on the way the meat samples are selected and

analyzed. There is huge amount of data posted in the Internet, but unfortunately no similar

data was found that can replace the current data. Moreover, the images were taken Tom live

mimals which move during image acquisition. The movement causes slight blur ctnd texture

distortion in ultrasound images. Movement blur causes fine texture to look coarser. Several

image processing techniques were tried out to reduce the blur but did not produce satisfactory

results.

The classifiers that produced the best results were fused by the GBF approach and the

results are show in Table 6.3, Table 6.4, Table 6.5, and Table 6.6. Each table shows the types

of the fused classifiers and the fusion results.

1 Fat estimation 1 1 Expriement / Classifier 1 Classifier 1 Type of 1 Festure 1 Whitening 1 Accuracy 1 Random 1 Notices 1

Table 6.1: Results based on the chernicd fat analysis data

label

HF Heteans

1 Mar biing assessrnent 1

--EF- _ _ -

super. MED Haralick no Y- 40.678% 1 no P- #52 unsuper. k-means Haralick

MED Haralick yes(l/l4) no 62.01% P. $55

type super.

unsuper.

L - J

-

l super. 1 MED 1 Laws 1 no Y- 1 61.01% 1 no *

name MED

k-means

- -

HLrnSas . w%EG8119

I super. 1 MED 1 Laws 1 yes(5/15) 1 no 1 46.44% 1 YeS 1

Notices Experirnent label

features Haralick Haralick

super. super.

unsuper. unsuper.

super.

Table 6.2: Results based on the animal marbling grade. The (*) indicates a nearly singular covariance matrix.

Classifier

type

super. unsuper. unsuper.

selection no

no

MED MED

k-means k-means

Classifier name

process no no

Haralick Haralick Haralick

MED 1 Laws yes(15/15) no no

k-means k-means

1 samples

52.54% 1 no 49.15% 1 no

Random samples

Type of features

Laws Laws

8.47% no

yes(l/l4) 1 no

MED 1 Laws

Y- no

Y-

P. $47 P- #48

Feature selection

no

yes(4/14) no

no

1

P. #58

60.68% 1 Y- Y e s no

Y= Earalick

63.73% 44.06% 64.40%

Whitening process

P. #59

71.86% 37.20% 77.96% no

Accuracy

Y= no

no

Y= no no

P- #72 P- #72

Fat estimation Classifier 1 Classifier 1 Featwes 1 Accuracy 1 Notices

Table 6.3: Fusion of the best supervised classifiers to estimate fat contents.

Type supervised

Marbling assessment Classifier 1 Classifier 1 Features 1 Accuracy 1 Notices

Table 6.4: Fusion of the best supervised classifies to assess the marbling grades.

name name

Type s u p e ~ s e d supewised

Fusion

Fat estimation Classifier 1 Classifier 1 Features 1 Accuracy 1 Notices

MED supeMsed 1 MED

64.24% Haralick Lam

p. #55 54.19% 65.92%

name MED MXD GBF

Table 6.5: Fusion of the best unsupe~sed classifier to estimate fat contents.

Fusion 1 GBF I - p. #71

Type unsupe~sed unsupervised

Fusion

Marbling assessment Classifier 1 Classifier 1 Features 1 Accuracy 1 Notices

name Haralick

Laws -

name k-means k-means

GBF

Table 6.6: Fusion of the best unsupervised classifiers to assess the marbling grades.

71.86% 63.73% 72.15%

Type supervised s u p e ~ s e d

Fusion

p. #99 p. #99

m e Haralick

Laws -

name k-means k-means

GBF

69.49% 76.27% 83.60%

p. #53 p. +?O p. #88

name Haralick

Lam -

77.96% 64.40%

p. #59 p. #72

86.40% 1 p. #94

Despite the fad that the ultrasound images used to conduct this experiments were having

low qualiw, the results are promising for using dtrasound images to assess rnarbling grades and

estimate fat content of live beef. Using a statisticd pattern recognition techniques accompanied

by a careful selection of texture feature sets were one reason for such good results. This research

does not claim to be the f h t one that used the statistical pattern recognition approach to grade

meat. Actudy , the main contribution wbich is weil known to statistical pattern recognition

commtinity, and it was not considered in other researches in the assessrnent of meat marbling

was the following:

1. careful selection of the feature sets, oamely, the Haralick's CO-occurrecne texture features,

and Laws' rnasks features,

2. an optimum technique for the classifiers performance evaluation, namely, the leave-one-

out method,

3. the whitening transformation process that removed the correlation and independency

between texture features,

4. ultrasound images were enhanced in contrast but were not atered. Filtering the images

was found by some experimental procedure is not helpful. In fact removal of speckle noise

degraded the classification results which indicates a relationship between speckle noise

and marbling grades. Brethour (241 and Y. Lui (21j showed that animal marbling and

s p e d e noise present in ultrasound images are correlated,

5. fbaJly, the proposed Generalized Bayesian Fusion approach (GBF) improved the resdts

significantly by combining different classifiers using the Bayes' theorem.

For practical applications, the accuracies values given in Table 6.1 and Table (6.2) shodd be

reexamined, unfortunately, because the data sample used to design the classification algorithms

was small and the classifiers accuracies are a h c t i o n of the data sample size. Typicdy the data

sample should be larger than the number of texture features, otherwise cornmon problem arises

which is hown as the curse of d2mensionality. Whenever the data set is small compared to the

number of features, the probability distribution function of the data will be under-estimated

or over-estimated. In this pro ject, the number of extracted features was 14 Haralick's and 15

L a d features which means that the data sample must be be much larger thao 60 images'.

To partially solve the problem, an artificial data was added to the data set and a procedure to

select the best features was ernployed. The leave-one out method was selected for evaluating

the classifiers performance because it uses the data sampIe more dectively at the expense of

the amount of computation needed for evaluating the classifiers performance. In leave-one out

method, ad the data except one sample is used to design the classifier. The sample that is not

used in the training process is used to test the classifier. This sample is inserted again, and

same process is repeated for al1 other samples.

From Table 6.1 and Table 6.2 we note that the Haralick's features, most of the time, gave

better results than Laws' features because the texture present in ultrasound images was mainly

a stochastic texture. This result confirmed the conclusions of the studies in [59] and [65]

which showed that Haralick's texture features outperform other texture features for identifjkng

stochastic texture. Haralick's features comput ation were based on the Co-occurrence matrix

which is an approximation of the second joint probability distribution of graylevels. Natur&y3

stochastic texture

The whitening

needs a statistical and probabilistic approach to characterize it.

process improved the results as it removed the correlation and the indepen-

dency between the texture features.

The result of fusion of two classifier was clearly always better than the results of individual

classifiers. It is an outcorne of the Bayesian approach since it ensures that the posterior proba-

bility is always equal or greater than the priori probability- That aIso means if we increase the

number of hsed classifiers the classification accuracy improves or at least we are certain that

it does not degrade..

6.2 Future Developments

For further development of the research, three different things could be investigated: usùig

different texture feature sets, fine tune the individual classifiers and fusion of more than two

classifiers.

Other than Haralick's and Laws' feature sets, there are many texture features that c m be

' ~ o m e authors claimed that the data should be at least 10 times the number of features.

103

used. For example, the Run Length Statistics which first proposed by GaUoway [661 proved

to be useful for stochastic texture. The basic idea of Gdoway is to extract texture properties

based on the lengths of runs of gray levels. A run of gray level is a consecutive pixels of the

same gray level in certain direction. For digital image the runs for all gray level are counted

and arranged in two dimensional matrix based on the length of the runs, such matrix is known

as the Run Length Matrix RLM. Elements of the FUM , p(i, j) , represent the number of

runs of length j and graylevel i. From p(i, j ) some texture features can be extracted such

as: short run emphasis (SRE), long run ernphasis (LRE), graylevel nonuniformity (GLN),

run length nonuniformity (RLN) and run percentage (RP). Two other features were added by

Chu [67] to complement the above feature namely, the low graylevel r u emphasis (LGRE)

and the high gray level run ernphasis (HGRE). Another useful technique for texture extraction

is the graylevel texture Merence method (GLDM) [68], [65J. By adding new feature sets,

useful information horn ultrasound image is extracted that might improve the accuracy of the

classification algoritthms.

The second area of improvement is to fine tune the individual classifiers. The potential one is

the k-means classifier. In k-means clustering the data sarnples are clustered into hyperspherical

clusters by optimizing certain objective function. The objective function used in this research

was the Euclidean distance between samples of the same class. Another objective function may

be developed so as to do two things simultaneously: minimize the Euclidean distance between

data points in the same cluster and maximize the Euclidean distance between cluster centers.

We may consider the k-means clustering as a discrete combinatorial optimization problem. For

any combinatorial problem that has c classes and n data samples, there are a very large number

of different ways to cluster the data. For such problem the caniinality is [69],

where

time, and

the expression ( ) is the number of different combinations of c abjects taken i ai a

defined as:

This research clustered 59 data samples into 3 clusters. By using Equation (6.1), the number

of possible clustering combinations are:

n o m this huge number of different possible dusters there is just only one correct solution.

The purpose of the objective function is to guide the algoritha to the correct solution. To

find the global solution and not to trap in a local solution the objective function must be fine

tuned and used in an algorithmic way. By using a suitable objective function and a guiding

algorithm, we ensure to get a suboptimal global solution. The Simulated Annealing algorithm

may be used to drive the k-means algorithm to search for the gIobal solution. The Simulated

A ~ e a l i n g algorithm can tuned by changhg certain parameters until a satisfactory solution

found. Moreover, the k-means algorithm can be replaced by lkzq k-means algorithm that

uses concepts £rom b y logic theory to cluster data 170, 691. The one that was used in this

research was the Hard k-means aigorithm that classified the data in crisp sense which means

that each data point was assigned to one, and only one, data cluster. In fuzzy clustering each

data point can have a partial membership to more than one class. The Euclidean distance

between the data points and the clusters prototypes c m be used with ceratin modification

as a membership function. To use Euclidean distance as a membership value, it should be

normdized and modified such that the rnembership function of a given data point should be

between O and 1.

The third suggested improvement is the fusion of more than two classifiers which improves

the result and at worst it will not degrades the resdt3.

Another apparoach other than using pattern recognition classifiers to estimate fat content

can be used. The extracted Haralick and Laws features can be used to build a multivaraite

linear regession model to estimate fat contents. For an accurate model, a complex nonlinear

model can be built by using a spline or interpolation procedure or applying some cuve fitting

techniques.

3 ~ a y e s ' theorem ensures that postenor-probabilities always equal or larger than prior-probabilities.

105

Bibliography

[l] Douglass Cross, H.R. et ai. An evaluation of the accuracy and uniformity of the USDA

beef quality and yield grading system. Technical Report D-M- 1980, Rep. to the Office of

Inspèctor General, USDA, 1980.

[2) Brian R. Thane and A. Dale Whîttaker. Automated assessrnent of marbling in ultrasonic

images of beef animals. In T;he Amencan Society of Agricdtural Engineers ASAE, n u b e r

90-7055, pages 24-27, Hyatt Regency Columbus,Columbus, Ohio, Jun. 1990.

[3] S.D.M. Jones, A.K.W. Tong, and S. Talbot. A survey of marbling fat in canadian beef

carcasses. Canadian Journal of Animal Science, 71:987-991, December 1991.

[4] Ralph B. Brown, J. W. Wilton, M.Leunissen, and 1. Mandeil. UZtrasound image enhacement

techniques for estimation of meat quality and yield in beef cattle. A ropert to Ontario Min-

istry of Agriculture, Food and Rural Affairs, Ontario Food Processing Research Program

Project FP4012, University of Guelph, April 1996.

[5I R.D. Bergen, J.J. McKinnoa, D.A. Christensen, and N. Kohle. Prediction of lean yield in

yearling bulls using red-time ultrasound. Canadian Journal of Animal Science, 76:305-311,

1996.

[6] Maxell G . Maginness. Methods and terminology for diagnostic ult rasound imaging systems.

Proceedings of the IEEE, 67(4) :641-653, April 1979.

[7] Takeo Sawatari Patrick N. Keating and Gene Zilinskas. Signal processing in acoustic

imaging. Proceeding iEEE, 67(4):496-510, April 1979.

[8] Gordon S . Kino. Acoustic imaging for nondestructive evaluation. Proceedzngs IEEE,

67(4):510-525, April 1979.

[9] James F. Havlice and Jon C. Taenzer. Medical ultrasonic imaging: An o v e ~ e w of pnnci-

ples and instrumentation. Proceedings IEEE, 67(4) :62&64O, April 1979.

(101 K. Kirk Shung. General e n g i n e e ~ g principles in diagnostic ultrasound. IEEE Engineering

in Medicine and Bzology Magazine, pages 7-13, Dec. 1987.

[ I l ] Jonthan Ophir and Nabil F. Maklad. Digital scan converters in diagnostic ultrasound

imaging. Pmceedings of IEEE, 67(4) :654-664, ApnI 1979.

[12] David H . Dameron. An inhomogeneous media mode1 for the deterrniniation of acoustic

parameters in tissues. IEEE %m. on Sonics and U'ltrasonzcs, 8 U-27(5) :244-248, Sep.

1980.

[13] Gerard E. Sleefe and Padmakar P. Lele. Tissue characterization based on scatterer number

density estimation. iEEE Tkans. on Ultrasonics, Ferroelectn'cs, and Fmquency Control,

35(6) :749-757, Nov. 1988.

[14] Udantha R. Abeyrante. On modeling the tissue response kom ultrasonic B-scan images.

IBEE Dans. on Medical Imagzng, 15 (4 ) :479-490, Aug. 1996.

(151 Luigi Landini and Lucio Verrazzani. Spectral characterization of tissues microstructure

by dtrasound: A stochastic approach. IEEE îhn . o n VZtrasonics, Ferroelectrics, and

Requency Contml, 37(5):448-456, Sep. 1990.

(161 Christoph B. Burchardt. Speckle in ultrasound %-mode scans. IEEE î+anr.on Sonics and

(ntrasonics, SU-25(1):1-6, January 1978.

(1 71 P.N.T. Wells and M. Halliwell. Speckle in ultrasonic imaging. Lrltrasonics, pages 225-229,

September 1981.

[18] Robert F. Wagner et al. Statistics of speckle in ultrasound Bscans. IEEE Thzs. on Sonics

and Ultrasonics, 3O(3) : 156-163, May 1983.

[19] V. Manoj Narayanan et al. Non-Rayleigh statistics of ultrasound backscatterd signais.

IEEE î h n s . on Ultnlsonics, Femelectrics, and h q u e n c y Control, 41 ( 6 ) :845-852, XOV.

1994.

[20] Akira Ishimaru. Theory and applications of wave propagation and scattering in random

media. Pmceedings of the IEEE, 66 (7) :lO3O-I06l, Jul. 1977.

[21] Y. Liu, D. J. Aneshansley, and J.R. Stouffer. Autocorrelation of ultrasound speckle and its

relationship to beef marbling. ASAE, 36(3):971-977, May-June 1993.

[22] Ferderic L. Lizzi et al. Relationship of ultrasonic spectral parameters to features of tis-

sue microstructure. IEEE trans. ultrasonic, femelectrics, and frequency control: UFFC-

33(3):319-329, May 1986.

[23] Roman Kuc. Ultrasonic tissue characterization using kurtosis. IEEE trans. ultrasonics,

femoelectTics, and frequency control, UFFG33(3) :273-279, May 1986.

[24] J.R. Brethour. Relationship of ultrasound speckle to marbling score in cattle. Journal of

Animal Science, 68:2603-2613, 1990.

[25] Kyouhei Qzutsumi, Tosihrio Nade, et al. Non-destructive, ult rasonic evaluation of meat

quality in live japanese black steers £rom colored images produced by a new ultrasound

scanner. Meut Science, 43(1) :6 1-69, 1996.

[26] B. Park, A.D. Whittaker, et al. Predicting intramuscular fat in beef Iongissimus muscle

from speed of sound. Journal of Animal Science, 72:lO9--116, 1994.

[27] B. Park, A.D. Whittaker, et al. Measuring intramuscular fat in beef with ultrasonic fre-

quency analysis. Journal of Animal Science, 72:ll?-l25, 1994.

1281 T.L. Perkins et al. Evaluation of ultraçonic estimates of carcass fat thickness and the

longissirnus muscle area in beef cattle. Journal of Animal Science, 70:1002--1010, 1992.

[29! S.D. Jones et al. The prediction of beef carcass lean content by an electronic probe, a

visual scoring system and carcass measurments. Canadzan Journal of Animal Science.

69:641-648, September 1989.

[30] M.T. Smith et al. Evaluation of ultrasound for prediction of carcass fat thickness and

longissimus muscle area in feedlot steers. Journal of An2mal Science, 70:29-37, 1992.

(311 Russell Muzzolini et al. Texture characterization using robust statistics. Pattern Rewgni-

tion, 27(l):llg-l34, 1994.

(321 Russell Muzzolini et al. Multiresolution texture segmentation with application to diagnostic

ultrasound images. IEEE Dans. Medicul Imdging, 12(1): 108-123, March 1993.

(331 B. Park A. D . Whittaker et al. Principles of ultrasound and measurements of intramuscular

fat. J. Anim. Sci., 70:942-952, 1992.

[34] Jea S.Lim and Hamid Nawab. Techniques for speckle noise removal. SPIE, 243:3544, July

1980.

[35] Anil K. Jain and C.R. Christensen. Digital processing of images in speckle noise. SPIE,

243:45-50, 3uly 1980.

[36] Robert L. Gdloway et al. A hequency diversity process for speckle reduction in real-tirne

ultrasonic images. IEEE transactions on Ultrasonics, Ferroelectrics, and Pequency control:

35(1):4549, Jan. 1988.

[37] Adrian N. Evans and Mark S. Nixon. Biased motion-adaptive temporal filtering for speckle

reduction in ediocardiography. IEEE tram. on Medical Irnaging, 15 ( l):39-50, Feb. 1996.

[38] J.D. McCauley et al. Fat estimation in beef ultrasound images using texture and adaptive

logic networks. 7kansaction of ASAE, 37(3):997-1002, 1994.

[39] Robert M. Haralick et al. Textural features for image classification. IEEE Pans. Sys.

Man. Cybem. , SMC-3:610-621, November 1973.

[40] Kevin Woods. Combination of multiple classifiers using local accuracy estimates. IEEE

Dans. on Pattern Analysis and Machine Intelligence, 19(4):405-410, April 1997.

(411 L. Lam and C.Y. Suen. Methods of combining multiple classifien and their applications

to handwriting recognition. pages 418-420, Jerusalem, 1994.

[42] Lei Xu and Adam Krzyzak. Methods of combinig classifiers and their applications to

handwriting recognition. IEEE %ns. on Systems Man and Cybernetics, 22(3) :418-435,

May-June 1992.

[43] Y.S. Huang and C.Y. Suen. A method of combinig multiple experts for the recognition

of unconstrained numerals. IEEE Buns.on Pattern Analysis and Machine Intelligence,

17(1):90-94, January 1995.

[44] F. Kimura and M. Shridhar. Handwritten numerical recognition based on multiple algo-

rithms. Pattern Recognition, 24(10):969-983, 1991.

[45] Tin Kam Ho et al. Decision combination in multiple classifier systems. lEEE Tkans.on

Pattern Analysis and Machine Intelligence, 16(1):66-75, January 1994.

[46] Ani1 K. Jain. hrtdnmentals of Digital Image Processing. Prentice Hall Information and

System Science Series. Prentice Hail, Englewood Cliffs, NJ 07632, first edition, 1989.

[47] B. Julesz e t al. Inability of humans to discriminate between visual texture that a agree in

second-order statistics- revisited, Preception, 2:391-405, 1973.

[481 Nam-Deuk Kim et al. Texture anal~sis using multiresolution analysis for ultrasound tissue

characterization. In 2yd Annual Reuiew of Pmgress in Quantitative NDE Conference.

Bowdoin College, Brunswick, Maine, July 28- Aug. 2 1996.

f49j Tissae Characterization for Beef Grading Uszng Teztwe Analysis of U2trasonic Images.

Proceedings of the 1993 IEEE ultrasonics symposium, 1993.

[50] J. Kittler. Featum Selection and Extraction. Handbook of Pattern Recognition. Academic

Press, 1986.

[51] E. Bruce Goldstein. Sensation and Perception. Wadsworth Publishing Company, Belmont,

Calfornia, 1989.

[52] Robert Schalkoff. Pattern Recognition: Statistical, Structural and Neural Approaches. John

Wiley & Sons, Inc., 1992.

1531 Naikuan Huang. M d o v Probabilistic F2mction Models for Texture-Based Image Segrnen-

tation. PhD thesis, Facdty of the Graduate School, University of Southern California,

Dec. 1983.

[54] G. R. Cross and A. K. Jain. Markov random field texture models. IEEE i"ramactions

Pattern Analysis and Machine intelligence, PAMI-3(1):25-39, Jan. 1983.

B. Julesz. Visuai pattern discrimination. IRE Dans. on Information Theoqj, 8(2):84-92,

1962.

Robert M. Haralick. S tatistical image texture analysis. In Handbook of Pattern Recognition

and Image Processing, pages 247-279. Academic Press, Inc., 1986.

Robert IV. Haralick. Statistical and structural approaches to texture. Proceedings of the

IEEE, 67(5):786-804, May 1979.

Keinosuke Fukunaga. Introduction to Statistical Pattern Recognition. Computer Science

and Scientific Computing. Academic Press, second edition, 1990.

3.M.H. Du Buf et al. Texture feature performance for image segmentation. Pattern Recog-

nition, 23(3/4) :29l-309, 1990.

K . Laws. Teztured Image Segmentation. PhD dissertation, University of Southern Califor-

nia, January 1980.

M.R. Chernick et al. Application of bootstrap and other resampiing techniques: Evaluation

of classifier performance. Pattern Recognition Letters, 3: 167-178, 1985.

Jean Vincent Moreau and Ani1 K. Jain. The bootstrap approach to dustering. Pattern

recognition handbook. pages 62-71, 1986.

Vijay K. Rohatgi. Statistical Inference. Number 7. John Wiley & Sons: 1986.

S. James Press. Bayesian Statistics: Principles, Models, and Applzcations. John Wiley &

Sons, 1988.

Richard W. Conners e t al. A theoretical cornparison of texture algorithms. IEEE Pans-

actions o n Pattern Analysis and Machine Intelligence, PAMI-S(3) :2O4-222, May 1980.

111

(661 M.M. Galloway. Texture andysis using graylevel nui lengths. Cornputer Graphics and

image Processing, (4 ) : 172-1 79, 1975.

[67] A. Chu et al. Use of gray value distribution of run lengths for texture analysis. Pattern

Recognition Letters, 11:415-420, June 1990.

[68] C. R. Dyer J. S. Weszka and A. Rosenfeld. A comparative study of texture measures for

terrain classification. IEEE Tram. Syst., Man, Cybem., SMG6:269-285, Apr. 1976.

[69] Timothy J. Ross. Fuzzy Logic with Engineering Applications. McGraw-Hill, Inc., 1995.

[701 Abraham Kandel. F u z y Techniques in Pattern Recognition John Wiley & Sons, 1982.

[71] Robert M . Hardick and Linda G. Shapiro. Glossay of computer vision terms. Pattern

Recognition, 24(1) :69-93, M a c h 1991.

[72) Peter J. Burt and Edward K. Adelson. The Laplacian pyramid as a compact image code.

IEEE Trans. on Communications, COM-31(4) :XK?-54O, April 1983.

[73] Donoho DL. Wavelet shrinkage and w.v.d: A 10-minute tour. In Y. Meyer and S. Roques,

editors, Progress in Wavelet Analyszs and Applications. Editions Frontieres, 1993.

[74] Johnstone I Donoho D. M i n i m a Estimation Via Wavelet Shrinkage. Dep. of Statistics,

S tanford University, 1992.

[75] John Sammon. A nonlinear mapping for data structure analysis. lEEE Tram. on Corn-

puters, C-18(5):401-409, ~May 1969.

1761 Wojciech Siedlecki e t al. Mapping techniques for exploratory pattern analysis. In B.S.

Gelsema and L.N. Kanal, editors, Pattern Recognition and Artificiai h~tellzgence~ pages

277-299. Elsevier Science Publishers B. V., 3ort h-Holland, 1988.

Appendix

Glossary

The following definitions are used hequently in the research and defined here for easy reference.

The definitions that are marked by (*) are taken fkom [7l].

Pictron'd pattern recognition*: refers to techniques which treats the image as a pattern and

either categorize the image or produce a description of the image.

The unit* : is the entity which is observed and whose measured properties constitute the

measurement pattern. The simplest and most practical unit to observe and mesure in the

pattern recognition of image data is oftea the pixel ( the gray tone intensity or gray tone inten-

sity). This what makes pictond pattern recognition so difEicult, because the objects requiring

analysis or identification are not single pixels but are often comple~ spatial formation of pixels.

The measurement space: data that are obtained by making certain measurement. Ultra-

sound images of anirnals having different marbling score can be considered as a measurement

space.

A pattern: each ultrasound image is a pattern. A set of ultrasound images (patterns) of

different marbling scores constitute the measurement space.

A class: a collection of patterns that have the same property. For example a set of ultrasound

image having the same marbling score. To allow for within class variations, classes are normally

described by a probability distribution function such as the Gaussian normal distribution.

Featuws, feature vectors and feature spaces: using an entire ultrasound image pixels for

classification is impractical and computationally extensive. A set of n data values or n feature

are computed hom. the image pixels by certain functions or algorithrns. The features are selected

so as to contain high amount of information relative to the discrimination between patterns of

different classes. The set of features can be viewed as a vector that has n components, or a

point lies in a space that has n-coordinates. The set of all feature vectors of all the images

constitute the feature space.

A classijier* : is a device or process that sorts patterns into classes.

The whztenzng hansfownatzon: a nonlinear transformation process t hat makes the featwes

uncorrelated and probably scaled. The covariance matrix of the feature space is used to obtain

a transformation matrix.

Sammons mapping algon'thm : a nonlinesr transformation a.lgorithm that maps points in

high dimensional space into two or t h e dimensional space. The Sammons algonthm a p

proximately preserve the Euclidean distance between all the points in both spaces, the high

dimensional space and the reduced one.

Appendix B

Image Filtering

B.l Speckle Noise

A first look at ultrasound image shows the dominance of speckle noise over other image

features. A question might arises regarding the effect of its reduction on the classification

results. Two different techniques for speckle noise filtering have been tried ,nameIy, Spatzal

averagzng of speckle and Wavele t Denozsing algorithm. Ot her fiIt ering techniques can be found

in [34].

Mathematically speaking spedde noise can be considered as an infinite sum of independent

, identical complex phasors with randorn amplitude and phase which has the form [46]:

where aR(x7 y) and ar(z, y) are zero mean, independent Guassian random variables with

variance c,. The intensity field is:

which has a Rayligh distribution with a variance o2 = 22a: and mean = E(s) = a*.

Speckle noise is different fkom other sort of noise that corrupts images in terms of two

aspects:

1- The signal to noise ratio for images corrupted by speckle noise is always lower than

images corrupted by other sort of noise, specifically, Guassian noise.

Z Normal Guassian noise is an additive noise that can be removed by low-pas filtering

the image. On the other hand speckie noise is a multiplicative noise which means it is a signal

dependent one. Removal

especidy because of the

image equal about 1.9 .

A through t heoretical

of speckie noise also removes part of the signal which is undesirable

lower signal to noise ratio. The signai to noise ratio for dtrasound

study of speckle noise can be found in [17], [16] and [18].

B.2 Spatial Averaging of Speckle

By taking the average of several pixels around a window of certain size, for example 3 x 3,

speckle noise c m be reduced significantly [46] and [35]. This operation is equivalent to low-pas

filtering the image by a Guassian kernel [72]. The same process of averaging c m be repeated

many times at the expense of reducing image resolution. 1

Image contrast ratio improves but the resolution also reduces by a factor of - where N2

N x N represent the window size used for averaging. Figure (B.l) shows an ultrasound image

and Figure (B.2) shows the same image filtered by a spatial averaging has a size of 3 x 3 .

B.3 Wavelet Denoising Algorit hm

This algorithm uses wavelet transform for noise removal and signal smoothing [73], [74]. Assume

we observe some unknown function f with Gaussian noise: yi = f (ti) + a.&i , i = 1, . .., n and

~i e N(0, l ) . The goal is to estimate the unknown function f . TO filter the signal yi to obtain

f ( k ) , the signal Yi is first must be transformed by a wavelet transform into scabspace domain.

The wavelet coefficients are thesholded. Then the signal is reconstructed to obtain an estimate

of the signal f (ti). There are many thresholding techniques with different objectives. One

of themis d , = ~ i g n ( d i j ) ( l & ~ l - A ) with A = QJ2709n . This thresholding rule gives an f i

Figure B. 1: Non-filtered, contrast enhanced ultrasound image.

Figure B .2: The image in Figure B-l filtered by an averaging filter of size 3x3.

117

estirnate of the signal f (ti) with a risk within a factor of 2.logn of the minimum risk when

using the optimal thresholding rule.

There are two difficdties for impleïnenting this algorithm. First the signal resolution need

to be a pawer of two so that a fast wrtvelet transformation algorithm can be used. Surprisingly,

fast wavelet transforms are faster than Fast Fourier Transform (FFT). The former has a com-

putational complexity of O(n) while the later has a complexity of O(n.log2(n)). The resolution

of the signal ( e.g. ultrasound image) can be changed t o be a power of two by appending zeros

to ultraSound image horizontally and vertically. The second problem is the estimation of the

thresholding value which is clearly depends on a (the speckle noise variance). The value O must

be k n o m a priori or it might be inferred from the image itself. The same image in Figure B.1

filtered by a wavelet denoising algorithm is shown in Figure B.3 below.

Figure B.3: Ultarsound image in Figure B-1 as filtered by the wavelet denoising algorithm.

8.4 Conclusion

Filtering of ultrasound images to reduce speckle noise increases the computationd burden and

worsen the classification result. This result was confirmed by observïng the separation between

classes before and after f3tering ultrasound images by the same procedure that was done for

selecting an ROI ??. Speckle noise is signal dependent, and any attempt to remove it will

degrade the desired signal. Apparently, speckle noise is the main contributer of the texture

present in ultrasound images. Speclde noise is a result of the interference of sound waves that

are scattered by tissues. Fat and meat tissues have a different scattering properties that reveal

themselves in ultrasound Mages as texture with difFerent statistical properties. Classification

algorithms in this thesis use features that are extracted from texture to categorize images into

different classes. Hence speckle noise reduction has not ben pursued further .

Appendix C

Sammon's Mapping Algorit hm

Since the feature space has high dimensional space it is very desirable to map it to lower di-

mensional space by linear or nonlinear transformation depending in the objective of the trans-

formation. Any data transfonnation that reduce the dimensionality of the data can be very

helpful to better understand the structure of the data. Due to the fact that planar images of

complex data can be perceived and analyzed by a human observer, the mapping methods can

be used as a powerful tool for data malysis. Noniinear mapping techniques is more efficient

than linear mapping techniques but they tend to be more complex to implement.

Due to the complexity of the nonlinear transformation algorit hms t hey are irnplemented

by iterative numerical methods. An efficient method that maps a high dimensional feature

space to two or three dimensional space called the Sa-mmon mapping a lgor i th , see ref. [75].

The main objective of the S m o n algorithm is that the transformation must preserve the

distance between the samples in both space. The Sxmmon rnapping algorithm is based upon a

point mapping of n vectors from a high dimensional h-space to a lower-dimensional 1-space

(typically two or three dimensions) such t hat the inherent data " structure" is approximately

preserved. We mean by the data structure the natural clustering of the data.

Let we have n vectors in a high dimensional space, the h-space. The main objective is to

transform these vectors to a lower dimensional space , the Z-space, such that 1 < h and the

distance between ail the points in the h-space are preserved in the l-space. Let the distance

between any two points (y and q} in the h- space as ej and the distance between any two

points {yi and y j ) in the 1-space as dY, . First start with an initial guessing of the n points in

the 1-space. Then compute the error of distances beheen the h-space and the i-space points

using the formula:

The next step is to try to minimize the error E by changing the value of dY, for a fked value

of d', using the steepest descent algorithm as follows,

Let ~ ( m ) be the mapping error after the mth iteration defined as:

where

the new 1- space configuration at time m+l is given by,

where a2 E (m)

= Eq(;mi) /l &,(rn)2 / and a is a constant that was determined empirically to be x 0.3 or 0.4.

The partial derivatives in Formula (C.5) are given by,

and

Other mapping techniques based on different criterion can be found in [76].

In this research nonlinear mapping used for many purposes such as:

1- To chose a region of an ultrasound image which contains the texture that could be used

to ident* the marbling score of the animal. Each time a certain region from ultrasound image

is selected the Haralick and Law features are computed and then projected using Sammon

algorithm. A selected region that shows the best separation in the feature space between

ultrasound images of dïfFerent marbling score was chosen to design the classifier.

2- The same procedure was used to select the size of the window used to extract Haralick

features. The optimal size was ernpirically found to be 45x45.

3- To gain more insight of the sigaificaace of the whitening process.

Index

algorithm

k-means, 27

Sammon, 40-42, 104, 107, 113

Simulated Annealing, 12, 105

Bayes' theorem, 27, 79, 80

hypothesis testing, 81

classes, 106

fat, 37

intra-class distance, 55

marbling, 57

prototypes, 25

classifiers

Adaptive logic neural networks, 14

Bayes', 75

definition, 107

design, 24, 44

evaluation, 45

fusion, 15, 74

Fuzzy k-means, 105

ISOCLUS, 13, 14

k-means, 27, 44

MAP, 26, 75

Maximum A posterior probability, 26

Maximum likelihood classifier, 13

m D , 25, 44, 75

Minimum Euclidean Distance, 25

CO-occurrence

example, 32

matrix, 31

window size, 41

curse of dimensionality, 54

distance

absolute value, 25

city block, 25

Euclidean, 25

intra-class, 55

minimum value, 25

feat ures, 106

CO-occurrence, 33

extraction, 13, 28, 42, 61

frequency domain, 12

Haralick, 33, 41

Haralick's vector, 42

Laws, 61

selection, 54

space, 39, 50

spatial domain, 12

vector, 40, 51

whitening, 50

mat r ix

C ~ - O C C U ~ ~ ~ R C ~ , 3 1

filtering

spatial averaging, 38, 109

temporal, 13

wavelet denoising dgorithm, 38, 109

fusion

averaged Bayesian, 76

by GBF, 79

by voting, 75

met hodologies, 75

of classifiers, 74

the generaiized Bayesian fusion (GBF),

79

image

Area Of Interest (AOI), 14. 40

Area of Interest (AOI) , 64

filtering, 38

histogram, 29

histogram equaiization, 37

photographie, 13

preprocessing, 37

transformation, 60

mar Ming

approaches, 3

commercial systems, 14

definition, 2

grades, 2

meat

grading, 1, 20

industry, 5

quality, 1

noise

speckle, 7

pattern recognition systenis

defini tion, 106

for grading, 33, 41

pro bability

first-order, 28

Guassian, 98

multirnodal density, 27

normal distribution, 45, 50, 56

Rayleigh, 7

second-order, 3 1

the Kurtosis value (K) , 8

space

classification, 23

feature, 23, 25, 39, 44, 50, 106

pattern, 23

the measurement , 106

speckle

ânalyticd models, 8

autocorrelation, 7, 9

modeling by stochastic theory, 7

noise, 38, 108

relation to marbling, 8

statistics

robust, 12

texture

andysis, 21

density Mage, 62

extraction, 12, 64

first-order probabilistic approach, 28

image, 62

Laws, 61

rotationdy invariant density image, 62

second-order probabilistic approach, 31

tissues

and ultrasound interaction, 3

characterization, 6

layered modeling, 6

random scatterers modeling, 6

texture, 4

transform

Sanimon, 107, 113

whitening, 50, 107

ultrasound

A-mode, 5, 9, 10

B-mode, 6, 17

compression waves, 3

cross-sectional images, 14

kequency, 4

imaging, 5

longitudinal, 14

propagation, 3

radio signal, 8

shear waves, 3

speed, 4, 10