Thesis an Final

Topological analysis of complex

networks using assortativity

Mahendra Rajah Piraveenan

A thesis submitted in fulfillment

of the requirements for the degree of

Doctor of Philosophy

School of Information technologies

The University of Sydney

May 2010

Declaration

I hereby declare that this submission is my own work and that, to the best of my knowledge

and belief, it contains no material previously published or written by another person nor

material which to a substantial extent has been accepted for the award of any other

degree or diploma of the University or other institute of higher learning, except where due

acknowledgement has been made in the text.

Mahendra Rajah Piraveenan

7 May, 2010

i

ii

Abstract

Mahendra Rajah Piraveenan Doctor of PhilosophyThe University of Sydney May 2010

Topological analysis of complexnetworks using assortativity

This thesis investigates assortative mixing in complex networks. Assortativity is the ten-dency whereby nodes preferentially connect with other nodes similar to themselves. Simi-larity of nodes could be interpreted in terms of node degrees, or in terms of other propertiesof nodes such as node states. Understanding the assortative mixing patterns in complexnetworks is important for a number of reasons, including classification of networks, design-ing growth models, interpreting node functionality, and successfully attacking or defendingnetworks. Moreover, such understanding could be utilized in a number of domains, in-cluding biological networks, technical networks and social networks.

This thesis analyses assortative mixing of directed and undirected networks, both at global(network) level, and local (node) level. The primary contribution of this thesis is at thelocal level, where it introduces the novel concept of local assortativity. Local assortativity isdefined as an individual node’s contribution to network assortativity, and mathematicallyderived for both undirected and directed networks. It is shown that local assortativityvalues of individual nodes provide information about node functionality, and the localassortativity distributions provide an additional quantitative tool for analysis of networktopologies. It is further demonstrated that complex networks could be classified in termsof these distributions, and four such classes exist in the case of undirected networks.The thesis also defines local assortativity in terms of node states, which is termed nodecongruity.

At the global (network) level, the thesis studies the relationship between assortative mixingand Shannon information content of networks, again for the directed and undirected cases.It is shown that, under certain conditions, these quantities are related by an informationpower law. The Shannon information content is also defined in terms of node states, andit is demonstrated that minimalistic and maximalistic networks could be found in termsof information content based on a given degree distribution.

Finally, the thesis presents a number of algorithms for assortativity related network designtasks. These include the Parallel Addition and Rewiring Growth (PARG) model, whichcould be used to grow a certain class of disassortative networks similar to Internet, andthe Assortative Preferential Attachment (APA) method which could grow networks witha given level of assortativity.

Acknowledgements

A Japanese proverb declares that when you have completed 95 percent of your journey,you are only half way there. Many were the times during this Ph.D candidature when Iappreciated the truth in this proverb. It has been a challenging journey to say the least.During this journey I once spent several weeks in a hospital bed. There was a semesterwhen I had to suspend my candidature. When I look back though, I find that the journeyhas been overall very enjoyable and ultimately fulfilling.

However, as Tim Cahill once said, a journey is best measured in friends rather than miles.There were many people who helped me pull through during the most difficult times, andmade it all worthwhile in the end. I feel utmost gratitude to them all. I cannot thankthem enough, but I shall make an attempt.

First of all I would like to thank my supervisors, Prof. Albert Y. Zomaya and Dr. MikhailProkopenko, for all their guidance and help throughout this journey. I have always felt thatI was very lucky to have you both as my supervisors. Mikhail, your advice and researchinsights, as well as all the time you spent with me working on our papers, are deeplyappreciated. I have been constantly amazed by your attention to detail and thoroughscrutiny of everything I send you. It was on your advice that I embarked on this journeyin the first place. Albert, you always encouraged me and helped me stay positive duringsome very difficult times. A Ph.D student needs confidence more than anything else andyou gave me that. A big and heartfelt thank you to you both.

I have enjoyed a productive and fun-filled work environment, thanks to my colleagues atboth CSIRO ICT centre and the University of Sydney. I would like to thank MatthewChadwick, Peter Wang, Joseph Lizier, Astrid Zeman, Don Price, Rose Wang, and OliverObst from CSIRO for their friendship and support. Similarly I would like to thank KhaledAlmiani, Young Choon Lee and Abdul Sikder from the University of Sydney for thefriendship shown. My thanks also go to all administrative staff from both CSIRO ICTcentre and the University of Sydney for their assistance.

Mrs. Vishaka Nanayakara, Head of the department of Computer Science and Engineering,Moratuwa University, Sri Lanka assisted this project immensely by providing workingspace and computer access whenever I visited Sri Lanka. I thank her for that, and forher encouragement and support. Thanks Kishan for assisting my research and discussingit with me, and thanks Sulochana, Prasad and Dulani for the language lessons. I alsowant to thank all staff from the same department for their friendship, encouragement andsupport.

iii

iv

I would like to thank my mentor and friend Sanjeev, who has always encouraged me tochoose a career in research. Throughout this candidature his encouragement and supporthas been invaluable.

Many friends helped me throughout this project to stay motivated and stay on course. Iwould like to thank them all. You all deserve to be mentioned by name, but I am unableto do so here and hope you will forgive me for that.

My teachers from Mahajana college, Union college and the University of Adelaide gaveme the foundation upon which I built this thesis. My heartfelt gratitude goes to them.

Finally, words are not adequate to express my gratitude towards my mother and father.You stood by me during the most difficult times. I will simply say, thank you for yourlove and for always being there when I needed you.

To Amma and Appa.

Publications

The following publications and manuscripts-under-review have resulted from the candida-ture for this degree:

1. M. Piraveenan, M. Prokopenko, and A. Y. Zomaya,“Local assortativeness in scale-free networks,” Europhysics Letters, vol. 84, no. 2, p. 28002, 2008.

2. M. Piraveenan, M. Prokopenko, and A. Y. Zomaya, “Assortativeness and informa-tion in scale-free networks,” European Physical Journal B, vol. 67, pp. 291-300,2009.

3. M. Piraveenan, M. Prokopenko, and A. Y. Zomaya, “Assortativity and growth ofinternet,” European Physical Journal B, vol. 70, pp. 275-285, 2009.

4. M. Piraveenan, M. Prokopenko, and A. Y. Zomaya, “Assortative mixing in directedbiological networks,” IEEE Transactions on computational biology and bioinformat-ics, in press, 2010.

5. M. Piraveenan, M. Prokopenko, and A. Y. Zomaya, “Local assortativeness in scale-free networks - addendum,” Europhysics Letters, vol. 89, no. 4, p. 49901, 2010

6. M. Piraveenan, M. Prokopenko, and A. Y. Zomaya, “On congruity of nodes incomplex networks,” Submitted to IEEE Transactions on computers, 2010

7. M. Piraveenan, K.A.D.N.K. Wimalawarne, M. Prokopenko, and A. Y. Zomaya,“Centrality of four-node motifs in metabolic networks,” Submitted to Theory inbiosciences, 2010 *

8. M. Piraveenan, M. Prokopenko, P. Wang, A. Zeman, “Decentralised multi-agentclustering in scale-free sensor networks,” book chapter, in J. Fulcher and L. C. Jain(eds.), Studies in Computational Intelligence (SCI), 115, 485-515, Springer, Berlin,2008 *

9. M. Piraveenan, M. Prokopenko, and A. Y. Zomaya, “Information-cloning of scale-free networks,” in Advances in Artificial Life: 9th European Conference on ArtficialLife (ECAL- 2007), Lisbon, Portugal, ser. Lecture Notes in Artificial Intelligence,F. A. e Costa, L. M. Rocha, E. Costa, and A. C. I. Harvey, Eds. Springer, 2007, vol.4648, pp. 925-935.

vi

vii

10. J. T. Lizier, M. Piraveenan, D. Pradhana, M. Prokopenko, and L. S. Yaeger, “Func-tional and structural topologies in evolved neural networks,” in Advances in ArtificialLife: 10th European Conference on Artificial Life (ECAL -2009), ser. LNCS/LNAI.Springer, 2009, vol. 5777-5778

11. M. Piraveenan, D. Polani and M. Prokopenko, “Emergence of Genetic Coding: anInformation-theoretic Model,” in Advances in Artificial Life: 9th European Con-ference on Artficial Life (ECAL -2007), Lisbon, Portugal, ser. Lecture Notes inArtificial Intelligence, F. A. e Costa, L. M. Rocha, E. Costa, and A. C. I. Harvey,Eds. Springer, 2007, vol. 4648, pp. 42-52 *

12. M. Piraveenan, M. Prokopenko, and A. Y. Zomaya, “Classifying complex networksusing unbiased local assortativity,” accepted, 12th International Conference on theSynthesis and Simulation of Living Systems (ALIFE -2010)

The papers marked with an asterick (*) have not directly contributed to this thesis.

Contents

Declaration i

Abstract ii

Acknowledgements ii

Publications vi

Contents viii

List of Figures xiii

List of Tables xvi

Nomenclature xvii

1 Introduction 1

1.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.4 Principal contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.5 Thesis structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Background 9

2.1 Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2 Degree-related distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.1 Degree distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

viii

CONTENTS ix

2.2.2 Excess degree (remaining degree) distribution . . . . . . . . . . . . . 10

2.2.3 Joint degree distribution . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2.4 Network assortativity . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.5 Scalar assortativity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.2.6 Limitations on minimal and maximal assortativity . . . . . . . . . . 15

2.2.7 Small-world Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2.8 Scale-free networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.3 Information content of networks . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.4 Complex networks in the real world . . . . . . . . . . . . . . . . . . . . . . 22

2.4.1 Protein-Protein Interaction (PPI) Networks . . . . . . . . . . . . . . 22

2.4.2 Transcription Networks . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.4.3 Gene Regulatory Networks (GRN) . . . . . . . . . . . . . . . . . . . 22

2.4.4 Cell Signalling Networks . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.4.5 Metabolic Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.4.6 Food webs (Ecological Networks) . . . . . . . . . . . . . . . . . . . . 25

2.4.7 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.4.8 Cortical Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.4.9 Scientific author collaboration Networks . . . . . . . . . . . . . . . . 25

2.4.10 Citation Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.4.11 Internet AS Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3 Assortativity and information in undirected networks 27

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.2 Information content of networks . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3 Classification of networks based on information content . . . . . . . . . . . 30

3.3.1 Minimalistic and maximalistic networks . . . . . . . . . . . . . . . . 33

3.4 Shannon information of real-world networks . . . . . . . . . . . . . . . . . . 36

3.5 Power-law of information-assortativity dependency . . . . . . . . . . . . . . 36

3.5.1 Slope and stability regions . . . . . . . . . . . . . . . . . . . . . . . . 42

3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3.7 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

CONTENTS x

4 Information content and assortativity in directed networks 48

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

4.2 Motivation for alternative assortativity definitions in directed networks . . . 49

4.3 Out-assortativity and in-assortativity . . . . . . . . . . . . . . . . . . . . . . 50

4.3.1 Canonical network examples . . . . . . . . . . . . . . . . . . . . . . 52

4.4 Assortativity of directed real world networks . . . . . . . . . . . . . . . . . 53

4.5 Assortativity and information content in directed networks . . . . . . . . . 56

4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5 Local assortativity in undirected networks 60

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

5.2 Definition of local assortativity . . . . . . . . . . . . . . . . . . . . . . . . . 62

5.3 Local assortativity distributions . . . . . . . . . . . . . . . . . . . . . . . . . 66

5.3.1 Local assortativity in model networks . . . . . . . . . . . . . . . . . 66

5.4 Local Assortativity in Scale-free networks . . . . . . . . . . . . . . . . . . . 68

5.5 Classification of networks using local assortativity . . . . . . . . . . . . . . 71

5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

6 Local assortativity in directed networks 77

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

6.2 Defining local assortativity in directed networks . . . . . . . . . . . . . . . . 78

6.2.1 Motivation for alternative local assortativity definitions . . . . . . . 82

6.2.2 Local out-assortativity and local in-assortativity . . . . . . . . . . . 83

6.2.3 Singularity cases of directed local assortativity . . . . . . . . . . . . 84

6.2.4 Distributions of local assortativity . . . . . . . . . . . . . . . . . . . 85

6.3 Local assortativity in canonical networks . . . . . . . . . . . . . . . . . . . . 86

6.4 Local assortativity Distributions of real-world Biological networks . . . . . . 88

6.4.1 Comparing various local assortativity measures . . . . . . . . . . . . 92

6.5 Local assortativity profiles and functionality of individual nodes . . . . . . . 94

6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

6.7 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

CONTENTS xi

7 Non-degree based assortativity 100

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

7.2 Scalar assortativity as a function of time . . . . . . . . . . . . . . . . . . . . 102

7.2.1 Model networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

7.3 Scalar assortativity in Random Boolean Networks . . . . . . . . . . . . . . . 106

7.3.1 Random logic: logic f1 . . . . . . . . . . . . . . . . . . . . . . . . . . 108

7.3.2 Logic f2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

7.3.3 Logic f3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

7.3.4 Combination of logical functions . . . . . . . . . . . . . . . . . . . . 110

7.4 Scalar assortativity and information content . . . . . . . . . . . . . . . . . . 111

7.5 Node congruity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

7.6 Distributions of node congruity . . . . . . . . . . . . . . . . . . . . . . . . . 121

7.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

8 A growth model based on local assortativity profiles 127

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

8.2 Local assortativity distributions of Internet at the AS level . . . . . . . . . 129

8.3 Growth models of Internet at the AS level . . . . . . . . . . . . . . . . . . . 131

8.3.1 Inet 3.0 model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

8.3.2 The Barabasi–Albert (BA) model . . . . . . . . . . . . . . . . . . . . 131

8.3.3 The Generalised Linear Preference (GLP) model . . . . . . . . . . . 131

8.3.4 The Interactive Growth (IG) model . . . . . . . . . . . . . . . . . . 133

8.3.5 The Positive Feedback Preference (PFP) model . . . . . . . . . . . . 136

8.3.6 Growth models and local assortativity distributions . . . . . . . . . 137

8.4 A network motif with negative local assortativity distribution . . . . . . . . 138

8.5 The PARG Model for Internet growth . . . . . . . . . . . . . . . . . . . . . 142

8.6 The local assortativity distribution of networks grown by the PARG model 145

8.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

9 Information cloning using assortativity 151

9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

9.2 Information cloning using Assortative Preferential attachment . . . . . . . . 153

9.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

CONTENTS xii

10 Conclusions 159

10.1 Summary of contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

10.1.1 Assortativity and Shannon information . . . . . . . . . . . . . . . . 159

10.1.2 Assortativity in directed networks . . . . . . . . . . . . . . . . . . . 160

10.1.3 Local assortativity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

10.1.4 Node congruity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

10.1.5 Parallel Addition and Rewiring Growth model . . . . . . . . . . . . 161

10.1.6 Assortative Preferential Attachment . . . . . . . . . . . . . . . . . . 161

10.1.7 Applications of assortative mixing . . . . . . . . . . . . . . . . . . . 161

10.2 Directions for future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

10.2.1 Local assortativity based sustained attack . . . . . . . . . . . . . . . 162

10.2.2 Quantifying the minimum assortativity limit . . . . . . . . . . . . . 163

10.2.3 Classification of directed networks . . . . . . . . . . . . . . . . . . . 163

10.2.4 Evolution of assortativity and local assortativity in networks . . . . 164

10.2.5 Local assortativity and rich club phenomena . . . . . . . . . . . . . 164

10.2.6 The investigation of more real world networks . . . . . . . . . . . . . 165

10.3 Epilogue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

A Data sources and software 167

A.1 Data sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

A.2 Software tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

B Evolution of assortativity in neural networks 170

B.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

B.2 Polyworld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

B.3 Inferring Functional Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 172

B.4 Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

B.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

C Rich club phenomenon and local assortativity 178

Bibliography 182

List of Figures

2.1 Excess degrees of nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2 The Caltech undergraduates Facebook network . . . . . . . . . . . . . . . . 23

2.3 The transcription network of C. glutamicum bacteria . . . . . . . . . . . . . 24

3.1 Shannon information and assortativity for different networks . . . . . . . . 30

3.2 Relationship between Shannon information and assortativity: γ = 1.0 . . . 34

3.3 Relationship between Shannon information and assortativity: γ = 2.3 . . . 35

3.4 The Escherichia coli metabolic network . . . . . . . . . . . . . . . . . . . . 40

3.5 Class A network corresponding to the Escherichia coli metabolic network . 41

3.6 Class B network corresponding to the Escherichia coli metabolic network . 42

3.7 The dependencies between b1 and Np, for different γ . . . . . . . . . . . . . 43

3.8 The dependencies between b1 and γ, for different Np . . . . . . . . . . . . . 43

3.9 The dependencies between d1 and γ, for different Np . . . . . . . . . . . . . 44

4.1 In-degrees and out-degrees of nodes with respect to a link . . . . . . . . . . 51

4.2 Model networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

4.3 A model network with perfect out-assortativity, imperfect in-assortativity . 54

4.4 Shift in assortativity coefficient with separate in-degrees and out-degrees . . 57

5.1 Excess degrees of nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

5.2 Local assortativity distribution of a regular lattice . . . . . . . . . . . . . . 67

5.3 Local assortativity distribution: assortative and non-assortative networks . 68

5.4 Local assortativity distribution: disassortative networks . . . . . . . . . . . 69

5.5 Local assortativity distribution: assortative networks . . . . . . . . . . . . 70

5.6 Local assortativity distribution: disassortative networks . . . . . . . . . . . 71

xiii

LIST OF FIGURES xiv

5.7 Local assortativity distribution: non-assortative networks . . . . . . . . . . 72

5.8 Examples of network classes . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

5.9 Examples of network classes - distributions . . . . . . . . . . . . . . . . . . 73

6.1 In-degrees and out-degrees of nodes with respect to a link . . . . . . . . . . 82

6.2 Model networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

6.3 Local in-assortativity distributions . . . . . . . . . . . . . . . . . . . . . . . 90

6.4 Local out-assortativity distributions . . . . . . . . . . . . . . . . . . . . . . 90

6.5 Rat GRN: Scatter plot of node ρout vs out-degree . . . . . . . . . . . . . . . 91

6.6 Mouse GRN: Scatter plot of node ρout vs out-degree . . . . . . . . . . . . . 92

6.7 Local assortativity ρ vs degree profile of E. coli transcription . . . . . . . . 93

6.8 Local assortativity ρd distribution vs degree: E. coli transcription . . . . . . 94

7.1 Star network with scalar assortativity L t = −1 . . . . . . . . . . . . . . . . 104

7.2 Ring network with scalar assortativity L t = −1 . . . . . . . . . . . . . . . 105

7.3 A scale-free network with scalar assortativity L t = −1 . . . . . . . . . . . . 105

7.4 The benzene-ring topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

7.5 Variation of scalar assortativity: logic f1 A . . . . . . . . . . . . . . . . . . 110

7.6 Variation of scalar assortativity: logic f1 B . . . . . . . . . . . . . . . . . . 111





7.11 Variation of scalar assortativity: alternating logics . . . . . . . . . . . . . . 116

7.12 Variation of scalar assortativity and entropy . . . . . . . . . . . . . . . . . . 117

7.13 Variation of scalar assortativity and mutual information: logic f2 . . . . . . 118

7.14 Variation of scalar assortativity and mutual information: logic f3 . . . . . . 119

7.15 Node congruity profile of M. musculus GRN: f1 . . . . . . . . . . . . . . . . 123



7.18 Node congruity profile of individual nodes: f2 . . . . . . . . . . . . . . . . . 124

7.19 Node congruity profile of individual nodes: f3 . . . . . . . . . . . . . . . . . 125

LIST OF FIGURES xv

8.1 Local assortativity distribution of Internet at the AS level A . . . . . . . . . 130

8.2 Local assortativity distribution of Internet at the AS level B . . . . . . . . . 130

8.3 Local assortativity distribution : Preferential Attachment . . . . . . . . . . 132

8.4 Local assortativity distribution: Interactive Growth model . . . . . . . . . . 132

8.5 Local assortativity distribution: PFP model A . . . . . . . . . . . . . . . . 133

8.6 Local assortativity distribution: PFP model B . . . . . . . . . . . . . . . . 134

8.7 Local assortativity distribution: BA model with varying parameters . . . . 135

8.8 Local assortativity distribution: Random network . . . . . . . . . . . . . . . 136

8.9 Local assortativity distribution: Star motif . . . . . . . . . . . . . . . . . . 139

8.10 Local assortativity distribution: a network motif A . . . . . . . . . . . . . . 140

8.11 Local assortativity distribution: A network motif B . . . . . . . . . . . . . . 141

8.12 A PARG model subnetwork . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

8.13 The local assortativity distribution: PARG model . . . . . . . . . . . . . . . 146

8.14 Degree distribution : PARG model . . . . . . . . . . . . . . . . . . . . . . . 146

8.15 Degree distribution of the real AS 98 network . . . . . . . . . . . . . . . . . 147

9.1 Information content I(r) as a function of r . . . . . . . . . . . . . . . . . . 153

9.2 Difficulty of recovery for γ = 1 . . . . . . . . . . . . . . . . . . . . . . . . . 154

9.3 Average of Dδ(r) for γ = 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

9.4 Standard deviation of Dδ(r) for γ = 1 . . . . . . . . . . . . . . . . . . . . . 155

9.5 Difficulty of recovery for r = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . 156



B.1 Assortativity trends in structural and functional networks . . . . . . . . . . 174

B.2 Clustering trends in structural and functional networks . . . . . . . . . . . 175

B.3 Closeness trends in structural and functional networks . . . . . . . . . . . . 176

C.1 The rich club coefficient in Internet AS level 1998 topology . . . . . . . . . 179

C.2 The cumulative average local assortativity vs ranked degree . . . . . . . . . 180

C.3 The cumulative average local assortativity vs the rich club coefficient . . . . 180

List of Tables

3.1 Shannon information computed for metabolic (substrate) networks . . . . . 37

3.2 Shannon information computed for transcription networks . . . . . . . . . . 38

3.3 Shannon information computed for Protein-Protein Interaction networks . . 38

3.4 Shannon information computed for Internet . . . . . . . . . . . . . . . . . . 38

3.5 Shannon information computed for citation networks . . . . . . . . . . . . . 39

3.6 Shannon information computed for collaboration networks . . . . . . . . . . 39

4.1 Assortativity in real world directed networks . . . . . . . . . . . . . . . . . 55

5.1 Classification of real world networks . . . . . . . . . . . . . . . . . . . . . . 72

6.1 Biological networks and their nodes with highest ρout or ρin . . . . . . . . . 95

7.1 Scalar assortativity with Benzene ring topology . . . . . . . . . . . . . . . . 106

8.1 Parameters of PARG model . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

8.2 A comparison between growth models . . . . . . . . . . . . . . . . . . . . . 148

xvi

Nomenclature

Notation

P (a) Probability of the event aP (a, b) Probability of the event a

⋂b

P (a|b) Probability of the event a given event b

Typefaces

X,Y, Z Variable namesx, y, z Specific values taken by the variables X,Y, ZX,Y,Z Sets of variablesx,y, z Values to X,Y,Z

Abbreviations

MI Mutual InformationGRN Gene Regulatory NetworkPPI Protein Protein InteractionKEGG Kyoto Encyclopaedia for Genes and GenomesAPA Assortative Preferential AttachmentAS Autonomous SystemPARG Parallel Addition and Rewiring GrowthIG Interactive GrowthPFP Positive Feedback PreferenceBA model Barabasi Albert modelLA Local AssortativityGLP Generalized Linear Preference

Variables Used

r Assortativity of a networkrd Assortativity of a directed network

xvii

NOMENCLATURE xviii

rout Out-Assortativity of a directed networkrin In-assortativity of a directed networkN Number of nodes in a networkM Number of links in a networkk Degree of a node (the number of links of a node). In the case of directed

networks, this is typically used to denote the degree of the target node.j Degree of a node (the number of links of a node). In the case of directed

networks, this is typically used to denote the degree of the source node.kin In-degree of a node (the number of links coming into a node). In the case

of directed networks, this is typically used to denote the in-degree of thetarget node

kout Out-degree of a node (the number of links going out of a node). In the caseof directed networks, this is typically used to denote the out-degree of thetarget node.

jin In-degree of a node (the number of links coming into a node). In the caseof directed networks, this is typically used to denote the in-degree of thesource node

jout Out-degree of a node (the number of links going out of a node). In the caseof directed networks, this is typically used to denote the out-degree of thesource node

γ Scale free exponent of a networkNp Maximum degree of a networkpk Degree distribution of a networkqk Excess degree (remaining degree) distribution of a networkqink Excess in-degree distribution of a network

qoutk Excess out-degree distribution of a network

ej,k Link distribution of a networkeoutj,k Out-out degree link distribution of a network

einj,k In-in degree link distribution of a network

ρv Local assortativity of a nodeρin Local in-assortativity of a node in a directed networkρout Local out-assortativity of a node in a directed network

Chapter 1

Introduction

‘All men are caught in an inescapable network of mutuality. Whatever affects one

directly, affects all indirectly.’ – Martin Luther King, Jr.

1.1 Objectives

This thesis is concerned with assortative mixing in complex networks. It investigates mix-

ing patterns based on node degree and non-degree attributes, in directed and undirected

networks, and in simulated and real world networks. It answers the question of how as-

sortative mixing affects the amount of Shannon information in a complex network. It

attempts to classify complex networks based on mixing patterns, and provides algorithms

for duplicating these mixing patterns, by growth or design, in synthesised networks. It

also investigates how individual nodes contribute to overall mixing patterns in networks.

Finally, it sheds light on how assortative mixing can be used to learn about functionality of

individual nodes, attack or defend networks, and evolve networks with specific topological

or information theoretic properties.

The goal of this thesis can be summarised as:

“To investigate assortative mixing in complex networks, based on node degree and other

attributes, including the local contribution of individual nodes to the overall mixing pat-

terns, the influence of mixing patterns on the information content of networks, and the

utility of mixing patterns in highlighting node roles and increasing network robustness.”

1.2 Motivation 2

1.2 Motivation

Networks are ubiquitous in today’s world. Communication networks such as world wide

web, telephone networks and mobile phone networks are changing the way we live and

we interact with other people. Social networks built on top of these, such as Facebook

and Twitter, are redefining ways of keeping in touch. Vast airline and rail networks have

given us access to the remotest parts of the world and reduced travel times by orders

of magnitude. Our survival depends on the functioning of a number of biological and

ecological networks. The energy needed for our domestic and industrial use is supplied by

electric power networks. Indeed, the interest and awareness about networks are not only

a trend in scientific research but also a social and cultural phenomenon of this age [41].

In a lot of systems that can be described as complex networks, the ability to function

properly arises not from individual components themselves, but by the way they interact.

In other words, the whole is more than the sum of the parts in these systems, and as

such they cannot be completely understood by a reductionist approach. A complete

description of the way the components of a networks are connected to each other is called

the network’s topology. Understanding the topology of a network is vital for understanding

its function, since the topology evolves (or is designed) to better undertake the function,

and the efficiency of network function is influenced by its topology. For this reason,

topological analysis of complex networks has been an intensely researched area in the last

decade. Scientists have taken advantage of the availability of fresh data from a number of

rapidly developing fields, including systems biology and computer networks, and new graph

theoretical techniques have been developed to tackle the increasingly large and complex

networks produced by this data. From simple random graphs to scale-free networks and

hierarchical, modular networks, graph theory has made great progress in modelling and

understanding the topology of various types of complex networks.

One may ask a number of questions about a network and its topology: What does it

visually look like? How large is it? How did it evolve or was developed? Why does

it display certain features, and what are the functional roles of these features? How to

design another network with the same features? and so on. To analyse and answer these

questions, a number of measures and metrics have been developed, by which a network

may be quantified. Degree distribution, modularity, clustering, centrality metrics, and

1.2 Motivation 3

motif analysis are some such measures. In this thesis we study another such feature,

namely the correlations between the properties of neighbouring nodes: the mixing patterns

[34, 81, 82, 92, 95, 108].

It is well known that in a number of social and biological networks, links do not connect

nodes randomly regardless of their attributes [82]. In a social network, people tend to be

friends with other people who are similar to them. Individuals with similar age, profession

or physical address tend to make more connections among themselves than mere chance

would dictate. This tendency is called assortative mixing. In an ecological network,

however, predator organisms tend to have links with prey organisms. In a sexual network,

most links tend to be between opposite genders. This is called disassortative mixing. Such

tendencies in mixing patterns can be quantified and interpreted in a number of ways. An

in-depth analysis and characterisation of assortative mixing in various types of complex

networks is the objective of this thesis.

The understanding of mixing patterns is important to the study of topology as well as

dynamics of networks for a number of reasons. Firstly, networks can be classified based on

the mixing patterns of nodes [50, 95] so that common topological traits can be identified in

networks from various domains (for example, biological and social networks), or networks

from a similar domain can be distinguished based on topology (eg., the transcription net-

works of E. coli and C. glutamicum). Secondly, we may gain insights into the evolution

or design of the networks, and may design growth models to duplicate the desired mixing

patterns. At an individual node level, the mixing patterns may highlight the key func-

tionalities of nodes. For instance, in a gene regulatory network, the key regulators can

be identified by the connecting patterns; and in an air-traffic network, the key airports.

As a direct consequence, understanding mixing patterns is vital to successfully attack or

successfully defend a network. In epidemiological networks, spread of infection may be

contained by taking out key nodes in terms of mixing patterns. In defence related net-

works, attackers with limited resources may bring down the entire network with targeted

attacks on a few key nodes. In social networks, targeting a few well connected people for

advertising campaigns is likely to be very successful. In sensor networks, analysing mixing

patterns may identify ‘fault lines’ where nodes display a continuous set of unacceptably

high sensor readings. In short, the understanding of mixing patterns is critically impor-

tant in all realms of complex networks including biological, technical and social networks.

1.3 Approach 4

Thus, the vitality of the study cannot be overstated.

1.3 Approach

In this thesis, we analyse assortative mixing in two levels: On the network level, where we

are concerned about average statistical indicators of assortative mixing in the network, and

on the node level, where we are concerned about how individual nodes contribute to the

overall patterns in network. We are concerned with both directed and undirected networks.

Even though simulated networks are often used to complement our analysis, the emphasis

is put on real world networks. The research spans biological, technical, and social networks.

Gene Regulatory Networks, transcription networks, Protein-Protein Interaction networks,

metabolic networks, neural networks, cortical networks, and food webs are some types of

biological networks we have analysed in this thesis. We have looked into scientific author

collaboration networks, paper citation networks, and Facebook among social networks.

Internet Autonomous System Level networks, software class diagrams and power grids

are examples of technical networks we have considered. Some applications of the concepts

introduced here are also presented in depth, and the connection between assortative mixing

levels and the information content of the network is analysed. The thesis attempts to

present a comprehensive analysis of assortative mixing in complex networks.

Whenever a novel concept (such as local assortativity ) is introduced, we attempt to apply

it first to some simple canonical networks, such as a regular lattice or star network. We

follow this by analysing simulated networks. The Assortative Preferential Attachment

method (introduced in chapter 3 ) is used to produce networks with a given level of

assortativity. This is followed by analysis of real world networks. Thus simulated and real

world networks are used to complement each other in this thesis. A set of data sources

used is given in appendix A.

Remark 1.3.1. The term assortativeness has been used in some of our publications

[91–93] to denote assortativity. We now consider that assortativity is a better term and

this term is used throughout this thesis and our later publications [94–96, 98]. These terms

are synonymous.

Remark 1.3.2. When analysing real-world networks, it should be remembered that the

finite-size of the networks may have an effect on the topological analysis. Following the

1.4 Principal contributions 5

standard practise in graph theory [41], all mathematical derivations assume infinite net-

work size unless otherwise stated. Most real world networks that we have analysed contain

hundreds or thousands of nodes.

1.4 Principal contributions

The main contributions of this thesis are:

• The introduction and formulation of the concept of local assortativity, for both

directed and undirected networks. This is a novel contribution to graph theory and

could be applied to any type of complex network. The subsequent introduction of

local assortativity distributions of networks.

• The utilisation of local assortativity distributions to classify networks. Four classes

of complex networks were identified using these distributions.

• Demonstration of how local assortativity can be used to highlight functionality of

nodes in networks, particularly biological networks.

• Growth models for complex networks based on their local assortativity profiles. In

particular, the Parallel Addition and Re-wiring Growth (PARG) model for Internet.

• Quantifying the relationship between assortativity and Information content in net-

works. Designing maximalistic and minimalistic networks in terms of information

content. The analysis of assortativity-information content landscape. The subse-

quent introduction of ‘information cloning’, re-growing damaged networks based on

their level of assortativity.

• The formulation of meaningful assortativity measures for directed networks (namely

out-assortativity and in-assortativity).

• The introduction of Assortative Preferential Attachment algorithm, to grow a net-

work with a given level of assortativity, subject to constraints.

• The introduction and formulation of the concept of node congruity. This is also a

novel contribution to graph theory, and while being similar to local assortativity,

1.5 Thesis structure 6

helps to highlight the interplay between topology and dynamics of a network. The

subsequent introduction of node congruity distributions.

The thesis contributes to assortativity based characterisation of networks on three

levels: (i) global (network) level (ii) local (node) level (iii) application level. The

most significant contribution of the thesis is the introduction of the concept of local

assortativity, which made the analysis possible at the node level for the first time.

The subsequent introduction of local assortativity distributions has enabled a new

classification of complex networks and provided a tool to analyse functionality of

nodes. It has also given new insights into network growth and evolution. The thesis

has also contributed at the global (network) level by introducing meaningful defini-

tions of assortativity for directed networks. While assortativity in directed networks

has already been defined [82], the thesis shows this definition was misleading, es-

pecially in the case of biological networks, and provides alternative definitions. It

also analyses the meaning of assortativity by exploring its connection to information

content at global level, again for both directed and undirected networks. Finally,

the thesis highlights a number of applications for the concepts introduced, including

information cloning, growth models, and targeted attacks of networks (or defence

thereof).

1.5 Thesis structure

This thesis is organised as follows:

• Chapter 2 provides the theoretical background for this thesis. Particularly, this

chapter presents the existing work upon which this thesis is built. The chapter in-

troduces a number of concepts related to network topology, including various degree

and link distributions. It describes the concept of assortative mixing at network level

and presents the related definitions. The chapter also provides a groundwork about

information theoretic concepts, which will be used throughout the thesis. Finally,

the chapter also briefly reviews a number of real world networks.

• Chapters 3-9 present the contributions of this thesis.


– Chapters 3 - 4 present contributions to this thesis at the ‘network ’ (global)

level.

∗ Chapter 3 is concerned with assortativity in undirected networks. It

analyses Shannon information content of (undirected) networks in terms

of their assortativity. Analysing the relationship between assortativity and

information content under a number of constraints, the chapter presents

minimalistic and maximalistic classes of networks based on their Shannon

information and shows that a number of real world networks lie between

these classes. Optimising Shannon information on the landscape of the

network’s parameter search-space, two regions of interest are identified: a

slope region and a stability region. Based on this the chapter explains

why certain parameters of real world scale-free networks are found within

a certain range.

∗ Chapter 4 deals with assortativity in directed networks. Putting an em-

phasis on biological networks which are directed, the chapter presents out-

assortativity and in-assortativity as better measures to analyse assortative

mixing in directed networks, compared to ‘general’ assortativity. The chap-

ter defines corresponding information content measures and uses them to

quantify the amount of information presented by out-assortativity and in-

assortativity. The chapter applies these measures to a number of real world

directed networks.

– Chapters 5 - 7 present contributions to this thesis at the ‘node’ (local) level.

∗ Chapter 5 introduces the concept of node-level (local) assortativity in

undirected networks. After presenting the derivation, the chapter intro-

duces local assortativity distributions and analyses such distributions for a

number of canonical, simulated and real world networks. The chapter also

highlights the possible applications for the novel metric of local assortativ-

ity.

∗ Chapter 6 extends the concept of node-level (local) assortativity to di-

rected networks. Particularly, complementing chapter four, this chapter

motivates and defines local in-assortativity and local out-assortativity. Fol-

lowing the derivations, the chapter applies local assortativity in directed


networks to highlight functionality of nodes in a number of directed bio-

logical networks.

∗ Chapter 7 analyses node-state based assortativity, which is named as

scalar assortativity. The chapter specifically introduces the concept of lo-

cal (node) congruity. The chapter illustrates how scalar assortativity of

node-states can be plotted as a function of time and used to analyse net-

work dynamics. The chapter also considers the information content of

the network in terms of node states. Finally, the chapter introduces and

defines node congruity, node congruity distributions, and highlights their

applications.

– Chapters 8 - 9 present some applications to the concepts demonstrated in the

previous chapters. More applications (subject to future research) are listed in

chapter 10.

∗ Chapter 8 is concerned with local assortativity distributions in Internet.

Pointing out that the existing growth models for Internet Autonomous

System Level networks do not match the local assortativity distributions of

real Internet AS networks, the chapter introduces a new growth model. The

growth model is named Parallel Addition and Rewiring Growth (PARG)

model and presented as a generic growth model to match a certain type

of local assortativity profile in any network. The chapter also provides a

comparative study of PARG and existing growth models.

∗ Chapter 9 investigates information-cloning recovery of scale-free networks

in terms of their information transfer, by using their level of assortativity.

It identifies a number of recovery features, and these features are inter-

preted with respect to two opposing tendencies dominating network recov-

ery: an increasing amount of choice in adding assortative or disassortative

connections, and an increasing divergence between the joint excess-degree

distributions of existing and required networks.

• Chapter 10 summarises the main conclusions, and identifies future research direc-

tions and application areas.

Chapter 2

Background

In this chapter we introduce a number of concepts on which the rest of the thesis is built.

The concepts introduced here include standard notions of graph theory and information

theory, as well as some of the recently introduced ideas. However, any new notion or

concept which is a contribution of this thesis is withheld for later chapters. We will start

with the formal definition of a network.

2.1 Network

Formally, a network (graph) is a set of nodes (vertices) connected by links (edges) [41, 85,

86]. It can be directed, where links originate from source nodes and end at target nodes,

or it can be undirected where there is no such distinction. We do not consider weighted

links [83], therefore all links are assumed to have the weight of unity.

The total number of links a node has is called the node’s degree k. In directed networks,

the number of incoming links to a node is its in-degree kin and the number of outgoing

links from the node is its out-degree kout. The largest number of links any node contains is

the network’s maximum degree Np . A network’s average degree k can be defined similarly.

A network can be fragmented unless otherwise stated; i.e it can contain a number of

disconnected components. The component with the biggest number of nodes is called the

giant component Smax.

2.2 Degree-related distributions 10

2.2 Degree-related distributions

Now, let us consider a network with N nodes (vertices) and M links (edges). We can

define the following degree-related distributions for this network.

2.2.1 Degree distribution

Let us say that the probability of a randomly chosen node having degree k is pk, where

1 ≤ k ≤ Np. The distribution of such probabilities is called the degree distribution pk of

the network.

In the case of directed networks, we may define in-degree distribution and out-degree dis-

tribution in a similar manner. The distribution of probabilities of nodes having a given

in-degree kin is defined as the in-degree distribution, pink . Similarly, the distribution of

probabilities of nodes having a given out-degree kout is defined as the out-degree distribu-

tion, poutk .

2.2.2 Excess degree (remaining degree) distribution

Excess degree distribution in undirected networks

Let us now consider a randomly chosen link in an undirected network. A node which is

reached by this link will have a number of other links connected to it. In other words, if

one has arrived at a node using a link, there are a number of ‘remaining’ paths or links

to traverse away from that node. This number is therefore called the excess degree or

remaining degree of the node (see figure 2.1). We may denote the probability of the node

at a random end of this link having excess degree k as qk. We call the distribution of such

probabilities as the Excess degree distribution [81, 82] qk of the network. It is also called

the remaining degree distribution [108].

This distribution is biased in favour of nodes of high degree, since more links end at a high-

degree node than at a low-degree one [81]. It is related to the original degree distribution

as follows:

qk =(k + 1)pk+1∑Np

1 kpk

, 1 ≤ k ≤ Np (2.1)


It should be noted that rather than considering the excess degree of the node at the end

of a link, we may instead consider the degree itself, which is the excess degree of that node

plus one to account for the link under consideration.

Figure 2.1: Excess degrees of nodes. Note that when the link between v1 and v2 isconsidered, it has a node of excess degree k = 3 at one end and a node of excess degreek = 2 at the other end.

Excess degree distribution in directed networks

A similar distribution can be defined in directed networks. Note however, that in the case

of directed networks, considering the ‘excess’ degree does not always make sense (since

some of the links cannot be used as ‘remaining paths’ to traverse away from the node, due

to directionality), and degrees rather than excess degrees are used in the literature to define

these distributions [81, 82]. Still, the distribution is defined as the probability distribution

of the node at a random end of a randomly chosen link having degree k. Therefore we

will continue to call it the excess degree distribution, with the understanding that degree

rather than excess degree is used as index in directed networks. Note however, that this

is not the degree-distribution, since it deals with degrees at the end of randomly chosen

links, rather than degrees of randomly chosen nodes. Thus the excess degree distribution

in directed networks will satisfy

qk =kpk∑Np

1 kpk

, 1 ≤ k ≤ Np (2.2)

where pk is the degree distribution. In directed networks, we may also define excess

in-degree distribution, and excess out-degree distribution. The probability distribution

of the target node of a randomly chosen directed link having in-degree kin is qink , the


excess in-degree distribution. Similarly, the probability distribution of the source node

of a randomly chosen directed link having out-degree kout is qoutk , the excess out-degree

distribution.

2.2.3 Joint degree distribution

Joint degree distribution in undirected networks

Let us consider an undirected link having a node with excess-degree j on one end and a

node with excess-degree k on the other end. Following [34] and [81], we can define the

quantity ej,k to be the joint probability distribution of the excess-degrees of the two nodes

at either end of a randomly chosen link. (For example, in Figure 2.1, the link between v1

and v2 will contribute to e2,3 and e3,2). As pointed out by [82], this quantity is symmetric

in its indices for an undirected graph. that is

ej,k = ek,j (2.3)

and it obeys the sum rules

∑

j

ej,k = qk (2.4)

∑

jk

ej,k = 1 (2.5)

Joint degree distribution in directed networks

In the case of directed networks, [82] defines the eout,inj,k as the probability distribution of

finding a directed link from a source node of jout out-degree and to a target node of kin

in-degree. Therefore the distribution is no longer symmetric, but it still obeys the sum

rules

∑

jout

eout,inj,k = qin

k (2.6)


∑

kin

eout,inj,k = qout

j (2.7)

∑

joutkin

eout,inj,k = 1 (2.8)

2.2.4 Network assortativity

Assortativity [21, 51, 81, 82, 114, 119, 123] is the tendency observed in complex networks

where nodes mostly connect with similar nodes. Typically, this similarity is interpreted

in terms of degrees of nodes [30, 65, 81, 108] (However, it is possible to define similarity

in non-degree terms, as we will describe later). Many complex networks in real world

show the tendency where highly connected nodes link with other highly connected nodes

(that is, nodes mix assortatively). The reverse is also true in some networks, where highly

connected nodes are more likely to make links with isolated, less connected nodes, i.e.

to mix disassortatively. In both cases, the probability of creating a link depends on the

degrees of both nodes. Averaging across the network, assortativity quantifies the tendency

for preferential association within the network [81, 91].

Naturally occurring networks display various levels of assortative mixing, and it becomes

necessary to quantify the level of assortative mixing in a complex network [81, 82, 108].

The measure proposed in [34, 81] defines assortativity as a correlation function in terms

of degrees at the network level. This correlation function yields zero for non-assortative

mixing and positive or negative values for assortative or disassortative mixing respectively.

In the case of undirected networks, If no preferential mixing occurs, then

ej,k = qjqk (2.9)

Therefore the correlation can be defined as

r =1σ2

q

∑

jk

jk (ej,k − qjqk)

(2.10)


where ej,k is the joint probability distribution of the excess degrees of the two nodes at

either end of a randomly chosen link. σq is the standard deviation of the excess degree

distribution of the network, qk. Similarly,

∑

j

jqj = µq (2.11)

where µq is the expected value or mean of the excess degree distribution. Therefore

network assortativity r can be defined also as:

r =1σ2

q

(

∑

jk

jkej,k)− µ2q

(2.12)

where µq and σq are both constants for the network.

Here r lies between −1 and 1, whereby r = 1 means perfect assortativity, r = −1 means

perfect disassortativity, and r = 0 means no assortativity (random linking).

If a network has perfect assortativity (r = 1), then all nodes connect only with nodes

with the same degree. For example, the joint distribution ej,k = qkδj,k where δj,k is the

Kronecker delta function, produces a perfectly assortative network. If the network has

no assortativity (r = 0), then any node can randomly connect to any other node. A

sufficiency condition for a non-assortative network is ej,k = qjqk. This is not a necessary

condition: other ej,k may also produce non-assortativity. For example, for the uniform

remaining degree distribution qk, the distribution ej,k = [qjδj,k + qjδj,(Np−1−k)]/2 will

produce a non-assortative network.

A similar definition has been proposed for vertex based assortativity coefficient in directed

networks [82]. Here the assortativity coefficient rd can be defined as

rd =1

σinq σout

q

∑

jk

jk(eout,inj,k − qin

j qoutk

) (2.13)

which can also be written as

rd =1

σinq σout

q

(

∑

jk

jkeout,inj,k )− µin

q µoutq

(2.14)


where eout,inj,k is the joint degree distribution, µin

q , µoutq are the means of the distributions

qink , qout

k respectively. Similarly, σinq , σout

q are the standard deviations of the respective

distributions.

2.2.5 Scalar assortativity

The degree-based definition for assortativity was extended in [82] to any scalar attribute

of a network. Accordingly, scalar assortativity in [82] is defined as

r =1

1−∑jk

ajbk

∑

jk

(ej,k − ajbk)

(2.15)

where aj and bk are the fraction of each type of end (source or target) of a link that is

attached to node of type j and node of type k . In undirected networks, where there is no

‘source’ or ‘target’ node, aj = bj . As before, ej,k is the fraction of links which have type j

of node at source and type k of nodes at target. Again, in undirected networks ej,k = ek,j .

2.2.6 Limitations on minimal and maximal assortativity

It is noted by [82] that perfect scalar disassortativity (r = −1) is not always possible. In

other words, while all links connecting same types of nodes will always mean r = 1.0, if

all links connect nodes of different types, this may not always mean that (r = −1). The

minimum value rmin in such cases is given by:

rmin =1

1−∑jk

ajbk

∑

jk

(−ajbk)

(2.16)

In general, −1.0 ≤ rmin ≤ 0.0. The explanation for not simply having r = −1.0 for a

maximally disassortative network is that a maximally disassortative network is normally

closer to a randomly mixed network than is a perfectly assortative network. When there are

several different vertex types, then random mixing will most often pair unlike vertices, just

like disassortative mixing. Therefore, it is appropriate that most disassortative networks

shows values closer to r = 0.0 compared to assortative networks [82]. One the other hand,


as we will explain in chapter 3, a network will be close to perfectly disassortative, i.e

r = −1.0, if, not only dissimilar nodes mix, but nodes which are the most dissimilar tend

to mix with each other (i.e, the scalar attributes of mixing nodes are at the extremes of the

scale. For example, in the case of degrees being the attribute considered, the biggest hubs

mix only with extremely peripheral nodes). While it is possible to design such networks

(as will be shown with the ‘Assortative Preferential Attachment’ later in the thesis), such

situations with real world networks are extremely rare.

For degree-based assortativity, we may make some specific observations regarding minimal

assortativity. The r = −1 case is possible only for symmetric excess degree distributions

where qk = q(Np−1−k), and ej,k = qkδj,(Np−1−k). In other words, for a network with excess

degrees 0, . . ., Np−1, a node with degree k must be linked to a node with a degree Np−1−k.

Nodes with identical degrees may still be connected in a perfectly disassortative network

(e.g., when their degree j is precisely in the middle of the distribution q, i.e., Np is odd

and j = (Np − 1)/2).

Perfect disassortativity is not possible for non-symmetric excess degree distributions qk,

because the ej,k distribution must obey the rules ej,k = ek,j , as well as∑j

ej,k = qk.

We denote the maximum attainable disassortativity (i.e minimum assortativity) as rmin,

where rmin < 0 (rmin = −1 only for symmetric qk). This limit and the corresponding

e(r=rmin)j,k can be obtained, given the distribution qk, via a suitable minimisation procedure

by varying ej,k under its constraints.

Perfect assortativity r = 1.0, on the other hand, is possible for any degree distribution and

excess degree distribution, as long as fragmented networks are not discounted. Indeed, if

we are to have more than one type of nodes, in terms of degree or otherwise, the network

would have to be necessarily fragmented to obtain r = 1.0, as any link between dissimilar

nodes will destroy perfect assortativity.

Finally, let us note that in terms of growing or constructing networks, in general we should

distinguish between difficulties in (i) constructing an ej,k distribution for a given degree

distribution pk, and (ii) growing the network for the calculated joint-degree distribution

ej,k. When one is constructing an ej,k distribution for a given degree distribution pk, the

cases of maximum disassortativity and maximum assortativity differ. Maximum assorta-

tivity is always possible, but maximum disassortativity is not. On the other hand, when

one is growing the network with the given ej,k, it may also not be possible to achieve


r = 1 for a given pk or qk. This is despite the fact that the required ej,k can be obtained

— the reason is that the network may not be large enough to accommodate all the nec-

essary connections. Thus, the maximum limit of assortativity rmax may also need to be

considered for networks that can be actually constructed. This is essentially due to the

finite-size effect as we pointed out in chapter 1, and not a mathematical constraint.

2.2.7 Small-world Networks

The average path length l of a network is defined as the average length of shortest paths

between all pairs of nodes in that network. For many real world networks, this average

path length is much smaller than the size of the network, that is l ¿ N . Such networks

are said to be showing the small world property [63, 79, 117].

The small world effect was famously demonstrated by Milgram with a network of acquain-

tances [75]. In Milgram’s experiment, several hundreds of randomly chosen people from

the US state of Omaha were asked to send a letter to a specified addressee from Boston.

The letters contained instructions that if they did not know the addressee, they should

mark their names on the letter and send the letter to anybody they think who might

know the target person. It turned out that the average number of hops required before

the letters were received by the intended addressees was only six: thus the ‘six-degrees of

separation’ [116] was demonstrated in this social network of acquaintances.

It has since been shown that a range of real world networks, including social networks, bio-

logical networks such as Gene Regulatory Networks, metabolic networks, Protein-Protein

Interaction networks, and signalling networks, as well as Internet show the small world

property [19, 41, 102]. Practically all real world networks studied in this thesis display the

small world property.

2.2.8 Scale-free networks

Scale-free networks are those networks that display similar topological features irrespective

of scale. Such networks are described by power law degree distributions, formally specified

as

pk = Ak−γu(k/Np) (2.17)

2.3 Information content of networks 18

u is a step function specifying a cut off at k = Np. The degree distribution of scale-free

networks can be specified by a number of parameters, including maximum degree Np,

scale-free exponent γ, proportion of out-lier nodes A, and average degree k. However, it

can be shown that there are only two independent parameters and the others could be

derived from these. In this thesis, we use the maximum degree Np and scale-free exponent

γ as the parameters to define degree distributions of scale-free networks.

Scale-free networks are impressively robust to random node failure and random damage

[17, 41]. To destroy or fragment such networks randomly, one would have to remove almost

all of its nodes [41]. This perhaps explains, at least partly, why scale-free architecture

is commonly found in many evolved networks in nature. This also means that targeted

attacks have to be designed specifically to effectively destroy such networks, and non-trivial

topological analysis of the network is necessary to identify the nodes to be targeted. It

is not always the case that targeting the hubs is the most effective way to attack such

networks, either. We will revisit this point later in the thesis.

Indeed, most real world networks are scale-free networks, including technical, biological

and social networks [22–24, 36, 41, 76, 89]. It is possible in some directed networks that the

in-degree distribution is scale-free but the out-degree distribution is not, or vice versa. For

example, the in-degree distributions of some transcription networks are scale-free, while

the out-degree distributions are exponential [19]. There are a number of growth models

which generate scale-free networks, and prominent among them is the Barabasi-Albert

model [15]. The subject of growth models is dealt with extensively in Chapter 8.

2.3 Information content of networks

Information Theory was originally developed by Shannon [106] for reliable transmission

of information from a source X to a receiver Y over noisy communication channels. Put

simply, it addresses the question of “how can we achieve perfect communication over an

imperfect, noisy communication channel?” [72]. When dealing with outcomes of imperfect

probabilistic processes, it is useful to define the information content of an outcome x which

has the probability P (x), as log21

P (x) (it is measured in bits): improbable outcomes convey

more information than probable outcomes. Given a probability distribution P over the

outcomes x ∈ X (i.e., over a discrete random variable X representing the process), and


defined by the probabilities P (x) ≡ P (X = x) given for all x ∈ X , the average Shannon

information content of an outcome is determined by

H(X) = −∑

x∈XP (x) log P (x) , (2.18)

henceforth we omit the logarithm base 2. This quantity is known as (information) entropy.

Intuitively, it measures, also in bits, the amount of freedom of choice (or the degree of

randomness) contained in the process — a process with many possible outcomes has high

entropy. This measure has some unique properties that make it specifically suitable for

measuring “how much “choice” is involved in the selection of the event or of how uncertain

we are of the outcome?” [106]. In answering this question, Shannon required the following

properties for such a measure H:

• continuity: H should be continuous in the probabilities, i.e., changing the value of

one of the probabilities by a small amount changes the entropy by a small amount;

• monotony: if all the choices are equally likely, e.g. if all the probabilities P (xi) are

equal to 1/n, where n is the size of the set X = {x1, . . . , xn}, then H should be a

monotonic increasing function of n: “with equally likely events there is more choice,

or uncertainty, when there are more possible events” [106];

• recursion: H is independent of how the process is divided into parts, i.e. “if a choice

be broken down into two successive choices, the original H should be the weighted

sum of the individual values of H” [106],

proving that entropy function −K∑n

i=1 P (xi) log P (xi), where a positive constant K rep-

resents a unit of measure, is the only function satisfying these three requirements.

The joint entropy of two (discrete) random variables X and Y is defined as the entropy

of the joint distribution of X and Y :

H(X, Y ) = −∑

x∈X

∑

y∈YP (x, y) log P (x, y) , (2.19)


where P (x, y) is the joint probability. The conditional entropy of Y , given random variable

X, is defined as follows:

H(Y |X) =∑

x∈X

∑

y∈YP (x, y) log

P (x)P (x, y)

= H(X, Y )−H(X) . (2.20)

This measures the average uncertainty that remains about y ∈ Y when x ∈ X is known

[72].

Mutual information I(X;Y ) [61] measures the amount of information that can be ob-

tained about one random variable by observing another (it is symmetric in terms of these

variables):

I(X;Y ) =∑

x∈X

∑

y∈YP (x, y) log

P (x, y)P (x)P (y)

. (2.21)

Mutual information I(X; Y ) can also be expressed via the conditional entropy:

I(X; Y ) = H(Y )−H(Y |X) . (2.22)

The amount of information I(X; Y ) shared between transmitted X and received Y signals

is often maximised by designers of communication channels, via choosing the best possible

transmitted signal X. Channel capacity is defined as the maximum mutual information

for the channel over all possible distributions of the transmitted signal X (the source).

The conditional entropy H(Y |X) is also called the equivocation of Y about X, and thus,

informally, the mutual information I(X; Y ) is equal to the difference between receiver’s

diversity H(Y ) and the equivocation of receiver about source H(Y |X). Hence, the channel

capacity is optimised when receiver’s diversity is maximised, while its equivocation about

the source is minimised.

Let us define the network’s information content (called information transfer by Sole and

Valverde [108]):

I(q) = H(q)−H(q|q′) (2.23)

where the first term is the Shannon entropy of the network, H(q) = −Np−1∑k=0

qk log(qk),

that measures the diversity of the degree distribution or the network’s heterogeneity, and

the second term is the conditional entropy defined via conditional probabilities π(k|k′) of

observing a node with k links leaving it, provided that the node at the other end of the


chosen link has k′ leaving links. Importantly, the conditional entropy H(q|q′) estimates

correlations in the network created by connecting the nodes with dissimilar degrees — this

component affects the overall diversity or the heterogeneity of the network, but does not

contribute to the amount of information within it. Informally, information content within

the network is the difference between network’s heterogeneity and assortative noise within

it [108].

In information-theoretic terms, H(q|q′) is the assortative noise within the network’s in-

formation channel, i.e., it is the non-assortative extent to which the preferential (either

assortative or disassortative) connections are obscured [99, 100]. Given the joint (remain-

ing) degree distribution ej,k, the information content can be expressed as:

I(q) =Np−1∑

j=0

Np−1∑

k=0

ej,k logej,k

qjqk(2.24)

Shannon information I(q) is a better, more generic measure of dependence than the cor-

relation functions that measure linear relations. Mutual information measures the general

dependence and is thus a less biased statistic [108]. Shannon information (2.24) can also

be seen as the Kullback-Leibler divergence K(ej,k ‖ qjqk) [62] (i.e., relative entropy) of

the product of two marginal distributions q from the joint distribution e. This divergence

amounts to the expected number of extra bits that must be transmitted in order to iden-

tify (on average) excess degrees of connected nodes j and k of the link (j, k) if they are

assigned using only the marginal distribution q, instead of the joint distribution ej,k. It

is evident that maximal information I(q) is attained when the product qjqk diverges the

most from the joint distribution ej,k, and minimal information I(q) is attained when the

product qjqk and the joint distribution ej,k diverge the least.

The entropy and information content described above are defined with respect to the degree

distribution and joint degree distribution, and there are alternative definitions where the

entropy of the network could be characterised by higher order correlations or community

structure [29].

It should also be noted that Shannon information contains no inherent directionality, and

various alternatives have been proposed. For example, transfer entropy [58, 104] measures

the average information contained in the source about the next state of the destination that

was not already contained in the destination’s past. It can be argued that transfer entropy

2.4 Complex networks in the real world 22

is the appropriate measure for predictive information transfer in spatiotemporal systems

[69]. In this thesis we follow Sole and Valverde [108] in using the mutual information to

represent information content (and not a directional transfer) within a network. However,

we will demonstrate how to extend it to directed networks by formulating it in terms of

directed distributions.

2.4 Complex networks in the real world

As mentioned in Chapter 1, mixing patterns of a number of real world networks are

investigated in this thesis. A brief introduction of each type of these networks is warranted:

2.4.1 Protein-Protein Interaction (PPI) Networks

The nodes are molecules of protein inside a cell, and the links represent any biochemical

interaction between them. The networks are undirected [57, 59, 73, 88, 111].

2.4.2 Transcription Networks

The nodes are regulatory genes and regulated proteins, and the links are the interactions

between them [37, 46, 59]. These are bipartite and directed networks.

2.4.3 Gene Regulatory Networks (GRN)

The nodes are genes, and the links are the inhibitory or inducing effects of one gene on the

expression of another gene [19, 43]. Note the subtlety that unlike transcription networks,

only genes are considered as nodes in these directed networks.

2.4.4 Cell Signalling Networks

The nodes are receptors and ligands inside (or in the vicinity) of a cell [44, 71]. The links

represent interactions between these receptors and ligands, which constitute a system of

signal transduction pathways inside the cell. The networks can be considered directed.


Fig

ure

2.2:

The

Face

book

netw

ork

ofth

eun

derg

radu

ate

stud

ents

inth

eC

alifo

rnia

Inst

itut

eof

Tec

hnol

ogy

[12]

.bl

ue:

mal

es,p

ink:

fem

ales

,gr

een:

gend

erin

form

atio

nno

tav

aila

ble.

See

the

inse

tw

here

asu

bnet

wor

kis

show

nto

illus

trat

eth

eco

nnec

ting

patt

erns

.


Fig

ure

2.3:

The

tran

scri

ptio

nne

twor

kof

C.gl

utam

icum

bact

eria

[25]


2.4.5 Metabolic Networks

The nodes are substrates belonging to one or more metabolic pathways inside a cell, and

links are biochemical reactions typically catalysed by enzymes acting on these substrates

[52, 90, 101, 113]. Note that this is sometimes called a ‘substrate network’ [55, 97]. A

complementary representation where the biochemical reactions are the nodes and the links

represent substrates is called a ‘reaction network’. We do not consider reaction networks

in this thesis.

2.4.6 Food webs (Ecological Networks)

The nodes are organisms in an ecosystem and the links represent predator-prey relation-

ships between them [56, 77, 108]. These networks can be considered undirected or directed

(prey to predator).

2.4.7 Neural Networks

The nodes are neurons belonging to an organism’s neural system and links are anatomical

connections between neurons [35, 45, 57]. These are undirected networks.

2.4.8 Cortical Networks

The term cortical networks is not a standard term in complex network literature, like the

terms used to denote other types of networks in this section. By this term we denote

the network of dependencies between various regions of the cerebral cortex (in a set of

primates)[6, 53, 112]. The nodes are regions in the cortex, and the links are functional

dependencies. Note that the nodes are not individual neurons. These are also undirected

networks.

2.4.9 Scientific author collaboration Networks

The nodes are authors of research papers, and a link exists between two authors if they

have co-authored at least one paper [80, 84]. These are undirected networks.


2.4.10 Citation Networks

The nodes are research papers (or other citable documents) and links denote citations

between these documents. These are directed networks [64].

2.4.11 Internet AS Networks

The nodes represent an Autonomous System present in the Internet and the links represent

a commercial agreement between two Internet Service Providers(who own the two ASs)[94].

Remark 2.4.1. Some of the networks mentioned here are ‘logical networks’, where the

‘links’ between nodes do not exist physically, but are only inferred logically based on func-

tionality. Gene Regulatory Networks (where a directed link is said to exist between node

A and node B if the expression levels of gene A affected the expression of gene B) are

an example of this. Other networks are ‘physical or anatomical networks’, where the links

exist physically. The neural network of C. elegans is an example of this (where the neurons

inside the organism are anatomically connected).

Remark 2.4.2. In this thesis, the term ‘neural network’ is used to mean the network

of neurons inside an organism. The term is often used by computer scientists to mean

an ‘Artificial Neural Network (ANN)’ (e.g. [31]), constructed to solve a computational

problem, such as an optimisation problem. We do not investigate such ANNs in this

thesis.

Chapter 3

Assortativity and information in

undirected networks

3.1 Introduction

Complex networks exhibit diverse mixing patterns, which are governed by a number of

parameters including maximum degree Np, average path length l, average degree k, and

in the case of scale-free networks, the scale free exponent γ [15, 16, 41]. Therefore, if we

consider a given node with degree k, there is an amount of uncertainty about the degrees

of the neighbours of this node. Depending on the network topology and mixing patterns,

this uncertainty, on average, is higher in some networks than others. Thus it becomes

possible, as we saw in chapter 2, to define the information content of the network in terms

of topology. If the information content is higher, we know, on average, what to expect in

terms of degrees at the ends of links. If the information content is zero, this means that

the topology is random, and no prediction can be made about degrees of nodes at the

ends of links.

This chapter is concerned with information content and assortativity in undirected net-

works. The definition for assortativity in undirected networks was proposed by Newman

[81] as described in chapter 2, and unlike in the directed networks case, this definition is

sufficient for the purposes of this thesis. However, since this thesis proposes to use assorta-

tivity as a tool to understand and analyse mixing patterns in networks, it is important to


understand what higher or lower levels of assortativity means in terms of the topological in-

formation content of network. If the assortativity is higher, does it mean that the network

topology is more predictable, or less? In other words, what is the relationship between

network assortativity and topological information content of the network? This chapter

investigates this question (for undirected networks), so that we can quantify the merits

of assortativity as a tool to provide information about the network. This investigation is

undertaken using simulated scale-free networks as well as real world networks.

This chapter is organised as follows. In section 3.2, we begin by studying an extensive

set of real-world networks to understand their information content and its relationship to

assortativity. In Section 3.3, we identify classes of minimalistic and maximalistic scale-

free networks in terms of Shannon information. For the studied classes, the information

is shown to depend non-linearly on the absolute value of the assortativity, with the dom-

inant term of the relationship being a power-law. We demonstrate that this relationship

subsumes that presented in [108]. In section 3.4 this dependency and classification is ex-

emplified using a range of real-world networks. Section 3.5 analyses the parameter search

space of scale-free networks in terms of the information power law, and two regions of

interest are identified: a slope region and a stability region. The implications of these

regions to network design in terms of maximising information content are explored. We

present the chapter summary in section 3.6.

3.2 Information content of networks

Let us recall the definitions of assortativity and information content in undirected networks

from chapter 2. Assortativity is defined as

r =1σ2

q

∑

jk

jk (ej,k − qjqk)

(3.1)

where ej,k is the joint probability distribution of the excess degrees of the two nodes at

either end of a randomly chosen link. σq is the standard deviation of the excess degree

distribution of the network, qk. Whereas, the Shannon information content is defined as


I(q) =Np−1∑

j=0

Np−1∑

k=0

ej,k logej,k

qjqk(3.2)

Sole and Valverde [108] were among the first to empirically analyse the relationship be-

tween assortativity and Shannon information, using a set of real world networks (which

were scale-free). Their conclusion was that the information (transfer) and assortativity

are correlated in a negative way: the extent of disassortativity increases with mutual in-

formation (see Figure 7 in [108]). We argue that this conclusion was only partially correct,

and influenced by the particular set of networks studied by [108].

To begin with, let us note that networks with the same assortativity r and the same

distribution qk could have different information contents I — because they may disagree

on ej,k. Moreover, most of the real world networks studied by Sole and Valverde [108] did

not show perfect (or nearly perfect) assortativity or disassortativity: the observed values

were between 0.4 and −0.2: a rather narrow subrange. Moreover, the compared networks

did not agree on average degree, degree distribution, etc. These reasons obscured the

conclusion reported in [108].

Therefore, first of all, we studied a more extensive set of real world networks as shown in

Figure 3.1. The networks that we studied include the scale-free networks considered by

Sole and Valverde (Tab. I in [108], including some technological and biological networks),

metabolic substrate networks (see Tab. 3.1), metabolic substrate networks without inor-

ganic components [50], transcription networks (see Tab. 3.2), protein-protein interaction

networks (see Tab. 3.3), Internet at Autonomous Systems (AS) level (see Tab. 3.4), cita-

tion networks (see Tab. 3.5), and collaboration networks (see Tab. 3.6). In selecting these

networks, we in particular made sure that the range of assortativity was increased. Figure

3.1 demonstrates that Shannon information is not negatively correlated with assortativity

(as conjectured by Sole and Valverde [108]), but is correlated with the absolute value of

the assortativity.

The Figure 3.1 demonstrates clearly that the correlation between assortativity and Shan-

non information is non-linear. However, these networks still have a range of topological

parameters (degree distribution, average degree etc) and as such a clear relationship can-

not be established by considering them all together. Indeed, Figure 3.1 shows no such

clear relationship. Thus a principled investigation becomes necessary. It is neverthe-

3.3 Classification of networks based on information content 30

less impossible to find enough real world networks where we can consider the topological

parameters in a controlled manner, one at a time, all other parameters being equal. How-

ever, as we have noted before, all the networks mentioned above are scale-free networks

[15, 19, 41, 50, 64, 94]. Therefore, it is suitable to undertake a principled analysis of this

relationship for scale-free networks based on theory and simulation.

0

0.2

0.4

0.6

0.8

1

1.2

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6

Info

rmat

ion

Con

tent

Assortativeness

Figure 3.1: Shannon information and assortativity for different networks. Circles: net-works considered by Sole and Valverde (Tab. I in [108]); squares: metabolic substratenetworks [2]; triangles: metabolic substrate networks without inorganic components [50];crosses: transcription networks [25]; stars: protein-protein interaction networks [3]; filledtriangles: Internet AS [5]; pluses: citation networks [64]; filled squares: collaborationnetworks [80, 84].

3.3 Classification of networks based on information content

We noted that networks with the same assortativity r and the same degree distribution pk (

and remaining-degree distribution qk) could have different information contents I because

they may still have different ej,k. Therefore, in establishing the relationship between I(q)

and r, we may classify networks according to the dependency of the distribution e(r=r′)j,k on


the assortativity r′. Within a class, the same distribution qk and the same assortativity r

result in the same information content I(r) (of course, when distribution qk changes, the

same r will typically correspond to different values I). Thus, each of these classes may

correspond to a different level of information content while the degree distribution is the

same.

Class A

As noted, the assortativity r is defined in terms of the joint distribution ej,k.

Theorem 3.3.1. If the distribution ej,k is given by the linear decomposition (3.3) for a

real number r′ > 0, then the network assortativity is precisely r′:

e(r=r′)j,k = r′ ( e

(r=1)j,k − e

(r=0)j,k ) + e

(r=0)j,k (3.3)

where e(r=1)j,k = qkδj,k and e

(r=0)j,k = qjqk. This is a sufficient but not necessary condition.

Proof. Substituting Eq. 3.3 into Eq. 3.1, we get

r =1σ2

q

∑

jk

jk(r′ ( e

(r=1)j,k − e

(r=0)j,k ) + e

(r=0)j,k − qjqk

) (3.4)

However e(r=0)j,k = qjqk. Therefore

r =1σ2

q

∑

jk

jk(r′ ( e

(r=1)j,k − e

(r=0)j,k )

) (3.5)

r =r′

σ2q

∑

jk

jk ( e(r=1)j,k − e

(r=0)j,k )

(3.6)

By definition of assortativity

1σ2

q

∑

jk

jk ( e(r=1)j,k − e

(r=0)j,k )

= 1 (3.7)

Thus it follows that


r = r′ (3.8)

Theorem 3.3.2. If r′ < 0 then

e(r=r′)j,k =

r′

rmin( e

(r=rmin)j,k − e

(r=0)j,k ) + e

(r=0)j,k (3.9)

where rmin ≤ 0 is the maximum attainable disassortativity.

Proof. Again, substituting Eq. 3.9 into Eq. 3.1, we get

r =1σ2

q

∑

jk

jk

(r′

rmin( e

(r=rmin)j,k − e

(r=0)j,k ) + e

(r=0)j,k − qjqk

) (3.10)

as e(r=0)j,k = qjqk,

r =1σ2

q

∑

jk

jk

(r′

rmin( e

(r=rmin)j,k − e

(r=0)j,k )

) (3.11)

r =r′

rminσ2q

∑

jk

jk ( e(r=rmin)j,k − e

(r=0)j,k )

(3.12)

By definition of assortativity

1σ2

q

∑

jk

jk ( e(r=rmin)j,k − e

(r=0)j,k )

= rmin (3.13)

It follows that

r =r′

rminrmin (3.14)

= r′ (3.15)


Theorem 3.3.3. If qk is symmetric then

e(r=r′)j,k = r′ ( e

(r=−1)j,k − e

(r=0)j,k ) + e

(r=0)j,k (3.16)

where e(r=−1)j,k = qkδj,(Nq−1−k).

Proof. Obtained from 3.9 noting that for symmetric qk, rmin = −1.

The templates (3.3) — (3.16) define a class of networks, class A. As intended, the same

distribution qk and the same assortativity r result in the same value I(r) within the class.

This is so simply because the templates define a unique distribution e(r=r′)j,k for a given r′,

and the distribution e(r=r′)j,k yields a unique information I(r′) according to the Eq. (3.2).

In particular, Shannon information within a non-assortative class A network (i.e., r′ = 0)

is zero: I(0) = 0.

Class B

Among many other possible classes, we may define another class, class B, by the following

template:

e(r=r′)j,k =

r′ + 12

e(r=1)j,k − r′ − 1

2e(r=rmin)j,k (3.17)

where e(r=1)j,k and e

(r=rmin)j,k , including e

(r=−1)j,k which replaces e

(r=rmin)j,k for symmetric distri-

butions, are computed as for the class A templates. For a non-assortative class B network,

the joint probability e(r=0)j,k is the average between the corresponding probabilities of per-

fectly assortative and disassortative networks: [e(r=1)j,k + e

(r=rmin)j,k ]/2.

Theorem 3.3.4. If the distribution ej,k is given by the linear decomposition (3.17) for a

real number r′ > 0, then the network assortativity is precisely r′.

Proof. Similar to the theorems above.

3.3.1 Minimalistic and maximalistic networks

We computed Shannon information for a wide range of degree distributions by substituting

the class A and class B templates (Eq. 3.3 – Eq. 3.17 ) into the Eq. (3.2). While a degree


distribution can be characterised in terms of many properties, e.g. the average degree, the

power law exponent γ, and the cut-off Np, there are only two independent variables in any

such characterisation, and as we mentioned in chapter 2, we choose the exponent γ and

the cut-off Np as our independent variables1. It is worth pointing out that the constraints

imposed by the connectivity structure of networks of finite size generate spontaneous

correlations which in turn may introduce a structural cut-off Np that possibly differs from

the natural one [32]. For this reason, we directly use the ej,k generated by the templates

into the Eq. (3.2) to calculate the information content of networks which would have

the given parameters, rather than actually growing these networks and measuring their

Shannon information.

0

0.5

1

1.5

2

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

Info

rmat

ion

Con

tent

Assortativeness

Figure 3.2: Relationship between Shannon information and assortativity for class-A(squares) and class-B (stars) networks, γ = 1.0, Np = 4.

Figure 3.2 (symmetric qk with γ = 1.0) and Figure 3.3 (asymmetric qk with γ = 2.3)

show Shannon information for both class A and class B networks. According to these

figures, the information content non-linearly and asymmetrically depends on the absolute

value of the assortativity, i.e. mutual information increases when assortativity varies from

a critical point r, in either positive or negative direction. This relationship subsumes1It should be noted that a network’s information content is independent of the network’s size N .


0

0.5

1

1.5

2

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

Info

rmat

ion

Con

tent

Assortativeness

Figure 3.3: Relationship between Shannon information and assortativity for class-A(squares) and class-B (stars) networks, γ = 2.3, Np = 4.

the one implied by Sole and Valverde [108]. Further calculations with varying γ and Np

produced similar figures. For γ 6= 1, the distribution qk is not symmetric, preventing

perfect disassortativity, and therefore, it is not possible to get close to the (r = −1) case

as shown by Figure 3.3.

Note that class B always corresponds to a higher I(r) compared to class A. Our conjecture

is that the template defining the class A networks is the minimalistic linear template: that

is, the information I(r) for the class A is minimal for a given r. In other words, any real-

world network with the same assortativity r, and the degree distribution parameters γ

and Np, should have higher information I(r). Similarly, the template defining the class B

networks is, we believe, the maximalistic linear template. That is, the information I(r)

for the class B is maximal for a given r, and real-world networks with the same r, γ and

Np, should have lower information I(r).

We verify this conjecture using a range of real-world networks in the following section.

3.4 Shannon information of real-world networks 36

3.4 Shannon information of real-world networks

We computed Shannon information for a set of real-world networks. For example, Table

3.1 is constructed using the metabolic network data from Centre for Complex Network

Research, University of Notre Dame [2]. These results are augmented with information

content computed for corresponding minimalistic and maximalistic networks, obtained

as follows. For a metabolic network with a given number of nodes N , assortativity r,

and the degree distribution parameters γ and Np, we generate a minimalistic class A

network, using the template (3.3) — (3.16), so that it shares the parameters N, r, γ, Np

with the original metabolic network. Analogously, a corresponding maximalistic network

is produced by using the template (3.17). For example, Figure 3.4 shows the metabolic

network for Escherichia coli (r = −0.162, I(r) = 0.49 bits), while Figure 3.5 and 3.6

show its corresponding minimalistic (r = −0.162, I(r) = 0.20 bits) and maximalistic

(r = −0.162, I(r) = 0.68 bits) networks2. It is evident that, although the three illustrated

networks have the same assortativity and the scaling exponent of the power-law degree

distribution, they differ topologically, and in terms of the information content. Table 3.1

demonstrates that information I(r) is always within bounds defined by the information

IA(r) for the corresponding minimalistic network and the information IB(r) for the cor-

responding maximalistic network. Analogously, tables 3.2, 3.3, 3.4, 3.5, 3.6 empirically

verify the respective bounds for other networks. These results confirm our conjecture

about minimalistic and maximalistic templates.

3.5 Power-law of information-assortativity dependency

It was stated in section 3.3 that the information content non-linearly and asymmetrically

depends on the absolute value of the assortativity. To further quantify this, we studied the

relationship I(r) for class A networks, by varying assortativity, γ and Np systematically

and calculating the corresponding Shannon Information using the Eq. 3.3 - 3.16 and Eq.2The networks shown in Figure 3.5 and 3.6 were constructed using the Assortative Preferential Attach-

ment (APA) method, which is a contribution of this thesis and described in the Appendix (Section 3.7).The method was used only to visualise the networks, while the information content and assortativity werecomputed directly using the distribution ej,k.

3.5 Power-law of information-assortativity dependency 37

Network N Np γ r IA(r) I(r) IB(r)

A. pernix 517 86 2.2 -0.181 0.14 0.34 0.75

A. fulgidus 1281 191 2.2 -0.173 0.14 0.42 0.66

M. thermoautotroph. 1138 167 2.2 -0.182 0.18 0.44 0.64

M. jannaschii 1103 160 2.2 -0.176 0.17 0.45 0.70

P. furiosus 790 114 2.1 -0.177 0.16 0.44 0.85

P. horikoshii 807 111 2.1 -0.176 0.22 0.43 0.80

A. aeolicus 1105 147 2.1 -0.193 0.19 0.38 0.67

C. pneumoniae 412 67 2.2 -0.151 0.11 0.32 0.68

C. trachomatis 467 80 2.3 -0.149 0.16 0.35 0.70

Synechocystis sp. 1486 233 2.1 -0.192 0.22 0.45 0.79

P. gingivalis 1052 161 2.2 -0.171 0.13 0.44 0.64

M. bovis 1102 193 2.2 -0.163 0.12 0.41 0.89

M. leprae 1106 177 2.2 -0.18 0.14 0.42 0.72

M. tuberculosis 1534 252 2.1 -0.179 0.20 0.43 0.86

B. subtilis 2217 410 2.1 -0.159 0.18 0.46 0.68

E. faecalis 1049 166 2.1 -0.186 0.18 0.40 0.90

C. acetobutylicum 1349 200 2.1 -0.187 0.17 0.44 0.86

M. genitalium 490 75 2.3 -0.184 0.13 0.49 0.42

M. pneumoniae 420 61 2.2 -0.189 0.05 0.40 0.50

S. pneumoniae 1116 180 2.1 -0.186 0.15 0.40 0.62

S. pyogenes 1087 176 2.1 -0.189 0.19 0.42 0.67

C. tepidum 953 136 2.1 -0.182 0.14 0.44 0.76

R. capsulatus 1808 283 2.1 -0.178 0.19 0.44 0.72

R. prowazekii 469 71 2.3 -0.161 0.08 0.37 0.67

N. gonorrhoeae 1104 169 2.1 -0.19 0.19 0.44 0.65

N. meningitidis 1032 160 2.2 -0.189 0.16 0.42 0.65

C. jejuni 993 153 2.2 -0.186 0.19 0.42 0.68

H. pylori 996 140 2.1 -0.196 0.21 0.45 0.74

E. coli 2316 430 2.1 -0.162 0.21 0.49 0.66

S. typhi 2403 444 2.2 -0.16 0.21 0.48 0.66

Y. pestis 1534 254 2.1 -0.168 0.19 0.42 0.86

A. actinomycetem-comit.

1046 154 2.1 -0.185 0.16 0.40 0.75

H. influenzae 1484 222 2.2 -0.179 0.20 0.48 0.63

P. aeruginosa 2023 364 2.1 -0.16 0.22 0.41 0.82

T. pallidum 506 87 2.2 -0.177 0.17 0.36 0.69

B. burgdorferi 433 78 2.3 -0.16 0.11 0.40 0.82

T. maritima 863 129 2.1 -0.186 0.19 0.37 0.67

D. radiodurans 2337 433 2.1 -0.157 0.24 0.45 0.60

E. nidulans 976 157 2.1 -0.177 0.14 0.43 1.00

S. cerevisiae 1559 260 2.1 -0.181 0.18 0.47 0.98

C. elegans 1207 208 2.1 -0.173 0.10 0.42 1.51

O. sativa 708 99 2.2 -0.167 0.09 0.47 1.50

A. thaliana 737 108 2.2 -0.172 0.09 0.46 1.51

Table 3.1: Shannon information I(r) computed for metabolic (substrate) networks andtheir corresponding class-A and class-B networks. N is the total number of substrates,temporary substrate-enzyme complexes, and enzymes [2].



C. diptheria 71 63 8.49 -0.84 0.85 0.97 0.99

C. efficiens 50 27 8.5 -0.69 0.65 0.82 0.83

C. glutamicum 539 104 1.86 -0.37 0.44 0.86 0.88

C. jeikeium 52 51 8.49 -1.00 1.00 1.00 1.00

Table 3.2: Shannon information I(r) computed for transcription networks and their cor-responding class-A and class-B networks. N is the total number of transcription factors[25].


H. pylori 714 54 1.26 -0.216 0.18 0.36 0.71

M. musculus 502 12 1.96 -0.073 0.12 0.18 1.17

H. sapien 1529 39 1.62 0.067 0.09 0.19 1.10

D. melanogaster 7485 178 1.17 -0.07 0.06 0.15 0.83

S. cerevisiae 502 12 1.96 -0.07 0.12 0.18 1.17

E. coli 1861 152 1.15 0.06 0.04 0.96 1.03

Table 3.3: Shannon information I(r) computed for Protein-Protein Interaction networksand their corresponding class-A and class-B networks. N is the total number of proteins[3].

3.2. This produced the following approximation:

I(r) =

a1 rb1 + c1 ed1 r − c1 if r ≥ 0

a2 |r|b2 + c2 ed2 |r| − c2 if r < 0(3.18)

where |r| denotes the absolute value of assortativity r, and the coefficients ai, bi, ci, di

depend on variables γ and Np. The critical assortativity at which the respective I(r) curve

attains its minimum is denoted as r. In general, r is specific for each degree distribution

qk, i.e. for each pair of γ and Np, however for class A networks, r = 0, and I(0) = 0 for

all γ and Np. For a symmetric distribution qk, the Eq. (3.18) reduces to

I(r) = a |r|b + c ed |r| − c (3.19)


AS 1998 3216 642 1.36 -0.198 0.20 0.55 0.58

AS 1999 4513 1018 1.21 -0.174 0.21 0.55 0.58

AS 2000 6474 1460 1.18 -0.16 0.18 0.62 0.83

Table 3.4: Shannon information I(r) computed for Internet and their corresponding class-A and class-B networks. N is the total number of autonomous systems [1].



Scientometrics 2729 164 2.84 -0.03 0.03 0.16 0.88

Small & Griffith 1024 232 2.77 -0.193 0.08 0.37 0.76

Self-organising maps 3773 740 2.88 -0.12 0.06 0.28 0.46

Small World 233 294 2.5 -0.303 0.15 0.66 0.72

Zewail 6652 331 2.63 0.002 0.03 0.20 0.21

Table 3.5: Shannon information I(r) computed for citation networks and their correspond-ing class-A and class-B networks. N is the total number of cited papers [1].


Astro Physics 16046 360 2.71 0.235 0.16 0.58 0.58

Condensed matter 16264 107 2.79 0.185 0.05 0.24 0.52

Condensed matt.2003

30460 202 2.74 0.178 0.09 0.22 0.40

Condensed matt.2005

39577 278 2.72 0.186 0.11 0.21 0.39

High-Energy Theory 7610 50 2.97 0.258 0.27 0.29 1.89

Table 3.6: Shannon information I(r) computed for collaboration networks and their cor-responding class-A and class-B networks. N is the total number of authors [80, 84].

A similar characterisation of class B networks revealed:

I(r) =

a3 (r − r)b3 + c3 ed3 (r−r) + g3 if r ≥ r

a4 |r − r|b4 + c4 ed4 |r−r| + g4 if r < r

For a symmetric distribution qk, r = 0.

The main term of the Eqs. (3.18) and (3.19) is the information power-law ai |r|bi which

dominates the correction term ci edi |r|. The rate coefficient bi(Np, γ) is the scaling ex-

ponent of the information power-law, reflecting how the amount of Shannon information

I would change with respect to a change in assortativity r. Figure 3.7 shows the rate

coefficient b1(Np, γ) against Np for various fixed exponents γ. This dependency can be

approximated by a function which is dominated by a power law for small Np:

b1(Np, γ) = µ(γ)Nν(γ)p + λ(γ) (3.20)

where ν(γ) < 0. For example, b1(Np, 1.0) ≈ 1.3N−0.28p +1.04, and b1(Np, 3.0) ≈ 0.75N−0.6

p +

1.47.

Conversely, Figure 3.8 traces the rate coefficient b1(Np, γ) against γ for various fixed cut-


Figure 3.4: The Escherichia coli metabolic network: r = −0.162, I(r) = 0.49 bits; γ =2.1, Np = 430. Figure is drawn with Cytoscape 2.5.1.

offs Np. It can be observed that the rate b1(Np, γ) tends to plateau when 4 < γ < 5,

and quickly diminishes when γ > 5. This creates a local “stability” region on the Np × γ

surface when Np > 20 and 4 < γ < 5. The stability region is also visible in Figure 3.9 that

shows the dependency of the correction coefficient d1(Np, γ) on γ for various fixed cut-offs

Np.

It can be also observed that the correction coefficient d1(Np, γ) moves toward its minimum

as Np grows and the exponent reduces below γ = 3.0 (Figure 3.9). This indicates that

in many real-world networks that are typically characterised by larger Np’s and the range

2.0 < γ < 3.0, the correction term plays a minor role.

We verified this conjecture by considering the assortativity range −0.4 ≤ r ≤ 0.6, which

corresponds to most real-world networks presented in this chapter (including social net-

works). In this case, it was observed that there is no need for a correction term at all, and

the relationships (3.18) and (3.19) can be simplified as follows:

I(r) =

a1 rb1 if r ≥ 0

a2 |r|b2 if r < 0(3.21)


Figure 3.5: Class A network corresponding to the Escherichia coli metabolic network:r = −0.162, I(r) = 0.20 bits; γ = 2.1, Np = 430. Figure is drawn with Cytoscape 2.5.1.

For a symmetric distribution qk, the Eq. (3.21) reduces to

I(r) = a |r|b (3.22)

These equations represent the ‘information power law’. The resulting coefficients b1(Np, γ)

do not differ from the coefficients obtained by fitting the relationships (3.18) and (3.19)

and shown in Figure 3.7 and Figure 3.8 – more precisely, the difference is within 0.004 or


Figure 3.6: Class B network corresponding to the Escherichia coli metabolic network:r = −0.162, I(r) = 0.68 bits; γ = 2.1, Np = 430. Figure is drawn with Cytoscape 2.5.1.

0.26%.

3.5.1 Slope and stability regions

If one attempts to optimise networks according to their information content (analogous

to optimising communication channels), assortativity becomes the main factor: the less

non-assortative is the network, the more information it can contain. The limit on maxi-

mally attainable disassortativity rmin implies that it is easier to maximise the information

content by increasing assortativity toward r = 1, as I(r = rmin) ≤ I(r = 1).

In the following analysis (based on the rates for the lower bound, i.e. class A), we point

out that new links and new node types (i.e., different degrees) in an evolving network

affect assortativity r much more than they do the degree distribution parameters Np and

γ. In other words, it is much easier to produce and explore a candidate network with a


1.2

1.25

1.3

1.35

1.4

1.45

1.5

1.55

1.6

1.65

20 30 40 50 60

Rat

e co

effic

ient

b

Cut-off Np

’16-G = 1.0’’16-G = 3.0’’16-G = 4.0’

Figure 3.7: The dependencies between the rate coefficient b1 and cut-offs Np, for differentγ. The points indicated by arrows are coefficients b1 for Np = 1000.

1.2

1.25

1.3

1.35

1.4

1.45

1.5

1.55

1.6

1.65

1.7

0 1 2 3 4 5 6 7 8 9

Rat

e co

effic

ient

b

Power-law scaling exponent

’Np = 16’’Np = 24’’Np = 40’’Np = 60’

’Np = 100’’Np = 500’

’Np = 1000’

Figure 3.8: The dependencies between the rate coefficient b1 and power law exponents γ,for different Np.


7

8

9

10

11

0 1 2 3 4 5 6 7 8 9

Cor

rect

ion

coef

ficie

nt d

Power law scaling exponent

’Np = 16’’Np = 24’’Np = 40’’Np = 60’

’Np = 100’’Np = 500’

’Np = 1000’

Figure 3.9: The dependencies between the correction coefficient d1 and power law expo-nents γ, for different Np.

different r, rather than different Np and/or γ.

When optimisation or evolutionary processes explore the slope region, 2.0 < γ < 3.0

(Figure 3.8), there is more freedom to generate and evaluate candidate networks. In this

region, even a small change in the scaling exponent γ brings a reward in a higher rate of

information b. Thus, the information content can be changed easily by modifying only

the assortativity r, i.e. in the slope region, the information content is most sensitive

to assortativity. Hence, the search becomes more efficient, and networks with higher

information content are found more easily. This feature may explain why the exponents

γ of real-world scale-free networks are mostly within [2.0, 3.0] range.

The slope region is upper-bounded by the stability region, 3.8 < γ < 5.0 (Figure 3.8).

Within the stability region, the different curves I(r) tend to be close to each other when

one varies Np and γ. For example, the relationships between information and assortativity

for class-A network (analogous to the one shown in Figure 3.3) for γ = 4.0 and γ = 5.0

would be very similar if Np > 20. The stability region creates a further structure in the

search-space defined by Shannon information. When networks evolve (or are explored in

the search-space) by changing either or both the cut-off Np and scaling exponent γ, the

3.6 Summary 45

information content tends to stay constant in the stability region as long as the assorta-

tivity is maintained at the same level. In other words, the informational fitness landscape

of evolving networks is smoother in the stability region: the expense taken to modify Np

and/or γ is not rewarded with more freedom to produce a higher information content.

At the lower range of scaling exponent, γ < 2.0 (Figure 3.8), the freedom to vary the

information content is reduced as well — simply because the rate coefficient b is smaller

for the high (e.g. real-world) cut-offs Np.

3.6 Summary

In order to appreciate the utility of assortativity as a tool in analysing networks, it is

important to understand the relationship between assortativity and information content

of a network. In this chapter, we analysed Shannon information of undirected scale-free

networks in terms of their assortativity. First of all, we disproved the relationship proposed

in [108]. Noting that the same assortativity r could correspond to different information

values I, we introduced a classification of networks according to the dependency of the

distribution e(r=r′)j,k on the assortativity r′, with the intention that, within a class, the

same distribution qk and the same assortativity r result in the same information I(r). We

observed that the two identified classes of networks provide lower and upper bounds, in

terms of Shannon information, for the considered real world networks.

We also demonstrated that the information content of scale-free networks depends non-

linearly (and asymmetrically) on the absolute value of the assortativity. The identified

dependency is symmetric when the corresponding remaining degree distribution is sym-

metric. We further studied class A (minimalistic) networks, and identified slope and

stability regions on the Np × γ surface. In the slope region, there is more freedom to gen-

erate and evaluate candidate networks since (i) the information content can be changed

easily by modifying only the assortativity r, and (ii) even a small change in the scaling

exponent γ brings a reward in a higher rate of information b. This feature may explain

why the exponents γ of real-world scale-free networks are usually within [2.0, 3.0].

The optimisation criteria defined according to information content of networks would

allow us to advance research into network’s resilience under node removal or percola-

tion/diffusion of adverse conditions. For example, one may consider a task of information-

3.7 Appendix 46

cloning of a scale-free network [91], given its fragment and some topological properties of

the original network. The “cloning”, interpreted information-theoretically, would aim at

attaining an equivalent information content of the resulting network which may disagree

with the original one in terms of specific node to node connections. This is further explored

in chapter 9. The next chapter focuses on assortativity in directed networks.

3.7 Appendix

In order to construct a network with a specific assortativity value r, given a degree dis-

tribution pk and a network size N , we developed and used the Assortative Preferential

Attachment (APA) method [91]. In this appendix we explain the APA method.

The excess degree distribution qk is obtained using equation (2.2). We use the ej,k com-

puted by templates (3.3) — (3.16) for class A, or (3.17) for class B, to grow a desired

network. We start by creating a ‘source pool’ and ‘target pool’ of unconnected nodes, each

of size N0 = N/2, with the intention of sequentially adding the nodes from source pool

to target pool. In the traditional preferential attachment method [15], the probability of

a new link between a source and a target node depends only on the degree of the target

node. In the APA method, however, the probability would depend on the degrees of both

source and target nodes. We therefore, probabilistically assign an ‘intended degree’ k to

each node in both pools such that the resulting degree distribution is pk.

Then we assign a probability distribution µ(k, j0), . . . , µ(k, jNp−1) to each target node

with the degree k, where µ(k, j) is the probability of a source with degree j joining the

target node with the degree k. The probability µ(k, j) is calculated as µ(k, j) = ej,k/pj ,

then normalised such that∑j

µ(k, j) = 1. The distribution µ(k, j) has to be biased by

division by pj , because each source node with degree j does not occur in the source pool

with the same probability. In other words, sequential addition would not maintain ej,k,

and the biased probability µ(k, j) accounts for that. Once µ(k, j) is generated, each source

node with degree j is added to the target pool and forms a link to a target node with

degree k with probability µ(k, j).

Example 3.7.1. if there are twice as many source nodes with degree j2 than those with

degree j1 (i.e., pj2 = 2 pj1), while ek,j2 = ek,j1 , then the biased probabilities µ(k, j1) and

3.7 Appendix 47

µ(k, j2) would be such that µ(k, j2) = ek,j2/pj2 and µ(k, j1) = ek,j1/pj1 = 2µ(k, j2). This

ensures that nodes with degree j1 (represented twice as scarce as the nodes with degree

j2) would find it twice as easy to form a link with a target node which has degree k.

When a target node with k degrees forms its last, k-th, link, all its probabilities µ(k, j)

are set to zero (i.e., this node will not form any more links). The grown network will thus

have the desired joint distribution ej,k, and hence the desired assortativity r′.

Remark 3.7.1. A number of other methods have been described in literature to grow a

network with a given level of assortativity. For example, see [51, 123].

Chapter 4

Information content and

assortativity in directed networks

4.1 Introduction

In chapter 3, we analysed assortativity and information content in undirected networks.

This chapter on the other hand is concerned with assortativity in directed networks. Many

naturally occurring networks, and biological networks in particular, are directed networks.

Transcription networks[25], neural networks [117], Gene Regulatory Networks (GRNs) [4],

and brain (cortical) networks [6] fall into this category. While assortativity of some bio-

logical networks, such as food webs, has been analysed by considering them as undirected

[81], generally we can get far better insights about their topologies if their directedness is

taken into account. As we show in this chapter, biological networks that may appear dis-

assortative when directedness is not considered, do in fact become assortative when they

are considered as directed networks. Therefore, it is necessary that a sound theoretical

background is developed for analysing assortativity in directed networks. In this chapter

(and chapter 6), we attempt this task, and use our results to analyse topological patterns

in directed biological networks.

As presented in chapter 2, Newman [82] defined assortativity for directed networks as a

correlation function, similar to the definition for assortativity in the undirected case. How-

ever, the meaning of this definition is not as sound. In the undirected case, assortativity

4.2 Motivation for alternative assortativity definitions in directed networks 49

measures the tendency of a node to connect with other nodes which have similar degrees.

In directed case, the ‘in-degree’ and ‘out-degree’ of nodes come into play. According to the

definition in [82], the assortativity for directed networks measures out-degree to in-degree

correlations. It would make more sense if assortativity instead measures the tendency for

nodes to connect with other nodes with similar out-degrees or similar in-degrees. With this

in mind, we propose alternative definitions for assortativity in directed networks, namely

the in-assortativity and the out-assortativity, in this chapter. A sound background for

these new definitions is laid by analysing some canonical networks. This in turn makes

it possible to define and analyse information content in real world directed networks, and

the relationship of information content to assortativity in directed networks.

This chapter is organised as follows. Section 4.2 presents the arguments for refined def-

initions of assortativity in directed networks. In section 4.3 the new correlation metrics,

out-assortativity and in-assortativity, are defined. Section 4.4 analyses out-assortativity

and in-assortativity in complex real world networks. In section 4.5, topological informa-

tion content is defined for directed networks, and the relationship of information content

and the assortativity metrics presented here is explored. Section 4.6 presents the chapter

summary.

4.2 Motivation for alternative assortativity definitions in di-

rected networks

Assortativity in undirected networks is the tendency for nodes to connect with nodes which

have similar degrees. As we saw in chapter 2, Newman [81, 82] defined assortativity in

directed networks as

rd =1

σinq σout

q

∑

jk

jk(eout,inj,k − qin

j qoutk

) (4.1)

where eout,inj,k is the joint degree distribution, and σin

q , σoutq are the standard deviations of

the distributions qink , qout

k respectively. According to this definition, we are looking at the

correlation between out-degrees of the source nodes and in-degrees of the target nodes.

4.3 Out-assortativity and in-assortativity 50

Therefore assortativity here is the tendency of nodes to connect with other nodes, whose

in-degrees are similar to the considered node’s out-degree (or vice-versa).

Consider a directed biological network where there are regulators and regulatees, such

as gene regulatory networks [19, 48]. Say the links in such networks are directed from

regulator to regulatee. A node which has high out-degree will be a dominant regulator.

However, the impact of the regulator in the network will be maximised if the nodes that

this regulator regulates, in turn regulate a lot of other nodes, i.e they themselves have high

out-degrees. Therefore we should be interested in out-degree to out-degree correlations

in such networks. Similarly, the nodes which are most likely to have complex regulation

patterns are those nodes which are regulated by many nodes, each of which in turn are

regulated by many other nodes. To measure this tendency, we need a quantity which

measures in-degree correlations.

Such correlations cannot be measured by Eq. 4.1 for directed assortativity. In other words,

rd fails to capture the ‘cascading’ effect. We therefore need alternative assortativity coeffi-

cients, which measure the tendencies where nodes preferentially connect with other nodes

with similar out-degrees to themselves or nodes preferentially connect with other nodes

with similar in-degrees. We call these tendencies as out-assortativity and in-assortativity

of a network.

4.3 Out-assortativity and in-assortativity

Let us define out-assortativity of a network as the tendency where nodes connect with

other nodes which have similar out-degrees to themselves. In-assortativity is, on the other

hand, the tendency where nodes connect with other nodes with similar in-degrees. Note

that these definitions fit well with the ‘generic’ definition of assortative mixing, where

similarity can be interpreted in terms of any given single quantity [34, 81, 82], where as

the previous definition of vertex assortativity in directed networks defined similarity in

terms of two different quantities (out-degree and in-degree) for a node pair.

To formally define out-assortativity and in-assortativity however, we first need to define

a few concepts in terms of vertex distributions. These definitions are similar but subtly

different from the ones we have reviewed in chapter 2.


Remark 4.3.1. In chapter 2, we have defined eout,inj,k as the probability distribution of a

link going into a node with j in-degree and out of a node with k out-degree.

Remark 4.3.2. In chapter 2, we have defined qoutk as the probability distribution of a link

going out of a node with k out-degree.

Remark 4.3.3. In chapter 2, we have defined qinj as the probability distribution of a link

going into a node with j in-degree.

In addition, let us present the following new definitions ( which are contributions of this

thesis).

Definition 4.3.4. eoutj,k distribution is the probability distribution of a link going into a

node of j out-degree, and out of a node of k out-degree.

Definition 4.3.5. qoutk is the probability distribution of a link going into a node with k

out-degree.

Definition 4.3.6. einj,k is the probability distribution of a link going into a node of j in-

degree, and out of a node of k in-degree.

Definition 4.3.7. qinj is the probability distribution of a link going out of a node with j

in-degree.

It is important to appreciate the subtle differences in these distributions. These concepts

are demonstrated in Figure 6.1.

Figure 4.1: In-degrees and out-degrees of nodes with respect to a link. Note the highlightedlink leaves from a node of in-degree two, and out-degree one. It goes into a node of in-degree three, and out-degree two. Note that the highlighted link will make contributionsto eout,in

1,3 , qout1 , qin

3 , eout1,2 , ein

2,3 , qout3 and qin

2 .

Now the out-assortativity and in-assortativity of a network could be defined.


Definition 4.3.8. The out-assortativity of a network is the tendency where nodes tend to

connect with other nodes with similar out-degrees.

This is formally defined as

rout =1

σoutq σout

q

(

∑

jk

jkeoutj,k )− µout

q µoutq

(4.2)

where σoutq is the standard deviation of qout

k , σoutq is the standard deviation of qout

k of the

network. Where a network has positive rout, it means that nodes with high out-degrees

tend to connect to other nodes with high out-degrees. If a network has negative rout, it

means that nodes with high out-degrees tend to connect to nodes with low out-degrees.

Definition 4.3.9. The in-assortativity of a network is the tendency whereby nodes tend

to connect with other nodes with similar in-degrees.

In-assortativity can be formally defined as

rin =1

σinq σin

q

(

∑

jk

jkeinj,k)− µin

q µinq

(4.3)

where σinq is the standard deviation of qin

k , σqin is the standard deviation of qink of the

network. If a network has positive rin, it means that nodes with high in-degrees tend to

connect to other nodes with high in-degrees. If a network has negative rin, it means that

nodes with high in-degrees tend to connect to nodes with low in-degrees.

4.3.1 Canonical network examples

Let us first look at some trivial examples of in-assortativity and out-assortativity with a

set of canonical networks as shown in Figure 4.2. A regular lattice (a) has rout = 1.0 and

rin = 1.0. A ring (f) with directed links of uniform orientation also would have rout = 1.0

and rin = 1.0. These networks are perfectly assortative in terms of out-degree as well

as in-degree. On the other hand, one may consider a range of star networks (b,c,d,e)

including single and multi-starts. These networks all have rout = −1.0 and rin = −1.0,

thus they are perfectly disassortative in terms of in-degrees and out-degrees.

4.4 Assortativity of directed real world networks 53

In general though, in-degree assortativity need not be similar to out-degree assortativity

in a network. Indeed, a network may be perfectly assortative in terms of out-degrees

and not perfectly assortative in terms of in-degrees, or vice versa. As a simple example,

let us consider the model network in Figure 4.3. This network has rout = 1.0 and rin =

−0.411, thus displaying perfect out-assortativity yet not perfect in-assortativity. A similar

example could be thought of in terms of modified start networks where it displays perfect

disassortativity in terms of out-degrees (rout = −1.0) yet not perfect disassortativity in

terms of in-degrees, or vice-versa.

Figure 4.2: Model networks: a) grid network with links directed uniformly; b) multi-star with links directed towards the hubs; c) multi-star with links directed towards theperipheral nodes; d) star with links directed towards peripheral nodes; e) star with linksdirected towards the hub; f) ring with directed links with uniform orientation.

4.4 Assortativity of directed real world networks

Now we may set out to analyse out-assortativity and in-assortativity in real world directed

networks. For the reasons given at the beginning of this chapter, our focus shall be on

biological networks. Table 4.1 shows the assortativity coefficients of a number of directed

networks, including neural networks, Gene Regulatory Networks, transcription networks,

cortex networks, and food webs. rd is the network assortativity according to Eq. 4.1 for

directed networks, whereas rout and rin represent the out-assortativity and in-assortativity

of networks respectively. r represents the assortativity when networks are considered

undirected, and is meaningless for directed networks and provided only for comparison.


Figure 4.3: A model network with perfect out-assortativity yet imperfect in-assortativity.

A clear tendency can immediately be observed in these values. First of all, the relative

level of assortativity is largely preserved over different ways of measuring it: there are

no dramatic changes. Secondly, the networks tend to be more assortative when their in-

assortativity and out-assortativity are considered than when they are combined together.

For example, consider the neural network of C. elegans. Even though the network appears

disassortative, with rd = −23 % , its rout = +10 % and rin = −9 %, both values shifted

considerably towards the (positive) assortativity side. Similarly, if we consider Chesapeake

lower food web, the network seems disassortative with rd = −45 %, even though when rin

and rout are considered separately, the values are rin = −6 % and rout = +21 %, again

both being shifted considerably towards positive values of assortativity. Finally, note that

many of the biological networks considered here are disassortative in terms of rd, but

becomes assortative in terms of rout and in some cases also rin. Figure 4.4 captures these

tendencies. Therefore, we may conclude that there is a weaker signature of disassortative

mixing when out-degrees and in-degrees are considered separately. Indeed, most directed

networks tend to be considerably disassortative when the tendency of nodes mixing with

other nodes which have in-degrees similar to their own out-degrees is considered. However,

these networks tend to be more assortative when the tendency of nodes mixing with other

nodes which have out-degrees similar to their own out-degrees is considered, or when the

tendency of nodes mixing with other nodes which have in-degrees similar to their own


Network Size N r I(r) rd I(rd) rout I(rout) rin I(rin)Neural networksC. elegans 297 -0.15 0.42 -0.23 0.46 0.1 1.01 -0.09 0.35GRNsrat (R. norvegicus) 819 0.86 1.65 0.31 0.95 0.64 2.24 0.59 0.75human (H. sapiens) 1452 -0.03 1.08 -0.03 0.61 0.2 1.41 -0.01 0.48mouse (M. musculus) 981 0.66 1.92 0.2 0.75 0.53 1.98 0.49 0.62C. elegans 581 -0.09 0.94 -0.12 0.68 0.36 1.06 0.01 0.38A. thaliana 395 -0.04 1.07 -0.12 0.61 0.16 1.31 0.03 0.52Transcription net-worksE. coli 1147 -0.26 0.96 0.06 0.36 0.17 1.26 0.03 0.11C. glutamicum 539 -0.37 0.84 -0.04 0.31 0.09 0.22 -0.01 0.13C. jeikeium 52 -1 1 undefined 0 -1 0 -1 0C. efficiens 50 -0.64 0.86 undefined 0 -1 0 -1 0Cortical networkshuman 994 0.17 0.19 0.13 0.19 0.17 0.19 0.17 0.19Macaque monkey 71 0.02 0.97 -0.01 0.41 0.06 0.4 -0.01 0.39Macaque sensory mo-tor cortex

47 0.01 0.68 -0.02 0.4 0.03 0.45 -0.02 0.5

Cat cortex 65 0.01 0.52 -0.05 0.32 -0.03 0.37 0.09 0.4Food websChesapeake Lower 170 -0.39 0.64 -0.45 0.57 0.21 0.7 -0.06 0.49Chesapeake Upper 193 -0.33 0.38 -0.38 0.58 0.1 1.61 -0.12 0.49Crystal river C 106 -0.33 0.49 -0.48 0.63 0.08 1.38 -0.14 0.47Crystal river D 90 -0.46 0.45 -0.54 0.65 0.06 1.18 -0.18 0.34Bay wet 2216 -0.12 0.39 -0.23 0.71 0.02 2.71 0.24 0.7Bay dry 2248 -0.11 0.39 -0.23 0.75 0.03 2.78 0.25 0.74

Table 4.1: Assortativity in real world directed networks. The table shows assortativitycoefficients calculated treating the networks as undirected and directed, and the out-degree and in-degree correlations. The table also shows corresponding mutual informationquantities. The source data for the biological networks is obtained from [6],[25],[4],[117],[1].

in-degrees is considered.

Furthermore, we note that rout values tend to be even more assortative than rin values.

While in-assortativity of the networks considered is still slightly disassortative (rin < 0),

out-assortativity remains slightly or strongly on the assortative side (rout > 0).

4.5 Assortativity and information content in directed networks 56

Note also that the cortical networks show comparatively less difference when rin and rout

are considered separately, from the case when rd is calculated as a single value. This could

be because of the fact that the cortical networks have a comparatively high link density,

and thus many pairs of nodes have links connecting them in both directions. Therefore

the effect of directionality is minimised, and in-degrees and out-degrees of nodes tend to

be similar.

4.5 Assortativity and information content in directed net-

works

To further understand these tendencies when out-degree distributions and in-degree dis-

tributions are considered separately for assortative mixing, let us investigate the mixing-

related mutual information in networks. In previous sections, we defined a number of new

degree or link distributions for directed networks. These include eoutj,k , ein

j,k, qink , and qout

k .

This was in addition to the distributions already defined and used, namely eout,inj,k , qin

k , and

qoutk for directed networks. We can make some interesting observations by looking at the

information content in terms of these distributions in directed networks.

As we discussed in chapter 2, Sole and Valverde [108] considered (Shannon) mutual in-

formation contained in the network, as an indicator of the amount of general correlation

between network nodes. It was explained that the maximum attainable information con-

tent defines the network’s capacity. It was shown in chapter 3 the information content

within a scale-free network increases non linearly with the absolute value of the assorta-

tivity [93], and can be related to network’s resilience under node removal or percolation

[91],[92], as will be shown in chapter 10.

In chapter 2, mutual information in terms of excess degrees was defined for undirected

networks as:

I (q) =Np∑

j=1

Np∑

k=1

ej,k log(

ej,k

qjqk

)(4.4)

Mutual information for directed networks could be defined similarly:

4.5 Assortativity and information content in directed networks 57

I (q) =Np∑

j=1

Np∑

k=1

ej,k log

(eout,inj,k

qinj qout

k

)(4.5)

Furthermore, we can define mutual information exclusively in terms of in-degrees or out-

degrees.

Iin (q) =Np∑

j=1

Np∑

k=1

einj,k log

(einj,k

qinj qin

k

)(4.6)

Iout (q) =Np∑

j=1

Np∑

k=1

eoutj,k log

(eoutj,k

qoutj qout

k

)(4.7)

-1

-0.5

0

0.5

0 2 4 6 8 10 12 14 16 18 20

asso

rtat

ivity

network ID

Figure 4.4: Shift in assortativity coefficient when in-degrees and out-degrees are consideredseparately. The points correspond to networks in Table 1. Crosses: directed assortativitycoefficients. Filled squares: directed out-assortativity coefficients. Circles: directed in-assortativity coefficients. Note the upward shift in values when out-assortativity and in-assortativity are compared to assortativity. Note that directed assortativity is undefinedfor networks 9, 10 due to division by zero.

By defining mutual information in terms of degree-related distributions in directed net-

works, we can make interesting observations about the information content in the network

topology.

4.6 Summary 58

Table 4.1 shows the values of the various mutual information measures that we have defined

for the same set of real world networks. We may observe that the highest information is

contained in out-degree mixing patterns. That is, it is the regulators that dominate

defining the connecting patterns of the network. the in-degree mixing patterns do not

contain that much information in general, and indeed contain less information than the

quantity I (q) in many cases. We may also note that in the cases of neural networks, all

information measures seem to reveal a similar amount of information content. This is due

to the high density of links in the neural network, where the in-degree distribution and the

out-degree distribution become quite similar to the degree distribution. (In the limiting

case of fully connected networks, in-degree distribution and out-degree distribution will

be identical, and also similar to the overall degree distribution, which will have its indices

‘doubled’ but otherwise identical to in-degree and out-degree distributions. the neural

networks with high density of links tend towards this limiting case.)

We also note that the neural networks contain the least amount of information of any kind

( Iout, Iin or I (q)), whereas Gene Regulatory Networks contain the most information. This

highlights the fact that there is less randomness in topology in some biological networks

compared to others. These might be the networks where topology plays a more important

role in functionality. According to this observation, topology seems to be less important

for the functioning of neural networks comparatively, even though a number of studies

have demonstrated the role of topologies in neural networks (eg: [60, 103]. We will visit

this point again when we discuss scalar assortativity in chapter 7.

4.6 Summary

In this chapter we analysed assortative mixing in directed networks and its relationship to

topological information content. We put particular emphasis on biological networks since

most biological networks are directed and their directedness critically influences their

functionality. We introduced new assortativity coefficients, the out-assortativity rout and

in-assortativity rin, and showed how these can be meaningful measures in understanding

network topology. We observed that the studied directed networks are more assortative

when in-degree mixing and out-degree mixing are considered separately, i.e., rin and rout

are generally higher than rd. Furthermore, out-degree mixing patterns contain the highest

4.6 Summary 59

amount of Shannon information, suggesting that the out-degree mixing patterns are the

most influential in the functionality of most biological networks. We also noted that due

to the high density of links, topology and mixing patterns play a less important role in

neural networks compared to other biological networks such as gene regulatory networks.

This completes our study of network assortativity and Shannon information content in

undirected and directed networks. The next three chapters will investigate node level

(local) assortativity.

Chapter 5

Local assortativity in undirected

networks

5.1 Introduction

The preceding chapters defined assortativity for both undirected and directed networks

in terms of degrees, and analysed the relationship between assortativity and information

content of networks. It was shown that, on average, assortativity quantifies the tendency

for preferential association within the network [81, 82, 93]. This, however, means that we

can measure how assortative the network is as a whole, but not how locally-assortative

are the individual nodes, or how do they contribute to the overall network assortativity.

To understand the local structure of the network, and the recurring local motifs in the

network, it is, nevertheless, useful to investigate how network assortativity emerges from

the individual assortative or disassortative tendencies of each node.

In this chapter, we propose the novel measure of local assortativity, which is a property

of a single node and indicates how similar a node’s immediate neighbourhood is to the

overall network1. Similarity can be interpreted in different ways, and for the moment, we

choose to interpret it as similarity in degrees (Chapter 7 examines alternative approaches

to defining similarity of nodes). We concentrate on the case of undirected networks in

this chapter (Local assortativity of directed networks is dealt with in chapter 6). We1Even though it is a property of a node, we choose to call it local assortativity rather than node

assortativity because it is influenced by a node’s ‘locality’, or neighbourhood.

5.1 Introduction 61

define local assortativity as an individual node’s contribution to the network’s assortativity.

Therefore, summing local assortativity values of all individual nodes should result in the

network assortativity. We show that, local assortativity of a node can be interpreted

as a scaled difference between the average excess degree of the node neighbours and the

expected excess degree of the network as a whole. Two networks with the same network

assortativity and similar degree distributions may have entirely different local assortativity

distributions, or profiles. Thus, local assortativity profiles can give a new perspective about

the design features of a network. For example, using local assortativity profiles, we could

see whether the largest hubs in a network are mostly connected to each other, or to smaller

hubs and peripheral nodes. Interconnected giant hubs may be a sign of robustness against

targeted attacks. As another example, the local assortativity distribution may indicate

whether all peripheral nodes are connected to more hub-like nodes (i.e., indicating presence

of star motifs), or to one another (indicating chain motifs). We analyse simulated and

real-world scale-free networks (e.g., biological networks) based on their local assortativity

profiles, and highlight motif and design features within them. This will show that the local

assortativity distribution is related to a network’s robustness against targeted attacks, and

that node roles can be classified based on their local assortativity values.

Finally, we study the local assortativity profiles of a number of model and real world net-

works, demonstrating that four classes of complex networks exist: (i) assortative networks

with disassortative hubs, (ii) assortative networks with assortative hubs, (iii) disassortative

networks with disassortative hubs , and (iv) disassortative networks with assortative hubs.

The classification of networks by local assortativity profiles is an important contribution

of this thesis, and could be used in developing growth models of networks and designing

targeted attacks of networks, as shown in later chapters.

This chapter is organised as follows: Section 5.2 introduces and defines local assortativity.

Section 5.3 introduces local assortativity distributions, and these distributions are anal-

ysed for a set of model (canonical) networks. In section 5.4, we analyse local assortativity

distributions of a set of simulated scale-free and real world networks. We also discuss the

implications of these distributions for the robustness and attack tolerance of each network.

In section 5.5 we identify four classes of networks based on their local assortativity distri-

butions, and discuss the significance of this classification. Section 5.6 presents the chapter

summary.

5.2 Definition of local assortativity 62

5.2 Definition of local assortativity

It may be recalled that network assortativity for undirected networks was defined in chap-

ter 2 as:

r =1σ2

q

(

∑

jk

jkej,k)− µ2q

(5.1)

where ej,k is the joint probability distribution of the remaining degrees of the two nodes

at either end of a randomly chosen link, µq is the mean and σq is the standard deviation of

the remaining degree distribution of the network, qk. Let us now concentrate on defining

local assortativity of a node. We propose to define local assortativity as the contribution

of a given node to the network assortativity, which means we need to determine how much

each node contributes to the term 1σ2

q

[(∑jk

jkej,k)− µ2q

]. Let us first look at the term

∑jk

jkej,k and the contribution of each node to this term. Suppose we visit all the nodes

in a network, and from each node in turn we visit all the links of that node. In a network

with N nodes and M links, the total visits we will thus make will be 2M , since each link

will be visited twice, once from each end. Suppose we build up the probability distribution

ej,k as we make these visits. Each link will add a probability of (1/2M) to the pair of (j, k)

where j and k are the remaining degrees of nodes at each end of the link. Thus, each visit

to a link will contribute jk/2M to the sum∑jk

jkej,k. Therefore, if we consider a node

with remaining degree j (Figure 5.1) which is connected to nodes with remaining degrees

k1, k2, . . . kj+1, it will contribute (jk1/2M)+ (jk2/2M)+ . . .+(jkj+1/2M) = j2M

j+1∑i=1

ki to

the sum∑jk

jkej,k. Let us denote the average remaining degree of a node’s neighbours as

k = 1j+1

j+1∑i=1

ki. Then we can represent the individual node’s contribution, αv, to the sum∑jk

jkej,k as

αv =j

2M

j+1∑

i=1

ki = (j + 1)jk

2M(5.2)

Now let us consider a node’s contribution to the term µ2q . To do so, let us first look at the


Figure 5.1: The considered node v has one remaining degree (j = 1), i.e., it has twoneighbours: v1 and v2 with three and two remaining degrees respectively (k1 = 3 andk2 = 2). The average remaining degree of the neighbours is k = 2.5.

following equivalent definitions of µq:

µq =1

2M

2M∑

m=1

km (5.3)

=1

2M

N∑

v=1

kv(1 + kv) (5.4)

where k is excess degree, m is a given end of an edge and v is a given node of the network.

We are especially interested in the last form (5.4) since it makes it obvious what each node

contributes to the term µq. It follows that

µq =1

2M(

N∑

v=1

kv +N∑

v=1

kv2) (5.5)

yielding

µ2q =

14M2

((N∑

v=1

kv)2 + (N∑

v=1

kv2)2 + 2

N∑

v=1

kv

N∑

v=1

kv2) (5.6)

Now, let us consider a single node (without loss of generality, let it be the node 1 with excess

degree k1), and its contribution to each of the three summation terms in the expression

above. Considering the first summation term, excess degree k1 contributes to it as follows:

k12 + 2(k1k2 + k1k3 + ........... + k1kN ) (5.7)

Among these, terms such as 2k1kj have to be ‘divided’ between node 1 and node j respec-

tively. These are multiplication terms, and we assume that an equal division is appropriate.


Therefore, we can consider that contribution of node 1 is:

k12 + (k1k2 + k1k3 + .... + k1kN ) = k1

N∑

j=1

kj (5.8)

Similarly considering the second summation term in 5.6, we may observe that the contri-

bution of node 1 is k12

N∑j=1

kj2.

Let us analyse the contribution of node 1 to the third summation term in (5.6). The third

summation term is given by

2N∑

i=1

ki

N∑

j=1

kj2 = 2(k1 +

N∑

i=2

ki)(k21 +

N∑

j=2

kj2) (5.9)

where i, j are node indices. The contribution of node 1 to the third term, again dividing

terms such as 2k1kj between node 1 and node j respectively, is

= 2k31 + k2

1

N∑

i=2

ki + k1

N∑

j=2

kj2 (5.10)

= k1

N∑

j=1

kj2 + k1

2N∑

j=1

kj (5.11)

Therefore, the total contribution of node 1, β1, to µ2q is given by:

β1 =

k1

N∑j=1

kj + k12

N∑j=1

kj2 + k1

N∑j=1

kj2 + k1

2N∑

j=1kj

4M2(5.12)

This can be further regrouped as

β1 =1

4M2(k1 + k1

2)(N∑

j=1

kj +N∑

j=1

kj2) (5.13)

Using equation (5.5) for µq, this can be reduced to:

β1 =k1 + k1

2

2Mµq (5.14)


Therefore, the contribution of a given node v to the term µ2q can be given by:

βv = (j + 1)jµq

2M(5.15)

where j is the excess degree of the node v.

The standard deviation is already a scaling term, and we need not worry about a single

nodes’ contribution to it.

Consequently, local assortativity of a node could be formally defined as the difference

between αv and βv, scaled by σ2q .

Definition 5.2.1. The local assortativity of node v is defined as:

ρv =αv − βv

σ2q

=j (j + 1)

(k − µq

)

2Mσ2q

(5.16)

where j is the node’s remaining degree, k is the average remaining degree of its neighbours,

and σq 6= 0.

By including the scaling term, we ensure that the equation for local assortativity satisfies

the condition

r =N∑

v=1

ρv (5.17)

If standard deviation σq is zero, then the definition (5.16) cannot be applied. This case,

however, can arise only if the network is homogeneous, i.e all nodes in the network have

the same degree, and network assortativity r = 1. Since local assortativity is defined as

the contribution of each node to the network assortativity, we state that in this case all

contributions are equal, and the local assortativity is set to ρ = 1/N .

Note that the sign of the local assortativity (positive or negative) is determined by the

difference between the average excess degree (k) of the neighbours and the global average

excess degree (µq). If the neighbours’ average is higher, then the node is assortative. If

the global average is higher, the node is disassortative. Therefore, the local assortativity

can also be interpreted as a scaled difference between the average excess degree of the

node’s neighbours and the global average excess degree. In other words, a node is locally

assortative if it is surrounded by nodes with ‘comparatively’ high degrees.

5.3 Local assortativity distributions 66

5.3 Local assortativity distributions

Since local assortativity is a property of a node, it is possible to construct local assortativity

distributions for a given network, plotting local assortativity values against degrees. Since

nodes with the same degree can have various local assortativity values, we could represent

either the total of the local assortativity values of nodes with a given degree k, or the

average local assortativity value for all nodes with a given degree k. In this thesis, we

calculate the average value of local assortativity for all the nodes in a network with a given

degree k, denoting this value as ρk. Therefore, if we denote by N(k) the number of nodes

with degree k, then the network assortativity is

r =∑

kN(k)ρk (5.18)

where ρ(k) is the average local assortativity of all nodes with degree k. It is easy to see

that this equation can also be written in the form:

r = N∑

kpkρk (5.19)

where pk represents the degree distribution of the network. Henceforth, we shall mostly

use the “average local assortativity vs degree”, ρk vs k, distributions. We will also look

at the local assortativity of individual nodes vs node ID (ρv vs vID) distributions.

Now let us look at the local assortativity distributions of some classical network structures

before analysing real world networks and their local assortativity.

5.3.1 Local assortativity in model networks

Regular lattice

Lattice like networks are common in some human-designed architectures, particularly

parallel computers [108]. For a lattice like network each node has the same degree and

remaining degree, therefore the variance of remaining degree distribution is 0. Since there

is only one type of nodes, the network is perfectly assortative (r = 1) and the local

assortativity of all nodes is 1/N . Therefore the local assortativity distribution resembles

a Kronecker delta function (Figure 5.2).

5.3 Local assortativity distributions 67

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0 5 10 15 20

aver

age

loca

l ass

orta

tiven

ess

degree

Figure 5.2: Local assortativity distribution, ρ(k) vs k, of a regular lattice with four nodesconnecting to each node (squares), and of a star graph (stars). Network size in both casesis N = 20.

Star Network

Star graph is another extreme example of complex networks in terms of topology. It is a

common motif, responsible for the small world feature in many networks, and an optimal

motif for low cost communication [108].

In a pure star graph, any given link has a peripheral node at one end, with degree one

(i.e remaining degree zero). It can be shown that a star graph is perfectly disassortative

(r = −1). Furthermore, any node in the star graph has either its remaining degree as

zero, or all of its neighbours’ remaining degrees as zero. It is easy to see that the term

represented by Eq. 5.2 reduces to zero in all cases. Thus the local assortativity reduces to

ρ = −j(j + 1)2M

µq

σ2q

(5.20)

Figure 5.2 shows the local assortativity distribution for a pure star graph: the central node

is much more locally-disassortative, as it connects with many dissimilar nodes, whereas the

peripheral nodes are less locally-disassortative since they connect to only one dissimilar

node.

5.4 Local Assortativity in Scale-free networks 68

5.4 Local Assortativity in Scale-free networks

Now we proceed to analyse the local assortativity distributions for simulated and real

world scale free networks. Most real world networks, including biological networks, social

networks and technological networks including Internet and World Wide Web are scale-

free networks with power law degree distributions [15, 41, 42] as mentioned in chapter

2. Hence we focus mainly on scale-free networks and their local assortativity profiles.

First of all, we look at simulated scale-free networks with various network assortativity

values, concentrating on nearly perfect assortativity, nearly perfect disassortativity, and

non-assortativity cases (r ≈ 1, r ≈ −1 and r ≈ 0 respectively)2. The networks are

simulated using the Assortative Preferential Attachment (chapter 3). Some typical results

are shown in Figures 5.3 and 5.43. The local assortativity for all nodes (ρv vs vID) is

shown as a scatter plot in Figures 5.5, 5.6, and 5.7.

-0.001

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0 2 4 6 8 10 12 14 16 18

avg.

loca

l ass

orta

tivity

degree

Figure 5.3: Local assortativity distributions for assortative networks (r ≈ 1, stars; r ≈ 0.5,squares) and non-assortative networks (r ≈ 0, empty squares): ρk vs k.

A number of interesting facts can be observed from these distributions. First of all, there

is a large number of disassortative nodes, ρ < 0, in any network, regardless of whether the2For comparison, we also present r ≈ 0.5 in Figure 5.3 and r ≈ −0.5 in Figure 5.43Even though r ≈ 0, it is rarely absolutely zero, and in the particular example shown in Figure 5.3 it

is positive

5.4 Local Assortativity in Scale-free networks 69

-0.007

-0.006

-0.005

-0.004

-0.003

-0.002

-0.001

0

0.001

0 2 4 6 8 10 12 14 16 18

avg.

loca

l ass

orta

tivity

degree

Figure 5.4: Local assortativity distribution for disassortative networks (r≈-1, stars; r ≈−0.5, squares): ρk vs k.

network is overall assortative, disassortative, or non assortative. Conversely, the number of

assortative nodes, ρ > 0, in a disassortative network is very small, or even zero. Secondly,

a large proportion of nodes in any network fall on the “slightly disassortative” area (just

below the zero axes in the scatter plots - see Figures 5.5, 5.6 and 5.7). These correspond to

peripheral nodes, which must exist in any type of scale-free network. The hubs however,

interestingly, have either high local assortativity or high local disassortativity. Specifically,

the hubs seem to be highly assortative in assortative (r ≈ 1) or non-assortative (r ≈ 0)

networks, whereas the hubs in disassortative networks (r ≈ −1) are highly disassortative

themselves. Thus the assortativity or disassortativity of the hubs seem to determine the

assortativity or disassortativity of the network to a large extent (though not in all networks,

as we will see later in the chapter), even though hubs are much smaller in number than

peripheral nodes. Finally, we observe that when the network is very assortative or very

disassortative, the nodes can be more easily clustered together based on local assortativity

(the horizontal stratification into levels of nodes, observed in Figure 5.5 and 5.6). There

are clear “levels” of hubs of a similar local assortativity, as well as provincial hubs and

inner peripheral nodes. Thus, local assortativity can be used to classify nodes based on

their function in these cases. In contrast, when the network is non-assortative (r≈0), nodes

5.5 Classification of networks using local assortativity 70

-0.0005

0

0.0005

0.001

0.0015

0.002

0.0025

0.003

0.0035

0 2000 4000 6000 8000 10000

Nod

e lo

cal a

ssor

tativ

enes

s

Node ID

Figure 5.5: Local assortativity distribution for assortative networks (r≈ 1): ρv vs vID.

cannot be easily grouped based on their local assortativity. There is no stratification into

levels (clusters) clearly visible in Figure 5.7. Furthermore, a small number of giant hubs

are likely to emerge in non-assortative networks (see Figure 5.7). When the network is

more assortative or more disassortative, a larger number of smaller hubs replace this small

group of giant hubs.

Why is it the case that the giant hubs are more likely to emerge in relatively non-assortative

networks? In assortative networks, hubs tend to connect to each other (this must happen

for high assortativity), preventing a single giant hub from emerging. In disassortative

networks, there are many smaller isolated hubs which are not connected to each other

(many star motifs). If these hubs make connections between each other or other provincial

hubs the disassortative nature begins to be compromised. It is therefore in non-assortative

networks that single giant hubs can emerge. This implies that non-assortative scale-free

networks (r ≈ 0) are more vulnerable to targeted attacks which destroy hubs. We will

elaborate more on this in chapter 10.


-0.0014

-0.0012

-0.001

-0.0008

-0.0006

-0.0004

-0.0002

0

0.0002

0 2000 4000 6000 8000 10000

Nod

e lo

cal a

ssor

tativ

enes

s

Node ID

Figure 5.6: Local assortativity distribution for disassortative networks (r≈ -1): ρv vs vID.

5.5 Classification of networks using local assortativity

In the previous section, we constructed model Barabasi–Albert scale-free networks [15] of

various assortativity levels and observed their local assortativity profiles. Specifically, we

used the Assortative Preferential Attachment method (APA) [91] to control the level of

assortativity, and observed as shown in Figures 5.3 and 5.4 that the profiles are more or

less symmetric with respect to the degree axis when assortativity is varied from r = 1.0 to

r = −1.0 while other network parameters are kept constant. We also noted that (i) globally

assortative networks have assortative hubs and disassortative peripheral nodes, and (ii)

globally disassortative networks have disassortative hubs and assortative peripheral nodes.

That is, the overall assortativity of the network is matched by that of the hubs. Thus,

we are able to classify the constructed model networks as either (i) assortative networks

with assortative hubs, or (ii) disassortative networks with disassortative hubs. This is not

surprising. However, one may ask whether there are also any disassortative networks with

assortative hubs, and vice-versa.


-0.0005

0

0.0005

0.001

0.0015

0.002

0 2000 4000 6000 8000 10000

Nod

e lo

cal a

ssor

tativ

enes

s

Node ID

Figure 5.7: Local assortativity distribution for non-assortative networks (r≈0): ρv vs vID.Giant hubs are marked by arrows.

Network assortativity r class

Human metabolic [7] 0.382 assortative with assortative hubs

Chimpanzee metabolic [7] 0.398 assortative with assortative hubs

Rhesus monkey metabolic [7] 0.363 assortative with assortative hubs

Astrophysics collaboration [80] 0.276 assortative with assortative hubs

Cond. mat. 2003 collaboration [84] 0.178 assortative with assortative hubs

Cond. mat. 2005 collaboration [84] 0.186 assortative with assortative hubs

High Energy Theory collaboration [84] 0.293 assortative with disassortative hubs

Network science collaboration [84] 0.46 assortative with disassortative hubs

H. sapien PPI [3] 0.075 assortative with disassortative hubs

E. coli PPI [3] 0.056 assortative with disassortative hubs

Internet AS 1998 [5] -0.198 disassortative with disassortative hubs

Internet AS 2008 [5] -0.198 disassortative with disassortative hubs

D. melanogaster PPI [3] -0.21 disassortative with disassortative hubs

M. musculus PPI [3] -0.057 disassortative with disassortative hubs

Crystal River C [1] -0.334 disassortative with disassortative hubs

Crystal River D [1] -0.467 disassortative with disassortative hubs

Lower Chesapeake [1] -0.391 disassortative with disassortative hubs

Scientometrics citation [1] -0.03 disassortative with disassortative hubs

Small, Griffith and Des. citation [1] -0.193 disassortative with disassortative hubs

Table 5.1: The real world networks studied and their classification.


Figure 5.8: Example of (a) an assortativenetwork with assortative hubs. H. sapienmetabolic network; r = 0.382. (b) an assor-tative network with disassortative hubs. H.sapien Protein Protein Interaction network;r = 0.075. (c) a disassortative network withassortative hubs. A model network with r= -0.109. (d) a disassortative network withdisassortative hubs. Crystal River D food-web; r = -0.467.

-0.01

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0 10 20 30 40 50 60 70

avg.

loca

l ass

orta

tivity

degree

-0.015

-0.01

-0.005

0

0.005

0.01

0.015

0 5 10 15 20 25 30 35 40

avg.

loca

l ass

orta

tivity

degree

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

0.05

1 2 3 4 5 6 7

avg.

loca

l ass

orta

tivity

degree

-0.3

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0 5 10 15 20 25 30

avg.

loca

l ass

orta

tivity

degree

Figure 5.9: Corresponding local assortativ-ity profiles of (a) an assortative networkwith assortative hubs. H. sapien metabolicnetwork. (b) an assortative network withdisassortative hubs. H. sapien Protein Pro-tein Interaction network. (c) a disassorta-tive network with assortative hubs. A modelnetwork. (d) a disassortative network withdisassortative hubs. Crystal River D foodweb.


To answer this question, let us first look at the model network given in Figure 5.8 c.

This network is made up of a number of interconnected star-like subnetworks. Each

subnetwork has a ‘core’ of hubs that are densely connected to one another (this is the so-

called ‘rich club phenomenon’ [39, 126]). The rest of the subnetwork seems to have mostly

disassortative connections. The subnetworks are then linked together with hub-to-hub

connections, further reinforcing the rich-club phenomenon. The overall assortativity of the

network is r = −0.109. However, as shown in Figure 5.9 c, the hubs are assortative. The

embedded subnetworks pattern can be repeated on larger scales, retaining the assortative

hubs with higher and higher degrees, while keeping the overall disassortativity. This

example demonstrates that it is possible to have disassortative networks with assortative

hubs, when the hubs show strong connectedness between one another but the rest of the

network has disassortative connections. This network represents a third class, where the

network is disassortative yet has assortative hubs.

Now let us consider some real world networks. The real world networks we studied included

most recent metabolic networks ( Kyoto Encyclopedia of Genes and Genomes (KEGG)

database), citation networks, Protein-Protein Interaction (PPI) networks, food-webs, and

Internet AS level networks among others. A list of the networks we analysed is shown in

Table 5.1. We were able to observe the following from our analysis.

Firstly, as in the case of model APA networks, some real-world assortative networks have

assortative hubs (for example, Figure 5.8 a; most other metabolic networks showed similar

profiles). Also many real-world disassortative networks have disassortative hubs, e.g., one

such food-web is shown in Figure 5.8 d. However, other assortative networks exhibit

disassortative hubs, such as the PPI networks of H. sapien (human) shown in Figure 5.8

c. A number of other PPI networks displayed a similar profile. These networks represent

the fourth class, namely the assortative networks with disassortative hubs.

Therefore, we can identify four classes of complex networks, namely: (i) assortative net-

works with assortative hubs, (ii) assortative networks with disassortative hubs, (iii) disas-

sortative networks with assortative hubs, (iv) disassortative networks with disassortative

hubs. There are several examples of real world networks for each of the first, second and

fourth cases, and we have shown representative examples of their local assortativity dis-

tributions in Figures 5.8 a, 5.8 b, and 5.8 d respectively . We did not find any real world

example of the third case among the networks we studied, however we have demonstrated

5.6 Summary 75

that in theory such networks could exist, as shown in the profile in Figure 5.9 c, and

real-world examples may yet be found as the range of networks studied is expanded. Note

that the networks with assortative hubs and disassortative hubs are not always visually

distinguishable, however the local assortativity profiles are able to highlight an important

topological difference in them.

5.6 Summary

Many real-world networks, including biological and technical networks, display degree as-

sortativity or disassortativity, where nodes preferentially connect to each other based on

their degrees. There can be various motifs or subnetworks within the overall network,

whose assortativity might be markedly different from the overall network assortativity4.

In other words, various nodes in a network contribute in different ways to network assor-

tativity, and in some cases, these nodes may display assortative tendencies even though

the network is overall disassortative, or vice versa.

In this chapter, we formally defined the novel measure of local assortativity to quantify

a node’s contribution to network’s overall assortativity, and studied local assortativity

profiles for both simulated and real networks. We showed that a node’s local assortativity

is proportional to the difference between the average excess degree of its neighbours and

the network’s overall average excess degree. Specifically, a node is locally assortative if

its neighbours have comparatively (i.e., compared with all nodes in the network) higher

degrees. It is important to realise that, in general, the nodes with the highest local

assortativity differ from the largest hubs (the nodes with the highest degrees). We pointed

out that any scale-free network must have a large number of locally-disassortative nodes in

it, but it may not have any locally-assortative nodes at all. The non-assortative networks

were shown to be more likely to have giant hubs, and therefore to be most vulnerable to

targeted attacks. In practical terms, local assortativity can be used to cluster nodes based

on their relative importance when faced with targeted attacks against the network.

Analysing a range of model and real-world networks, we observed four classes of networks,

namely: (i) assortative networks with assortative hubs, (ii) assortative networks with dis-4For example, a star motif by itself is disassortative, even though an assortative network might have

many star motifs in it

5.6 Summary 76

assortative hubs, (iii) disassortative networks with assortative hubs, and (iv) disassortative

networks with disassortative hubs. Real-world examples for three classes were identified,

and a model network was constructed as an example for the fourth class (class iii).

The local assortativity profiles provide an additional quantitative tool for analysis of net-

work topologies. For instance, these profiles highlight important topological differences in

otherwise seemingly indistinguishable networks. This may help in understanding global

network properties and dynamics: e.g., (a) growth models for real-world networks may

be constructed in such a way that the grown artificial networks not only satisfy global

characteristics, but also agree with local assortativity profiles of the real-world counter-

parts [93]; (b) robustness of networks may be analysed in terms of an attack targeting

the nodes with higher local assortativity; (c) motifs within networks can be studied in

terms of their average local assortativity; (d) The role of the nodes with the highest local

assortativity in regulatory processes (e.g., reaction cascades) may be highlighted. Some of

these applications are demonstrated in the following chapters.

Chapter 6

Local assortativity in directed

networks

6.1 Introduction

This chapter is concerned with local assortativity in directed networks. As discussed

earlier, specific assortativity definitions are needed in the case of directed networks, such

as out-assortativity and in-assortativity. Since local assortativity has been introduced

for undirected networks, it is logical to extend this concept to directed networks, and

in particular, to analyse the contribution of individual nodes to out-assortativity and in-

assortativity. Moreover, this chapter is focused on directed biological networks, since local

assortativity profiles of biological networks are much more informative when directness is

taken into account. We also highlight in this chapter how local assortativity could be used

in biological networks to understand the functionality of individual nodes.

This chapter is organised as follows: Section 6.2 introduces and defines local assortativity

in directed networks. In section 6.3 we introduce the corresponding local assortativity

distributions. In section 6.4 we look at a set of canonical networks to illustrate these

concepts. Section 6.5 analyses local assortativity in real world directed networks, with

a particular focus on biological networks. In section 6.6 we highlight the utility of local

assortativity in understanding the functionality of nodes in biological networks. Section

6.7 presents the chapter summary.

6.2 Defining local assortativity in directed networks 78

6.2 Defining local assortativity in directed networks

Let us recall first the definition of assortativity in directed networks as presented in chapter

2:

rd =1

σinq σout

q

(

∑

jk


q µoutq

(6.1)

where eout,inj,k is the joint degree distribution, µin

q , µoutq are the means of the distributions

qink , qout

k respectively. Similarly, σinq , σout

q are the standard deviations of the respective

distributions.

Our criticism of this definition aside, let us first look defining the corresponding local

assortativity. As before, we propose to define local assortativity as the contribution each

node makes to the network assortativity, where network assortativity is given by equation

6.1. That is, we need to determine how much contribution each node makes to the term

1σin

q σoutq

(

∑

jk


q µoutq

First of all, we point out that the degrees j and k in this expression are understood to be

the out-degree of the source node jout and in-degree of the target node kin respectively.

We use k to indicate properties of ‘target’ nodes, and j to indicate properties of ‘source’

nodes: this is meaningful when considering a link (j, k). When considering properties of

any individual node, such as its in-degrees and out-degrees, we stay with the notation kout

for out-degrees and kin for in-degree, although this is not strictly necessary1.

Consider the contribution of each node to the term∑jk

jkeout,inj,k . As explained in chapter

5, suppose we visit all nodes in a network, and in turn from each node, we visit all links

that depart from that node, keeping the directionality of the links in mind. In a network

with N nodes and M links, the total number of visits we will thus make will be M. Again,

lets assume we build up the probability distribution eout,inj,k as we make these visits. Each

link will add a probability of 1M to the pair of (j, k), where k is the in-degree of the target

1 That is, for a given node, kout and jout should both be understood uniquely as out-degrees, while kin

and jin should both refer to in-degrees.


node, and j is the out-degree of the source node, in accordance with the definition of

eout,inj,k . Thus, each visit to a link will contribute jk/M to the sum

∑jk

jkeout,inj,k . Therefore,

considering a node with out-degree j which is connected to nodes with in-degrees k1,

k2, . . . kj it will contribute

α1 = (jk1/M) + (jk2/M) + . . . + (jkj/M) =j

M

j∑

i=1

ki

to the sum∑jk

jkeout,inj,k . Let us denote the average in-degree of a nodes’ neighbours as

kin = 1j

j∑i=1

ki. Then the individual node’s contribution to the sum∑jk

jkeout,inj,k is

α1 =j

M

j∑

i=1

ki = jjkin

M

Noting that j is the out-degree of the node concerned, it is clearer now to denote it as

jout. Therefore

α1 = (jout)2kin

M

An alternative definition is also possible, if we considered each node and all links that

come into that node in turn. In this case, we can show that the contribution to the sum∑jk

jkeout,inj,k is:

α2 = (kin)2jout

M(6.2)

Here jout is the average neighbour out-degree, neighbours being those nodes from which

this node can be reached, and kin is the in-degree of the node concerned. Therefore, let

us say that the ‘average’ contribution of a node to the term∑jk

jkeout,inj,k is the average of

the above two quantities, α1 and α2. We will note this as αd. Therefore

αd = (jout)2kin

2M+ (kin)2

jout

2M(6.3)

Now let us consider a node’s contribution to the term µinq µout

q . It can be seen that the


expectation of distribution qink , can be written as in two equivalent forms:

µinq =

1M

M∑

m=1

kinm =

1M

N∑

v=1

(kinv )2 (6.4)

where kin is the in-degree of a source node of a given link m (the first form), or the

in-degree of a given source node v (the second form). Similarly

µoutq =

1M

M∑

m=1

koutm =

1M

N∑

v=1

(koutv )2 (6.5)

where kout is the out-degree. The last two expressions lead to

µinq µout

q =1

M2

N∑

v=1

(kinv )2

N∑

v=1

(koutv )2 (6.6)

Without loss of generality, let us consider the contribution of node 1 to the above expres-

sion. Eq. 6.6 rewritten as:

1M2

((kin

1 )2 +N∑

v=2

(kinv )2

)((kout

1 )2 +N∑

v=2

(koutv )2

)(6.7)

We assume that a term such as (kin1 )2 is contributed fully by node 1, whereas a multiplica-

tion term such as (kin1 )2(kin

2 )2 is contributed to equally by node 1 and node 2. Therefore,

node 1 contributes

(kin1 )2(kout

1 )2

M2+

(kout1 )2

N∑v=2

(kinv )2

2M2+

(kin1 )2

N∑v=2

(koutv )2

2M2(6.8)

This yields1

2M2

((kout

1 )2N∑

v=1

(kinv )2 + (kin

1 )2N∑

v=1

(koutv )2

)(6.9)

and using expressions (6.4) and (6.5), can be further reduced to

12M

((kout

1 )2 µinq + (kin

1 )2 µoutq

)(6.10)

Thus we obtain the contribution of node v to the term µinq µout

q as


βd =1

2M(kout

2µinq + k2

inµoutq ) (6.11)

The standard deviations are already used as scaling terms, so we need not worry about their

contributions. Therefore, we can now define a node’s contribution to directed assortativity

of a network, represented by equation (6.1), by using αd, given by expression (6.3), and

βd, given by expression (6.11):

ρd =αd − βd

σinq σout

q

(6.12)

Definition 6.2.1. A node’s contribution to the assortativity of a directed network rd is

defined as:

ρd =1

2Mσinq σout

q

(kout2(kin − µin

q ) + kin2(jout − µout

q )) (6.13)

where kout is the out-degree of the node under consideration, kin is the average in-degree

of its neighbours (to which node v has an edge) and jout is the average out-degree of its

neighbours (from which node v has an edge) . σinq 6= 0, σout

q 6= 0.

By including the scaling terms σinq and σout

q , we ensure that the equation for local assor-

tativity for a directed network satisfies the condition

rd =N∑

i=1

ρd (6.14)

It may also be illustrative to look at ρd as the average of two quantities:

ρ1 =1

Mσinq

σoutq (kout

2(kin − µinq )) (6.15)

and

ρ2 =1

Mσinq

σoutq (kin

2(jout − µoutq )) (6.16)

In general ρ1 and ρ2 are not equal for individual nodes.


6.2.1 Motivation for alternative local assortativity definitions

It was demonstrated in chapter 4 that the definition of assortativity given by Eq. 6.1 is

inadequate to analyse assortative mixing in directed networks, and alternative correlation

coefficients out-assortativity and in-assortativity were proposed. Using the same argu-

ments, we may present a case for defining local out-assortativity and local in-assortativity.

For example, let us consider a directed biological network where there are regulators and

regulatees, such as gene regulatory networks. A node which has high out-degree will be

a dominant regulator. However, the impact of the regulator in the network will be max-

imised if the nodes that this regulator regulates, in turn regulate a lot of other nodes, i.e

they themselves have high out-degrees. Therefore, to understand the importance of such

nodes in the networks, we need a quantity that favours nodes that have high out-degree

and are connected to other nodes with high out-degrees. Similarly, the nodes which are

most likely to have complex regulation patterns are those nodes which are regulated by

many nodes, each of which in turn are regulated by many other nodes. To measure this

tendency, we need a quantity which favours nodes with high in-degree which are (direc-

tionally) connected to other nodes with high in-degrees.

Such quantities cannot be obtained by decomposing Eq.6.1 for directed assortativity. How-

ever, they can be obtained by decomposing out-assortativity and in-assortativity, as shown

in the following section.

Figure 6.1: In-degrees and out-degrees of nodes with respect to a link. Note the highlightedlink leaves from a node of in-degree two, and out-degree one. It goes into a node of in-degree three, and out-degree two.


6.2.2 Local out-assortativity and local in-assortativity

Now we may define local assortativity for directed networks in terms of exclusively out-

degrees and exclusively in-degrees. We define local out-assortativity of a node as a node’s

contribution to the network’s out-assortativity. The derivation is presented in Appendix.

Definition 6.2.2. The local out-assortativity of a node is given by

ρout =jout

2Mσoutq σout

q

(jout(kout − µout

q ) + kin(jout − µoutq )

)(6.17)

where jout is the node’s out-degree, kin is the node’s in-degree, kout is the average out-degree

of the ‘target’ neighbours to which this node has a directed link, and jout is the average

out-degree of the ‘source’ neighbours from which this node is reachable via a directed link.

Furthermore, µoutq , µout

q are the expectations of the distributions qoutk and qout

k respectively;

σoutq , σout

q are the standard deviations of the same quantities.

Local out-assortativity can be interpreted in the following way. It is a linear combination of

two terms, (kout−µoutq ) and (jout−µout

q ). The first term represents the difference between

the average out-degree of target nodes from this node, and the average out-degree of target

nodes globally (that is, the expected out-degree of a node at the end of a directed link).

Similarly, the second term represents the difference between the average out-degree of

source nodes that are neighbours to this node, and the average out-degree of source nodes

globally. That is, both terms compare the local average with the global average. The

overall local out-assortativity is a scaled linear combination of these terms. Therefore,

ρout is increased if local average of a node’s neighbours, in terms of out-degrees, is higher

than the global average. On the other hand, ρout is reduced, if the global average of out-

degrees is higher than the local averages around a given node. In this case, the node tends

to become locally out-disassortative. This interpretation is similar to local assortativity

in the undirected case, as proposed in chapter 5.

Similarly, we may define local in-assortativity of a node as a node’s contribution to the

network in-assortativity.

Definition 6.2.3. Local in-assortativity of a node is given by

ρin =kin

2Mσinq σin

q

(kin(jin − µin

q ) + jout(kin − µinq )

)(6.18)


where jout is the node’s out-degree, kin is the node’s in-degree, kin is the average in-degree

of the ‘target’ neighbours to which this node has a directed link, and jin is the average

in-degree of the ‘source’ neighbours from which this node is reachable via a directed link.

µinq , µin

q are the expectations of the distributions qink and qin

k respectively, and σinq , σin

q are

the standard deviations of the same quantities.

Note that the interpretation of ρin, in terms of the differences between local and global

averages, is similar to that given for ρout above.

The local out-assortativity and local in-assortativity indeed satisfy the sum rules

rout =N∑

i=1

ρout (6.19)

rin =N∑

i=1

ρin (6.20)

6.2.3 Singularity cases of directed local assortativity

When defining directed local assortativity, it is important to carefully consider what hap-

pens when one of the degree distributions, qink , qin

k , qoutk , qout

k , are Kronecker δ functions

(i.e., only one type of degree exists), making the variance zero. A few combinations are

worthy of attention here.

Let us consider the in-assortativity. Suppose σinq and σin

q are both zero. Let us then look

at the degree kin for which qink , qin

k are non-zero. If both these distributions are non-zero

at the same point (kin), then we need to obtain network assortativity equal to 1, since all

nodes have the same in-degree. Therefore, in this case we define local in-assortativity of

a node as;

ρin =kin

M=

1N

(6.21)

where kin is the in-degree of the node concerned.

Let us note that both these distributions cannot be Kronecker δ functions and non-zero

at different points (for different kin ). If qink is a δ function, then there is only one type

of in-degrees present in the network, and qink also must be a δ function and be non-zero


at the same point for the same value of kin. For the same reason, if σinq = 0 then it must

follow that σinq = 0 too.

Similarly, if σoutq = 0, then σout

q would have to be zero too, and we can define ρout as

ρout =kout

M=

1N

(6.22)

Let us point out, however, that in real-world networks of reasonable size these singularity

conditions rarely occur.

6.2.4 Distributions of local assortativity

Since local assortativity is a property of a node, it is possible to construct local assorta-

tivity distributions for a given directed network, plotting local assortativity values against

degrees (in-degrees or out-degrees). As was done in chapter 5, we may calculate the aver-

age local assortativity value for all nodes with a given in-degree kin, or a given out-degree

kout. We propose that ρout should be plotted against out-degree, since this quantity mea-

sures the contribution of a node to the out-degree correlation, while ρin should be plotted

against in-degree, since this quantity measures the contribution of a node to the in-degree

correlation. ρd is plotted against node degree. If we denote by N(kout) the number of

nodes with out-degree kout, by N(kin) the number of nodes with in-degree kin, and by

N(k) the number of nodes with degree k, the following equations hold true:

rd =∑

kN(k) ρd (k) (6.23)

where ρd(k) is the average ρd of all nodes with degree k;

rout =∑

kout

N(kout) ρout (kout) (6.24)

where ρout(kout) is the average ρout of all nodes with out-degree kout;

rin =∑

kin

N(kin) ρin (kin) (6.25)

where ρin(kin) is the average ρin of all nodes with in-degree kin.

6.3 Local assortativity in canonical networks 86

In the following sections, we will mainly consider ρout vs kout distributions, and ρin vs kin

distributions. We will not concentrate on ρd vs k distributions since as we pointed out

earlier, the quantity ρd is less helpful in understanding the topological role of nodes in

directed networks. However, we will give a few examples of these plots for comparison.

6.3 Local assortativity in canonical networks

Figure 6.2: Model networks: a) grid network with links directed uniformly; b) ‘inward’multi-star with links directed towards the hubs; c) “outward’ multi-star with links directedtowards the peripheral nodes; d) ‘outward’ star with links directed towards peripheralnodes; e) ‘inward’ star with links directed towards the hub; f) ring with directed links ofuniform orientation.

Before analysing local assortativity profiles for real-world directed networks, let us look

at these profiles for some important but simple topologies. These are the same topologies

that we considered in Chapter 4, and the corresponding figure is reproduced here for

convenience (Figure 6.2 ). Let us first consider the ρout vs kout profiles. As we mentioned

in earlier chapters, star networks are commonly used in communication networks and are

an important motif embedded in larger networks [108]. A few varieties of star topologies

are possible as shown in Figure 6.2, and as described in chapter 4 all of these topologies

result in disassortative networks. As we show below, the local assortativity distributions in

6.3 Local assortativity in canonical networks 87

these cases are δ functions, and all nodes are disassortative in nature. Grid layout, on the

other hand, results in a perfectly assortative network, and all nodes are assortative, even

though here too, the distribution is a δ function. The ring topology results in a perfectly

assortative network, with all nodes equally contributing to this assortativity, therefore

the ρout vs kout is again a delta function. Similar or complementary results are obtained

for ρin vs kin distributions. We formally present these observations below, which can be

mathematically derived and have been verified by simulation.

Regular lattice (a)

ρout =1N

δj,kout (6.26)

ρin =1N

δj,kin (6.27)

where kout is the out-degree of a node, and kin is the in-degree.

Inward multi-star (b)

ρout = − 1N − n∗

δj,n∗ (6.28)

ρin = − 1n∗

δj,N−n∗ (6.29)

where n∗ is the number of hubs (three in the example in Figure 6.2).

Outward multi-star (c)

ρout = − 1n∗δj,N−n∗ (6.30)

ρin = − 1N − n∗δj,n∗ (6.31)

where n∗ is the number of hubs (three in the example in Figure 6.2).

Outward star (d)

6.4 Local assortativity Distributions of real-world Biological networks 88

ρout = −δj,N−1 (6.32)

ρin = − 1N − 1

δj,1 (6.33)

Inward star (e)

ρout = − 1N − 1

δj,1 (6.34)

ρin = −δj,N−1 (6.35)

Ring (f)

ρout = ρin =1N

δj,1 (6.36)

N is the total number of nodes in the network in all these cases.

Now we proceed to analyse assortativity and local assortativity distributions in simulated

and real world directed scale-free networks.

6.4 Local assortativity Distributions of real-world Biological

networks

Now let us consider local assortativity distributions of some of the networks in Table 4.1

from Chapter 4, where we considered network level assortativity. We will look at both

ρout vs out-degree distributions and ρin vs in-degree distributions. The local assortativity

distributions of four different types of biological networks are shown in Figures 6.3, 6.4.

It can be observed that, in the rat (R. norvegicus)Gene Regulatory Network, the distri-

butions are non-linear and have assortative hubs. However, in some cases, such as E. coli

transcription network (local in-assortativity) and C. elegans neural network (again local

in-assortativity) the hubs are disassortative. Furthermore, in the human cortical network

also the largest hubs are disassortative, though the provincial hubs seem assortative. As

pointed out in the previous chapter, it is possible to classify networks (directed networks


in this case) based on whether hubs are assortative or disassortative. Specifically, net-

works can be classified as (i) assortative networks with assortative hubs, (ii) disassortative

networks with assortative hubs, (iii) assortative networks with disassortative hubs, (iv)

disassortative networks with disassortative hubs.


-0.016

-0.014

-0.012

-0.01

-0.008

-0.006

-0.004

-0.002

0

0.002

0 50 100 150 200 250 300 350 400

loca

l ass

orta

tivity

degree

-0.002

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0 20 40 60 80 100 120 140 160 180 200

loca

l ass

orta

tivity

degree

-0.004

-0.002

0

0.002

0.004

0.006

0.008

0.01

0 10 20 30 40 50 60 70 80 90

loca

l ass

orta

tivity

degree

-0.045

-0.04

-0.035

-0.03

-0.025

-0.02

-0.015

-0.01

-0.005

0

0.005

0 20 40 60 80 100 120 140

loca

l ass

orta

tivity

degree

Figure 6.3: Local in-assortativity distribu-tions of (a) E. coli transcription network (b)Rat Gene Regulatory Network (c) Humancortical network (d) C. elegans neural net-work.

-0.001

0

0.001

0.002

0.003

0.004

0.005

0.006

1 2 3 4 5 6 7

loca

l ass

orta

tivity

degree

-0.002

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0 20 40 60 80 100 120 140 160

loca

l ass

orta

tivity

degree

-0.004

-0.002

0

0.002

0.004

0.006

0.008

0.01

0 10 20 30 40 50 60 70 80 90

loca

l ass

orta

tivity

degree

-0.008

-0.006

-0.004

-0.002

0

0.002

0.004

0.006

0.008

0 5 10 15 20 25 30 35 40

loca

l ass

orta

tivity

degree

Figure 6.4: Local out-assortativity distribu-tions of (a) E. coli transcription network (b)Rat Gene Regulatory Network (c) Humancortical network (d) C. elegans neural net-work.


-0.004

-0.002

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0 20 40 60 80 100 120 140 160

loca

l ass

orta

tivity

degree

Figure 6.5: Rat Gene Regulatory Network: Scatter plot of node ρout vs out-degree. Notethe several ‘branches’ in the plot, which seem to indicate that nodes with similar degreescan have very different ρout values depending on their topological placement. The nodesat the highest branch are the ones topologically in the best position to regulate the othernodes, while the nodes in the lowest branch are in the worst position to do so. Localout-assortativity highlights this property.

In the case of directed networks, this classification can be done for both out-degree correla-

tions and in-degree correlations. For example, the rat Gene Regulatory Network would fall

into the first class, for both in-degree and out-degree correlations. Other networks, such as

E. coli transcription, would fall into different classes depending on whether out-degree or

in-degree correlation is considered. Moreover, if the hubs with high out-degree are assor-

tative, this means that these hubs are regulators which regulate other regulators, thereby

highly influencing the expression patterns of the whole network. Similarly, if the nodes

with larger in-degrees are assortative, they form ‘sinks’ of the regulating signals. Most

biological networks that we studied have assortative hubs and disassortative peripheral

nodes when out-degree and in-degree are considered separately.

It is also possible to plot individual node-degrees on X axis and local assortativity on Y

axis as a scatter plot to get a different kind of local assortativity profile. This profile

better highlights the individual nodes with highest ρin or ρout. For example, the ρout vs

out-degree plot for R. norvegicus (rat) Gene Regulatory Network is shown in the figure


-0.002

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0 20 40 60 80 100 120 140

loca

l ass

orta

tivity

degree

Figure 6.6: House mouse M. musculus Gene Regulatory network: Scatter plot of nodeρout vs out-degree. Again, note the several ‘branches’ of plot indicating nodes with similarout-degree but differing local out-assortativity.

6.5. The corresponding figure for M. musculus (mouse) is shown in figure 6.6. Note

that the nodes with the highest ρout can be easily highlighted in these plots. These are

the regulators of the network. Furthermore, we may observe certain ‘branches’ in the

profile, where nodes with similar degrees seem to have vastly different out-assortativity

values. This highlights the fact that node-degree (or out-degree) alone cannot be used to

determine the regulating effect of a node upon the rest of the network.

6.4.1 Comparing various local assortativity measures

We should note that for directed networks, the ρout and ρin profiles are most informative

in understanding the network topology. Treating them as undirected networks leads to

misleading impressions, while using the ρd does not give as much information about node

roles as ρout and ρin do. To understand these points, let us first plot the local assor-

tativity distributions of a transcription network, treating it as undirected network. The

(undirected) transcription network of E. coli is shown in Figure 6.7.

The figure shows the local assortativity ρ vs degree distributions for the transcription


-0.16

-0.14

-0.12

-0.1

-0.08

-0.06

-0.04

-0.02

0

0.02

0 50 100 150 200 250 300 350 400

avg

loca

l ass

orta

tivity

degree

Figure 6.7: Local assortativity ρ vs degree profile of E. coli transcription network. Herethe network was considered as undirected. Contrast with Figures 6.4, 6.3, 6.8.

network, considered as undirected network. It would seem from this profile that this net-

work has a completely negative and smooth local assortativity profile (similar to Internet

AS networks, as we will see in chapter 8), and hubs are disassortative. The mechanisms

responsible for growing such networks are quite different from the mechanisms that grow

networks with assortative hubs [94]. However, as Figures 6.3, 6.4, 6.8 show, we know that

this transcription network has a complex assortativity profile with hubs assortative or

disassortative depending on the correlations we are interested in (out-degree or in-degree

correlations). Therefore, it is clear that the (undirected) ρ distribution can be misleading

when applied to directed networks, and directed local assortativity must be used.

Let us now look at the measure ρd. The distributions of this measure for E. coli tran-

scription networks is given in Figure 6.8. This shows a complex assortativity profile with

assortative hubs. This measure has directedness embedded in it, but it was derived from

equation (6.1). If the ρd of a node is high, it means that the node has high out-degree

coupled with its neighbours having high in-degree or the node has high in-degree coupled

with its neighbours having high out-degree. It is a regulator surrounded by regulatees,

or vice versa. Therefore we cannot say much about the global impact of this regulator.

The Figure 6.8 seems to show two large hubs, one highly assortative and another slightly

6.5 Local assortativity profiles and functionality of individual nodes 94

-0.005

0

0.005

0.01

0.015

0.02

0.025

0 50 100 150 200 250 300 350 400

loca

l ass

orta

tivity

degree

Figure 6.8: Local assortativity ρd distribution vs degree: E. coli transcription network,considered directed. Contrast with the undirected case above.

disassortative. In fact, the profile seems ‘split’ and moving towards these two hubs. On

the other hand, when we consider ρout and ρin separately, the trend is much clearer, as

is shown in Figures 6.3 a and 6.4 a. We have a highly disassortative in-degree hub and a

highly assortative out-degree hub. Thus the quantities ρout and ρin have more utility in

identifying assortative or disassortative mixing in directed networks.

In summary, we can say that local assortativity profiles of directed networks are most

informative when ρout and ρin are plotted against out-degree and in-degree respectively.

Considering their undirected local assortativity profiles can give misleading information

about their growth mechanisms and phase of growth, while considering their ρd values

tend to combine and confuse out-degree and in-degree trends.

6.5 Local assortativity profiles and functionality of individ-

ual nodes

The local assortativity profiles can be used to make some interesting observations about

the functionality of individual nodes in a directed biological network, and thus simulate

further research about these nodes at an individual level. For example, the nodes with

6.5 Local assortativity profiles and functionality of individual nodes 95

No network name Nodes with highest ρin

[highest kin in brackets]Nodes with highest ρout

[highest kout in brackets]

1 E. coli transcription arcA [crp] fnr [several nodes but notfnr]

2 C. efficiens tran-scription

several nodes but not pcaR[pcaR]

all [all]

3 C. glutamicumtranscription

arnR [glxR] ramB [several but notramB]

4 C. jeikeium tran-scription

all except dtxR [dtxR] dtxR [all except dtxR]

5 Bay wet food web Respiration, water POC,raptors [raptos]

Herb. amphipods,det. am-phipods, BC [pred. shrimp]

6 Bay dry food web Respiration, water POC,raptors [raptors]

Herb. amphipods,det. am-phipods,BC [pred. shrimp]

7 Crystal C food web string grey, striped an-chovy, silver jelly [detrius]

BI, zooplankton, moharra[BI]

8 Crystal D food web moharra, gold spotted kill-fish, silverside [detrius]

BI, bay anchovy,silverside[BI]

9 Lower Chesapeakefood web

pytoplankton, picoplank-ton, meroplankton [POC]

pytoplankton, picoplank-ton, meroplankton [POC]

10 Upper Chesapeakefood web

output, stripped bass,perch [POC]

net pytoplankton, pi-coplankton, ciliates [POC]

11 Human GRN RHOV [RHOV] RHOA [RHOA]

12 House mouse GRN LOC100046796[LOC100046796]

Ppp3r2 [Ppp3r2]

13 Rat GRN LOC690845 [LOC690845] Adcy1 [Adcy1]

14 A. thaliana GRN AT5GO8100 [AT5GO8100] AT2GO2000 [AT2GO2000]

15 C. elegans GRN F23B2.13 [F23B2.13] F23B2.13 [F23B2.13]

Table 6.1: A list of biological networks and their nodes with highest ρout or ρinvalues. Thenodes with highest out-degrees kout and highest in-degrees kin are shown in correspondingbrackets for comparison. POC stands for Particulate Organic Carbon. BI for BenthicInvertebrates and BC for Benthic Crustaceans. In Gene Regulatory Networks the nodesare genes, while in transcription networks, the nodes are transcription factors.

the highest local assortativity (both ρout and ρin ) for a range of networks are given

in Table 6.1. For comparison, the nodes with highest out-degrees and in-degrees are

also listed in the table. It is apparent that the nodes with highest out-degrees or in-

degrees are not necessarily the strongest regulators / regulatees in the table. This (the

6.6 Summary 96

similarity) seems to be the case only among the Gene Regulatory Networks we analyzed.

Among the transcription networks and food webs, the node with the highest out-degree

is often different from those with highest ρout. For example, in the case of C. glutamicum

transcription network, the transcription factor arnR is the node with the highest ρout,

while the transcription factor glxR is the node with the lowest ρout and highest out-

degree. Therefore it could be argued that while the transcription factor glxR has the

highest out-degree, the transcription factor arnR has more influence on regulating other

genes. A similar line of argument could be used to highlight the utility of ρin. Therefore,

local assortativity quantities can be used to gain information about node functionality

that is not apparent from node-degrees.

The considered examples demonstrated that local assortativity profiles of biological net-

works can be used to highlight interesting topological properties of individual nodes or

groups of nodes.

6.6 Summary

In this chapter we introduced and analysed local assortativity in directed networks. Again,

we put particular emphasis on biological networks, since directedness is important to

the functionality of these networks. We extended the concept of local assortativity to

directed networks, defining local out-assortativity ρout and local in-assortativity ρin and

deriving the formulations for these quantities. We analysed local assortativity profiles of

directed biological networks, and attempted to relate out-assortativity and in-assortativity

of individual nodes with their functions.

It was explained that when out-degree and in-degree mixing are considered separately, the

assortativity or disassortativity of nodes especially hubs become more explicit, i.e., the

nodes with relatively low ρd can have relatively high ρin or ρout and vice versa. It was

shown also that the local out-assortativity can be used to identify the regulators which are

most influential, since high out-degree (or in-degree) does not necessarily imply high ρout

(or ρin) and vice versa. Local assortativity profiles can also be used to identify nodes and

groups of nodes which are ‘interestingly’ placed topologically — that is, with the ability

to regulate or to be regulated.

6.7 Appendix 97

It is possible to classify directed networks based on local assortativity, similar to the

classification presented in the previous chapter for undirected networks. Networks can be

classified into four classes based on either out-assortativity or in-assortativity, therefore in

total, sixteen classes of networks could, in theory, exist. It would be interesting to find out

which classes of these networks actually exist among real world directed networks. This is

however considered to be beyond the scope of this thesis, which seeks to shed some light

on the fundamentals of assortative mixing. Nevertheless this can be one of the research

questions which could be addressed in the follow up work to this thesis.

In the previous chapter and this chapter we introduced and analysed local assortativity

for undirected and directed networks respectively. In the next chapter we will look at how

this concept can be extended to similarity based on node states.

6.7 Appendix

This appendix presents the derivation of the expressions for ρout and ρin. Therefore we

need to analyse contributions to terms appearing in equations (6.17) and (6.18) for ρout

and ρin respectively.

First of all, we note that µinq and µout

q can be equivalently defined as

µinq =

1M

M∑

m=1

kinm =

1M

N∑

v=1

kinv kout

v (6.37)

and

µoutq =

1M

M∑

m=1

koutm =

1M

N∑

v=1

koutv kin

v (6.38)

Note that µinq is the ‘expected in-degree’ when a link goes out from a node. From any

node, kout links go out, therefore it has to be multiplied by kout when we consider the

node v. Similarly, note that µoutq is the ‘expected out-degree’ when a link comes into a

node. Any node has kin links going into it, therefore it has to be multiplied by kin when

we consider the node v.

Remark 6.7.1. We may note that

µoutq = µin

q (6.39)

6.7 Appendix 98

However, µoutq and µin

q are not always equal.

Having defined these expected degrees, we analyse the equation (6.17) for ρout. We begin

by considering the contribution to its first term:∑jk

jkeoutj,k , where j, k are out-degrees. One

the one hand, it is produced via the neighbours reachable from the node:

αout1 = (jout)2kout

M(6.40)

where jout is the out-degree of the node considered, and kout is the average out-degree of

the neighbours reachable from the node. That is, we have considered all links that depart

from the node as contributing to the quantity∑jk

jkeoutj,k .

On the other hand, we should also consider how much all links that reach a given node

contribute to it. In this case, the contribution to∑jk

jkeoutj,k is

αout2 = joutkinjout

M(6.41)

where kin is the in-degree of node considered, and kout is the average out-degree of the

neighbours from which the node can be reached.

As was done previously, we take the average of these quantities as the contribution of a

given node, αout, yielding

αout = joutjoutkout

2M+ joutkin

jout

2M(6.42)

This expression captures the contribution to the term∑jk

jkeoutj,k .

We follow by considering the contribution to the second term µoutq µout

q , obtained, using

equations (6.5) and (6.38), as follows:

µoutq µout

q =1

M2

N∑

v=1

kinv kout

v

N∑

v=1

(koutv )2 (6.43)

=1

M2

(kin

1 kout1 +

N∑

v=2

kinv kout

v

)((kout

1 )2 +N∑

v=2

(koutv )2

)(6.44)

6.7 Appendix 99

Considering a single node (without loss of generality, we choose the node 1), we obtain its

contribution as:

kin1 (kout

1 )3

M2+

kout1 kin

1

N∑v=2

(koutv )2

2M2+

(kout1 )2

N∑v=2

koutv kin

1

2M2(6.45)

We assume equal contribution when two nodes are involved in a term, hence division by

two. This can be further reduced to

12M2

(kout

1 kin1

N∑

v=1

(koutv )2 + (kout

1 )2N∑

v=1

koutv kin

1

)(6.46)

yielding, for any node:

βout =1

2M

((jout)2µout

q + kinjoutµoutq

)(6.47)

Therefore, we obtain

ρout =αout − βout

σoutq σout

q

(6.48)

resulting in

ρout =jout

2Mσoutq σout

q

(jout(kout − µout

q ) + kin(jout − µoutq )

)(6.49)

Similarly, we obtain

ρin =kin

2Mσinq σin

q

(kin(jin − µin

q ) + jout(kin − µinq )

)(6.50)

Chapter 7

Non-degree based assortativity

7.1 Introduction

In previous chapters we analysed assortativity and local assortativity in complex networks.

In defining assortativity, we have defined similarity of nodes in terms of degrees. However,

as we mentioned in chapter 2, similarity could be defined in terms of other properties of

nodes as well. In this chapter we investigate the concepts of scalar based assortativity (i.e

assortative mixing of nodes based on node states or properties of nodes other than degree)

and local scalar assortativity, which we call node congruity.

Nodes of complex networks may have a number of properties. Some of these properties

may be boolean in nature, taking one of two states. For example, in a social network,

each node may have a gender (male or female), or in a neural network, each neuron may

be spiking or not spiking at a given time. Other properties may take integer values, such

as the age of people in social networks, and yet other properties may be continuous real

numbers, such as reading of sensors in a sensor network. In each of these cases, the node

states have a distribution. For example, let us say that in a binary network, the node

state distribution is such that most nodes have state ‘1’ rather than ‘0’. One may then

ask, does that mean that if we pick a random neighbour of a node, that neighbour is more

likely to have state ‘1’? That is not always the case.

To illustrate this, consider a star network with the central node (the hub) having state ‘0’,

and all other nodes having state ‘1’. While most nodes have state ‘1’, neighbours of most

7.1 Introduction 101

nodes have state ‘0’. In other words, even though most nodes have state ‘1’, if we pick a

random link, finding a ‘1’ at an end of this link is not more likely than finding a ‘0’. In

fact these likelihoods are equal, and are influenced by the topology.

The knowledge about the likelihood of finding a given state at the end of a link is quite

important to understanding a complex network and its dynamics [87]. For instance, in

a sensor network, relatively high readings of temperature in a chain of direct neighbours

may point to a potential fault line. In a social network, we may be interested to know

whether people who are directly connected are in similar age groups, or even whether they

have similar habits that can be quantified [38]. In a neural network, it may be important

to understand if all the neurons which spike at the given time are directly connected. A

number of other examples could be provided from other domains of complex networks. In

short, measuring the tendency in a network where directly connected nodes have similar

properties is critically important in understanding the network’s dynamics. In this chapter

we analyse this tendency, by generalising and extending the concept of scalar assortativity,

as described below.

Even though similarity between nodes can be interpreted in many ways, assortativity

has been primarily defined by similarity of degrees of nodes [81, 82, 92, 93]. Thus, the

assortativity coefficient is related to network topology and is constant for that network

while the topology remains unaltered. The concept of assortativity was extended by

Newman [82] to measure similarity of scalar attributes of nodes (other than degree) as we

saw in chapter 2 - this was called scalar assortativity by [82].

The state of the node, whether it is a boolean, discrete or continuous quantity, is an at-

tribute of the node, and similarity of nodes can be interpreted in terms of this attribute.

Moreover, unlike node degree, the node state will change with time, therefore when sim-

ilarity is defined in terms of node states, the assortativity coefficient of a network varies

with time as well. Therefore, it is possible to measure scalar assortativity over time and

analyse its tendencies as a way of understanding the dynamics of the network.

In this chapter, we analyse scalar assortativity coefficient as a function of time, based on

node states. We primarily use networks with boolean states in simulating the dynamics,

though we use the topologies of real world networks. We show that network scalar assorta-

tivity carries information about the network’s dynamics that cannot be described by either

the topology alone or by the state distribution alone, and we quantify this information

7.2 Scalar assortativity as a function of time 102

using information theoretic measures. Furthermore, following [92], we also define the local

contribution of an individual node to the global scalar assortativity, which we call node

congruity. We analyse a number of model and real-world networks and their dynamics

using these concepts.

This chapter is organised as follows: Section 7.2 introduces scalar assortativity as a func-

tion of time. Section 7.3 analyses scalar assortativity in boolean networks. Section 7.4

investigates the relationship between scalar assortativity and network information content

in terms of node states. Finally, section 7.5 introduces and defines node congruity. In

section 7.6 we present the chapter summary.

7.2 Scalar assortativity as a function of time

We saw in chapter 2 that scalar assortativity could be defined as

r =1

1−∑jk

ajbk

∑

jk

(ej,k − ajbk)

(7.1)

where aj and bk are the fraction of each type of end (source or target) of a link that is

attached to node of type j and node of type k . In undirected networks, where there is

no ‘source’ or ‘target’ node, aj = bk. Furthermore, ej,k is the fraction of links which have

type j of node at source and type k of node at target.

Let us say that a given node at a given time t is in state yt. For simplicity, let us assume

that yt takes only integer values (though the concept of scalar assortativity is applicable

to continuous node states with appropriate binning). Most of the examples we present in

this chapter, in fact, assume binary node states. Following the excess degree distribution,

let us define distribution qty as the probability distribution of finding a link with node state

yt at an end of a link at time t. Similarly, let us define distribution ety,z as the probability

distribution of finding a link with node state yt at one end of the link and node state zt

at the other end of the link. Let us also say the expectation of qty at a given instant t is

denoted as µtq and the standard deviation of the same distribution at time t as σt

q. Then

network scalar assortativity L t is defined as:


L t =1

(σtq)

2

[∑yz

yz(ety,z − qt

yqtz

)]

(7.2)

Equivalently, we can also write

L t =1

(σtq)

2

[(∑yz

yzety,z

)− (µt

q)2

](7.3)

where µtq is the expected value of node state at the end of a link at time t = 0.

Let us note that if the scalar assortativity L t = 1, it means all links in the network have

the same node states at either side of the link. In a non-fragmented network, this also

means that all nodes must have the same state. If L t = −1, it means that all links have

nodes with dissimilar states on either side of them.

If L t = 0, it means that a link is equally likely to have similar or dissimilar node states

on either side of the link.

We should note that L t = 0 does not imply a random distribution of nodes states. Indeed,

scalar assortativity is a measure of the influence of topology in the ‘expected’ node state

at the end of a link. Therefore, if the expected value of the node state distribution is

equal to the expected value of qtz, then scalar assortativity should be zero. The following

examples with some model networks will illustrate this point further.

7.2.1 Model networks

Before analysing real world networks, we consider the scalar assortativity of some canonical

networks with trivial node state distributions. For simplicity, let us consider binary node

states, where node state can be either 1 or 0. Note that regardless of the number of

possible states and their discrete / continuous nature, network scalar assortativity can

take any real value between 1 and −1. This is the case even when the node states are

binary.

Perfect positive scalar assortativity L t = 1.0 is possible if and only if all nodes are in the

same state, regardless of the topology (unless the network is fragmented). Therefore let

us concentrate on the cases which show perfect negative scalar assortativity L t = −1.0.


Star network with dissimilar node as a hub

Figure 7.1: Star network with scalar assortativity L t = −1

Star topology is an important motif in many real world networks including communication

networks, Local Area Networks (LAN), and regulatory networks [108]. We alluded to this

network at the start of this chapter in discussing the motivation for this work. The star

network shown in Figure 7.1 with binary nodes states has scalar assortativity of L t = −1.

It should be noted that even though most nodes have similar states, the scalar assortativity

shows extreme negative correlation. This is the simplest case with perfect negative scalar

assortativity. L t = −1 is not possible for all network topologies however. In scale free

networks, it may not be possible to achieve L t = −1 for any combination of node states,

simply due to the topology. However, if the scale free network is a tree, then a set of node

states can be found such that L t = −1.

In general, a given topology will have a maximum positive scalar assortativity L tmax ≤ 1.0

and a maximum negative scalar assortativity L tmin ≥ −1.0. The exact values of these

depend not only on the topology but also on the number of possible states (if there are

more than two states).

Ring network with nodes having alternating states

The ring network, as shown in Figure 7.2, also shows L t = −1.


Figure 7.2: Ring network with scalar assortativity L t = −1

Scale-free topology with perfect negative scalar assortativity

Figure 7.3: A scale-free network with scalar assortativity L t = −1

As an example for the simple scale-free network (in this case, also a network with a tree

topology) showing perfect negative scalar assortativity, we present the network in Figure

7.3. As the figure shows, this network with the given node states has perfect negative

scalar assortativity.

A random network or any network with randomly distributed (binary) node states would

asymptotically reach L t = 0 with the network size approaching infinity.

We have utilised a number of topologies above to demonstrate the occurrences of extreme

scalar assortativity values (L t = 1.0 , L t = −1.0) and L t = 0. However, it is important

7.3 Scalar assortativity in Random Boolean Networks 106

to note that network scalar assortativity is not determined by topology alone. Indeed,

even for very simple topologies, the whole range of scalar assortativity values from 1.0

to −1.0 are possible. To demonstrate this, let us consider the simple ‘benzene-ring’ like

topology in Figure 7.4. Assume that, in seven time steps, the nodes take the node states

shown in Table 7.1. As Table 7.1 also shows, the scalar assortativity goes from 1.0 to

−1.0, while the topology remains the same. We will show that large fluctuations in scalar

assortativity are possible in other topologies also, including scale-free networks. Thus,

scalar assortativity provides more information about the node states and dynamics of

networks than the network’s degree-based assortativity (a correlation measure of network

topology) or statistical measures such as the standard deviation of node states (correlation

measures on node state distribution).

Node T=1 T=2 T=3 T=4 T=5 T=6 T=7

1 0 1 1 1 1 1 1

2 0 0 0 0 0 0 0

3 0 0 1 1 1 1 1

4 0 0 0 0 0 0 0

5 0 0 0 1 1 1 1

6 0 0 0 0 0 0 0

7 0 0 0 0 1 1 1

8 0 0 0 0 0 0 0

9 0 0 0 0 0 1 1

10 0 0 0 0 0 0 0

11 0 0 0 0 0 0 1

12 0 0 0 0 0 0 0

scalar assortativity 1.0 -0.14 -0.33 -0.60 -0.71 -0.84 -1.0

Table 7.1: The states for seven time steps and corresponding scalar assortativity for thenetwork shown in Figure 7.4

7.3 Scalar assortativity in Random Boolean Networks

To understand scalar assortativity as a function of time, we simulated network dynamics

on a number of boolean networks and measured their scalar assortativity against time. We

used the topologies of a number of real world networks (eg: E. coli transcription network),


Figure 7.4: A benzene-ring like topology shows scalar assortativity ranging from L t = 1 toL t = −1 depending on node states. Note that the states shown in this figure correspondto L t = −1.

but in simulating the dynamics, we assumed that their node state would be either ‘zero’ or

‘one’ (i.e we considered them as boolean networks). Particularly, we utilised the topologies

of Gene Regulatory Networks and transcription networks, since it has been shown that

boolean networks are good models for these types of real world networks [18, 19]. The

interpretation of the boolean states is expressed or not-expressed states of the genes. We

implemented a number of logic functions in the nodes to simulate the dynamics, as listed

below.

1. logic f1: The nodes are simply assigned a boolean state (‘0’ or ‘1’) with probabilities

1−p and p. The previous state of the node considered or other nodes do not influence

the current state. If p = 0.5, the node will be randomly assigned ‘1’ or ‘0’ with equal

likelihood.

2. logic f2: The nodes follow the ‘average state’ of all their neighbours with probability

p. Specifically:

• If node state yv = 0 and average neighbour state z > 0.5, then with probability

p = Az, the node changes state to yv = 1. A is a parameter of the logic.

• If node state yv = 1 and average neighbour state z < 0.5, then with probability

p = Az, the node changes state to yv = 0

3. logic f3: The nodes ‘oppose’ the states of their neighbours with probability p. Since


a node will have a number of neighbours, a node will choose one node from its

neighbours and change its state to ‘oppose’ that neighbours state. The probability

of a neighbour being chosen is proportional to the neighbour’s degree. That is, nodes

with more connections are more likely to be ‘opposed’ by their neighbours. Formally,

for the concerned node v with degree dv, choose a node w among the neighbours

with probability pw such that

pw =kw∑dv1 kw

(7.4)

and change the node state of node v such that yv 6= yw.

We ran a number of simulation experiments, implementing the above logical functions in

the nodes of networks. In a given simulation experiment, all nodes had identical logical

behaviour. However, the logic function that is run on nodes could change with time (e.g.,

nodes implementing f1 for T1 time-steps and then f2 for T2 time-steps, periodically.).

The node states were synchronously updated, with the updating order random and shuf-

fled for each time step. The simulation results for the boolean network with the E. coli

transcription network topology are given below.

7.3.1 Random logic: logic f1

When node states are randomly assigned (with P (1) = p), the scalar assortativity remains

close to zero for any number of time steps. We tried changing the value p periodically,

so that the proportion of ‘1’ states changes with time. The result of such a simulation

experiment is shown in Figure 7.5, where parameter p is periodically changed from p = 0.2

to p = 0.8. We see that despite the change in the proportion of ‘1’ states, the scalar

assortativity remains close to zero. A similar example is shown in Figure 7.6, where

parameter p is periodically and linearly (rather than like a step-function) changed from

p = 0.2 to p = 1.0. Again, we see that the scalar assortativity remains close to zero

throughout the simulation time.

These results are easy to explain. Scalar assortativity does not depend only on the dis-

tribution of states, but it depends also on the placement of states topologically. If the

topological assignment is random, then despite the variations in the state distribution,


the scalar assortativity will be close to zero. It can be shown that this result is valid for

networks with any number of states, not just binary state networks.

7.3.2 Logic f2

As seen above, logic f2 is implemented in such a way that nodes tend to (stochastically)

follow the states of their neighbours. Intuitively, this should mean that the scalar assorta-

tivity must increase, since links are increasingly likely to have nodes with similar states at

each end over time. The result of a simulation for 200 time-steps where nodes implement

this logic is shown in Figure 7.7. The nodes are initialised randomly. Indeed, we could

see from this figure that the scalar assortativity starts from close to zero and increases

exponentially and stabilises at L = 0.63. We observe that comparatively the proportion

of ‘1’s do not change much. The result of another run of the same simulation is shown in

Figure 7.8 where the proportion of ‘1’ states actually decreases, while scalar assortativity

still increases exponentially.

From these results, it is clear that scalar assortativity can vary by orders of magnitude

while the distribution of states remain nearly unchanged. In this case, the implemented

dynamics, which encourages neighbouring nodes to have similar states, is responsible for

the eventual high (positive) scalar assortativity.

7.3.3 Logic f3

Logic f3 is implemented in such a way that nodes stochastically ‘oppose’ the state of their

neighbours, with the neighbours with the highest degree having more likelihood to be

‘opposed’. Intuitively, this should mean that the scalar assortativity must decrease from

zero, since links are increasingly likely to have nodes with opposite states at each end. The

results of two separate simulation runs for 200 time-steps where nodes implement this logic

are shown in Figure 7.9 and 7.10. Again, the nodes are initialised randomly. We could

see from these figures that scalar assortativity indeed decreases from zero and stabilises

on considerably negative values (around L = −0.5). In Figure 7.9, the proportion of

‘1’s slightly increases with time, whereby in Figure 7.10, the proportion of ‘1’s slightly

decreases with time; however, in both cases, the scalar assortativity decreases by an order

of magnitude. These results further confirm that scalar assortativity can vary by orders


-0.2

0

0.2

0.4

0.6

0.8

1

0 50 100 150 200 250 300 350 400

Sca

lar

asso

rtat

ivity

/ m

ean

time

Figure 7.5: Variation of scalar assortativity vs time for a boolean network having thetopology of E.coli transcription network. The nodes implement a simple logic to randomlyassign node states. The state distribution is varied, periodically having a high proportionof ‘1’ states. Note that while the state distribution peaks periodically, scalar assortativitydoes not change much and stays close to zero. Stars: mean of state distribution. Crosses:network scalar assortativity.

of magnitude while the distribution of states remain nearly unchanged. In this case, the

implemented dynamics, which discourages neighbouring nodes to have similar states, is

responsible for the eventual negative scalar assortativity.

7.3.4 Combination of logical functions

To further verify the results above, we combined the logic functions mentioned above along

the time axis. For example, logic f1 was implemented on all nodes for t = 60 time steps

followed by logic f2 for t = 20 time steps on all nodes. This process is repeated to create a

periodic combination of logic f1 and logic f2. The results of such an experiment are shown

in Figure 7.11. We may see that when random logic f1 is implemented, (with p = 0.5), the

scalar assortativity remains close to zero. When logic f2 is implemented though, scalar

assortativity raises by an order of magnitude. The proportion of ‘1’ states either increases

or decreases depending on the node state distributions when the logic is flipped (from logic

7.4 Scalar assortativity and information content 111

-0.2

0

0.2

0.4

0.6

0.8

1

0 50 100 150 200 250 300 350 400

Sca

lar

asso

rtat

ivity

/ m

ean

time

Figure 7.6: Variation of scalar assortativity vs time for a boolean network having thetopology of E.coli transcription network. The nodes implement a simple logic to randomlyassign node states. The state distribution is varied, periodically having a high proportionof ‘1’ states. Note that while the state distribution peaks periodically, scalar assortativitydoes not change much and stays close to zero. Stars: mean of state distribution. Crosses:network scalar assortativity.

f1 to logic f2), but in all cases the change in the proportion of ‘1’ states is small compared

to the change in scalar assortativity. When the logic is flipped again (from logic f2 to

logic f1), scalar assortativity drops back immediately close to zero. We combined logic f3

with logic f1 and logic f2 and obtained similar results. These results confirm that scalar

assortativity is highly influenced by topological placement of node states (node values),

and as such provides information about the network dynamics that cannot be obtained by

just analysing the node state distributions of the network. In the next section, we attempt

to quantify the information provided by scalar assortativity.

7.4 Scalar assortativity and information content

In the previous sections we have seen that scalar assortativity can convey more information

about the states of the network than just conveyed by the network’s state-distribution.


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 20 40 60 80 100 120 140 160 180 200

Sca

lar

asso

rtat

ivity

/ m

ean

time

Figure 7.7: Variation of scalar assortativity vs time for a boolean network having thetopology of E.coli transcription network. The nodes implement a simple logic to proba-bilistically follow the average state of their neighbours. The initial node states are random.Note that while the state distribution does not change much, scalar assortativity increasesexponentially from near zero (scalar non-assortativity) to positive scalar assortativity.Stars: mean of state distribution. Crosses: network scalar assortativity.

How can we quantify this? In other words, what is the relationship between scalar as-

sortativity and the information contained in the network in terms of its node states? To

answer this, we should define the information content of a network in terms of node states.

In chapter 3 the relationship between assortativity and degree-based information content

was analysed in detail. Here we undertake a similar, albeit brief, analysis.

We saw in chapter 2 that Shannon information I(q) is a more generic measure of depen-

dence than the correlation functions that measure linear relations. In [93, 96, 108], the

entropy and information content were defined with respect to the degree distribution and

joint degree distribution — purely in topological terms, irrespective of node states. Now

we will attempt to define these in terms of node states in a network.

At first glance, one may wish to define the entropy of a network, using the probability

distribution ut that is defined via the probabilities utz of encountering a node at the state


0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 20 40 60 80 100 120 140 160 180 200

Sca

lar

asso

rtat

ivity

/ m

ean

time

Figure 7.8: Variation of scalar assortativity vs time for a boolean network having thetopology of E.coli transcription network. The nodes implement a simple logic to proba-bilistically follow the average state of their neighbours. The initial node states are random.Note that while the state distribution does not change much, scalar assortativity increasesexponentially from near zero (scalar non-assortativity) to positive scalar assortativity.Stars: mean of state distribution. Crosses: network scalar assortativity.

zt anywhere in the network, at time t:

H(ut) = −∑

z

utz log ut

z (7.5)

In this chapter however, we are interested in node states as well as the topology, and

therefore, shall define entropy and information content in terms of node state distributions

that depend on the link distribution. Such network entropy can be defined as

H(qt) = −∑

z

qtz log qt

z (7.6)

where qtz is, at time t, the probability (proportion) of links with a node (at one end) in the

state z. Since qtz is dependent on link distribution, the entropy defined by Equation (7.6)

also depends on the network topology, and is not just the entropy of node states, defined

by Equation (7.5).


-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

0 20 40 60 80 100 120 140 160 180 200

Sca

lar

asso

rtat

ivity

/ m

ean

time

Figure 7.9: Variation of scalar assortativity vs time for a boolean network having thetopology of E.coli transcription network. The nodes implement a simple logic to proba-bilistically oppose the state of their neighbours. The initial node states are random. Notethat while the state distribution does not change much, scalar assortativity decreases expo-nentially from near zero (scalar non-assortativity) to negative scalar assortativity. Stars:mean of state distribution. Crosses: network scalar assortativity.

The defined entropy measures are contrasted in Figure 7.12. Note that the boolean net-

work having the topology of E. coli transcription network is simulated here, with logic f2

implemented in nodes. As seen before, the logic f2 will ensure that scalar assortativity

of the network will increase with time until it stabilises at a maximum value. We may

note that the entropy H(ut) decreases but the entropy H(qt) increases with time. This

is due to the fact that the former is not dependent on topology, and simply reflects the

proportion of zeros and ones, while the latter depends on topology and reflects the scalar

assortativity of the network.

Similarly, mutual information in terms of node states can be defined as:

I(qt

)=

∑y

∑z

ety,z log

ety,z

qtyq

tz

(7.7)

where ety,z is the proportion of links connecting, at time t, the nodes with states y, z


-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0 20 40 60 80 100 120 140 160 180 200

Sca

lar

asso

rtat

ivity

/ m

ean

time

Figure 7.10: Variation of scalar assortativity vs time for a boolean network having thetopology of E.coli transcription network. The nodes implement a simple logic to proba-bilistically oppose the state of their neighbours. The initial node states are random. Notethat while the state distribution does not change much, scalar assortativity decreases expo-nentially from near zero (scalar non-assortativity) to negative scalar assortativity. Stars:mean of state distribution. Crosses: network scalar assortativity.

respectively; qty is the proportion of links, at time t, with a node (at one end) in the state

y; and similarly, qtz is the proportion of links, at time t, with a node (at one end) in the

state z.

Now we can analyse how this mutual information changes with scalar assortativity. To do

so, we looked at the two logical functions (other than f1) of random boolean networks,

plotting network mutual information as well as scalar assortativity. The results are given

in the Figures 7.13, 7.14 respectively. From the figures we may see that the information

content matches the absolute values of scalar assortativity. That is, the more assortative

or disassortative the network is, the more information it contains about expected states

at the end of links. The beginning of each simulation where scalar assortativity is close to

zero contains the least amount of information. We also note that there is no evidence for

just positive scalar assortativity containing more information or vice-versa. Therefore, as

suggested in [93] for assortativity and information content regarding degrees, we postulate

7.5 Node congruity 116

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 50 100 150 200 250 300 350 400

Sca

lar

asso

rtat

ivity

/ m

ean

time

Figure 7.11: Variation of scalar assortativity vs time for a boolean network having thetopology of E.coli transcription network. The nodes alternatively implement two logics,one assigning node states randomly and the other logic where nodes probabilisticallyfollow their neighbour’s states. Note that when the second logic is implemented, scalarassortativity increases rapidly, and when the first logic is implemented, scalar assortativitydrops back to near zero. The proportion of nodes with state ‘1’ comparatively does notchange much. Stars: mean of state distribution. Crosses: network scalar assortativity.

that information content in a network regarding note states has a positive correlation with

the absolute value of scalar assortativity. A detailed study of this correlation, as done in

chapter 3 for information content and degree-based assortativity, is a subject of future

research.

7.5 Node congruity

The concept of local assortativity [92] was introduced in chapter 5 to quantify the con-

tribution of an individual node to network assortativity. Since the scalar assortativity

L t measures similarity of nodes globally, the local scalar assortativity, denoted λt, can be

defined for each node as the node’s contribution to the scalar assortativity L t, at time

t. We choose for simplicity to call this local property λt the node congruity. We believe

that congruity is a suitable term as it quantifies the extent to which a node is similar


2

2.005

2.01

2.015

2.02

2.025

2.03

0 50 100 150 200 250 300

Ent

ropy

time

Figure 7.12: Variation of entropy vs time for a boolean network having the topology ofE. coli transcription network, simulated with logic f2. Note that as scalar assortativityincreases with time (even though not shown in this figure, logic f2 ensures that it willincrease with time as we have seen before), the entropy H(ut) decreases but the entropyH(qt) increases. Crosses: H(ut). Stars: H(qt).

(congruent) to its neighbours. In this section we derive the expression for node congruity

λt.

Following chapter 5 and chapter 6, we propose to derive node congruity as the contribution

of a given node to the network scalar assortativity, which means we need to determine

how much each node v contributes to the term

1σ2

q

[(∑yz

yzety,z

)− (µt

q)2

]

Let us first look at the term∑yz

yzety,z (which is calculated over node states) and the

contribution of the node v in the state yv to this term.

Suppose we visit all the nodes in a network, and from each node in turn we visit all the links

of that node. In a network with N nodes and M links, the total visits we will thus make

will be 2M , since each link will be visited twice, once from each end. Suppose we build up

the probability distribution ety,z as we make these visits. Each link will add a probability of


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 50 100 150 200 250 300

Sca

lar

asso

rtat

ivity

/ M

I

time

Figure 7.13: Variation of scalar assortativity and mutual information vs time for a booleannetwork having the topology of E. coli transcription network. Note that informationcontent increases with the increase in positive scalar assortativity. Simulated with logicf2. Stars: mutual information. Crosses: network scalar assortativity.

(1/2M) to the pair of (y, z) where y and z are the node states of nodes at each end of the

link. Thus, each visit to a link will contribute yz/2M to the sum∑yz

yzety,z. Therefore, if

we examine the node v with state yv and degree dv which is connected to nodes with states

z1, z2, . . . zdv , it will contribute (yvz1/2M) + (yvz2/2M) + . . . + (yvzdv/2M) = yv

2M

dv∑i=1

zi

to the sum∑yz

yzety,z. Let us denote the average of node states of a node’s neighbours as

z = 1dv

dv∑i=1

zi. Then we can represent the individual node’s contribution, αv, to the sum∑yz

yzety,z as

αv =yv

2M

dv∑

i=1

zi =yv

2Mdvz (7.8)

Now let us consider a node’s contribution to the term (µtq)

2. To do so, let us first examine

the definition of µtq:

µtq =

12M

2M∑

m=1

ym =1

2M

N∑

w=1

dwyw (7.9)


-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

0 50 100 150 200 250 300

Sca

lar

asso

rtat

ivity

/ M

I

time

Figure 7.14: Variation of scalar assortativity and mutual information vs time for a booleannetwork having the topology of E.coli transcription network. Note that information con-tent increases with the increase in negative scalar assortativity. Simulated with logic f3.Stars: mutual information. Crosses: network scalar assortativity.

where m is an end of a link; ym is the state of the node at the end m; while dw is a

node-degree and yw a node state. The equivalence of the two representations used in this

definition, is yielded by the replacement of every set of links connected to some node by

that node scaled by its degree. It follows that

(µtq)

2 =

(1

2M

N∑

w=1

dwyw

)2

(7.10)

(µtq)

2 =1

4M2(d1y1 + d2y2 + ..... + dNyN )2 (7.11)

Now, let us consider the node v (without loss of generality, let it be the node 1 with node

state y1), and its contribution to the expression above. The terms with index 1 are:

14M2

((d1y1)2 + 2d1y1(d2y2 + d3y3 + ........... + dNyN )) (7.12)

Among these, terms such as 2d1y1djyj have to be ‘divided’ between node 1 and node j


respectively. These are multiplication terms, and we assume that an equal division is

appropriate. Therefore, we can consider that contribution of node 1 is:

14M2

((d1y1)2 + d1y1(d2y2 + d3y3 + ........... + dNyN )) (7.13)

=1

4M2(d1y1(d1y1 + d2y2 + d3y3 + ........... + dNyN )) (7.14)

=1

4M2

(d1y1

N∑

v=1

dvyv

)(7.15)

Therefore, the contribution of a given node v to the term (µtq)

2 can be given by:

βv =1

4M2

(dvyv

N∑

w=1

dwyw

)(7.16)

βv =1

2M

(dvyvµ

tq

)(7.17)

The standard deviation is already a scaling term, and we need not worry about a sin-

gle node’s contribution to it. Combining Equations (7.8) and (7.17) we formally define

congruity of a node.

Definition 7.5.1. congruity of a node λtv is given by

λtv =

αv − βv

(σtq)2

= yvdv

(z − µt

q

)

2M(σtq)

2 (7.18)

Congruity can be interpreted as a scaled difference between (i) the average state of the

node’s neighbours, and (ii) the average state across the whole network (i.e., the expected

global or network-level state). If the node’s local neighbours are in the states that are com-

paratively ‘higher’ than the globally expected value, then the node’s congruity is positive.

On the other hand, if the neighbours are in the states that are comparatively ‘lower’ than

the globally expected value, then the congruity of the node is negative. Thus, congruity

also quantifies the extent of how much the states of the node’s immediate neighbours differ

to the network as a whole. From the definition and derivation of congruity, it also follows

7.6 Distributions of node congruity 121

that the sum of congruities λtv over all nodes is equal to network scalar assortativity L t,

at any time t. That is,

L t =N∑

v=1

λtv (7.19)

7.6 Distributions of node congruity

Since congruity is a property of a node, it is possible to construct node congruity distribu-

tions for a given network, just like local assortativity distributions mentioned in chapters

5 and 6. We may plot node congruity values against degrees, or we may calculate the

average node congruity value for all nodes with a given degree k. If we denote by N(k)

the number of nodes with degree k, the following equations hold true.

L t =∑

kN(k)λt (k) (7.20)

where λt(k) is the average congruity λt, at time t, of all nodes with degree k.

L t = N∑

kpkλt (k) (7.21)

where pk is the degree distribution of the network, being independent of time.

The Figures 7.15, 7.16, 7.17, 7.18, 7.19 show some examples of congruity distributions of

networks. Figure 7.15 shows the node congruity distribution of M. musculus Gene Regu-

latory Network, simulated according to f2 described above, until the scalar assortativity

stabilises at its maximum (which was, in this case L =0.94). Thus, this network at the

considered point in time has near perfect scalar assortativity. We note that the congruity

distribution shows a strong correlation between node degree and node congruity. That is,

it is the hubs which have the highest congruity. However, we may note that the relation-

ship between node congruity and node degree is not linear. That is, there are some nodes

which seems to have higher or lower congruity than predicted by a linear correlation with

degrees. This shows that the overall placement of a node in the network (not merely the

degree of the nodes), as well as the overall distribution of node states across the network,

plays a part in node congruity.


Figure 7.16 shows the congruity distribution of E. coli transcription network, simulated

according to f3 described above, until the scalar assortativity stabilises at its minimum

(which was, in this case L t = −0.52). Let us note that, as mentioned above, minimal

scalar assortativity is harder to achieve in a network topology, since it requires neighbour-

ing nodes to have different values, and the topology may make this harder to achieve. We

note that the congruity distribution shows again strong correlation between node degree

and node congruity, with the hubs having the highest negative congruity. Again, we may

note that the relationship between node congruity and node degree is not linear. Figure

7.17 shows the node congruity distribution of E. coli transcription network, simulated

according to random logic: logic f1. Here the scalar assortativity remains close to scalar

non-assortativity (L t = 0.10) and we may see that there is no recognisable correlation be-

tween node congruity and node degree. Other simulated networks confirmed the patterns

in the results described above.

When a network has maximal scalar assortativity, (L t = 0.94) does it mean that all nodes

in the network will have positive node congruity, or merely the majority of nodes will?

This question cannot be answered by the plots above, since the average node congruity is

plotted against degree. In the following two Figures 7.18, 7.19, we show node congruity

of all individual nodes, where network scalar assortativity is either maximal or minimal.

It can be noted that when scalar assortativity is maximal, all nodes have positive node

congruity. However, when scalar assortativity is minimal, quite a few nodes still have

positive congruity. This is again a property of congruity, since as we explained above, it is

not always possible for a node to be different from all of its neighbours. However, it could

be easily similar to all of its neighbours, if almost all nodes have similar states anyway.

It is important to note that this result is true only for binary states — specifically, if the

number of states are comparable to, or higher than, the number of nodes, then it is much

easier for the nodes to be dissimilar. Thus node congruity profiles gives us interesting

insights about the interplay between node states, average neighbour-degree, and network

size in a network with complex dynamics.


0

0.001

0.002

0.003

0.004

0.005

0.006

0 20 40 60 80 100 120 140

node

con

grui

ty

node degree

Figure 7.15: Node congruity profile. The M. musculus Gene regulatory network is simu-lated with logic f2. L = 0.94.

-0.16

-0.14

-0.12

-0.1

-0.08

-0.06

-0.04

-0.02

0

0.02

0 50 100 150 200 250 300 350 400

node

con

grui

ty

node degree

Figure 7.16: Node congruity profile. The E. coli transcription network is simulated withlogic f3. L = −0.52.


-0.015

-0.01

-0.005

0

0.005

0.01

0 50 100 150 200 250 300 350 400

node

con

grui

ty

node degree

Figure 7.17: Node congruity profile: The E. coli transcription network is simulated withlogic f1. L = 0.10.

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

0.009

0.01

0 100 200 300 400 500 600 700 800 900 1000

node

con

grui

ty

node ID

Figure 7.18: Node congruity profile with individual nodes. The M. musculus Gene regu-latory network is simulated with logic f2. L = 0.94.

7.7 Summary 125

-0.025

-0.02

-0.015

-0.01

-0.005

0

0.005

0.01

0.015

0.02

0 200 400 600 800 1000 1200

node

con

grui

ty

node ID

Figure 7.19: Node congruity profile with individual nodes. The E. coli transcriptionnetwork is simulated with logic f3. L = −0.52.

7.7 Summary

The dynamics of a network is influenced by its topology, and the topology of a network

evolves as a result of its functional requirements and dynamics. Therefore, the patterns

in a network’s dynamics and its topology are closely interdependent. In this chapter, we

considered scalar assortativity as a function of time. It measures the tendency of nodes

in a network to make links with similar nodes, where similarity is interpreted in terms

of node states, rather than node degrees. High positive scalar assortativity of a network

means that connected nodes tend to have similar node states at a given time, whereas high

negative scalar assortativity means that connected nodes tend to have dissimilar states.

Scalar assortativity can vary with time and show tendencies which give information about

the dynamics of the network. Using simulated boolean networks, we showed that networks

which initially have nodes with random states, can in time achieve either high positive

or high negative scalar assortativity, depending on the functionality of the nodes (the

logic that the nodes implement). We also showed that such networks can have high

positive or high negative scalar assortativity, even if nodes are no more likely to be in one

7.7 Summary 126

state than the other (the state distribution is more or less uniform). We pointed out a

number of scenarios where scalar assortativity could be used to measure dynamics of real-

world networks. Introducing appropriate entropy and information content measures, we

quantified the relationship between network scalar assortativity and information content

of networks.

Finally, we introduced node congruity as an individual node’s contribution to scalar as-

sortativity, and showed that the node congruity can be interpreted as a scaled difference

between (i) the average state of the node’s neighbours, and (ii) the average state across

the entire network (i.e., the expected global state). Using the concept of congruity, we

showed that congruity distributions provide an additional tool to understand a network’s

dynamics. The introduced tools may be used in quantifying properties of complex net-

works, and contribute to studies of functional motifs, dynamic behaviour, growth models,

etc.

This chapter concludes our theoretical analysis of assortativity in complex networks in

global and local levels. The next two chapters will present two application scenarios of

assortative mixing.

Chapter 8

A growth model based on local

assortativity profiles

8.1 Introduction

In the previous chapters we introduced and analysed local assortativity. While the ap-

plications of this concept are wide spread, it is pertinent to present some applications

here which have been the focus of our research. In this chapter we tackle the subject of

growth models based on the local assortativity profiles. Our focus here is on presenting a

growth model for Internet Autonomous Systems level network. However, it will be shown

that this growth model could be utilised to simulate the evolution of any network which

matches a certain type of local assortativity profile.

In the past decade, multiple large-scale complex networks have been analysed in terms of

their global topology and local structure [15, 16, 68, 91–93, 100, 108]. One network that

has been given much attention is the Internet Autonomous System (AS) level network,

where each node represents an Autonomous System present in the Internet and the edges

represent a commercial agreement between two Internet Service Providers(who own the

two ASs). Such an agreement defines whether they agree to exchange data traffic and how

to charge each other. As such, the AS graph is the ‘control plane’ of Internet. The AS

network of Internet has seen very rapid growth over the recent years (from about 3000

nodes in 1998 to about 25000 nodes in 2008) and the growth of this network has been well


documented with snapshots of the network being available on a regular basis [5]. As such,

Internet AS networks present very realistic opportunities to gain insight into the evolution

of complex networks.

The global structure of Internet AS networks is known to have a power law degree distribu-

tion (with scale-free exponent γ = −2.2 as reported in [41, 42] for networks at that time)

and a tier architecture. It is also known to display community structure and rich-club

phenomenon[125]. We will show in this chapter that Internet AS networks are disassorta-

tive with disassortative hubs, with their local assortativity profiles becoming almost linear

for high degrees. Indeed, these profiles seem to hint that the evolution or growth of In-

ternet is driven by fundamentally different design principles to that of most social and

biological networks in the previous chapters.

Several growth models exist to simulate the growth of the Internet, including the Inet

3.0 model [118], the Barabasi–Albert model [14], the Generalised Linear Preference model

[33], the Interactive Growth model [125], and the Positive Feedback Preference (PFP)

model [124]. Some of these models are capable of matching the degree distribution, and

community structure (including the rich-club phenomena) of the Internet AS networks.

However, these models do not generate topologies that match the local assortativity dis-

tribution of the Internet. In fact, as we will show, these models mostly generate local

assortativity profiles that are much more similar to biological and social networks that we

have considered in the previous chapters.

Therefore in this chapter a new growth model for Internet is presented, which we call the

Parallel Addition and Rewiring Growth (PARG) model. The PARG model satisfactorily

explains the local assortativity distribution of the Internet, while retaining the ability to

reflect the scale-free nature, and other properties explained by existing growth models.

The PARG model we present rearranges links in parallel to addition of nodes, as is the

case with real Internet growth. We make detailed comparisons between PARG model and

other existing growth models. Finally, we outline possible applications and significance of

the new growth model.

This chapter is organised as follows: In section 8.2 we present the local assortativity profiles

for real world Internet AS networks. In section 8.3 we present a review of existing growth

models for Internet, and analyse the local assortativity profiles and other topological

features of the networks produced by these models compared to the real AS networks.

8.2 Local assortativity distributions of Internet at the AS level 129

In section 8.4 we motivate the new growth model. In section 8.5 the PARG model is

presented in detail while in section 8.6 the performance of this model is analysed. Section

8.7 provides the chapter summary.

8.2 Local assortativity distributions of Internet at the AS

level

We introduced local assortativity distributions in chapter 5. Let us consider the local

assortativity distributions of Internet AS networks. Such distributions for AS networks

in the years of 1998, 1999, 2000, and 2008 are shown in Figures 8.1,8.2, along with their

network assortativity values.

It could be observed from the figures that these networks fall into class (iv) we introduced

in chapter 5, namely disassortative networks with disassortative hubs. Furthermore, the

local assortativity profiles are very smooth unlike most of the profiles shown in chapter

5, and they become nearly linear for higher values of degree. It could also be observed

that no degree shows a positive average local assortativity (the profiles consist entirely

of negative values). We produced the local assortativity profiles of AS networks in the

intervening years (2000 - 2007) and obtained qualitatively identical results. Therefore it

is worth exploring what network growth / evolution mechanisms could produce networks

which could have such local assortativity profiles. This assertion brings us to the topic of

growth models for Internet, which we can use to simulate the evolution of Internet over

the years.

A number of growth models exist to simulate the growth of Internet [125], and most of

these models are developed specifically to model Internet. Some of the most prominent

models are Inet 3.0 model [118], the Barabasi–Albert model [14], the Generalised Linear

Preference model [33], the Interactive Growth model [125], and the Positive Feedback

Preference Model[124]. Let us briefly overview these existing models.

8.2 Local assortativity distributions of Internet at the AS level 130

-0.09

-0.08

-0.07

-0.06

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0 200 400 600 800 1000 1200 1400 1600

avg.

loca

l ass

orta

tivity

degree

9899

2000

Figure 8.1: Local assortativity distribution of Internet at the AS level, in years 1998(r = −0.198: diamonds), 1999 (r = −0.174: pluses), 2000 (r = −0.16: squares).

-0.025

-0.02

-0.015

-0.01

-0.005

0

0.005

0 500 1000 1500 2000 2500 3000 3500

avg.

loca

l ass

orta

tivity

degree

AS2008

Figure 8.2: Local assortativity distribution of Internet at the AS level, in 2008, August(r = −0.13).

8.3 Growth models of Internet at the AS level 131

8.3 Growth models of Internet at the AS level

8.3.1 Inet 3.0 model

The Inet 3.0 model [118] is capable of matching the degree distributions of real AS Graphs.

Given the degree distribution of the AS network that it needs to model, the Inet 3.0

mechanism assigns degrees to the given number of nodes to match the desired degree

distribution, then connects these nodes using a three step process. Nodes are connected

to other nodes with ‘free’ degrees using weighted linear preference. However, it has been

noted that the model typically generates 25 percent less links than the real extended AS

graphs [125].

8.3.2 The Barabasi–Albert (BA) model

The BA model [14] first explained how a power law degree distribution can arise from

a growth model, by introducing preferential attachment of nodes along with growth. In

the BA model, new nodes attach themselves preferentially to nodes which already have a

higher number of links. That is, the probability of an existing node i with degree ki to be

selected is

pi =ki∑j kj

(8.1)

The BA model has inspired many growth models that followed and has been used as a

starting point in some of them [33, 125]. However, it has not been proposed specifically

to be a growth model for Internet, and networks produced by it generally do not match

the parameters of AS networks (even apart from the local assortativity profiles).

8.3.3 The Generalised Linear Preference (GLP) model

The GLP model [33] improves on the BA model by splitting the growth in two parts; (i)

the addition of new nodes (ii) the addition of new links between existing nodes. starting

with m0 nodes connected by m0 − 1 links, it performs one of the following operations

at each time step. (i) with probability p, m new links are added between nodes chosen


-0.0006

-0.0005

-0.0004

-0.0003

-0.0002

-0.0001

0

0.0001

0.0002

0.0003

0 5 10 15 20 25 30 35 40 45 50

loca

l ass

orta

tivity

degree

Figure 8.3: Local assortativity distribution of a network grown with preferential attach-ment (The Barabasi–Albert Model). The network size is that of Internet AS network in1998.

-0.001

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

0.009

0 20 40 60 80 100 120 140 160

avg.

loca

l ass

orta

tivity

degree

IG

Figure 8.4: Local assortativity distribution of a network grown using the InteractiveGrowth model proposed by [125]. The network size is that of Internet AS network in1998.


-0.05

-0.045

-0.04

-0.035

-0.03

-0.025

-0.02

-0.015

-0.01

-0.005

0

0.005

0 200 400 600 800 1000 1200

loca

l ass

orta

tivity

degree

Figure 8.5: Local assortativity profile of a network grown using the PFP model for δ =0.021

preferentially, and (ii) with probability 1− p, one new node is added and connected to m

existing nodes chosen preferentially. The probability of an existing node i with degree ki

to be selected is

pi =ki − β∑j kj − β

(8.2)

where β is a parameter, which when set to zero reduces the mechanism to exactly that of

BA. Thus, it generalises the BA mechanism and, with suitable choice of β, matches the

real AS graphs in degree distribution, clustering coefficient and path lengths [33, 125].

8.3.4 The Interactive Growth (IG) model

The Interactive Growth model has been proposed recently [125] to model the rich club

phenomena in the real AS graphs. The model starts with a random graph of m0 nodes

and the same number of links. At each time step (i) with 40 percent probability, a new

node is connected to one host node and the host node is connected to two peer nodes (ii)

with 60 percent probability, a new node is connected to two host nodes and one of the


-0.09

-0.08

-0.07

-0.06

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0 200 400 600 800 1000 1200 1400

loca

l ass

orta

tivity

degree

Figure 8.6: Local assortativity profile of a network grown using the PFP model for δ =0.042

host nodes (randomly selected) is connected to one peer node1. Thus three new links are

added at each time step.

The IG model thus determines the link density a priori, without parameterising it. How-

ever, it has been argued that the model captures the degree distribution and link distri-

bution of the real AS graphs [125]. In addition, the model is able to capture the rich-club

phenomena [39, 126]. A rich-club is defined in terms of degree-based rank r of nodes, and

the rich-club connectivity ϕ(r). The degree-based rank denotes the rank of a given node

when all nodes are ordered in terms of their degrees, highest first. This is then normalised

by the total number of nodes. The rich-club connectivity is defined as the ratio of actual

number of links over the maximum possible number of links between nodes with rank

less than r. Thus, it is possible to calculate the rich-club connectivity distribution of a

network, ϕ(r) over r. It has been shown that the IG model is able to very closely match

the ϕ(r) over r distribution of real AS graphs [125].1There is no difference in ‘type’ between host nodes and peer nodes. A node which acted as a host

node during one node addition could be selected as a peer node during another node addition. Any nodeto which the incoming node is directly connected is called the ‘host’ node, and any node then selected tomake additional links with the host node is called the ‘peer’ node


-0.014

-0.012

-0.01

-0.008

-0.006

-0.004

-0.002

0

0.002

0.004

0.006

0.008

0 50 100 150 200 250

loca

l ass

orta

tivity

degree

-0.001

-0.0008

-0.0006

-0.0004

-0.0002

0

0.0002

0.0004

0.0006

0.0008

0.001

0.0012

0 10 20 30 40 50 60

loca

l ass

orta

tivity

degree

-0.00014

-0.00012

-0.0001

-8e-005

-6e-005

-4e-005

-2e-005

0

2e-005

4e-005

6e-005

8e-005

0 5 10 15 20 25 30

loca

l ass

orta

tivity

degree

Figure 8.7: Local assortativity distribution of networks grown with the Barabasi–Albertmodel, with various parameters Elink =0.25 (green filled square), 0.5 (red circle), 0.75(blue circle), 1.0 (green cross), 1.25 (red star), 1.5 (green star) and 1.75 (red plus). Thenetwork size is that of Internet AS network in 1998. Note that the parameter Elink = 1.0gives compatible number of links to that of Internet AS 98 Network.


-0.004

-0.003

-0.002

-0.001

0

0.001

0.002

0.003

2 4 6 8 10 12 14

loca

l ass

orta

tivity

degree

Figure 8.8: Local assortativity distribution of a network grown as an Erdos–Renyi randomgraph. The network size is that of Internet AS network in 1998.

8.3.5 The Positive Feedback Preference (PFP) model

The PFP model is derived from the IG model [124]. It follows a similar mechanism,

except that to select the host and peer nodes it uses a non-linear preference called Positive

Feedback Preference. In PFP, the selection probability of a node is determined by

pi =k

1+δlog(ki)i∑

j kj1+δlog(kj)

(8.3)

where δ is a parameter of the model, e.g δ = 0.021 (which is the recommended value of

the parameter by [124]). The non-linear preference is utilized to achieve better similarity

to the AS graph’s attributes compared to the IG model, for example in terms of getting

the appropriate maximum degree.

The PFP model compares favourably with almost all measurable attributes of the AS

graph, including the rich club coefficients [124], but not the local assortativity profiles, as

we will show.


8.3.6 Growth models and local assortativity distributions

Most of the models reviewed above have been shown to model the degree distribution and

community structure (in terms of rich-club connectivity) of the Internet reasonably well,

as well as accounting for the Internet’s scale-free nature. However, these models have not

been validated for Internet’s local assortativity distributions. Therefore, we used each of

these models to grow networks and calculated the local assortativity distributions. Our

observation was that these models fail to capture the local assortativity distribution of

Internet, as Figures 8.3, 8.4, 8.7, 8.8 show.

The Barabasi–Albert model does not show a negative local assortativity profile 8.3. This

model could be used to grow the networks with a constant (Elink) which is a parameter

of the model. We varied this parameter and the results are shown in Figure 8.7. None

of the profiles in this figure match the real AS 1998 network profile, the size of which we

used in these experiments. Furthermore, none of the profiles match the maximum degree

of the AS 1998 network either. We should note that only the value Elink = 1.0 matches

the number of links in AS 1998 network. Similar results were obtained for the Generalised

linear Preference model.

Some models display assortative hubs, such as the Interactive Growth model, and the

Erdos–Renyi random network model [15, 16, 82]. Such models each have a number of

parameters, and we verified that changing these parameters do not affect the overall local

assortativity profile. We studied the PFP model in detail as it has been shown to be

the best model around to model AS networks by some distance [124]. However, the local

assortativity profiles generated by the PFP model do not completely match the real AS

network profiles, as shown in Figures 8.5, 8.6. As these figures show, the PFP model

is capable of producing local assortativity profiles with disassortative hubs, though the

values for smaller degrees are slightly positive. In any case the profile is not smooth or as

evenly spread out as that of the AS networks is. Specifically for higher values of δ which

seem necessary to produce disassortative hubs, the profile seems punctuated, i.e there is a

big ‘gap’ between the degrees of the biggest hub and other networks. This is not surprising

given the nature of the positive feedback mechanism of the model, in which ‘the rich not

only get richer, but get disproportionally richer’, as the authors of the model point out

[124], and that effect is even more pronounced for high δ, which implies strong positive

feedback. Therefore, while the PFP model overall is quite effective in modelling the AS

8.4 A network motif with negative local assortativity distribution 138

networks, it produces a punctuated local assortativity profile which also shows positive

values for small degrees.

Figures 8.1, 8.2, 8.3, 8.4,8.5, 8.6, 8.7, 8.8, illustrate that the existing models produce

local assortativity profiles that do not match those of the real Internet AS networks (see

also the table 8.2), warranting a new growth model. Such a new model should capture

not only the local assortativity distribution of Internet, but also the attributes already

captured by the existing models — namely degree distributions and community structure.

In the next sections we present such a growth model, which we call the Parallel Addition

and Rewiring Growth (PARG) model. Before presenting a step-by-step description of the

model, we explain the motivation behind it.

8.4 A network motif with negative local assortativity dis-

tribution

To motivate the new growth model, it is important first to recognise a network motif which

has the property of negative local assortativity distribution with disassortative hubs, as

well as being scale-free. Then a network can be constructed by connecting together such

motifs or growing them in a scale-free manner.

Let us first note that a star motif has these properties at a very elementary level [92].

A star motif can have nodes with only two different degrees, and all nodes are locally

disassortative — the local assortativity distribution of a star motif is shown in Figure 8.9

[92].

Clearly, if we consider a number of larger star motifs with varying maximum degrees

as a network, the local assortativity distribution of the overall network would be always

negative and increase (in absolute value) with degree. However, these star motifs have to be

interconnected to form a single network, without compromising the locally disassortative

nature of the nodes. This can be accomplished if the hubs in the stars are connected to

peripheral nodes in other stars, so that the links are disassortative in nature. The likelihood

of hub-to-hub links must be reduced, so that they only form a very small proportion of

all links in the network. Such a network is shown in Figure 8.10. Note that this network

is scale-free.


-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0 2 4 6 8 10 12 14 16 18 20

avg.

loca

l ass

orta

tiven

ess

degree

Figure 8.9: Local assortativity distribution of a star motif with highest degree = 19.

It is evident that a growth model that satisfies the local assortativity distribution of the

Internet has to contain many instances of the pattern above. However, when the network

is sufficiently large, a number of assortative links can also appear, as they are not likely

to affect the distribution in a big way. Indeed, Internet AS networks are by no means a

hierarchy of stars as the above motif is. On the other hand, AS networks have been shown

to display the ‘rich-club’ phenomena, where most of the hubs are densely connected to

each other [39, 124–126]. Nevertheless, many such hierarchies of stars must be interwoven

in the Internet topology for the overall network to maintain a negative local assortativity

profile.

We also made a few other observations about the topology of Internet in designing our

growth model. We noted that nodes are constantly being deleted as well as added in

the Internet topology. For example, between January 2004 and February 2004, 2431 new

nodes were added while 2061 were deleted, making a net increase of 370 nodes [5]. This

however, is not mainly due to Internet Service providers going out of business, but due

to the permanent variation of interconnections [41]. Connections between AS members

are constantly rearranged and may flicker, and if a AS node has only a few connections,

it is actually possible that all connections may be shut down from time to time [41]. At


Figure 8.10: A network motif that displays negative local assortativity distribution cor-related with node degree. Note that this is essentially a hierarchy of stars with hubsconnected together via linking nodes.

such times, it appears as if the node has been deleted. The actual node mortality is small

compared to this. Therefore, it is important that node deletion, as well as addition, needs

to be explicitly modelled. None of the existing growth models take this into account. While

the network is growing on average and as such can be modelled by purely joining nodes, the

connection patterns are affected by the deletion. For example, the preferential attachment

model assumes that all joining nodes preferentially attach themselves to existing nodes.

However, those nodes that are deleted are not likely to be chosen in a preferential way;

therefore the resulting patterns in the link distribution are not wholly captured.

Taking these facts into account, we present a growth model that resembles the preferential

attachment mechanism only in part. That is, nodes join preferentially with existing hubs,

making the formation of giant hubs possible. At the same time, another mechanism

is at work, which disfavours assortative links. That is, assortative links are replaced

by disassortative links, giving way to the emergence of star-like motifs and ultimately a


-0.3

-0.25

-0.2

-0.15

-0.1

-0.05

0

2 4 6 8 10 12 14 16

loca

l ass

orta

tivity

degree

Figure 8.11: The local assortativity distribution of the network motif displayed in figure8.10.

negative local assortativity profile.

Let us note here, however, that the growth model we present here is a generic model to

generate disassortative local assortativity profiles. While it has been motivated by the

Internet, it is not a model limited to Internet growth. However, with a suitable set of

parameters it captures reasonably well other features of Internet AS networks such as

degree distribution, maximum degree and rich club coefficients.

We do not explicitly model deleting nodes in PARG model, though we do explicitly model

deleting links. Service agreements between Internet Service Providers (ISP) appear and

disappear all the time, and as such the disappearance of links have to be modelled ex-

plicitly. An ISP going out of business would be comparably less frequent. Furthermore,

the constant deletion of links in parallel to the growth of network seems to be one of the

driving forces behind the negative local assortativity profile and disassortative hubs that

the AS network displays. Now we proceed to present a step-by-step description of the

PARG model.

8.5 The PARG Model for Internet growth 142

8.5 The PARG Model for Internet growth

The PARG model contains two mechanisms of growth. One is the node-addition mech-

anism, and the second is the link rearrangement mechanism. These mechanisms work in

parallel. Specifically, each time a node is added to the network some links are rearranged

stochastically. The rate of rearrangement is a parameter of the model.

Our node-addition mechanism closely reflects the BA model of Internet growth [14], as it

has been shown that this model sufficiently explains the scale-free nature and power law

degree distributions of Internet. Below we present the model in detail.

The model starts the growth from a small initial network of size N0. The initial network

could be a simple random graph.

At each time step a new node is added to the network.

• The new node stochastically makes Nadd number of links with existing nodes. That

is, the joining node makes a number of links with the expected number of links being

Nadd.

The new node connects to the existing nodes preferentially. That is, a node’s probability

to be selected to have a link with the joining node is proportional to the number of its

existing links. Formally, the probability of an existing node i with degree ki to be selected

is

pi =ki∑j kj

(8.4)

After each node addition:

• Probabilistically choose and delete Ndel number of assortative links in the network.

This is done in the following fashion:

– Choose Ncut number of the highest degreed nodes from the network (sort nodes

based on degree and choose the first Ncut number of nodes.)

– Each link in the selected node is stochastically deleted with a probability that

is inversely proportional to the degree of that node.

– The actual probability is calculated so that the expected number of link dele-

tions is maintained at Ndel.


Formally, if a node with degree d from the network has degree-based rank rankd ≥ Ncut,

the probability of a link of that node being deleted is:

p =Ndel

d dcut(8.5)

where dcut corresponds to the degree of the node that has exactly the rank of rankd.

Otherwise (if rankd < Ncut) the probability of a link of that node being deleted is:

p = 0 (8.6)

• Delete the chosen links.

• For each deleted link, add two links to the network;

– The node with the higher degree among the two nodes that were connected by

the deleted link, node s, is chosen as the node to create these new links from.

– Another two nodes p1, p2 are selected from the network.

– These nodes are selected in anti–preferential fashion. That is, nodes in the

network are sorted according to degree, the highest degreed first. The proba-

bility of a node being selected is proportional to its rank in the sorted list. The

higher the rank (in terms of absolute value), the higher the probability is to get

selected, e.g., a node that is ranked 20th is twice as likely to be selected than a

node that is ranked 10th.

– Two new links are created: one connecting s with p1, and another connecting

s with p2.

This process is repeated until the desired number of nodes are added.

Nadd, Ndel and Ncut are parameters of our model. For example, the values of these

parameters that were used in to simulate the AS 1998 network are summarised in table

8.1 (Other suitable values were used to simulate other AS networks). The parameters

Nadd and Ndel together determine the number of links in the network.

The idea behind the second mechanism is that assortative links are deleted and replaced

by disassortative links. Note that we are selecting links from high-degreed nodes to be

deleted, so these links are likely to be assortative. When the links are replaced, though,


PARG Top 1%rich club

Top 2% rich club Maximum Degree

Nadd Ndel Ncut

0.04 4.8 1 + 0.04N 27 % 11 % 650

0.006 4.8 1 + 0.2N 35 % 18 % 655

AS 98 37 % 17 % 641

Table 8.1: Parameters of PARG model and rich-club phenomena based on AS 98 networks.N is the network size.

the node with the higher degree (s) is chosen for one end of these links — but the nodes

at the other end (both p1 and p2) are chosen anti-preferentially. This effectively results in

disassortative links. The deletion / replacement of such links counters, to some extent, the

nature of preferential association where hubs are preferred to form links. The end result

is that while hubs are allowed to form, they are discouraged to form links with other hubs

excessively. The hubs among AS network connect mostly to relatively peripheral nodes,

and while they maintain links with other hubs, such links are only a small proportion of

all the links the hubs may have.

Let us note also that the PARG model growth will result in some nodes being ‘dropped’

from the network, even though such occurrences will be rare. Specifically, when a link is

deleted and replaced by two links to the node which has the higher degree, the node with

lower degree may drop out of the network if it had only that link which has been deleted.

As we explained above, this however is also the case with real AS networks, where nodes

do drop out temporarily during network growth. This is yet another aspect of Internet

growth that the PARG model captures.

Figure 8.12 shows a sub-network of 350 nodes grown by the PARG network. It is possible

to observe that the network contains many star motifs and the hubs are often not directly

connected, though some assortative links are visible too. A visual comparison with the

Internet AS networks is not possible since these are much larger and the design patterns

cannot be clearly discerned from a figure. However, as we show in the next section, the

PARG model seems to capture well the topological design patterns of Internet, including

the local assortativity distribution.

In the next section, we analyse the performance of the PARG model.


8.6 The local assortativity distribution of networks grown

by the PARG model

Figure 8.12: A sub-network of 350 nodes grown by the PARG model. The original grownnetwork contained 3000 nodes, out of which 350 nodes were randomly chosen with theirlinks to illustrate the connection patterns. Note that it is highly similar, albeit bigger, tothe motif we proposed earlier ( in Figure 8.10).

Figure 8.13 shows the local assortativity distribution of two networks produced by the

PARG model. It can be noted that the PARG model produces a local assortativity dis-

tribution which is negative with disassortative hubs and becomes linear for high degrees

— similar to the real Internet AS network. As Figures 8.14, 8.15 show, the networks

produced by PARG model are also scale-free and their degree distributions are compatible

with the degree distributions of the Internet. Table 8.1 shows that the PARG model can

produce rich-club coefficients and maximum degrees comparable with AS graphs. Thus,

the PARG model is successful in producing the desired local assortativity profile while

retaining other aspects of Internet that are already modelled.

We undertook a detailed comparison study of the PARG model with other existing growth


-0.08

-0.07

-0.06

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0 100 200 300 400 500 600 700

avg.

loca

l ass

orta

tivity

degree

PARG 3000PARG 1000

Figure 8.13: The local assortativity distribution of two networks grown by the PARGmodel (Network size = 1000 nodes and 3000 nodes respectively). Number of Links = 2000and 6100 respectively. Note that the second network corresponds roughly to the size ofInternet AS network in 1998.

0

200

400

600

800

1000

1 10 100

freq

uenc

y

degree

Figure 8.14: Degree distribution of a network grown by the PARG model. Nodes = 3000.Links = 6700 (roughly corresponds to AS 98 network). Network assortativity r = −0.28.


0

200

400

600

800

1000

1200

1400

1 10 100

freq

uenc

y

degree

Figure 8.15: Degree distribution of the real AS 98 network. Nodes = 3000. Links = 6100.Network assortativity r = −0.198.

models. For this purpose we considered the Barabasi–Albert (BA) model, Interactive

Growth (IG) model, Generalized Linear Preference (GLP) model, Positive Feedback Pref-

erence (PFP) model, as well as our PARG model, contrasted with the real AS networks.

Using PARG model, we have grown networks compatible with both AS 98 (3000 nodes)

and AS 2008 (25000 nodes) networks. We considered the power law exponent γ and

assortativity r of the grown networks, as well as the ability to produce scale-free charac-

teristics, the nature of hubs, the negative local assortativity profiles, and the ability to

produce community structure. Our results are summarised in Table 8.2.

Table 8.2 shows that only the PARG model is able to produce a network that has a

negative local assortativity profile and disassortative hubs, as is the case with real AS

networks. Meanwhile, the PARG model reasonably retains the ability to model other

aspects of Internet topology, such as degree distribution and community structure.

It is pertinent here to discuss in some detail the capacity of the PARG model to produce the

rich-club phenomena. The PARG model has been motivated by a desire to grow networks

which show negative local assortativity profiles with disassortative hubs. However, having

disassortative hubs does not mean that the hubs cannot be interconnected. It merely

means that a very high proportion of the links of these hubs are connected to comparatively


Net

wor

kA

S‘98

AS‘

03A

S‘08

PAR

GPA

RG

BA

IGG

LP

PFP

Inet

Size

3000

1600

025

000

3000

2500

030

0030

0030

0030

0030

00

Scal

e-fr

eeY

esY

esY

esY

esY

esY

esY

esY

esY

esY

es

Pow

erla

wex

pone

ntγ

-2.2

2-2

.16

-2.0

4-2

.08

-1.9

6-2

.9-2

.1-2

.1-2

.2-2

.1

asso

rtat

ivity

r-0

.198

-0.1

4-0

.13

-0.2

8-0

.26

-0.0

9-0

.28

-0.2

-0.2

3-0

.2

Ass

orta

tive

hubs

No

No

No

No

No

Yes

Yes

Yes

No

Yes

nega

tive

loca

las

sort

ativ

ity

Yes

Yes

Yes

Yes

Yes

No

No

No

Yes

/No

No

Ric

hcl

ubY

esY

esY

esY

esY

esN

oY

esN

oY

esN

o

Tab

le8.

2:A

com

pari

son

betw

een

grow

thm

odel

san

dth

ere

alIn

tern

etA

Sne

twor

k.

8.7 Summary 149

peripheral nodes. It should be noted that, as pointed out in [92], the rich-club phenomenon

is in some way connected to local assortativity, albeit that the rich-club phenomenon is

concerned with hubs only and cannot be used to analyse peripheral nodes alone. In other

words, the rich-club coefficient represents the cumulative local assortativity aggregated

from the highest-degreed nodes toward the smaller degreed nodes [92]. The relationship

between the rich-club coefficient ϕ(r) and the local assortativity ρ(r), both plotted over

rank r, has been explored in Appendix C. On the other hand, networks which show positive

local assortative profiles with assortative hubs (class I), and networks which show negative

local assortative profiles with disassortative hubs (class IV) may both display strong rich

club connectivity. As such, rich club connectivity and local assortativity remain related

but independently relevant concepts to analyse complex networks. The PARG model is

able to capture both in a way comparable to real AS graphs. The one-percent and two-

percent rich club connectivities of the PARG model is shown in Table 8.1, along with the

maximum degrees of the produced networks.

8.7 Summary

In this chapter, we have presented a new growth model — the PARG model — which

grows networks with negative local assortativity profiles and disassortative hubs. It is a

dynamic model that includes two parallel mechanisms: a node addition mechanism which

is similar to the preferential attachment, as well as a link rearrangement mechanism which

ensures a negative local assortativity distribution for the network. The growth model

satisfies the local assortativity distribution for real AS networks. The model also captures

link deletion and nodes dropping out (mostly temporarily) as a result, which occurs in

real AS networks but has not been hitherto captured by existing growth models.

We have compared the PARG model with existing growth models for Internet. We found

that the PARG model captures the degree distribution and rich-club phenomena as well

as do other existing models, in addition to being unique in capturing local assortativity

profiles. (We observed that the PFP model produces severely punctuated and not perfectly

negative local assortative profiles). The PARG model, though motivated as a growth model

to explain local assortativity profiles of the Internet, could be used to model any network

that has negative local assortativity profiles with disassortative hubs. Therefore, together

8.7 Summary 150

with the measure of local assortativity, this growth model has the potential to greatly aid

the simulation, design and analysis of complex networks in general.

Chapter 9

Information cloning using

assortativity

9.1 Introduction

In this chapter, we consider a task of information-cloning of a scale-free network using as-

sortative mixing, given a fragment and some topological properties of the original network.

Following the previous chapter where a growth model was presented which makes use of

the local assortativity profiles to evolve a network with given topological features, this

chapter explores another application of analysing assortative mixing in detail. The “clo-

ning” is interpreted information-theoretically: the resulting network may disagree with the

original one in terms of specific node to node connections, but is required to have equiv-

alent information content. The information-cloning task is partly motivated by needs of

network manufacturing, where an “assembly-line” starts with a fragment and continues

with “manufacturing” the rest, subject to topological constraints. Another motivation is

regeneration of scale-free networks which are prone to percolation/diffusion of adverse con-

ditions, as well as removal of highly connected nodes. Both demands (topology-oriented

manufacturing and regeneration) are referred in this chapter as network recovery.

Recovery of networks can be attempted and evaluated in various ways. In this chapter,

we aim at a general measure in terms of mutual information contained in the network.

More precisely, we propose to judge success of the recovery with respect to the amount of

information content regained by a resulting network.


The extent of assortativity affects network’s resilience under node removal or percola-

tion/diffusion of adverse conditions [81]. Our objective is an investigation of how success-

ful is a network recovery in terms of assortativity and information content. We note that

this objective is different from investigation of networks’ robustness properties such as

error tolerance, attack survivability, or network fragmentation that have been extensively

studied [17, 40, 78]. For example, Moreno et al. [78] explored robustness of large scale-free

networks faced with node-breaking avalanches (cascading failures when a failure of a node

triggers subsequent failures of neighbours), and investigated how the random removal of

nodes in a fixed proportion affects the global connectivity and functionality of scale-free

networks. Stauffer and Sahimi studied scale-free networks with annealed disorder [110],

when the links between various nodes may temporarily be lost and reestablished again

later on, and observed a number of critical phenomena, e.g. “the existence of a phase

diagram that separates the region in which diffusion is possible from one in which diffu-

sion is impossible”. Their study did not investigate, however, the role of assortativity and

information content in the diffusion process.

Utilising the Assortative Preferential Attachment (APA) method introduced in chapter 3,

we investigate here recovery of scale-free networks in terms of their information content. In

chapter 3, we argued that networks with the same assortativity r and the same distribution

qk could have different information contents I — because they may disagree on ej,k — and

observed that, under certain conditions, the information transfer non-linearly depends on

the absolute value of the assortativity (i.e. mutual information increases when assortativity

varies in either positive or negative direction). For example, this relationship for class A

networks (see chapter 3) and a set of topological parameters is illustrated in Figure 9.1

which is similar to Figure 3.2 from chapter 3. Now we can capitalise on the fact that, under

certain conditions, the knowledge of r allows one to determine the information content I(r)

uniquely. Specifically, we intend to recover a network by growing the missing fragments

in such a way that the resulting assortativity (and hence, the information content) is as

close as possible to the original one, while other network parameters are kept constant.

9.2 Information cloning using Assortative Preferential attachment 153

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

Info

rmat

ion

Tra

nsfe

r

Assortativeness

Figure 9.1: information content I(r) as a function of r, for a qk distribution with γ = 1;’+’ indicate Np = 4; ’×’ indicate Np = 8; ’∗’ indicate Np = 12; ¤ indicate Np = 16.This figure is reproduced from [91] and is similar to Figure 3.2 for class A networks.

9.2 Information cloning using Assortative Preferential at-

tachment

We utilized the APA method (described in chapter 3 Appendix) to grow and/or recover

scale-free networks with varying assortativity values1. Each experiment involved a set of

networks with fixed degree distributions pk (that is, fixed γ = 1, or γ = 3, and Np = 16),

and varying assortativity values r = 1, r = 0 and r = rmin. In the case γ = 1, the

disassortativity extreme rmin = −1. Each original network (for each r) was grown with

APA, and resulting information content I0(r) provided the point of reference. Then the

network was progressively modified by removing a certain percentage (deficit) δ of nodes

and the links connected to these nodes (δ varied from 1% to 99%). The APA method

was applied to each modified network, and information content Iδ(r) was computed for

the recovered network. The information distance Dδ(r) = |I0(r) − Iδ(r)| determined the

success of the recovery in terms of information content. The experiments were repeated

10 times for each deficit level δ, and averaged into Dδ(r).1When recovering a network, the target pool contains all the existing nodes of the original network, i.e

the ‘fragment’ that is used to recover the networks forms the target pool [91]


We begin our analysis with symmetric distributions, γ = 1 and Np = 16. The most

challenging cases involve recovering highly assortative (e.g., perfectly assortative, r = 1)

or highly disassortative (e.g., perfectly disassortative, r = −1) networks. These cases are

more difficult than recovering non-assortative networks (r = 0) because the probabilistic

assigning of intended degrees to target nodes with existing links may deviate from the

intended ej,k, but any such deviation would not harm non-assortative networks. Figure

9.2 plots Dδ(r) for both extreme cases r = 1 and r = −1. It can be observed that, if

the deficit level δ is below a certain threshold δ0, a full recovery of information content is

possible: Dδ(r) = 0 for both r = 1 and r = −1. As the deficit level δ increases, it becomes

harder to recover the transfer, but the distance Dδ(r) grows slower and stabilises after

reaching a certain height. However, at a certain critical level δt, there is a final transition

to the region where the method cannot always follow the intended ej,k and departs from

the corresponding templates. This results in a higher variance of the information distance

when δ > δt (especially visible in Figure 9.2, right, for r = −1, which is less robust than

the case r = 1). Figures 9.3 and 9.4 plot, respectively, average and standard deviation of

Dδ(r) over 10 experiments: the critical levels δt are evident, pinpointing phase transitions

as the deficit surpasses the level δt.

0

0.5

1

1.5

2

0 10 20 30 40 50 60 70 80 90 100

Info

rmat

ion

Dis

tanc

e

Deficit %

0

0.5

1

1.5

2

0 10 20 30 40 50 60 70 80 90 100

Info

rmat

ion

Dis

tanc

e

Deficit %

Figure 9.2: Difficulty of recovery for γ = 1. Left: r = 1 (δ0 ≈ 20%, δt ≈ 95%). Right:r = −1 (δ0 ≈ 10%, δt ≈ 70%).

Figure 9.5 plots Dδ(r) for the non-assortative case r = 0. Interestingly, a full recovery

is possible in this scenario for either very low or very high deficit level δ. The reason for

such symmetry is simple: the low levels δ present no challenge as the missing network

fragments are small, while the high levels δ leave the method a lot of freedom in choosing

the random (non-assortative) connections. For example, if a non-assortative network is


0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

0 10 20 30 40 50 60 70 80 90 100

Ave

rage

of I

nfor

mat

ion

Dis

tanc

e

Deficit %

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

0 10 20 30 40 50 60 70 80 90 100

Ave

rage

of I

nfor

mat

ion

Dis

tanc

e

Deficit %

Figure 9.3: Average of Dδ(r) for γ = 1. Left: r = 1 (δ0 ≈ 20%, δt ≈ 95%). Right: r = −1(δ0 ≈ 10%, δt ≈ 70%).

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0 10 20 30 40 50 60 70 80 90 100

Sta

ndar

d D

evia

tion

of In

form

atio

n D

ista

nce

Deficit %

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0 10 20 30 40 50 60 70 80 90 100

Sta

ndar

d D

evia

tion

of In

form

atio

n D

ista

nce

Deficit %

Figure 9.4: Standard deviation of Dδ(r) for γ = 1. Left: r = 1 (δ0 ≈ 20%, δt ≈ 95%).Right: r = −1 (δ0 ≈ 10%, δt ≈ 70%).

regrown completely anew, it will attain the point-of-reference information transfer. Thus,

there is a maximal difficulty (symptomatic of bell-shaped complexity curves) at the mid-

range of δ. We should also note that the information distance Dδ(r) is overall much smaller

than that of the cases of highly assortative (disassortative) networks, as it is significantly

less difficult to find non-assortative connections. The transition point δt noted in the plots

for extreme r’s can now be explained in the light of the complexity curve. There are two

tendencies contributing to the recovery process: one is trying to reduce the difficulty as δ

approaches 100% (more choice, or freedom, left by the higher deficit in constructing the

desired ej,k), while the other is increasing the difficulty (the ej,k of the existing links in

the target pool diverges more from the required ej,k).

We noted earlier (in chapter 2) that if γ = 1, the resulting excess degree distribution qk

is uniform, hence symmetric. For other values of γ, the resulting qk is not symmetric.


-0.01

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0 10 20 30 40 50 60 70 80 90 100

Info

rmat

ion

Dis

tanc

e

Deficit %

-0.01

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0 10 20 30 40 50 60 70 80 90 100

Info

rmat

ion

Dis

tanc

e

Deficit %

Figure 9.5: Difficulty of recovery for r = 0. Left: γ = 1. Right: γ = 3.

Perfect disassortativity is possible only for symmetric qk, and therefore, for γ > 1, e.g.

γ = 3, it is not possible to get close to the (r = −1) case. Nevertheless, the recovery

behaviour is similar to the one observed in the scenarios for γ = 1.

Figure 9.5, right, shows a familiar bell-shaped complexity curve for non-assortative net-

works, r = 0. Figure 9.6, left, showing r = 1, has an extra feature. In addition to expected

full recovery δ0 threshold for low deficit levels, and transition recovery δt for high deficit

levels, there is a mid-range δm level where the amount of choice available for recovery

completely dominates over the divergence of the existing ej,k from the required ej,k. The

information distance is minimal at δm as the full recovery is attained. Figure 9.6, right,

showing r = rmin ≈ −0.52, is similar to its counterpart from symmetric degree distri-

bution (γ = 1): there are detectable levels of full recovery δ0 and transition recovery δt.

Similar results are observed with γ = 4 (Figure 9.7).

0

0.2

0.4

0.6

0.8

1

1.2

0 10 20 30 40 50 60 70 80 90 100

Info

rmat

ion

Dis

tanc

e

Deficit %

0

0.2

0.4

0.6

0.8

1

1.2

0 10 20 30 40 50 60 70 80 90 100

Info

rmat

ion

Dis

tanc

e

Deficit %

Figure 9.6: Difficulty of recovery for γ = 3. Left: r = 1 (δ0 ≈ 5%, δm ≈ 55%, δt ≈ 95%).Right: r = rmin ≈ −0.52 (δ0 ≈ 22%, δt ≈ 75%).

9.3 Summary 157

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 10 20 30 40 50 60 70 80 90 100

Info

rmat

ion

Dis

tanc

e

Deficit %

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 10 20 30 40 50 60 70 80 90 100

Info

rmat

ion

Dis

tanc

e

Deficit %

Figure 9.7: Difficulty of recovery for γ = 4. Left: r = 1 (δ0 ≈ 5%, δm ≈ 68%, δt ≈ 95%).Right: r = rmin ≈ −0.50 (δ0 ≈ 22%, δt ≈ 75%).

The experiments were also repeated for different distribution lengths Np, and medium

assortativity values r. The latter cases showed intermediate profiles, where Dδ(r) bal-

ances between the two identified tendencies (increasing freedom of choice and increasing

divergence of ej,k) as δ approaches maximum deficit.

9.3 Summary

In this chapter, we applied Assortative Preferential Attachment (APA) method to grow

and/or recover scale-free networks in terms of their information content. APA achieves

a required assortativity value, and hence the information content, for a given degree dis-

tribution and network size. The method covers the extreme cases of perfect assortativity

and perfect disassortativity, where the latter is only achievable if the specified degree

distribution is such that the corresponding excess degree distribution is symmetric.

We identified a number of recovery features: a full-recovery threshold, a phase transi-

tion for assortative and disassortative networks when deficit reaches a critical point, and

a bell-shaped complexity curve for non-assortative networks. Two opposing tendencies

dominating network recovery are detected: the increasing amount of choice in adding as-

sortative/disassortative connections, and the increasing divergence between the existing

and required networks in terms of the ej,k.

The investigation of network robustness, under node removal, random disintegration or

targeted attacks, could be undertaken from a number of perspectives, as we pointed out.

9.3 Summary 158

This chapter only studied it from an information cloning perspective, where the aim was

to recover the information content of the damaged networks. In chapter 10 we will explore

robustness and assortativity from a topological perspective, especially under sustained

targeted attacks, as part of a list of future research topics.

This chapter along with the previous chapter served to illustrate some applications of

assortative mixing. We will present the conclusions of the thesis in the following chapter.

Chapter 10

Conclusions

This thesis presented a comprehensive examination of assortative mixing in complex net-

works. It investigated directed and undirected networks, simulated and real world net-

works, biological, technical, and social networks. The analysis was undertaken at network

level and node level, and a number of algorithms inspired by this analysis, including growth

models and heuristics for network recovery, were presented as well.

This chapter summarises the material presented in this thesis and indicates possible di-

rections for further research, which indeed are many and interesting. Section 10.1 will list

the contributions of this thesis, whereas in section 10.2 directions for future work will be

proposed.

10.1 Summary of contributions

The contributions of, and conclusions reached by, this thesis are summarised below.

10.1.1 Assortativity and Shannon information

It was shown that in scale-free networks, the Shannon information content is correlated

non-linearly to the absolute value of assortativity. Under certain assumptions, this corre-

lation can be expressed as an information power law. We showed that minimalistic and

maximalistic networks (class A and class B) could be defined in terms of the information

10.1 Summary of contributions 160

content, for scale-free networks whose topology is defined by the scale-free exponent and

maximum degree. All the considered real world networks with the same parameters were

shown to have an information content between that of equivalent class A and class B net-

works. We studied the parameter search space of scale-free networks and demonstrated

that there is a slope region and a stability region, and within the slope region there is

higher freedom to optimise for information content, which could be the reason why most

real world networks have their scale-free exponents within this region.

10.1.2 Assortativity in directed networks

We introduced new global assortativity measures for directed networks, namely out-

assortativity and in-assortativity, and demonstrated their relevance. It was shown that

many real world networks which are considered disassortative could in fact be assortative

when out-assortativity and in-assortativity were considered separately. It was also shown

that out-degree mixing patterns consist the highest amount of Shannon information for

the networks studied.

10.1.3 Local assortativity

The most important contribution of this thesis was the introduction of the novel concept

of local assortativity. Local assortativity was defined as a node’s contribution to network

assortativity and mathematically derived for both directed and undirected networks. In

undirected networks, a node’s local assortativity is the scaled difference between the aver-

age excess degree of its neighbours and the network’s overall average excess degree. Local

assortativity distributions can be constructed and used to analyse both simulated and real

world networks. We identified four classes of undirected networks based on these profiles,

namely (i) assortative networks with assortative hubs, (ii) assortative networks with disas-

sortative hubs, (iii) disassortative networks with assortative hubs, and (iv) disassortative

networks with disassortative hubs. The local assortativity profiles provide an additional

quantitative tool for analysis of network topologies. It was shown that the non-assortative

networks are more likely to have a small number of highly assortative hubs, and therefore

are most vulnerable to targeted attacks. In general, local assortativity based rank of nodes

could be used as a guide for choosing nodes in targeted attacks, as opposed to hub based

10.1 Summary of contributions 161

ranking or betweenness centrality based ranking.

We introduced local out-assortativity and local in-assortativity as the corresponding mea-

sures in directed networks. We showed that, particularly in biological networks, these

quantities can be used to discern information about the functionality of nodes. For ex-

ample, the local out-assortativity can be used to identify the regulators which are most

influential in regulatory networks.

10.1.4 Node congruity

The concept of node congruity was proposed in this thesis, which was defined as a node’s

contribution to the scalar assortativity of a network, based on node states. As such, the

scalar assortativity of a network and the congruity distribution are functions of time and

reflect the dynamics of the network. Just as local (node) assortativity distributions provide

an additional tool to understand a network’s topology, node congruity distributions provide

an additional tool to understand a network’s dynamics.

10.1.5 Parallel Addition and Rewiring Growth model

We demonstrated that the existing growth models for Internet AS networks do not ad-

equately capture their local assortativity profiles. We introduced the Parallel Addition

and rewiring growth (PARG) model which does so, while satisfactorily matching (with

the right set of parameters) other topological features of Internet AS networks. It was

shown that the PARG model in general could be used as a growth model to produce

disassortative networks with disassortative hubs.

10.1.6 Assortative Preferential Attachment

The Assortative Preferential Attachment (APA) method was introduced which can grow

a scale-free network with a given level of assortativity.

10.1.7 Applications of assortative mixing

While quantifying assortative mixing can be useful in a number of network design scenarios,

we highlighted a few examples in this thesis. We showed that the task of information

10.2 Directions for future work 162

cloning of networks could utilise knowledge of assortative mixing, and local assortativity

profiles can be used to ‘plan’ targeted attacks against networks. Local assortativity is also

useful to identify node roles in biological networks.

10.2 Directions for future work

The research described in this thesis has introduced a number of new concepts to graph

theory and network science, as well as expanded on some existing concepts. Thus there

is a lot of scope for future research, both in developing the theory further and applying

the measures introduced to new sets of data. Specifically, the following directions can be

pursued.

10.2.1 Local assortativity based sustained attack

Albert and colleagues [17] first considered error and attack tolerance of complex networks.

In their work, they removed nodes from complex networks one by one until all nodes are

extracted, and studied the variation of topological properties in networks due to these

removals. They removed nodes in two separate orders.

1. random order

2. degree order (highest degree first)

They analysed three topological properties:

1. network diameter

2. Size of largest component

3. the average size of the rest of the components

and their conclusion was that while random networks can disintegrate relatively easily

under random attacks, scale-free networks are much more resilient against random node

removal. However, targeted attacks on hubs can cause the scale-free networks disintegrate

(decompose) quickly.


One may consider a number of criteria other than node degrees to select target nodes for

such attacks. These may include betweenness centrality, closeness centrality, node clus-

tering coefficient etc. Since local assortativity is a property of a node, local assortativity

based attack may also be considered. It will be interesting to compare the effectiveness of

attacks based on these quantities for a range of simulated and real world networks.

This line of research would also need to consider suitable metrics that can quantify network

robustness, so that the modes of attack mentioned above can be compared effectively.

Network diameter has the disadvantage that, as soon as the network fragments, it becomes

infinity. Albert and colleagues [17] produced a number of persistent attack profiles for

complex networks, including random networks, world wide web and Internet. However,

they did not attempt to define network robustness as a single quantity for networks. A

number of researchers in later years have attempted to define topological robustness, but

they have all dealt with non-persistent attacks, concentrating on the average effect of

single node removals, rather than continuous node removals. Defining network robustness

under persistent targeted attacks as a single quantity would be an interesting theoretical

aspect of future work related to local assortativity based attacks.

10.2.2 Quantifying the minimum assortativity limit

In chapter 2 we noted that perfect disassortativity is not possible for non-symmetric excess

degree distributions qk, because the ej,k distribution must obey the summation rules. We

denoted the minimum attainable assortativity as rmin, and noted that it can be obtained

for a given qk (or pk) by a suitable minimisation procedure of varying ej,k under constraints.

Since for any scale-free networks, the degree distribution can be uniquely identified by two

parameters (eg: the scale-free exponent γ and the maximum degree Np), it would be

interesting to study the relationship of these parameters to the minimum assortativity of

the degree distribution represented by them. That is, the variation of rmin in the Np × γ

surface could be investigated.

10.2.3 Classification of directed networks

In chapter 5 we classified undirected complex networks based on their local assortativity

profiles. In chapter 6, we pointed out that a similar classification could be undertaken for


directed networks. Since directed networks could be classified based on in-assortativity

as well as out-assortativity, sixteen (four times four) such classes are possible in theory.

However, some of these classes may have no real world examples, and their non existence

could be interpreted in terms of design requirements of networks in each domain (for

example, biological networks). This is another direction of future research.

10.2.4 Evolution of assortativity and local assortativity in networks

Attempts have been made to study the evolutionary tendencies of assortativity in complex

networks. This investigation is hampered by the fact that topological data throughout the

evolutionary process is not available for many real world networks, particularly biological

networks. Technological networks, on the other hand, have taken much shorter time

to evolve (often years or decades compared to the millions of years taken by biological

networks), and often the topological data is available from the beginning of their evolution

to the present time. As we have mentioned in chapter 8, Internet AS network is a good

example where evolutionary history of topology is available. The evolution of assortativity

in such networks would be interesting to study.

In the case of biological networks, simulated systems whose biological validity is guaranteed

to some extent by extensive research could be considered. We present an example of such

a study in Appendix B, where we investigate the evolution of assortativity in the neural

networks of a set of agents in an artificial life computational ecology, named Polyworld

[66].

The evolution of local assortativity and its distribution also could be studied. It was sug-

gested in [92, 94] that networks that belong to various classes based on local assortativity

could be in different phases of evolution. In other words, the transition between classes

might be a good indicator of growth history. Therefore, studying the evolution of local

assortativity profiles could lead to better understanding of the evolution phases of the

network under consideration.

10.2.5 Local assortativity and rich club phenomena

We made some comments about contrasting local assortativity with the rich club connec-

tivity in chapter 5. Rich club connectivity is defined as an average connectivity of nodes

10.3 Epilogue 165

that have more than a specified number of degrees [125]. While the latter is computed

over sub graphs (rich clubs), the former is a measure of a specific node. In particular, it

is possible to measure local assortativity for any peripheral node, but it is not possible to

compute rich club connectivity for peripheral nodes alone, as they would have to belong to

a rich club. However, as we show in appendix C, one may consider a cumulative average

local assortativity, Rk, by aggregating ρk for all degrees higher than k. Contrasting Rk

with the corresponding rich club connectivity reveals that these quantities are correlated,

but the correlation is non-linear. A more detailed comparison between these measures is

a subject of future research.

10.2.6 The investigation of more real world networks

While we have considered a significant range of networks in this thesis, the analysis of local

assortativity could be extended to a vast number of networks that we have not considered.

These may include transportation networks [50], networks of friends in social network

websites such as Facebook and Tweeter [12], the world wide web [16], and biological

networks of the organisms that we have not considered, but to name a few. Each of

these networks have their own evolutionary dynamics and could have interesting local

assortativity profiles as a result.

It should be noted that the research directions proposed above only serve as examples

of potential avenues for future work. The concept of assortative mixing is applicable in

a number of domains, and in each domain a vast number of research problems could be

investigated.

10.3 Epilogue

This thesis investigated assortative mixing in complex networks, by analysing mixing

patterns in global (network) and local (node) level, both for directed and undirected

networks. The thesis also analysed the Shannon information content of networks in terms

of assortativity, and presented a number of algorithms and heuristics which could be

used to grow networks with specific assortativity related properties. The most important

contribution of the thesis was the concept of local assortativity (and the related concept of

10.3 Epilogue 166

node congruity) and the utility of this concept was amply demonstrated for the analysis of

a number of real world networks. The thesis presents a comprehensive body of knowledge

about assortative mixing, and it is hoped that this knowledge will be drawn upon and

expanded by the complex networks community.

Appendix A

Data sources and software

In this appendix we list and briefly comment about the data sources that were used in

constructing real world networks. We also present a list of freely available software tools

that we have used in our analysis (apart from the software developed during the course of

this project).

A.1 Data sources

Following is a list of data sources from which the real world networks analysed in this

thesis were constructed or directly downloaded.

1. CCNR data: The metabolic networks investigated in chapter 3 were downloaded

from the Centre for Complex Network Research, University of Notre Dame website

[2]. It should be noted that these metabolic networks include the so-called currency

metabolites [20, 54, 70, 105, 115]. The current consensus in the metabolic net-

work research community is that the currency metabolites should be removed before

the topology, and particularly the appearance of motifs, in metabolic networks is

analysed [19, 54, 70, 97]. This has reduced the biological relevance of the CCNR

metabolic networks. However, the networks in the CCNR data are still real world ex-

amples of complex networks, and as such could be used as examples in investigating

information content, as done in chapter 3.

A.2 Software tools 168

2. KEGG data: The Kyoto Encyclopaedia of Genes and Genomes (KEGG) contains,

among others, a regularly updated database of metabolic pathways of more than a

thousand organisms [7]. The metabolic networks analysed in this thesis (other than

those in chapter 3) were constructed from the KEGG database. The networks were

constructed using NeAT (Network Analysis Tools from Universite Libre de Bruxelles,

Belgium) [8]. The currency metabolites were removed.

3. CAIDA data: The Cooperative Association for Internet Data Analysis (CAIDA)

maintains a regularly updated database which contains information about topology,

traffic, routing, security and performance of Internet. We used topological data at

the Autonomous Systems Level from this database [5].

4. MIMI data: The Gene Regulatory networks were downloaded from the Michigan

Molecular Interaction Database, University of Michigan [4].

5. DIP data: The Protein-Protein Interaction networks were downloaded from Database

of Interacting Proteins, University of California, Los Angeles [3].

6. Cortical Network data: The cortical networks were constructed from a number of

sources. The primary source was the Collations of connectivity data on the Macaque

brain website [6]. Supplementary data from the Brain Connectivity toolbox [109]

and the Sums database of the Van Essen lab [10] was also used.

7. Corynebacteria Data: The CoryneRegNet 4.0 - A reference database for corynebac-

terial gene regulatory networks was used to construct the transcription networks

of Coryne bacteria [25]. These networks were directly downloaded into Cytoscape

using the Cytoscape plug-in available [26]

We have commented above on the primary data sources and/or those which needed

some curing which merited explanation. A number of other sources were also utilised

to download networks, and these have been cited appropriately throughout the thesis.

A.2 Software tools

1. Cytoscape: Cytoscape is an open source bioinformatics software platform for visu-

alising molecular interaction networks and integrating these interactions with gene

A.2 Software tools 169

expression profiles and other state data [11, 107]. We used Cytoscape primarily

to visualise complex networks that were under investigation. All figures visualising

complex networks in this thesis were produced using Cytoscape (unless otherwise

stated). Cytoscape was also used sometimes as a complementary tool of analysis

(e.g. to double check computation of network properties, such as degree distribu-

tions). It should be emphasised however that all values of network properties were

primarily computed using software developed during this project.

Several independent researchers have developed plug-ins for Cytoscape to enhance

its capacity, and we utilized some of these: namely, the Network analyser plug-in [9]

and the Coryne network database plug-in [26]

2. Pajek: Pajek (Slovene word for Spider) is a program, for Windows, for analysis and

visualisation of large networks. It is freely available, for noncommercial use [13]. We

used Pajek as a complementary visualisation tool, especially for large networks and

for three dimensional visualisation of networks.

3. The Brain Connectivity Toolbox: The brain connectivity toolbox provides access

to a large selection of complex network measures in Matlab. Such measures aim to

characterise brain connectivity by neurobiologically meaningful statistics, and are

increasingly used in the description of structural and functional connectivity data

sets [109]. The brain connectivity toolbox was used in this project to double check

the computation of assortativity.

Note that network analysis in this project was undertaken primarily with in-house

developed Java software. The code base will be made available as open source

software in the near future.

Appendix B

Evolution of assortativity in

neural networks

B.1 Introduction

The nature of evolutionary trends in complex networks has been subject to much debate

[27, 49, 74]. In this appendix, our interest lies in the manner in which the topology of neural

networks adapt under evolutionary pressure. Specifically, we investigate the evolution of

assortativity of neural networks of agents in the Polyworld artificial life system [120, 121].

We examine both the actual structure of these networks, and their logical structure.

The logical structure of the neural networks is explored by inferring functional networks

[47, 53] from statistical dependencies between the time series of each node in the underlying

structural network. Here, we use mutual information [72] and transfer entropy [104] to

measure the statistical dependencies between the neurons. We then examine the trends

in assortativity of the topologies of the structural and functional networks with respect to

evolutionary time. We also examine the trends in a a few other topological measures for

comparison.

We find several interesting trends in the topologies, with the trends in the structural and

transfer entropy-based functional networks being most similar. These networks become

more non-assortative, more clustered, and adopt shorter average path lengths with evolu-

tionary time. These trends are significant in that they imply the networks are taking on

B.2 Polyworld 171

a more “small-world” [117] character over evolutionary time.

We begin by providing background on the Polyworld artificial life system [122], and the

manner in which simulations are run here. We then describe how functional networks are

inferred using the mutual information and transfer entropy measures. Subsequently, we

present and discuss the trends identified in assortativity with evolutionary time, as well

as compare them with trends in clustering and average path lengths. Finally we present

the conclusions that could be inferred from these trends.

B.2 Polyworld

Polyworld [122] is a computational ecology evolving populations of haploid agents, each

using a suite of primitive behaviours (move, turn, eat, mate, attack, light, focus) un-

der continuous control of an Artificial Neural Network (ANN) employing summing and

squashing neurons with synapses that adapt via Hebbian learning. The wiring diagram

of the ANN is encoded in the organism’s genome, via a statistical description of the num-

ber of neural groups of excitatory and inhibitory neurons, synaptic connection densities,

ordered-ness of connections, and learning rates. Input to the ANN consists of pixels from

a rendering of the scene from each agent’s point of view, like light falling on a retina.

The agent morphologies are simple and fixed, but agents’ interactions with the world and

each other are fairly complex, as they replenish energy by seeking out and consuming

food or by killing and eating other agents. They reproduce when two collocated agents

simultaneously express their mating behaviours, using a number of crossover points and a

mutation rate that are also contained in the parental genomes [122].

The simulation is initially seeded with a uniform population of agents that have the min-

imum number of neural groups and a nearly minimal number of neurons and synapses.

While predisposed to some potentially beneficial behaviours, such as running towards food

(green) and away from aggression (red; see [122] for details on colour use in Polyworld),

these seed organisms are not a viable species. Without evolution they cannot sustain their

numbers through their reproductive behaviours and will inevitably die out.

As simulations progress both the structural architecture of the ANNs and the activation

of every neuron at every time step are recorded for every agent. Here we use these

B.3 Inferring Functional Networks 172

neural activation recordings to determine functional networks for each agent and compare

functional network assortativity to the underlying structural network assortativity.

B.3 Inferring Functional Networks

Two remote neural nodes are defined to be functionally connected where they exhibit

statistical dependence in time [47, 53]. The nodes considered could be voxels in BOLD

recordings (e.g. [53]), or neurons in an artificial neural network (as are used here). A

functional network is then formed from a set of functional connections. Inferring functional

networks from time-series of node states therefore involves two distinct steps: (i) making

some measure of the statistical dependence or closeness between each node pair, then (ii)

deciding whether each closeness value should constitute a link between the node pair. The

closeness measure and the inferred links can be either directional or undirectional.

Functional networks may be used to infer the underlying structural network where this is

unknown. More importantly, functional networks provide insight into the logical structure

of the network and how this changes as a function of network activity (regardless of whether

the underlying structure is known).

In the work presented in this appendix, we use information-theoretical measures [72] for

the closeness of each pair X and Y . The mutual information between X and Y has been

introduced earlier (in chapter 2). This is a symmetric measure of the common information

between X and Y . Though it has been used in literature to measure directed information

transfer from one variable to another, this is not valid: it is a symmetric measure of

statically shared information (which is useful in its own right).

Alternatively, the transfer entropy [104] is a directed measure of dynamic information

transfer from one variable to another. It quantifies the information provided by a source

node about a destination’s next state that was not contained in the past of the destina-

tion. Specifically, the transfer entropy from a source node Y to a destination X is the

mutual information between the previous state of the source yn and the next state of the

destination xn+1, conditioned on the past k states of the destination x(k)n :

TY→X(k) =∑

xn+1,x(k)n ,yn

p(xn+1, x(k)n , yn) log2

p(xn+1|x(k)n , yn)

p(xn+1|x(k)n )

. (B.1)

B.4 Results and Discussion 173

The transfer entropy may be measured for any two time series X and Y and is always a

valid measure of the predictive gain from the source, but only represents physical infor-

mation transfer when measured on a causal link [67].

Here, we compute functional networks for each agent from the Polyworld simulation using

both mutual information and transfer entropy as separate measures of closeness. The

continuous activation levels are first discretised in four levels, and a history length k = 1

is used for the transfer entropy (this renders it more towards an inference of causal effect

than information transfer [67, 69]).

Several options are then available for deciding whether each pair of areas should be con-

sidered functionally connected based on their closeness. One could assign links to a given

number or percentage of pairs based on the largest closeness values, or could use an ap-

proach based on the statistical significance of the closeness measure, e.g. [28]. Here, the

number of functional links was designed to match the proportion of links in the underlying

structural network, and the largest such closeness values were assigned links. A (directed)

link exists in the structural network between two neurons where the source neuron is

an input to the target neuron. We consider both processing and input neurons in the

functional network.

B.4 Results and Discussion

We constructed the functional networks for each agent, and evaluated the assortativity

of each of these and the underlying structural networks (which had between 13 and 159

neurons, and 52 on average). We then averaged the assortativity over sets of 100 sequential

agents ordered by birth. The results are plotted with respect to evolutionary time in

Figure B.1. Clearly, in all cases the assortativity reaches a relatively steady state within

5000 – 12000 steps in evolutionary time. This aligns with previous studies of trends in

the complexity of the neural networks in Polyworld [120] where the complexity is driven

upwards over the initial 5000 or so steps of evolution before the agents find a “good

enough” solution. At this point the drive for evolutionary change somewhat stagnates, as

is reflected in the steady state of the measures here.

In general, the transfer entropy-inferred functional networks show a similar trend to the

structural networks. Interestingly, the transfer entropy-inferred functional networks had a


-0.80

-0.60

-0.40

-0.20

0.00

0.20

0.40

0 2000 4000 6000 8000 10000 12000

Ass

orta

tivity

Evolutionary time

Figure B.1: Assortativity trends in structural and functional networks versus evolutionarytime. Assortativity is plotted for structural networks (red line), mutual information-inferred functional networks (violet ×), and transfer entropy-inferred functional networks(blue ¤). Error bars indicate the standard error of the mean.

slightly smaller overlap (mean 17.6± 0.1%) with the underlying structural networks than

the mutual information-inferred functional networks (mean 19.1±0.1%). It is possible that

the transfer entropy performs better at inferring the general interaction structure between

modules or regions in the structural network (thereby capturing the general topological

trends) without necessarily inferring the precise links any better.

As shown in Figure B.1, the structural networks tend to exhibit negative assortativity:

this is not surprising as it is a known general characteristic of biological networks evolved

under external pressure [108]. This is because negative assortativity supports connectiv-

ity between diverse elements in the network, an important feature for producing complex

behaviour. Unsurprisingly also, the mutual information-inferred networks exhibit pos-

itive assortativity (since mutual information is maximised for similar elements), while

the transfer entropy-inferred networks exhibit negative assortativity (since transfer en-

tropy is minimised for similar elements). More interestingly, the structural and transfer


0.00

0.20

0.40

0.60

0.80

1.00

1.20

0 2000 4000 6000 8000 10000 12000

Clu

ster

ing

Coe

ffic

ient

Evolutionary time

Figure B.2: Clustering trends in structural and functional networks versus evolution-ary time. Clustering coefficient is plotted for structural networks (red line), mutualinformation-inferred functional networks (violet ×), and transfer entropy-inferred func-tional networks (blue ¤). Error bars indicate the standard error of the mean.

entropy-inferred networks become more neutrally assortative over time (i.e. less negatively

assortative). While this may seem surprising, it is possibly an artifact of the elements in

the network becoming more closely coupled as they evolve and therefore become more

similar, or perhaps reflects the increased clustering occurring over evolutionary time.

To verify this, we also considered the clustering coefficient and closeness of these networks

in a similar manner. The clustering coefficient of a node characterises the density

of links in the environment closest to a vertex. Formally, the clustering coefficient C

of a node is the ratio between the total number y of links connecting its neighbours

and the total number of all possible links between all these z nearest neighbours [41]:

C = 2y/ (z (z − 1)).

C =2y

z (z − 1)(B.2)

The clustering coefficient C for a network is the average C over all nodes. Closeness

centrality of a node v is defined as the mean geodesic distance (shortest path length)


1.00

1.50

2.00

2.50

3.00

3.50

4.00

0 2000 4000 6000 8000 10000 12000

Clo

sene

ss c

entr

ality

Evolutionary time

Figure B.3: Assortativity trends in structural and functional networks versus evolutionarytime. Closeness centrality is plotted for structural networks (red line), mutual information-inferred functional networks (violet ×), and transfer entropy-inferred functional networks(blue ¤). Error bars indicate the standard error of the mean.

between the node and all other nodes in the network [126]. Closeness centrality is formally

defined as CC (v) =∑

dG(v, t) where v 6= t and dG(v, t) is the shortest path distance

between nodes v and t.

Our results for these measures are shown in Figures B.2 and B.3. The figures show that the

structural and transfer entropy-inferred networks get more clustered as they evolve. The

mutual information-inferred networks however exhibit a decrease in clustering coefficient.

Finally, B.3 shows that the closeness centrality is reduced with evolutionary time for all

networks. Given the previous results, this is unsurprising as all imply diversification of

connectivity across the network with evolutionary time. In fact, taken together these

results suggest that the networks are becoming more small-world [117] with evolutionary

time. Importantly though, recall that all measures reach a steady state here: the neural

networks do not continually improve on these desirable features, but stop developing once

a good enough solution is found.

B.5 Conclusion 177

B.5 Conclusion

We used the Polyworld artificial life system to study evolution of assortativity in neural

networks. We constructed functional networks and analysed the underlying structural

networks. Our investigation revealed clear trends in assortativity with evolutionary time.

Namely, the structural networks, as well as functional networks inferred with transfer

entropy, became more non-assortative with evolution. The structure and activity in the

networks became more integrated over time, as may be expected in the evolution of com-

plex distributed processes. In particular, considered together with other topological mea-

sures, it was evident that both the structural and functional networks take on more of a

small-world character as the evolution progresses.

Our results also showed interesting differences between the use of mutual information and

transfer entropy in inferring functional networks. The transfer entropy-inferred functional

networks showed trends in assortativity more similar to those of the underlying structural

networks, and also provided more intuitive insights into network activity.

Appendix C

Rich club phenomenon and local

assortativity

We would like to contrast the measure of local assortativity with the measure of rich-club

connectivity[39, 126].

A rich-club is defined in terms of degree-based rank r of nodes, and the rich-club con-

nectivity ϕ(r). The degree-based rank denotes the rank of a given node when all nodes

are ordered in terms of their degrees, highest first. This is then normalised by the total

number of nodes. The rich-club connectivity is defined as the ratio of actual number of

links over the maximum possible number of links between nodes with rank less than r.

Thus, it is possible to calculate the rich-club connectivity distribution of a network, ϕ(r)

over r.

While the rich-club connectivity is computed over sub graphs (rich clubs), local assortativ-

ity is a measure of a specific node. In particular, it is possible to measure local assortativity

ρv for any peripheral node, but it is not possible to compute ϕ(r) for peripheral nodes

alone, as they would have to belong to a rich club. One may consider a cumulative average

local assortativity, R(k), by aggregating average local assortativity for a given degree (see

chapter 5) ρk for all degrees higher than k. Contrasting R(k) with the corresponding ϕ(r)

reveals that these quantities are correlated, but the correlation is non linear. Specifically,

the cumulative average local assortativity R(k) has some correlation with the rich club

connectivity coefficient, so that the rich club connectivity resembles the integral of average

Rich club phenomenon and local assortativity 179

local assortativity on hubs.

To contrast these measures we studied the relationship between them in more detail, using

the Autonomous System Level topology of Internet (1998) as an example.

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

Ric

h C

lub

Co

effi

cien

t

rank degree r

Figure C.1: The rich club coefficient ϕ(r) vs ranked (percentage) degree in Internet ASlevel 1998 topology.

Figure C.1 shows the rich club connectivity of this network vs the percentage rank of

degrees for the Internet AS network. We plotted the cumulative local assortativity, as

shown in Fig C.2, starting from the nodes with the highest degree (the ‘smallest’ rank)

and accumulating the local assortativity values as we go towards lower degrees.

It is evident from Fig C.1 and Fig C.2 that these two measures are correlated. However,

cumulative local assortativity goes through a sharper transition, compared to the rich club

connectivity coefficient. To illustrate this we have plotted the cumulative average local

assortativity vs rich club connectivity coefficient (by coupling points of the same rank

degree together) in Figure C.3, which confirms that there is no linear relationship between

these two quantities.

In summary, the following differences between these measures can be observed:

• In contrast to the rich club coefficient measure, we do not need to consider a sub-

graph to calculate the local assortativity of a node. Local assortativity is a quantity


-0.05

-0.045

-0.04

-0.035

-0.03

-0.025

-0.02

-0.015

-0.01

-0.005

0 10 20 30 40 50 60 70 80 90 100

Cu

mu

lati

ve

avg

Lo

cal

asso

rtat

iven

ess

rank degree r

Figure C.2: The cumulative average local assortativity R(k) vs ranked (percentage) degreein Internet AS level 1998 topology.

-0.05

-0.045

-0.04

-0.035

-0.03

-0.025

-0.02

-0.015

-0.01

-0.005

0 10 20 30 40 50 60 70 80 90 100

Cu

mu

lati

ve

avg

Lo

cal

asso

rtat

iven

ess

Rich Club Coefficient

Figure C.3: The cumulative average local assortativity vs the rich club coefficient inInternet AS level 1998 topology.


we define for each node in the topology, while rich club phenomena necessitate con-

sidering a rich club subgraph.

• By extension, the local assortativity measure can be used against any kind of nodes,

including peripheral nodes; where as using the rich club phenomena peripheral nodes

can only be studied as part of an extended rich club.

Thus these two quantities remain related yet independently relevant measures of mixing

patterns in complex networks. A more detailed qualification of their relationship is subject

to future research.

Bibliography

[1] (2007) Pajek datasets. [Online]. Available:http://vlado.fmf.uni-lj.si/pub/networks/data

[2] (2008) Centre for Complex Network Research, University of Notre Dame. [Online].Available: http://www.nd.edu/networks/index.htm

[3] (2008) Database of Interacting Proteins, University of California, Los Angeles.[Online]. Available: http://dip.doe-mbi.ucla.edu/dip/Download.cgi

[4] (2008) Michigan Molecular Interaction Database, University of Michigan. [Online].Available: http://mimi.ncibi.org/MimiWeb/main-page.jsp

[5] (2008) The Cooperative Association for Internet Data Analysis. [Online]. Available:http://www.caida.org/home

[6] (2009) Collations of connectivity data on the Macaque brain. [Online]. Available:www.cocomac.org/

[7] (2009) The Kyoto Encyclopaedia for Genes and Genomes. [Online]. Available:http://www.genome.jp/kegg

[8] (2009) NeAT Network Analysis Tools, Universite Libre de Bruxelles. [Online].Available: http://rsat.ulb.ac.be/rsat/index neat.html

[9] (2009) Network Analyzer: Java plug-in for Cytoscape. [Online]. Available:http://med.bioinf.mpi-inf.mpg.de/netanalyzer/index.php

[10] (2009) Sums database, Van Essen lab. [Online]. Available:http://brainvis.wustl.edu/wiki/index.php/Sums:About

[11] (2010) Cytpscape. [Online]. Available: http://www.cytoscape.org/

[12] (2010) International Network for Social Network Analysis. [Online]. Available:http://www.insna.org/software/index.html

[13] (2010) Pajek. [Online]. Available: http://pajek.imfm.si/doku.php

[14] R. Albert and A.-L. Barabasi, “Emergence of scaling in random networks,” Science,vol. 286, pp. 509–512, 1999.

BIBLIOGRAPHY 183

[15] ——, “Statistical mechanics of complex networks,” Reviews of Modern Physics,vol. 74, pp. 47–97, 2002.

[16] R. Albert, H. Jeong, and A.-L. Barabasi, “Diameter of the world-wide web,” Nature,vol. 401, pp. 130–131, 1999.

[17] R. Albert, H. Jeong, and A.-L. Barabasi, “Error and attack tolerance of complexnetworks,” Nature, vol. 406, pp. 378–382, 2000.

[18] M. Aldana, “Boolean dynamics of networks with scale-free topology,” Physica D,vol. 185, no. 1, pp. 45–66, 2003.

[19] U. Alon, Introduction to Systems Biology: Design Principles of Biological Circuits.London: Chapman and Hall, 2007.

[20] M. Arita, “The metabolic world of escherichia coli is not small.” Proceedings ofthe National Academy of Sciences, vol. 101, no. 6, pp. 1543–1547, February 2004.[Online]. Available: http://dx.doi.org/10.1073/pnas.0306458101

[21] G. Bagler and S. Sinha, “Assortative mixing in protein contact networks and proteinfolding kinetics,” Bioinformatics, vol. 23, no. 14, pp. 1760–1767, 2007.

[22] A.-L. Barabasi, “Scale-free networks: A decade and beyond,” Science, vol. 325, no.5939, pp. 412–413, 2009.

[23] A.-L. Barabasi, R. Albert, and H. Jeong, “Scale-free characteristics of random net-works: The topology of the world-wide web,” Physica A, vol. 281, pp. 69–77, 2000.

[24] A.-L. Barabasi and E. Bonabeau, “Scale-free networks,” Scientific American, vol.288, pp. 50–59, 2003.

[25] J. Baumbach, “Coryneregnet 4.0 - a reference database for corynebacterial generegulatory networks,” BMC Bioinformatics, vol. 8, 2007.

[26] J. Baumbach and L. Apeltsin, “Linking cytoscape and the corynebacterial referencedatabase coryneregnet,” BMC Genomics, vol. 9, p. 184, April 2008.

[27] M. A. Bedau, “The evolution of complexity,” in Mapping the Future of Biology, ser.Boston Studies In The Philosophy Of Science, A. Barberousse, M. Morange, andT. Pradeu, Eds. Netherlands: Springer, 2009, vol. 266, pp. 111–130.

[28] L. M. A. Bettencourt, G. J. Stephens, M. I. Ham, and G. W. Gross, “Functionalstructure of cortical neuronal networks grown in vitro,” Physical Review E, vol. 75,no. 2, p. 021915, 2007.

[29] G. Bianconi, “The entropy of randomized network ensembles,” Euro-physics Letters, vol. 81, no. 2, p. 28005, 2008. [Online]. Available:http://dx.doi.org/10.1209/0295-5075/81/28005

[30] V. D. Blondel, A. Gajardo, M. Heymans, P. Senellart, and P. Van Dooren, “Ameasure of similarity between graph vertices: Applications to synonym extractionand web searching,” SIAM Review, vol. 46, no. 4, pp. 647–666, 2004.

BIBLIOGRAPHY 184

[31] J. Boedecker, O. Obst, N. M. Mayer, and M. Asada, “Initialization and self-organizedoptimization of recurrent neural network connectivity,” HFSP Journal, vol. 3, no. 5,pp. 340–349, 2009.

[32] M. Boguna, R. Pastor-Satorras, and A. Vespignani, “Cut-offs and finite sizeeffects in scale-free networks,” The European Physical Journal B - CondensedMatter, vol. 38, no. 2, pp. 205–209, March 2004. [Online]. Available:http://dx.doi.org/10.1140/epjb/e2004-00038-8

[33] T. Bu and D. Towsli, “On distinguishing between internet power law topology gen-erators,” Proceedings of INFOCOM, 2002.

[34] D. S. Callaway, J. E. Hopcroft, J. M. Kleinberg, M. E. J. Newman, and S. H.Strogatz, “Are randomly grown graphs really random?” Physical Review E, vol. 64,no. 4 Pt 1, October 2001.

[35] A. Cangelosi and D. Parisi, “A neural network model of caenorhab-ditis elegans: The circuit of touch sensitivity.” Neural Pro-cessing Letters, vol. 6, pp. 91–98, 1997. [Online]. Available:http://www.wormbase.org/db/misc/paper?name=WBPaper00002929

[36] A. Cavagna, A. Cimarelli, I. Giardina, G. Parisi, R. Santagati, F. Stefanini, andM. Viale, “Scale-free correlations in bird flocks,” 2009, arXiv:0911.4393. [Online].Available: http://arxiv.org/abs/0911.4393

[37] S. Cho, S. G. Park, D. O. H. Lee, and B. C. Park, “Protein-protein interactionnetworks: from interactions to networks.” Journal of Biochemistry and MolecularBiology, vol. 37, no. 1, pp. 45–52, January 2004.

[38] K. K. S. Chung, L. Hossain, and J. Davis, “Exploring sociocentric and egocentricapproaches for social network analysis,” in KMAP 2005: Kowledge Management inAsia Pacific, Wellington, New Zealand, 2005.

[39] V. Colizza, A. Flammini, M. A. Serrano, and A. Vespignani, “Detecting rich-clubordering in complex networks,” Nature Physics, vol. 2, pp. 110–115, 2006.

[40] P. Crucittia, V. Latora, M. Marchiori, and A. Rapisarda, “Error and attack toleranceof complex networks,” Physica A, vol. 340, p. 388394, 2004.

[41] S. N. Dorogovtsev and J. F. F. Mendes, Evolution of Networks: From BiologicalNets to the Internet and WWW. Oxford: Oxford University Press, January 2003.

[42] M. Faloutsos, P. Faloutsos, and C. Faloutsos, “On power-law relationships of theinternet topology,” Computer Communications Review, vol. 29, pp. 251–262, 1999.

[43] P. Fernandez and R. V. Sole, “The role of computation in complex regulatory net-works,” in Scale-free Networks and Genome Biology, E. V. Koonin, Y. I. Wolf, andG. P. Karev, Eds. Georgetown, TX: Landes Bioscience, 2006, pp. 206–225.

[44] ——, “Neutral fitness landscapes in signalling networks,” Journal of The RoyalSociety Interface, vol. 4, no. 12, pp. 41–47, 2007.

BIBLIOGRAPHY 185

[45] T. C. Ferree and S. R. Lockery, “Mathematical analysis of neu-ral networks for chemotaxis in c. elegans,” 1997. [Online]. Available:http://www.wormbase.org/db/misc/paper?name=WBPaper00022274

[46] T. C. Freeman, L. Goldovsky, M. Brosch, S. van Dongen, P. Maziare, R. J. Gro-cock, S. Freilich, J. Thornton, and A. J. Enright, “Construction, visualisation, andclustering of transcription networks from microarray expression data,” PLoS Com-putational Biology, vol. 3, no. 10, p. e206, 10 2007.

[47] K. J. Friston, “Functional and effective connectivity in neuroimaging: A synthesis,”Human Brain Mapping, vol. 2, pp. 56–78, 1994.

[48] N. Geard, “Modelling gene regulatory networks: Systems biology to complex sys-tems,” University of Queensland Technical report, 2004.

[49] S. J. Gould, “The evolution of life on earth,” Scientific American, vol. 271, no. 4,pp. 62–69, 1994.

[50] R. Guimera, M. Sales-Pardo, and L. A. Amaral, “Classes of complex networksdefined by role-to-role connectivity profiles,” Nature Physics, vol. 3, no. 1, pp.63–69, January 2007. [Online]. Available: http://dx.doi.org/10.1038/nphys489

[51] Q. Guo, T. Zhou, J.-G. Liu, W.-J. Bai, B.-H. Wang, and M. Zhao, “Growingscale-free small-world networks with tunable assortative coefficient,” Physica A:Statistical and Theoretical Physics, vol. 371, no. 2, pp. 814–822, November 2006.[Online]. Available: http://dx.doi.org/10.1016/j.physa.2006.03.055

[52] V. Hatzimanikatis, C. Li, J. A. Ionita, and L. J. Broadbelt, “Metabolic networks:enzyme function and metabolite structure,” Current Opinion in Structural Biology,vol. 14, no. 3, pp. 300–306, June 2004.

[53] C. J. Honey, R. Kotter, M. Breakspear, and O. Sporns, “Network structure of cere-bral cortex shapes functional connectivity on multiple time scales,” Proceedings ofthe National Academy of Sciences, vol. 104, no. 24, pp. 10 240–10 245, 2007.

[54] M. Huss and P. Holme, “Currency and commodity metabolites: theiridentification and relation to the modularity of metabolic networks.” IET SystemsBiology, vol. 1, no. 5, pp. 280–285, September 2007. [Online]. Available:http://view.ncbi.nlm.nih.gov/pubmed/17907676

[55] H. Jeong, B. Tombor, R. Albert, Z. N. Oltvai, and A. L. Barabaai, “The large-scaleorganization of metabolic networks,” Nature, vol. 407, p. 651, 2000.

[56] L. N. Joppa, J. Bascompte, J. M. Montoya, R. V. Sole, J. Sanderson, and S. L.Pimm, “Reciprocal specialization in ecological networks,” Ecology Letters, vol. 12,no. 9, pp. 961–969, 2009.

[57] B. H. Junker and F. Schreiber, Analysis of Biological Networks (Wiley Series inBioinformatics). Wiley-Interscience, March.

BIBLIOGRAPHY 186

[58] A. Kaiser and T. Schreiber, “Information transfer in continuous processes,” PhysicaD, vol. 166, no. 1-2, pp. 43–62, 2002.

[59] F. Kepes (Ed), Biological Networks. Singapore: World Scientific, 2007.

[60] S. Knock, A. McIntosh, O. Sporns, R. Ktter, P. Hagmann, and V. Jirsa, “The effectsof physiologically plausible connectivity structure on local and global dynamics inlarge scale brain models,” Journal of Neuroscience Methods, vol. 183, no. 1, pp. 86– 94, 2009.

[61] A. Kraskov, H. Stogbauer, and P. Grassberger, “Estimating mutual information,”Physical Review E, vol. 69, no. 6, p. 066138, 2004.

[62] S. Kullback and R. A. Leibler, “On information and sufficiency,” Annals of Mathe-matical Statistics, vol. 22, pp. 49–86, 1951.

[63] V. Latora and M. Marchiori, “Efficient behavior of small-world networks,” PhysicalReview Letters, vol. 87, no. 19, p. 198701, 2001.

[64] E. A. Leicht, G. Clarkson, K. Shedden, and M. E. J. Newman, “Large-scalestructure of time evolving citation networks,” European Physical Journal B, vol. 59,pp. 75–83, May 2007. [Online]. Available: http://arxiv.org/abs/0706.0015

[65] E. A. Leicht, P. Holme, and M. E. J. Newman, “Vertex similarity in networks,”Physical Review E, vol. 73, p. 026120, October 2006.

[66] J. T. Lizier, M. Piraveenan, D. Pradhana, M. Prokopenko, and L. S. Yaeger, “Func-tional and structural topologies in evolved neural networks,” in Advances in ArtificialLife: Tenth European Conference on Artificial Life (ECAL ’09), ser. LNCS/LNAI.Springer, 2009, vol. 5777-5778.

[67] J. T. Lizier and M. Prokopenko, “Differentiating information transfer and causaleffect,” European Physical Journal B, vol. 73, no. 4, pp. 605–615, 2010.

[68] J. T. Lizier, M. Prokopenko, and A. Y. Zomaya, “The information dynamics ofphase transitions in random boolean networks,” in Artificial Life XI - Proceedingsof the Eleventh International Conference on the Simulation and Synthesis of LivingSystems, S. Bullock, J. Noble, R. Watson, and M. A. Bedau, Eds. MIT Press, 2008,pp. 374–381.

[69] ——, “Local information transfer as a spatiotemporal filter for complex systems,”Physical Review E, vol. 77, no. 2, p. 026110, 2008.

[70] H. Ma and A. P. Zeng, “Reconstruction of metabolic networks fromgenome data and analysis of their global structure for various organisms.”Bioinformatics, vol. 19, no. 2, pp. 270–277, January 2003. [Online]. Available:http://bioinformatics.oxfordjournals.org/cgi/content/abstract/19/2/270

[71] A. Ma’ayan, S. Jenkins, S. Neves, A. Hasseldine, E. Grace, B. Dubin-Thaler,N. Eungdamrong, G. Weng, P. Ram, J. J. Rice, A. Kershenbaum, G. Stolovitzky,R. Blitzer, and R. Iyengar, “Formation of regulatory patterns during signal propa-gation in a mammalian cellular network,” Science, vol. 309, no. 1078, 2005.

BIBLIOGRAPHY 187

[72] D. J. MacKay, Information Theory, Inference, and Learning Algorithms. Cam-bridge: Cambridge University Press, 2003.

[73] S. Maslov and K. Sneppen, “Specificity and stability in topology of protein net-works,” Science, vol. 296, no. 5569, pp. 910–913, May 2002.

[74] J. Maynard Smith, “Time in the evolutionary process,” Studium Generale, vol. 23,pp. 266–272, 1970.

[75] S. Milgram, “The small world problem,” Psychology Today, vol. 1, p. 61, 1967.

[76] M. Mitchell, “Complex systems: Network thinking,” Artificial Intelligence, vol. 170,no. 18, pp. 1194–1212, 2006.

[77] J. M. Montoya, S. L. Pimm, and R. V. Sole, “Ecological networks and their fragility.”Nature, vol. 442, no. 7100, pp. 259–264, July 2006.

[78] Y. Moreno, J. B. Gomez, and A. F. Pacheco, “Instability of scale-free networks undernode-breaking avalanches,” Europhysics Letters, vol. 58, pp. 630–636, 2002.

[79] M. E. J. Newman, “Models of the small world,” Journal of StatisticalPhysics, vol. 101, no. 3, pp. 819–841, November 2000. [Online]. Available:http://dx.doi.org/10.1023/A:1026485807148

[80] ——, “The structure of scientific collaboration networks,” Proceedings of the Na-tional Academy of Sciences, vol. 98, pp. 404–409, 2001.

[81] ——, “Assortative mixing in networks,” Physical Review Letters, vol. 89, no. 20, p.208701, 2002.

[82] ——, “Mixing patterns in networks,” Physical Review E, vol. 67, no. 2, p. 026126,2003.

[83] ——, “Analysis of weighted networks,” July 2004. [Online]. Available:http://arxiv.org/abs/cond-mat/0407503

[84] ——, “Finding community structure in networks using the eigenvectors of matrices,”Physical Review E, vol. 74, p. 036104, 2006.

[85] ——, “Mathematics of networks,” in The New Palgrave Dictionary of Economics,S. N. Durlauf and L. E. Blume, Eds. Basingstoke: Palgrave Macmillan, 2008.

[86] ——, “The physics of networks - physics today november 2008,” Physics Today,November 2008.

[87] M. E. J. Newman, A. L. Barabasi, and D. J. Watts, Eds., The Structure and Dy-namics of Networks. Princeton University Press, 2006.

[88] B. O. Palsson, Systems Biology: Properties of Reconstructed Networks, 1st ed. Cam-bridge University Press, January 2006.

BIBLIOGRAPHY 188

[89] J. Park and M. E. J. Newman, “Statistical mechanics of networks,” PhysicalReview E, vol. 70, no. 6, pp. 066 117+, Dec 2004. [Online]. Available:http://dx.doi.org/10.1103/PhysRevE.70.066117

[90] M. Parter, N. Kashtan, and U. Alon, “Environmental variability and modularityof bacterial metabolic networks.” BMC evolutionary biology, vol. 7, no. 1, p. 169,September 2007.

[91] M. Piraveenan, M. Prokopenko, and A. Y. Zomaya, “Information-cloning of scale-free networks,” in Advances in Artificial Life: 9th European Conference on ArtificialLife (ECAL-2007), Lisbon, Portugal, ser. Lecture Notes in Artificial Intelligence,F. A. e Costa, L. M. Rocha, E. Costa, and A. C. I. Harvey, Eds. Springer, 2007,vol. 4648, pp. 925–935.

[92] ——, “Local assortativeness in scale-free networks,” Europhysics Letters, vol. 84,no. 2, p. 28002, 2008.

[93] ——, “Assortativeness and information in scale-free networks,” European PhysicalJournal B, vol. 67, pp. 291–300, 2009.

[94] ——, “Assortativity and growth of Internet,” European Physical Journal B, vol. 70,pp. 275–285, 2009.

[95] ——, “Local assortativeness in scale-free networks — addendum,” Europhysics Let-ters, vol. 89, no. 4, p. 49901, 2010.

[96] ——, “Assortative mixing in directed biological networks,” IEEE Transactions oncomputational biology and bioinformatics, in press, 2010.

[97] ——, “Centrality of four-node motifs in metabolic networks,” Theory in biosciences,under review, 2010.

[98] ——, “On congruity of nodes in complex networks,” IEEE Transactions on comput-ers, under review, 2010.

[99] M. Prokopenko, “Guided self-organization,” HFSP Journal, vol. 3, no. 5, pp. 287–289, 2009.

[100] M. Prokopenko, F. Boschetti, and A. J. Ryan, “An information-theoretic primer oncomplexity, self-organization, and emergence,” Complexity, vol. 15, no. 1, pp. 11–28,2009.

[101] E. Ravasz, A. L. Somera, D. A. Mongru, Z. N. Oltvai, and A. L. Barabasi, “Hier-archical organization of modularity in metabolic networks,” Science, vol. 297, no.5586, pp. 1551–1555, August 2002.

[102] M. Rubinov, S. A. Knock, C. J. Stam, S. Micheloyannis, A. W. F. Harris, L. M.Williams, and M. Breakspear, “Small-world properties of nonlinear brain activity inschizophrenia,” Human Brain Mapping, vol. 30, pp. 403–416, 2009.

BIBLIOGRAPHY 189

[103] M. Rubinov, O. Sporns, C. van Leeuwen, and M. Breakspear, “Symbiotic relation-ship between brain structure and dynamics,” BMC Neuroscience, vol. 10, no. 1,p. 55, 2009. [Online]. Available: http://www.biomedcentral.com/1471-2202/10/55

[104] T. Schreiber, “Measuring information transfer,” Physical Review Letters, vol. 85, p.461, 2000.

[105] S. Schuster, T. Pfeiffer, F. Moldenhauer, and I. Koch, “Exploring the pathway struc-ture of metabolism: decomposition into subnetworks and application to mycoplasmapneumoniae,” Bioinformatics, vol. 18–2, pp. 351–361, 2002.

[106] C. E. Shannon, “A mathematical theory of communication,” The Bell System Tech-nical Journal, vol. 27, pp. 379–423, 623–656, July, October, 1948.

[107] P. Shannon, A. Markiel, O. Ozier, N. S. Baliga, J. T. Wang, D. Ramage, N. Amin,B. Schwikowski, and T. Ideker, “Cytoscape: A software environment for integratedmodels of biomolecular interaction networks,” Genome Research, vol. 13, no. 11, pp.2498–2504, 2003.

[108] R. V. Sole and S. Valverde, “Information theory of complex networks: on evolutionand architectural constraints,” in Complex Networks, ser. Lecture Notes in Physics,E. Ben-Naim, H. Frauenfelder, and Z. Toroczkai, Eds. Springer, 2004, vol. 650.

[109] O. Sporns, M. Rubinov, and R. Kotter, “Brain connectivity tool-box,” 2009, http://www.brain-connectivity-toolbox.net/. [Online]. Available:http://www.brain-connectivity-toolbox.net/

[110] D. Stauffer and M. Sahimi, “Diffusion in scale-free networks with annealed disorder,”Physical Review E, vol. 72, p. 046128, 2005.

[111] U. Stelzl, U. Worm, M. Lalowski, C. Haenig, F. H. Brembeck, H. Goehler,M. Stroedicke, M. Zenkner, A. Schoenherr, S. Koeppen, J. Timm, S. Mintzlaff,C. Abraham, N. Bock, S. Kietzmann, A. Goedde, E. Toksoz, A. Droege, S. Kro-bitsch, B. Korn, W. Birchmeier, H. Lehrach, and E. E. Wanker, “A human protein-protein interaction network: a resource for annotating the proteome,” Cell, vol. 122,no. 6, pp. 957–968.

[112] A. Tang, C. Honey, J. Hobbs, A. Sher, A. Litke, O. Sporns, and J. Beggs, “Informa-tion flow in local cortical networks is not democratic,” BMC Neuroscience, vol. 9,no. Suppl 1, p. O3, 2008.

[113] A. Varma and B. O. Palsson, “Stoichiometric flux balance models quantitatively pre-dict growth and metabolic by-product secretion in wild-type escherichia coli w3110.”Applied and Environmental Microbiology, vol. 60, no. 10, pp. 3724–3731, October1994.

[114] A. Vazquez, “Growing network with local rules: Preferential attachment, clusteringhierarchy, and degree correlations,” Physical Review E, vol. 67, no. 5, p. 056104,May 2003. [Online]. Available: http://dx.doi.org/10.1103/PhysRevE.67.056104

BIBLIOGRAPHY 190

[115] A. Wagner and D. A. Fell, “The small world inside large metabolicnetworks.” Proceedings of Royal Society London B Biological Sciences,vol. 268, no. 1478, pp. 1803–1810, September 2001. [Online]. Available:http://dx.doi.org/10.1098/rspb.2001.1711

[116] D. J. Watts, Six Degrees: The Science of a Connected Age. New York: Norton,2003.

[117] D. J. Watts and S. H. Strogatz, “Collective dynamics of small-world networks,”Nature, vol. 393, no. 6684, pp. 440–442, June 1998. [Online]. Available:http://dx.doi.org/10.1038/30918

[118] J. Winick and S. Jamin, “Inet3.0: Internet topology generator,” University of Michi-gan, Tech. Rep. UMCSE-TR-456-02, 2002.

[119] R. Xulvi-Brunet and I. Sokolov, “Reshuffling scale-free networks: from random toassortative.” Physical Review E, vol. 70, no. 6 Pt 2, p. 066102, 2004.

[120] L. Yaeger, V. Griffith, and O. Sporns, “Passive and driven trends in the evolutionof complexity,” in Proceedings of the Eleventh International Conference on the Sim-ulation and Synthesis of Living Systems (ALifeXI), Winchester, UK, S. Bullock,J. Noble, R. Watson, and M. A. Bedau, Eds., pp. 725–732. Cambridge, MA, USA:MIT Press, 2008.

[121] L. Yaeger and O. Sporns, “Evolution of neural structure and complexity in a compu-tational ecology,” in Proceedings of the Tenth International Conference on Simula-tion and Synthesis of Living Systems (ALifeX), Bloomington, Indiana, USA, L. M.Rocha, L. S. Yaeger, M. A. Bedau, D. Floeano, R. L. Goldstone, and A. Vespignani,Eds., pp. 330–336. MIT Press, 2006.

[122] L. S. Yaeger, “Computational genetics, physiology, metabolism, neural systems,learning, vision and behaviour or polyworld: Life in a new context,” in Proceed-ings of the Artificial Life III Conference, Santa Fe, NM, USA, C. G. Langton, Ed.,pp. 263–298. Addison-Wesley, 1994.

[123] J. Zhou, X. Xu, J. Zhang, J. Sun, M. Small, and J. Lu, “Generating an assortativenetwork with a given degree distribution,” International Journal of Bifurcation andChaos, vol. 18, no. 11, pp. 3495–3502, 2008.

[124] S. Zhou, “Understanding the evolution dynamics of internet topology,” PhysicalReview E, vol. 74, p. 016124, 2006.

[125] S. Zhou and R. J. Mondragon, “Towards modelling the internet topology - theinteractive growth model,” Physical Review E, vol. 67, p. 026126, 2003.

[126] ——, “The rich-club phenomenon in the internet topology,” IEEE CommunicationsLetters, vol. 8, pp. 180–182, 2004.

Thesis an Final

Documents

Transcript of Thesis an Final