An Interdisciplinary Perspective on Artificial

Immune Systems

Jon TimmisDepartment of Electronics andDepartment of Computer Sciencejtimmis@cs.york.ac.ukhttp://www-users.cs.york.ac.uk/jtimmis

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Artificial what?

Artificial Immune Systems: A typical definition

AIS are adaptive systems inspired by theoretical immunology and observed immune functions,

principles and models, which are applied to complex problem domains

[De Castro and Timmis,2002]

But I think this might be a bit limiting in terms of definition ..

A bit of history … Developed from the field of theoretical

immunology in the mid 1980’s. Suggested we ‘might look’ at the IS

1990 – Ishida first use of immune algorithms to solve problems

Forrest et al – Computer Security mid 1990’s Hunt et al, mid 1990’s – Machine learning ICARIS conference series, ARTIST network

History (cont.) Started quite immunologically grounded

Bersini’s work with Varela Forrest's work with Perelson

Kind of moved away from that, and abstracted more

Now there seems to be a move to go back to the roots of immunology and greater interaction … but how do we manage this interaction to make it worth

while for all concerned …. ?

What does engineering have to do with immune systems?

Unique to individuals Distributed Imperfect Detection Anomaly Detection Learning/Adaptation Memory Feature Extraction Diverse ..and more

Robust Scalable Flexible Exhibit graceful

degradation Homeostatic

Systems that are:Computational Properties

Example Application Areas

Computer Security

Computer Security

OptimisationOptimisationRobotic Control

Robotic Control

Data-Mining and

classification

Data-Mining and

classification

Anomaly Detection

Anomaly Detection

Network models

Clonal Selection

Negative selection

Bone Marrow

What is the Immune System ?

a complex system of cellular and molecular components having the primary function of distinguishing self from not self and defense against foreign organisms or substances (Dorland's Illustrated Medical Dictionary)

The immune system is a cognitive system whose primary role is to provide body maintenance (Cohen)

Immunologists Disagree“There is an obvious and dangerous

potential for the immune system to kill its host; but it is equally obvious that the best minds in

immunology are far from agreement on how the immune system manages to avoid

this problem”

Langman, R. E. and Cohn, M., Editorial Summary, Seminars in Immunology, vol. 12, pp. 343-344, 2000

What is the Immune System ?

S/NS

Cohen

Varela

Matzinger

• The are many different viewpoints

•Lots of common ingredients (??)

•All tell us about information processing …

Clonal Selection as an example for information processing

Immune Responses - continual information processing

Antigen Ag 1 Antigens Ag1, Ag2

Primary Response Secondary Response

Lag

Response to Ag1

Anti

body Concentration

Time

Lag

Response to Ag2

Response to Ag1

...

...

Cross-Reactive Response

...

...

Antigen Ag1 + Ag3

Response to Ag1 + Ag3

Lag

An `artificial immune system’ in an engineering

contextKeeping ATM’s working

ATMs High usage machines Don’t go wrong that often, but if they do it can

be expensive Create logs when they go wrong It is possible to use that data to immunise a

system at a number of levels via an Adaptable Error Detection system

Adaptable error detection as a means to improved availability Error detection

Improved error detection enhances availability Error detection techniques usually exploit known

systems profile for detecting error states and behaviour These error detection techniques are limited to the

detection of errors known at design-time of systems Adaptable error detection is aimed at detecting errors

that were not known during the design-time of systems

A Framework for AIS

Algorithms

Affinity

Representation

Application

Solution

AIS

[De Castro and Timmis, 2002]

Within the AIS Framework

Representation Sequence of states --> fatal state

Affinity measure Similarity of sequences (weighted)

Algorithm Dynamic clonal selection

[De Lemos et al, 2007]

Architecture for Immune AED

[De Lemos et al, 2007]

Results

AISEC v1 AISEC v2

Accuracy

Mean detectionTime interval

85.78%(6)89.93%(.2)

86.67%(5)91.53%(.16)

0:11:21:22(0:5:20:16) 0:01:03:30 (0:0:9:35)

0:12:31:10 (0:3:36:37)

0:02:25:41 (0:0:6:16)

[De Lemos et al, 2007]

A bit of time for reflection …

Are we really capturing immune system complexity in our AIS?(or should we even care?)

modelling

Analyticalframework/

principle

A Framework for Thinking about and Developing AIS

Biologicalsystem

Simplifyingabstract

representation

Bio-inspiredalgorithms

Probes,Observations,experiments

DC activation, T-cell clonality

Mathematical models

Construct a computational

model

Abstract into algorithms

suitable for an application

Analysable, validated systems that fully exploit the underlying biology

[Stepney et al, 2005]

Interdisciplinary interaction via immune

modelling

What is in it for both sides?

Modelling Approaches Mathematical

E.g. Differential equations Computational

Various calculi Agent based modelling UML

We are investigating a number of different approaches at the moment to see which (if any) are useful (both to us and immunologists)

UML UML = Unified Modelling Language

Collection of 13 diagrams for general purpose modelling

Mostly used in software engineering for modelling “the real world”...

Diagrams fall into 2 categories Structural Behavioural

Modelling Complex Systems with UML Most of the diagrams in UML we

have not found to be that useful Ones that we have:

Class diagrams: what things are State diagrams: how things behave Activity diagrams: how things interact

UML Perspectives Conceptual

Concepts of the domain Implementing classes are related, but doesn't

have to be one-to-one mapping Specification

Interfaces Implementation

Code specifics

State Chart - Clonal Selection

[Bersini, 2006]

Process Oriented Approaches Processes are again a natural way to think about

biological systems Investigating two approaches of modelling this way Current research is investigating the development of a

pattern language for complex systems (at many levels) Modelling infrastructure (tool set, and method) for the

modelling of complex systems - our drive is the immune system

Occam- is our target language which allows us to build large-scale, highly parallel simulation

Currently working with the IIU at York on the development of models of expansion and contraction of blood vessels in lymph nodes and also the formation of granulomas under certain infections (also making use of UML in this context)

Extensible Architecture for Homeostasis

http://www.bioinspired.com/research/xArcH/index.shtml

-Calculus The -calculus [Milner 1999]. A

process calculus designed to model communicating mobile systems.

What is mobility?

Stochastic -Calculus• -calculus is good for qualitative analysis of

systems, Stochastic allows quantitative.• Associates every activity with a rate parameter

r [0, ].

Why use -Calculus? Can model the interactions between biological

components directly, possibly more intuitive (in some cases) than ODE modeling.

Can perform qualitative analysis through their bi-simulation equivalence.

Can perform quantitative analysis through simulation SPiM, BioSpi.

Through analysis can hopefully abstract what it is about the biological system that gives it its behaviour.

Some interesting immunology: Tunable T-cell receptors Classic immunology suggests a clear recognition of self/non-self

by randomly generated repertoire of cells - how is this possible? Tunable activation threshold (TAT): Proposed by [Grossman,

1992] to help explain mechanisms for self-tolerance. T Cells are mostly discussed and are viewed as having tunable

thresholds with which dictate proliferation and differentiation and therefore react only to changes in the environment and not any one specific interaction

The implications are: Self-reactive T-cells can exist but …. .. they require generally higher affinity for antigen, or a higher

avidity is required, i.e. the rate and amount by at which peptides are presented is faster for antigen.

One small part … Excitatory and Inhibitory factors are produced when the T cell

binds via its T Cell Receptor A war of phosphorylation between a kinase and a phosphatase. If

kinase activity is higher than phosphatase causes phosphorylation. If phosphatase activity is higher than kinase

causes dephosphorylation.

Why might this TAT idea be useful to engineers? The real-world is hard, and building systems that

can cope with a variety of input, that changes over time, is difficult

If we could have a system of agents that can tune themselves to tolerate, or not, certain input .. that would be very useful .. It would allow us to to begin to capture homeostasis ….

Look at patterns of response

Lymphocyte Entry to the Lymph Node through High Endothelial Venules

http://www.cosmos-research.org

On-going modelling work Collaboration with the Infection and

Immunology Unit at York Early stages (no simulation as yet, still

under development), have some basic models

Provide support for the hypothesis: The increase in lymphocyte numbers in lymph

node during an immune response is a direct result of migration rather than proliferation of existing lymphocytes in the lymph node

38

Lymph Nodes

Immune organs where adaptive immune response initiated and antibodies produced

Hundreds throughout body

Cells enter though blood or lymphatic system

39

Venules

Small blood vessels Bring de-oxygenated blood to the

veins from capillary bed

40

High Endothelial Venules (HEV) Certain areas of the lymph node

venule network are made up of HEVs HEVs characterised by tall and plump

endothelial cells

Endothelial CellEndothelial Cell

41

HEVs in a Lymph Node

42

Pericytes

Cells that wrap around small blood vessels Act as scaffolding Similar to smooth muscle cells

Constriction and dilation regulates diameter and blood flow of vessel

Endothelial Cell

Pericyte

43

Lymphocyte Migration (1)

Lymphocytes enter lymph node through HEVs Initiate in a rolling process Under certain conditions, lymphocytes slow

and squeeze though between endothelial cells

44

Lymphocyte Migration (2) Rolling, slowing and migration mechanism

controlled by cell surface molecules and receptors (selectins, integrins, chemokines)

45

Lymphocyte Migration (3)

A chemical signal molecule (chemokine) emitted in HEV crucial to lymphocyte migration HEVs facilitate lymphocytes migration but

exclude other leukocytes (white blood cells) Quarter of circulating lymphocytes leave

blood after entering HEV Migration through venule takes between

10 and 20 minutes

46

Number of cellsin millions

Experimental data

Our immunologists have measured Number of lymphocytes in a node during

response Relationship between pericyte dilation

(distance from vessel) and blood vessel size

47Lumen Size in nm Venule Perimeter in nm

PericyteDistancein nm

What are we doing with this? Developing UML models of the rolling process

For the most part this has been done. Developing simulations

First without space, then with space Output will be (in the first instance) a graph showing

lymphocyte numbers over time Number of challenges

• Time, space etc. Importantly, we are reviewing the process of modelling.

What assumptions do we make What problems do we encounter What tools work and what don’t (and why)

A wider field than ever before? Three types of ‘AIS’ people:

1. ‘Literal’ school : Those who try and build things to do what the IS does (e.g. security systems)

2. ‘metaphorical’ school: Those who use the IS as inspiration, but may be far from the what they IS actually does e.g. optimisation algorithms

3. ‘modelling’ school: Those who try and understand the IS through a series of models (computational and mathematical) e.g. models of self/non-self or tunable activation thresholds

[Cohen, 2007]

The great possibility for interaction Use of modelling tools and the

development of new tools CoSMoS project http://www.cosmos-

research.org Engage the experimentalist

They want predictions - models should be able to help

Through good modelling, engineering can also reap the benefit through a greater understanding of the immune system

References [Cohen, 2007] Computing the state of the body. Nature Rev. Imm. 7, 569-574

(2007) [De Lemos et al, 2007] R. De Lemos, J. Timmis. M. Ayara, and S. Forrest. Immune

Inspired Adaptable Error Detection for Automated Teller Machines. IEEE SMC Part B. [Forrest and Beachemin, 2007] Computer Immunology. Immunological Reviews. Vol.

216. [Timmis 2007] J. Timmis. Challenges for Artificial Immune Systems. Natural

Computation. [Stepney et al. 2006] S. Stepney, R. Smith, J. Timmis, A. Tyrrell, M. Neal and A.Hone.

Conceptual Frameworks for Artificial Immune Systems, International Journal of Unconventional Computing. 2006.

[De Castro and Timmis,2002] L. De Castro and J. Timmis. Artificial Immune Systems; A New Computational Intelligence Paradigm. Springer. 2002.

[Farmer et al, 1986] Farmer, J. D., N. H. Packard and A. Perelson. "The Immune System, Adaptation, and Machine Learning." Physica D 22(1-3) (1986): 187-204

[Owens et al,2008] Owens, N, Timmis, J. Tyrrell, A. and Greensted, A. Modelling the Tunability of Early T-cell Signaling Events. ICARIS 2008.

Acknowledgements Paul Andrews /Susan Stepney /

Amelia Ismail (CoSMoS) Lisa Scott, Mark Coles (IIU) Nick Owens / Andy Greensted / Andy

Tyrrell (Xarch)