An Investigation of a Methodology for the Development of Artificial Immune Systems: A Case-Study in Immune Receptor Degeneracy

Transcription

1 An Investigation of a Methodology for the Development of Artificial Immune Systems: A Case-Study in Immune Receptor Degeneracy Paul Simon Andrews Submitted for the degree of Doctor of Philosophy University of York Department of Computer Science September 2008

2 Abstract For nearly half a century, biology has provided a rich source of inspiration for computing systems, the result of which is a large body of work that includes biologically-inspired algorithms such as artificial immune system (AIS). Despite the effort that has been invested in engineering these algorithms, relatively little research has focused on how best to extract the underlying biological properties from which inspiration is taken. This thesis aims to address this failing by following a more principled approach to the development of an AIS. We begin by exploring the current state of AIS, and examine how immunology has been used to inspire AIS to date. This leads us to identify a methodology for developing AIS that incorporates a number of explicit modelling stages to extract the key features of the biological system. An examination of the immunological literature identifies our immune inspiration: immune receptor degeneracy and the mechanism of patterns of response to provide immune specificity. Our first step in developing an AIS based on these properties is to build an agent-based simulation to explore them free of any engineering application bias. We then investigate the idea of tunable activation thresholds for immune detectors, which results in the generation of a single pattern of response from a detector population when subject to a stimulus. Using the insight gained from these investigations, we construct a framework to allow patterns of degenerate tunable detectors to be incorporated into AIS as a data pre-processing stage. This framework it then instantiated for a simple pattern classification AIS. To conclude we analyse the process we have followed to develop our AIS and assess the benefits and drawbacks of the approach we have taken, showing how a more principled approach can be applied to the design of biologically-inspired algorithms. 2

3 Contents 1 Introduction Motivation Computing and Biology An Approach to Bio-inspired Algorithm Design Artificial Immune Systems Thesis Details Goal, Themes and Justification Content Overview Research Questions and Contribution Publications Applied Artificial Immune Systems The Immune System An Overview Organs, Cells and Molecules Lymphocytes Clonal Selection Theory and Self Non-self Immune Network Theory Danger Theory Artificial Immune Systems for Engineering Component Representations and Affinity Measures Negative Selection Algorithms Clonal Selection Algorithms Immune Network Algorithms Danger Theory Algorithms Current Opinion on Applied AIS Structure

4 CONTENTS Applications and Usefulness Future Properties Conclusions Exploiting the Immune System Bio-Inspired Algorithm Design through Conceptual Frameworks The Conceptual Framework Approach and AIS Modelling and Simulating Immune Systems Mathematical and Formal Approaches Agent Based Approaches Diagrammatic and Software Engineering Approaches Understanding Immunology through Models Developing a New AIS: A Methodology Identifying Biology for a Novel Artificial Immune System Levels of Immune Abstraction for AIS Immunology Today The Science of Self Non-self Discrimination? A Clash of Methodologies Implications for AIS Inspiration The Cognitive Immune System The Role of the Immune System Immune Specificity Immune Cognition Inspiration for AIS Degeneracy In Biology In the Immune System In AIS Degeneracy and Patterns for AIS A Simulation of Patterns of Degenerate Detectors in a Lymph Node Goal Returning to Immunology Lymph Nodes T H Activation in a Lymph Node Model Basis Scope Biological Components and Behaviours

5 CONTENTS 5.4 Agent Based Model Lymph Node Environment Chemokine Space and Paracortex Agents The Simulator Java Implementation Chemokine Space Calibration An Example Run Experiments and Results Patterns of T H Agent Response Thresholds and Shape Lengths Conclusions The Adaptable Lymphocyte Goal and Motivation The Adaptable Lymphocyte Hypothesis Investigating TAT Behaviours The TAT Equation AIS-like Data Example Population Patterns Generating Patterns of Response Population Pattern Parameters Conclusions A Degenerate and Tunable Artificial Immune System Degeneracy and Patterns of Response for Engineering Algorithm Framework for Data Pre-Processing Data Pre-processing The Algorithm Framework Population Pattern Parameters AIS Instantiation Application and Data Algorithm and Response Shape Processing PoRTuDe Algorithm Experiments and Results Experimental Set-up Iris Results Breast Cancer Results Ionosphere Results

6 CONTENTS Results Discussion Conclusions Reflections on Following the Conceptual Framework Approach The Conceptual Framework for AIS in Context Examination of CFAIS Stages Identifying Biology Probes Models and Simulations Frameworks and Algorithms The CFA as a Whole The Lack of an Inter-Disciplinary Approach Domain Expertise Feedback to Biology Adapting the CFA Conclusions and Future Work A Lymph Node Simulation Supplementary Results 197 A.1 Results with Length A.1.1 Recognition Threshold A.1.2 Recognition Threshold A.1.3 Recognition Threshold A.1.4 Antigen 0, 2 and 7 Response to Different T H Set A.2 Results with Length A.2.1 Recognition Threshold A.2.2 Recognition Threshold A.2.3 Recognition Threshold A.3 Results with Length A.3.1 Recognition Threshold A.3.2 Recognition Threshold A.3.3 Recognition Threshold B Adaptable Lymphocyte Supplementary Results 222 B.1 TAT behaviours B.2 Generating Patterns B.3 Pattern Parameters C Pattern Classification Supplementary Results 233 C.1 Iris Data Set

7 CONTENTS C.2 Cancer Data Set C.3 Ionosphere Data Set References 249 7

8 List of Tables 5.1 Parameters for the Java simulator of the agent-based lymph node simulator Ten randomly generated Real-valued T H shapes of length Ten randomly generated Real-valued antigen shapes of length Percentage of times a T H agent is activated by an antigen agent over 50 simulation runs with recog = Median iterations for a T H agent to be activated by an antigen agent from over 50 simulation runs with recog = Part 1 of the similarity between antigen agents shapes based on respective affinities Part 2 of the similarity between antigen agents shapes based on respective affinities Ten randomly generated Real-valued T H shapes of length 2, reproduced from table Three randomly generated Real-valued antigen shapes of length 2, reproduced from table Classification results for k-nearest neighbour on the iris, ionosphere and breast cancer data sets for k = 1, 3 and The best twenty classification results for the iris data set based on the mean classification over fifty separate runs The worst ten classification results for the iris data set based on the mean classification over fifty separate runs The best twenty classification results for the breast cancer data set based on the mean classification over fifty separate runs The worst ten classification results for the breast cancer data set based on the mean classification over fifty separate runs

9 LIST OF TABLES 7.6 The best twenty classification results for the ionosphere data set based on the mean classification over fifty separate runs The worst ten classification results for the ionosphere data set based on the mean classification over fifty separate runs A.1 Ten randomly generated Real-valued T H shapes of length 2, reproduced from table A.2 Ten randomly generated Real-valued antigen shapes of length 2, reproduced from table A.3 Part 1 of the similarity between antigen agents shapes based on respective affinities, reproduced from table A.4 Part 2 of the similarity between antigen agents shapes based on respective affinities, reproduced from table A.5 Percentage of times a T H agent from table A.1 is activated by an antigen agent from table A.2 over 50 simulation runs with recognition threshold, recog = A.6 Median iterations for a T H agent from table A.1 to be activated by an antigen agent from table A.2 over 50 simulation runs with recognition threshold, recog = A.7 Percentage of times a T H agent from table A.1 is activated by an antigen agent from table A.2 over 50 simulation runs with recognition threshold, recog = A.8 Median iterations for a T H agent from table A.1 to be activated by an antigen agent from table A.2 over 50 simulation runs with recognition threshold, recog = A.9 Percentage of times a T H agent from table A.1 is activated by an antigen agent from table A.2 over 50 simulation runs with recognition threshold, recog = A.10 Median iterations for a T H agent from table A.1 to be activated by an antigen agent from table A.2 over 50 simulation runs with recognition threshold, recog = A.11 An alternative set of ten randomly generated Real-valued T H shapes of length A.12 Ten randomly generated Real-valued T H shapes of length 4, shown to 6 significant figures A.13 Ten randomly generated Real-valued antigen shapes of length 4, shown to 6 significant figures

10 LIST OF TABLES A.14 Part 1 of the similarity between antigen agents shapes from table A.13 based on respective affinities A.15 Part 2 of the similarity between antigen agents shapes from table A.13 based on respective affinities A.16 Percentage of times a T H agent from table A.12 is activated by an antigen agent from table A.13 over 50 simulation runs with recognition threshold, recog = A.17 Median iterations for a T H agent from table A.12 to be activated by an antigen agent from table A.13 over 50 simulation runs with recognition threshold, recog = A.18 Percentage of times a T H agent from table A.12 is activated by an antigen agent from table A.13 over 50 simulation runs with recognition threshold, recog = A.19 Median iterations for a T H agent from table A.12 to be activated by an antigen agent from table A.13 over 50 simulation runs with recognition threshold, recog = A.20 Percentage of times a T H agent from table A.12 is activated by an antigen agent from table A.13 over 50 simulation runs with recognition threshold, recog = A.21 Median iterations for a T H agent from table A.12 to be activated by an antigen agent from table A.13 over 50 simulation runs with recognition threshold, recog = A.22 Ten randomly generated Real-valued T H shapes of length 8, shown to 5 significant figures A.23 Ten randomly generated Real-valued antigen shapes of length 8, shown to 5 significant figures A.24 Part 1 of the similarity between antigen agents shapes from table A.23 based on respective affinities A.25 Part 2 of the similarity between antigen agents shapes from table A.23 based on respective affinities A.26 Percentage of times a T H agent from table A.22 is activated by an antigen agent from table A.23 over 50 simulation runs with recognition threshold, recog = A.27 Median iterations for a T H agent from table A.22 to be activated by an antigen agent from table A.23 over 50 simulation runs with recognition threshold, recog =

11 LIST OF TABLES A.28 Percentage of times a T H agent from table A.22 is activated by an antigen agent from table A.23 over 50 simulation runs with recognition threshold, recog = A.29 Median iterations for a T H agent from table A.22 to be activated by an antigen agent from table A.23 over 50 simulation runs with recognition threshold, recog = A.30 Percentage of times a T H agent from table A.22 is activated by an antigen agent from table A.23 over 50 simulation runs with recognition threshold, recog = A.31 Median iterations for a T H agent from table A.22 to be activated by an antigen agent from table A.23 over 50 simulation runs with recognition threshold, recog = B.1 Ten randomly generated T H shapes, reproduced from table B.2 Three randomly generated antigen shapes, reproduced from table C.1 Mean and standard deviation of classification accuracy for the iris data set from fifty runs of the PoRTuDe algorithm using a k value of 1 and a range of µ, α and δ values C.2 Mean and standard deviation of classification accuracy for the iris data set from fifty runs of the PoRTuDe algorithm using a k value of 3 and a range of µ, α and δ values C.3 Mean and standard deviation of classification accuracy for the iris data set from fifty runs of the PoRTuDe algorithm using a k value of 5 and a range of µ, α and δ values C.4 Mean, standard deviation, lowest and highest classification accuracy from fifty runs of five different test/train splits of the iris data set for a range of k, µ, α and δ values C.5 Mean and standard deviation of classification accuracy for the cancer data set from fifty runs of the PoRTuDe algorithm using a k value of 1 and a range of µ, α and δ values C.6 Mean and standard deviation of classification accuracy for the cancer data set from fifty runs of the PoRTuDe algorithm using a k value of 3 and a range of µ, α and δ values C.7 Mean and standard deviation of classification accuracy for the cancer data set from fifty runs of the PoRTuDe algorithm using a k value of 5 and a range of µ, α and δ values

12 LIST OF TABLES C.8 Mean, standard deviation, lowest and highest classification accuracy from fifty runs of five different test/train splits of the cancer data set for a range of k, µ, α and δ values C.9 Mean and standard deviation of classification accuracy for the ionosphere data set from fifty runs of the PoRTuDe algorithm using a k value of 1 and a range of µ, α and δ values C.10 Mean and standard deviation of classification accuracy for the ionosphere data set from fifty runs of the PoRTuDe algorithm using a k value of 3 and a range of µ, α and δ values C.11 Mean and standard deviation of classification accuracy for the ionosphere data set from fifty runs of the PoRTuDe algorithm using a k value of 5 and a range of µ, α and δ values C.12 Mean, standard deviation, lowest and highest classification accuracy from fifty runs of five different test/train splits of the ionosphere data set for a range of k, µ, α and δ values

13 List of Figures 1.1 The original conceptual framework for bio-inspired algorithm design from [Stepney et al. 2004] The organs of the immune system and their locations with the body. Reproduced from [de Castro & Timmis 2002b] The structure of an antibody showing the variable regions. Reproduced from [de Castro & Timmis 2002b] The clonal selection theory showing negative selection, proliferation and differentiation. Reproduced from [de Castro & Timmis 2002b] A layered framework for AIS. Reproduced from [de Castro & Timmis 2002b] A 2-dimensional depiction of shape-space. Reproduced from [de Castro & Timmis 2002b] The original conceptual framework for bio-inspired algorithm design from [Stepney et al. 2004] The original meta-framework for integrating computational domains from [Stepney et al. 2004] A conceptual framework for developing AIS A depiction of the levels of abstraction within the stages of our framework for developing AIS Co-respondence according to [Cohen 2000b] Areas of a lymph node from [Goldsby et al. 2003] showing the cortex, paracortex and medulla T H state model showing T H cell activation behaviours APC state model showing presentation and activation behaviours

14 LIST OF FIGURES 5.4 The two layers of the lymph node model cellular space with example values Cellular space Moore neighbourhood An example of the stages involved in the diffusion rule for a single cell in the chemokine space A snapshot of the lymph node degeneracy simulator user interface Chemokine space of lymph node model upon initialisation Chemokine space of lymph node model after 25 iterations, where w = 25, h = 25, chem-max = 100 and chem-prod = 25% Chemokine space of lymph node model after 25 iterations, where w = 25, h = 25, chem-max = 100 and chem-prod = 5% Chemokine space of lymph node model after 25 iterations, where w = 25, h = 25, chem-max = 10 and chem-prod = 25% Chemokine space of lymph node model after 25 iterations, where w = 25, h = 25, chem-max = 1000 and chem-prod = 25% Percentage of times a T H agent is activated by antigen agent 0 over 50 simulation runs with recognition threshold, recog = Percentage of times a T H agent is activated by antigen agent 2 over 50 simulation runs with recognition threshold, recog = Percentage of times a T H agent is activated by antigen agent 7 over 50 simulation runs with recognition threshold, recog = Box and whisker plot showing how long a T H agent takes to become activated by antigen agent 0 over 50 simulation runs with recognition threshold, recog = Box and whisker plot showing how long a T H agent takes to become activated by antigen agent 2 over 50 simulation runs with recognition threshold, recog = Box and whisker plot showing how long a T H agent takes to become activated by antigen agent 7 over 50 simulation runs with recognition threshold, recog = Percentage of times a T H agent is activated by antigen agent 0 over 50 simulation runs with recognition thresholds, recog = 2, 4 and Percentage of times a T H agent is activated by antigen agent 2 over 50 simulation runs with recognition thresholds, recog = 2, 4 and Percentage of times a T H agent is activated by antigen agent 7 over 50 simulation runs with recognition thresholds, recog = 2, 4 and

15 LIST OF FIGURES 6.1 Original tunable activation threshold behaviour from [Grossman & Paul 1992] Relationship between constant excitation, the excitation index and an activation threshold set to trace 0.1 above the excitation index Relationship between excitation and excitation index to a varying input stimulus created from a Gaussian distribution An illustration of the constant tuning of the excitation index to rising and falling levels of excitation generated from two Gaussian distributions The effect of parameter α on the speed at which the excitation index tunes to a constant excitation Example of how the excitation effects the excitation index at lower excitation values to compare with figure Reproduction of TAT behaviours graph in figure 6.1 using equation The tuning of T H agent shapes 0 to 4 from table 6.1 to a constant stimulus by antigen 0 from table The tuning of T H agent shapes 5 to 9 from table 6.1 to a constant stimulus by antigen 0 from table The tuning of T H agent shape 1 from table 6.1 to a constant stimulus by antigens 0, 2 and 7 from table T H detector population sizes for shapes 0 to 4 from table 6.1 to a constant stimulus by Ag 2 from table T H detector population sizes for shapes 5 to 9 from table 6.1 to a constant stimulus by Ag 2 from table Active T H detector population sizes for all shapes from table 6.1 to a constant stimulus by Ag 2 from table The population response of three different antigens, Ag 0, 2, and 7 to the same set of degenerate tunable detectors The population response of three different antigens, Ag 0, 2, and 7 to the same set of degenerate tunable detectors with larger α The population response of three different antigens, Ag 0, 2, and 7 to the same set of degenerate tunable detectors with larger θ The population response of three different antigens, Ag 0, 2, and 7 to the same set of degenerate tunable detectors with low µ

16 LIST OF FIGURES 7.1 The patterns of degenerate tunable detector framework for AIS (DTD-AIS), which incorporates a data pre-processing stage into an application specific algorithm Response shape of 5 instances of the Setosa iris data class, generated using algorithm Response shape of 5 instances of the Virginica iris data class, generated using algorithm Response shape of 5 instances of the Versicolor iris data class, generated using algorithm Box and whisker plot for fifty runs of PoRTuDe with the iris data set for varying value of δ Box and whisker plot for fifty runs of PoRTuDe with the iris data set for varying value of α Box and whisker plot for fifty runs of PoRTuDe with the iris data set for varying value of µ Box and whisker plot for fifty runs of PoRTuDe with the breast cancer data set for varying value of δ Box and whisker plot for fifty runs of PoRTuDe with the breast cancer data set for varying value of α Box and whisker plot for fifty runs of PoRTuDe with the breast cancer data set for varying value of µ Box and whisker plot for fifty runs of PoRTuDe with the ionosphere data set for varying value of δ Box and whisker plot for fifty runs of PoRTuDe with the ionosphere data set for varying value of α Box and whisker plot for fifty runs of PoRTuDe with the ionosphere data set for varying value of µ A conceptual framework for developing AIS A.1 Percentage of times a T H agent (detector ID) from table A.1 is activated by antigen agent 7 from table A.2 over 50 simulation runs with recognition threshold, recog = A.2 Percentage of times a T H agent (detector ID) from table A.1 is activated by antigen agent 2 from table A.2 over 50 simulation runs with recognition threshold, recog = A.3 Percentage of times a T H agent (detector ID) from table A.1 is activated by antigen agent 6 from table A.2 over 50 simulation runs with recognition threshold, recog =

17 LIST OF FIGURES A.4 Percentage of times a T H agent (detector ID) from table A.1 is activated by antigen agent 1 from table A.2 over 50 simulation runs with recognition threshold, recog = A.5 Percentage of times a T H agent (detector ID) from table A.1 is activated by antigen agent 3 from table A.2 over 50 simulation runs with recognition threshold, recog = A.6 Percentage of times a T H agent (detector ID) from table A.1 is activated by antigen agent 5 from table A.2 over 50 simulation runs with recognition threshold, recog = A.7 Percentage of times a T H agent (detector ID) from table A.1 is activated by antigen agent 4 from table A.2 over 50 simulation runs with recognition threshold, recog = A.8 Percentage of times a T H agent (detector ID) from table A.1 is activated by antigen agent 9 from table A.2 over 50 simulation runs with recognition threshold, recog = A.9 Percentage of times a T H agent (detector ID) from table A.1 is activated by antigen agent 8 from table A.2 over 50 simulation runs with recognition threshold, recog = A.10 Percentage of times a T H agent (detector ID) from table A.1 is activated by antigen agent 0 from table A.2 over 50 simulation runs with recognition threshold, recog = A.11 Percentage of times a T H agent (detector ID) from table A.11 is activated by antigen agent 0 from table A.2 over 50 simulation runs with recognition threshold, recog = A.12 Percentage of times a T H agent (detector ID) from table A.11 is activated by antigen agent 2 from table A.2 over 50 simulation runs with recognition threshold, recog = A.13 Percentage of times a T H agent (detector ID) from table A.11 is activated by antigen agent 7 from table A.2 over 50 simulation runs with recognition threshold, recog = A.14 Box and whisker for a T H agent from table A.11 takes to become activated by antigen agent 0 from table A.2 over 50 simulation runs with recognition threshold, recog = A.15 Box and whisker for a T H agent from table A.11 takes to become activated by antigen agent 2 from table A.2 over 50 simulation runs with recognition threshold, recog =

18 LIST OF FIGURES A.16 Box and whisker for a T H agent from table A.11 takes to become activated by antigen agent 7 from table A.2 over 50 simulation runs with recognition threshold, recog = B.1 The tuning of T H agent shapes 0 to 4 from table B.1 to a constant stimulus by antigen 0 from table B B.2 The tuning of T H agent shapes 5 to 9 from table B.1 to a constant stimulus by antigen 0 from table B B.3 The tuning of T H agent shapes 0 to 4 from table B.1 to a constant stimulus by antigen 2 from table B B.4 The tuning of T H agent shapes 5 to 9 from table B.1 to a constant stimulus by antigen 2 from table B B.5 The tuning of T H agent shapes 0 to 4 from table B.1 to a constant stimulus by antigen 7 from table B B.6 The tuning of T H agent shapes 5 to 9 from table B.1 to a constant stimulus by antigen 7 from table B B.7 Active T H detector population sizes for all shapes from table B.1 to a constant stimulus by Ag 0 from table B B.8 Active T H detector population sizes for all shapes from table B.1 to a constant stimulus by Ag 2 from table B B.9 Active T H detector population sizes for all shapes from table B.1 to a constant stimulus by Ag 7 from table B B.10 The population response of three different antigens, Ag 0, 2, and 7 from table B.2 to the same set of degenerate tunable detectors from table B B.11 The population response of three different antigens, Ag 0, 2, and 7 from table B.2 to the same set of degenerate tunable detectors from table B.1 with smaller α B.12 The population response of three different antigens, Ag 0, 2, and 7 from table B.2 to the same set of degenerate tunable detectors from table B.1 with larger α B.13 The population response of three different antigens, Ag 0, 2, and 7 from table B.2 to the same set of degenerate tunable detectors from table B.1 with smaller θ B.14 The population response of three different antigens, Ag 0, 2, and 7 from table B.2 to the same set of degenerate tunable detectors from table B.1 with larger θ

19 LIST OF FIGURES B.15 The population response of three different antigens, Ag 0, 2, and 7 from table B.2 to the same set of degenerate tunable detectors from table B.1 with µ = B.16 The population response of three different antigens, Ag 0, 2, and 7 from table B.2 to the same set of degenerate tunable detectors from table B.1 with µ = B.17 The population response of three different antigens, Ag 0, 2, and 7 from table B.2 to the same set of degenerate tunable detectors from table B.1 with µ =

20 List of Algorithms 1 The original negative selection algorithm of [Forrest et al. 1994] CLONALG for pattern matching [de Castro & Von Zuben 2002] ainet for data clustering [de Castro & Von Zuben 2000] Dendritic Cell Algorithm adapted from [Greensmith et al. 2005] Algorithm for updating chemokine values in the chemokine space Algorithm for the agent-based Java simulator Population response generation algorithm Pseudocode for the PoRTuDe pattern classification AIS

21 Acknowledgements To Jon for the guidence and supervision. To my parents for all their support over the years. To Helena for the support and patience that made this thesis possible. 21

22 Declaration Elements of the research reported in this thesis are based on my contributions to the following previously presented publications: [Andrews & Timmis 2005a, Andrews & Timmis 2006, Andrews & Timmis 2007, Andrews & Timmis 2008, Timmis et al. 2008a]. 22

23 CHAPTER ONE Introduction We present in this introductory chapter both a motivation and examination of the work detailed in this thesis. First, section 1.1 provides the thesis motivation and inspiration by describing the background to the work. This is followed in section 1.2 with an examination of the structure and content of the thesis, stating our goal, research questions, contribution and providing a content overview of the thesis. 1.1 Motivation The relationship between biology and computing in research has a long and rich history. For instance, biologically-inspired computing dates back to the 1960s with the idea of using evolutionary theory to evolve solutions to computational problems [Fogel 1998]. However, despite the large quantity of work that has been presented since, there is very little that addresses how bio-inspired algorithms should be built. The motivation behind the work presented in this thesis is to investigate this issue within the context of a case-study that develops an algorithm inspired by the human immune system Computing and Biology Approaches that combine elements of computing and biology typically fall into one of two categories: the use of biological metaphors to influence the design of software and hardware in computer systems (biologically inspired computing); 23

24 1.1 Motivation and the use of computing tools and methods to model and simulate biological systems (computational biology). [Paton 1991] examines the distinction between biologically (bio) inspired computing and computational biology based on subject and source: the subject is the thing that is modelled; and the source provides ideas for the construction of the model. Under this classification, bio-inspired computing is concerned with computational subjects using biological sources, whereas computational biology focuses on biological subjects using computational sources. This distinction neatly reveals the different motivations behind the research, where computational biology methods aim to understand the biological system being modelled, whilst bio-inspired approaches aim to capture problem solving strategies observed in nature for use in engineering systems. This thesis concerns elements of both these motivations. Within the natural world a vast array of strategies exist to solve problems and facilitate life. These include: evolution via natural selection exerted on all biological organisms; co-operation amongst organisms of the same and different species, and the subsequent formation of societies; neural systems that control organism behaviour; the specialisation of cells within multi-cellular organisms; immune systems that fight against harmful bacteria and viruses; and the mechanisms of hereditary and their encoding in genetic material such as DNA and RNA. The inspiration this biology provides to the computer engineer has resulted in a broad range of bio-inspired computing paradigms and techniques, which have been applied to a vast range of computational problems. Examples of these techniques include: evolutionary computation and genetic algorithms [Mitchell 1998, Fogel 1998]; artificial neural networks [Gurney 1997]; artificial life models [Adami 1998]; swarm intelligence [Kennedy & Eberhart 2001]; DNA computing [Amos 2005]; and artificial immune systems [de Castro & Timmis 2002b]. Each area of bio-inspired computing has found varying levels of success and application to a wide variety of problems. Arguably one of the most successful of these is the genetic algorithm (GA) developed by [Holland 1975]. Interestingly, the GA was developed as a result of a wider study into adaptability in natural and artificial systems. It was from the theoretical models of adaptability built during this study, that GAs were developed. However, despite the principled beginning of the GA and its success as a bio-inspired approach, there has been very little subsequent work addressing the general question of what kind of process or methodology should be adopted when trying to develop a new algorithm inspired by a biological system. One attempt to tackle this is presented by [Stepney et al. 2004], who argue that bio-inspired algorithms are best developed within 24

25 1.1 Motivation a conceptual framework that incorporates the modelling (both mathematical and computational) and analysis of the biological system to aid understanding of that system before any algorithm is implemented An Approach to Bio-inspired Algorithm Design Th conceptual framework approach (CFA) of [Stepney et al. 2004, Stepney et al. 2005] is a response to the observation that the majority of bio-inspired algorithms reported in the literature proceed directly from (often naive) biological observations to algorithm implementation. By following a more principled approach to algorithm design, it is hoped that any resulting algorithms will better capture the biological properties from the system from which they take inspiration, and reduce any unwanted biases that might be introduced during algorithm implementation. Figure 1.1 presents the stages of the CFA as suggested by [Stepney et al. 2004]. The CFA begins with the biological system that is providing the inspiration for the bio-inspired algorithm. Once this system is identified, the algorithm development proceeds through four connected stages: Probes : Constructed from observations and experiments, these probes provide a view of the biological system, which is most likely to be partial and noisy. Simplifying abstract representations : These are models of the biological system built from the view provided by the probes. Once built these models should be validated. Analytical computational frameworks : Built from the insight gained from the Figure 1.1: The original conceptual framework for bio-inspired algorithm design from [Stepney et al. 2004] 25

26 1.1 Motivation simplifying abstract representations, these frameworks provide principles for designing and analysing bio-inspired algorithms. Again these should be validated via mathematical analysis, benchmark problems and engineering demonstrations. Bio-inspired algorithms : Specialised algorithms applicable to solving problems from a range of domains can be instantiated from the computational frameworks and design principles. These algorithms may contain as much biological realism as appropriate. Both the simplifying abstract representations and analytical computational frameworks stages are exercises in modelling. It is interesting to note that the CFA seems to suggest that the development of bio-inspired computing approaches would benefit from incorporating ideas from the other area where biology and computing combine identified in section 1.1.1: the use of computational biology in the form of biological model creation and analysis. Since the publication of the CFA, a number of AIS researchers including [Newborough & Stepney 2005, Twycross & Aickelin 2005, Guzella et al. 2007b, Owens et al. 2007] have taken inspiration from these ideas. However, what is currently lacking in the literature is a full instantiation of the CFA that follows the process right from identifying biology from which to take inspiration, through the construction and analysis of models of the biology, to the development of a bioinspired algorithm. If support is to be given to the claim that following the CFA will benefit the development of bio-inspired computing, then the ideas of the CFA need to be used and analysed in practice Artificial Immune Systems The CFA ideas presented by [Stepney et al. 2004, Stepney et al. 2005], are investigated in the context of artificial immune systems (AIS). AIS combine elements of immunology with the engineering sciences (both computational and mathematical approaches). Previously, bio-inspired computing and computational biology were highlighted as being two distinct areas which have fused elements of biology and computing. According to current opinion in the AIS community [Timmis et al. 2008a], immune inspired examples of both these areas are classified as AIS. From the bio-inspired computing point of view, [de Castro & Timmis 2002b] define AIS as: adaptive systems, inspired by theoretical immunology and observed immune functions, principles and models, which are applied to problem solving. 26

27 1.2 Thesis Details [Cohen 2007] goes further in the classification of AIS to identify three schools of scientist. The first two fit into the bio-inspired computing category and are distinguished by how the biological inspiration is applied: the literal school builds in silico systems to do what the real immune system does such as protect computers from viruses; the metaphorical school looks for inspiration from the immune system to build computational systems but do not try and mimic what it does in practice. The third type fits within the computational biology category incorporating those who aim to understand immunity through the development of computational and mathematical models. It is clear, therefore, that the field of AIS offers a rich and varied source of research, and one that encapsulates examples of work typical to most research areas at the interface between computing and biology. The conceptual framework inspired approach we follow in this thesis takes elements from each of these areas in the quest to develop bio-inspired algorithms. AIS is a relatively small area of bio-inspired computing compared to other bioinspired approaches such as genetic algorithms and artificial neural networks. AIS are typically applied to many of the same applications other bio-inspired approaches are applied to such as learning, anomaly detection and optimisation. The vast majority of these AIS have been inspired by three key immune ideas: negative selection, clonal selection and immune networks. The immune system, however, is a immensely rich system comprising far more than these three mechanisms. This has been recognised by various researchers such as [Hart & Timmis 2005] who suggest future AIS should possess properties far richer than those already in AIS today. 1.2 Thesis Details We describe here various details concerning the content and structure of the thesis. First, we present the goal and justification for our work. Next, we present a content overview of the thesis, summarising each stage of the work and how it fits with our thesis goal. We then identify our main research questions and contributions, and finally we highlight the papers that have been published whilst working towards this thesis, noting where these influence the content of the thesis chapters. 27

28 1.2 Thesis Details Goal, Themes and Justification The core of this thesis aims to investigate the idea that the CFA of [Stepney et al. 2004] can be used in practice to develop a bio-inspired algorithm. Our goal, therefore, is to: Develop an artificial immune system based on novel immunology by following the key ideas of the conceptual framework approach of [Stepney et al. 2004]. As noted in section there is currently no example in the literature of a full instantiation of the CFA that follows the process right from the biological system of inspiration to the development of a bio-inspired algorithm. To address this, we will take a case-study based approach and develop a new AIS starting with the inspiring immunology, through the modelling stages to a specific algorithm applied to an engineering problem. There are, therefore, two main themes running through the thesis: an investigation of the CFA, and an investigation into immunology not previously used for AIS inspiration. The justification for following a more principled approach to bio-inspired algorithm development such as the CFA is one of both algorithm performance and structure. With regard to performance, we highlighted in section that most bio-inspired approaches are not based on a process of understanding the biological property via modelling and simulation before the algorithm was developed, but that most still perform well. However, arguably one of the most successful bio-inspired algorithms, GAs, was originally developed within a principled investigation of the underlying process of adaptation, which has arguably been to the benefit of GAs, producing a powerful and popular problem solving approach. With regard to the structure of bio-inspired algorithms, a more principled approach aims to decrease the chance of failing to properly investigate the properties from which inspiration is being taken, and thus missing key aspects and understandings of those properties. In addition, an approach such as the CFA can provide an explicit traceable route from the biology to the final algorithm. By appreciating this route and having a deeper understanding of the properties present in the bio-inspired algorithm, it should be possible to highlight more easily which aspects of the algorithm can be improved and how this can be done. As well as believing that bio-inspired algorithms will benefit from following a more principled approach to their construction (a claim we explore in this thesis), [Stepney et al. 2004] also state there should be a positive feedback to biology from the construction of biological models and the insight these bring. Whilst this may 28

29 1.2 Thesis Details be the case, it is not within the scope of this thesis to investigate this issue within the technical work presented. We have chosen the area of AIS for our case-study in developing a bio-inspired algorithm using the CFA for a number of reasons to which we alluded in section These reasons can be summarised as follows: The work describing the CFA in [Stepney et al. 2004, Stepney et al. 2005] is discussed within the context of AIS, therefore AIS seems a sensible area in which to investigate the CFA further. Immunology is an immensely rich and complicated system compared to the relatively simple ideas behind evolution via natural selection that inspires algorithms such as GAs. A principled approach might help to explore the richness of immunology in a manageable way. There is recognition within the AIS community that current AIS generally do not offer anything over other bio-inspired approaches [Hart & Timmis 2005, Garrett 2005]. There is scope, therefore, within AIS to develop novel algorithms, and the CFA might assist us in doing this. The area of AIS combines examples of both bio-inspired algorithms and modelling/simulation of immune mechanisms, thus encompasses both the modelling and algorithm development aspects required for the CFA. One aspect of bio-inspired algorithm design the CFA does not address is what biology might be applicable to take inspiration from, and how such biology can be identified. We investigate this issue and explore the vast array of immunology to identify properties that have not previously inspired AIS. This is not the approach most often taken when developing bio-inspired algorithms where there is a specific, or class of, application/s in mind. We are, therefore, taking an approach to AIS development based on biological properties rather than the more typical application based approach. In doing this, we remove any bias from our choice of immunology due to a specific application, instead focussing on the properties as they appear in the immune system. The knowledge gained from the exploration of a generic immune property in the modelling stages of the CFA should highlight suitable engineering applications to which they can be applied Content Overview We present here an outline of the thesis, highlighting how the work presented addresses our goal of developing an AIS based on alternative immune concepts 29

30 1.2 Thesis Details by following the ideas of the CFA of [Stepney et al. 2004]. The thesis tells the story of our development of an AIS, presenting a chronological investigation of various alternative immune ideas for AIS inspiration, the development of models and simulations, and finally the construction of an algorithm framework and a specific AIS algorithm. Naturally, therefore, the thesis structure follows closely the stages of the CFA. Our first task is to cover the background of our chosen bio-inspired area, AIS, in chapter 2. In order to appreciate the field of AIS, we first provide a background section on the immune system. This covers the major actors (organs, cells and molecules) of immune system plus the main immune theories that have inspired AIS to date. We then discuss the structure of applied AIS (those immune inspired systems that have been applied to engineering problems) and the main algorithms they employ. The chapter on AIS is concluded with a critical discussion on the current opinion on AIS. In chapter 3 we expand on the main topic that runs through the thesis, the CFA of [Stepney et al. 2004] for the development of bio-inspired algorithms. First we present a detailed overview of the CFA, followed by an investigation of how the CFA has influenced AIS to date. The CFA suggests that AIS should be developed via a process of modelling to understand the underlying biological concepts before proceeding to specific algorithm implementations. We therefore identify a number of approaches that have been used to model and simulate immune mechanisms, and thus would be suitable tools when following the CFA. We conclude chapter 3 by outlining how we are going to apply and adapt the ideas of the CFA to develop an AIS within the limitations of a PhD. This results in the conceptual framework for AIS (CFAIS). Before we can follow our identified method for developing an AIS, we need to address the issue of identifying suitable immunology from which to take inspiration. Chapter 4 begins by examining the immunology that has previously been used to inspire AIS and compares this to some current strands of thinking by immunologists, showing an apparent discord between many immunologists regarding some of the major conceptual ideas of immunology. We assess the impact this discord has on AIS, and proceed to investigate the alternative immune model of [Cohen 2000b], which presents a complex systems view of the immune system. Inherent within this model are concepts that can inspire AIS and we focus further on the concepts of immune receptor degeneracy (receptors can bind many different targets) and the notion of patterns of response to provide immune recognition. In the context of the CFA summarised in figure 1.1, the work presented in chapter 4 aims to identify exactly what the biological system we are 30

31 1.2 Thesis Details looking at is, and to start the process of probing that system to provide sufficient detail to move forward to the construction of a model and simulation (simplifying abstract representation). Chapter 5 details our first attempt at elaborating and investigating the ideas of immune detector degeneracy and their patterns of response, by developing a model and simulation inspired by immune cell activation in a lymph node. In order to do this, we need to return to immunology to identify the suitable biological details necessary for the model and simulation construction. The work detailed in chapter 5, therefore, encompasses both the probes and simplifying abstract representation stages of the CFA. The primary motivation for the lymph node simulation is to investigate whether we see patterns of detector response to an antigenic stimulus and what these patterns look like, based on the assumption that the detectors are degenerate. A lymph node is chosen as our inspiration as it is the site of the initial recognition event of the adaptive immune system, thus a likely place where patterns of response might emerge. The simulation we construct shows how sets of randomly generated degenerate immune detectors react to different antigenic stimuli, and how for similar stimuli, similar patterns of response emerge from a set of detectors. These responses, however, are heavily dependent on the static recognition thresholds of the detectors. It is not apparent from the simulation that we can move to the next stage of the CFA as it is unclear how patterns of response could be used effectively as part of an AIS algorithm. Therefore, we decide to look further at the issue of choosing recognition thresholds in chapter 6. In the discussion in chapter 4 on alternative immune properties, we touch on the adaptable lymphocyte theory of [Grossman & Paul 2000]. Further investigations of this theory in chapter 6 identify the tunable activation thresholds (TAT) model [Grossman & Paul 1992, Grossman 1993] whereby recognition thresholds are tuned to cell excitation levels over a period of time. The lymph node simulation in chapter 5 identifies the sensitivity of detector recognition thresholds, so we proceed to investigate the TAT model within the context of degeneracy and patterns of response. This investigation performs an empirical assessment of the TAT behaviour using a simple simulation. This enables us to show that we can achieve the TAT behaviours described by [Grossman & Paul 1992], as well as showing the suitability to a potential AIS. We then investigate the behaviour of sets of degenerate tunable detectors that are randomly generated and show how these can be combined to produce a single pattern of response for the detector population upon an antigenic stimulus. Significantly, we show that similar stimuli produce similar patterns of response from the same sets of detectors. The work 31

32 1.2 Thesis Details presented in chapter 6 is essentially a repeat of the stages of the CFA employed in the lymph node simulation of chapter 5, namely probes and a simplifying abstract representation in the form of our simulation. However, the work is informed by the outcome of the lymph node simulation of chapter 5, and importantly enables us to move on to the next stage of the CFA. Chapter 7 details our application of the final two stages of the CFA: an analytical framework with algorithm design principles, and the instantiation of the framework producing an AIS (bio-inspired algorithm). Based on the TAT model investigated in chapter 6, we identify that patterns of response can be used in an engineering context as a projection or mapping of input data onto the space of randomly generated detectors. This leads us to present an AIS algorithm framework in which the generation of patterns of response is employed as a data preprocessing stage. This pre-processing stage takes input data vectors and generates a pattern of response from a set of detectors, processes these patterns and feeds the resulting data into an application specific algorithm. The investigations of the patterns of response in chapter 6 also enable us to present guidelines on how to use the parameters of the TAT model to generate the patterns. To complete our pass through the CFA, chapter 7 instantiates our framework to develop an AIS for pattern recognition. This AIS uses the patterns of response to pre-process data vectors, which are then classified by a k-nearest neighbour algorithm. The AIS is tested using three data sets typically used to benchmark pattern recognition approaches, and we examine the performance of the algorithm using various parameter settings. To conclude the work of the thesis, chapter 8 reflects on the experience of developing an AIS by following the ideas laid out by the CFA. We begin by summarising the work presented in chapters 4, 5, 6 and 7 with respect our AIS design approach identified in chapter 3, the CFAIS. We then take a detailed look at each of the stages of the CFAIS analysing how we have approached the task of identifying biology for inspiration, how we have probed that biology, the role played by modelling, simulation, frameworks and algorithms, and finally looking at the CFAIS as a whole. Next, we elaborate on the drawback of not taking an interdisciplinary approach to the CFAIS, commenting on the role played by domain experts, and how we could achieve feedback to biology from our work. We then comment on different ways the CFA could be modified. Finally, we conclude the thesis by summarising our application of the CFAIS, identifying future work, and returning to assess how we have addressed our research questions laid out in the next section. 32

33 1.2 Thesis Details Research Questions and Contribution Based on the descriptions of our research above, there are two main research questions that we will be addressing during the course of the thesis: 1. Is the CFA a suitable method for developing a bio-inspired algorithm such as an AIS? What are the pros and cons of the CFA? Are there ways in which it can be improved? 2. How can the concepts of degeneracy and patterns of response be incorporated into an AIS? What are the implications of degeneracy and patterns of response to an AIS? Is there a suitable type of application for such an AIS? The first of these research questions is assessed throughout the technical work presented in the thesis, with the main discussion points arising in chapters 3, 4 and 8. The second research question is mainly explored in chapters 5, 6 and 7. In addressing the research questions, we make the following contributions to the areas of AIS and the development of bio-inspired algorithm design: Present the first reported work that follows a complete pass of the CFA from the biology through the modelling stages to an AIS. Outline a methodology inspired by the CFA for developing AIS in the context of a PhD. Investigate alternative immunology that has not previously been used to inspire AIS and suggest that competing theories can all inspire AIS. Develop an agent-based simulation tool for investigating degenerate detectors and their reactions to different antigenic stimulus. Investigate the idea that lymphocytes can tune their activation thresholds and what this means in the context of AIS. Extract an algorithm framework that incorporates tunable degenerate detectors into AIS. Develop a simple pattern classification algorithm based on our AIS algorithm framework. 33

34 1.2 Thesis Details Examine issues surrounding the design of bio-inspired algorithms including why we should follow a more principle approach, the role of modelling and domain expertise, and the types of biology suitable for algorithm inspiration. Suggest how the CFA can be adapted for the benefit of bio-inspired algorithm development Publications A number of publications have resulted from working towards this thesis and form the basis of much of the work presented within. These publications, along with how they relate to this thesis, are: [Andrews & Timmis 2005a] : I am the principal author on this conference paper that influenced many of the ideas presented in chapter 4. [Andrews & Timmis 2007] : I am the principal author on this book chapter that extends the ideas presented in the previous publication [Andrews & Timmis 2005a]. Much of chapter 4 is reproduced from this work. [Andrews & Timmis 2006] : I am the principal author on this conference paper, most of which is reproduced in chapter 5. [Andrews & Timmis 2008] : I am the principal author on this conference paper, most of which is reproduced in chapter 6. [Timmis et al. 2008a] : My contributions to this journal article have formed the basis for parts of chapters 2 and 3. In addition to these, the following have been published that are relevant to this thesis, but they do not form a core part of the work presented: [Andrews & Timmis 2005b, Timmis & Andrews 2007, Read et al. 2008]. 34

35 CHAPTER TWO Applied Artificial Immune Systems We have chosen artificial immune systems (AIS) as the bio-inspired subject area for our exploration of the conceptual framework approach (CFA) of [Stepney et al. 2004]. As part of our goal is to develop an AIS to solve an engineering problem (an applied AIS), we explore the background on applied AIS in this chapter. First, however, it is necessary to provide a background on the immune system in section 2.1, covering the main actors and immune theories that have inspired AIS to date. Following this in section 2.2, we review current AIS, discussing their structure and the main algorithms they employ. We then critique current opinion on applied AIS in section 2.3, focusing on structural aspects of AIS, the applications they have been applied to, how useful they have been, and the future properties AIS should employ. We conclude the chapter in section 2.4 by discussing the current state of applied AIS with respect to our work. 2.1 The Immune System The human immune system (referred to simply as the immune system from this point forth) has been studied for well over 100 years, but is still far from being completely understood. This background section covers the basic operating details of the immune system along with many of the key discoveries and theories of the immune system s function. The view presented, however, does not reflect the disquiet that exists amongst many immunologists regarding the key tenets of immunology. This subject is addressed later in section

36 2.1 The Immune System An Overview Immunology concerns the study of the immune system and the effects of its operation on the body. The immune system is normally defined in relation to its perceived function: a defence system evolved to protect its host from pathogens (harmful micro-organisms such as bacteria, viruses and parasites) [Goldsby et al. 2003]. It comprises a variety of specialised cells that circulate and monitor the body, various extra-cellular molecules, and immune organs that provide an environment for immune cells to interact, mature and respond. The collective action of immune cells and molecules forms a complex network leading to the detection and recognition of pathogens within the body. This is followed by a specific effector response aimed at eliminating the pathogen. This recognition and response process is vastly complicated with many of the details not yet properly understood. In mammals, the immune system can be classified into two components based on functionality: a less specific component termed innate immunity and a more specific component termed adaptive (or acquired) immunity. The mechanisms of innate immunity are generic defence mechanisms that are non-specific to particular examples of pathogen, but act against general classes of pathogen. They are encoded within the genes of the species, and do not adapt during the lifetime of the individual. Examples include the inflammatory response, phagocytic immune cells (those that can ingest and kill pathogens), anatomic barriers such as skin, and physiologic barriers such as temperature. By contrast, the mechanisms of adaptive immunity enable the immune system to adapt to previously unseen pathogens based upon exposure to them [Goldsby et al. 2003]. This is achieved through a learning mechanism that operates during the lifetime of the individual. Additionally, once exposed to a pathogen, memory mechanisms exist to allow the immune system to remember the pathogen. This enables a faster and more effective secondary response that can be elicited against the pathogen if it is encountered again. The adaptive and innate arms of the immune system interact to provide the body with a comprehensive defence mechanism against pathogens Organs, Cells and Molecules The organs of the immune system are distributed throughout the body (see figure 2.1) and can be classified into two main groups. The first is the central (primary) immune organs made up of the bone marrow and the thymus. These are the sites of production and maturation of immune cells. The second group is the 36

37 2.1 The Immune System Figure 2.1: The organs of the immune system and their locations with the body. Reproduced from [de Castro & Timmis 2002b] peripheral (secondary) immune organs, which include the lymph nodes, tonsils, spleen, Peyer s patches and lymph vessels. These immune organs provide transport mechanisms and sites for immune cells to interact and operate. The class of cells that belong to the immune system are the leukocytes, often referred to as the white blood cells. All blood cells are produced in the bone marrow, which consists of the soft tissue that resides inside most elongated bones. The leukocytes can be divided into two distinct groups, those responsible for adaptive immunity, and those of innate immunity. The cells of adaptive immunity are known as the lymphocytes, and include the B and T cells. These cells are named after their place of maturation: the bone marrow and thymus. The thymus is an organ situated near the heart, which provides an environment where T cells are selected to provide the appropriate immune reactivity. This process will be explored in more detail later in section The innate immune cells can be classified into two main groups, the monocytes and granulocytes. These can be subdivided further into different cell types 37

38 2.1 The Immune System based on their action, for example, the monocytes include macrophages and dendritic cells. The precise detail of this classification is not important to the current discussion, however, the general behaviours of the innate immune cells are. Both the monocytes and some granulocytes are phagocytic, thus can ingest and kill some pathogens such as bacteria. Additionally, the monocytes act as antigen presenting cells (APCs), whereby portions of the ingested pathogen are displayed on their cell membrane as antigens to be recognised by T cells [Goldsby et al. 2003]. To produce the required defence from pathogens, the immune cells circulate through the body s tissues and organs via the blood and lymph vessels. Lymph is the term given to the fluid derived from the tissues of the body and contains a mixture of immune cells, other bodily cells and bacteria. This lymph drains through the lymph vessels and collects in the lymph nodes. The lymph nodes are small solid structures (about the size of a pea) located at various positions around the body. They provide an environment for immune cells to interact with each other and with pathogens, and is therefore the site where adaptive immune responses occur [de Castro & Timmis 2002b]. All immune cells, and the majority of other cells of the body, possess protein molecules on their surface that act as receptors to other extra-cellular molecules. When a sufficiently strong chemical bond occurs between a receptor and another molecule (a ligand), a cascade of intra-cellular signals is initiated, the outcome of which depends on the initiating receptors. This process provides a mechanism for recognition at the molecular level. There exists two types of immune cell receptor: innate receptors that have evolved to recognise specific molecules; and the unique receptors of lymphocytes that are generated during the life time of the individual to recognise previously unseen molecules. The later of these molecules are generically known as antigens, a term given to any molecular structure that can chemically bind to the unique receptors of T and B cells. The antigen receptors of the B cell are called antibodies, and those of the T cell are called T cell receptors (TCR). They are both generated via a stochastic process (elaborated in section 2.1.3), and are vital to the body s adaptive immune response. Immune molecules are defined as those molecules that stimulate, or are used by, immune cells for the purposes of the immune system [Cohen 2000b]. They include cytokines, immune cell receptors, antibodies, enzymes, plasma proteins and adhesion molecules. The cytokines, for example, are signalling molecules secreted by both immune and other bodily cells, which are then detected via specific cellular receptors. Cytokines work via either an autocrine or paracrine action, whereby they are taken up by the cell that secreted them (autocrine) or a nearby cell (paracrine). Many different types of cytokine exist and their effects in- 38

39 2.1 The Immune System clude the activation, differentiation, growth, movement and death of many types of cell [Cohen 2000b] Lymphocytes As previously noted, the lymphocytes include the B and T cell populations. The main job of B cells in adaptive immunity is the production of antibodies. These antibodies can either be attached to the surface of the B cell and act as a receptor for antigen recognition, or be secreted from the cell to bind to antigens that are free in solution. This marks the antigen for deletion by phagocytic cells. An individual B cell only produces one type of antibody, one that is unique for that B cell, and produced in large quantities. A B cell that has not yet encountered any antigen that binds its antibodies is called a naive B cell. Once antigen binding occurs, the naive B cell is activated, initiating a process of cellular proliferation and differentiation into effector B cells (see section 2.1.4). This results in a population of clones of the original B cell being produced, which are responsible for secreting large numbers of antibodies. Some B cells also differentiate into long lived memory cells, providing protection from future infection by the same pathogen whose antigen activated it [Goldsby et al. 2003]. The specific portions of an antigen to which lymphocytes receptors bind are called epitopes. The specificity of a lymphocyte receptor is, therefore, defined by its ability to bind with a single unique epitope. The strength of this binding is measured in terms of an affinity, whereby a high affinity between receptor and epitope results from a tight molecular binding [Goldsby et al. 2003]. Figure 2.2 shows the structure of an antibody: a protein dimer, formed out of two identical molecular chains, producing a symmetrical Y-shaped molecule. These identical chains are themselves made out of two different molecular chains, the light chain and the heavy chain. Each of these consists of a constant (C) region and a variable (V) region. The C region is similar for all antibodies, whereas the V region is generally unique to each B cell. This V region is the primary site where binding of antigen occurs, and hence determines the specificity of the antibody molecule. The genes encoding the structure of V regions are contained within sets of gene segments called gene libraries, each encoding a different part of the V region. These libraries contain many different variations of the same gene segment. As the B cell matures, a randomly selected individual from each gene library comes together to form a DNA strand encoding the entire V region. There are millions of different possible V regions that can be produced this way, thus each antibody will generally have a unique V region on each of the light 39

40 2.1 The Immune System Figure 2.2: The structure of an antibody showing the variable regions. Reproduced from [de Castro & Timmis 2002b] and heavy chains. Once the B cell has produced a functional receptor, the process of gene re-arrangement stops and so each B cell can only produce one type of antibody [de Castro & Timmis 2002b, Janeway et al. 2001]. Like B cells, T cells each express unique antigen receptors. These TCRs are structurally different from the antibody, being constructed from just two different molecular chains. Each chain, however, has a C and V region, which is generated via the same mechanism as for antibodies. TCRs only recognise epitopes that are bound to a receptor called major histocompatability complex (MHC), which is expressed on the surface of nearly all cells in the body. The epitopes that are presented by cells on their MHC receptors are examples of the proteins found within the cell. MHC exists in two varieties: MHC-I and MHC-II. MHC-I is expressed by nearly all bodily cells, whereas MHC-II is only expressed by APCs. T cells are classified into two sub-populations dependent on the class of MHC to which they bind. T helper (T H ) cells interact with epitopes bound to MHC-II, whereas T cytotoxic (T C ) cells interact with antigens bound to MHC-I. This distinction is important as it means that T H cells will encounter pathogenic epitopes which have been ingested by APCs (such as dendritic cells) then presented by their MHC- 40

41 2.1 The Immune System II receptors. These T H cells go on to activate other cells such as B and T C cells. When activated, a T C cell exerts cytotoxic activity against any cell expressing the epitope to which it binds, resulting in the death of that cell. Typically, these are virus infected cells and tumours [Goldsby et al. 2003]. T cells follow the same cellular activation, proliferation and differentiation mechanisms described above for B cells, although with no affinity maturation of their antigen receptors Clonal Selection Theory and Self Non-self It was highlighted above that the antigen receptors of lymphocytes differ. The body contains millions of different lymphocytes, thus millions of different receptors with different specificities are expressed, the combination of which is known as the receptor repertoire of the individual. In order to provide an adaptive immune response, this repertoire undergoes a selection mechanism during the lifetime of the individual. This selection mechanism, called the clonal selection theory, was formulated by [Burnet 1959] to explain how an adequate number of lymphocytes capable of recognising a particular pathogen can be produced to combat it. The theory was formulated when very little was known about the function of lymphocytes, and nothing was known about the structure of their receptors, however it has since been shown to operate on both B and T cell populations. It has also become the central tenet of adaptive immunity, so much so that immunology became know as the science of self non-self discrimination [Tauber 2000]. The theory states that on a sufficient antigen bind, activation of lymphocytes occurs. Once activated, clones of the lymphocyte are produced expressing identical receptors to the original lymphocyte that encountered the antigen. Thus a clonal expansion of the original lymphocyte occurs. The four basic principles of clonal selection, as quoted verbatim from [Janeway et al. 2001], are: 1. Each lymphocyte bears a single type of receptor with a unique specificity 2. Interaction between a foreign molecule and a lymphocyte capable of binding that molecule with a high affinity leads to lymphocyte activation 3. The differentiated effector cells derived from an activated lymphocyte will bear receptors of identical specificity to those of the parental cell from which that lymphocyte was derived 4. Lymphocytes bearing receptors specific to ubiquitous self molecules are deleted at an early stage in lymphoid cell development and are therefore absent from the repertoire of mature lymphocytes 41

42 2.1 The Immune System The last of these principles is known as clonal deletion. It prevents randomly generated lymphocyte receptors from recognising the body s own tissue molecules and evoking an immune response against them [Janeway et al. 2001]. Figure 2.3 summarises the clonal selection theory. With the formulation of the clonal selection theory and its requirement for clonal deletion, [Burnet 1959] introduced into immunology the explicit notion of molecular material being either part of the self or not [Tauber 2000]. Consequently, one of the major tasks of the immune system became the classification of antigens as either self or non-self. To achieve this distinction required the absence of any self-reacting lymphocytes from the available repertoire, so any antigen reacting with a lymphocyte receptor must implicitly be non-self and should be removed. Although purely speculation at the time, clonal deletion was subsequently discovered to exist in the body via a mechanism called negative selection that operates on lymphocytes during their maturation. For T cells this occurs in the thymus, which provides an environment rich in APCs presenting self-antigens. Immature T cells that strongly bind these self-antigens undergo a controlled death (apoptosis). Thus, the T cells that survive this process should be unreactive to self-antigens. The property of lymphocytes not to react to the self is Figure 2.3: The clonal selection theory showing negative selection, proliferation and differentiation. Reproduced from [de Castro & Timmis 2002b] 42

43 2.1 The Immune System called immunological tolerance [de Castro & Timmis 2002b]. Since the original clonal selection theory was proposed, a number of modifications have been made to incorporate more recent experimental observations. One such observation revealed that during the clonal expansion of B cells (but not T cells), the average antibody affinity increased for the antigen that triggered the clonal expansion. This phenomenon is called affinity maturation. This results in the immune response being more effective upon a subsequent exposure to the antigen, owing to the higher affinity of the antibody for the antigen. Affinity maturation is caused by a somatic hypermutation and selection mechanism that occurs during the clonal expansion of B cells. Somatic hypermutation alters the specificity of antibodies by introducing random changes to the genes that encode their V regions. This hypermutation mechanism is believed to be proportional to the affinity of the antigen-antibody binding, so the higher the antibody affinity the less mutations it suffers. After the mutations have occurred, the B cells that produce higher affinity antibodies are preferentially selected to differentiate into effector and memory cells, so over the course of an immune response, the average population affinity of antibodies increases [Goldsby et al. 2003, de Castro & Timmis 2002b] Immune Network Theory [Jerne 1974] proposed an immune network theory to help explain a number of the observed emergent properties of the immune system, such as learning and memory. In general immune network theories view the immune system as a regulated network of molecules and cells that recognise each other producing a self-organising behaviour even in the absence of antigen. The original network of [Jerne 1974] was based on antibody interactions and states that any antibody epitope could be bound and recognised by a subset of the total antibody repertoire. Individuals of this recognising set have their own set of recognising antibodies etc, forming a unique network of interactions. Antibody epitopes that recognise other antibody epitopes are called paratopes and the epitopes capable of being recognised are called idiotopes. The set of idiotopes characterised by an antibody is called the antibody s idiotype, and thus these networks are often referred to as idiotypic networks. In an idiotypic network, the activation or suppression of an immune response can result from an antibody recognising another antibody or other antigen. These networks express a self-organising behaviour without the need for stimulation by non-self antigens. It is the perturbations to the network (for example by a 43

44 2.1 The Immune System pathogenic antigen) that causes an immune response to occur. As a consequence of their dynamic behaviour, immune networks explain the mechanisms of learning and memory from the viewpoint of interacting populations of immune agents rather than the property of single agents [de Castro & Timmis 2002b]. The biological evidence for immune networks is not strong compared with that for clonal selection theory, and consequently they have remained on the periphery of immunological thinking. However, a number of authors such as [Farmer et al. 1986] and [Varela & Coutinho 1991] have expanded on the original immune network ideas Danger Theory [Matzinger 2002] explains how the original clonal selection theory placed the antigen-specific cells of adaptive immunity (most notably the T H cell) at the heart of the decision of whether or not to initiate an immune response. This decision was achieved through the deletion of the self-reacting lymphocytes, so that responses would only be initiated against non-self. It was discovered, however, that T H cells themselves require a co-stimulatory signal from non-antigen-specific APCs in order to initiate an effective adaptive immune response. As a consequence, it could not be assured that immunity would only be directed against non-self, as APCs express on their surfaces both self and non-self antigens. To address this, [Janeway 1992] proposed the infectious non-self model that suggested APCs could discriminate between self and non-self by detecting, using germline encoded receptors, evolutionarily conserved pathogen-associated molecular patterns (PAMPs) unique to bacteria. As an alternative explanation, [Matzinger 1994] proposed the danger theory, which has gained much popularity amongst immunologists in recent years. The danger theory states that APCs, such as dendritic cells, are themselves activated via an alarm: danger signals. These activated APCs will then be able to provide the necessary co-stimulatory signal to the T H cells that subsequently control the adaptive immune response. The danger signals are emitted by ordinary cells of the body that have been injured due to attack by pathogen. For example, the intra-cellular contents released due to uncontrolled (necrotic) cell death could provide such signals. Dendritic cells detect these danger signals along with other safe signals whilst collecting antigen in the peripheral parts of the body. They are able to integrate the signals to make a decision as to the safety of their environment. If a dangerous environment is detected, the dendritic cell can stimulate T H cells and present the antigen they have collected in that dangerous environment, thus 44

45 2.2 Artificial Immune Systems for Engineering initiating the adaptive immune response [Mosmann & Livingstone 2004]. The danger theory presents a number of important consequences for the field of immunology. Firstly, with the danger signals arising from normal cells of the body, an immune response is no-longer initiated by the specialised cells of the immune system. Secondly, the adaptive immune response is itself controlled by the action of innate immune cells, thus blurring the distinction between the adaptive and innate arms of the immune system. Lastly, the notion of self non-self discrimination is replaced with a danger non-danger metaphor, whereby foreign non-self no longer necessarily initiates an immune response. 2.2 Artificial Immune Systems for Engineering The immune system has been a rich source of inspiration for engineering systems as it is observed to exhibit properties such as robustness, adaptability, learning, memory, recognition, feature extraction, diversity, scalability and multiple interactions on a variety of timescales [Timmis et al. 2006, Dasgupta 1999]. Whilst many different AIS have been developed and applied to problem solving over the past 20 years, a layered framework for engineering AIS by [de Castro & Timmis 2002b] demonstrates the general structure of most applied AIS. This framework is used in this section as a template for their summary, and is shown in figure 2.4. It takes as its starting point the application domain for the AIS, followed by three design layers to be considered before the required AIS is engineered. The design layers are as follows: Component Representations : how the components of the system are to be represented Affinity Measures : how the interactions between the components of the system are to be quantified Immune Algorithms : how the components of the systems are going to interact to determine the system dynamics The representations and affinity measures used in AIS tend to be applicable to any type of immune algorithm, and are discussed together in section The type of immune algorithm used in an AIS is normally used as a system of AIS classification. These algorithms are typically inspired by the immune theories covered in section 2.1, and fall into one of four groups: negative selection, clonal selection, immune networks and danger theory algorithms. Each of these will be discussed in sections 2.2.2, 2.2.3, and below. 45

46 2.2 Artificial Immune Systems for Engineering Figure 2.4: A layered framework for AIS. Reproduced from [de Castro & Timmis 2002b] Component Representations and Affinity Measures The most influential concept to affect the representation of components in AIS is shape-space introduced by [Perelson & Oster 1979]. They theoretically investigate how large an antibody repertoire must be in order to provide effective defence, by viewing the immune system as a molecular recognition device designed to identify foreign shapes. They define the antigen combining region of an antibody in terms of N shape parameters. These parameters are not explicitly identified, but suggestions include geometric quantities specifying molecular sizes and shapes, or molecular charges. An antigen that binds the antibody combining region can be described by the same N parameters. By combining these parameters into a vector, both the antibody and antigen can be represented as points Ab and Ag respectively in an N-dimensional vector space called shape-space, S. By ignoring the complementary structure of the antibody and antigen bind, a perfect bind occurs when Ab = Ag. The antibody-antigen complementarity can, therefore, be measured as the distance between Ab and Ag using a suitable metric in S. If all of the N parameters contribute equally to the antibody specificity, then a metric such as Euclidean distance can be used. A volume, V, can then be defined in S as the volume in which the possible repertoire of antibodies and antigens can fall. For each individual antibody, a small region V ε can also be defined, where ε is a distance threshold in S. Any antigen that falls within the V ε of an antibody is said to be bound by that antibody. Figure 2.5 shows how a 2-dimensional space-space scheme would work. 46

47 2.2 Artificial Immune Systems for Engineering Figure 2.5: A 2-dimensional depiction of shape-space. Reproduced from [de Castro & Timmis 2002b] [Perelson & Oster 1979] note that a strict binding threshold between components could be replaced with a probability that decreases with distance from the antibody. They also note that previous work by [Edelstein & Rosen 1978] shows that the shape of an antibody could be defined by a continuous function instead of a point vector. Even though the work of [Perelson & Oster 1979] used antibodies and antigens, [de Castro & Timmis 2002b] point out that the shape-space representation can be applied to any type of receptor and molecule that binds it. The vast majority of AIS that use immune receptor and antigen components employ the notion of shape-space. AIS components will, therefore, typically be represented as attribute strings of N parameters, for example an antibody component would be described by the vector Ab =< Ab 1, Ab 2,..., Ab N >. The choice of vector attributes determines the shape-space type of which there are three main types identified by [de Castro & Timmis 2002b]: Real-valued : attributes of all the components are real numbers Hamming : attributes of all the components are from a finite alphabet Symbolic : attributes can be of any type, including symbols such as age and name. The choice of component attribute type naturally determines the metric in S that can be used to measure the similarity (affinity) between AIS components. It is 47

48 2.2 Artificial Immune Systems for Engineering typical in AIS that each of the N component attributes contributes equally to the affinity measure. For Real-valued shape-spaces Euclidean distance is normally used as the affinity measure. Given two Real-valued vectors, A and B, of length N, the Euclidean distance between them is calculated by: EuclideanDistance = N (A i B i ) 2 (2.1) Other affinity measures for Real-valued shape-spaces such as Manhattan distance are equally applicable. The affinity measures used for the Hamming shape-spaces are determined by the alphabet used. By far the most widely used alphabet is binary. Two of the most popular affinity measures for the binary alphabet are the Hamming distance and the r-contiguous bit rule. The Hamming distance is simply calculated by applying the XOR operator to the two components that are being measured. The r-contiguous bit rule calculates the affinity as the length of the largest contiguous region between the two components. The affinity measures used for symbolic shape-spaces are very much dependent on the nature of the symbols that make up the attribute strings, and thus defining this measure can be a non-trivial task. For all affinity measures a recognition threshold (ε above) can be set to determine whether recognition between components has occurred. i= Negative Selection Algorithms Negative selection algorithms are inspired by the main mechanism in the thymus that produces a set of mature T cells capable of binding only non-self antigens. The first negative selection algorithm was proposed by [Forrest et al. 1994] and applied to computer security to detect data manipulation (non-self) caused by a virus. Other typical applications for negative selection include network security and fault detection [Dasgupta et al. 2004]. The starting point of the algorithm of [Forrest et al. 1994], which is typical of all negative selection algorithms, is to produce a set of self strings, S, that define the normal state of the system. The task then is to generate a set of detectors, D, that only bind/recognise the complement of S. These detectors can then be applied to new data in order to classify them as being self or non-self. The set of detectors, D, is generated by the process described by algorithm 1. The original negative selection algorithm used binary strings to represent the self and detector components, and the r-contiguous bit rule as the affinity measure. Negative selection algorithms have also been used with real-valued components, such as 48

49 2.2 Artificial Immune Systems for Engineering input : S = set of seen known self elements output: D = set of generated detectors begin repeat randomly generate potential detectors and place in set P determine affinity of each member in P with each member in self set S if at least one element in S recognises a detector in P according to a recognition threshold then detector is rejected else add to the set of available detectors D end until until stopping criteria has been met end Algorithm 1: The original negative selection algorithm of [Forrest et al. 1994] [Gonzalez & Dasgupta 2003]. In recent years, a lot of theoretical work has been invested in the negative selection algorithm, most notably a series of work by [Stibor et al. 2004, Stibor et al. 2005a, Stibor et al. 2005b, Stibor et al. 2006a, Stibor et al. 2006b]. These works highlight a number of limitations of negative selection algorithms, including poor performance compared to statistical anomaly detection techniques [Stibor et al. 2005b] and the difficulty of generating detectors in high dimensional spaces [Stibor et al. 2006b] Clonal Selection Algorithms Clonal selection algorithms have taken inspiration from the antigen driven affinity maturation process of B cells and the associated hypermutation mechanism. These AIS also often use the idea of memory cells to retain good solutions to the problem being solved. [de Castro & Timmis 2002b] highlight two important features of affinity maturation in B cells that can be exploited from the computational viewpoint. The first feature is that the proliferation of B cells is proportional to the affinity of the antigen that binds it, thus the higher the affinity, the more clones that are produced. Secondly, the mutations suffered by the antibody of a B cell are inversely proportional to the affinity of the antigen it binds. Applying these two features, [de Castro & Von Zuben 2002] developed an AIS called CLONALG, which has been used to performed the tasks of pattern matching and multi-modal function optimisation [de Castro & Timmis 2002a]. For the example of pattern matching, a set of patterns, S, to be matched are considered to be anti- 49

50 2.2 Artificial Immune Systems for Engineering input : S = set of patterns to be recognised, n the number of worst elements to select for removal output: M = set of memory detectors capable of classifying unseen patterns begin Create an initial random set of antibodies, A forall patterns in S do Determine the affinity with each antibody in A Generate clones of a subset of the antibodies in A with the highest affinity. The number of clones for an antibody is proportional to its affinity Mutate attributes of these clones inversely proportional to its affinity. Add these clones to the set A, and place a copy of the highest affinity antibodies in A into the memory set, M Replace the n lowest affinity antibodies in A with new randomly generated antibodies end end Algorithm 2: CLONALG for pattern matching [de Castro & Von Zuben 2002] gens. The task of CLONALG is to then produce a set of memory antibodies, M, that match the members in S. This is achieved via algorithm 2. Clonal selection algorithms share many similarities with evolutionary algorithms [Newborough & Stepney 2005], although importantly the selection and mutation mechanisms are influenced by the affinities of antibody-antigen matching [de Castro & Timmis 2002b]. Due to this similarity, many of the theoretical approaches applied to evolutionary algorithms are applicable to clonal selection algorithms also. [Timmis et al. 2008b] summarises much of the theoretical work done on AIS to date. This includes the work of [Clark et al. 2005] who develop an exact Markov chain model of the clonal selection algorithm called the B-cell algorithm (BCA) [Kelsey & Timmis 2003], proving its convergence. They go on to show how the model can be applied to give insight into optimal parameter settings for the BCA in a function optimisation landscape. Other AIS that have been inspired by the adaptive immune mechanisms of B cells are AIRS [Watkins et al. 2004], a supervised learning algorithm, and IA that has been used in numerous applications and well studied [Cutello et al. 2004, Cutello et al. 2005] Immune Network Algorithms Immune network algorithms have their basis in the continuous ordinary differential equation models used by theoretical immunologists to explore the perceived behaviour of real immune networks. Examples include the models by [Farmer et 50

51 2.2 Artificial Immune Systems for Engineering al. 1986] and [Varela & Coutinho 1991]. One of the main differences between the discretised immune network algorithms is that they interact with their environment (i.e. antigens), whereas the continuous models typically do not [de Castro & Timmis 2002b]. The main difference between immune network algorithms and other immune algorithms is that the components of the system not only interact with antigenic components, but with the other components in the AIS. Two examples of immune network algorithms are RAIN [Timmis & Neal 2001] and ainet [de Castro & Von Zuben 2000], which attempt to use the basic concepts of immune network theory to solve problems such as pattern recognition and data clustering. ainet consists of a network of antibody components that adapt to match a population of input components (antigens) to be clustered. ainet is essentially a modified input : S = set of patterns to be recognised, nt network affinity threshold, ct clonal pool threshold, h number of highest affinity clones, a number of new antibodies to introduce output: N = set of memory detectors capable of classifying unseen patterns begin Create an initial random set of network antibodies, N repeat forall patterns in S do Determine the affinity with each antibody in N Generate clones of a subset of the antibodies in N with the highest affinity. The number of clones for an antibody is proportional to its affinity Mutate attributes of these clones inversely proportional to its affinity, and place the h number of highest affinity clones into a clonal memory set, C Eliminate all members of C whose affinity with the antigen is less than a pre-defined threshold (ct) Determine the affinity amongst all the antibodies in C and eliminate those antibodies whose affinity with each other is less than a pre-specified threshold (ct) Incorporate the remaining clones in C into N end Determine the affinity between each pair of antibodies in N and eliminate all antibodies whose affinity is less than a pre-specified threshold nt Introduce a number (a) of new randomly generated antibodies into N until until a stopping condition has been met end Algorithm 3: ainet for data clustering [de Castro & Von Zuben 2000] 51

52 2.2 Artificial Immune Systems for Engineering version of CLONALG (described above) with an added mechanism of suppressive interactions between the antibody components. It works as described in algorithm 3. The resulting set of network antibodies that is generated represents an internal image of the antigens to which they have been exposed [de Castro & Timmis 2002b]. ainet has found wide use in the area of optimisation, and many adaptations have been made to the algorithm such as [Andrews & Timmis 2005b]. [Galeano et al. 2005] provides a good review of the different immune networks that appear in the literature. Relatively little theoretical work exists for immune network algorithms, although [Timmis et al. 2008b] do highlight various issues. Due to their network structure, immune networks would be open to theoretical techniques used in the study of other networks such as small-world and scale-free networks [Barabasi & Bonabeau 2003]. Additionally many of the techniques used to study the continuous theoretical immune models are also relevant Danger Theory Algorithms Compared to the immunology that has inspired negative selection, clonal selection and immune networks, danger theory is a relatively new addition to the field of immunology. Danger theory inspired algorithms are, therefore, still in their infancy. [Garrett 2005] states that these algorithms should provide an alternative to the negative selection approach, as danger theory has a number of appealing properties from a computational perspective. For example, ideas from danger theory should focus events on what is harmful instead of just non-self, reporting only the detection of something dangerous. [Secker et al. 2003] proposed to explore the relevance of danger theory to web mining by investigating the use of danger signals to provide context for searches. [Aickelin et al. 2003] examined the possibility of an intrusion detection system based on danger theory by investigating how various intrusion scenarios can be detected via the use of different computer danger signals. This has been extended by [Greensmith et al. 2005, Greensmith et al. 2006] with the dentritic cell algorithm (DCA) which is presented in algorithm 4. This introduced the notion of danger, safe and PAMP signals that contribute to the context of a data signal being presented at any given time. The context is integrated via a process inspired by the role of dendritic cells and removes the need to define what self is, although it is necessary to define what the danger, safe and PAMP signals are. The DCA is the first concrete danger inspired AIS algorithm and has been applied to anomaly detection [Greensmith et al. 2006] and behaviour classification 52

53 2.3 Current Opinion on Applied AIS input : S = set of data items to be labelled safe or dangerous output: L = set of data items labelled safe or dangerous begin Create an initial population of dendritic cells (DCs), D Create a set to contain migrated DCs, M forall data items in S do Create a set of DCs randomly sampled from D, P forall DCs in P do Add data item to DCs collected list Update danger, PAMP and safe signal concentrations Update concentrations of output cytokines Migrate dendritic cell from D to M and create a new DC in D if concentration of co-stimulatory molecules is above a threshold end end forall DCs in M do Set DC to be semi-mature if output concentration of semi-mature cytokines is greater than mature cytokines otherwise set as mature end forall data items in S do Calculate number of times data item is presented by a mature DC and a semi-mature DC Label data item as safe if presented by more semi-mature DCs than mature DCs otherwise label as dangerous Add data item to labelled set M end end Algorithm 4: Dendritic Cell Algorithm adapted from [Greensmith et al. 2005] on a robotics platform [Oates et al. 2007]. There is currently very little theoretical work examining danger theory algorithms such as the DCA, although this is unsurprising as they are the newest of the AIS paradigms. 2.3 Current Opinion on Applied AIS In recent years, a large body of work has amassed on applying AIS to specific engineering problems and the analysis of various aspects of these AIS. The most representative picture on the current state of AIS can be gauged from the proceedings of the only dedicated annual AIS conference: the International Conference on Artificial Immune Systems (ICARIS) [Timmis & Bentley 2002, Timmis et al. 2003, Nicosia et al. 2004, Jacob et al. 2005, Bersini & Carneiro 2006, de Castro et al. 2007, Bentley et al. 2008]. We critique here some of the main themes raised within the AIS community on the current state of applied AIS. This discussion 53

54 2.3 Current Opinion on Applied AIS is split into three areas: AIS structure in section 2.3.1; AIS applications and their usefulness in section 2.3.2; and future properties of applied AIS in section Structure As highlighted in section 2.2, [de Castro & Timmis 2002b] introduced a layered framework for engineering AIS that identifies the three basic AIS design elements: component representations, affinity measures and immune algorithms. [Bentley 2005] suggests that a layer is missing from this structure and introduces the notion of an artificial tissue component to AIS. The claim is that because the immune system s job is to protect the tissue (cellular structure) of the body, an immune system without tissue is meaningless. The job of an artificial tissue is to move away from a direct mapping between data and antigens, and to introduce a tissue layer between the AIS and the specific problem. This is much like the genetic representation in a genetic algorithm, providing a dynamic encoding, which can be modified according to the problem. The result is the presentation of a consistent interface to an AIS regardless of the underlying data. An analogy is drawn by [Bentley 2005] between the artificial tissue providing an innate response and the AIS providing the adaptive response. As a preliminary investigation, two tissue algorithms are presented by [Bentley 2005]. Whilst an interesting concept, the benefits of these artificial tissue algorithms are not clear and artificial tissues have not been taken up by the AIS community. The only exception has been [Le Martelot et al. 2008] who implement an artificial tissue using systemic computing, which is a tool similar to a process-calculus. Much like artificial tissue, gene libraries (see section 2.1.3) are an addition to the structure of AIS that have seldom been used. Gene libraries were introduced to AIS by [Hightower et al. 1995], and have recently been re-investigated by [Cayzer et al. 2005, Cayzer & Smith 2006], who suggest that they form metalearning which could be useful for AIS. Benefits of gene libraries are said to include improving non-self coverage and reducing the cost of negative selection detector generation, mapping the antigen population more accurately and helping to deal with co-evolution of antigens. The results of the work of [Cayzer & Smith 2006] suggest gene libraries show much promise, and it is an area that would benefit from more investigation. [Hart & Ross 2004] argue that the set of matching mechanisms (affinity measure) of the layered framework of [de Castro & Timmis 2002b] makes AIS distinct from other bio-inspired algorithms. However, the implications of choosing a matching rule from an engineering view point has not been well studied. 54

55 2.3 Current Opinion on Applied AIS A series of work by [Hart & Ross 2004, Hart 2005, Hart 2006, Hart et al. 2006] addresses this weakness by investigating a simulation of an idiotypic network showing how (amongst other things): the shape of the recognition region in a real-valued shape-space can be altered to change the properties for the resulting idiotypic networks; the recognition radius and recognition thresholds play on the property of network formed; and the way affinity is defined affects the topology of the network. These investigations highlight the importance of understanding the shape-space and affinity measures used in AIS so that they are appropriate for the chosen AIS application. Similarly, [Dilger 2006] examines a number of the distance functions (such as Euclidean and Manhattan) used in shape-spaces and how they can affect the affinity function of an AIS. [Garrett 2003] points out that the normal data vector representation of components in AIS is too simplistic, and naive with respect to the biology. This single vector assumption ignores the fact that a B cell paratope and epitope are not the same thing (see section 2.2.4). Alternative representations do exist, however, for example, the original paper of [Farmer et al. 1986] represents B cells as two vectors: one for each of the paratope and epitope. This representation leads to different dynamics in a generic immune network model built by [Garrett 2003]. More recently, [McEwan et al. 2008] discuss issues surrounding the use of shape-space in AIS, arguing that many of the abstractions that have been made over the years are not appropriate. Works such as these can only benefit AIS as they aim to include a small part of the complexity seen in the real immune system. Hybrid approaches to AIS are suggested by [Garrett 2005] to provide more powerful methods to solve certain problems. An example of a hybrid approach is presented by [Andrews & Timmis 2005b] who take inspiration from an immune mechanism called receptor editing to augment ainet (see section 2.2.4) with an additional mutation operator. This operator is similar to ideas from other evolutionary computing domains and helps to introduce diversity and improve performance of ainet. [Freitas & Timmis 2003] (extended in [Freitas & Timmis 2007]) examines AIS in the context of data mining tasks such as classification and anomaly detection. They advocate a problem-oriented approach for data mining AIS, in that the design should be tailored to the data and the application. This conclusion stems from the inductive bias inherent in all data-mining algorithms. The inductive bias is simply a basis for favouring one hypothesis or data model over another, without which a choice could not be made. Any algorithm that performs generalisation, therefore, must have inductive bias. To maximise the performance in an AIS, an understanding of the nature of the data should be gained before designing the 55

56 2.3 Current Opinion on Applied AIS AIS with an inductive bias suitable to the target data and application. [Freitas & Timmis 2007] show that inductive bias affects the choice of each AIS element in the layered framework of [de Castro & Timmis 2002b], and as a final point they list a set of limitations in existing AIS and suggest future research to overcome these. A strong engineering study such as this can benefit AIS greatly by encouraging an informed approach to developing better problem solving results. Lastly, [Stepney et al. 2003] examines the structure and nature of AIS in terms of the grand challenge for non-classical computation 1. AIS are identified as novel bio-inspired approaches that challenge some of the major classical computational paradigms Applications and Usefulness Although applied AIS are not as widely used as many other bio-inspired paradigms such as evolutionary algorithms and neural networks, the body of work describing their applications has become substantial over the last decade. Accordingly, [Hart & Timmis 2005] (and extended in [Hart & Timmis 2008]) have investigated the application areas of AIS, and considered the contribution AIS have made to these areas. Their survey of AIS is not exhaustive, but attempts to produce a picture of the general areas to which they have been applied. Over 100 papers were classified into 12 categories that were chosen to reflect the natural groupings of the papers. Some of these categories are broad, whereas some are narrow, with a new category created when there was more than one paper reporting a particular application area. The 12 identified categories, in the order of the most papers first, were: clustering/classification; anomaly detection (e.g. detecting faults in engineering systems); computer security; numerical function optimisation; combinatoric optimisation (e.g. scheduling); learning; bio-informatics; image processing; robotics (e.g. control and navigation); adaptive control systems; virus detection; and web mining. The authors go on to note that these categories can be summarised into three general application areas of learning, anomaly detection and optimisation. [Hart & Timmis 2005] observe that the three main application areas of AIS are partly a result of their historical development rather than being chosen to exploit the distinctive features present in the immune system. Having been inspired by the view of the immune system as a defence system performing pattern recognition, AIS practitioners are guilty of producing simplistic models of observed immune mechanisms and applying these in isolation to what appeared to be suit

57 2.3 Current Opinion on Applied AIS able problems. Secondly, many AIS practitioners arrived in the field with a strong evolutionary computation background, and thus naturally apply AIS to similar types of problem as other evolutionary computation paradigms. For a paradigm such as AIS to be considered useful, [Hart & Timmis 2005] argue that it is not sufficient for it to simply outperform other algorithms, but it should contain features not contained within other paradigms. It is these features that make a paradigm distinctive. The way forward for AIS should therefore involve the selection of application areas that map the problem features to mechanisms expressed by the immune system. For each of the three identified AIS application areas highlighted above, [Hart & Timmis 2005] review whether or not applying AIS to these areas brings any benefits that could not have been gained via alternative methods. They conclude that for each area, AIS do not seem to offer anything over alternative approaches. They state, however, that immune approaches have typically been applied to static data sets, but they may in fact be more applicable to dynamic data by incorporating a life-long memory mechanism such as that seen in the real immune system. [Garrett 2005] attempts to assess the usefulness of different types of AIS. It is argued that a useful algorithm is both distinct from other algorithms, and effective at performing its required function. Thus, distinctiveness and effectiveness are used as metrics to measure the usefulness. To measure distinctiveness, the following questions quoted verbatim from [Garrett 2005] are asked: 1. Does the new method contain unique symbols, or can the features of this method be transformed into the features of another method, without affecting the dynamics of the new method? 2. Are the new method s symbols organised in novel expressions, or can its expressions be transformed to become the same as some other method, without affecting its dynamics? 3. Does the new method contain unique processes that are applied to its expressions, or can its processes be transformed to become identical to some other method, without affecting its dynamics? An AIS is deemed distinctive if the answer to at least one of these questions is yes. Likewise, a method is deemed effective if the answer to at least one of the following questions, quoted verbatim from [Garrett 2005], is yes : 1. Does the method provide a unique means of obtaining a set of results? 2. Does the method provide better results than existing methods, when applied to a shared benchmark task? 57

58 2.3 Current Opinion on Applied AIS 3. Does the method allow a set of results to be obtained more quickly than another method, on a benchmark test? [Garrett 2005] considers a truly useful AIS to be one which has been classified as being both distinctive and effective. By examining the research literature, [Garrett 2005] assesses the usefulness of three AIS types: negative selection, clonal selection, and immune network algorithms. Applying the distinctiveness questions, the answer no is given to the first two question and yes to the last question for all three AIS types. (It is interesting to note the contradiction here with [Hart & Ross 2004] who state that the affinity measures of AIS do make them distinct from other approaches.) Each of negative selection, clonal selection and immune networks are therefore said to be distinctive as they contain unique algorithmic processes. In applying the effectiveness questions, negative selection is considered to be effective as it answers yes to the first question, having been assessed to provide unique results. Clonal selection algorithms are considered to be effective part of the time as the answer to the second question is sometimes. Immune network algorithms answer yes to the first two questions, so are also deemed effective. In summary, [Garrett 2005] states that the AIS paradigm has provided three distinct types of method that can in some cases produce effective results. Whilst reflection is a vital part to the scientific process, it is difficult to see how the assessment of the usefulness of AIS by [Garrett 2005] is in itself useful. Firstly, all AIS approaches have been grouped into one of the identified general types of AIS to be assessed for their usefulness. This does not seem to be helpful, as too many generalisations and assumptions have to be made about specific AIS approaches. For example, two AIS inspired by the clonal selection algorithm, such as CLONALG [de Castro & Von Zuben 2002] and the B-cell Algorithm [Kelsey & Timmis 2003], may possess very different, but equally useful properties. By considering them as the same type of algorithm, these properties may by lost for the purposes of evaluation. Secondly, as the distinctiveness and effectiveness questions are answered via an empirical investigation, it can be argued that the answers to these question are entirely subjective, depending on the level of abstraction at which the AIS are examined. For example, the third of the distinctiveness questions looks at the distinctiveness of the processes in the algorithm. On one level, it could be argued that evolutionary algorithms and clonal selection algorithms do not contain distinct processes as both are population based algorithms with mutation and selection mechanisms. [Garrett 2005] argues differently, stating that they are distinct as these mechanisms in clonal selection algorithms are unique, being related to fitness of the solution. However, by this 58

59 2.3 Current Opinion on Applied AIS argument two evolutionary algorithms with slightly different mutation and selection mechanism to each other could also be defined as distinct. In the end, this becomes simply a process of classification, and it is not clear how this can benefit the future development of AIS. By looking at the theoretical link between extremisation problems (a more general form of function optimisation) and an AIS (the B cell algorithm (BCA)), [Hone & Kelsey 2004] suggest a range of possible new applications for AIS as well as approaches to rigorous analysis with theory of dynamical systems. Other theoretical work is invaluable to determining how useful AIS can be, for example the in-depth study of [Stibor et al. 2004, Stibor et al. 2005a, Stibor et al. 2005b, Stibor et al. 2006a, Stibor et al. 2006b, Stibor 2007] mentioned in section 2.2.2, sheds serious doubt on the effectiveness of negative selection algorithms. The work by [Clark et al. 2005] on the convergence BCA is also useful for engineering as it elaborates the convergence abilities of the algorithm based on the function landscape of the problem. A strong candidate for further theoretical investigations includes the dendritic cell algorithm (see section 2.2.5, which should enable the AIS community to better understand how the algorithm works and to what type of problem it is suitable Future Properties In suggesting the way forward for AIS, [Hart & Timmis 2005] believe these systems have been reasonably successful at solving the problems they have been applied to, but do not offer sufficient advantage over other paradigms available to the engineer. To address this and tap into the unexploited potential of AIS, they identify three key ideas mostly missing in the AIS domain: Innate Immunity : The innate immune system has been mainly ignored by AIS practitioners so far (with the recent exceptions of danger theory inspired AIS covered in section 2.2.5). This is in contrast to immunology where in the last decade there has been a resurgence of interest in the mechanisms of the innate immune system, such as signalling, and its role in controlling the adaptive response [Germain 2004]. Interactions : The immune system does not operate in isolation. Through the interactions of the immune, neural and endocrine systems, organisms achieve a steady internal state in varying environments (homeostasis). Life-long learning : A key property of the immune system is life-long learning. True life-long learning whereby a system is required to improve its perfor- 59

60 2.4 Conclusions mance as a consequence of its lifetime s experience, has not been utilised in AIS. In summary, [Hart & Timmis 2005] propose a list of features they believe AIS will be required to possess a combination of, if the field of AIS is to carve out a computational niche. These future AIS features, quoted verbatim from [Hart & Timmis 2005], are: 1. They will exhibit homeostasis 2. They will benefit from interactions between innate and adaptive immune models 3. They will consist of multiple, interacting components 4. Components can be easily and naturally distributed 5. They will be required to perform life-long learning Both [Garrett 2005] and [Twycross & Aickelin 2005, Twycross & Aickelin 2007] also call for more innate inspired AIS. It is noted that a number of the suggestions of [Hart & Timmis 2005] are being investigated. The DCA of [Greensmith et al. 2006] is a truly innate inspired AIS, and the role of homeostasis for engineered systems is addressed by [Neal & Timmis 2005, Owens et al. 2007] in the context of the immune system. It seems clear that there is still plenty of un-tapped immune inspiration for new AIS and suitable applications to apply these to. 2.4 Conclusions We have presented in this chapter a picture of the current state of applied AIS, starting with the immunology that has inspired them, coverage of their structure and main algorithms, and finally a critical look at the current opinion on applied AIS. Like all areas of bio-inspired computing, the main structures and processes present in the algorithms stem from the underlying structures and processes in the real system. These were identified for AIS in section 2.1, in which we gave an overview of the immune system focussing on the main cells and processes that have inspired AIS to date. The immune system, however, is a vastly more complicated and rich biological system, comprising many different cell populations and mechanisms that have not inspired AIS. In recognition of this, it is part of the goal of this thesis to investigate immunology that has not previously inspired AIS, a subject we will examine in more detail in chapter 4. 60

61 2.4 Conclusions Despite the main classes of AIS that have been inspired by different immunology (negative selection, clonal selection, danger theory and immune networks), the general framework for AIS presented by [de Castro & Timmis 2002b] neatly summarises their structure into three components: component representations, affinity measures and immune algorithms. All major AIS share these components, and they are so integral to AIS design that it is highly likely that any new AIS will share these components too. The critique of AIS in section 2.3 focused on three main areas: how the structures they employ can differ from the generic framework of [de Castro & Timmis 2002b]; the applications AIS have been applied to and how well they have performed; and future properties for AIS. It is clear from all these views that whilst current AIS have been successful at many of the problems they have been applied to, AIS are not fulfilling their perceived potential and should strive to achieve more. To overcome this, some authors (see section 2.3.1) suggest that AIS are missing key structures such as artificial tissues and gene libraries, or that AIS should be hybridised with other approaches. Others have suggested using simulation to explore the algorithm structures such as affinity measures and shape spaces, and that AIS should be specialised based on the data and application to which they are being applied. By analysing AIS from the point of view of their application, [Hart & Timmis 2005] argue that these applications may not be the most appropriate for AIS. Instead applications should be chosen to exploit the features present in the AIS. [Hart & Timmis 2005] also believe that current AIS do not exploit all the potentially useful properties of the immune system. They therefore suggest a list of such properties (see section 2.3.3) that future AIS should employ. We believe that this list is not exhaustive, but we are very much in agreement that AIS should possess more and different properties from the immune system than they currently do. It is our belief that much of the criticism of AIS in section 2.3 is justified, and we especially feel that moving away from the negative selection, clonal selection, danger theory and immune network views is important for the field of AIS to move forward. To do this we also believe that a more principled approach to algorithm design is needed to extract these properties, which forms the subject for the next chapter. It is important to stress that at this initial stage we are taking a view on AIS design free from application, focussing purely on immune properties as seen in the immune system. As we progress down this route, application specific issues will be addressed when any developed AIS is applied to solve an engineering problem. 61

62 CHAPTER THREE Exploiting the Immune System In this chapter we explore how we are going to exploit and extract immune system properties in a principled way to enable us to develop an AIS. In section 3.1 we elaborate our main inspiration, namely the CFA of [Stepney et al. 2004] for the development of bio-inspired algorithms. This is followed in section 3.2 by an investigation of how the CFA has influenced AIS to date. The CFA suggests that AIS should be developed in a way that incorporates modelling to understand the underlying biological concepts. We therefore examine in section 3.3 a number of approaches that have been used to model and simulate immune mechanisms that might be applicable when following an approach inspired by the CFA. In section 3.4 we examine how modelling can be used within the context of AIS to understand immune properties. We conclude the chapter in section 3.5 by outlining a methodology adapted from the CFA that we follow to develop an AIS within the limitations of a PhD. 3.1 Bio-Inspired Algorithm Design through Conceptual Frameworks Despite the large number of bio-inspired algorithms that have been developed over the last half a century, very little work exists that explicitly deals with how new bio-inspired algorithms should be developed. In section 1.1.2, we introduced the ideas of [Stepney et al. 2004] for developing bio-inspired algorithms who attempt to address this and: propose that bio-inspired algorithms are best developed and anal- 62

63 3.1 Bio-Inspired Algorithm Design through Conceptual Frameworks ysed in the context of a multidisciplinary conceptual framework that provides for sophisticated biological models and well-founded analytical principles. [Stepney et al. 2004] The outline structure of the conceptual framework from figures 1.1 is reproduced here in figure 3.1, for convenience. The conceptual framework shown in figure 3.1 provides us with a structured and principled approach to the development of a bio-inspired algorithm. It starts with the biological system that is providing the inspiration for bio-inspired algorithms. [Stepney et al. 2004] say little about what might make a suitable biological system to take inspiration from, so we assume that any part, process or mechanism of any biological system is open to investigation. This is an issue to which we will return in chapters 4 and 8. As noted in chapter 1, once a suitable biological system has been identified, the conceptual framework proceeds through four distinct stages. Both the simplifying abstract representations and analytical computational frameworks stages fall under the heading of modelling. Models can take many forms and are simply abstractions that aid description and/or understanding. We will return to the subject of modelling and simulation in section 3.3 where definitions are given. Proceeding through the four stages of the CFA need not be a uni-directional process, hence the double ended arrows in figure 3.1. This permits the flow of information back and forth between stages depending on the results or insight gained from each stage. For example, the results from implementing the bioinspired algorithm may influence the validation of the models, suggest model improvements or further probes to the biological system. The framework also allows for multiple models of the same biological system and multiple computa- Figure 3.1: The original conceptual framework for bio-inspired algorithm design from [Stepney et al. 2004] 63

64 3.1 Bio-Inspired Algorithm Design through Conceptual Frameworks tional frameworks from the same models. Owing to the broad range of skills captured by the CFA, [Stepney et al. 2004] state that the process of constructing a complete framework for bio-inspired algorithms is necessarily an interdisciplinary approach requiring, at a minimum, collaboration between a biologist, mathematician and computer scientist. The CFA attempts to increase the chances of extracting the underlying properties of biological systems that are important to provide the observed behaviours from which we are attempting to take inspiration. It is usual for bio-inspired algorithms to proceed directly from a naive biological model to an algorithm, with little investigation of the relevant biological properties. It is this approach, termed reasoning by metaphor by [Stepney et al. 2004], that the CFA tries to get away from by providing a more principled approach. Not only should this be of benefit to the design of bio-inspired algorithms, but also to the biological system under study. The sophisticated models that the CFA calls for could provide insight into the real biological system and suggest avenues for further biological research and experimentation. [Stepney et al. 2004] also introduce the concept of meta-frameworks for bioinspired computation for the purpose of studying the similarities between a series of conceptual, mathematical and computational frameworks. The aim is to develop more integrated and generic frameworks that capture any unifying properties. To achieve this, [Stepney et al. 2004] suggest the conceptual framework shown in figure 3.2, which follows the same structure as the framework in figure 3.1. Instead of a biological system being under study, a series of frameworks is probed to build meta-representation, meta-frameworks and novel unified algorithms. Figure 3.2: The original meta-framework for integrating computational domains from [Stepney et al. 2004] 64

65 3.2 The Conceptual Framework Approach and AIS A number of meta-questions are also proposed by [Stepney et al. 2004] to form the framework probes, which will influence the kinds of meta-model built. These questions address concepts such as openness, diversity, interaction, structure and scale, and are collectively called the ODISS questions. The resulting answers from asking each question across a number of frameworks being studied, will feed into the construction of the meta-framework. The ODISS questions of [Stepney et al. 2004] are as follows: Openness : Addresses the desire for non-halting, continually evolving and growing systems. Typical questions would address how much openness is necessary, how it is controlled and how is the system maintained in the presence of openness. Diversity : Deals with the heterogeneity present in the structure, behaviour and interactions within a biological system. Questions of diversity ask how much diversity is necessary within and between levels of structure, can we talk about an average agent and how does diversity overcome fragility. Interactions : Concerns how agents interact with the environment and other agents. Interaction questions would investigate how the interactions take place within and between structural levels and how do communications influences the system. Structure : Biological systems have structure on many different levels such as molecular, cellular, organisational, societal etc. Questions about structure might address how to recognise these levels, how they affect interaction and what the relationships are between physical and informational structures. Scale : The scale of biological systems is typically large and consists of many components. Questions of scale would address how big a system must be to show emergence, whether or not behaviour changes as scale does and how diversity affects scale. As stated in chapter 1, we propose to use the CFA ideas of [Stepney et al. 2004] as the basis for developing a novel AIS. It will, however, be necessary for us to tailor the approach to our needs, to which we will return in section The Conceptual Framework Approach and AIS Having outlined the main ideas of the CFA, we examine in this section how the CFA is applicable to AIS and where it has already influenced work in the 65

66 3.2 The Conceptual Framework Approach and AIS AIS community. It was stated by [Stepney et al. 2004] that many bio-inspired algorithms fall into the trap of reasoning by metaphor, proceeding straight from simplistic immune models directly to a bio-inspired algorithm. This is certainly true of the many applied AIS, for example negative selection algorithms described in section Whilst it is true that negative selection takes place in the body, it forms just one part of a much more complicated cellular selection process. One the main sites where negative selection occurs is the selection of T cells in the thymus. During thymic selection, not only do T cells undergo negative selection (where strongly self-reactive cells are deleted), but they are also subject to positive selection to ensure that those T cell that do survive can recognise MHC, itself a self peptide. This selection process as a whole, therefore, produces a set of detectors that can essentially recognise some elements of self strongly enough to make them effective at binding to APCs, whilst at the same time fail to recognise other elements of self too strongly. The majority of negative selection AIS ignore the fact that negative selection is just one part of a wider selection process in the body. The view of negative selection as the process that achieves self non-self discrimination is naive and out of context. In the previous chapter we outlined another framework that can be applied to AIS, namely the layered framework of [de Castro & Timmis 2002b] described in section 2.2 and shown in figure 2.4. Although it may appear that the layered framework and the CFA are not compatible, we believe this is not the case. The layered framework describes the development of AIS as consisting of three main components: the component representations (e.g. a string of bits), the affinity measures (e.g. Euclidean distance) and immune algorithms. Any AIS developed by following the CFA is highly likely to contain the same three components, thus the layered framework would easily sit within the last stage of the CFA when instantiating an AIS to a specific application. The layered framework could also be applicable to earlier simulation stages of the CFA, as again, these are likely to contain representations of immune components, measures of affinity and an algorithm to describe how the components will interact. It is also likely that the choice of these three components is consistent over the various modelling, simulation and algorithm stages of the CFA as each stage will inform the one to follow. The main difference between applying the layered framework to a simulation rather than an applied AIS is in the application domain that drives the choices of the three main AIS components. For a simulation, the application domain would be analogous to the biology you are investigating and the goal of the simulation, thus the choice of component representation, affinity measures and immune algorithm are going to depend on the modelling techniques being employed and 66

67 3.2 The Conceptual Framework Approach and AIS what the simulation intends to show. [Stepney et al. 2004] provide an example of how the CFA could be applied to AIS by analysing the structure of population based (clonal selection, negative selection) and network based (idiotypic networks, cytokine networks) AIS. For network based models, the immune system is probed from the perspective of a system with network interactions, with network elements (e.g. cells) co-operating and competing. From these probes, you develop a suitable mapping between the biological properties and the components for artificial systems. These components then form the basis for the topology and dynamics in new biological models, or the re-examination of old models (such as the network of [Farmer et al. 1986]) but in the context of the CFA. The models provide an understanding of the working of the system. This leads to new metaphors and frameworks with appropriate representations for the components and methods for assessing their interactions and processes that interact upon these components. The instantiation of the these frameworks allows the relevant properties to be captured for the algorithm being developed. [Stepney et al. 2005] extends [Stepney et al. 2004] by describing mathematical techniques for analysing the state of AIS, and providing a high level case-study involving a number of biological components to which the ODISS questions are applied. This case-study describes a piece of on-going research presented in [Neal & Timmis 2005], which investigates the property of homeostasis for autonomous robotic systems. Biological homeostasis is provided by components such as the immune system, the neural system and the endocrine system, to which the system of [Neal & Timmis 2005] contains analogies. It is these systems that the ODISS questions are applied to, for example, diversity is present in various different cell types such as neurons, endocrine gland cells and immune cells. Interactions between these are via simple communication channels over varying time periods such as fast neural synapses and slow chemical diffusion. Whilst an interesting and promising system, this case-study is a very high level overview of what the CFA is about. There are no reported details of any implementation or modelling work regarding the artificial components in the system. It does, however, show how the ODISS questions can be applied to an artificial system. [Newborough & Stepney 2005] apply many of the ideas behind [Stepney et al. 2004] to produce a generic framework for population-based bio-inspired algorithms. The framework is constructed more in the vein of the meta-framework described in section 3.1 and summarised in figure 3.2, by looking not at a biological system, but at a group of bio-inspired algorithms. These algorithms are not in themselves computational frameworks or particular exemplars of the algorithms, 67

68 3.2 The Conceptual Framework Approach and AIS but the generic components and behaviours behind them. The algorithms examined are generic genetic algorithms, negative selection, clonal selection, particle swarm optimisation and ant colony optimisation. Examination of the algorithms is carried out at a high level, simply by empirically investigating the components and interactions for the algorithms and generalising across these. They abstract the basic underlying concepts from each class of algorithm. Six algorithmic stages are said to exist in each algorithm (quoted verbatim from [Newborough & Stepney 2005]): 1. Create: make novel members of the population 2. Evaluate: evaluate each individual for its affinity to the solution 3. Test: test if some termination condition has been met 4. Select: select certain individuals from the current generation, based on their affinity, to be used in the creation of the next generation 5. Spawn: create new individuals for the next generation 6. Mutate: change selected individuals Once situated in this generic framework, generalisations can be made that take features from one specific algorithm and apply them to others. Such properties investigated were niching and elitism. A prototype implementation of the above is presented for an FPGA as a proof of concept allowing the user to pick features. This work is one of the first examples where ideas behind the CFA are investigated, and in implementation terms it is the most complete. It shows a generic framework inspired by biology but not restricted by any particular domain. However, this work does not follow the CFA ideas in full, with no explicit modelling of biological concepts. The CFA influenced the paper of [Twycross & Aickelin 2005] who present a general meta-framework for models incorporating innate immunity. A table of six general properties of the innate immune system is presented and it is claimed that AIS will need to incorporate properties such as these to realise functions of the immune system. While [Stepney et al. 2005] use meta-frameworks to analyse artificial models for essential features and commonalities, here [Twycross & Aickelin 2005] use it to analyse biological models. This is because few artificial models exist, and it is claimed that this allows biology to have more of an influence on the meta-framework. The ODISS questions are then applied comparing innate and adaptive immune cells and mechanisms. It is not clear quite what framework 68

69 3.3 Modelling and Simulating Immune Systems is present in the work of [Twycross & Aickelin 2005] as the CFA seems only to provide an argumentation structure around analysing innate immunology and its possible benefit for AIS. In a similar approach to [Twycross & Aickelin 2005], [Guzella et al. 2007a] highlight a class of T cell, T regulatory cells, as inspiration for AIS. They suggest that incorporating these cells might lead to more biologically plausible models and algorithms that achieve better results in real-life problems. Two models of regulatory T cells are described that could form the starting point in a CFA type approach. 3.3 Modelling and Simulating Immune Systems The CFA argues for explicit modelling of biological systems from which inspiration is being taken to understand the underlying properties of that system. There are many modelling approaches applicable to AIS, but before reviewing modelling work on the immune system, it is useful to define what we mean by a model and a simulation. In the broadest sense, a model is simply an abstract representation of a thing, such as an entity, phenomenon or process. A model can take many forms, for example mathematical, logical, diagrammatic and physical. They are used to help us understand and communicate problems. Based on this definition of a model, a simulation is simply the implementation of a model over time. Simulations also often provide key concepts such as spatial organisation, and they allow us to analyse the behaviour of the models. It is noted that model and simulation are often confused and used interchangeably within the literature. This makes it hard to cover each topic separately, we therefore discuss them together in this section. The process of modelling involves identifying the components of a system under investigation, along with the component relationships, behaviours and interactions with each other and their environment. The result of this depends on the modelling techniques being employed. In section we noted that current thinking in AIS views models and simulations of the immune system as examples of AIS alongside those applied AIS covered in section 2.2. Like applied AIS, the idea of component representation and affinity measures described in section also applies to many of the models and simulations of the immune system. These differ from the applied AIS by defining the interactions between the components in an application neutral way. The behaviour of the modelled AIS is used to understand the immune system rather than to solve an engineering problem. [Forrest & Beauchemin 2007] note that there is a vast range of modelling ap- 69

70 3.3 Modelling and Simulating Immune Systems proaches applicable to model the immune system, each with their own advantages and disadvantages operating at different levels of abstraction. Owing to the vast number of modelling approaches, it is not possible to cover them comprehensively here, instead we give an overview of the types of techniques that have been used to model and simulate the immune system. We have split these techniques into the following groups: mathematical and formal approaches in section 3.3.1; agent-based approaches in section 3.3.2; and diagrammatic and software engineering approaches in section Mathematical and Formal Approaches An overview of many mathematical techniques used for modelling the immune system is provided by [Perelson & Weisbuch 1997]. A large number of these approaches involve the use of differential equations, although other techniques can be applied such as Boolean networks [Weisbuch & Atlan 1988] and the work of [Kelsey et al. 2008] who present and analyse a Markov chain model of a cytokine network. Differential equation based models typically model population level behaviours and do not explicitly represent each individual such as a cell. One of the first, and most influential, differential equation models was that of [Farmer et al. 1986] who performed an in-depth investigation of the original idiotypic immune network ideas of [Jerne 1974], providing insight into their dynamics. The model of [Farmer et al. 1986] focused on how the stimulation level of a population of B cells is affected by antigen binding, binding to other B cells, and suppression by other B cells. Immune networks are still being studied by AIS researchers such as [Schmidtchen & Behn 2006] who present a minimal model of an idiotypic network investigating how random evolution can lead to a highly organised steady structure. Differential equation models have also been used to study the interaction of cytokines and immune cells by [Hone & van den Berg 2007], who develop a generic model called an artificial cytokine network (ACN). The model consists of a network of cytokines that are produced by cell types undergoing an external stimulation. Cytokine production is a function of the cytokine concentrations, external stimuli and number of cells. [Hone & van den Berg 2007] present numerical results of an ACN with two cytokines, one cell type and a single external stimulus. This study has been extended by [Read et al. 2008], who ran a number of empirical experiments to get an understanding of the network s parameters and behaviours in order to assess its suitability for inspiring applied AIS. They concluded that the behaviour of the ACN was too unpredictable to be of any obvious 70

71 3.3 Modelling and Simulating Immune Systems benefit in an engineering system. Recently, process calculi have been applied to models of the immune system such as [Owens et al. 2008]. Process calculi are formal languages from computer science that are used to specify concurrent systems. As biological systems are inherently concurrent, these types of languages seem well suited to biological modelling. [Timmis et al. 2008a] provides further reasons for using process calculi such as the π-calculus (and variants) that include graphical tool support, simulators, qualitative and quantitative analysis of biological systems, and the ability to translate a π-calculus model into a continuous time Markov chain model. In summary, equation-based models are powerful and often rigorous tools for describing and investigating population behaviours, but often assume that these populations are homogeneous and well mixed. They can also be difficult for non-mathematicians to appreciate, and are typically based on big biological abstractions and large sets of assumptions. They do, however, benefit from tools and techniques that can check their correctness Agent Based Approaches [Forrest & Beauchemin 2007] provide a review of many of the modelling approaches in immunology, with a focus on agent based modelling (ABM). In ABM, components such as cells (and sometimes molecules) are represented individually as agents rather than as homogeneous populations such as in differential equation techniques. Different agent types typically represent different immune cells types. These agent types are encoded with simple rules extracted from the real biology that govern how they behave and interact. ABM techniques typically employ a concrete notion of space such as that used in cellular automata-like models [Kleinstein & Seiden 2000]. The advantage of ABM is that it allows the observation of agent population dynamics as they emerge from the interactions of individual agents. An example of an ABM is given by [Beauchemin et al. 2006] who investigate the dynamics of in vitro infection with a strain of influenza. Their model has an explicit notion of space (often overlooked in equation based models), consisting of a cell layer based on a hexagonal 2-dimensional grid. Each grid cell represents a real epithelial cell, whereas influenza virons are modelled as continuous realvalued concentrations at each grid location that diffuses over time. The cells can be in one of four states: healthy, dead, infected, and infected and secreting. This in silico model is run in tandem with in vitro experiments that inform the ABM, providing realistic simulation parameters. This leads to a high-cohesion in the 71

72 3.3 Modelling and Simulating Immune Systems model to biological accuracy. ABM has not only been applied to biologically realistic situations such as the model of [Beauchemin et al. 2006] just described. [Salazar-Bañuelos 2008] presents an agent-based model of immune responses from the perspective of swarms to investigate more philosophical issues of immune recognition. ABM has also been used to study more computational aspects of applied AIS such as the series of work by [Hart & Ross 2004, Hart 2005, Hart 2006, Hart et al. 2006] mentioned in section These works use a simulation of an idiotypic network to investigate how different models of shape-space and affinity affect the dynamics of the network such as memory capacity and the structures formed, emphasising the need for careful choice of parameters in the engineered systems. To aid the development of agent-based simulations, a number of generic simulators exist that are applicable to immune simulations such as IMMSIM [Kleinstein & Seiden 2000], Simmune [Meier-Schellersheim & Mack 1999] and MASON [Luke et al. 2005]. IMMSIM enables to the construction of cellular automata-like simulations of immune behaviours, containing agents such as T cells, B cells, antibodies and APCs. Simmune is a simulation environment that provides a suite of tools to construct models from data describing cell behaviour across many scales from molecular interactions to cell population behaviours. It was originally created to model immune properties, but is applicable to a wide range cell scale biological models. Whilst IMMSIM and Simmune are geared towards immune simulation, MASON has been designed to be applicable to more generic agent based simulations. MASON is essentially a Java library that provides generic functionality for agent-based simulations such as agent control, structures for representing space and visualisation tools. One major drawback of the agent-based simulators is gaining an appreciation of how the provided generic behaviours have been implemented and the types of assumptions that might be built in to the simulation infrastructure The main advantage of agent-based models is the ability to describe the behaviours of each individual, creating a good mapping to the actual biology with fewer large assumptions than equation-based models. Their close mapping to the biology also makes the agent-based approach more accessible to the experimental biologist. Another important feature of agent-based models is that in addition to encoding agent behaviours, they can include a description of the spatial environment in which the agents interact, which can often be an important factor in simulating the required behaviours. One of the typical drawbacks of agent approaches is their lack of scalability. The computing power needed to simulate each agent individually often limits the number of agents to a fraction of the num- 72

73 3.3 Modelling and Simulating Immune Systems ber in the actual system. Whether this issue with scalability becomes a problem is normally specific to the system being simulated Diagrammatic and Software Engineering Approaches The majority of diagrams used in immunology, and biology as a whole, are informal pictorial representations that aim to clarify a textual description. Examples appear in this thesis, such as in figure 2.3. Although useful, these cartoons lack any kind of formal syntax and semantics, and typically do not capture abstractions. Diagrammatic modelling languages are more structured than the biological cartoons, providing a language for both relationships and structure. A number of diagrammatic languages have been specifically designed for modelling molecular networks in systems biology, which could be appropriate for modelling molecular aspects of immunology. An example of of a diagrammatic language is the process diagrams of [Kitano et al. 2005], which are applied to capture molecular state transitions. In addition to the tools developed by biologists, there is a wealth of diagrammatic modelling experience in computer science and software engineering that has culminated in the unified modelling language (UML) [Fowler 2000]. The UML consists of a set of 13 different types of diagram that can model different aspects of structure and behaviour. The advantage of the UML is its non-domain specific nature and subsequent ability to capture abstractions. The UML (and related diagrams such as statecharts) have started to become a powerful tool in modelling aspects of biological systems. By far the most advanced use of the UML and statecharts in immunology is that of [Efroni et al. 2003], who have built a sophisticated and predictive model of T cell maturation in the thymus using a simulation tool called reactive animation, which combines the execution of statecharts and other behavioural diagrams. In addition to the UML, there are other techniques used in software engineering, which [Bersini 2006] suggests can facilitate the development and communication of immune modelling. These include object oriented technologies such as object oriented programming and design patterns [Gamma et al. 1995]. The perceived benefit is the clarification of immune objects and their relationships. To support this, [Bersini 2006] provides an example of how clonal selection can be modelled with a simple state diagram. Design patterns are also proposed by [Babaoglu et al. 2006] as an alternative route to exploiting biology for the benefit of computing techniques. They suggest design patterns can be extracted and abstracted from biology to transfer knowledge to the field of distributed comput- 73

74 3.4 Understanding Immunology through Models ing. These design patterns form a bridge between the engineering and biological systems. [Babaoglu et al. 2006] succeed in identifying a number of suitable patterns common in biological systems, such as diffusion, replication, stigmergy and chemotaxis, that can be applied to distributed computing problems. The main advantage of diagrammatic approaches to modelling is to capture the behaviours and components of the system you are investigating in a structured and accessible way. There is also a lot of expertise in computer science in general purpose modelling languages such as UML that can often be applicable to modelling biology. Simulation tools, such as Reactive Animation [Efroni et al. 2005], can also be constructed to execute the behaviours described in the diagrammatic models. Unlike the agent-based simulators, few simulators run from diagrammatic models, and those that do (Reactive Animation included) are proprietary tools. 3.4 Understanding Immunology through Models [Timmis et al. 2006] argue that bio-inspired computation, such as AIS, has reached an impasse. While the systems that have been built to date have been effective at solving engineering problems, they are often based on crude approximations of the fundamental biological principles. A gulf therefore exists between the real and artificial systems. They state that the current approach to designing bioinspired systems fails to get us any closer to completely novel computational paradigms, and it is rare that studies add to the understanding of the underlying biological system. [Timmis et al. 2006] therefore believe that a reassessment is needed of how biology is exploited for the benefit of computational systems. Echoing the view of [Stepney et al. 2004] highlighted in section 2.2, [Timmis et al. 2006] state there must be closer interaction between computer scientists and biologists to generate useful models, with a desire for long term insights not just a focus on short term applications. Considering the history of AIS, [Stepney et al. 2004] suggest that many have drifted away from the immunological inspiration that had fuelled their development, and that AIS practitioners are failing to capture the complexity and richness that the immune system offers. Many of the original AIS had their origins in the field of theoretical immunology such as [Perelson & Oster 1979, Farmer et al. 1986, Varela & Coutinho 1991]. Indeed, two of the earliest engineering applications for AIS can be attributed to [Bersini 1991] and [Forrest et al. 1994], which had their roots firmly in the immunology on which the algorithms were based. In the early part of this decade, however, many AIS had drifted away from the 74

75 3.4 Understanding Immunology through Models inspiration and more towards engineering considerations, with most of the AIS enhancements inspired more by the engineering aspects rather than immunology. This need not be a bad thing for AIS (see the problem-oriented approach of [Freitas & Timmis 2007] described in section 2.3.1), but the field of AIS can only benefit from more immune inspired work. In recent years, there has been a gradual shift in some AIS back towards paying more attention to the underlying biological system that serves as inspiration. For example, the development of the DCA (see section 2.2.5) involved the input from real biological experimentation as inspiration. However, there was no reported sophisticated biological modelling of the forms covered in section 3.3 to understand the underlying biology as is suggested by [Stepney et al. 2004] and [Timmis et al. 2006]. The only biological models that exist are those in the form of cartoons, such as the figure of dendritic cell differentiation into mature or semimature reported in [Greensmith et al. 2006]. Other examples of this shift back to the underlying biology include [Wilson & Garrett 2004] and [Jacob et al. 2004], who have used modelling techniques to build AIS in order to understand underlying immune properties. Different levels of abstraction appear in the type of models we build of the immune system described in section 3.3. For example, many of the differential equation models are quite abstract in their assumption of a well mixed system and the removal of spatial aspects of the biology. Agent-based modelling (ABM) makes less of these assumptions and hence can be said to be a less abstract form of modelling. Owing to this, [Forrest & Beauchemin 2007] argue that ABM might be a more appropriate tool for modelling immunology due to the ease with which one can incorporate knowledge into the model. This allows multiple experiments to be run with ease. One difficult aspect of ABM, however, is defining the right level of abstraction for each agent in the model, such as movement rules and interaction behaviours. Different choices will affect how the simulation operates. We believe in the need to choose modelling techniques that are suited to the nature of the immune processes being modelled. For example, if the immune process is an emergent behaviour, then the designer needs to choose a modelling technique that allows emergence to occur. Such modelling tools are used extensively by the Artificial Life (ALife) community. ALife, as described by [Langton 1992], takes a synthetic, bottom-up approach to biology, putting together the interacting elements of a system in order to understand it. The interactions between these elements produce the observed emergent dynamic behaviours. We also suggest, therefore, that the models used by the ALife community may be suitable in the design process of AIS inspired by immune models with emer- 75

76 3.5 Developing a New AIS: A Methodology gent behaviours. This makes the ABM approaches (section 3.3.2) particularly well suited. In addition to the tools that have already been used to model the immune system and the proposed ALife tools, techniques that have been utilised in other biological modelling areas may be appropriate to immune modelling, and should thus be investigated. For example the work by [Johnson et al. 2004] use the object-oriented methods mentioned by [Bersini 2006] to model intra-cellular processes. An effective example of a modelling technique taken from a different area of modelling is given by [Chao et al. 2004], where a stochastic age-structured model, often used in ecology, has been applied to modelling immune cell populations and transitions. 3.5 Developing a New AIS: A Methodology In chapter 1 we highlighted our aim of applying the ideas of the CFA to the development of a novel AIS. In this chapter we have examined the CFA and its influence on AIS, and it is clear from the ideas in section 3.1, the CFA proposes a more principled route to developing bio-inspired algorithms than current techniques. It is also clear from section 3.2 that there have been no examples presented in the literature which apply the CFA from start to finish to develop a new bio-inspired algorithm. As described in section 1.2.1, the goal of this thesis is to develop an AIS based on novel immunology by following the ideas of the CFA. We therefore present in this section an outline of the CFA inspired method we follow to develop an AIS, detailing which ideas of the CFA we are applying and any deviations we make from the CFA. We apply the ideas of [Stepney et al. 2004] summarised in section 3.1 that relate to the development of a single instance of a bio-inspired algorithm. The stages in figure 3.1 will therefore form the basis of the method we adopt to develop a novel AIS. We will not be using the meta-framework as shown in figure 3.2 as we will only be developing a single AIS. Also, whilst the ODISS questions are interesting to ask of a bio-inspire algorithm, we will not be directly applying them to the technical work presented in the rest of this thesis. One of the key points of the CFA highlighted in section 3.1 is that it is necessarily an interdisciplinary process involving at the very least, a computer scientist, a mathematician and a biologist. In the context of a single person PhD, it is not possible to be an expert in each of these three areas, thus the main limitation we have in applying the CFA is the lack of domain expertise, especially with regards to the immunology. Owing to this, the following limitations will apply to our application of the CFA: 76

77 3.5 Developing a New AIS: A Methodology 1. Choosing biological system: The knowledge of the biological system chosen for inspiration before we apply the probes will not be of the same level of a biological specialist. In addition, the biological system is not open to direct analysis by us, with the main source of knowledge coming from text-books and the research literature. 2. Probes: The types of probe we can use are severely limited by our inability to directly probe the system and a lack of access to a domain expert. 3. Models: The type of models we are able to build will be biased to the computational approaches (such as those presented in section 3.3.2) rather than the more mathematical approaches. 4. Depth: As only one person is implementing each stage, the amount of detail we can go into at each stage would not be as deep as if domain experts were doing each stage separately. We will return to discus these limitations once we have presented our adapted CFA. In addition to addressing the above limitations, we can tailor the CFA to our specific biological domain (the immune system) and the algorithm to be developed (AIS). Thus we have adapted the original framework shown in figure 3.1 to take these limitations into account and to specialise it for the area of immunology and AIS. Our conceptual framework for AIS (CFAIS) is presented in figure 3.3. The structure of this framework is the same as that of the original, but the stages have been adapted as follows: Biological system Immunology: research literature : We have specialised our domain of interest to the immune system, and limited it to the relevant research literature. Probes Probes: read, investigate : The only probes open to us is reading the literature available. Simplifying abstract representations Simplifying computational models : As our expertise lies in the computational field, any models that we build will biased towards the computational models rather than mathematical. Analytical computational frameworks Algorithm frameworks : The computational models will naturally refine to algorithm frameworks from which an AIS specific to an application can be instantiated. 77

78 3.5 Developing a New AIS: A Methodology Figure 3.3: A conceptual framework for developing AIS. Bio-inspired algorithms Artificial immune systems : We have specialised the resulting algorithm to be an applied AIS, similar in form to those described in section 2.2. In the CFAIS, we allow the same transitions between stages as the original conceptual framework. This provides transitions back and forth between modelling each stage. This enables us to re-evaluate if the outcome of any stage is unsatisfactory. The idea is not to pre-define our route but to allow the process to evolve based on the results from each stage. During the modelling and simulation stages of the CFAIS we will apply modelling techniques that seem appropriate for the purpose the model we wish to create. The simplest instantiation of the CFAIS will be linear from the immunology through modelling and algorithm framework construction to a specific AIS. Figure 3.4 summarises the flow of increased abstraction as we move from the real biological system to the artificial representation inherent in the AIS. Despite the limitation of lack of domain expertise identified above, by following the CFAIS we should still gain insight into the CFA of [Stepney et al. 2004] and Figure 3.4: A depiction of the levels of abstraction within the stages of our framework for developing AIS. 78

79 3.5 Developing a New AIS: A Methodology provide the most detailed investigation of these ideas that has been presented in the literature to date. Even by developing models that do not contain the depth we may wish, we believe we are still doing better than the majority of the reasoning by metaphor approaches presently employed to develop AIS. Another issue that the lack of domain expertise impacts upon, is the validation that models and simulations are producing the desired behaviours. This is an inherently difficult task, and one which ideally we would like to carry out by comparison to a real system. Without domain expertise and access to the actual biological system and relevant data, this is impossible in the context of this PhD. Instead, we will simply assess whether the models and simulations produce qualitatively similar behaviours to those being modelled. To conclude, we outline how the work presented in the next five chapters fits into the stages of CFAIS: Chapter 4 : We probe the immunology research literature to identifying candidate immunological from which to take inspiration, focusing on degeneracy and patterns of response. Chapter 5 : We explore further immunology to identify a place in the body where degenerate patterns of response might occur, and build an agent-based simulation to investigate these properties further. Chapter 6 : Using the results from the agent-based model, we return again to the immunology research literature to investigate how activation thresholds for cells can be tuned. We investigate these ideas using a simple simulation, assessing their applicability to engineering. Chapter 7 : Based on the insight gained from investigating tunable activation thresholds, we construct a framework to allow patterns of degenerate tunable detectors to be incorporated into AIS. We then instantiate this framework with a simple pattern classification AIS. Chapter 8 : We look back at the process we have followed and comment on our journey through the CFAIS. 79

80 CHAPTER FOUR Identifying Biology for a Novel Artificial Immune System Having identified how we plan to develop a novel AIS in the previous chapter, we need to identify what the inspiration for this AIS is going to be. In the context of the CFAIS this chapter identifies the biological properties we are going to take inspiration from, probing that system to provide sufficient detail to move onto the construction of a models and simulations. We begin in section 4.1 by identifying that different levels of abstraction exist in the types of immunology that can inspire AIS. We then move on to examining novel and current immunology that has previously been used to inspire AIS in section 4.2, showing an apparent discord between many immunologists regarding some of the major conceptual ideas of immunology. In section 4.3 we proceed to investigate the alternative immune model of [Cohen 2000b], which presents a complex systems view of the immune system. Inherent within this model are concepts that can inspire AIS and we focus further on the concepts of immune receptor degeneracy and the notion of patterns of response to provide immune recognition in section 4.4. We conclude the chapter in section 4.5 by discussing our plan to exploit degeneracy and patterns of response for a novel AIS. 4.1 Levels of Immune Abstraction for AIS If we examine the AIS algorithms in section 2.2, we can identify two clear levels in the type of biology that has inspired them from section 2.1. The first are the high 80

81 4.1 Levels of Immune Abstraction for AIS level concepts of the immune system, whereas the second are the low level mechanisms that are believed to achieve the high level concepts. Examples include the concept of self non-self discrimination, achieved via the mechanism of negative selection, and the concept of danger due to the mechanisms of the dendritic cells that aggregate danger signals. The idea that AIS should take inspiration from the higher level of biological abstraction (the immune system concepts) is advocated by [Twycross & Aickelin 2007]. They state that immune concepts (called systemic models by [Twycross & Aickelin 2007]) should be employed to structure the design of AIS. They identify that the self non-self discrimination model is dominant in AIS, but it is at odds with current immunological thinking. AIS researchers should therefore look at more contemporary models such as danger theory, maintenance and homeostasis. Echoing the views of many other authors before them such as [Stepney et al. 2004, Timmis et al. 2006], [Twycross & Aickelin 2007] advise building AIS based on more realistic biological models that can help inform the biology by providing feedback on the systemic models of the immune system. We agree with this view, and it is one of main the themes addressed by this thesis. [Twycross & Aickelin 2007] also state that AIS can be made more biologically realistic by drawing inspiration from organisms with only innate immune systems, for example plants and invertebrates. They state that the majority of organisms don t have an adaptive immune system, and in organisational terms the innate immune system is simpler. This leads to the claim that AIS should start with simpler innate based models before building adaptive ones. Whilst we agree that we should not limit AIS to vertebrate inspired immune systems (those with an adaptive element), why should we limit ourselves to non-vertebrate immune systems? Many of the mechanisms of the adaptive immune system (such as clonal selection) are well understood and have inspired successful AIS, so why should we not take inspiration from these? The assumption that we should start with simpler innate models before building adaptive ones also seems to assume that we can separate out these layers (a problematic issue we elaborate on below), and that the non-vertebrate innate immune system has essentially the same functionality as the vertebrate innate immune system. However, a vertebrate without an adaptive immune system is severely compromised, and it may be that non-vertebrates have evolved extra functionality in their innate immune systems. Thus a non-vertebrate innate immune system and a vertebrate innate immune system are not likely to be the same thing. Whilst we should take a principled approach to what we are doing and break things down into more easily understandable chunks, specifying that 81

82 4.1 Levels of Immune Abstraction for AIS we shouldn t look at some parts of immunity is not helpful, especially when we are dealing with such a complicated system as the immune system. A more interesting question might be why do vertebrates have this extra layer of adaptive complexity in their immune systems, whereas non-vertebrates do not? To return to the issue of splitting the innate and adaptive immune systems, [Neal & Trapnell, Jr. 2007] highlight the dangers of drawing separation between such subsystems within biological systems. Drawing these separations can be problematic as the real biology may be so inter-twined that separation is impossible. Often the subsystems identified by researchers are biased by their research interests and the separation of these subsystems may not be the best way of splitting the system. To elaborate, [Neal & Trapnell, Jr. 2007] state the following set of assumptions (quoted verbatim) which often afflict computer scientists: 1. The sub-systems and components as described by biologists are circumscribed by natural boundaries and by implication have some degree of functional separability over and above that likely to be found for similar sized arbitrary sets of linked components selected at random. 2. That sub-systems and components as identified by the biologists will be of interest and utility to computer scientists seeking inspiration. 3. That biologists really believe that they have identified the important features of the immune system when describing these sub components. 4. That it is valid to make computational simplifications and caricatures by using off the shelf components when building computational analogues of the sub-systems and their components. If none of these assumptions were valid, then biology and the subsequent development of bio-inspired systems would be meaningless. However, [Neal & Trapnell, Jr. 2007] claim that if we can accept the first two assumptions, talk to biologists to ascertain that the third holds, and guard against the fourth by reexamining the biology to avoid over simplification, then both the biology and bio-inspired system development maintain value. It is suggested that following the CFA of [Stepney et al. 2004] can help cope with the third and fourth assumptions. The work of [Neal & Trapnell, Jr. 2007] raises an important issue most often ignored by AIS practitioners. Understanding the limitations of the way we perceive the biology may become an important step in validating the models and algorithms we build. 82

83 4.2 Immunology Today 4.2 Immunology Today It is clear from the immunological research literature that many immunologists fundamentally disagree on the mechanisms responsible for many observed immune properties. This disagreement gives rise to many (often conflicting) theories of immune function. In section 4.2.1, an example is given on one such disagreement documented in a series of articles from 2000 published in volume 12(3) of Seminars in Immunology. It is highlighted in section that much of the disagreement arises from a clash of research methodologies being employed by the immunologist. In section we conclude that any sound immune theory can inspire AIS The Science of Self Non-self Discrimination? The formulation of the clonal selection theory gave to immunology an explicit notion of an immune self and a mechanism by which the adaptive immune system could discriminate between self and non-self molecules in the body. With the adoption of the clonal selection theory as the key principle of adaptive immunity, [Tauber 2000] notes that by the 1970s, the immune self had become the defining idea in immunology with the field itself being referred to as the science of self non-self discrimination. It is evident, however, from a series of articles [Anderson & Matzinger 2000, Bretscher 2000, Cohen 2000a, Grossman & Paul 2000, Langman & Cohn 2000b, Medzhitov & Janeway 2000, Tauber 2000, Silverstein & Rose 2000], that this view is beginning to change. In these articles, many leading immunologists discuss their views on the nature and importance of self non-self discrimination in the immune system. The level of disagreement is neatly summarised in the editorial summary, in which [Langman & Cohn 2000a] state that: There is an obvious and dangerous potential for the immune system to kill its host; but it is equally obvious that the best minds in immunology are far from agreement on how the immune system manages to avoid this problem. [Tauber 2000] provides a commentary on the models proposed in the articles by the other immunologists, stating that each model falls to various degrees between the ideas of Burnet (clonal selection theory, see section 2.1.4) and Jerne (immune network theory, see section 2.1.5). The debate amongst immunologists is thus a continuation of the arguments that have run for decades between these two points of view. The minimal model of self non-self discrimination presented by [Langman & Cohn 2000b] is identified as the one closest to the original ideas 83

84 4.2 Immunology Today of Burnet, whereas the model of [Cohen 2000a] is the closest to Jerne s view. The model of [Langman & Cohn 2000b] considers only the adaptive immune cells to be involved in the recognition of non-self antigen, and thus the initiation of the immune response, in the body. Conversely, the model of [Cohen 2000a] removes the requirement for self non-self discrimination, viewing the immune system as complex, emergent system. This model allows all immune cells to recognise both self and non-self antigens and form an immune dialogue with the body s tissues in order to fulfil the role of body maintenance. Between the extremes of the [Langman & Cohn 2000b] and [Cohen 2000a] models fall those of [Bretscher 2000, Medzhitov & Janeway 2000, Anderson & Matzinger 2000, Grossman & Paul 2000]. The model of [Bretscher 2000] attempts to reconcile the original ideas of Burnet with contemporary immunological observations by including the role of innate immune cells acting as antigen presenting cells (APCs) during the initiation of an immune response. [Medzhitov & Janeway 2000] and [Anderson & Matzinger 2000] present the views of the infectious nonself model and danger theory respectively (see section 2.1.6). [Grossman & Paul 2000] introduces adaptable lymphocytes that react to the rate of change in excitatory signals. This is achieved by constantly tuning their activation thresholds, only responding when they can no longer adjust due to a rapid change in excitation. This response is dependent on many conditions that include antigen concentrations, activation levels of APCs and danger signals. In addition to the models summarised above, other positions are presented including that of [Silverstein & Rose 2000]. Although no immune model is presented, they state that the concept of self non-self discrimination is a delusion, arguing that the immune system has evolved in complexity gradually over time. To cope with changing circumstances, control mechanisms have been evolved in parallel to amplify or dampen the immune response. Such an immune response thus becomes a balance between protection and damage dependent on the parameters (e.g. quantity and quality) of the immunogenic stimulus A Clash of Methodologies Much of the difference of opinion regarding the immune models presented above in section 4.2.1, can be explained from the point of view of the research methodology employed in formulating the model. To highlight this we can examine the minimal model of [Langman & Cohn 2002], its criticisms by [Efroni & Cohen 2002, Efroni & Cohen 2003] and a response by [Cohn 2003]. [Langman & Cohn 2002] state that for the immune system to counteract the 84

85 4.2 Immunology Today attack of pathogen, it must generate and regulate new specificities during the lifetime of an individual, and that this requires a mechanism of self non-self discrimination. The minimal model considers only the antigen specific lymphocytes of adaptive immunity, and assumes that during the embryonic development of an individual, maternal protection provides an environment containing only self antigen. Thus, during this early stage in life, a process can take place that produces a lymphocyte population capable of determining self from non-self for the duration of the individual s life. [Efroni & Cohen 2002] criticise the reductionist logic of [Langman & Cohn 2002] used to try and show the minimal model as the only reasonable model of the immune system s function. They state that the reasoning used by [Langman & Cohn 2002] neither matches the observed behaviour of the immune system, nor is an appropriate method for understanding its complexity. Instead, they consider the immune system to be a complex system [Bar-Yam 1997] and thus advocate the use of complex systems research tools to understand how the immune system works. They go on to concede that reductionism has provided immunology with much information regarding the specific agents of the immune system, but claim that it is not possible to deduce its functions by simply dismantling it. Instead, they believe that it is now time to acquire knowledge about the immune system from the bottom up, building models to examine how networks are created and properties emerge at the level of the system as whole. The differences between the research philosophies is summed up by Cohn [Cohn 2003] in his response to [Efroni & Cohen 2002]. [Cohn 2003] simply equates complex systems research to computer modelling and believes that the major difference between the two positions is that [Langman & Cohn 2002] wish to put in place a framework for the functioning of the immune system before computer modelling, whereas [Efroni & Cohen 2002] wish to build mathematical and computational models in order to discover such a framework. Cohn goes on to claim that: Biological complexity is not a problem solved by building a mathematical, philosophical, or computer programming web around any random collection of observations. This is in stark contrast to [Cohen et al. 2004] point of view: Immunology needs precise mathematical modeling and computer simulation to help us understand the emergence of immune specificity from the collective co-response. The interactions are simply too complex to be grasped by intuition 85

86 4.3 The Cognitive Immune System Implications for AIS Inspiration The immunological inspiration on which the majority of AIS are built, comes from either the clonal selection theory [Burnet 1959], or immune network theory [Jerne 1974]. Although these are competing and contradictory theories for the functioning of the immune system, they have both been able to inspire examples of AIS that satisfactorily perform their desired tasks. Thus, from an AIS perspective, both the clonal selection and immune network theories are equally useful for the purpose of providing the AIS designer with inspiration. It is important, however, that the theory chosen from which to take inspiration is appropriate, based on the behaviours required from the AIS. It seems clear that the success of an AIS practitioner owes much to the theories presented by the immunologist. However, many immune processes are not well understood, and there is little agreement amongst many immunologists regarding many of the key immune principles. This raises the question that if the immunologists are themselves unclear as to the functioning of many immune processes, where does this leave the AIS practitioner when deciding which aspects of immunological theory to take inspiration from? As highlighted in the previous paragraph, the competing theories of clonal selection and immune network have both provided inspiration for many successful AIS. We therefore suggest that alternative theories of immune processes can be equally useful for inspiring AIS. Indeed, we believe that the AIS practitioner should actively investigate these theories as they are likely to highlight new and different models of immune processes, and thus alternative ideas from which to take inspiration. These alternative ideas should lead to new functionalities for AIS. 4.3 The Cognitive Immune System In section 4.2, the cognitive immune model of [Cohen 2000a] was identified as a novel theory of the immune system. In [Cohen 2000b, Cohen 2001, Cohen et al. 2004] this model is elaborated, presenting a holistic immune system that is a complex, reactive and adaptive system. We chose to investigate this view of the immune system further as possible inspiration for AIS for two main reasons. First, the model is rich and contains many ideas that could inspire AIS. In addition to this, many of the ideas within this model are appealing from a computational point of view. We describe the cognitive immune system model though sections to 4.3.3, followed by detailing aspects suitable for AIS inspiration in section

87 4.3 The Cognitive Immune System The Role of the Immune System The cognitive view of the immune system states that the role of the immune system is to repair and maintain the body, rather than simply discriminating between self from non-self antigen. Following this argument the removal of pathogen is beneficial to the health of the body, thus defence against pathogen is just a special case of body maintenance. The rationale for believing in the maintenance property of the immune system is based on its perceived inputs and outputs. The inputs are simply the molecular shapes sensed when bonding to immune cell receptors occurs. The response to this input is the changing of immune cell states and activities causing cells to grow, replicate, die, move and differentiate along with the modification of tissue support and supply systems. Collectively these result in inflammation: the range of processes that the immune system has on the body. The output of the immune system can, therefore, be considered as inflammation. In order to achieve body maintenance, the immune system must select and regulate the inflammatory response according to the current condition of the body. This condition is assessed by both the adaptive and innate immune cells, which are required to recognise both the presence of pathogen (non-self antigen) and the state of the body s own tissues (self antigen). The specificity of the immune response, therefore, is not just the discrimination of danger or the distinction of self non-self, but the diagnosis of varied situations followed by the evocation of a suitable response. In summary, the maintenance role of the immune system requires it to provide three properties: Recognition : to determine what is right and wrong Cognition : to interpret the input signals, evaluate them, and make decisions Action : to carry out the decisions The properties of immune recognition and cognition are explained in the following sections, whilst the property of immune action can be achieved via the effector functions of the various immune cells Immune Specificity According to the clonal selection theory, immune specificity is a property of the somatically generated immune receptors of the T and B cells, which both initiate and regulate the immune response. Initiation is achieved via the binding between an antigen and a receptor that is specific to it. The response will stop only when 87

88 4.3 The Cognitive Immune System there is no antigen or receptor left for binding. [Cohen 2000b] points out, however, that immune receptors are intrinsically degenerate, being able to bind more than one antigen. Immune specificity, therefore, cannot be purely dependent on molecular binding as no one receptor can be specific to a single antigen. Instead, affinity, the strength of binding between a receptor and its ligand, is a matter of degree. In the cognitive immune model, immune specificity requires the diagnosis of varied conditions in the body and the production of a specific inflammatory response. This specificity emerges from the co-operation between immune agents (cells and molecules), despite receptor degeneracy and the pleiotropism and functionally redundancy of these immune agents. Pleiotropism is the property of a single immune agent to produce more than one effect, for example the same cytokine is able to kill some cells whilst stimulate others depending on various conditions. Functional redundancy refers to the ability of one class of immune agents to perform the same function as another. For example apoptosis can be induced by different immune cells. There are two processes provided by [Cohen 2000b] to explain the generation of immune specificity in the cognitive model: co-respondence and patterns of elements. Co-respondence is a process whereby the agents of the immune system respond simultaneously to different aspects of its target, and to its own response. This co-operation results in a specific picture of a pathogen emerging. As previously noted in section 4.3.1, immune receptors provide the input to the immune system by recognising molecular shapes. There are three different types of immune receptor that recognise different aspects of antigen. These are the antigen receptors of the T and B cells and the innate receptors of macrophages. The T cell receptors are restricted to recognising processed fragments of antigen peptides bound to a MHC molecule, whereas the B cell receptors (antibodies) recognise the conformation of a segment of antigen. The innate receptors of macrophages don t recognise antigen, but immune molecules. These molecules form a set of ancillary signals that describe the context in which the T and B cells are recognising antigen. These ancillary signals can be classified into three classes: The state of body tissues: some receptors detect molecules only expressed on damaged cells The presence and effects of pathogen: some receptors are unique to infectious agents such as bacterial cell wall The states of activation of nearby lymphocytes: some receptors detect immune molecules produced by lymphocytes 88

89 4.3 The Cognitive Immune System In addition to interacting with their target object, the T cells, B cells and macrophages use immune molecules to communicate their response to each other, and other tissues of the body. This forms an immune dialogue comprised of an on-going exchange of chemical signals between the immune cells. Subject to this exchange of information, they update their own responses accordingly, be it to increase or decrease the vigour of their response. The process of co-respondence is illustrated by figure 4.1. Patterns of elements can generate immune specificity owing to the specificity Figure 4.1: Co-respondence, as presented in [Cohen 2000b]. Rectangles represent objects, ellipses within rectangles represent states of objects, arrows designate directions of relationships and items separated by broken lines can be combined to generate joint products. 89

90 4.3 The Cognitive Immune System of a pattern extending beyond that of the individual elements that make up the pattern. Immune patterns are a complex arrangement of populations of immune agents, which, through their individual activity, produce a specific pattern of activity. For example, a pattern can emerge toward a particular antigen from the overlapping reactions of a population of degenerate immune receptors. Even though each immune receptor is non-specific to its target, the result of all the receptor reactions together will be unique, and thus specific to that antigen. Patterns can also be built with the help of immune agent pleiotropism and functional redundancy. The ability of different immune agents each being capable of responding to a situation with a number of different immune effects, allows more response options to be available than just having a single mapping between immune agent and its effect. Thus, specificity emerges through a co-operative pattern of degenerate, redundant and pleiotropic immune agents Immune Cognition [Cohen 2000b] claims the immune system achieves its role of body maintenance through a cognitive strategy. In biology, cognition is often related to the workings of the brain and properties such as awareness and conscious thinking. [Cohen 2000b] gives a more general definition of cognition as: a particular way of dealing with the world, or adjusting to the environment. Examples of systems in a biological organism include the respiratory, renal, nervous and immune systems. Of these, [Cohen 2000b] considers only the nervous and immune systems to be cognitive. They differ from the non-cognitive systems in the way they utilise internal images of their environment and the processes of self-organisation to make deterministic decisions. Self-organisation in cognitive systems is typified by the processes of learning and acquisition of memory. This involves the attainment of new behaviours and abilities from experience, which are then stored for later reference. [Atlan & Cohen 1998] proposes a formal theory of self-organisation in biological systems based on the net increase in information, which is summarised in the context of the immune system. This highlights two conditions needed in a system for selforganisation to occur: redundancy of information and unpredictability, i.e. noise. To generate new information, redundant copies of old information are perturbed by noise, resulting in the net gain of information. Both the adaptive and innate arms of the immune system are seen to selforganise via the creation of information from noise and redundancy. The adaptive arm self-organises with the construction of the T cell and B cell antigen re- 90

91 4.3 The Cognitive Immune System ceptor repertoires. Redundancy is provided by the proliferation of cellular clones, and noise by the random genetic mutations that produce the variable regions of the antigen receptors. These clones are then subjected to selection (T cell maturation in the thymus and the affinity maturation of B cells) to produce the immune receptor repertoire. Self-organisation of the innate arm of the immune system occurs by the generation of the actual response repertoire of the immune system via the fine tuning of the set of innate immune responses. This determines the types of response that will be associated to the signals perceived by the lymphocyte receptor repertoire. Through their interaction, entities are able to create abstract images of each other. For example, the teeth of an animal encodes a functional image of its diet. Cognitive systems build internal images that map the environment in which they exist, and help to inform the host on how to satisfy its needs. Biological cognitive systems such as the brain and the immune system make use of two types of image: innate images inherited from parents; and acquired images that are built from life experience. The innate images can be split further into three categories: feature detectors that filter system input; attention preferences that direct a cognitive system to particular types of input; and motive forces that cause the cognitive system to act, influencing the behaviour of the system. Examples of immune images are present in the process of co-respondence. Innate feature detectors and attention preferences filter out the information they require from their target. Innate motive forces exist in the form of the innate response repertoire that undergoes self-organisation to produce the actual immune response repertoire. This resulting response repertoire is classed as an adaptive image, providing the immune system with an internal map of its possible immune responses. Other adaptive images are generated by the immune system such as the patterns of immune agents distributed around the body. These include the entire T cell repertoire as an image of the T cell selections that have occurred in the body. To make a decision is to choose from a number of available options. In a deterministic system this is possible if the system has options available to it, and it has an internal history detailing its experience. This history can be formed out of the types of image discussed above. The decision making process then becomes the process of associating a particular circumstance encountered in the external environment, with a class of action present within the system in the form of internal images, such as an internal motive. The output generated will be deterministic, but as the elements of the internal history can be so many, and so complex, the choice appears to be unpredictable. 91

92 4.3 The Cognitive Immune System A cognitive system was described earlier as using internal images and the forces of self-organisation to make deterministic decisions. It is the outcome of these decisions that are proposed to perform body maintenance. The decision that the immune system must make is the choice of an inflammatory response to the perceived input. In the immune system, the decision making process becomes the association between the current immune environment perceived through the receptor repertoire (images of perceptions), with a suitable inflammatory response from the response repertoire (images of responses). This decision is proposed to be achieved through the process of co-respondence described above [Cohen 2000b]. Memory is also a property of cognitive systems and is formed via learning from past experiences. In the immune system, memory is expressed through the differences that are seen to arise between a primary and secondary infection from a particular pathogen. During the first infection, the immune system must expand its T and B cell repertoire to recognise the pathogenic antigen, and memory T and B cells are selected that express the innate effector response needed to kill the pathogen. During the second infection, the immune system can draw on its experience from the first infection to rid itself of the pathogen immediately. Thus, the memory T and B cells no longer need the full string of signals needed to produce the effector response in the first infection. Immune memory can, therefore, be considered as the replacement of the context of an infection. This memory is never in a final state, and continues to evolve during the lifetime of the individual Inspiration for AIS From the description of Cohen s cognitive immune model given in the previous section, a number of properties can be identified that could provide inspiration for AIS. It is noted that some of these ideas are present in immune network theory (section 2.2.4) and have already been used as inspiration for AIS. These properties, however, are still highlighted here as they form key parts of the cognitive immune model, and the way in which they are integrated into this model may highlight alternative inspiration. As highlighted in section 4.1, there are two levels of scale of biological inspiration that can be identified from the cognitive immune model: the high level concepts that describe the functioning and behaviour of the immune system; and the lower level mechanisms that are proposed to achieve the described functions. The high level concepts include: 92

93 4.3 The Cognitive Immune System Cognition : The immune system is a cognitive system that can make unconscious decisions dependent on the information presented to it. Images : Adaptive and innate images provide a history of past immune encounters. Self-Organisation : This encompasses the abilities of learning and memory. Maintenance : The role that Cohen sees the immune system as fulfilling, rather than the discrimination of self from non-self. Co-operation : The immune response is a collaborative effort between the innate and adaptive immune agents. Emergent Behaviours : The observed immune responses and properties, such as immune specificity, emerge from the functioning of immune agents rather than a one-to-one mapping between a receptor and an antigen. Examples of the low level mechanisms include: Multiple Immune Agents : Co-respondence involves the interactions of different agents such as macrophages, T and B cells. Signalling Networks : Immune agents communicate using an immune dialogue of signalling molecules. Feedback : Positive and negative feedback help to co-ordinate the immune response. Degeneracy : The degeneracy of antigen receptors provides a many-to-one relationship between the receptors and specificity of recognition. Pleiotropia and Redundancy : The pleiotropic and redundant nature of immune agents are also important in providing specificity of response. By highlighting these properties, it is then possible to identify application areas to which they might be applied. For example, the notion of the immune system as a concurrent and reactive maintenance system, lends itself well to application domains that operate in dynamic environments, such as embedded systems and robotics. For other properties such as degeneracy, however, it is not entirely clear how they can be beneficial to an AIS, so it can be useful to investigate these properties in more detail. This is carried out for degeneracy in the next section, identifying it as an important biological property and how it might be a useful algorithmic recognition property. 93

94 4.4 Degeneracy 4.4 Degeneracy In this section we look into greater detail at degeneracy by probing deeper into the research literature on the subject. In sections and 4.4.2, we identify degeneracy as a property prevalent in biology as well as the immune system. We then examine in section the influence degeneracy has had on AIS to date In Biology [Edelman & Gally 2001] define degeneracy in the context of biology as: the ability of elements that are structurally different to perform the same function or yield the same output. It is a ubiquitous biological property present at most levels of biological organisation, not just in the immune system. Degeneracy appears at each of the genetic, cellular, system and population levels of biology, for example (from [Edelman & Gally 2001]): Genetic code : different sequences can encode the same polypeptide. Protein folding : different polypeptides can fold to become both structurally and functionally equivalent. Intra-cellular signalling : parallel pathways of e.g. hormones transmit degenerate signals. Connectivity in neural networks : connections and dynamics are degenerate. Body movements : different patterns of muscle contractions can produce equivalent movements. Inter-animal communication : there are many ways to transmit the same message, e.g. via language. [Edelman & Gally 2001] argue that the omnipresence of degeneracy in biology is a result of it being conserved and favoured by natural selection. As natural selection can only operate on populations of genetically dissimilar organisms, many different overlapping genes and gene networks will tend to contribute to a phenotypic feature that is undergoing selection. Degenerate systems will thus be maintained as the selection process cannot assign the responsibility of this feature to any particular gene or network. 94

95 4.4 Degeneracy Degeneracy is also, according to [Edelman & Gally 2001], accompanied with complexity, playing a key role in complex biological systems. They define a complex system as: one in which smaller parts are functionally segregated or differentiated across a diversity of functions but also as one that shows increasing degrees of integration when more and more of its parts interact This relationship between complexity and degeneracy is investigated in earlier work by [Tononi et al. 1999], who use information theoretical concepts to develop functional measures of degeneracy and redundancy in a neural network 1. The difference between degeneracy (as defined above) and redundancy, is that redundant items capable of carrying out the same function are identical rather than degenerate items that are structurally different. By using a complexity measure for neural networks, [Tononi et al. 1999] were able to show that systems with high degeneracy also expressed high complexity. They also show that degeneracy is low in systems where the individual elements can affect the output independently, whereas degeneracy is high in systems where different elements can at the same time affect the output in similar ways and have independent effects. Importantly, degenerate elements were observed to produce different outputs in different contexts, thus making degenerate systems extremely adaptable to changes in their environment. Lastly, the relationship between degeneracy and redundancy was examined, showing that degenerate systems must express a degree of functional redundancy, whereas a fully redundant system is not necessarily degenerate In the Immune System Degeneracy in the immune system is exemplified by the degenerate nature of lymphocyte receptors. As highlighted in section 4.3.2, this has repercussions for immunological thinking by conflicting with the logic behind the clonal selection theory. A number of these repercussions are investigated in the this section. The concept of degeneracy in the immune system is not new. [Parnes 2004] provides a chronology for the use of the term degeneracy in immunology over the last 35 years. It is clear from this that there is no consensus on what exactly degeneracy is, and the word degeneracy has been used to explain a number of different concepts. For the purpose of the rest of this thesis, the definition of antigen receptor degeneracy given by [Cohen et al. 2004] is adopted: 1 The measures developed are considered applicable to any biological network or complex system. 95

96 4.4 Degeneracy [Antigen receptor degeneracy is the] capacity of any single antigen receptor to bind and respond to (recognize) many different ligands This definition is compatible with the general definition of degeneracy in biological systems given by [Edelman & Gally 2001] in section [Parnes 2004] argues that in the light of receptor degeneracy the challenge for immunology is to retain a meaningful explanation of immune activity. When taken seriously, the implication of degeneracy produces an alternate view of the immune system that does not fit with existing ideas. This can been seen from the description of the cognitive immune model given in section 4.3. As [Parnes 2004] points out, this model attempts to cover the gap between the immunologist s idea of an immune response and what is known about the properties of adaptive immune components such as the inherent degeneracy of antigen receptors. [Cohen et al. 2004] reports that the degeneracy of antigen receptors leads to two consequences for immune receptor recognition, both of which have been proven experimentally 2 : Poly-clonality : A single antigen epitope is able to activate different lymphocyte clones Poly-recognition : A single lymphocyte clone is able to recognise different antigen epitopes Most immune responses do not express extreme poly-clonality as mechanisms such as clonal competition exist to restrict it. Clonal competition should favour those lymphocyte clones whose receptors have the greatest affinity to an antigen epitope, thus they should proliferate over other clones. However, the existence of poly-recognition causes more problems for the traditional clonal selection theory view as it relies on the strict specificity of lymphocyte clones. The degeneracy of antigen receptors, therefore, provides the biggest challenge to the validity of the clonal selection theory. As an example of the power of receptor degeneracy, [Cohen 2001] provides a description of colour vision. The human eye possesses millions of colour receptors called cones, of which there are only three types (red, green and blue). These receptors are degenerate, each responding to broad range of light wavelengths, which overlap between the different cone types. Even though there are only three types of receptor, the human brain is able to perceive thousands of different and specific colours. Colour specificity, therefore, is not encoded by the cone receptors, but achieved via subsequent neuronal firings. Likewise, immune specificity 2 See [Cohen et al. 2004] for appropriate references 96

97 4.4 Degeneracy is likely to be encoded in the patterns of degenerate, co-responding lymphocytes and their allied cells, not in the initial clonal activation of lymphocytes. In addition to the cognitive immune model, other models exist that incorporate receptor degeneracy to try and explain the functioning of the immune system. Most notably from a computational view point, the model of [Leng & Bentwich 2002] describes the immune system as a fuzzy system. This is an attempt to compensate for the defects in the two-valued self non-self classification. The model keeps all the basic tenets of the clonal selection theory, but replaces the antigen-receptor selection and binding mechanism with a fuzzy recognition process. This process uses stimulation thresholds of a set of fuzzy lymphocytes functioning as a statistical clone for the activation of an immune response [Parnes 2004]. [Leng & Bentwich 2002] conclude that in a fuzzy immune system the response to self or non-self antigen may include many effector functions, not just the response or no response view of the original clonal selection theory. The fuzzy immune system view is shared by others such as [Sercarz & Maverakis 2004], who advocate the use of fuzzy logic models to help understand receptor degeneracy and its implication to the functioning of the immune system In AIS To the author s knowledge, there are no instances in the AIS literature where degeneracy, defined in the terms above in section 4.4.2, has been explicitly addressed. Fuzzy logic, however, has been used in association with AIS by a number of authors. [Alves et al. 2004] use an AIS to induce a set of fuzzy classification rules for the purposes of data mining. The discovered rules are of the form IF (fuzzy conditions) THEN (class), but fuzzy logic is not used as part of the AIS itself. [Gomez et al. 2003] use a real-valued negative selection algorithm to generate a set of fuzzy detector rules given a set of self samples. The detectors are then used to determine whether a new sample is self or non-self. Again, this approach does not incorporate fuzzy logic into the actual AIS. [Nasaroui et al. 2002] present a web data mining algorithm that does use fuzzy logic within the actual antigen-antibody matching mechanism. The algorithm is based on an immune network model by [Knight & Timmis 2001] called AINE, which uses artificial recognition balls (ARBs) to represent n-dimensional data items that use thresholds to match against antigens and other ARBs in the network. [Nasaroui et al. 2002] adapt the ARBs to represent fuzzy sets of data items rather than a single data item, removing the need for a crisp thresholding measure. It is important to note that algorithm performance appears to be the main mo- 97

98 4.5 Degeneracy and Patterns for AIS tivation behind the described works to introduce fuzzy mechanisms into an AIS. Each attempts to improve on perceived weaknesses in previous AIS approaches to an application by augmention with fuzzy logic. Work by [Kaers et al. 2003], however, approaches fuzzy logic from a slightly more immunologically inspiration perspective. They investigate antibody detectors based on fuzzy set theory to capture the graded nature of the physical matching between antibody and antigen. These detectors are, however, are only investigated in the context of an application, by comparing a pattern classifier with and without fuzzy inspired detectors. The work shows that the fuzzy morphology of the antibody can have a positive effect on a pattern classification algorithm. 4.5 Degeneracy and Patterns for AIS In this chapter we have been exploring the element of this thesis laid out in section to investigate alternative immune ideas for the inspiration of an AIS. Within the context of following the CFAIS described in section 3.5, we have been concerned with the identification of the immunology to provide the algorithm inspiration and subsequent probing of the immunological research literature to elaborate on that immunology. Our investigation has led us through the conflicting views many immunologists have regarding the immune systems and to an in-depth look at the cognitive immune model of [Cohen 2000b]. A key aspect of this model it the property of degeneracy that is being both discussed [Cohen et al. 2004, Parnes 2004, Sercarz & Maverakis 2004] and modelled [Tieri et al. 2007] by immunologists, but which has failed to filter down into AIS. Our inspiration for a new AIS, will therefore focus on immune receptor degeneracy and the concept of patterns of response as a mechanism for exploiting this degeneracy. Hence, the following definitions will be used for the remainder of this thesis: Degeneracy : This refers to antigen receptor degeneracy and to the capacity of any single antigen receptor to bind and respond to many different ligands (see section 4.4.2). This is a property of the chemistry of molecular binding, such that antigen receptors can bind many different antigens to different degrees. Pattern of Response : This refers to the concept of patterns of responding elements introduced by [Cohen 2000b] (see section 4.3.2). The perceived consequence of antigen receptor degeneracy is the lack of one-to-one specificity between an antigen receptor and an antigen. Patterns of response of degenerate receptors is one of the concepts used to explain how this consequence 98

99 4.5 Degeneracy and Patterns for AIS can be overcome, whereby antigen specificity emerges from the pattern of all responding degenerate immune detectors. Essentially, patterns of response enables the power of degenerate detectors to be exploited by establishing a many-to-many mapping between the antigen and degenerate detectors that recognise those antigen. In terms of an AIS the many-to-many mapping between detectors and the structures they are trying to detect (the target) is novel. The majority of AIS have a one-to-many or one-to-one mapping between detector and target, thus there is an apparent mis-match between what we know about the recognition properties of the real immune system, and those of AIS. This mis-match can be addressed by investigating a many-to-many mapping using the concept of patterns of response, whereby patterns emerge toward a particular target (antigen) from the overlapping reactions of a population of degenerate detectors. As the notion of detector and target is an inherent part of AIS, the idea of degeneracy and patterns of response should be applicable to the majority AIS and their applications. An AIS incorporating degenerate detectors and patterns of response will have consequences for the dynamics of its algorithm. Recognition will emerge from the collective response of a population of detectors and the algorithm will be required to integrate this population response in some way. Aside from the argument of biologically faithfulness, an AIS that incorporates patterns of degenerate detectors might be able to provide improved scalability over existing AIS. Greater scalability can be achieved as the capacity to discriminate patterns collectively by a set of degenerate detectors should be greater than by single detectors (see the example of colour vision in section 4.4.2). In the introductory chapter to this thesis we posed two main research questions (section 1.2.3) that we wished to address. The details outlined in this chapter led to the second of these questions: how can the concepts of degeneracy and patterns of response be incorporated into an AIS? It it this question that we address in the following three chapters. We start in the next chapter by investigating the issues of detector degeneracy and patterns of response in the context of a simple simulation case-study. 99

100 CHAPTER FIVE A Simulation of Patterns of Degenerate Detectors in a Lymph Node The previous chapter identified our desire to investigate the concepts of degeneracy and patterns of response as possible inspiration for a novel AIS. Within the context of the CFAIS, our next step, which is addressed in this chapter, is to build a simplifying abstract representation (in this case a simulation) that captures and investigates these concepts. First, in section 5.1 we identify the goal of the work presented in this chapter followed by a return to looking at immunology in section 5.2 to identify suitable biology for simulation construction. The work detailed in this chapter, therefore, encompasses both the probes and simplifying abstract representation stages of the CFAIS. In section 5.3, we establish the scope of our simulation and detail its construction from the immunology details in section 5.2. Then, in section 5.4 we describe our agent-based model followed by presenting the simulator in section 5.5. Section 5.6 explores the issue of patterns of response by detailing a set of experiments. The results of these experiments are concluded in section 5.7 within the context of our goal to develop a novel AIS following the CFAIS. In the conclusion of the work, the decision is made not to proceed further with the simulation presented in this chapter, instead, it establishes the goal for the work presented in the next chapter. 100

101 5.1 Goal 5.1 Goal The goal of this chapter is to develop a simulation to provide insight into how patterns of degenerate detectors might be incorporated into an AIS. We have identified receptor (detector) degeneracy and patterns of response as the inspiration for a novel AIS, based on the property of degeneracy being important across the biological domain. Detector degeneracy in the immune system is expressed by the recognition of antigens being a matter of degree rather than an absolute. Patterns of response have been suggested by [Cohen 2000b] as a way to produce immune recognition from a set of degenerate detectors. It is not entirely apparent, however, what form patterns of degenerate detectors might take in the real immune system. Before we can proceed to developing an AIS, we want to investigate whether patterns of response will emerge from a simulation built using simple bio-inspired building blocks, and if they do, what do they look like. The simulation presented in this chapter, therefore, is exploratory in the context of elaborating the ideas of degeneracy and patterns of response. We will examine behaviours of the simulator qualitatively to provide insight into patterns of response. To achieve our goal, we need to: Investigate immunology background to identify where patterns of response might emerge in the immune system. Build a simulation based on the outcome of this investigation, which incorporates the assumption that immune detectors are degenerate. Investigate whether any behaviours emerge that we can identify as being patterns of response from a set of detectors. Analyse the behaviours we observe with respect to our thesis goal of developing a novel AIS via the CFAIS. 5.2 Returning to Immunology In this section we examine the immune literature to identify a suitable system to model that might express patterns of degenerate cells responding to an antigenic stimulus. In section we identified lymph nodes as an example of a secondary immune organ, providing an environment where immune responses to antigen in the lymph may be triggered and develop. They therefore act as filters, capturing and responding to foreign antigen that have entered the body via 101

102 5.2 Returning to Immunology entry points such as the skin. If we are to assume that the antigen receptors of T H cells are degenerate, then the lymph node will be one place in the body where we would expect to see patterns of T H cell response. In this section we investigate this immunology further, first describing lymph nodes in more detail and then examining how T H cells become activated in lymph nodes Lymph Nodes The human body contains many hundred lymph nodes situated at various points in the lymphatic system. They are small bean shaped structures (about the size of a pea in humans) rich in both lymphocytes and antigen presenting cells (APCs) such as dendritic cells. Each lymph node is connected to the lymphatic system via a number of afferent lymph vessels through which lymph can enter the node, and a single efferent lymph vessel transports lymph away from the node. Lymph nodes are also connected to the circulatory system via a lymphatic artery and vein. It is through the lymphatic artery that lymphocytes (mainly naive T and B cells) enter the lymph node. As lymph drains though the node, any antigen present is captured and processed by APCs for presentation to T H cells. An adaptive immune response is initiated if the antigen presented by the APC activates a T H cell. Antigen may also be transported into the lymph node by APCs that have captured the antigen in the periphery of the body and then migrated to the node via the lymphatics [Goldsby et al. 2003, Abbas & Lichtman 2003]. The lymph node can be functionally separated into three distinct areas each supporting a different cellular environment: the cortex, the paracortex and the medulla. Figure 5.1 demonstrates the arrangement of these areas in a lymph node. The cortex supports mainly B cells and various APCs (such as macrophages and dendritic cells), the paracortex supports mainly naive T H cells and dendritic cells, and the medulla contains mostly lymphocytes including the antibody producing plasma cells. As lymph drains through the lymph node, it slowly percolates though each of the three regions. In the paracortex, the dendritic cells trap and process any foreign antigen and present it via MHC-II to the naive T H cells. These T H cells then help activate B cells on the edge of the paracortex leading to B cell proliferation. This proliferation takes place in the germinal centres of the cortex, and results in antibody producing plasma cells, some of which migrate to the medulla. This whole process results in the lymph leaving the lymph node being enriched with antibodies and lymphocytes [Goldsby et al. 2003]. The segmentation of the lymph node into the three different areas is due to the presence of specific signalling molecules called chemokines. Both naive T H cells 102

103 5.2 Returning to Immunology Figure 5.1: Areas of a lymph node from [Goldsby et al. 2003] showing the cortex, paracortex and medulla. and APCs activated due to exposure to antigen, express the same cell-surface receptor for a chemokine produced only in the paracortex. This has the effect of attracting both of these cell types into the same area, thus enabling their interaction. Likewise, naive B cells are concentrated in the cortex as they express a receptor for a different chemokine produced only in the cortex. Once T H and B cells have been activated by antigen/apcs, they lose their chemokine receptors from the cell surface, and migrate towards each other. Thus the structure of the lymph node keeps each of the T and B cells populations in close proximity to the appropriate APCs and apart from each other until they are in a state in which they are ready to interact with each other [Abbas & Lichtman 2003] T H Activation in a Lymph Node Naive T H cells become activated by APCs presenting MHC-II to which antigenic peptides are bound (MHC-P). In order for this activation to take place, a certain level of stimulation is required, an issue determined by affinity and avidity. In section 2.2 we introduced affinity, which is simply the strength of binding between a single binding site (e.g. T H cell receptor) and a single ligand (e.g. an MHC-P complex). It can be quantitatively measured using a dissociation constant K d, which represents the concentration of a molecule X required to occupy half of the combining sites of another molecule Y present in a solution. Hence, a smaller 103

104 5.3 Model Basis K d represents a stronger or higher affinity [Abbas & Lichtman 2003]. Avidity on the other hand is a measure of the strength of binding between molecules or cells when there is more than one binding site present [Janeway et al. 2001]. T H cells become activated when the concentration of MHC-P complexes on an APC reaches a sufficient threshold level [Anderton & Wraith 2002]. In other words, T H cells become activated when an avidity threshold is met. This means a T H cell activation is affected by both the affinity between the T H cell receptor and antigenic peptides presented by the APC, and the concentration of these ligands present. It is possible, therefore, for an APC presenting high concentration of MHC-P complexes with weak affinity to activate a T H cell, and conversely an APC presenting a low concentration of MHC-P complexes with high affinity not to activate a T H cell. Once a naive T H cell has become activated it initiates a process of cellular proliferation and differentiation into effector T H cells. These effector cells play a crucial role in activating both B and T C cells, which are then able to neutralise pathogens. 5.3 Model Basis In this section we investigate the basis for our model by first outlining in section the type and scope of model presented in this chapter. Then, in section 5.3.2, we extract the various biological components and their behaviours from the biological description above that will form the basis of the implementation details that will follow Scope The previous section described how the activation of naive T H cells in the paracortex of the lymph node provides the initial recognition event of the adaptive immune response to lymph-borne antigen. Based on this and the goal presented in section 5.1, we identify our model as an agent-based model of the activation of T H cells in the paracortex of a lymph node, assuming T H cell receptors are degenerate. This degeneracy is expressed simply in the rules of interaction between the TCR with the MHC-P expressed by APC agents, such that we allow a bind to occur between any TCR, MHC-P pair that will subsequently produce a proportional level of excitation to the T H agent. By building such a model, we expect to see different sets of T H cells becoming activated to different antigens. It is these patterns of response that we test for and analyse. We choose to model and simulate using an agent-based approach for many of 104

105 5.3 Model Basis the reasons outlined in section An agent-based approach allows us to represent individual cells and molecules as agents each with their own behaviours, rather than well mixed populations assumed in population-based models. We believe an agent-based approach is also conceptually closer to the real biological system as direct analogies exist in the model and less abstractions will need to be made. In the model, we choose to reduce the dimensionality of the space to two spatial dimensions rather than three. Our justification is that doing this reduces the complexity of the implementation, whilst still enabling the elements of the system to move and interact in a non-trivial way. Also, we assume that biologically, the structure of paracortex is homogeneous (a fair assumption we believe), thus any 2-dimensional plane through the space will be structurally similar. We specifically take into account in the model the idea that the biological system is inherently noisy, as this may impact on the types of pattern of response that may emerge. By noisy, we refer to behaviours such as cellular interactions that appear non-deterministic at a higher level, with cells constantly moving and interactions occurring when cells are physically adjacent. Also, APCs can present different levels of antigen to T H cells, thus we take into account the idea of avidity described in section above, whereby APCs can ingest and present different levels of antigen to T H cells. We decide to implement our own agent-based model and simulation rather than use a pre-existing simulation tool such as those outlined in section Whilst we believe these tools could model and simulate our system, the main advantage of implementing our own is that we have full control of the behaviours of the system and can therefore build exactly the type of simulation we decide is necessary. We will therefore have more confidence that the simulation is doing what it has been designed to do as there are less unknowns regarding how technical issues such as agent interactions are handled Biological Components and Behaviours The first step in constructing an agent-based model is to identify the relevant components and their behaviours from the biology described in section 5.2. The following components will form the basis for structures and agents of the simulation: Lymph Node : This provides the location where cells and molecules interact. It consists of three main areas: a paracortex, a cortex and a medulla. 105

106 5.3 Model Basis Paracortex : The paracortex area of the lymph node emits a chemokine to attract naive T H cells and APCs presenting MHC-P, and thus provides the area where they interact. T H cell : These cells can be in different behavioural states and present TCRs that interact with MHC-P on APCs. APC : These can ingest any number of antigens, which are presened as peptides (P) bound to MHC. A variable number of these MHC-P complexes can be presented to the TCRs on T H cells. Like T H cells, APCs can be in different behavioural states. Antigen : These can be ingested by APCs and subsequently presented as MHC-P complexes. MHC : The receptors of the APCs that bind with peptides to form MHC-P complexes. TCRs : The receptors of T H cells that bind to MHC-P complexes to provide excitation to the T H cell. Chemokine : This is emitted from the paracortex, which produces a gradient to attract the naive T H cells and APCs presenting MHC-P to the paracortex. From the description in section 5.2, there are two cells that exhibit the most complex behaviours. These are the T H cells and the APCs. To capture their main behaviours, we present state diagrams similar to those used in UML [Fowler 2000]. Figures 5.2 and 5.3 are state diagrams that summarise the behaviours for the T H cell and APC agents respectively. Both these state diagrams begin with the start state (the solid circle) followed by a create transition that delivers the naive cell into the lymph node. It is from this point on that we are interested in the behaviour of the T H cell and APCs. This assumes that anything that happened before the cell arrived in the lymph node does not affect the behaviour of these cells, and that all cells arrive in the lymph node in the same cellular state. This is an important point to clarify as it makes a big assumption about the biology. However, it is a suitable assumption as it would be intractable to model the behaviour of these cells throughout their entire lifecycle when we are only interested in subset of the agent behaviours. The behaviour of the T H cells represented by figure 5.2 can be summarised as follows. The T H cell enters the lymph node via the create transition and is in the naive state and attracted by chemokine. In this diagram, the naive state is 106

107 5.3 Model Basis Figure 5.2: T H state model showing T H cell activation behaviours. a super-state represented by the large box that contains sub-states and transitions. Once in the attracted by chemokine state, the T H cell has two possible behaviours: move and continue to be in the attracted by chemokine state; or bind APC and transition into the interact with APC state. From the interact with APC state, the T H cell again has two behaviours open to it: disassociate APC and move back to the attracted by chemokine state; or activate and transition out of the naive super-state into the activated state. In the activated state, the T H cell only has the move randomly transition that keeps it in the activated state. The behaviour of the APCs represented by figure 5.3 can be summarised as follows. The APC enters the lymph node via the create transition and into the not presenting MHC-P state (i.e. a naive APC see section 5.2.1). From this state, the APC has two behaviours available: move randomly and stay in the not presenting MHC-P state; or ingest antigen and transition into the presenting MHC- P super-state and the attracted by chemokine sub-state. Once in the attracted by chemokine state, the APC can do one of three things: ingest antigen and stay in the attracted by chemokine state; move and continue in the attracted by chemokine state; or bind T H cell transiting to the interact with T H state. There is then one behaviour for the APC in this state and that is to disassociate T H and transition back into the attracted by chemokine state. 107

108 5.4 Agent Based Model Figure 5.3: APC state model showing presentation and activation behaviours. Clearly, in the real biological system there are many additional behaviours and states, but we have created these diagrams from the perspective highlighted above in section 5.2. Hence, we are not interested in what happens after a T H becomes activated, but the behaviour of the cells that lead up to this event. 5.4 Agent Based Model Having outlined the scope and background biology of the model in section 5.3, we present here how the agent-based model is constructed. The two main aspects of the agent-based model that we must identify are the agents themselves, and the environment in which they interact. In section we identified the main components of the biology in which we are interested, and it is these components that will form the agents and their environment in the agent-based model. As we are interested in the behaviours of the T H cells and their interaction with APCs presenting antigen, it is sensible that these three components (T H cells, APCs and 108

109 5.4 Agent Based Model antigen) are the agents in our system. The MHC component will be an element of the APC agent and the TCR component will be an element of the T H agent. The environment, therefore, will consist of a lymph node that contains a paracortex region and a chemokine gradient. Based on the agent and environment aspects just identified, we elaborate here on how these have been implemented in our agent-based model. In section we detail the lymph node environment, followed by how the chemokine gradient has been modelled in section Finally, we describe the three agent types in section Lymph Node Environment The approach we have taken to model the environment is similar to that of [Maree et al. 1999] who have modelled the movement of Dictyostelium disciodeum amoebae due to a chemical gradient. They use a hybrid cellular space/partial differential equation model, where the cellular space is used to represent the physical details of the amoebae and the partial differential equation models the chemical gradient. In our model, the lymph node is represented as two separate layers in a cellular space: a chemokine space and an agent space. This agent space provides the environment where the agents of the model can move and interact. The chemokine space models the action of the chemokine produced by the paracortex to attract naive T H cells and APCs presenting antigen. The cellular space layers are implemented as 2-dimensional grids of cells, with the agent space placed directly on top of the chemokine space. Both grids therefore share the same dimensions and co-ordinate system, for example grid reference (2, 3) in the agent space would relate directly to the same grid reference in the chemokine space. Figure 5.4 depicts this two layer space model. Cells in the agent space layer can be either empty or contain one of the three agents types: antigen, APC or T H cell. The contents of the cells in the chemokine space are integer values that model a concentration of chemokine. The neighbourhood used for the agent and chemokine spaces is the Moore neighbourhood which consists of nine cells: the original cell and the eight surrounding neighbour cells (those to the north, northeast, east, southeast, south, southwest, west and northwest) shown in figure 5.5. Wrap-around occurs between the right and left hand edges of the cellular spaces, but not at the top and bottom. This allows us to model the flow of agents from top to bottom through the lymph node. In abstract terms, we can consider the top of the space as modelling the afferent lymph vessels (the entry points to the lymph node), and the 109

110 5.4 Agent Based Model Chemokine Space APC Ag T H Ag APC Ag Agent Space Figure 5.4: The two layers of the lymph node model cellular space with example values. NW N NE W C E SW S SE Figure 5.5: The Moore neighbourhood where C = central cell, N = north, NE = northeast, E = east, SE = southeast, S = south, SW = southwest, W = west and NW = northwest. bottom as modelling the efferent lymph vessel (the exit point of the lymph node). Time is represented in the model by discrete steps (iterations). At each iteration all the cells in the chemokine space update, agents move in the agent space, and agent interactions occur Chemokine Space and Paracortex In the chemokine space model, each cell holds an integer value representing a chemokine concentration. A diffusion rule is then implemented so that a chemo- 110

111 5.4 Agent Based Model kine gradient emerges. As mention in the previous section, the idea of splitting the behaviour of a chemical gradient into a different layers in a cellular-based simulation is not new, but typically the gradient is generated using equationbased techniques [Maree et al. 1999]. The approach we take here is an alternative based on the physical laws of equilibrium, such that each grid cell attempts to reach a local equilibrium at each update. We deem this to be acceptable as our aim is to simply to provide the qualitative behaviour of a chemokine gradient which can be used by the model agents. In this case, the gradient allows the agents to perform chemotaxis: a directed random walk towards the production of the attractant [Goldsby et al. 2003]. At each iteration of the chemokine space model, the chemokine values update according to a diffusion rule, which is applied to all cells. The diffusion rule works as follows: for each cell, integer division by 9 is performed on the chemokine value and the result is shared between the nine neighbours as defined above. The remainder, R, from this division is then shared out randomly between the nine neighbours. An example of how this would work for one cell is shown in figure 5.6. Here, the original value of 95 is divided by 9 and each neighbour is assigned a value of 10. Next, the remainder of 5 is shared randomly to the neighbour cells. When applied to every cell, the effect of the diffusion rule in the space is to smooth the chemokine concentration over the entire chemokine space, whilst leaving a level of variability at the local level. To model the production of chemokine molecules in the paracortex, an area is defined in the middle of the chemokine space in which chemokine can be generated. To provide a stable chemokine gradient, the level of chemokine that exits the space at the top and bottom of the chemokine space during an iteration is re-injected into random locations in the paracortex region. A summary of the chemical space update rules is given by algorithm 5. We return to analyse the behaviour of the chemical space in section Figure 5.6: An example of the stages involved in the diffusion rule for a single cell in the chemokine space. 111

112 5.4 Agent Based Model begin Create a two-dimensional grid of cells, chem space, to hold chemokine values Define a sub-region of chem space, chem prod, as a chemokine production area Populate each cell in chem space with a randomly generated integer value between 0 and a user defined maximum foreach iteration do Initialise a copy of chem space, chem space copy, with each cell set to 0 Set level of overspill chemokine, overspill, to 0 foreach cell, c, in chem space do Integer divide value in c by 9 and assign the quotient to q and remainder to r foreach moore neighbour (show in Figure 5.5), n, of c do if n is outside chem space then Increment overspill by q else Increment the value of cell n in chem space copy by q end end foreach r do Generate a random number in the range 0 to 8 and increment the associated moore neighbour by 1 end end Copy values in chem space copy to chem space foreach overspill do Choose a cell in chem prod at random and increment its value in chem space by 1 end end end Algorithm 5: Algorithm for updating chemokine values in the chemokine space Agents As highlighted above, there are three agent types in our model: APC, antigen and T H agents. Both the antigen and T H agents have a Real-valued vector associated with them which models their molecular shape (in the case of the T H agent, this is the shape of the TCR). This shape is used when calculating the affinity of binding between antigen and TCR. In our model, we use the Euclidean distance (see equation (2.1)), which is typical in AIS as described in section The behaviours of the three agent types are as follows: 112

113 5.4 Agent Based Model Antigen agent : The movement of the antigen agents is designed to mimic the movement of real antigen draining through a lymph node from top to bottom. Therefore, they can only move down or sideways in the agent space, i.e. movements to the east, west, south, southeast and southwest Moore neighbours. At each update, a neighbour is chosen at random to move to. Once an antigen leaves the bottom of the agent space it re-enters the top of the agent space at a randomly determined cell location to mimic a constant flow of antigen through the node. The antigen agents do not themselves initiate any interactions with other agents. APC agent : The states expressed in figure 5.3 outlined above in section determine the behaviours of APC agents. The movement rule of APCs is determined by its state, with active APCs (those presenting MHC-P) following the chemokine gradient to the paracortex, otherwise they move to a randomly determined Moore neighbour. When following the chemokine gradient in the chemokine space, an APC agent will move to the neighbour with the highest chemokine value. Upon encountering an antigen agent in its Moore neighbourhood, an APC agent will ingest the antigen agent removing it from the agent space. The APC s ingested antigen count will then be incremented. This count directly relates to a concentration of MHC-P that is presented by the APC, and there is no limit to the amount of MHC-P an APC can present. The MHC-P is simply represented by the Real-valued shape of the original antigen agent that was ingested. T H agent : Like the APC agents, the states expressed in figure 5.2 outlined above in section determine the T H agent behaviours, and the movement rule is determined by its state. Naive T H agents follow the chemokine gradient to the paracortex, whilst activated T H agents move to a randomly determined Moore neighbour. T H cell agents each have an activation threshold that is used to determine when they change state from naive to activated. Only naive T H cell agents interact with APC agents, and these APC agents must be in the activated state, presenting an MHC-P shape vector. When such an APC agent is in the Moore neighbourhood of a naive T H cell agent, the avidity between the TCR shape and the MHC-P is calculated. This is done by multiplying the MHC-P concentration of the APC by the affinity between the TCR and MHC-P. This avidity is then compared to the activation threshold of the T H cell agent. If the avidity is above the activation threshold then the T H cell agent becomes activated. 113

114 5.5 The Simulator 5.5 The Simulator In the previous two sections we have specified our agent-based model, detailing the agents, their behaviours, and their environment. In this section we detail a simulator that implements this agent-based model, and will allow us to investigate the desired patterns of response behaviour outlined in section 5.1. First, we detail the Java simulator that has been implemented in section 5.5.1, followed by the calibration of the chemokine space settings in section Lastly, we detail the behaviours observed in an example run of the simulator in section Java Implementation The simulator has been designed to implement the agent-based model outlined above and allow experimentation into how a set of T H agents reacts in a system with a single antigen type. Over multiple simulations we can then compare how different T H agent sets react to different antigen shapes. The simulator has been implemented in the Java object-oriented programming language and can be run either interactively via a graphical user interface (GUI) or on the command line allowing for batch simulation. Figure 5.7 shows the simulator GUI with the parameter settings and run details on the right side of the window. The left side Figure 5.7: A snapshot of the lymph node degeneracy simulator user interface. 114

115 5.5 The Simulator shows the space model containing the overlaid agent and chemokine space. We believe an object-oriented language to be a suitable choice of implementation language as the agent types of our model naturally translate to object classes of the simulator. Additionally, we can describe various behaviours such as movement in a generic way and have agent classes inherit from these. The simulator can be configured via a number of user-defined parameters that are outline in table 5.1. These parameters relate to both the configuration of the agents as well as the space model. Given a set of parameters, the simulator executes the algorithm given in algorithm 6 to produce a simulation run. The algorithm starts with an initialisation phase where the chemokine gradient in the chemokine space is first established (pre-itns), then the required number of agents are injected into the agent space. From here the simulation is run Table 5.1: Parameters for the Java simulator of the agent-based lymph node simulator. Parameter Symbol Description Width w Width of the cellular space in number of cells. Height h Height of the cellular space in number of cells. Number of Pre-Iterations pre-itns Number of simulation iterations before agents inserted. Number of itns Number of simulation iterations once agents Iterations Chemokine Producer Percentage Maximum Chemokine Level Number of APCs Number of Antigens Number of T H Cells Recognition Threshold Antigen Shape T H Cell Receptor Shapes chem-prod chem-max apc-num ag-num th-num recog ag-shp th-shps inserted. Percentage of total chemokine space set to be the chemokine producing area in the centre of the chemokine space. Maximum allowed chemokine value of a chemokine space cell upon initialisation of the simulator. Number of APC agents to be inserted. Number of antigen agents to be inserted. Number of T H cell agents to be inserted. Avidity threshold to determine whether a T H cell becomes activated upon interaction with an APC. Vector of real numbers that represents the antigen shape. List of vectors that represent the unique receptors for each T H cell in the simulation. 115

116 5.5 The Simulator begin # Initialisation Phase # Initialise chemokine space, chem-space, with random chemokine values (see algorithm 5) foreach pre-iteration in pre-itns do Update chemokine in chem-space according to algorithm 5 end Generate apc-num APC agents, ag-num antigen agents and th-num T H agents Inject agents into random positions in the agent space, agent-space # Interaction Phase # foreach iteration in itns do Update chemokine in chem-space according to algorithm 5 foreach APC, antigen and T H agent do Move agent according to the movement rules of agent type end foreach APC, antigen and T H agent do Interact with other agents in the Moore neighbourhood according to the interaction rules of agent type end Collect and display agent statistics end end Algorithm 6: Algorithm for the agent-based Java simulator. for the required number of iterations (itns) and at each iteration, the chemokine space is updated, agents move then interact, and agent population statistics are updated along with the display Chemokine Space Calibration The update rules for the chemokine space were described in section 5.4.2, along with the desired behaviour of a gradient that allows agents to perform chemotaxis and move in a purposeful but random way towards the source of the chemokine. To achieve this we need the overall effect of the chemokine space model to provide a stable chemokine gradient flowing out of the paracortex region, whilst containing a small amount of variability to the chemokine values at the local level. By identifying parameters that achieve this gradient behaviour, we can use these when investigating the agent response behaviour in the next section. There are five parameters in table 5.1 that affect the behaviour of the chemokine space: h, w, pre-itns, chem-prod, and chem-max. Figure 5.8 shows a chemokine space with w = 25, h = 25 that has been initialised with random val- 116

117 5.5 The Simulator ues at each cell between 0 and 100, where lighter cells depict higher chemokine values. After 25 iterations a gradient like that shown in figure 5.9 forms, which remains stable for subsequent iterations. The height (h) and width (w) parameters affect the number of iterations before a stable gradient forms, with larger values requiring more pre-itns for the gradient to even out over the space. A stable gradient will form after a number of iterations that is greater than or equal to the larger of h and w, therefore, pre-itns should be set to this value. The chem-prod parameter determines the size of the chemokine production area. The lower the value, the less cells there are that produce chemokine. This has the effect of increasing the chemokine value of these cells as there are less cells for chemokine to be added to once it leaves the chemokine space from the top and bottom. Consequently, the gradient between the highest chemokine values (in the centre of the space), and the lowest values near the top and bottom, increases. We can see this in figure 5.10, which shows a thin central band of chemokine producing cells in a 25 by 25 grid with chem-prod set to 5%. If we compare to figure 5.9, which has chem-prod set to 25%, we see a much wider chemokine producing band. The chem-max parameter affects the smoothness of the chemokine gradient. The lower the value, the coarser the gradient with increased local variability between cell values. This is illustrated with figures 5.11 and 5.12, which show the stable state of the chemokine gradients for a chem-max = 10 and chem-max = 1000 respectively. Having investigated each of the parameters that affect the chemokine gradient, we can identify settings that will produce the desired chemokine gradient behaviour mentioned above: We need to set pre-itns to be greater than or equal to the larger of height (h) and width (w) parameters so that the gradient has time to form. We can set chem-prod to equal 25% as this gives us a visually suitable paracortex area with a chemokine gradient that decreases to the top and bottom boundaries. We can set chem-max to equal 100 as this gives us an acceptable balance between a smooth gradient, but with enough local variability for agents that follow it to achieve a purposeful random walk. 117

118 5.5 The Simulator Figure 5.8: Chemokine space of lymph node model upon initialisation, where w = 25 and h = 25. Lighter cells represent higher chemokine values. Figure 5.9: Chemokine space of lymph node model after 25 iterations, where w = 25, h = 25, chem-max = 100 and chem-prod = 25%. Lighter cells represent higher chemokine values. 118

119 5.5 The Simulator Figure 5.10: Chemokine space of lymph node model after 25 iterations, where w = 25, h = 25, chem-max = 100 and chem-prod = 5%. Lighter cells represent higher chemokine values. Figure 5.11: Chemokine space of lymph node model after 25 iterations, where w = 25, h = 25, chem-max = 10 and chem-prod = 25%. Lighter cells represent higher chemokine values. 119