Agent-Based Architecture of Selection Principle in the Immune System

Transcription

1 Agent-Based Architecture of Selection Principle in the Immune System Yoshiteru Ishida Graduate School of Information Science Division of Applied Systems Science Nara Institute of Science & Technology Ikoma, Nara, Japan Phone: , Fax: January 13, 2000 Abstract Based on some features of the immune system (the selection-based mechanism compatible with Edelman s selectionist principle, self/nonself-reference and negative/positive selection), we proposed the immune algorithm. The algorithm proceeds in three steps: diversity generation, establishment of self-tolerance, and memorizing non-self. The algorithm may be used typically to deal with the system by distributed agents where the system (the self) as well as the environment (the non-self) are unknown or cannot be modeled or difficult to treat even when modeled. Agent-based architecture is proposed with the application to adaptive system where the knowledge about environment is not available. Adaptive noise neutralizer is formalized and simulated for a simple plant. Keywords: immune algorithm; agent-based architecture; selection system; adaptive control; disturbance rejection.

2 1 Introduction Mathematical models of immune systems have been studied in mathematical immunology focusing on population dynamics of specific cells and antibody, inspired by the concept of idiotypic network [1]. Recently, many studies ([2, 3, 4, 5, 6] to mention only few) tried to extract information processing principles from the immune system for the application to information processing systems. In this paper, rather than building a detailed mathematical model of immune systems, we tried to explore the principle in immune system focusing on the information processing capabilities as done in artificial neural networks and genetic algorithm. The motivation of our research is to (1) explore the principle of information processing done in the immune system, and (2) develop information models based on the principle in immune systems, as the neural net is developed based on that in brain systems. Our aim, therefore, is to build new information models, rather than to build a precise model of the immune system. Although complete imitation of biological information processing may not lead to the good information processing by machine, some of their sophisticated way of information processing gives us a new paradigm. 2 Immune Systems and Information Processing Important findings in immunology from a viewpoint of information processing may be summarized under the following keywords: Specificity; Diversity; Tolerance; and Memory. We will discuss implications of each of the features above. In the discussion below, we mention the following principles of biological systems from the viewpoint of information processing that can be extracted not only from the immune system but from biological systems in general in their cell level. E The units (cells) are homogeneous in structure and its potential, but it will be specialized in its function. E The units can self-replicate with mutation and crossing-over. E The units can change its attributes such as mutation rate, life span, reproduction rate etc., triggered by some events. As for diversity, it is novel to artificial systems to prepare diversity beforehand, since economy and resource restriction is critical for artificial systems. However, this feature certainly seems to be imperative under the situation where the system is exposed to totally unpredictable environment. This feature is made possible by the other principle of biological system; homogeneous unit can self-replicate with mutation and crossing-over. Generation of diversity by the genetic recombination may potentially give and insight to building new information models, since most of the current models depend upon copying for storage and transfer of information.

3 Tolerance and Memory can be regarded as the adaptation process to the self and the non-self, respectively. Adaptation to self seems to be first carried out to be insensitive to the self. In this sense, acquiring self-tolerance can be seen as a learning process for longterm memory and memorizing non-self can be seen as a learning process for short-term memory. For the agent-based architecture, more importance is placed on the principle of biological system that the unit can change its attributes such as mutation rate, life span, reproduction rate etc., triggered by some events. This paper will demonstrate the agent-based architecture in the application to the adaptive noise neutralizer presented in section 6. The immune algorithm [7], is neutral to the architecture as presented in section 5. So far we have discussed informational features of the immune system, which will be elaborated as an immune algorithm in the next section. We also extracted some principles of biological systems, which may be common to several levels (i.e. such as cell, individual, and social group) for self-organizing system. Although more elaboration and refinement is required for the extracted principles, they would give some implications for artificial systems, which will not be discussed in this paper. At this point, we note that although we focus on the specific feature of the immune system, its information processing seems to share common background with many other parallel distributed systems 1. In fact, compared with neural networks, the immune system is dedicated to maintain material identity of the self, while neural system is dedicated to mental identity of the self. In this level, genetic system may be dedicated to maintain the identity of species. 3 Selection-Based Mechanism of Recognition and Categorization Edelman proposed The Theory of Neural Group Selection (TNGS) or Neural Darwinism [3] where he claimed that selection based (as opposed to instruction based) recognition mechanism can apply to the neural systems. His argument is that recognition and categorization carried out in the unlabeled world should depend on selection as done in the immune system. Conventional AI assumed labeled world and used labeled primitive for knowledge representation. However, both neural systems and immune systems can recognize and categorize being confronted with unlabeled world at first. He and his colleague demonstrated 1 General character of biological information processing is that it has physical and material basis by interaction and synthesis of high molecules as opposed to pure information basis found in the computer. The immune system is not exception. Recognition is done by physically moving units (i.e. lymphocyte). Communication among units are done physically by interleukin. Its recognition is determined by physical pattern of recognizing and recognized site (i.e. paratope and epitope). Its action is physical (i.e. removal of non-self and keeping physical identity of the self.)

4 that recognition and categorization in unlabeled world is actually possible with TGNS by building the system; Darwin, Darwin II, and Darwin III [8]. In fact, his work aims at more ambitious one: building and explaining consciousness by TNGS as stated in his series of publications [3, 8, 9]. Our proposal of adaptive mechanism extracted from the immune system is basically compatible with the idea behind TNGS except that our adaptive mechanism is more biased to the following character of the immune system: self/nonself-reference and positive/negative selection. His definition of recognition is the continual adaptive matching or fitting of elements..., a matching that occurs without prior instruction [9]. By this kind of recognition (not by the template matching of conventional AI), it is possible to deal with unlabeled and ever changing environment. We take up the task of adaptive noise neutralization which is essentially unlabeled task with changing environment. Other than not requiring prior explicit information ( instruction ) [9], he stressed that there is no explicit information transfer between the environment and organisms... in his view of evolution. It is by this selection-based and implicit transfer of information combined with the recognition that can deal with modeling in engineering and knowledge transfer in AI. To demonstrate this fact, we take up the task of adaptive noise neutralization, since modeling the environment (noise) explicitly is impossible. 4 The Immune Algorithm: The Immune System as a Super Adaptive System The significant character of the immune system is its super adaptiveness: it continuously adapts not only to the changing environment (non-self) but also to the changing self. Continuous adaptation is driven by the diversity continuously fixed at the top of the system, motivated by the concept of somatic hyper mutation for generating the diversity of antibody [10]. Thus, the immune system assumes the changing self as well as the changing non-self. Neural networks and even genetic algorithm assumes an adaptation of the set of parameters to a given problem. However, the immune algorithm that will be presented in this section assumes always changing parameters to a set of problems or to continuously changing set of disturbances (i.e. changes themselves). Thus, the problems and tasks for the immune algorithm is quite different from those for the neural networks and the genetic algorithm. The immune algorithm is not meant for a fixed problem or task; but rather for ever changing problems or for changes themselves in the environment. (Genetic algorithm should also treat the similar problems to those by the immune algorithm if analogy with biology is seriously taken. However, genetic algorithm seemed to be used for a fixed problem or for an optimization of a fixed problem.)

5 To illustrate our immunity-based approach, we use the metaphor of weighing. There are roughly two methods of weighing objects: one is using a scale that requires a sophisticated central processing mechanism that maps weight to the number; another is using a balance that uses many types of balance weights and compare the weight of the target object with some set of balance weights. The former is often accurate and efficient, once the mechanism of mapping is devised. However, the latter method of object-against-object offers a distributed and robust way of weighing. The network view of the immune system can be understood with this metaphor; regarding many types of balance weights as recognizing agent (immune related cells such as B-cells and T-cells that react only with specific antigen) and action of weighing by balance as recognition by paratope and epitope with spatial complementarity. The result of recognition is used to activate other recognizing agent, similarly to the fact that the result of balance is used to determine more appropriate balance weight against the target object. This object-against-object weighing is more robust, since the weighing mechanism is a simple comparison and that information is distributed into many balance weights. Although this metaphor is rough, it seems to reflect many important informational features of the immune network. However, it may be more appropriate to consider the immune network more sophisticated than the simple balance weighing system in the following points: (1) Each agent has not only information but a recognizing mechanism itself. (Each recognizing agent is comparable with a balance weight equipped with balance rather than only balance weight.) (2) Recognizing agent activated by an encounter with the antigen will reproduce its clone to enhance the ability of elimination of the antigen. (Balance weights, when used, can self-reproduce the balance weights of the same type.) (3) The reproduction above will be performed with mutation to increase the affinity with the antigen. (Balance weight, when used, can reproduce not only the same type, but s- lightly different types, hence enhancing precision in weighing.) Next, the genetic mechanisms used in the evolution and in the immune system are compared. For the immune system, environment with which it must interact is not only the non-self from outer world but also the self from internal world. An important difference from Genetic Algorithm is that each agent may interact (e.g. stimulation, inhibition) with each other where in Genetic Algorithm genes evolve independently. Finally, in Genetic Algorithm crossing-over is used to mix genes, however in the immune system, only genetic recombination is used for attaining diversity 2. In sum, the significance difference of the immune algorithm from Genetic Algorithm is; 2 In the battle between host and parasite, host has two defense systems: cellular resistance and systematic immune defense. In cellular resistance, the parasite tries to catch up by evolving molecular that bind, and the host tries to escape by evolving proteins that do not bind. Thus, it is hypothesized that the host uses sexual combination for escape [11]. In immune defense, the roles are reversed; immune system tries to catch up.

6 E its super adaptive character driven by continuous diversity generation; E its adaptation occurs in many levels: structural level and parameter level; E its self-reference as well as nonself-reference. The most naive immune algorithm has the following three steps. (1) Generation of Diversity: Diversity of recognizer 3 in its specificity is generated. (2) Establishment of Self-Tolerance: Recognizers are adjusted to be insensitive to known pattern(self) during developmental phase. (3) Memory of Non-Self: Recognizers are adjusted to be more sensitive to unknown pattern (non-self) during working phase. The outline of the immune algorithm is depicted in Fig. 1 (left). It should be stressed that the algorithm is a general method for modeling systems adaptively. The algorithm may be used to model the system where the system (the self) as well as the environment (the non-self) are unknown or cannot be modeled. Diversity Generation 1.Diversity Generation (Driving Continuous Change) FILTERING Establishment of Self Tolerance REFERENCE SELF SYSTEM (SELF) ENVIRONMENT (NON-SELF) REFERENCE 3.Memory of Non-self (Driving Parameter Change) REFERENCE SENSITIZATION NON-SELF Memory of Non-self REFERENCE 2.Establishment of Self Tolerance (Driving Structural Change) REPRODUCTION and MUTATION Figure 1: Schematic Diagram of the Immune Algorithm(Left) and Immune Algorithm for An Agent-based Architecture(Right) The typical examples of the target to be treated by the immune algorithm may be the followings: 3 We will use the word recognizer for the unit (cell) that has only recognizing and communicating capabilities. The word agent is used for the unit that has more intelligence and autonomy; adaptation and self-replication capability as well as recognition capability.

7 E Noise neutralization in signal level (as presented in section 6). Moreover, abnormal event handling in general such as faulty robot identification and elimination in a robot group, fault detection and recover, computer virus check and elimination, etc. E Group task achievement by cooperation of robots where task may change from time to time. E Adjusting the model structurally and tuning parameters for ever changing self (system itself) as well as the non-self (environment). E Group decision making by filtering out the minor or the inconsistent opinions in a society. Since noise neutralization in signal level will be discussed in detail in section 6, we briefly mention other possible applications here. For the group task achievement by cooperation of robots, diverse population of robots that differs in character must be generated by genetic recombination at the step of Generation of Diversity. This diversity generation may be done in hardware level or in software level depending upon the diversity required. After the diversity is generated, it is imperative to establish a self consistency at the step of Establishment of Self-Tolerance. This can be done typically in the agent architecture for tasks such as group task achievement by filtering out robots (agents) that may disturb the cooperation or may not contribute to the task. Finally, parameters of the remained robots will be adjusted to increase the performance at the step of Memory of Non-self. It should be noted that the internal adaptation at the step of Establishment of Self-Tolerance as well as the external adaptation at the step of Memory of Non-self will not converge due to the continuous change driven at the step of Generation of Diversity. Thus, after the self is established, these three steps can proceed in parallel. 5 Implementation and Architecture of The Immune Algorithm The immune algorithm may be used typically to model the system by distributed agents (called agent-based architecture). In the step of Generation of Diversity, genetic coding for recognizer could be used in a similar manner to that of Genetic Algorithm. In that case, the genetic code of recognizers will be recombined to guarantee the diversity of their specificity. Then, recognizers are developed from their coding in this step. In the step of Establishment of Self-Tolerance, the recognizers which are specific to self pattern are removed for agent-based architecture. In the step of Memory of Non-Self for agent-based architecture, the recognizers which actually recognized the unknown pattern during working phase will be activated. Activated agents would have any of the following properties resulting in higher affinity with the encountered non-self.

8 E Elongation of life span or lower death rate to attain immune memory, E Reproduction of clones (i.e. agents of same type), E Higher rate of mutation. Fig. 1 (right) shows an schematic diagram of the immune algorithm for an agent-based architecture. The immune algorithm may apply to any system where environment is unpredictable. One typical domain may be fault diagnostic systems where the knowledge of fault is intrinsically unpredictable. In other words, fault is by its nature unexpectable, since expectable event can be generally avoidable by design. The technique seems promising, since the diversity of faults is enormous as the system becomes large-scale. Another domain may be control of systems where the knowledge about disturbances imposed is not available. This technique may be especially required for the situation that control signal and disturbance cannot be discriminated in the signal level. We will discuss the application to the control domain in the next section. In conventional control system, disturbance recognizer (sensor) and controller (effector) are physically and conceptually separated. However, in this adaptive disturbance neutralizer, they are identified as agents. Agents are rather divided by the disturbance patterns upon which agents act. To each disturbance pattern of signals, different agent which is capable of producing neutralizing signal is supposed to be activated. That is, each agent has the capability of recognizing one specific disturbance signal and that of neutralizing the disturbance. 6 Applications of the Immune Algorithm to Adaptive Noise Neutralization Instead of estimating the models of the plant and the disturbances, a number of agents are prepared a priori. Each agent is activated to a disturbance peculiar to the agent, and the activated agent injects a signal, called neutralizing signal, to the control system. The total sum of the neutralizing signal is to cancel the effect of disturbances to the control system. 6.1 Control architecture Fig. 2 (Left) shows a schematic diagram of the adaptive control system with the noise neutralizer. In the figure, the block of the adaptive noise neutralizer is composed of a system of agents connected in parallel. Each agent is essentially a shaping filter, which receive the error signal e as its input and the input is shaped to make the output of the filter. The population and parameters of agents in the neutralizer is variable during the operation of the control system. Unnecessary agents become inactive or are deleted, and new elements are created.

9 Adaptive Noise Neutralizer x(n) y(n) j j agent 1 Adaptive Noise Neutralizer d e agent 2 v r + - e v + + Controller Plant y... agent p Figure 2: Block Diagram of Adaptive Noise Neutralizer (Left) and Adaptive Noise Neutralizer Consisting of Agents Working in Parallel (Right) For simplicity, only periodical disturbances are considered. This assumption implies that the disturance can be represented by a sum of a finite number of determnistic and periodic functions. In this case, every element of the neutralizer has a pair of signals of the finite lengths. One of its pair, x j (n) represents a pattern of input signal to the plant. Another element y j (n) of the pair represents the output of the plant for the input x j (n). The pairs of signals {x j (n),y j (n), n =1, 2,,N j } are generated by some means a priori. We further assume that the pair of recognition signal and neutralizing signal is given for those different agents. We call x j (n) genes. This pair is actually acquired during long evolutionary process in real biological system. The problem concerning the generation of the pairs of the signals will be discussed later. The idea of the neutralizer and the pairs of the signals in each component (agent) is illustrated in Fig. 2 (right), which is a detailed diagram of the noise neutralization component shown in Fig. 2 (left). We focus on the processing in each agent. One of the prerequisite for agent is autonomy; agent can decide and act by itself based on the information it collected. Agent in the adaptive noise neutralizer also recognizes the error signal and reacts based on the signal and its current state. Thus, the adaptive noise neutralizer with only one agent could work, the performance is quite low though. First, agent j receives the error signal and store the recent N yj signals in e(k) at the time τ as follows: e j (τ,i)=s(τ N yj +1+i)(i =0, 1, 2,,N yj 1) Whether the data set e j (τ, i) and y j (i) are alike or not will be determined by the following correlation coefficient:

10 R j (τ) = y j (i)e j (i) y j (i)e j (τ,i) 1 Nyj N yj i=0 (y j (i) y j (i)) 2 Nyj i=0 (e j (τ,i) e j (τ,i)) 2 When R j (τ) λ for an appropriate threshold λ(0 <λ<1), the agent recognizes the signal similar to y j (i) in the error signal e(k) at the time τ. Each agent can be in one of the three states: non-sensitized; semi-active; and active state. Agent reacts on the error signal differently depending upon the state of the agent. E If the agent is in non-sensitized state, it will become semi-active state when it recognizes the signal similar to y j (i) in the error signal e(k). E If the agent is in semi-active state, it will become active state when it recognizes the signal similar to y j (i) in the error signal e(k) repeatedly. If it does not recognize the signal repeatedly, the agent will be removed. (i.e. In order to emit the neutralizing signal, the agent must recognize the signal similar to y j (i) in the error signal e(k) repeatedly; periodic signal. ) E If the agent is in active state, it will emit the neutralizing signal when it recognizes the signal similar to y j (i) in the error signal e(k). The neutralizing signals from all the active agents will be summed up, and the total neutralizing signal is considered as the output of the adaptive noise neutralizer. 6.2 Simulation and performance of the noise neutralizer The transfer functions of plant and controller for numerical simulations are; P (s) = 250,C(s) =40, s(s + 130) respectively. In this specific simulation, the control input r(t) is assumed to be 0, hence only disturbance signals are imposed at the point d in Fig 2. Thus, the step of Establishment of Self-Tolerance is omitted for this simulation. When there are control signals, discrimination between the control signal and disturbance is critical. As an initial set of gene data, we use primitive one shown in Fig. 3 (left). In the simulation, ten different gene data with different length of the base are used. Since genes will evolve in the immune algorithm, initial set of gene may be arbitrary. However, primitive one is required so that they can compose many type of disturbance signals. Adaptation done in the step Memory of Non-Self, gene will mutate. In this simulation, two points of the base of triangle (Fig. 3 (left)) changes. Noise is asssumed to be a periodic disturbance. Fig. 3 (right) shows an error caused by one period of disturbance. Fig. 4 (left) shows the time evolution of error when the

11 Noise Intensity base length time Figure 3: Initial Gene Data; Ten different length of the base are prepared initially (left) and Error caused by the disturbance (right) disturbance is imposed with period 100 step (1step =.005 sec). The periodic disturbance is successfully rejected by the system. This means that there are active agents whose neutralizing signals in total can neutralize the imposed disturbance. The disturbance is also known to be neutralized gracefully from a period to period as step increases. This comes from the adaptation of agents, which is the result from adaptation of active agents; higher affinity by higher rate of mutation in this case (as in the step of Memory of Non-self in the immune algorithm). In another simulation, the responses to the initial (at 0 step ) and second (after step imposition of other disturbances) encounter with the same noise are be compared (Fig. 4 (right)). The disturbance at secondary encounter is more effectively neutralized than that at the first encounter. This again comes from the adaptation of the agent; elongation of the life time in this case. In this simulation, we do not assume the communication among agents. However, agents could communicate and cooperate in eliminating the disturbance and memorizing the disturbance pattern. For example, if the neutralizing signal can affect the other agents then it would be close to the Jerne s network [1]. Further, if the neutralizing signal from agents can be an error signal to other agents, then agents may be connected by signal similarly to the Jerne s network [1]. For the actual biological immune system, different types of cells communicate and cooperate extensively during elimination process as known under the name of interleukin or cytokine. Bersini proposed the adaptive control mechanism [12] combining reinforcement learning (Q-learning) and recruitment mechanism [4] hinted from the immune system. His motivations for applying to adaptive control seems to be similar to ours; i.e. modeling difficulty.

12 Noise Intensity Active Population time time Figure 4: Time evolution of error when the disturbance is imposed with period 100 time step (left); and time evolution of error when first encounter with the disturbance of a type at step 0 and again at step after imposition of different disturbance from step to (right). He used recruitment mechanism of the immune system that can be used for the intermittent refreshing of the current actors which proved to be useful for treating the cases when no solution extists or if one solution extists but needs to be improved as well as for improving the adaptability of the controller interacting with unstable process [12]. We took up the task of disturbance rejection which seems to be more analogous to the task of the immune system, hence allows to reflect more features of the immune system: the selectionist principle, self/nonself-reference, and positive/negative selection. 7 Conclusion We proposed a naive concept of the immune algorithm based on the selection-based a- gent architecture compatible with Edelman s selectionist principle, self/nonself-reference, and positive/negative selection. We also proposed that the agent-based architecture to implement the idea and applied it to the task of adaptive noise neutralization. The immune algorithm proposed is neutral to both architecture, and promising for the system where the models of system itself and the environment are not available. Further, the problem of how initial agents should be prepared is also discussed. The architecture should be elaborated and refined. For example, direct interactions (activation and inhibition) among agents may enhance the adaptability, which is left for the future research.

13 Acknowledgements The first version of the simulator had been implemented by Hidetoshi Kubota in his Master s thesis in The similator is currently being modified by Syusaku Siotani. This work has been supported in part by a General Research Grant (C ) by the Ministry of Education, Science and Culture. References [1] N. K. Jerne, The Immune System Sci. Am. Vol. 229, No.1, pp , [2] J. D. Farmer, N. H. Packard, and A. S. Perelson, The Immune Systems, Adaptation, and Machine Learning Physica, 22D, 187, [3] G. M. Edelman, Neural Darwinism: The Theory of Neural Group Selection, Basic Books, New York, [4] H. Bersisni and F.J. Varela, The Immune Recruitment Mechanism: A Selective Evolutionary Strategy Proc. ICGA 91, [5] Y. Ishida, Fully Distributed Diagnosis by PDP Learning Algorithm: Towards Immune Network PDP Model, Proc. of IJCNN 90, San Diego, [6] S. Forrest, A.S. Perelson, L. Allen, and R. Cherukuri, Self-Nonself Discrimination in a Computer, in Proceedings of 1994 IEEE Symposium on Research in Security and Privacy, [7] Y. Ishida and N. Adachi, An Immunological Algorithm and Its Application to Disturbance Rejection, Technical Report of NAIST, NAIST-IS-TR 95030, [8] G. M. Edelman, The Remembered Present: A Biological Theory of Consciousness, Basic Books, New York, [9] G. M. Edelman, Bright Air, Brilliant Fire: On the Matter of the Mind, Basic Books, New York, [10] S. Tonegawa, Somatic Generation of Antibody Diversity, Nature, 302, pp , [11] H. Bremermann, The Adaptive Significance of Sexuality in S.C. Sterans Ed. The Evolution of Sex and Its Consequences, Basel. pp , [12] H. Bersisni, The Immune Network and Adaptive Control Technical Report, No. IR/IRDIA/91-9, 1991.