Using evolutionary techniques to improve the multisensor fusion of environmental measurements

Using evolutionary techniques to improve the multisensor fusion of environmental measurements A.L. Hood 1*, V.M.Becerra 2 and R.J.Craddock 3 1 Technologies for Sustainable Built Environments Centre, University of Reading, United Kingdom 2 School of Systems Engineering, University of Reading, United Kingdom 3 Thales Research and Technology (UK), Reading, United Kingdom ABSTRACT * Corresponding author: fx016817@reading.ac.uk Within the environmental sciences, the ability to accurately collate, combine and present data from various sensors is important to validate research. Whilst measuring data from a single sensor is a standard practice, fusing the output of more than one sensor can be a complicated and computationally intensive process. Outlined is a brief introduction to the art of multisensor fusion and a suggestion that fusion optimisation and analysis could be improved with the use of evolutionary techniques common within the field of artificial intelligence. Evolutionary techniques are used in a wide array of research areas, but their application and initial parameter settings are often unique to the specific task at hand. By combining these two relatively established practices it is suggested that a new approach could be determined to provide a combined sensory output far superior to that of individual readings. Keywords: Artificial intelligence, sensor, fusion, evolutionary techniques, environmental 1. INTRODUCTION The ability to measure environmental variables plays a key part in climate research. However, like all sensors, environmental devices can be impacted by accuracy failures, obstruction and the inability to adapt to changes both measurable and unseen. These problems are only compounded when mounted upon evolving staging environments (such as moveable platforms, autonomous vehicles or harsh climate). Combining the outputs of multiple sensors can offer a durable sensor platform. If one data stream fails, the others can be used to compensate and in some cases provide estimates of the missing values. Further advancements can be made by using a combination of sensors that offer a variation in coverage areas; thus extending the operational range of the data gathering. Multisensor fusion (MSF) provides the possibilities for the combination of such data. Many models have been developed often uniquely for the task in hand, although no specific model for such environmental data fusion exists. In addition, MSF would benefit from a further understanding of the use of self-adapting algorithms which can keep track of ever evolving environments. One such approach is the Darwinian inspired evolutionary algorithm, which is a generic, flexible and versatile framework for complex optimisation and search problems. 1

2. THE BASICS OF MULTISENSOR FUSION The field of sensor fusion has its historical background within defence applications [1][2]. Sensor fusion can be described as the process of combining multiple data from several sensor (or data) sources, such that the result is in some cases better than would be possible when sensor data is used individually. This combined data improves accuracy and allows for more specific inferences than would be possible from a single source. Whilst data fusion is often seen as the combining of data from multiple sources of the same or different types, the definition also includes the fusion of data from a single data source such as measurements taken at different times [3]. The application of sensor fusion can provide many benefits. These include the increase in spatial and temporal coverage of sensors, increased precision and a reduction in sensory degradation. When combined and evaluated, the data provided can improve the accuracy of the inferences being assessed. 2.1 Examples Of MSF Figure 1 The multiple sensors within humans [4] One of the most efficient and critical sensory fusion systems is contained within the mammalian brain. The human brain for example is capable of collating data from our internal and external sensors and combining them to provide input into the decision making areas. If we were able to only focus on one sensory input at a time, for example sound, our perception of our environment would be incomplete and thus have an effect on how we make inferences. As shown by Figure 1, sensory systems can be a combination of sensors, each of which can be given a specific strength or impact value. Whilst it is often remarked that humans possess five primary sensors, our perception will be further defined by the supplementary use of secondary sensory inputs such as priorioception (position), equilibrioception (balance) and thermoception (temperature). A common civilian use of sensory fusion takes place within the control systems of many modern cars [5]. With the intention of improving safety, vehicles are equipped with multiple specific sensors to monitor the internal and external environment whilst in motion. By combining the measurements derived, a central control unit can make situational inferences and in some cases override the human controls (Figure 2). 2

Figure 2 An example multisensor architecture for improving vehicular safety [5] The consolidation of multiple data streams also allows for reasoning to continue if one or more input sources show signs of degradation. Weather conditions, sensor failure and interference can all cause an input to fail or produce inaccurate results. Whilst this would cause a severe reduction in accuracy using a single sensor system, the outputs of other sensors can be combined with a priori knowledge to derive alternatives. 2.2 Common Industry Models Although there are examples of multisensor use in civilian applications, the field has matured predominantly within defence applications such as tracking, identification and targeting. Emerging from this research have been various architectures conceived with the intention of offering a structure for the combination of outputs and inputs. The JDL (Joint Directors of Laboratories) architecture [6] (Figure 3) is widely used in both the defence and civilian fields, but it is believed to be quite inflexible [7]. The authors provided a definition for their hierarchical approach describing data fusion as: A multilevel, multifaceted process dealing with the automatic detection, association, correlation, estimation and combination of data and information from single and multiple sources Each of the defined levels provides a processing approach to further tailor the data provided from the outputs. In addition, a priori knowledge is provided by a database and allowances are included for human interaction to further refine the inference making process. In addition, other approaches such as the British Waterfall Fusion Process Model [8] and the adopted Boyd OODA loop [9] have been developed to offer alternatives which provide flexibility in the development of multisensory systems. 3

Figure 3 The JDL Architecture [3] 3 OUTLINE OF EVOLUTIONARY TECHNIQUES First developed in the 1950 s, evolutionary computation has developed over the decades into a powerful and versatile heuristic based global search and optimisation method. Taking its inspiration from the biological evolutionary process, the technique allows a system to move towards a solution within a search space by the combination and mutation of a population of possible candidates. Whilst the initial set-up and problem representation is often regarded as more an art than a science, genetic techniques have been used in a wide arrange of applications and are embraced as a subset of the field of artificial intelligence. 3.1 The Influence From Nature Biological entities can be described based on the composition of their genes; the DNA strings which offer a building block to the overall structure. These small DNA segments refer to specific features of the organism and can be broken down into individual sections which describe a variable referring to the value of this feature. A simple example may refer to a section responsible for the colour of eyes and will have a variable which when set refers to blue, brown or green. An individual can be described as the combination of each of these genes, known as a genome, and the population of identical genomes as a genotype. When this set of genes is referenced to its observed (or external) equivalent, it is known as its phenotype. Evolutionary computation utilises an approach similar to the biological representation (Figure 4) of an organism to define a particular solution found within a population of solutions. As in nature, these populations can be mated (combined) to produce offspring (new solutions) which share the traits of their parents. Ideally those traits will best enable an organism to increase its performance within its set environment. This can be evaluated with the use of a fitness function; a value which determines how well its phenotype fits within a set of parameters. 4

Figure 4 The comparative representation of biological systems and evolutionary computation [10] Similar to the natural occurring effects of population combination, evolutionary techniques adopt the use of gene mutation; random mutations of individual variables to provide variety in the genotype of the solution. 3.2 Basic Structure Of An Evolutionary Algorithm 3.2.1 Representing the problem One of the first and important tasks when developing an evolutionary approach is the design of a candidate solution. This will represent the mapping from genotype to phenotype and will be created to accurately reflect a solution to the problem stipulated. As with a great deal of evolutionary solutions, this task will be made significantly easier with a stronger understanding of the problem and search space involved. The simplest approach to candidate representation is a binary genotype. Each variable to be considered is presented as a binary 0 or 1 value and can form a string of bits of infinite length. However, depending on the nature of the problem, consideration can be given to integer, real, floating or permutation approaches. Each variation produces its own unique challenges and care should be given where variables are linked and need to be kept together. Numerical representations can be made for non-numerical values, for example the set {0,1,2,3} can be used to produce the equivalents of the set {North, East, South, West} Of utmost importance is the maintenance of the link between the genotype and phenotype mappings. Failure at the design stage will often cause candidate solutions to be created which have phenotype mappings which are completely unsuitable for the search space and the problem at hand. 3.2.2 The fitness function In order to measure the suitability of a candidate solution to the problem, a fitness function must be derived to provide a numerical 'score' of strength. As an example, if the task was to derive an integer that maximises the value of x 2 using a candidate pool of 5 bit strings, then the solution 01110 the fitness f(x 2 ) would be calculated as 14 2 = 196. 5

The fitness function is vital to the evolutionary approach. It offers the primary mechanism for determining proximity to an ideal solution and allows for the ranking of pooled candidate solutions during parent selection. 3.2.3 Selecting mating pairs As with natural evolution, the main onus on parental selection is to choose those 'mates' that offer the greatest strengths within the defined environment. The intention is to choose those parents with high fitness scores with the aim that recombination of their genes will, in some cases, maintain the best features from each of the parent candidates. There are many approaches to this task and each should be evaluated on a problem by problem basis. Commonly used methods include the roulette wheel, ranking tournaments and fitness proportional selection. Initially it may be intuitive to always select the top two ranked candidates for recombination, however this approach is likely to result in the process stopping at a local optimum rather than the global optimum solution. 3.2.4 Recombination and the injection of mutation Figure 5 Examples of one point and two point crossover [11] Once a process of parental selection has taken place, a decision needs to be taken regarding the crossover (mating) process. There are many documented approaches to this part of the algorithm, some specific to certain problem domains, but Figure 5 shows a commonly used one or two point crossover. During one bit crossover, a single point in the parents genotype is identified and the genes split into a head and a tail. The process recombines these genes head to tail from the represented parents to produce the required number of offspring. An important addition to this evolutionary approach is the introduction of the biological equivalent gene mutation. Using a user defined random process, one (or sometimes more) genes are selected to have their value altered (mutated). In a binary string genotype, this would result in one of the bits being switched (0 to 1 or 1 to 0). This application has been shown to be the key driver to 'bouncing' a solution out of a local optimum and allowing a wider search area to be considered. Once a new population of candidates has been created a population model needs to be adhered to. This may dictate that all parents are replaced by an equivalent number of children, or that only the top x percent of solutions from the previous generation are included. 6

Figure 6 The evolutionary process Evolutionary algorithms require a stopping criterion. This could be after a certain number of generations, or when the best solution has remained constant for a determined period of time. Until this threshold is met, the algorithm will continue (Figure 6) to loop. 4. ENVIRONMENTAL SENSING AND SENSOR FUSION MODELS 4.1 Sensor Use In Climate Science Sensor arrays are currently in use for systems responsible for weather forecasting and measuring the Earth's resources. These rely on a combination of visible, microwave, infrared, laser and MMV (millimetre-wave) sensors to collate information about rainfall, cloud cover, snow density amongst others. Due to the nature of the entities being observed, there are many instances of source obstruction (such as cloud cover), signal interference (solar storms) and sensor failure (often due to the difficultly regarding maintenance). Many of these issues can be either overcome or mitigated by the use of multiple, overlapping sensors and a combination of mathematical techniques. 4.2 Applying MSF Models To Environmental Measuring One of the most commonly used MSF models is the JDL data fusion architecture. As can be seen from Figure 3, the process is split into a series of levels; each responsible for processing the data to further refine its inferences. Whilst this model was not initially developed with civilian use in mind, it is possible to adapt it for other implementations [12]. Climate science is rich in data sources to facilitate the investigation of the impact of data combination. Raw data from one entity can be processed with the inferences from another to provide a clearer, more accurate depiction of that being studied. Information that is being measured at discrete time intervals can be estimated and combined with that measured using a differing scale. This offers a greater scope for climate modelling and further understanding of the relationships between the information being observed. 5. CONCLUSIONS / FURTHER WORK There are a large number of questions that need to be addressed in order to fully appreciate the possibilities of combining the techniques discussed for environmental applications. Firstly among these is the investigation of existing MSF models and how they can be applied to environmental problems. Could an existing model be adapted slightly, or does a new specific approach need to be 7

taken? Would it be better to design a more generic framework which could have a use outside of the parameters of this research? This is the initial task required before venturing into the intricacies of MSF. Any model must have at its core the driver of self-adaptation. Environmental sensors are affected by the environment they are measuring. There are both natural (e.g. weather) and unnatural (e.g. sensor degradation) impacts on sensor functioning which can be addressed by adding awareness to the model and evolving new strategies in situ. This is particularly important for sensors placed in locations where regular maintenance is difficult or must be scheduled in advance (including those placed on satellites or in harsh environments). Whether evolutionary algorithms offer the most robust approach to this problem, or only offer benefits at initialisation stage (such as optimum sensor placement) will need to be assessed. However, such algorithms work well in adapting environments and may prove very useful in areas where the search space is constantly changing. Finally, examples will need to be provided to highlight the benefit, if any, of utilising MSF in environmental measuring. It is easy to assume that utilising more than one data stream would offer a richer, more concise view of the environment being measured. However, automating this process will have to be proven to offer a more robust, trusted information source, that provides tangible benefits to the researchers undertaking such work. 6. REFERENCES [1] E. Waltz and J.Linus, Multi-sensor Data Fusion, Artech House, Norwood, MA 1990 [2] Hall, Mathematical Techniques in Multi-Sensor Data Fusion, Artech House, Norwood, MA 1992 [3] W. Elmenreich, An Introduction to Sensor Fusion, Vienna University of Technology, Austria, November 2002 [4] PULSE, Consumer electronics and the future of engaging the senses: an ergonomist s perspective. [Online]. Available: http://pulse.pdd.co.uk/2011/11/consumer-electronicsand-the-future-of-engaging-the-senses-an-ergonomists-perspective/ [5] Delphi, Delphi sensor fusion for automotive safety, [Online]. Available: http://delphi.com/manufacturers/auto/safety/active/ [6] White, F.E., A Model for Data Fusion, Proc. 1st National Symposium on Sensor Fusion, 1988 [7] Steinburg, Bowman, White, Revisions to the JDL data fusion model, Proceedings of the 1999 IRIS Unclassified National Sensor and Data Fusion Conference, May 1999 [8] Markin, et al, Technology foresight on data fusion and data processing, The Royal Aeronautical Society, 1997 [9] Boyd, A discourse on wining and losing, Air University Library, Maxwell, AL, May 1987 (unpublished) [10] Maslov, Gertner, Mutli-sensor fusion: an evolutionary algorithm approach, Information Fusion 7, 304-330, 2007 [11] Johnston, R, Applications of Genetic Algorithms in Chemistry, University of Birmingham, [Online], Available: http://www.tc.bham.ac.uk/~roy/research/ga.html [12] Llinas, et al, Revisiting the JDL Data Fusion Model II, Proceedings of the Seventh International Conference on Information Fusion (FUSION 2004) 8