Verification of an Agent-Based Model

Transcription

1 Verification of an Agent-Based Model David A. Ostrowski Ford Research Laboratory Ford Motor Company Abstract This paper presents efforts towards the verification of an Agent-Based Model. Software Engineering principles are first applied to verify first structural then functional levels of the model [Pressman 1990]. Finally, techniques of docking are applied to test the model at a higher level of abstraction [Burton, Obel, 1999]. The ABM in which these techniques are applied is designed to model the evolution of the infrastructure of an alternative fuel energy source. These techniques can be generally applied to any ABM development. This verification is deemed necessary to the institutionalization of ABM as well as gaining specific model results. Through these steps, the author makes a strong connection between ABM modeling efforts and overall software development. Contact David A. Ostrowski Ford Research Laboratory Ford Motor Company Infotronics and System Analytics Department SFL building, MD # Village Road Dearborn, MI Tel: dostrows@ford.com Acknowledgement: The author would like to thank John Sullivan and Craig Stephan of the Ford Research Laboratory for helpful discussions and sharing of their ABM model results for the docking comparison. Agent Based Modeling, Software Testing, Docking, Verification

2 Verification of an Agent-Based Model David A. Ostrowski ABM models, by definition, are complex. In their use, ABM is applied to solve problems that exist out of the range of typical mathematical techniques[ Ferber 1995]. This characteristic presents many challenges with regards to their verification. This has led to difficulties in their industrial applications since their results do not maintain the same levels of confidence. This paper presents the efforts to define testing to verify a specific Agent-Based Model. Three different methodologies are applied at successive levels of abstraction. Structural testing is first employed. Specifically, path testing is used to examine all conditions of crucial parts of the system. Due to its level of complexity, it is unrealistic to test all paths of the model as a completely integrated unit. In order to test the critical components, separate driver routines are employed to perform these tests. The second approach is to test the system from a functional perspective. This allows for the examination of the system as an integrated whole. In order to successfully test this approach, equivalence classes are constructed. These classes are constructed for the specific goal of the experiment. Boundary conditions are presented in order to sufficiently test the system in accord with the intended outcome of specific variables. The last approach to be considered is to derive some type of verification from the approach of docking. The ABM presented here is compared to results from another model that is approaching the solution with a slightly different set of assumptions. The model considered in this paper is applied to the evolution of alternative fuel infrastructures. The model containing agents that represent consumers and fuel station proprietors, interact on a grid representing an urban landscape. Consumers make considerations with regards to the purchase or retention of alternative fuel vehicles. Fuel station proprietors make decisions on the investment of an alternative fuel infrastructure. Decisions are influenced by the maintenance of the agents' ability above a given threshold parameter as well as the choice between driving routes and traffic congestion. The model demonstrates the transition parameters in the acceptance of an alternative fuel infrastructure. Early adopters of alternative fuel vehicles are considered as agents in the model. These agents weigh factors such as the distance of station, destination, traffic flow and current levels of fuel in each path decision. Information in this model is both limited and global [Lampner Johnson 2002]. The agents maintain a history of paths taken as well as knowledge of current fuel station locations on the grid. They employ learning through their decision process utilizing this dynamic memory. The agents make a conscience decision as to what path to take to work each day. This specific model places emphasis on the evolution of a given infrastructure with strong consideration given to the geographical areas in the model along with the interest of modeling congestion and its influence on the evolution of such an infrastructure. Model Description The model is represented as a rectangular grid in which drivers travel to specified locations. In each time step, the driver makes a commute. At every four-month interval drivers decide between driving either a gasoline or alternative fuel powered vehicle. Agents maintain a utility function that is viewed as a residual fuel level over the average traveled distance. Agents strive to maintain their utility function over a given threshold. If the threshold is not maintained it will influence the decision of purchase or maintenance of a alternatively fueled vehicle. Agents choose between alternative paths on the grid to the specified location. Paths are designated by proximity between chosen destinations as well as the location of fueling facilities. A secondary utility is placed on the traffic congestion representing the percentage of driver agents taking the same paths. Given a recent window of acceptable residual levels, agents choose to avoid high traffic scenarios. Both utility thresholds are assigned around a random distribution between the drivers agent population. Stations consider the investment into an alternative fuel station. This is determined through a utility function that considers the traffic threshold. It maintains that if a specific traffic level is above a given threshold, then support

3 alternative fuels. After making an investment, they choose to maintain the infrastructure based on an acceptable utility level. If the station proprietor decides to discontinue its support, then they do not reconsider it for the remainder of the simulation. Structural Verification The first step in verification is to apply structural testing. This necessitates an examination of the overall structure from a software design perspective. When scaled up with a substantial number of agents, there are too many paths to follow. Due to this issue, the system is tested out at a simplified level with paths in the crucial logic considered for a single agent. The model was designed with the station agents and consumer agents as separate objects. Objects and their associated methods are listed in figure 1.0. Three major areas of logic in the system are considered : decision for energy source under each time interval, the station proprietors decision at each time level and the travel path decisions. The associated input test data sets (example figure 2.0) were considered to generate each path through the logic. To enable this, a single driver agent was applied against the grid containing at most 20 station agents. The data sets associated allow for a direct conversion to the associated data. Consumer Stations Init() stationlocation [25,25] [30,30] InitFuel() threshold DefinePaths() fuellevel 16 5 DefineFuelDictatedPaths() fueltype 1 0 TravelTo() numberfillups 2 0 FindPosition() DisplayAllPaths() Init() InfrastructureEvaluation() Figure 1.0 Classes and methods Figure 2.0 Designated input data for paths tested in TravelTo() method All of the relevant tests were executed at a basic level. For testing of the components including the evaluation of traffic congestion, five agents were applied to the model to test the associated properties. Since congestion was measured as a percentage of agents this allowed for a realistic first round of testing. Functional Verification As an integrated software unit, it is more convenient to consider the ABM model from the perspective of functional testing. This is accomplished through the establishment of equivalence classes [Sommerville 1996]. The classes are established around a specific goal or specified outcome for the models execution. The initial goal for the model is to define a relationship between placements on a grid to provide for a successful or unsuccessful transition to an alternative fuel source. In this example, a successful transition is considered if the ratio of consumers who have adopted the updated technology is over forty percent. Figure 3.0 shows the identified equivalence classes that characterize the three conditions of failure, success and equilibrium state. Here the equilibrium state is defined as where the initialized agents keep their alternative fuel vehicles ( in this case 10%) with a small ratio converting to alternative fuel vehicles from conventional. The associated characteristics include the number of stations, average distance from destination, average distance between stations. These classes were initially developed through intuition and then clarified by a series of tests using an equilibrium parameter. For the testing, equivalence levels were maintained in the other classes. Boundary values were tested around the established partitions.

4 .characteristics equivalence Classes boundary values number of Stations Failure of Infrastructure Unit representation: number of stations Equilibrium Success of infrastructure Avg Distance from Destination Failure of Infrastructure ,21 Equilibrium ,20 Success of infrastructure ,17 Average Miles from other Stations Failure of Infrastructure ,13 Equilibrium ,17 Success of infrastructure ,19 Average Distance from Agent Initialization Failure of Infrastructure ,13 Equilibrium ,15 Success of infrastructure ,17 Figure 3.0 Associated Characteristics, Equivalence Classes and Boundary Conditions Docking Docking is applied by drawing comparisons between the current model presented and a second ABM which is designed for a similar purpose. [Burton 2003] This second ABM was created by Stephan and Sullivan of Ford Research Laboratory [Stephan, Sullivan 2004] for the examination of an alternative fuel adoption. These models maintain both similarities and differences. Among the similarities is that both models employ driver agents and station agents with both agents either choosing to invest in an alternative fuel infrastructure. The models vary in the approach to utility functions and driving patterns. Example output is demonstrated in figure 4,5,6. The figures demonstrate the goal of the dependence of random vs. judicial placement of fueling stations between the two different models. Figure 4 and 5 demonstrate the results of a failure and successful fuel adoption. Figure 6 demonstrates both scenarios based only on the specific placement of stations. Both of the models demonstrate an acceptance or decline simply out of the judicious placement of the stations without modifying any other variables in the model. Due to the layout of the agents and how they travel to work, the layouts and how they interact are not duplicated exactly. The first model demonstrates the characteristic of the stations to eventually find a plateau while the second model shows the proprietor agents to trend towards higher acceptance rates H2 Drivers (% of fleet) H2 Stations Figure 4.0 Model 1. Failure of Adoption of Alternative Fuel

5 H2 Vehicles (% of fleet) H2 Vehicles Figure 5.0 Model 1. Successful Transition H2 Vehicles (% of fleet) H2 Stations (number) Years Figure 6.0 Model 2 (Docking Comparison Model) Both successful and unsuccessful transition. Conclusion An ABM can be viewed as a complex software system. Like software, they can only be disproved as opposed to be proved. ABMs have provided intuition into the solution of problems which are intractable. Given this, the institutionalization of ABM must be provided with set applications of methodologies by which to demonstrate a level of correctness. Here, steps have been presented to establish means by which verification can be applied to such models. Three different methods were employed to perform verification of our model. The first, structural verification allowed for a very basic approach to the model through the means of identifying critical paths that must have some confidence established around in order to be able to verifiy their correctness. The second approach at a higher level of abstraction is to apply functional methods of testing in order to examine the model as a black box which test cases are established by a set goal for the output and the test conditions are built to define equivalence classes around which the boundary values are to be tested. Finally, a high-level comparison is made between the results of two similar models. This test established that the fundamental relationship between specific initial locations of alternative fuel stations can influence the long-term acceptance of the infrastructure.

6 References [Beizer, Boris], "Software Testing Techniques", Data Systems Analysts, Inc. Van Nostrand Reinhold Company, [Beizer, Boris] "Black Box Testing: Techniques for Functional Testing of Software and systems", John Wiley and Sons Inc. New York, NY 1995 [Burton, 2003], "Computational Laboratories for Organizational Science: Questions, Validity and Docking", Computational and Mathematical Organization Theory: 9, pp , 2003, Kluwer Academic Publishers [Burton R., Obel B.], "Agent Simulation: Applications, Models and Tools", Proceedings of the Workshop on Agent Simulation: Applications, Models and Tools", October 15-16, 1999, The University of Chicago [Ferber, 1995] "Multi-Agent systems: An Introduction to Distributed Aritificial Intelligence", Addison-Wesley, 1995 [ D. Lamper,N. F.Johnson], "The Science of Complexity", Dr. Dobbs Journal, October 2002, pp [ Pressman, Roger S.1990], "Software Engineering: A Practitioners Approach", McGraw Hill, 1990 [Sommervile 1996] "Software Engineering", Addison-Wesley, 1996 [Stephan, Sullivan], "An Agent-Based Hydrogen Vehicle/Infrastructure Model",. CEC2004, Portland, OR, 2004, pp