Integrating New Cost Model into HMA-Based Grid Resource Scheduling

Transcription

1 Integrating New Cost Model into HMA-Based Grid Resource Scheduling Jun-yan Zhang, Fan Min, and Guo-wei Yang College of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu , China {jyzhang, minfan, Abstract. Grid systems can provide a virtual framework for management and scheduling of resources across different domains. This paper proposes an HMA-base grid resource scheduling system to implement resource finding and scheduling. A new cost model is also given, considering resource finding cost and resource deciding cost beyond traditional model. In succession, the new cost model is integrated into the HMA-base grid resource scheduling system. Our experiment shows that optimal solution under traditional cost model is no longer optimal under our model. Keywords. Grid, resource scheduling, Agent, cost model 1 Introduction In traditional distributed computing environments (DCEs), resource management systems (RMSs) were primarily responsible for allocating resources for tasks [3]. They also performed functions such as resource discovery and monitoring to support their primary roles. With large amount of distributed resources and users, grid systems can provide a virtual framework for management and scheduling of resources across different domains, and they have been the focus of much research activities in recent years. A computational grid is an emerging computing infrastructure that enables effective access to high performance computing resources [5]. Resource management and scheduling are key grid services, where issues of utilizing grid resources reasonably by minimizing total cost represent a common concern for most grid infrastructure and scheduling algorithm developers. Resource management and scheduling in grid systems [1] [2] is challenging due to: (a) geographical distribution of resources; (b) resource heterogeneity; (c) autonomously administered grid domains having their own resource policies and practices; and (d) grid domains using different access and cost models. In this paper, we adopt Hierarchical Multi-Agent-based (HMA-based) methodology to grid resource scheduling, achieved by integrating cost into the HMA-base grid resource scheduling system to implement minimal cost resource scheduling. M. Li et al. (Eds.): GCC 2003, LNCS 3033, pp , Springer-Verlag Berlin Heidelberg 2004

2 Integrating New Cost Model into HMA-Based Grid Resource Scheduling 653 The paper is organized as follows: Section 2 introduces the traditional cost model. In section 3, the HMA-based grid resource scheduling system is described. In section 4, we integrate new cost model into our HMA-based grid resource scheduling systems. Comparing experimental results are included in section 5 and the paper concludes in section 6. 2 Traditional Cost Model The traditional cost model of the Internet forms the basis of most call admission, routing and reservation algorithms today. The model combines the cost classifying, switching, queuing and scheduling at a node with the cost of transmission over the next link in one abstract figure associated with each node-link pair (See Figure 1)[6]. Based on this model, the total cost of a system can be defined as: C = Cs + Cq + Cl (1) Where, C denotes the total cost of system; Cs denotes the cost of switching and scheduling; Cq denotes the cost of queuing; Cl denotes the cost of link. Fig. 1. Traditional Cost Model Although simple, this model has proven effective and robust in designing many popular network protocols. Shortest Path Routing, for example, uses this cost model to find the route between a pair of origin-destination nodes by minimizing the sum of per-hop costs.

3 654 J.-y. Zhang, F. Min, and G.-w. Yang 3 HMA-Based Grid Resource Scheduling System In our resource scheduling system, we consider the whole grid system as a Global grid, which is made up of n Grid Domains. Let GD i (i = 1, 2,, n) denote the ith Grid Domain (See Figure 2). The GD i is an autonomous, administrative and interactive entity consisting of a set of resources, services and users managed by a single Management Agent (MAgent i ). In our system, we divide each GD into two subdomains: (a) a resource domain (RD) which signifies the resources within the GD; and (b) a user domain (UD) which signifies the users within the GD. Fig. 2. HMA-Based Grid Resource Scheduling System First of all, we define γ = (γi, j) n k as system resource matrix, where γi, j specifies how many type j resources GD i possesses. We also define ξ = (ξi, j) n k as system processing power matrix, where ξi, j specifies processing power of GD i for the jth type of resource, clearly ξij 0 for any legal i, j. In this system, Resource Deciding Agent (RDAgent) maintains an n n table D which records the distance between GDs, where D ij denotes the distance between GD i and GD j. It also holds a quadruple Θ (θ, γθ, ξ, μ) to describe each GD in detail provides by each MAgent. Where, (a) θ denotes serial number of current GD, θ = 1, 2,, n; (b) γθ = [γθ1, γθ2,, γθk] specifies local resources; (c) ξθ = [ξθ1, ξθ2,, ξθk] denotes the processing power of current GD; (d) μ = [μ1, μ1,, μl] denotes the users of UDθ, μ i (i 0)signifies the ith user within UDθ. At the beginning, RDAgent initializes the distance table D and quadruple Θ (θ, γ, ξ, μ) according to the information submitted by all existed MAgents. When RDAgent polls MAgents on its own initiative, MAgents would refresh the table and the quadru-

4 Integrating New Cost Model into HMA-Based Grid Resource Scheduling 655 Fig. 3. A New Cost Model ple if the update happens within current GD. Otherwise, MAgents needn t submit the update and only keep the update information locally. Let TR denote a task requirement originated from a GD, it includes the explicit user IP address, and its original GD. It also includes the information of resource requirement. The requirement amount of each type of resource is denoted as x 1, x 2,, x K respectively. After a TR is recognized and analyzed by Resource Scheduling Agent (RSAgent), RSAgent orders Resource Finding Agent (RFAgent) to find the location of resources which can meet the requirement TR. At first, RFAgent queries RDAgent for resource information. Because RDAgent has a distance table D and a quadruple Θ, after RFAgent finishes the finding processing, it can get a resource deployment matrix RDP n k. The columns specify RS and the rows specify Grid Domains. If RS m locates at GR θ and γ θm x m, we let RDP θm =(γ θm, D iθ ), else we let RDP θm = (0, ). Now RFAgent examines this matrix RDP n k, if RFAgent finds a column whose elements are all (0, ), it will ask the RDAgent to update table D and quadruple Θ. After update table D and quadruple Θ, if matrix RDP n k still does not satisfy the TR, the next time update will start after a period of time (which we set to 5 minutes). If appropriate resource combination cannot be found after some times (which we set to 3 times), RFAgent concludes that the TR cannot be met and the task cannot be implemented, and sends this message to RSAgent. RSAgent will cancel this task and notify the user. If appropriate resource combination can be found, RFAgent sends this resource combination information to RSAgent. RSAgent will schedule resources according to this information and return result to the user, which procedure will be given in the next section. 4 Integrating New Cost Model into HMA-Based Grid Resource Scheduling System Generally speaking, resource schedulers and resource managers tend to choose the nearest resource because they make decisions depending upon traditional cost model [4]. Now we present a new cost model based on grid systems; therefore, the cost of resource finding and cost of resource deciding will affect the total cost significantly.

5 656 J.-y. Zhang, F. Min, and G.-w. Yang Thus, the system total cost C consists of three parts: cost of resource finding C F, cost of resource deciding C D, and cost of resource scheduling Cs. Its architecture is illustrated with Figure 3. On the view of Figure 3, C S can be further divided into processing cost C P and transmission cost C T. The total cost of resource scheduling C can be defined as: C = C F + C D + C P + C T (2) Now we can integrate the new cost model into HMA-based grid resource scheduling systems to implement reasonable scheduling with minimal cost. On receiving a TR, RSAgent will forward resources and processing power requirement to RFAgent. RFAgent examines this matrix RDP n k, if it finds a column whose elements are all (0, ), it will ask the RDAgent to update table D and quadruple Θ. We use integer variable count to record the update times. After refreshing table D and quadruple Θ, if matrix RDP n k still does not satisfy the TR, the next time update will start after a period of time (about 5 minutes). Therefore, the cost of finding resources can be measured by count and denoted as C F. That is to say, C F = count. If count > M (M is a constant and M >0. In our paper, we set M = 3), then let C F =, RFAgent consider the TR cannot be met and tell this message to RSAgent. RSAgent will cancel this task and notify the user. If only one element is not (0, ) in each column, this means only one kind combination of resources and Grid Domains can satisfy TR. So RSAgent has no choice but select this combination as the only solution regardless of cost C. If there is more than one resource combinations can satisfy TR, RDAgent will list all possible resource combinations and make decisions which combination to be chosen. The combinations can be signified as follows: CB (TR) = [RA, RA,, RA ] (3) 1 2 K where RAi represents the GD which offers the ith resource, 0 for no requirement of respective resource. The requirement of each type of resource can be met by some GD. Each type of resources is selected independently, so we need choose among at most n values each time. Accordingly, the time complexity is O(n k) instead of O(n k ). If some CB α which satisfies TR and includes λ GD, which means there are λ kinds resources can be provided locally, and (k λ) kinds resources locate at other GDs. We can define C D as: C D = (k λ)/k (4) Resource scheduling cost C S is made up of processing cost C P and transmission cost C T. The stronger the processing power of GD, the smaller the C P of GD. For ξ i, j represents computation power of GDi for resource j, then k ξ = i= 1 is the total computational power when choosing respective combination. ξ i, j (5)

6 Integrating New Cost Model into HMA-Based Grid Resource Scheduling 657 Accordingly, we use processing power s reciprocal to signify C P, i.e. C P = 1/ξ. Because we assume the task to be processed cannot be divided further, the GD is unique. The father the distance between two GDs, the larger the C T. Accordingly, we use logarithm of the distance between GDs and GD i to signify C T, so we have: k C T = log Dij j= 1 (6) Table 1. Distance between GD i (unit: hop) GD 1 GD 2 GD 3 GD 4 GD 5 GD 6 GD 7 GD 8 GD 9 GD 10 GD GD GD GD GD GD GD GD GD GD 10 0 Table 2. Initial Information about all GDs Table 3. Task Requirements Information γθ ξθ μθ Task1 γ13= 5 γ21 = 3 γ 53 = μ51 Task2 γ34 = 1 γ71 = μ92 Task3 γ12 = 9 γ31 = 7 γ81 = 1 γ10,1 = μ13 Task4 γ11 = 2 γ44 = μ74 Task5 γ61 = 4 γ92 = 5 γ10,3 = μ61 Task6 γ33 = 6 γ73 = 3 γ81 = 1 γ94 = μ45 Task7 γ24 = 2 γ62 = μ10,1 Task8 γ41 = 3 γ83 = 3 γ93 = μ33

7 658 J.-y. Zhang, F. Min, and G.-w. Yang traditional cost model average cost new cost model sum of resource requirement Fig. 4. Performance comparison Based on equation (2), the system total resource scheduling cost is: C = count + (k λ)/k + 1/ξ + log k Dij j= 1 (7) RDAgent will choose the CB with minimal C and recommend to RSAgent. RSAgent would perform scheduling according this CB and return results to corresponding user. In this paper, we don t compare costs of two tasks, so how many resources (such as communication traffic, computing overhead and so on) are needed is not our concern when we calculate total cost C. That is to say, the requirement we compute is relative but not absolute. 5 Comparing Experiment We simulate a system containing 10 GDs. 4 types of resource are provided. Distance between these GDs are listed in Table 1, initial information about all GDs are listed in Table 2, and task requirements information are listed in Table 3. Figure 4 shows average cost of tasks under two conditions: 1, optimal solution with traditional cost model and 2, optimal solution with new cost model. Here we can see that optimal solution under traditional cost model may not be optimal under new cost model. 6 Conclusion With the growing popularity of middleware dedicated at making so-called grids of processing and storage resources, network based computing will soon offer to users a

8 Integrating New Cost Model into HMA-Based Grid Resource Scheduling 659 dramatic increase in the available aggregate processing power. We propose an HMAbased grid resource scheduling system and a new cost model is given. Especially, we formalize resource information and task requirement in detail. Comparing experiment shows that optimal solution under traditional cost model is no longer optimal under our model. References [1] I. Foster, C. Kesselman, and S. Tuecke, The anatomy of the Grid: Enabling scalable virtual organizations, Int l Journal on Supercomputer Applications, [2] K. Krauter, R. Buyya, and M. Maheswaran, A taxonomy and survey of Grid resource management systems, Software Practice and Experiance, Vol. 32, No. 2, Feb 2002, pp [3] F. Berman, R. Wolski, S. Figueira, J. Schopf, and G. Shao, Application-level scheduling on distributed heterogeneous networks, in Proc Supercomputing, Pittsburgh, PA, USA, [4] B. Davie, S. Casner, C. Iturradle, D. Oran, J. Wroelawski, Integrated Services in the presence of Compressible Flows, Internet Draft, February [5] I. Foster and C. Kesselman, The GRID: Blueprint for a New Computing Infrastructure, Morgan-Kaufmann, [6] Kazem Najafi and Alberto Leon-Garcia, A Novel Cost Model for Active Networks, Communication Technology Proceedings 2000, vol.2, pp