A Framework for Scope, Cost and Effort Estimation for Service Oriented Architecture (SOA) Projects

Transcription

1 A Framework for Scope, Cost and Effort Estimation for Service Oriented Architecture (SOA) Projects Liam O Brien NICTA, Tower A, London Circuit, Canberra, ACT 2601, Australia RSISE, Australian National University, Canberra, ACT 0200, Australia Liam.OBrien@nicta.com.au Abstract Determining the scope, size, cost and effort of a Service Oriented Architecture (SOA) project is important to managing the risk of cost blowout happening during the project, building the business case and securing the funding for the project. Little work has been published on examining the various aspects of SOA projects and in having a systematic approach to determine the scope, size, cost and effort for such projects. This paper examines in detail various types of SOA projects and proposes the SMAT- AUS framework for capturing and using details about various aspects of SOA projects to determine the scope and estimate cost and effort. We describe the framework in detail and show the various dimensions of the framework that will impact on the scope, cost and effort including the technical and social/cultural/ organizational aspects, as well as the maturity of the organization in terms of its experience in undertaking SOA projects. We also give details of a case study using the framework. 1. Introduction Many organizations are adapting and transforming their business processes and IT systems to meet new challenges and respond to customer demands. To support this transformation many of them have already started SOA initiatives and have implemented or are implementing SOA-based systems. The promise of cost-efficiency, agility, adaptability and legacy leverage has attracted many organizations to try the SOA approach and in many cases make substantial investments in new systems that are based on SOA. As organizations begin work on their SOA initiatives there are various types of projects that they may undertake. Organizations have to determine what is involved in each of these projects i.e. determine their scope and size, determine what activities need to take place and what needs to be developed and/or purchased and determine how much these projects will cost in terms of effort and cost. In many cases (especially in government contexts) business cases with detailed budgets have to be prepared before the necessary approvals can be obtained from those funding the work. Current approaches to determining the scope, cost and effort involved in SOA projects include asking experts within an organization how much effort is involved in specific tasks and building in contingencies, or using some existing project management tools to make a determination of how much the development of a specific service will cost. However there is a lack of approaches to systematically analyze the scope and size of the various types of SOA projects (especially the nontechnical dimension) and determine what is involved in such projects and estimate the effort required and costs involved. This paper outlines an initial approach to developing a framework for capturing what is required in scoping, sizing and cost and effort estimation for different types of SOA projects. The rest of this paper is structured as follows. Section 2 outlines various types of SOA projects and activities within them. Section 3 outlines the current approaches to cost and effort estimation in SOA projects. Section 4 outlines the SMAT-AUS framework for scope, cost and effort estimation for SOA projects. Section 5 gives details of a recent application of the framework. Section 6 concludes the paper with details of planned developments within the framework and plans for applying the framework further. 2. Types of SOA Projects When an organization wants to develop SOAbased systems it first needs to understand what services it will need. In many cases an organization will

2 develop its Business Enterprise Architecture (BEA) to clearly identify what business processes it will need for the future and how these will be supported by the various services, IT systems, infrastructure and platforms. Identifying what services are needed can be done using a top-down or a bottom up approach or a combination of both. In the top-down approach the set of business processes are analyzed and the services that are needed are identified. In a bottom-up approach the existing processes and IT systems are analyzed and services needs are identified. Regardless of the approach that is used the identification of what services are needed is a major activity that the organization should undertake. For the purposes of this paper we assume that the set of services have been identified and are outlined in the BEA. The BEA should also outline the SOA initiative that will be put in place to deliver the required set of services. The BEA should act as a blueprint for the SOA initiative and used as a starting point for various types of SOA projects. The types of SOA projects that an organization may engage in include: Service Mining: the identification and mining of services from existing/legacy systems; Service Development: the development of services from scratch; Application Development: development of applications using services; Service Integration: the integration of common or shared services with existing systems; SOA Infrastructure: the development and/or acquisition of an SOA infrastructure; SOA Governance: the development of governance policies and procedures (including establishing and monitoring of Service Level Agreements); SOA Architecture Analysis: analysis of the architecture of SOA-based systems to determine if it meets specific QoS requirements (e.g. performance or security). For each of these types of project there are various activities and other factors that will involve expenditure of effort and cost. Common to each of these projects will be the costs involved in development of detailed project plans and management of the project. Further costs may include the development of a business case for the project to submit for funding approval. These cost factors have to be taken into account in determining the overall cost of an SOA project. 2.1 Identification and Mining of Services When an organization decides to use SOA it normally wants to leverage its investment in its existing systems. As a result there is a need to determine what service needs can be satisfied by mining and reusing components from its existing systems. In order to do this there are several activities that it will need to be undertaken including: Understanding the existing systems and identifying potential components that could satisfy service needs. Details about the service needs and the existing systems and components need to be captured. This may involve documenting the existing systems and may even entail using architecture reconstruction [13]. Determining what needs to be done with those components to migrate and reuse them as services. This may involve wrapping the components, changing the internals of the components and building new interfaces. Determining the cost and effort of mining, changing and reusing the components. The effort and cost will be associated with making the changes to the components, developing the wrappers, etc. Details about what additional infrastructure (software, hardware, licenses, etc.) is required needs to be captured and will be part of the additional cost factors. A method to support scoping and effort and cost estimation in mining and reusing existing components is the Software Engineering Institute s Service Migration and Reuse Technique (SMART) method [5]. In our framework the SMART method may be incorporated along with the various details for this type of project as well as the artifacts generated. 2.2 Development of Services As part of an organization s SOA strategy it will need to develop services from scratch that are specific to its organization needs and cannot be reused from anywhere else. Different approaches to developing services can be used such as the traditional spiral development models to more agile approaches. The approach used can also be top-down which is driven by business processes, Model Driven Architecture [11] or bottom-up which is driven by existing legacy and componentization of legacy. When dealing with the development of services the quality-of-service (QoS) aspects of the services can be a major concern. One of the main issues is the problem of time-boxed development of services versus designing for QoS. If a set of services have to be developed in a particular timeframe especially in more agile approaches how is QoS for the services handled?

3 When developing services an organization will have specific QoS requirements which the services and applications will have to meet. These requirements can guide the development of the services and applications but there are few development methods that can guarantee that the resultant services will meet their QoS needs. Detailed testing of the services will need to be done to determine if in fact they actually meet their QoS requirements. The main activities involved in the development of services (which are similar to the development of other software) are: Determining service requirements. The service requirements are captured; Developing the service architecture and a detailed design for each service. The service s architecture and design are documented; Implementing the services. Code and other artifacts need to be built; Testing of the services. This may involve building additional software to call/interface with the services. Some of the main requirements in building services are flexibility, granularity of services and making it easy to integrate and manage services. Thus architecting and designing service may involve more effort in order that these aspects are correctly handled. There are lots of tools and technology available to support services development in SOA but there is still a considerable effort and cost involved in the development of services. As well as the development cost (to carry out the activities listed above) there are other cost factors that will have to be taken into consideration. Acquisition of development and testing environments for the services (hardware, software, tools, licenses, etc.), up-skilling and capability development of architects and developers and training in specific SOA technologies and WS-* standards. 2.3 Application Development from Services Development of applications from services can use services that are either home-grown or provided by external organizations. These services will need to be integrated into a complete application that meets the needs of the business processes of the organization. The main activities involved in development of an application include: Determining and capturing the requirements for the application; Developing and documenting architecture and detailed design of the application; Implementing the application. Pieces of the application are already available as services so these will have to be interfaced with the rest of the application. If services are used dynamically then this needs to be accounted for within the application. Also the nonavailability of services/service exceptions has also to be handled. Developing the glue code that integrates the set of services and handles data transformations, service exceptions, etc. is all part of the application development and may be where most of the effort and cost is expended; Testing of the application. This takes on more significance as the services consumed by the application will need to be thoroughly tested and if services are dynamically identified and used this adds further complications. As well as the development cost there are other costs that have to be taken into consideration. Acquisition of development, testing and runtime monitoring environments (hardware, software, tools, licenses, etc.), up-skilling and capability development of architects and developers and training in specific technologies are additional cost factors that form part of the overall cost of application development. 2.4 Integration of Services As an organization moves forward with its SOA strategy services will be identified, either internal or external to the organization, which will need to be integrated with the organization s existing systems. The set of activities involved in such a project include: Determining what systems the services will integrate with. Details about the services and the existing systems need to be captured; Understanding the existing systems and identifying the changes that will need to be made to each system to enable integration. Details about what changes will need to be made to each system need to be captured; Determining the scope of the work involved and estimating the cost and effort of doing the work. The scope of the integration may involve acquiring new infrastructure and middleware to enable the services to be integrated; Determining what the impact of different architecture alternatives will be on various quality attributes and QoS concerns for the systems. This may involve doing evaluation of the architecture of the resultant SOA-based system; Developing SLAs for the organization s systems (if not already in place) and establishing QoS requirements for the services that the systems consume. Negotiating SLAs

4 for the services with the service providers. Capturing QoS requirements and making sure these are negotiated and established in SLAs is an important activity to ensure the success of an SOA initiative. We are currently developing a method for determining the scope, size, effort and cost of service integration projects. We are developing a set of templates for data capture and exploring what constitutes the set of factors of a cost function for service integration. All of these are being incorporated into the framework. 2.5 Development/Acquisition of an SOA Infrastructure To execute and manage SOA applications and services, an organization needs an SOA infrastructure. An SOA infrastructure consists of several pieces that support various aspects of SOA including security, governance, management, orchestration and resourcing (e.g. virtualization). There are many products from various vendors such as IBM, Microsoft, SAP, Oracle, etc. that an organization can acquire. Each product provides a set of technologies that cover most of what an organization needs. The main activities involved in the acquisition of an SOA infrastructure include: Determining the requirements for the SOA infrastructure; Evaluating several infrastructures from different vendors that may be suitable. It may be a case that more than one infrastructure satisfies the requirements or there is a need to have some combination; Selecting and acquiring the various pieces of the infrastructure. This will involve acquiring software, hardware, licenses, etc.; Customization of the infrastructure within the organization. This may involve having consultants from the vendors work with the organization s personnel; The customization work may require considerable effort for the organization. As well as the cost of this effort other cost factors such as technology licenses, hardware, training on the technology, consultancy costs, infrastructure testing, etc. all have to be taken into account in the determining the overall cost of the acquiring the SOA infrastructure. In some cases an organization may decide to undertake development of some or all of their SOA infrastructure needs themselves. In this case the activities will be similar to the activities outlined above for the development of applications (in this case the application is the SOA infrastructure). The effort and costs involved in doing this may be much greater so an organization will have to give careful consideration to going down such a path. In some cases an organization may have to go down this path if the infrastructure or environment is quite specialized and no existing vendor has a product that can be customized for this situation. 2.6 SOA Governance SOA governance is an essential activity to manage many levels of SOA decisions correctly. There are various activities that need to be undertaken to establish proper governance of an SOA initiative within an organization (adapted from [12]). These include: Developing the strategy and goals for the SOA initiative (determining what is governed and why?); Determining where funding will come from, who has ownership and who gives the necessary approvals (who owns what? what gets funded and by whom?); Determining the necessary structures, processes and governance mechanisms that need to be in place within the organization; Determining for each governance process what are the roles, responsibilities and procedures for managing SOA activities; Developing policies and enforcement mechanisms for policies related to the use of standards, security, release of new versions of application and services and re-use of services; Developing a set of metrics to show progress of the project and the overall SOA initiative. Determining what are the business outcomes to be achieved and how are they measured and by what metrics; Determining what the behavioral model for SOA governance will be. Determining what are the incentives, penalties and rewards for appropriate SOA Behavior. More research still needs to be done in various areas of governance and clear paths through all of the issues in governance have to be developed that will lead an organization to success with its SOA initiative. Research and experience is needed to determine what governance issues are core to the success of an SOA initiative and need to be tackled early in the SOA lifecycle and which can be left to later. Furthermore there is a need to determine what areas can be automated and what tools are there to help automate governance. One of the areas of SOA governance that we are working on is on Service Level Agreements and

5 the use of methods and tools to determine the performance and QoS of services and applications that can be used in developing and negotiating SLAs. All of the activities above will involve costs and maybe an organization will not get it right the first time and SOA governance may evolve as the organization gets more experienced with its SOA initiative. 2.7 SOA Architecture Analysis The analysis of the architecture of an SOA-based system may examine many quality characteristics of that architecture such as performance, scalability, security, adaptability, etc. In analyzing the architecture there may be similar tasks undertaken for a range of characteristics. For the purposes of this paper we will concentrate on the main activities involved in analyzing the performance and scalability of the architecture. The analysis can be done in several ways from load testing of the system to building performance and scalability models of the system and running simulations on these. We will outline the activities involved in the latter, which include: Determining and capturing the performance and scalability requirements for the application; Understanding the architecture of the system and identifying the services, workflow and the physical deployment architecture (servers, networks, etc); Obtaining unloaded performance data for the various services and workflows and understanding the demand that will be on the system; Building a model of the system based on the services, workflow and servers and parameterizing the model with the performance data; Running simulations on the model based on the demand to determine the response time, throughput and capacity of the various pieces of the architecture and other metrics that may be of use; Determining if the system meets the requirements and if not where in the architecture things needs to change (software change, additional CPUs on servers, additional servers, etc); Using the analysis data to negotiate and establish service level agreements for the system; An SOA-based system may use services that are either home-grown or provided by external organizations. If external services are used it may be more difficult to carry out the analysis as it may be difficult to get access to performance data. Service level agreements with the service providers of these external services will have to be negotiated and be in place in order to make any guarantees about the QoS of the system. A further complication on analyzing the architecture for QoS is that a system may dynamically identify services to use, so it is not possible at design time to know what services will actually be used. All of the activities above will involve effort and cost and there are other cost factors that have to be taken into consideration. In the example above additional costs could included the acquisition of performance testing and runtime monitoring environments (hardware, software, tools, licenses, etc.), obtaining the performance data and contracting of an organization to do performance and scalability analysis, are additional costs that could form part of the overall cost of architecture analysis of SOA-based systems. 3. Current Approaches to Cost and Effort Estimation for SOA Projects A comprehensive review of software development cost estimation studies by Jørgensen and Shepperd [6] and subsequent work by Shepperd [7] categorizes several different types of estimation approaches. The main ones are: Algorithmic or Parametric models this approach provides a model which is typically in the form of a function or set of functions that relates size of the task to the degree of effort (cost) to perform it. COCOMO is one of the best know examples and it has evolved in the COCOMO II model [10] and COCOTS (for component-based projects). The model can be calibrated to local environments; Analogy this approach characterizes projects into a set of features (e.g. number of interfaces, number of functional requirements, etc.). Data from completed projects is stored in a database and when a new project is identified the problem becomes finding similar projects to this in the database; Expert Judgment this approach involves an expert or group of experts studying what needs to be done and deriving an estimate of the effort and duration of a project which can then have a cost associated with it based on unit costs (such as the cost of hiring architects/developers for a month). Function Point this approach is used to measure the amount of business functionality a system provides to a user. Once the number

6 of function points for a system is known the level of productivity of the development team developing those functions can be used to estimate the overall effort (cost) of the development of the system. Alternatively the function points can be converted to lines of code and an algorithmic model such as COCOMO can be used to do effort estimation. In examining the literature on SOA cost and effort estimation very little seems to have been published on the topic. For realization of individual services some people have used the COCOMO II software development cost-estimation model [1] or Scrum [2]. For the total cost of an SOA project Linthicum [3] outlines a set of general guidelines which are outlined below. The use of Function Point in an SOA environment has not been examined in much detail and it has been observed that major issues may occur when applying function points to SOAs [4]. Almost nothing has been reported in the literature on SOA on analogy-based or expert judgment based estimation. Tansey and Stroulia [1] outline an approach to an integrated model for cost/value in service oriented computing. To determine the cost of a service they used COCOMO II in estimating the cost of creating or modifying a service in an SOA. They use BPEL definitions to represent a service in its current state and future state and used COCOMO II (either the Post Architecture or Early Design model depending on the status of the project) to determine the cost of the services based on user calibration. The tool they developed prompts for additional cost associated with the project. There has been almost no application of Function Point approaches to estimation in SOA projects. According to Santillo [4] there are many issues with using function point estimation on SOA projects. The main issue is in boundary positioning which deals with what parts of the system belong to the part of the software to be measured and which are outside or independent of it. The main concern is determining the unit of size in order to be ale to apply the function point approach. In many cases an organization that wants to implement SOA-based systems hires a contracting firm or if it is a large project one of the large system integrators to undertake the development of the SOA. Many of these consultancy or system integrator organizations use their own proprietary approaches to determine the cost of an SOA implementation. Many use complex models defined in spreadsheets to determine what the cost will be and once the cost has been decided with the contracting organization usually the consulting organization starts the project. When the contracting organization receives the costing from the contractor they don t have any way of validating the costs and they have no framework against which they can determine if all of the factors that should be included have been included. Linthicum [3] provides some guidelines to determining the cost of an SOA project and states that many organizations don t have a clue how to approach this, and in many cases grossly underestimate the cost of their SOA. In many cases the cost of the SOA is usually underestimated in order to gain approval for the project and the true cost of the SOA which is usually not revealed until later is often much higher. Linthicum states that time is needed in order to understand what needs to be done and prepare the budget. Determining the scope of an SOA effort is a major task and time must be set aside for this and it will involve some cost. Understanding what needs to be done involves understanding the domain in detail including: Number of data elements Complexity of data storage technology System complexity Service complexity Process Complexity New services needed Enabling technology Applicable standards Potential risks The Cost of the SOA = (Cost of Data Complexity + Cost of Service Complexity + Cost of Process Complexity + Enabling Technology Solution). Linthicum provides some details about how to determine the elements of the cost such as: Cost of Data Complexity = (((Number of Data Elements) * Complexity of Data Storage Technology) * Labor Units). The Complexity of the Data Storage Technology is expressed as a % (between 0 and 1) with Relational being 0.3, Object-Oriented being 0.6 and ISAM being 0.8. The labor unit is the amount of money it takes to understand and refine one data element. A similar approach to determining the other elements can be used but details are not provided. Linthicum outlines that these formulas should be used with an organization s own project management and project costing methods. Overall there is very little published work on scope, cost, and effort estimation for SOA projects. There is no framework that brings together the various types of projects, the various elements of the cost and effort or talks about the various dimensions of scope,

7 cost and effort such as the technical as well as the social/cultural dimension. Also in dealing with costs of SOA projects there has been little written about how to determine not only the development cost but the total cost of ownership of a system or the total life time cost of the system over a number of years. All of these factors have been taken into account in the development of our framework for scope, cost and effort estimation for SOA project which we outline in the next section. The following sections outline more details about the various components of the framework. 4. A Framework for Scope, Cost and Effort Estimation for SOA Projects Based on a review of the literature on effort/cost estimation in SOA projects, studying different types of SOA project and identifying the need for an overall framework we have developed the SMAT-AUS Scope, Cost and Effort Estimation Framework for SOA projects. An overview of the framework is given in Fig. 1. The framework consists of the following: There are different types of SOA projects and an organization may undertake one or more of these project types For each type of project there are: o Methods that can be supported by tools for determining what needs to be done (scope), and what are the activities and other sources of expenditure of effort and cost o Templates for capturing data about a project (e.g. information on services, legacy systems, stakeholders, etc) o A set of cost factors which are specific to each project type o A set of cost functions for determining the costs within each project type A set of effort and cost factors which may be common across one of more project types An SOA Maturity Model that is used across all project types to determine the organization s ability to be able to undertake and successfully complete an SOA project. Depending on how mature an organization is this may have an impact on the effort and cost of an SOA project For each project type there are two significant dimensions that have to be taken into consideration. These are the Technical dimension which involves the technical work involved and the Social/Cultural dimension which involves the social/cultural/ organizational issues within an SOA project. Figure 1. Overview of the SMAT-AUS Scope, Cost and Effort Estimation Framework. 4.1 Project Types The set of project types are the set outlined in Section 2. For each project type there is a set of methods, templates, cost functions and project type specific cost factors. Methods: each project type will have one or more methods associated with it to determine the scope, effort and cost factors of the work that needs to be done. An example of the type of method is shown in Fig. 2. This method is for determining the scope of the work and the cost and effort, risk and difficulty of the Service Integration project type. Templates: templates are used to capture data about artifacts such as details about existing systems, services (Fig. 3) and various options that might be available, etc. Templates can also be used to capture risks and issues for particular types of projects. Cost Functions: with the variety of SOA project types there is not going to be a single cost function that covers all of the effort and cost estimation of every type of project or the entire set of costs associated with a particular project type. As a result several cost functions may be needed and some of these may be specific to a particular project type. For example the effort and cost of developing policies and service level agreements would be specific to the SOA Governance project and these would be quite different from effort and cost estimation of Service Development. Existing effort and cost estimation functions and methods will be examined for applicability to the various project types and use within the framework.

8 4.3 SOA Maturity Model Figure 2. Outline of Method to Determine the Scope, Cost and Effort for Service Integration. Project Type Specific Cost factors: factors of cost and effort specific to each type of project are captured within the framework. For an SOA Infrastructure project the acquisition of an Enterprise Service Bus would be specific to that type of project. These effort and cost factors may not be applicable to other types of projects. 4.2 Common Factors The common effort and cost factors are factors that are similar across a number of project types. Examples of the type of factors that may be shared include those for hardware, training, software licenses and development environments. The overall cost and effort for a project will be a combination of the common cost and effort factors plus factors specific to a project type. Further common factors may include the effort and cost of development of the project plan and detailed budget, effort and cost of project management, and effort and cost of getting stakeholder buy-in. Not all of the effort and cost factors may relate to technical work and some additional factors may be developed for the Social/Cultural dimension. Figure 3. Shows an Example of a Template to Capture Details about a Particular Service. The SOA Maturity Model is an aggregate of certain factors that could have an impact on the cost and effort required for a project. Examples of factors include whether or not a detailed Business Enterprise Architecture has been produced and is available for use on the project, the skill level of the architects and developers within the organization in the use of SOA technologies and whether or not the organization has undertaken SOA projects before or this is their first venture into the SOA world. Each of these will have an impact on effort and cost for an SOA project. Further research is being carried out and the list of factors is still being developed and it is hoped to be able to distill these down to some coefficient of a cost function. 4.4 Dimensions of SOA Projects As mentioned earlier there are two main dimensions of SOA projects which are covered within the framework. Technical the technical dimension deals with the technical work involved including development of services, acquisition of hardware and software, licenses, etc. Social/Cultural/Organizational this dimension deals with the people and organization issues such as communication with and getting buy-in from stakeholders, the impact of change within the organization, training of people on new processes as a result of moving to a new SOA-based system, upskilling of developers and architects, organizational restructure, incentives to development teams to use services, etc. It is important that these considerations are taken into account on an SOA project as many projects fail because of the social, cultural and organizational issues within an organization not being addressed [8, 9]. In order to handle and resolve many of these issues substantial effort may be required and additional costs may have to be incurred within a project. Methods will need to be developed for scoping this type of work. Templates, cost functions and identifying project specific cost factors for this type of work will also need to be developed. 4.5 Other Considerations for SOA Projects In determining the scope, effort and costs involved in an SOA project there are other considerations that should be taken into account. Development vs Total System Ownership vs Life Time costs the effort and cost of a project could be just for the development of the software/service or

9 infrastructure with no consideration of the other ownership costs to put the SOA-based system into production. Alternatively the effort and cost estimate could include the total ownership costs, such as additional hardware, software, licenses, training, etc., of putting the system into production. Alternatively again the cost could be not only for total ownership cost, to put the system into production, but also the cost of maintenance and evolution of the system over a certain number of years. This may be more difficult to estimate as the future direction of a system may not be known. These may require additional methods, costs functions and identification of additional cost factors. Outsourced service provisioning costs associated with an outsourcing organization providing one or more services and the associated infrastructure could also form part of the cost of a project. In many cases outsourcing of services for an organization may be more cost effective than the organization developing and hosting the services themselves. An additional set of cost factors and cost functions may have to be developed for this case. Operation of the SOA-based system an SOAbased system may use services where there is a cost to the organization of using these services (pay-per-use). These additional costs have also to be factored into the total cost of ownership of the system and life time cost. Automation As SOA technologies mature more automation may be built into the various technologies such as SOA development environments. As more activities are automated less effort and cost may be associated with the various types of projects. 5. Case Study and Uses of the Framework In a recent engagement with a government agency in Australia we examined the costing of an early SOA initiative. The agency has embarked on an SOA initiative to introduce an Enterprise Service Bus to integrate several applications to improve operations. They have identified a set of requirements which they turned into a set of components and services needs. Using a group of experts they developed a cost and effort estimate for the SOA project. In applying the SMAT-AUS Framework we identified several areas where they had not fully scoped the work that would be involved and the cost and effort involved. They had identified most of the technical work involved but there was one area that was lacking, which was the effort and cost involved in semantically integrating the systems. The various systems hold similar but subtly different information in different formats. Semantic integration will be an important activity for the agency especially as it is planned to integrate further systems using the SOA approach. The agency had identified that there will be a changes in operations for the end users of the systems. However they had not determined the scope of the change and the cost and effort involved. Using the framework we outlined several areas that they will need to work on, such as identifying what exactly will change, what will be involved in training and updating the skills of the end users, the need for a change manager to be involved in the project and for that person to work with both the end user community, business analysts and technical people to make sure that the change in operations and the developed systems will be used properly and that the changes will occur with everyone working in unison. Parts of the SMAT-AUS framework are still being developed. As we develop these and extend other pieces it is envisioned that we will continue to evaluate the framework on real SOA projects to determine its usefulness. We can see the framework being used in a similar way as in the case study with other organizations. The effort involved in applying the framework will depend of the type of application. It can be used to help those starting out on an SOA initiative to help scope and determine what needs to be done and the cost and effort factors involved. This will involve a lot of effort over a period of time to fully scope and apply the various elements of the framework. The framework can also be used to validate that all of the various elements of an SOA project have been properly identified and taken into consideration and proper effort and cost estimates have been developed. We expect that it will take less effort and time to apply the framework in this context. 6. Discussion and Future Work The development of the complete SMAT-AUS Scope, Cost and Effort Estimation Framework will help organizations to get a better understanding of what needs to be done in scoping and sizing of SOA projects and in determining the effort and cost for such projects. The framework identifies not only the Technical dimension of the work but another dimension as well, the social/cultural/organizational dimension. It is important that factors associated with the social/cultural/organizational dimension are taken into account when determining, scope, effort and cost of SOA projects. Ignoring this dimension altogether or not paying adequate attention to it can be detrimental to a project. For each project type a set of methods (and tools), templates, cost functions, and cost factors (common or

10 specific) are being developed and can be used by organizations to do a better job of scoping the work on such projects and validating the costing estimation work done by third party organizations. NICTA is developing the EffortWatch tool which is an analogybased cost estimation tool and applying it on effort and cost estimation of SOA-based system. This tool is being incorporated into the framework. Development of most of the components of the framework is still ongoing and we are working with organizations to apply various pieces of the framework. Collaboration with various organizations to understand their SOA projects and determine the cost factors, the social/cultural/organizational dimension and understand how the maturity of the organization impacts on the success of projects is also ongoing. As the various components of the framework are developed it is planned to apply the framework on further projects to refine and mature the framework. Acknowledgments: NICTA is funded by the Australian Government as represented by the Department of Broadband, Communications and the Digital Economy and the Australian Research Council through the ICT Centre of Excellence program. [8] Virgo, P., Why to we never learn? The pre-conditions for public sector systems success, Transformation: Promoting new thinking in the public sector, Mian, E. (ed). never_learn.pdf. [9] Krigsman, M., 10 Reasons for IT Project Failure, May 2008, e. [10] Boehm, B.W., Clark, B., Horowitz, E., Madachy, R. Shelby, R., and Westland, C., Cost Models for Future Software Life Cycle Processes: COCOMO 2.0, Annals of Software Engineering, : p [11] Harding, C., Achieving Business Agility through Model-Driven SOA, January [12] Marks, E. A., Perspectives on SOA Governance, URL: _05_2324/ EMarks pdf [13] O Brien, L., Smith D. and Lewis, G., Supporting Migration to Services using Software Architecture Reconstruction, Software Technology and Engineering Practice, Budapest, Hungary, September 24th 25th, References [1] Tansey, B. and Stroulia, E., Valuating Software Service Development: Integrating COCOMO II and Real Options Theory, First International Workshop on the Economics of Software and Computation, at ICSE 2007, May [2] Doelen R., How Much Will Your SOA Cost?, March [3] Linthicum, D., How Much Will Your SOA Cost?, March 2007 [4] Santillo, L., Seizing and Sizing SOA Applications with COSMIC Function Points, Software Measurement European Forum, May 2007 Rome, Italy. [5] Lewis, G., Morris, E., O Brien, L., Smith, D., and Wrage, L., SMART: The Service-Oriented Migration and Reuse Technique, CMU/SEI-2005-TN-029, Software Engineering Institute, USA. [6] Jørgensen, M., and Shepperd, M.J., A Systematic Review of Software Development Cost Estimation Studies. IEEE Transactions on Software Engineering 33(1): (2007). [7] Shepperd, M., Software project economics, Future of Software Engineering (FOSE 07) at ICSE 2007.