Methodology for risk management in systems development

Methodology for risk management in systems development Vrassidas LEOPOULOS Mechanical Engineer Dpt National Technical University of Athens Iroon Polytechniou 9 GREECE Konstantinos KIRYTOPOULOS Mechanical Engineer Dpt National Technical University of Athens Iroon Polytechniou 9 GREECE Abstract: - Aim of this paper is to provide a framework, able to identify and aid the assessment of risks on specific systems structures. A company (provider) develops the aforementioned systems for a client. Risk management is considered as a key tool for decision making in the early and most critical phase of this development, namely the bidding phase. The goal is a method for the use of Risk Management as an integrative decision factor during the determination of the most appropriate system among all the possible ones, for each specific bid. The paper is illustrated by a Case Study for the development and installation of a Production Planning and Control System, which presents the approach and the benefits stemmed from the method. Key-Words: - risk management, bidding process, systems solutions 1 Introduction A systems provider, when submitting a proposal for undertaking the implementation of a system, considers often alternative technical solutions. However, most of the times, only one solution has to be chosen as the best and final to incorporate in the offer to the possible customer. Bid managers have to capture the critical decision factor that will point out the system with the higher possibility to be accepted by the customer. On the other hand, this possibility should be balanced to the ability of the provider (contractor) to fulfil the terms and agreements of the contract if the project is finally assigned. Moreover, that procedure is limited by the short period of time that the bidding lasts. In this paper a method is proposed, that uses Risk Management as an integrative decision factor during the determination of the most appropriate system, for each specific bid. The study is based on an innovative methodology, which reduces significantly the time required to prepare and evaluate different system solutions. Furthermore, Risk Management is integrated in the methodology processes and aids the Bid Manager to distinguish between the high and low risk systems, in terms of bidding success probability and development feasibility. The rest of this paper is organised as follows: In Section 2 the problem of the study is formulated. The method proposed for the solution of the problem is described in Section 3. In Section 4 one can find the presentation of the case study that illustrates the method, and the results are analysed. Finally, in Section 5 the conclusions and opportunities for further research in the topic are presented. 2 Problem Formulation The description of the method follows three processes. The first one concerns the bidding process itself and how this is organised in practice. The second one concerns the sub-process of the development of potential technical (systems) solutions fulfilling the client s specifications. And the last one is the Risk Management sub-process and the decision support. 2.1 Bidding Process In a provider client relationship for systems development, the bidding process has been recognised as the most critical phase during the whole development s life cycle [1]. The intrinsic uncertainty may either be used in favour of the provider as an opportunity or against him as a threat. The typical bidding process is composed of eight

steps as those presented in Fig. 1 [2]. RFP exploration Preliminary RFP evaluation Proposal preparation start Industrial Organisation Technical solution development Bid Profile development Decision value of the proposal for the customer. Bid profile describes the set of features characterising the proposal and the relative strength of each one considering customer s required performances, described explicitly or not, and competitors behaviour. The Decision step, should provide the Best and Final Offer. During this step the best technical solution for the proposed system is selected and all the other features of the proposal, such as price asked, subcontractors, etc., are definitely decided. Process Completion step is used in order to assess the efficiency of the specific proposal and provide the lessons learned to the company. Process completion Fig. 1: Bidding Process Steps The Request For Proposal (RFP) Exploration step is the market research for new RFPs. This step is a perfect opportunity for implementing every effective method or trick we have in quiver in order to achieve more and earlier information, since this could be a great competitive advantage [3]. The Preliminary RFP Exploration step is used in order to examine the level of coherency of the RFP with the providers profile and scope of business. If the RFP is relevant, then an abstract of RFP baseline is documented for further analysis. A positive outcome of the previous phase leads to the Proposal Preparation Start step, which purpose is to set up the proposal team in terms of resources assigned, procedures adopted and tools to be used. This step includes also the planning phase defining proposal budget and schedule. The Industrial Organisation step integrates the politics as far as the customer or the market is concerned. Moreover, during this step a first make or buy decision is taken and necessary partners or sub-contractors are considered. The Technical Solution Development step is usually the kernel process for systems development providers. Expected result is a set of different technical solutions (systems) meeting the customer requirements. Systems components have to be decided along with the partners and sub-contractors needs for each. This is a time consuming process, which normally runs until the very last moment of a bid and most times in parallel with bid profile development and decision process, as well [3]. The Bid profile development step is linked to the 2.2 Solutions construction Although many types and alternatives of systems could be used in order to serve the same functionality, only one will be finally promoted and selected by the customer. In a nutshell, apart from any other factor interfering to customer s decisionmaking procedure, the technical solution offered is a dominant criterion. Another basic assumption is that the solution should fulfil the requirements but also should be capable of being developed within the available budget of the customer. However, the cost estimation of a technical solution is quite difficult. The earlier in the project one is trying to estimate the cost, the more difficult he succeeds. The cost envelope described by Smith [4] and can be depicted as in Fig. 2. In any case the Bid Manager and his/ her team are challenged to propose the best possible in terms of specifications system within the available budget that the customer is willing to pay. Estimated Project Cost Maximum Cost Minimum Cost Feasibility study Bid Implementation Phase of the project Claims Settlement Delivery Actual Cost Fig. 2: Cost estimating in different phases

2.3 Risk Management The first approach of the formal introduction of Risk Management (RM) techniques in the scientific community, was the effort put by Hammer [5] to apply it on technical solutions. The need that urged Hammer to do that was to avoid the technical risks events that may lead to the failure of complex systems, such as aeroplanes, oil industries, etc. RM has also been included in the theory of investments as financial risk and now it is probably the most advanced sector of RM [6]. Furthermore, RM has been applied to human errors and occupational safety [7]. Recent trends suggest the application of RM during the bidding process and its adoption by the senior management [3]. The steps of Risk Management can be summarised as in Fig. 3 [8]. The first step of the process is the Development of a Risk Management Plan. This Plan sets the base for Risk Management, including elements such as, the frequency of reporting, the milestones and everything that a project plan would be consisted of. The second step of the process is the identification of risks that might affect the proposal or the project if the provider wins the auction. Identification is an important step of the process since bid managers cannot cope with problems that have not been identified. There are many techniques indicating the way to identify risks, such as checklists, experts interviews, etc [9]. The third and fourth steps of the process address the analysis issue. Depending on the amount of information available or desirable, analysis could be either qualitative or quantitative. Several techniques are available for risk analysis. Some of Development of Risk Management Plan Follow Up and Control Risk Identification Mitigation Actions Plan Qualitative Analysis Quantitative Analysis Fig. 3: Risk Management Steps them are three point estimate, decision trees, Monte Carlo simulation, etc [10]. Next step for RM is the Mitigation Action Plan, i.e. the definition of specific and effective response, in order to smooth or completely eliminate the risk that may put the project (or the bid) into jeopardy. Preventive or corrective actions maybe used in order to obtain the minimisation of risk [8]. The last step, which is the Follow Up and Control of risks, aims to assure that the outcome from the previous steps is still valid as the time passes by, the mitigation actions defined are really efficient and that every new risk is registered. Especially for the implementation of Risk Management during the bidding process, this last step has to be performed in the limited time interval between the issue of a Request For Proposals (RFP) and the due date of the delivery of the proposal to the customer. In many cases changes that should be controlled are witnessed at the last moment, especially concerning external risks. 3 Problem Solution The method presented in this paper aims to integrate the techniques of risk management and cost estimating during the bidding process so as to come up with a solution that may improve the quality and the efficiency of proposals. The core conception on which the method is based on, is the creation of a Corporate Memory that includes all the technical solutions and its subelements that have been built in the past. This memory is enhanced with two types of information. The first is that each element is linked to the actual cost that comes as a return from experience, and the second one is that each element is linked to the related risks that have been experienced (Fig. 4). The benefit of introducing such a kind of Memory is double. The first advantage that could be easily deduced is that the Bid Manager and his team may construct in very short time several technical solutions for the requested system by assembling system s components (technical elements) that have been used in the past. Moreover, each one of these solutions goes with a relevant cost that is calculated in respect to the real values that have been defined from the return of experience. On the other hand, as stated earlier, each system s component (technical element) goes with a list of related risks. These risks have an impact either on cost or development time, given that the system s functionality is not negotiable.

the following Case Study. Fig. 4: Structure of Corporate Memory An efficient use of these risks may transform Smith s diagram (Fig. 2) and narrow the breadth between the minimum and maximum estimated cost, especially during the early phases of the project. As one can see in Fig. 5, the breadth between minimum and maximum cost is significantly reduced when the Corporate Memory is used. This result is explained by the use of the Corporate Memory that is the use of more precise Knowledge acquired from the experience of the provider. The use of that Knowledge reduces the envelope of the uncertainty, which intrinsically exists at projects early phases. The method as presented here is spotted on the Technical Solution Development step of the Estimated Project Cost Maximum Cost Minimum Cost Feasibility study Bid Breadth without Corporate Memory Breadth with Corporate Memory Implementation Phase of the project Claims Settlement Delivery Fig. 5: Deviation of cost variation Actual Cost Bidding process and its practical use is described in 4 Case study The Case study concerns an RFP issued by a privet sector industry in Greece. The firm is one of the leading in the field of packaging industry in Greece operating as a sub-contractor of several multinational firms. The request concerned the development of a dedicated software system for the Production Planning and Control (PPC) of the productive unit of the firm. The system should support the PPC functions within the existing infrastructure. Potential provider (Provider, in the rest of this paper) was a Greek consulting company, activating in customised development, consulting and installation of industrial and administrative systems. Experienced engineering professionals and academic researchers founded the company in 1992. The IT department employees 10 industrial and software engineers of MSc and PhD level with a turnover of 150 million drs. The mainstream work concerns customised development and installation of management information systems. The budget and due date restrictions being extremely tight, the Provider had to implement the methodology described earlier in order to come up with a more accurate estimation of cost and development time. The study was focused on the technical solution development step of the Bidding process (Fig. 1), especially on the capability of the Provider to develop alternative system solutions by using the Corporate Memory. Although just an initial corporate memory has been used, the outcome was that the proposal was ready in shorter time with estimations closer to the real outcome. This Corporate Memory has been constructed with the PRIMA tool, which is a dedicated tool to the development of technical solutions and Risk Management during the Bidding Process, and its functions stem from the method described in this paper. The systems structure and components (technical solution) proposed is presented in an abstract level, for reasons of brevity, in Fig.6. The modelling of the technical solution follows the principles of Product, Process and Resource (PPR). This means that each system is decomposed to sub-systems and processes that may built it and the process is decomposed to sub-processes and resources that perform it. This type of modelling allows the easy reuse of components that have been previously used from the provider.

In this Case Study, the proposal team identified the key sub-systems the requested system consists of. Fig. 6: Construction of a system solution using the Corporate Memory of the PRIMA tool PPC sub-systems were the MRP, Scheduler, H/W Network and the necessary Interfaces. The bidding team with the aid of the PRIMA tool, searched the Corporate Memory in order to find the processes and resources that have been used in the past to construct each one of these sub-systems. The first three subsystems (MRP, Scheduler and H/W Network) had been, indeed, developed again in the past so the only thing that remained to be done was a slight customisation. The fourth sub-system included the necessary Interfaces, demanding different effort since it had to do with the specific installed infrastructure of the customer. The systems components picked up from the Corporate Memory carry with them the risks that have been faced in the past. Figure 7 depicts the PPR structure for the technical solution of the Case Study. In addition to the PPR elements, explosion shapes are included, which stand for risks attached to the corresponding element. For instance, it has been experienced in the past that it is difficult to find programmer experienced in MRP coding. This means that the project, once undertaken by the provider, may be threatened by a delay or / and an increase in cost, which has already been measured from actual data from previous development of similar systems. The benefit of the use of the methodology was that the provider managed to make a better approximation of the cost of the system offered. The bidding team took into account the actual cost of similar system components developed in the past and moreover, the cost of risks that were very likely to be faced again, since they had already appeared in similar projects. In other words, the bidding team was enabled to offer a more reasonable price and schedule. In addition a better confidence that the terms and agreements of the contract, if the project was assigned, as it really did, was possible. 5 Conclusion further research PRODUCT Production Planning & Control Insufficient Network Incompatibility PROCESS MRP Scheduler H/W Network Interfaces RESOURCE RISK Coding Coding Implementation Coding Integration Specifications Specifications Programmer B Programmer B Technician Programmer A Programmer A S/W Engineer Prod. Manager H/W engineer Mech. Engineer Difficulty to find programmer available Bugs / Specifications inconformity Fig.7: Corporate Memory Risks attached to PPR model

The use of a Corporate Memory integrates risk in the technical solution for the development of a dedicated software system. A competitive advantage stems for the provider adopting the method. As stated in this paper, the reuse of system s components may narrow the envelope for the estimated cost during the bidding process, giving thus a better estimation of the project s results. The method as presented here is focused on the Technical Solution Development but can easily be prolonged to cover all the steps of the bidding process. The authors, within the PRIMA project (see Acknowledgements) are now further investigating the benefits of risk integration in other aspects of a proposal apart from the technical solution, such as the external risk stemming from the interaction of the proposed system with the environment (customer and competitors). References: [1] C.Chapman, S.Ward, J.Bennell, Incorporating uncertainty in competitive bidding. International Journal of Project Management, Vol. 18, 2000, pp. 337-347. [2] PRIMA, Description of as-is process methods synthesis, expression of needs and identification of key requirements, PRIMA project deliverable, 2000. [3] Tatsiopoulos I., Leopoulos V., Kirytopoulos K., 2001, Risk as a strategic decision factor for the competitive bidding process in contract manufacturing, IFIP Conference, Aalborg, Denmark. [4] Smith N., 1995 Project cost estimating, Thomas Telford Ltd., London. ISBN: 0 7277 20322 5 [5] Hammer R.W., 1972, Handbook of system and product safety, Englewood Cliffs, N.J.: Prentice Hall. [6] Alexander C., 1999, Risk Management and Analysis: Measuring and Modelling Financial Risk, John Wiley & Son Ltd. [7] Kirwan B., 1998, Human Error Identification techniques for risk assessment of high risk systems-part 1: review and evaluation of techniques, Applied Ergonomics Vol: 29 (3), pp. 157-177. [8] Project Management Institute, 2000, Project management book of knowledge. PMI. [9] R.Wideman, 1992, Project and program risk management: a guide to managing project risks and opportunities. PMI. [10] Kliem R, Ludin I., 1997, Reducing Project Risk, Gower Publishing Ltd, UK. Acknowledgments: Several researchers and managers contributed to the work presented in this paper. People from academia and industry have been working for more than two years now towards the common goal of delivering a method and tool which will help to the improvement of competitiveness and effectiveness of industries during the bidding process. The work has been formally settled in the scope of PRIMA (Project Risk Management) project, which is partly funded by the European Commission and has been expanded through a case study concerning systems solutions building by a Greek Consulting Company. Objectives of the PRIMA project is to define develop and disseminate a Management by Risk method and the associated software tools, which are a Risk Corporate Memory and a Decision Support System for bidding. Partners of the PRIMA projects are important European organisations: Alcatel Space Industries, University of Toulouse 1, CR2A- DI, Andalusian Association for Research and Industrial Cooperation, Sociedad Anonima de Instalaciones de Control, National Technical University of Athens, Hellenic Company for Space Applications, Politecnico di Milano and Snamprogetti.