A Cross-Industry Configuration Management Maturity Assessment

Transcription

1 Industrial Case Study A Cross-Industry Configuration Management Maturity Assessment A collaborative study by

2 2

3 PUBLISHED BY P3 ingenieurgesellschaft mbh and Institute for Information Management in Engineering (IMI) at the Karlsruhe Institute of Technology (KIT) Authors P3 ingenieurgesellschaft mbh Karlsruhe Institute of Technology (KIT) Institute for Information Management in Engineering (IMI) Dipl.-Wi.-Ing. Stefan Bierbrauer Tel Dipl.-Wi.-Ing. Till Reymann Tel M. Sc. PMP Masoud Niknam Tel masoud.niknam@kit.edu Prof. Dr. Dr.-Ing. Dr. h.c. Jivka Ovtcharova Tel jivka.ovtcharova@kit.edu 213 P3 ingenieurgesellschaft A Cross-Industry Configuration Management Maturity Assessment 3

4

5 TABLE OF CONTENTS EXECUTIVE SUMMARY INTRODUCTION OF PARTNERS INTRODUCTION Background & Motivation Configuration Management Maturity Model Configuration Management Maturity Assessment RESULTS DESCRIPTION OF THE SAMPLE RESULTS CM EFFECTS AND APPLICATION RESULTS MATURITY ASSESSMENT CONCLUSION SUMMARY AND FUTURE RESEARCH AUTHORS REFERENCES A Cross-Industry Configuration Management Maturity Assessment Table of contents 5

6 EXECUTIVE SUMMARY Growing complexity in product development, manufacturing and maintenance requires sophisticated management and systematic utilization of product information throughout the entire product lifecycle. To obtain a high level of quality and safety, as well as to identify and mitigate risks, Configuration Management (CM) has become an essential and well-recognized management discipline. Configuration Management is applied wherever products, facilities or services are generated with complex requirement profiles. Due to the holistic nature of CM, its principles can be implemented in all granularities and environments: From small to large organizations, from physical to virtual products. CM provides the means to structure products and services adequately, control their evolution effectively and ensure quality with respect to all existing interdependencies and stakeholders. Utilizing a CM maturity model developed by the Institute for Information Management in Engineering (IMI) at the Karlsruhe Institute of Technology (KIT) and its associated appraisal material developed in collaboration with P3 ingenieurgesellschaft, this study examines the maturity of Configuration Management in different industries. The core of the study is a survey, for which 67 respondents from ten different industries have provided an insight on their application of CM. By analyzing the survey results, the study not only provides a standardized view on the maturity of CM in different industries, but also identifies the weaknesses and fields of improvement which challenge companies today. We are very pleased to present the results of this study on the following pages. Although CM principles have widely spread across the industries in the past years, the maturity of CM application and the related strengths and weaknesses of in-field Configuration Management is hardly assessable or measureable. 6

7 INTRODUCTION OF PARTNERS Founded in 1996 as spin-off of the Fraunhofer Institute for Production Technology (IPT) and the University RWTH Aachen, the P3 ingenieurgesellschaft provides international management consulting and innovative engineering services to the automotive, aviation, telecommunication and energy industries. P3 is privately owned with 2.1 employees and partners worldwide. With Configuration Management as a strategic business area, the P3 CM service portfolio encompasses consulting services on processes, methods and tools, as well as operational implementation and support for various industries worldwide. The effective and efficient management of information throughout engineering processes is of essential importance for economic success. The Institute for Information Management in Engineering supports enterprises by providing innovative and practical IT-solutions and methodologies ensuring successful and sustainable competitive advantage in research and industry. IMI s research fields of Life Cycle Engineering, Collaborative Engineering and Virtual Engineering complement one another in a holistic approach for integrating process- and system information in a unified product knowledge model. IMI operates the Lifecycle Engineering Solutions Center (LESC) which offers a platform for interdisciplinary research at the KIT. LESC is an innovative research facility which aims at empowering interdisciplinary collaboration between internal and external research institutions, as well as the knowledge exchange and technology transfer to both the industry and society. A Cross-Industry Configuration Management Maturity Assessment Introduction of partners 7

8 INTRODUCTION Background & Motivation Configuration Management (CM) is a discipline that aims at providing consistency and accuracy of product knowledge throughout its lifecycle. It is the discipline to monitor and control the consistency of the product with its documented requirements throughout all lifecycle phases [1, 2, 3]. CM ensures that products and facilities, including all the systems, equipment and components, are accurately described at all times [2]. In addition, CM involves all parties which contribute and/or benefit from the data generated and traced. It is noted by many researchers and practitioners that implementing effective Configuration Management processes not only improves the safety in organizations, but also has direct positive impacts on return on investment, Product Lifecycle costs, on-time deliveries and product quality [1, 3, 4]. Industry best practice has shown that a set of four enablers contributes to the implementation and improvement of CM in enterprises and organizations: Product Identification & Modularization, Product Structuring & Documentation, Requirement and Change Management, CM Information Technology. Although the benefits of effective CM disciplines in place are prominent in all professional organizations, there have not been many works to present a clear roadmap to evaluate CM maturity in the field, prioritize their improvement activities and implement missing elements step-by-step [2, 5]. This lack of transparency escalates in higher levels when organizations intend to compare their know-how in the field with best practices in the industry. Therefore, it is highly beneficial for organizations to have a framework to evaluate their CM activities in order to identify their gaps and focus on future improvements in a more efficient manner [6]. Accordingly, the main purpose of this study is to utilize and validate the Configuration Management maturity model developed by the Institute for Information Management in Engineering (IMI) [5, 7], shown in the following chapter, and its associated appraisal material developed in collaboration with P3 Ingenieurgesellschaft. Furthermore, the CM practices of different industrial organization shall be evaluated in a way towards understanding the primary problem areas and future focus areas in the field. Therefore, the model shall enable a cross-industry comparison of the application of CM disciplines. Configuration Management Maturity Model The CM maturity model was developed at IMI based on a comprehensive review of the associated state-of-the-art standards [9, 1, 11, 12] and maturity models such as CMMI [13, 14] or the Systems Engineering Capability Maturity model [15]. The primary dimensions of Configuration Management discipline according to this model are illustrated in Figure 1. As one of the support disciplines in Systems Engineering and Product Lifecycle Management, the extent to which the CM discipline permeates an organization s structure and activities represents the organization s CM maturity. The concept of Maturity and Organizational Alignment [8] is used for categorizing different maturity levels of organizations. Figure 2 represents the authors proposition for four CM maturity levels. 8

9 strategy & performance Strategy & Performance represents: The existence and alignment of CM strategy and policy with the overall strategy, vision and mision of the entreprise. The suitable performance indicators to measure this alignment and the achievement of CM goals. process Process represents: The availability of standard processes that can stabilize the CM activities among all organizational units during all lifecycle phases and the approach to test and measure the efficiency of such processes. information technology Information Technology represents: The information system tools being used by the entreprise and the level up to which their functionalities match the necessities. The level of integration of these information systems into the business processes of the organization. organization & value-stream Organization & Value-Stream represents: The organizational structure of the enterprise and the level up to which the roles and responsibilities are defined with respect to CM. The way to cope with CM complexities through the supply chain e.g. dealing with sub-contractors. knowledge & support Knowledge & Support represents: The level up to which the entreprise and key personnel are knowledgable in CM areas. The initiatives and efforts of the management in supporting the CM activities throughout the organization. Primary Dimensions of Configuration Management Figure 1 level 1: initial not managed at all or managed only by some autonomous individuals level 2: managed managed by some projects level 3: standard managed by most of the projects in a standard way level 4: optimizing managed by the entire organization in a standard way implementing continuous improvement Configuration Management Maturity Levels Figure 2 A Cross-Industry Configuration Management Maturity Assessment Introduction 9

10 Following is a detailed description of CM maturity dimensions and sub-dimensions: Level 1: Initial At this level, there is no specific Configuration Management strategy and policy in place. People just rely on their own understanding of objectives behind CM application that might not be in line with the project or organization s overall strategy. The processes are ad hoc and different among different personnel. There is nobody responsible for managing and enforcing the processes and CM relies on individuals and their tacit knowledge. Changes are made by individuals with their interpretation of situations rather than having a full understanding of the change impacts. There is no standardized set of terminology and definitions for CM in place and therefore different opinions are introduced as standards based on different backgrounds and fields of work. The organization structure does not appreciate the necessary roles for CM activities and these activities are seen as less important compared to other positions in each project. There is no training identified for Configuration Management and the level of management support is limited. Configuration Management requirements have not been considered in the choice of information technology. Therefore, only a few of the needed functionalities are supported in the projects. In the best case, functions are provided by self-developed tools that are either offline or not connected to the main IT system. Level 2: Managed At this level, the generic need for a formal CM discipline is still missing, however some specific methods are in place. CM strategy and policy are developed for some of the projects but there is no official strategy in line with the organization s strategy in place. Therefore, objectives are only defined and pursued on project level for a few projects. Some CM processes are identified in combination with other processes at the beginning of projects and are accessible by everyone. These processes are developed and managed based on the knowledge of involved personnel in specific projects and do not comprise previously used processes and lessons learned from other projects. Therefore, such processes are not standard in all projects. Configuration changes are managed according to the project processes but not crossfunctionally coordinated among all stakeholders. CM roles and responsibilities are assigned at the beginning of each project and vary depending on the specific opinion, knowledge and experience of project managers in different projects. Information systems, whether developed in-house or chosen from Commercial Off-The-Shelf (COTS) products, are used for CM activities and requirements of each project. There may be similarities among the systems used in different projects but there is no standard choice or framework for choosing the IT system for the whole organization. Level 3: Standard At this level, due to the recognized need for CM discipline, a clear strategy as well as a CM policy is developed for all projects. The strategy is in line with the overall organization strategy and objectives are clearly defined and controlled during the whole lifecycle of the products. Standard CM processes are in place for most of the projects and are accessible by all stakeholders. These processes are developed and managed based on the knowledge of CM experts. However, such processes are not updated based on feedback from the field or lessons learned. Configuration changes are managed according to the project processes, and based on the standards, they are cross-functionally coordinated with all stakeholders. The importance of CM is understood by higher level management of the organization and they support the implementation of the discipline. A standard and specific CM terminology and knowledge source is in place which is developed based on the CM expert knowledge. CM roles and responsibilities are assigned in a standard way at the beginning of most/all projects based on the organization and specific project needs. Standard functionalities of CM are the rationale behind the choice of information systems. However, there is no regular benchmark for updating the IT system. The IT system is standard for most/all projects and therefore there is a central repository for all information. The importance of CM is understood by some of the personnel in charge of specific projects and only in those projects is the implementation of the discipline supported. However, the terminology introduced in the beginning of the project is based on a limited knowledge and different from project to project. 1

11 Level 4: Optimizing At this level, there is a specific Configuration Management strategy and policy in place for the whole organization. This strategy is aligned with the overall enterprise s strategy and is deployed at different levels of the organization i.e. strategic, tactical and operational levels. There are quantitative measures to evaluate the fulfillment of overall CM objectives on a regular basis. Such a strategy is updated in regular time intervals with improvement scenarios and new KPIs. Standard CM processes are defined for the whole organization and are accessible in a central process repository for all stakeholders. These processes have specific owners who are responsible for providing training and developing the processes further by collecting field feedback. There is a defined methodology for change management which involves clear guidelines for cross-functional change impact analysis. CM terminology is distinguished from other disciplines terminology and standardized throughout the whole organization. The terminology, along with a CM knowledge base, is available for all stakeholders and its usability is assured. Smart decisions about whether to have a centralized CM organization or to scatter the CM-related personnel among the projects can be made based on the complexity level and specific requirements of the organization. Roles and responsibilities are defined and communicated to the employees. There is an organized practical training for CM which is consistent all along the organization and the knowledge gained from different situations in different projects is used for updating the training. CM support originates from top management and is deployed at operational level. Management promotes internal and external benchmarks related to CM features and overall CM discipline in the organization is continuously improved. When choosing information technology, the CM functionalities and requirements are taken into consideration. Any IT system to be used is checked for correspondence and alignment to the main IT system of the organization first. The CM-related information is globally available throughout the whole organization and a clear authorization system is in place for creating, using or modifying this information. A Cross-Industry Configuration Management Maturity Assessment Introduction 11

12 Configuration Management Maturity Assessment Description, Framework and Methodology of the Study In order to rapidly and easily distribute the appraisal material to all target persons and at the same time collect as much detailed information as possible, the online survey method was selected [9]. The survey was created at IMI and was reviewed and improved by CM experts at P3. In order to enhance the quality and user-friendliness, the survey was made available in both English and German. The survey was sent to more than 15 CM-related professionals and also made available in specific professional online communities. During the one-month data collection period a total of 67 full responses were collected which, considering the time and detailed level of information and expertise needed to fill the survey, is a very satisfactory rate. Altogether there were 53 questions in the survey, which were assigned to seven categories corresponding to the five maturity model s primary dimensions described above, as well as General and Demographic questions. Prior to distribution, the survey was pre-tested by P3 with an average response time of 15 minutes. 12

13 RESULTS DESCRIPTION OF THE SAMPLE Distribution of the Participating Organizations (Private vs. Public) 28 % public sector 72 % private sector Distribution of the Participating Organizations Figure 3 Distribution of Industry Sectors In this chart, the distribution of the participating industrial sectors is illustrated. As shown, the participants involved are from a diverse set of industrial sectors. The Aerospace, Transportation, Military and Automotive sectors supply the most participants. IT military, defence, government communication services 3 % 4 % 11 % 12 % 11 % automotive transportation 12 % consulting electronics & mechanics 5 % 7 % 2 % 33 % aerospace healthcare Distribution of Industry Sectors Figure 4 A Cross-Industry Configuration Management Maturity Assessment Results Description of the sample 13

14 Size of Participants Organizations More than 5 percent of the participants originate from large enterprises where the activities are distributed mostly among various geographical locations and the importance of an intraorganizational standardization and integration is evident. Approximately 44% of the participants were part of SMEs (1 to 25 employees) and medium to large (25 to 1 employees) organizations where some level of operational flexibility exists and increased partnership and outsourcing requires interorganizational standardization and integration. less than 1 3 % 9 % 14 % 1 to 5 5 to % 21 % more than to 1. Size of Participants Organizations Figure 5 Level of Participants Involvement in CM Activities and their Typical Positions in the Organization The following figure shows the level of involvement in CM activities among the participants. More than 8% consider their job to be directly related to CM, and 2% are knowledgeable and active in this discipline but consider their job to be only indirectly asociated with CM. Although the option was available, no respondent considered their job to have no association to CM activities. As a result, all participants had a comprehensive understanding of CM, including its application characteristics in their organization. This ensures both reliable and meaningful results. 19,3 % indirectly involved 8,7 % directly involved Level of Participants Involvement in CM Activities Figure 6 14

15 RESULTS CM EFFECTS AND APPLICATION Estimations from Participants on the Level of CM Effectiveness on Lead Time, Cost and Quality The majority of participants believe that Configuration Management has a direct positive effect on product lead time, cost and quality. This interpretation from industrial practitioners complements the academic point of view [1, 2, 3] and reflects the high importance of Configuration Management as a stand-alone and independent support discipline. 78,9% 8 number of responses (%) ,5% 3,5%,% direct positive effect indirect positive effect no effect negative effect Opinion on Effect of CM on Product Leadtime and Cost Figure 7 CM Effects on Reliability, Availability, Maintainability and Safety (RAMS) Almost 8 percent of the respondents appreciate the effect of CM on Reliability, Availability, Maintainability and Safety of their product or facilities. 8 78,9% number of responses (%) ,1%,%,% considerable positive effect limited positive effect no effect negative effect Opinion on Impact of CM on RAMS Figure 8 A Cross-Industry Configuration Management Maturity Assessment Results CM effects and application 15

16 Level of CM Implementation throughout all Participating Organizations As mentioned previously, an implementation of CM as a holistic discipline can be found in almost any production environment. The figure below supports this view. However, it is interesting to learn about the quality and extent of CM application in organizations, provided with the maturity assessment in the following chapters. number of responses (%) ,9% 45,6% 3,5% CM is implemented in whole organization CM is implemented in some entities only CM is not implemented Level of CM Implementation throughout Organizations Figure 9 CM Assessment History Although Configuration Management has shown to exert a strong influence on product cost and lead time reduction as well as quality improvement, the level up to which organizations focus on evaluating its potential and improving its application is rather low. Only 28 percent of the participants have used assessment frameworks for this discipline of which the only official framework is CMMI. This leaves the majority of organizations less aware of their CM activities. 28,1 19,3 % assessed (formally & informally) 71,9 8,7 % never assessed History of CM Assessment Figure 1 16

17 Product-related Level of CM Application (Hardware, Software, both, none) Although CM history started with hardware, it is equally important today for software. A successful organization shall therefore apply this discipline to hardware, software as well as their interfaces in order to be able to trace the changes in each artifact to the right requirement. Although almost all participants have products which include both software and hardware, not all of them apply CM to both. A small amount of respondents even stated that they do not apply CM at all. number of responses (%) ,1% 17,6% 19,3% 7,% both software & hardware only hardware only software none Product-related Level of CM Application Figure 11 Percentage of CM Application in different Lifecycle Phases According to the answers, CM is applied throughout the complete product lifecycle. Almost 9% of the participants stated that there is application of CM in Design & Development phase, whereas there is a low level of CM application in the early and late phases of the Product Lifecycle. 1 89,5% number of responses (%) ,8% 7,2% 66,7% 28,1% planning & prestudy design & development manufacturing & assembly operation & maintenance decommissioning & disposal Percentage of Organizations applying CM in different Lifecyle Phases Figure 12 A Cross-Industry Configuration Management Maturity Assessment Results CM effects and application 17

18 RESULTS MATURITY ASSESSMENT Organizations Self-estimation of their CM Maturity In the following figure, the participants self-estimation of their organizations overall CM maturity is illustrated. It is based on the maturity levels described in chapter Configuration Management Maturity Model. When we compare the selfestimations with the final results from the detailed maturity assessment, considerable differences show. These differences justify the necessity for awareness about the status of CM activities at all times. What is believed to be the real status might not always be true since various quantitative measures and dimensions are not considered most of the time. initial 12,3 % 12,3% optimizing 21 % 54,4 % standard managed Organizations Self-estimation of their CM Maturity Figure 13 Dimension-based Maturity Overall Maturity under each Dimension The chart below illustrates the maturity of Configuration Management primary dimensions (as defined in chapter Configuration Management Maturity Model ) in all participating organizations. It can be observed that almost all dimensions maturity is close to level 2 or the Managed level. This shows the high general potential for improvement in this discipline especially in the level of management support and knowledge creation and reuse as well as utilization of the information technology for supporting the processes. strategy & performance knowledge & support process organization & value-stream information technology 1 = initial 2 = managed 3 = standard 4 = optimizing Overall Maturity under each Dimension Figure 14 18

19 Dimension 1: Strategy & Performance As described before, the strategy dimension of CM maturity model includes subjects such as creation of CM strategy and policy, deployment of this strategy in different organization levels and units as well as measuring the CM performance by utilizing KPIs. The following figure illustrates the maturity of the strategy dimension among organizations with a considerable amount of participants. The results suggest that the Military, Defense and Government as well as Aerospace industry have the highest average maturity of about 2.7 which is close to the Standard level. However, in other industries such as Transportation and IT, there is much room for improvement. maturity level 3, 2,5 2, 1,5 1, 2,8 2,72 1,71 2,75 1,92,5 automotive aerospace transportation military & government information technology Maturity Assessment Strategy Figure 15 Regarding the CM Strategy, the highest potential lies in the development and utilization of suitable KPIs for CM performance measurement as well as communication of the CM strategy to all stakeholders. This is shown in the following figure. 3, maturity level 2,5 2, 1,5 1, 2,58 2,37 2,18 1,88,5 CM strategy & policy strategy deployment communication of strategy KPIs Maturity Assessment Strategy Sub-dimensions Figure 16 A Cross-Industry Configuration Management Maturity Assessment Results Maturity Assessment 19

20 Dimension 2: Process The Process dimension of the CM maturity model includes subjects such as standard CM processes including Configuration Identification, Change Control, Status Accounting and Audit. Furthermore the management and maintenance of processes, accessibility of processes, process ownership and customizability of CM processes are part of this dimension. The following figure illustrates the maturity of the process dimension among organizations with a considerable amount of participants. The results suggest that the Military, Defense and Government as well as Aerospace industry have the highest average maturity of about 2.7 which is close to the Standard level. maturity level 3, 2,5 2, 1,5 1,,5 2,18 2,73 1,75 2,75 2,28 automotive aerospace transportation military & government information technology Maturity Assessment Process Figure 17 The highest potential for improvement among the various processes can be identified in the process interface between CM and Supply Chain, as well as the Configuration Audit process. Industry experience shows that the integration of suppliers is a big challenge to manufacturers and integrators of complex products. Furthermore, the measuring of a company s CM process quality is often neglected and therefore improvement potential remains hidden. Both experiences are supported by the figure below. maturity level 3, 2,5 2, 1,5 1, 2,4 2,39 2,25 2,19 2,61 2,4 2,4 2,18 2,9 2,26 2,44 2,33,5 consideration of subcontractors CI identification product structure management baselining engineering change management change impact analysis change implementation CM status accounting configuration auditing allocation of process owners stakeholder s access to CM information process customizability Maturity Assessment Process Sub-dimensions Figure 18 2

21 Dimension 3: Information Technology The IT dimension of the CM maturity model includes subjects such as the integration of CM IT systems with the main IT system, user-friendliness of the CM IT system, support of IT systems in change management and accessing and managing workflows, as well as the authorization capabilities of the CM IT system. The following figure illustrates the maturity of the Information Technology dimension among organizations with a considerable amount of participants. The results suggest that the Aerospace industry has the highest average maturity of about 2.7 which is close to the Standard level. The Transportation industry has a maturity of 1.13 which represents high potential for improvement for this industry in the IT dimension. maturity level 3, 2,5 2, 1,5 1,,5 2,21 2,74 1,13 1,98 1,94 automotive aerospace transportation military & government information technology Maturity Assessment Information Technology Figure 19 The results show that across the participating industries, a need for improvement can be assigned to the IT support for Engineering Change Management, whereas identification (e.g. naming), versioning and variant management seem to be supported by IT on a higher level. 3, maturity level 2,5 2, 1,5 2,12 2,19 2,12 1,93 2, 2,32 2,32 2,9 1,,5 CM IT integration in other systems userfriendliness of CM IT systems IT engineering change management support Tracking the change status using IT systems IT support for traceability of changes IT support for naming, versioning and variant management IT support for workflow management authorization capabilities of CM IT system Maturity Assessment Information Technology Sub-dimensions Figure 2 A Cross-Industry Configuration Management Maturity Assessment Results Maturity Assessment 21

22 Dimension 4: Organization & Value-stream The Organization & Value-stream dimension of the CM maturity model includes subjects such as CM organization structure, roles and responsibilities, cross-functional collaboration, sub-contracting activities as well as the involvement of customers and stakeholders in CM. The following figure illustrates the maturity of the Organization & Value-stream dimension among organizations with a considerable amount of participants. The results suggest that the Aerospace industry has the highest average maturity of about 2.5. On average, all organizations have more or less the same range of maturity in this dimension. Nevertheless there is still a high potential for improvement. maturity level 3, 2,5 2, 1,5 1,,5 2, 2,54 1,91 2,31 2,33 Maturity Assessment Organization & Value-stream Figure 21 The involvement of sub-contractors shows the highest improvement potential among the CM tasks. This clearly reflects the challenges which result from the approach of risk sharing across the product lifecycle with a global supply chain. Sharing risks means shared responsibility, e.g. in the development and design field. 3, 2,5 2,17 2,34 2,14 2,41 2, 1,5 1, 1,79,5 org. complexity & needs consideration roles & responsibility definition cross-functional collaboration involvement of sub-contractors in task distribution involvement of stakeholders in requirement changes automotive aerospace transportation military & government information technology maturity level Maturity Assessment Organization & Value-stream Sub-dimensions Figure 22 22

23 Dimension 5: Knowledge & Support The Knowledge & Support dimension of the CM maturity model includes subjects such as the utilization of standard CM terminology, CM trainings, accessibility of knowledge sources and best practices, CM promotion by top management, communication of CM benefits to stakeholders as well as external and internal benchmarking. The following figure illustrates the maturity of the Knowledge & Support dimension among organizations with a considerable amount of participants. maturity level 3, 2,5 2, 1,5 1, 2,8 2,32 1,79 2,21 2,14,5 Maturity Assessment Knowledge & Support Figure 23 CM benefits are hardly known across a whole company. In fact, industry experience shows that there is a strong concentration of knowledge in the engineering departments despite the fact that CM is a holistic discipline. Improvement potential can only be fully exploited if CM is implemented across the whole company. As a consequence, CM knowledge must be distributed widely and on a level which is understood by all stakeholders. 3, 2,5 2, 1,5 1, 2,12 2,19 2,12 1,93 2, 2,32,5 terminology & knowledge support availability of CM training availability of CM standards, resources & lessons learned top management support & CM promotion communication of CM benefits to stakeholders utilization of CM benchmarks automotive aerospace transportation military & government information technology maturity level Maturity Assessment Knowledge & Support Sub-dimensions Figure 24 A Cross-Industry Configuration Management Maturity Assessment Results Maturity Assessment 23

24 Overall Maturity In this section the CM maturity of industries with a considerable percentage of participants is individually illustrated. strategy & performance knowledge & support process organization & value-stream information technology automotive aerospace transportation military & government IT 1 = initial 2 = managed 3 = standard 4 = optimizing Overall Maturity All industries Figure 25 24

25 automotive aerospace transportation military & government information technology Overall Maturity Industy-specific = initial 2 = managed 3 = standard 4 = optimizing Figure 26 A Cross-Industry Configuration Management Maturity Assessment Results Maturity Assessment 25

26 Industry Comparison The overall maturity of industries with a considerable percentage of participants is presented in the following chart. The rather substantial difference in CM maturity among the various industries can be explained with the sensitive nature of products from a CM standpoint, such as those in the aerospace and military industries. However, as the Standard and Optimizing levels are not reached by any of the organizations yet, more focus on various CM aspects and application level is recommended for all industries, especially the ones with a low maturity level. maturity level 3, 2,5 2, 1,5 1, 2,11 2,61 1,66 2,4 2,12,5 automotive aerospace transportation military & government IT industry Overall Maturity of Industries Figure 27 Size-based Comparison The following charts illustrate a comparison of the overall maturity of CM dimensions in large organizations (more than 1 employees) with SMEs and medium to large organizations (between 5 to 1 employees). A considerable difference in all dimensions can be observed, specifically in the Information Technology and Knowledge & Support dimensions, where SMEs and medium-to-large organizations do not reach Level 2 maturity ( Managed ). 26

27 4 maturity level ,8 2,16 1,93 2,9 1,89 strategy & performance process information technology organization & value-stream knowledge & support SMEs and Medium to Large Organizations Figure 28 4 maturity level 3 2 2,41 2,43 2,33 2,34 2,19 1 strategy & performance process information technology organization & value-stream knowledge & support Large Organizations Figure 29 This observation can be confirmed from industry experience point of view, with the following possible explanations: SMEs with growing responsibility and product complexity, such as system suppliers to the Aerospace or Transportation industry, are facing substantial challenges to enhance their CM processes, information systems or knowledge transfer. Often, the capacity to do so is not available and a clear priority is given to the operational business. Regarding the information technology support for CM, large companies frequently implement extensive IT solutions with a broad range of CM functionality, such as PLM and ERP systems. In SMEs, the use of IT systems often has a pragmatic background with self-programmed or open-source solutions. Here, in many cases, CM functionality is iteratively added to the existing systems. Configuration Management roles in SME organizations are often allocated to resources which at the same time fulfill other roles, such as project managers, quality managers or design engineers. Large companies often have the capacity to define organizational units with dedicated resources to fulfill CM tasks. A Cross-Industry Configuration Management Maturity Assessment Results Maturity Assessment 27

28 CONCLUSION Summarizing the results of the study, several aspects can be identified as major findings for the cross-industry participants: 1. Positive Effects of CM Application Although the benefits of CM seem to be hard to measure and to communicate, a large majority of participants attribute positive effects on lead time, cost and quality to Configuration Management. CM is widely considered as essential to businesses throughout the Product Lifecycle, with lower rates of CM application in early and late lifecycle phases. A high majority of participants also give CM credit for having considerable positive effects on Reliability, Availability, Maintainability and Safety (RAMS). This is particularly important for safety-critical environments where a significant amount of energy and investment is dedicated to improving the aforementioned variables. 2. Self-estimation vs. CM Maturity Assessment Prior to the assessment, over two third of the participants estimated their level of CM maturity to be Standard (Level 3) or higher, based on their own knowledge and experience. Compared to this estimation, the assessed maturity levels are significantly lower: The overall average of the measured results shows a maturity level close to Managed (Level 2) and does not exceed a value of 2.75 in any of the industries. This is mainly related to a considerable potential for improvement in the dimensions of Knowledge & Support, Information Technology and Organization & Value-stream. In order to explain this significant deviation between selfestimation and maturity assessment, several factors can be discussed. One possible factor could be the transparency among organizational units and functions: Due to a lack of reliable indicators explaining the status of CM application in organizational units and functions applying CM, a full view and transparency is hardly given. Another factor for this deviation could be the prioritization of CM sub-disciplines: Certain sub-disciplines of CM are prioritized, such as Change Management which is implemented on a relatively high level. Thus, other CM sub-disciplines, such as Baseline Management or Configuration Auditing are given a lower priority and are therefore assessed with a lower maturity. 3. Potentials for Improvement Top Management Support & Performance Indicators The top management support and promotion of CM seems to be a crucial aspect. This is also related to the measuring and communication of CM benefits, which seems to challenge the CM supporters. The character of CM as a supportive function rather than a productive one makes it difficult to point out certain benefits, supported by concrete numbers and facts. Identifying and measuring appropriate CM key figures to convince top management and CM skeptics will be a key field for improvement for a more solid establishment of CM to industries and organizations. Supply Chain Involvement The involvement of the supply chain in CM processes and tools preoccupies most participants. Shifting responsibilities (e.g. design and development or maintenance) to a global supply chain lowers cost and risk to a certain extent. Nevertheless, internal complexity increases in terms of organization, processes and tools along the Product Lifecycle. Consequently, Configuration Management needs to include suppliers adequately in terms of CM roles, CM process interfaces as well as tool interfaces. Vice versa, suppliers must strive to create their CM environment to be as flexible as possible in order to adapt to customer processes and tools. Information Technology For an efficient application of CM throughout the different Product Lifecycle phases, it is essential to have adequate Information Technology support. The results show a particular potential for improvement regarding IT support of Engineering Change Management processes. As technical changes can occur at any point in time along the Product Lifecycle, industry experience shows that especially the transitions between the lifecycle phases (e.g. between development and manufacturing) need to be well-reflected in the IT environment. 28

29 SUMMARY AND FUTURE RESEARCH This study focuses on performing a cross-industry assessment to analyze the current application status of Configuration Management in various industrial organizations by utilizing the Configuration Management Maturity Model described above. The results, including the participants feedback, imply that the model is satisfactorily capable of covering various CM aspects. The appraisal material is designed to a sufficient level of detail for achieving descriptive assessment purposes, as can be observed in the figure below. 14 % further detail necessary 86 % adequate coverage The Participants Opinion on Level of Model s Coverage on CM Aspects Since maturity assessment frameworks are proven to be powerful tools for continuous improvement initiatives, the Institute for Information Management in Engineering (IMI) and P3 ingenieurgesellschaft aim at a further development of this initiative. Future collaboration will focus on research aspects in order to: Enhance the functionality of the model for prescriptive purposes, in order to generate suitable roadmaps for the assessed organizations to improve their maturity level Utilize a weighting system for a better evaluation of maturity with respect to the importance and priority of CM practices Figure 3 Develop qualitative appraisal material for a more detailed assessment of CM in each organization Create a database of benchmarks and best practices for each dimension and sub-dimension to be used for assessment and improvement purposes Develop innovative solutions to enhance CM application utilizing the assessment results, e.g. performance indicators, decision support methods and tools, best practices for supply chain integration A Cross-Industry Configuration Management Maturity Assessment Conclusion and summary 29

30 AUTHORS Prof. Dr. Dr.-Ing. Dr. h.c. Jivka Ovtcharova, is a full professor and leader of the Institute for Information Management in Engineering (IMI) at Karlsruhe Institute of Technology (KIT). Besides she is the leader of the Lifecycle Engineering Solution Center (LESC) and the director for Process and Data Management in Engineering (PDE) department at the Research Center for Information Technology in Karlsruhe (FZI). Her research activities are focused on the interface between Engineering and Computer Science in the area of continuous information management throughout the product lifecycle. M. Sc. PMP Masoud Niknam is a research associate at the Institute for Information Management in Engineering and PhD candidate at Karlsruhe Institute of Technology (KIT). His research interests are associated with Maturity Models, Configuration and Change Management as well as Lean Product Development. His background comprises education and industrial experience in international universities and organizations. He is currently working on an EU-FP7 Marie Curie project entitled PURESAFE and is focusing on developing a Configuration Management Maturity Model. Dipl.-Wi.-Ing. Stefan Bierbrauer is a senior consultant for P3 Ingenieurgesellschaft in the fields of Configuration Management, Product Lifecycle Management and Environmental Compliance. Besides his consulting activities in different industries such as aerospace, electronics and railway, he is focusing on the collaboration and knowledge transfer between industry and research. He is experienced as trainer, guest lecturer and speaker to international conferences. Dipl.-Wi.-Ing. Till Reymann is a senior consultant for P3 Ingenieurgesellschaft. His professional activities take place in Configuration Management, Product Structuring & Data Management, Modularization and Portfolio Management. Foremost, he gained his experience in the aviation industry, but especially due to the wide-ranged applicability of Configuration Management, the scope has extended to other fields like e.g. border security. Like his colleague, he is engaged in trainings and lectures. 3

31 REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [1] [11] [12] [13] [14] [15] [16] [17] [18] [19] Sage, A.P., Rouse W.B.: Handbook of Systems Engineering and Management, Jon Wiley & Sons, 211. International Atomic Energy Agency: Application of Configuration Management in Nuclear Power Plants, Safety Reports Series No. 65, 21. Hass, A.M.: Configuration Management Principles and Practice, Addison-Wesley Professional, 23. International Aerospace Quality Group IAQG: Configuration management guidelines, 21, Available at accessed Sep 211. Niknam, M., Bonnal, P., Ovtcharova, J.: Configuration Management Maturity in Scientific Facilities, to be published in International Journal of Advanced Robotic Systems special issue on Telerobotics and Systems Engineering in Scientific Facilities, 213. Röglinger, M., Pöppelbuss, J., Becker, J.: Maturity models in business process management, Business Process Management Journal, Vol. 18 Iss.2, pp , 212. Niknam, M., Ovtcharova, J.: Towards Higher Configuration Management Maturity, Proceedings of the International Product Lifecycle Management Conference (PLM13), July 6-1, Nantes, 213. Batenburg, R., Helms, R. W., Versendaal, J.: PLM roadmap: stepwise PLM implementation based on the concepts of maturity and alignment, International J. of Product Lifecycle Management, 1(4), , 26. ISO/IEC 1227: Systems and Software Engineering, Software and Systems Engineering Standards Committee, International Standard Organization, 28. BS ISO 17: Quality Management Systems Guidelines for Configuration Management, British Standards, 23. Tech America: ANSI/EIA-649-B Configuration Management Standard, 211. Department of Defence DoD: MIL-STD-346 Standard for Configuration Management, 213 Accessed online on Humphrey, W.S.: Characterizing the Software Process: A Maturity Framework, Software Engineering Institute, CMU/SEI-87-TR-11, ADA182895, CMMI Product Team: CMMI for Development, Version 1.3 (CMU/SEI-21-TR-33). Software Engineering Institute, Carnegie Mellon University, 21, Government Electronics and Information Technology Association GEIA: Systems Engineering Capability Model (EIA-731.1), Electronic Industries Alliance (EIA), 22. Crawford J.K.: Project Management Maturity Model, Taylor & Francis gr., 26. Kuvaja, P., Bicego, A.: Bootstrap A European Assessment Methodology, Software Quality Journal, Issue 3, p , Project Management Institute PMI: A Guide to Project Management Body of Knowledge, Project Management Institute, 21. Czaja, R., Blair, J.: Designing Surveys: A guide to divisions and procedures, Pine Forge P., Dec 9, 24. A Cross-Industry Configuration Management Maturity Assessment Authors and references 31

32 +++ Aachen +++ Bangalore +++ Beijing +++ Belgrad +++ Berlin +++ Braunschweig +++ Bremen +++ Bristol +++ Boulder +++ Cornebarrieu +++ Dallas +++ Durham +++ Düsseldorf +++ Hamburg +++ Hildesheim +++ Hounslow +++ Ingolstadt +++ Livermore +++ Madrid +++ Manresa +++ Morristown +++ München +++ Newport Beach +++ Peking +++ Quebec +++ Rom +++ Shanghai +++ Simsbury +++ Solms +++ Stuttgart +++ Toulouse +++ Troy +++ Wichita +++ Wolfsburg +++