Measuring Conceptual Knowledge Among NPP Maintenance Personnel A Tool For Knowledge Management

Size: px

Start display at page:

Download "Measuring Conceptual Knowledge Among NPP Maintenance Personnel A Tool For Knowledge Management"

Sophia Curtis
6 years ago
Views:

1 Measuring Conceptual Knowledge Among NPP Maintenance Personnel A Tool For Knowledge Management Pia Oedewald and Teemu Reiman Organization Research VTT, Technical Research Centre of Finland pia.oedewald@vtt.fi, teemu.reiman@vtt.fi Abstract The paper presents key findings of a longitudinal study on development of knowledge among new maintenance workers at a Nordic nuclear power plant. A measure of conceptual knowledge was developed in cooperation with the technical experts and trainers at the case plant. The power plant recruited ten maintenance workers during a two years period. The conceptual knowledge of the ten new workers was measured three times during their first two years of tenure. Also reference group of experienced maintenance workers completed the conceptual knowledge questionnaire. In the paper the construction of the measure is presented and the results of the measurements are discussed in light of knowledge management at the nuclear industry. The results show that the conceptual knowledge of the newcomers increased after training, but difference in the scores between the individuals was very high both before and after the training. I. INTRODUCTION Long work experience and excellent technical know-how of the maintenance personnel are often seen as success factors for safety and reliability of nuclear power. An ongoing change of workforce generation calls for tools to analyze the existing know-how of the personnel in order to create effective training programs [4, 5]. Maintenance work in nuclear power plants is not routine-like activity, which could be carried out just by following the procedures. It requires different types of skills and knowledge. Practical craftsman skills, overall understanding of the functioning of and couplings between the systems as well as technical knowledge about the materials and equipments are needed in maintenance work [13, 14, 15, 16]. Thus, in addition to the tacit knowledge about the power plant in question, understanding of the theoretical basis of technical phenomena and work processes are essential contents of the know-how of the personnel. Conceptual knowledge means the understanding of the theoretical basis of the technical phenomena, tools and procedures that one is working with. Theoretical, in this context, should not be understood as scientific or irrelevant for practice. Instead, it means that one comprehends the principles underlying the power production and process equipment. [7]. Good conceptual knowledge is a prerequisite for adequate decision making in situations where the predefined procedures do not apply. It is also crucial for effective communication between technicians, foremen and control room operators. Furthermore, it plays an important role when specific tasks need to be verbally explained to people not familiar with the machinery and the working practices at the power plant. In more general terms conceptual knowledge is the basis for (and a mediator of) on-the-job learning. Accurate understanding of the local workplace jargon and knowledge about the principles of the nuclear power process enables the workers to integrate and evaluate the vast amount of information they face every day. Studies in manufacturing [8] and pulp and paper industry [9] have shown that the workers conceptual knowledge is related to the quality of the work and job satisfaction. The level of conceptual knowledge can vary even among the experienced employees [9]. Different work groups at one facility can have varying conceptual knowledge profiles. Thus long work experience does not always mean high levels of conceptual knowledge. This is important to bear in mind when designing training programs where the newcomers learn from the experienced workers. It is often assumed that experienced personnel in well performing nuclear power plants possess high levels of conceptual knowledge in addition to their practical hands-on experience of the equipment. To our knowledge, this assumption has not been verified in maintenance organizations. Furthermore, it is not known how conceptual knowledge develops in practice or by training. This paper reports a case study where a measure of conceptual knowledge was developed in order to analyze the development of expertise among maintenance employees. II. THE CASE STUDY A longitudinal study was carried out at a Nordic nuclear power plant during The company operates two BWR-type reactors which have been in commercial use approximately for 25 years. The plant has excellent operating records. Copyright 2007 IEEE, Except as Authored Under US Government Contract 368

2 The maintenance newcomers expectations towards work in a nuclear power plant, their experiences of the training and coaching and the development of their conceptual knowledge were analyzed. The purpose was to evaluate the success of the training program developed by the power plant. Furthermore, the study aimed for identifying generic training needs in the maintenance department. The power plant recruited altogether 10 maintenance workers (eight technicians and two foremen) during Six of the recruits had previous experience as contractor or trainee. Still most of them participated in a structured training program arranged by the power plant. Those who had worked as trainees just before the beginning of the employment had previously participated in introductory courses and were expected to know the basics of e.g. radiation protection and maintenance procedures as well as the layout and functional principles of the plant. For the purpose of this study the newcomers were interviewed two or three times during First interviews were carried out when they had been employed approximately four weeks by the power plant. Second interviews were conducted a year later and the last ones when they had been working around two years. A conceptual knowledge questionnaire was completed each time. The study was able to follow six of the 10 newcomers for two years. Seven newcomers participated to the second test and six to the third one. No one of the newcomers resigned from the organization during the study, but some were otherwise unavailable during data gathering. In addition, 43 experienced maintenance workers completed the conceptual knowledge questionnaire. 32 respondents reported working experience of two years or more. They were thus used as a control group. III. THE MEASURE OF CONCEPTUAL KNOWLEDGE The measure of conceptual knowledge was developed in cooperation with technical experts, maintenance engineers, and trainers from the power plant. The purpose was to design questions that are relevant for all the maintenance workers as well as some specific questions in each technical area that would differentiate expert from novice. Specific questions for each technical field were created. The questionnaire contains altogether 75 multiple choice questions in nine knowledge areas: - mechanical engineering (10 questions) - electrical engineering (9) - I&C (10) - radiochemistry (3) - technical specifications (5) - process control (10) - design basis of the plant (10) - maintenance planning (9) - maintenance procedures. (9) Four to three response options were given in each question, one of which was always I don t know. Correct answer was scored one point, wrong answer minus one, and I don t know zero points. The maximum score was thus 75 points, and a minimum minus 75. The respondents were not allowed to use literature when completing the questionnaire. They were instructed that the questionnaire contains specific questions from different technical areas and they are not expected to know everything. The questions where of the following type: Decrease of the temperature of the feedwater causes a) an increase in the reactor power level b) a decrease in the reactor power level c) nothing Diversity principle aims at preventing common cause failures Diversity means that a) each safety function must be ensured by a similar function b) the plant can be operated when 2 out of 4 similar subsystems are operational c) the functioning of the equipment belonging to redundant subsystems must be based on physically different principles What is corrosion? a) oxidation of metal b) a reaction when metal is exposed to acid c) embrittlement by radiation IV. RESULTS The overall scores of the first test (n=10) ranged from -5 to 34. Mean score of the entire group was 20,7. Process control and electrical engineering related questions received plenty of wrong answers. Maintenance procedures and radiochemistry, themes that were emphasized in the introductory course, were known relatively well. Surprisingly, however, only one of ten knew the right answer to a question concerning the definition on redundancy and 2/10 to the question concerning the diversity principle. These questions were considered as being basic safety related knowledge that every worker should possess. The scores improved during the first year on the average (fig 1.). Only one of the seven respondents scored lower in the second test than in the first. However, especially the radiochemistry related knowledge was forgotten and the questions regarding redundancy and diversity were still difficult for most respondents. 369

3 6 Kokonaispistemäärä Value yht 1 1 testauskerta Fig. 1. Development of conceptual knowledge during the first year among maintenance newcomers (n=7). Y-axis shows the overall scores of the test 1 and 2 (x-axis). The scores of the last test (n = 6) ranged from 15 to 51 points the mean score being 34 (fig. 2.). All the respondents scored higher than at the first test. Two of the 75 questions were missed by all the respondents. Those related to process control and information security issues. 6 Value yht Case Number Fig. 2. Total scores of the respondents in the newcomers group. The results of the experienced workers group (tenure 2-30 years, mean 21,3 years) ranged from -5 to 55 points, the mean score being 31,6 (fig. 3.). The overall score did not correlate with the tenure (r = -.28, p = 0.128) Case Number Fig 3. Total scores of the respondents in the experienced worker s group. Basic concepts were not self-evident for the experienced group either. Only 60 % of the experienced maintenance workers knew the correct meaning of the diversity or the unit sievert. As the Table 1 suggests experienced workers tended to be optimistic about their knowledge concerning the process control area. TABLE I. MEAN SCORES OF THE DIFFERENT KNOWLEDGE AREAS OF THE EXPERIENCED WORKERS GROUP AND THE NEWCOMERS THIRD TEST GROUP Scores of the different knowledge areas a Newcomers (n=6) Mechanical engineering (max 10) 5,3 3,9 Electrical engineering (max 9) 2,8 2,3 I&C (max 10) 3,7 3,3 Radiochemistry (max3) 2,3 2,0 Technical specifications (max 5) 4,7 4,9 Process control (max 10) 2,0 1,3 Design basis of the plant (max 10) 3,5 3,3 Maintenance planning (max 9) 4,5 4,8 Maintenance procedures (max9) 5,2 5,7 Total score (max 75) Experienced workers (n=32) a. correct minus wrong answers V. DISCUSSION The measure of conceptual knowledge was sensitive enough to differentiate between individual respondents in the maintenance organization. In addition to that, it provided information on the easy to digest as well as the difficult areas of maintenance expertise, and identified concepts with contradictory interpretations among the personnel. Correcting 370

4 these misunderstandings may save the organization from several human errors. On the basis of this pilot study one cannot judge whether the level of conceptual knowledge in the case organization was sufficient or not. When the questions were designed it was assumed that the experienced maintenance experts would manage most of them. Clearly this was not the case. Phrasing of the questions may have been unclear in some cases. In spite of this, it seems fair to say that the conceptual knowledge was rather low among some respondents. Further, the maintenance managers and training people saw difficulties with certain questions surprising. The study did not analyze the relations between the conceptual knowledge and actual work performance. However, based on previous studies it can be hypothesized that high levels of conceptual knowledge at least on a team level contributes to effective communication, better quality of decision making, higher work motivation and well-being at work. The power plant was willing to invest on the evaluation of the conceptual knowledge because they saw those as important factors for nuclear power plant reliability [cf. 11, 12]. The study provided a profile of the conceptual knowledge among maintenance personnel at one well performing nuclear power plant. It also revealed the necessity of evaluating the training needs among the experienced workers. Refresher courses on e.g. safety and radiation protection principles and basic technical areas might be beneficial at the case organization. In addition to training, there are many other approaches to support the development of conceptual knowledge. Work process modeling, new division of tasks and roles and appropriate instructions support the development of overall picture over the plant [7,12]. An experienced maintenance team might manage daily tasks without perfect understanding about the analyzed fault mechanisms and details of the working methods because the gradually formed tacit knowledge works well enough in most situations. In spite of that, several event investigations have illustrated how the frailties of the knowledge surface when unusual disturbances [2, 3] or new parties challenge the routines. Analyzing the conceptual knowledge of the personnel is a proactive measure to tackle this problem. The newcomers overall conceptual knowledge improved significantly during the first two years. In fact, the mean score of the newcomers third test was higher than the mean score of the experienced employees group. Due to the small sample size no significant conclusions can be drawn from that. It is possible that repeating the test created a learning effect. The respondents may have red up on the themes between the tests. On the other hand, those who attended to the structured training program saw the training program and on-the-job coaching quite effective. Those who received less structured training and coaching were less satisfied with their apprentice time. Still some of them got excellent results in the conceptual knowledge test. The longer the newcomers had worked the more they expressed the need for more profound knowledge about the power plant. This is understandable from the viewpoint of problem-based learning. Personal experiences on practical problems are important for learning motivation to develop [1]. As the results from the experienced workers test suggest it is not the tenure per se that differentiates the experts from less knowledgeable workers. Personal orientation towards the work is also important for the development of expertise [10]. Those newcomers who scored extraordinarily well at the third test had slightly different orientation towards maintenance work from the beginning. They seemed to be more aware of the uncertainties in complex systems and they emphasized the importance of an overall picture of the plant. Recognizing uncertainties in the technology and risks in the activities motivates learning [6, 10]. This is important to take into account when designing training programs. Training should not focus too much on routines and practical solutions but encourage studying issues outside the immediate work content and questioning current practices. REFERENCES [1] C. Chan, J. Burtis and C. Scardamalia, Knowledge building as a mediator of conflict in conceptual change. Cognition and Instruction 15, 1997 pp [2] S. P. Feldman, The culture of objectivity: Quantification, uncertainty, and the eveluation of risk at NASA. Human Relations 57, 2004, pp [3] A. Hopkins, Lessons from Longford. The Esso Gas Plant Explosion. Sydney: CCH, [4] J. Kettunen, T. Reiman, and B. Wahlström, Safety management challenges and tensions in the European nuclear power industry. Scandinavian Journal of Management, in press. [5] T. Kuronen and N. Rintala, The prerequisites for successful knowledge sharing in nuclear power plants. CSNI Workshop on Better Nuclear Plant Maintenance: Improving Human and Organizational Performance, 3-5 October, Ottawa, Canada, [6] U-M. Klemola and L. Norros, Analysis of the clinical behaviour of anaesthetists: recognition of uncertainty as basis for practice. Medical education 31, 1997, pp [7] A. Leppänen, Improving the mastery of work and the development of the work process in paper production. Ergonomics, training and workplace change, vol 56, [8] A. Leppänen, personal communication. [9] A. Leppänen, E. Tuominen and A-M. Vieno, The level of production technology, conceptual mastery of work, perceived work characteristics and job satisfaction in paper production. In J. Rantanen, et al. (Eds.) Work in the Information Society. People and Work. Research Reports 8. Helsinki: Finnish Institute of Occupational Health, pp , [10] L.Norros, Acting under unceratinty. The core- task analysis in ecological study of work. VTT Publications 546. Espoo:VTT, [11] P. Oedewald and T. Reiman, Using subjective measures to monitor the systems capability to manage complexity evidence from the nuclear industry and health care, In E. Hollnagel and E. Rigaud, (Eds.), Proceedings of the Second Resilience Engineering Symposium, 8-10 November, Antibes, Juan-Les-Pins, France. [12] P. Oedewald and T. Reiman, Enhancing maintenance personnel s job motivation and organizational effectiveness. CSNI workshop on Better 371

5 nuclear power plant maintenance: improving human and organizational performance. Ottawa, Canada, 3-5 October [13] P. Oedewald and T. Reiman, Core task modelling in cultural assessment: A case study in nuclear power plant maintenance. Cognition, Technology & Work 5, 2003, pp [14] C. Perin, Shouldering risks. The culture of control in the nuclear power industry. New Jersey: Princeton University press, [15] J. Reason, Managing the risks of organizational accidents. Aldershot:Ashgate, [16] T. Reiman, Assessing organizational culture in complex sociotechnical systems Methodological evidence from studies in nuclear power plant maintenance organizations. VTT Publications 627. Espoo: VTT,

Systemic view on safety as a safety culture factor

Systemic view on safety as a safety culture factor Interaction between Individuals, Technology and Organization A Systemic Approach to Safety in Practice IAEA Vienna, 10-13 June 2014 Pia Oedewald, VTT