ORGANIZATIONAL REVIEW. Operational Performance Analysis (OPA) TNA_OPA Rev.3_17Feb2017 Page 1 of 23

Transcription

1 ORGANIZATIONAL REVIEW Operational Performance Analysis (OPA) TNA_OPA Rev.3_17Feb2017 Page 1 of 23

2 Organizational Performance Analysis (OPA) 1. Introduction The purpose of the OPA is to provide training management with the information needed to make a decision as to whether training is required and, if so, what training strategy to use. It consists of two related activities: problem analysis and training requirements analysis. This exercise is designed to provide managers with answers to the following questions: - What exactly is the problem that training is expected to solve? - What causes it? - What are the training and non-training solutions? - What management actions are needed to make development of a training solution effective, including the provision of necessary resources? After completion of the OPA, managers will be in a position to make a balanced decision as to which training approach should be followed and what resources should be allocated to the development of a Training Plan if this is the chosen training solution. Problem analysis is based on the detection of a performance problem which has to be analysed in order to determine what causes it and what actions must be taken to solve it. Solutions may imply training and/or non-training actions. If training is a solution, the OPA helps managers determine what sort of training should be undertaken. Primarily, Preliminary Study enables managers to justify the development of a training plan to respond to the real job performance needs. In particular, it avoids simply preparing a training plan just because an operational or training manager thinks it is a good idea or has a vague notion that just by training staff the organization will somehow become ultra-performing. It eliminates nice-to-know courses that have no direct relevance to the employee s job and merely allow staff to escape from the workplace for a few days. In other terms, OPA prevents time, effort and money from being wasted developing ineffective or useless training. 2. Analysing a performance problem 2.1 The identification of a performance problem The identification, or detection, of a performance problem stems from some directly observable or measurable facts or events that indicate a difference between actual and acceptable performance. Such facts or events are called symptoms of a problem. The key point is determining the acceptable standard of performance. This standard will either be a product standard or a process standard against which the actual product or TNA_OPA Rev.3_17Feb2017 Page 2 of 23

3 process will be compared. A symptom is generated when there is an alert that the acceptable standard of performance is not being achieved. Therefore, the symptom is a consequence of a performance problem affecting the acceptable product/process standard. Perceptions of symptoms may vary according to whether they come from operations managers or from users of an organization s services. In cases where there are divergent opinions, it is necessary to analyse the detected performance problem objectively, first by verifying the facts and then by seeking the cause(s) if the performance problem is confirmed (failure to meet a defined standard) Estimating the importance of a performance problem The importance of a performance problem can be expressed in terms of the harmful effects resulting from the problem. Some factors to consider are: - the magnitude of the problem; - the duration of the problem; - how widespread and frequent the problem is; - factor affecting personnel, such as risks of personal injury; - factors affecting service, such as equipment out of service, delays in reporting. Estimating the importance of a problem should go hand-in-hand with estimating the potential value of eliminating it. For example, will eliminating the problem lead to increased performance/productivity, decreased costs, increased revenues, increased job satisfaction, less absenteeism, fewer workplace incidents/accidents or reduced delays? Such factors are not always quantifiable in terms of monetary gain although it is desirable to obtain such information whenever possible. The main purpose of estimating the importance of a problem and the potential value of eliminating it is to enable management to determine whether it is worthwhile allocating resources to proceed with the analysis of the problem or, if two or more performance problems have been detected, the priority in which they should be analysed Identifying the causes of performance problems A cause is an explanation of a symptom usually a deficiency that produces a problem. It is critically important to identify causes carefully since lack of precision can lead to erroneous decisions concerning what corrective actions should be taken in the form of solutions to the problem. Examples of major causes of performance problems are given on Figure 2.2. Causes are frequently interdependent. While some causes of performance problems are clearly linked to deficiencies in the training process (the upper, shaded half), others are linked to structural or procedural deficiencies at the level of the organization (lower half) although they may indirectly affect the training process. Some causes may be partially linked to training and partially to procedural aspects, such as recruitment procedures which will have an impact on training needs. TNA_OPA Rev.3_17Feb2017 Page 3 of 23

4 Analysis must be done to determine whether training could help overcome the problem and, if so, what kind of training. All non-training causes of problems should be reported to management. However, while it is not the job of the course development team to provide solutions to these problems, the team should make it clear that without appropriate responses to these other causes, providing a training solution will not be enough to attain acceptable standards of job performance in the future. Frequently, the best solution to a performance problem consists of a mix of training and non-training solutions. It should never be expected or assumed that training alone can resolve all deficiencies. 3 Anticipated problems Quite often, there may be no actual performance problem in a given sector of civil aviation activity but due to manpower forecasts (such as recruitment to replace a significant number of forthcoming retirees or to respond to expected increases in activity), emerging technology or planned organizational or procedural changes, it is anticipated that problems will appear if no appropriate action is taken in time. 4 Problem analysis using a systems approach Any organized operation can be considered to be a system, a set of interrelated elements working to achieve a common goal. This provides a convenient means for analysing the organization and defining performance problems accurately. A system has the following elements: - inputs (e.g. personnel, equipment, financial resources, external regulations); - process (e.g. procedures, structures and rules which ensure through a series of interrelated internal mechanisms that the inputs are transformed into the desired output); - output, also called product (which should meet the standard (goal) of the system); - feedback (which is a mechanism to measure the actual output against the desired output (standard) and return information to the system about any discrepancies so that corrective action can be taken, e.g. modifying inputs or elements of the process). Causes of performance problems may be external (directly linked to inappropriate inputs that come from other systems or subsystems) or internal (part of the system process itself). Identifying the system affected is paramount in clearly defining performance problems, particularly because this system is usually linked to other systems or subsystems that have to be considered in the analysis. TNA_OPA Rev.3_17Feb2017 Page 4 of 23

5 Examples of some deficiencies in a system (causes of performance problems): Inputs: - inadequately qualified personnel; - inadequate equipment or specifications; - insufficient financial resources. Process: - poorly designed working procedures and methods; - inappropriate organizational structure; - inadequate flow of information. Output: - quality of the product does not match specifications (standard); - insufficient quantity of products (less than specified). Feedback: - standard (goal) not adequately defined; - inaccurate measuring instruments or methods; - information about output deficiencies not fed back to the system (lack of communication); - inadequate or erroneous information about the output fed back to the system (distortion of communication); - information not available when required. As part of the OPA, it should be prepared a diagram of the system(s) affected by the identified performance problem, using the feedback models presented in this section. Furthermore, it is necessary to prepare an organizational chart showing where the job(s) associated with the performance problem is (are) situated in the structure as well as the job s vertical and horizontal linkages with other jobs on whose output it depends or for which it provides inputs. 5 Solutions to performance problems Only when the preceding process of analysing the performance problem has been completed objectively and a decision has been taken that something must be done to resolve the problem should solutions be examined. Whether they are training or nontraining solutions, they should always be based on established facts, never on the whims and fancies of management. Not all solutions will have the same cost or return on investment and these aspects need to be constantly borne in mind. The costeffectiveness of a solution is of prime importance for training management. TNA_OPA Rev.3_17Feb2017 Page 5 of 23

6 5.1 Training solutions Training solutions are varied and some alternative potential sources of training are enumerated hereafter: - develop new competency-based training - revise and upgrade/update existing training; - adapt courses obtained from training providers; - organize on-the-job training; - use alternative training based on already-prepared training material available from other sources: manufacturers consultants global training institutions Other factors which may also determine the best choice of a training solution imply a preliminary evaluation of where, when, how and for whom training will be implemented. At this stage, the target population primarily affected by a training solution should also have been identified. Training may be conducted: - in the Organization itself or by sending trainees to external institutions (national/foreign); - in one continuous programme or by alternating training with job experience; - through day-release of staff, evening classes or at home study via a distancelearning package; - through a combination of the above training approaches. Validated, competency-based in-house training provides an effective training solution because it is tailored to meet specific job requirements or address specific performance problems Non-training solutions Whenever a practicable non-training solution is identified, it should of course be referred to the relevant operational/technical sectors. However, the impact of such a solution on training (if any) should be assessed. For example, introducing new work procedures or automation processes may create new training. When the solution to a performance problem requires both training and non-training responses, as is frequently the case, it is important to establish an action plan. For example, it may not be effective to conduct training until after implementation of one or more of the nontraining solutions. 6. Cost-effectiveness of training solutions TNA_OPA Rev.3_17Feb2017 Page 6 of 23

7 Although the cost-effectiveness of a training solution is useful information for managers, notably for justifying external resources required for a training plan, it is often very difficult to measure financial benefits accurately. In many cases, such as training destined to improve safety, where it is fundamental to avoid loss of lives or accidents to personnel, the measurement of cost-effectiveness does not apply. Training has to be conducted, whatever the cost, to reduce risks to life. The notion of cost-effectiveness has validity when there may be two or more possible training solutions and management wishes to compare the alternatives. Taking into account the benefits and the costs of alternative solutions as well as possible constraints, the most suitable solution (or combination of solutions) may be recommended for consideration by management. The most suitable training solution is that which provides the required number of adequately trained personnel, at the lowest cost, in the most practical manner, and in the most acceptable time frame. 7. Evaluation Plan A plan for evaluating the effect of training on the organization s operational performance should also be prepared at the end of the OPA and included in the report for management. This plan entitled Evaluation Plan, should define what factors or aspects must be evaluated. Indicators which should be derived from the symptoms of the problem must be defined as well as the means for monitoring them and the person who will be responsible for this. The evaluation plan will give management a tool for measuring the impact of the training project on the organization. 8. Preparing a report on the OPA The report on the OPA marking the end of this step in the course development process is destined for management. It should include the following elements: i. a short introduction explaining the context of the report; ii. a concise definition of the performance problem; iii. a list of the main symptoms that indicated a performance problem; iv. a concise statement of the potential value of eliminating the problem; v. a explanation of the causes of the problem; vi. a concise description of the system(s) affected by the performance problem, a presentation of the system(s) in schematic form and preparation of an organizational chart; vii. a statement of the proposed training/non-training solution(s); viii. a statement of the expected benefits from implementing a training solution (if a training solution is proposed); ix. a concise statement of the aim (goal) of the training solution (if a training solution is proposed); TNA_OPA Rev.3_17Feb2017 Page 7 of 23

8 x. a statement of the resources; xi. a concise statement of any constraints on the proposed training solution; xii. an analysis of the cost-effectiveness of training solutions (if necessary); xiii. a Training Project Planning and Control Sheet; xiv. an evaluation Plan. 9. Quality Assurance Checklist for the OPA 1. Has the preliminary analysis been coordinated with operations? 2. Have symptoms of a performance problem been identified and described? 3. Has the performance problem been identified and specified? 4. Have causes of the performance problem been identified and described? 5. Has an analysis of the system and subsystems been carried out? 6. Has the system affected by the problem been clearly identified? 7. Has an organizational chart been included? 8. Have training and non-training solutions been identified? 9. If there is more than one training solution, has the selected solution been justified? 10. Has the target population affected by the training solution been identified? 11. Have any constraints on developing a training plan been identified? 12. Has a Training Planning and Control Sheet been prepared? 13. Has an Evaluation Plan that includes indicators derived from symptoms been prepared? TNA_OPA Rev.3_17Feb2017 Page 8 of 23

9 Appendices OPA basic tools FORM I: Interview FORM II: Basic Questionnaire FORM III: Survey TNA_OPA Rev.3_17Feb2017 Page 9 of 23

10 FORM I: INTERVIEW 1. Organization (Company) name? 2. What is the goal of the Organization? Vision? Mission? 3. What are the sub-systems (departments, sections, etc.) or elements (people, basic and special equipment, etc.), and how do they fit together? 4. For the major sub-systems, identify the following: - Goals; - Inputs; - Processes; - Products (services). 5. Please provide an organigram of the Organization/company, illustrating interconnections. 6. Give an analysis and assessment of the main problems affecting the company. 7. Develop a cause/effect chain to isolate problems created by professional incompetence. 8. Identify the content required for a staff training programme. TNA_OPA Rev.3_17Feb2017 Page 10 of 23

11 FORM II: BASIC QUESTIONNAIRE 1. Main problems facing the Organization a) What are the main problems and how are the various departments affected? b) What is the measurable impact of each problem on the product (service) of the company? c) Are there other symptoms of the problem such as: - Customer complaints; - Staff complaints; - Significant competency gaps between two employees performing the same task; - Core tasks of the job avoided or neglected? 2. What is the potential benefit of eliminating each of the main problems identified? - Improved performance and productivity; - Greater job satisfaction, less absenteeism; - Lower costs; - Other factors. 3. Main causes of the problems - What are the main causes of the problem(s), and what evidence supports your conclusions? - How are the causes inter-related, and what is their relative importance? Consider possible causes in relation to: a) Organization and structure; b) New technology; c) Personnel management; d) IT systems; e) Environment; f) Existing training. a) Organization and Structure - Is the structure of the organization flexible enough to adapt to changes? - Are all the necessary conditions in place for the Organization to function efficiently? - Are inputs adequate in terms of the quantity, quality, timeliness and means of delivery? - Are tasks properly assigned? - Are any sections (work units) overburdened or under-used? - Are the roles and objectives clearly defined throughout the structure? - What are the standards (objectives), if any, for quality of service, productivity, etc. and are they updated? b) New Technology TNA_OPA Rev.3_17Feb2017 Page 11 of 23

12 - Is it necessary to incorporate new international civil aviation standards or technological advances? - Has new equipment recently been installed, or scheduled for instalment in the near future? c) Personnel Management - Are there job descriptions, and do they reflect the jobs actually being done? - Are task assignments reasonable? - Do the persons involved agree to their roles? - Do the employees feel that the objectives are appropriate? - Are the objectives sufficiently flexible to adapt to changing needs? - Is there a pool of adequately qualified people from which to recruit? - Are there irregularities in recruitment procedures (delays, lack of information for candidates, etc.)? - Do the selection criteria reflect the requirements of the position, and if so, are they fulfilled? - Is the right person assigned to the task? - Are staff frequently promoted, or do they change jobs for other reasons? - Are individual employees rewarded for good performance? - Are the rewards appropriate and predictable? - Is unsatisfactory performance sanctioned? - Is there career planning? If so, how does it affect the training scheme? d) IT System - Is there an integrated IT system? - Are the IT systems fully effective? - Is there adequate and timely feedback? Top-down; Bottom-up; Among peers? - Is the information made available where and when it is needed? - Is the system open to new ideas, changing needs, etc.? e) Environment - Is team design or distribution causing any problems? - Are spare parts and replacement units available? - Are sufficient numbers of adequate tools available? - Are any difficulties experienced in using the available work aids? - Are there any other environmental factors causing problems on the job such as facilities, lighting, furniture, climate control, etc.? - Can you specify the real impact of employees personal circumstances on job performance (housing, food, commutes, etc.)? - Are there any other environmental factors that should be considered? f) Existing Training (for managers, directors, top officials, etc.) - Does the current training programme have a clear purpose, and if so is it being achieved? - Are the main training objectives stated? If you feel the programme objectives are inadequate, can you create new ones? TNA_OPA Rev.3_17Feb2017 Page 12 of 23

13 - Does the current programme meet the needs? - Is training content relevant to the job? - Is the training material suited to the level, pre-requisites and aptitude of trainees? - Are the teaching methods adapted to the technical content and learning methods of the trainees? - Are the facilities and training material satisfactory? - Are there enough qualified teachers, trainers and instructors? - Are there any issues with inexperienced teachers, trainers and instructors? - Can the existing training programme be evaluated? - Is there an evaluation system? - Is practical on-the-job training part of the programme, and if so is real-life on-thejob training combined with formal classroom training? - Are there people available to conduct on-the-job training? - Is there equipment available for practical training? TNA_OPA Rev.3_17Feb2017 Page 13 of 23

14 FORM III: SURVEY Survey General Information: Position held: Department: In view of your position in the Office, we kindly ask that you help us in our work by participating in this survey. Please read the following instructions very carefully before completing the survey: 1. Please take a good look at Page 1 which describes problems identified in the companies of our corporation as a result of our investigations, together with a list of some of the symptoms of those problems. 2. To the right of each symptom are Columns A and B. In, please indicate how often you think the problems occur in your company by marking the appropriate box with an X. 3. In, please assess the severity of the problem in your company, marking the appropriate box with an X. 4. The list of identified symptoms is followed by a blank space which you should use to describe any other symptoms you have encountered. 5. On the last page, each problem is listed with two columns: Causes and Solutions. In the Causes column, please say what you think the main cause of the problem is. In the Solutions column, please suggest ways you can think of to solve the problem. TNA_OPA Rev.3_17Feb2017 Page 14 of 23

15 Form III: SURVEY A) The management functions, powers and inter-departmental relationships are not defined. Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) Several people doing the same task 2) Delays in execution of tasks and business processes 3) Dissatisfaction among staff 4) Ambiguous responsibilities 5) Unnecessary use of labour force 6) Inadequate control of activities 7) Objectives and proposed actions are not clearly stated. 8) The desired level of quality is not achieved. 9) Tasks are avoided or neglected. 10) Little interaction among departments TNA_OPA Rev.3_17Feb2017 Page 15 of 23

16 B) There are no standards or other mechanisms to accurately measure the performance of individual workers and teams. Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) Promotions are awarded without a precise assessment of workers performance levels. 2) Incentives are not based on performance levels. C) No effective selection procedures have been created. Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) Staff are selected who lack the experience or aptitude for the position, and who then perform poorly. TNA_OPA Rev.3_17Feb2017 Page 16 of 23

17 D) There is no free flow of communication and/or no means to ensure communication. Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) Information is erratic. 2) Information is not timely. 3) Task execution is delayed. 4) There is not a clear understanding of the problems, proposed actions and the purpose of the tasks. 5) Unnecessary use of labour force 6) Individual workers or units (teams, departments) do not meet expectations. 7) Information is not obtained from the workers in a timely and systematic manner. Their opinions are not known, and they have limited involvement in organizational changes. TNA_OPA Rev.3_17Feb2017 Page 17 of 23

18 E) The staff training, education and development plan does not meet the present and future needs of the company. Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) Not all workers are given adequate training. 2) Resources are wasted (time, materials, etc.). 3) Educational content and materials are not adapted to needs. F) The incentive system is not effective (salaries that do not match performance, significant differences between flight and ground crew, etc.). Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) Workers are demotivated, apathetic and uninterested, and do not feel a part of the company. 2) Employees potential, skills and knowledge are underused. 3) Workers do not see how important their contribution is to the company. 4) Staff turnover TNA_OPA Rev.3_17Feb2017 Page 18 of 23

19 G) The control function is not an active and systematic process aimed at improving service. Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) The accounts do not give a realtime indication of the financial position. 2) Systems to monitor instruments and inputs such as spare parts and materials do not exist or are defective (potential for automation). 3) The desired results are not achieved, and resources are wasted (time, labour force, materials, etc.). 4) The service is not delivered to the desired standard of quality. TNA_OPA Rev.3_17Feb2017 Page 19 of 23

20 H) There is no assessment mechanism to measure the impact of training initiatives on staff performance. Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) Training programmes are not systematically improved (educational material, methods, etc.). 2) Resources are wasted (time, materials). 3) Workers do not see how important their contribution is to the company. 4) The assessment of course outcomes is unrealistic. TNA_OPA Rev.3_17Feb2017 Page 20 of 23

21 I) The structure of the organization is not flexible enough to respond to change. Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) There is no economic/operational independence, resulting in poor service delivery. 2) Slow uptake of automation and other changes needed to improve corporate performance. J) Management expertise is sub-optimal. Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) Frequent, poorly planned and unproductive meetings 2) Overwork 3) Inability to communicate with subordinates (poor knowledge of their skills, initiatives and personal traits) 4) Centralized decision making with no delegation of authority or responsibility 5) Time wasted on executive functions 6) Lack of creativity and initiative 7) Failure to involve staff and foster teamwork 8) Lack of knowledge about the rules governing external business TNA_OPA Rev.3_17Feb2017 Page 21 of 23

22 K) Limited financial or functional autonomy Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) Delays in the purchase of inputs essential for business 2) No dynamic, effective relationship between the central and dependent units 3) Small investments are not made in a timely fashion. 4) Staff are demotivated, irritated and unhappy. 5) Large scale investment procedures are hampered by State restrictions. L) Strategic planning and leadership do not meet requirements. Symptoms Always Almost always Sometimes Almost never Never Very Serious Not very 1) Time pressure in introducing new technology or other necessary changes 2) Time pressure in training staff 3) Loss of business and customers 4) Resources (staff, tools) are not used appropriately. 5) Existing capacities are not optimized. 6) Tasks are not prioritized, resulting in haphazard daily problem solving. TNA_OPA Rev.3_17Feb2017 Page 22 of 23

23 Causes and Possible Solutions Problem Causes Solutions Training Solutions A Non-Training Solutions B C D E F G H I J K L TNA_OPA Rev.3_17Feb2017 Page 23 of 23