Review of Quality Assurance in SKB s Repository Research Experiments

Transcription

1 SKI Report 2007:11 Research Review of Quality Assurance in SKB s Repository Research Experiments T.W. Hicks January 2007 ISSN ISRN SKI-R-07/11-SE

2 SKI Perspective Background SKB is preparing a license application for the construction of a final repository for spent nuclear fuel in Sweden. This application will be supported by the safety assessment SR-Site for the post-closure phase. As a preparation for SR-Site SKB has recently produced the SR- Can safety assessment, which is currently in review. The assessment of long-term safety is based on a broad range of experimental results from laboratory scale, intermediate scale and up to full scale experiments. It is essential that there is a satisfactory level of assurance that experiments have been carried of with sufficient quality, so that results can be considered to be reliable within the context of their use in safety assessment. SKI has initiated a series of reviews of SKB s methods of quality assurance and their implementation. This project in particular addresses SKB s quality assurance of experiments related to the buffer and backfill. These include characterisation of material properties in small scale experiments (Clay Technology AB in Lund Sweden), intermediate scale experiments addressing various aspects of buffer evolution as well as experiments with full-scale canister and buffer components mainly for confirmation and demonstration (Äspö Hard Rock Laboratory). Purpose of the Project The purpose of this project is to assess SKB s quality assurance with the view of providing input for the preparation of the SR-Site safety assessment. This has been achieved by examination of a number of SKB experiments using a check list, visits to the relevant facilities, and meetings with contractors and a few members of the SKB staff. As a background for understanding various approaches to handle quality issues, the quality programmes carried out as part of the Drigg (UK), WIPP (USA) and Yucca Mountain (USA) projects are briefly discussed. Results The results show that the efforts involving quality assurance are increasing within the SKB programme and in general appear to be satisfactory for ongoing experiments and measurements. Nevertheless, the level of detail in descriptions of QA requirements for experimental work is probably lower than for the other programmes included in this study. In addition, the link between experimental work and its use in safety assessment as well as in the decision-making process is in some cases not entirely clear. It needs to be decided how data from experiments carried out prior to formal QA should be handled in SR-Site. Future work Quality aspects will be further analysed as part of the review of SKB s SR-Can safety assessment. Additional scrutiny of this subject will be needed also for the subsequent stages of SKB s programme Project Information SKI project manager: Bo Strömberg Project Identification Number: and

3

4 SKI Report 2007:11 Research Review of Quality Assurance in SKB s Repository Research Experiments Galson Sciences Ltd. 5 Grosvenor House Melton Road Oakham Rutland LE15 6AX United Kingdom January 2007 This report concerns a study which has been conducted for the Swedish Nuclear Power Inspectorate (SKI). The conclusions and viewpoints presented in the report are those of the author/authors and do not necessarily coincide with those of the SKI.

5

6 Executive Summary SKB is preparing licence applications for a spent nuclear fuel encapsulation plant and repository which will be supported by the SR-Site safety report. A separate safety report, SR-Can, has been produced by SKB in preparation for the SR-Site report. SKI is in the process of reviewing the SR-Can safety report. In preparation for this review, and with a view to building confidence in SKB s research activities and understanding SKB s handling of data and other information, SKI has examined SKB s application of QA measures in the management and conduct of repository research and development projects that support the SR-Can safety assessment. These preliminary investigations will serve to support the preparation of more detailed quality and technical audits of SKB s repository safety assessment after the submission of a licence application. SKI s approach to this QA review is based on the consideration of quality-affecting aspects of a selection of SKB s research and development activities. As part of this review, SKI identified the need to examine quality-related aspects of some of the many experiments and investigations that form part of SKB s repository research programme. This report presents the findings of such a review, focusing on experiments concerned with the properties and performance of the engineered barrier system. First, in order to establish a broad understanding of QA requirements for repository scientific investigations, QA procedures implemented in the management of research and development activities for the low-level radioactive waste repository near Drigg in the UK and the Waste Isolation Pilot Plant and Yucca Mountain repository projects in the US were studied. The QA procedures for experiments and tests undertaken in these projects were compared with those implemented by SKB. Key findings are: QA programmes have been implemented for each repository development programme in response to regulatory requirements. The need for regular audits of the application, suitability and effectiveness of QA systems has been stressed in regulations and top-level QA requirements documents. In some cases, evidence of such audits has been presented in support of facility safety cases. The project QA programmes include requirements for scientific investigations that address the planning and performance of investigations as well as data management. Top-level QA documents for the US repository projects include detailed descriptions of requirements relating to the conduct of scientific investigations. Such detailed QA requirement descriptions have not been identified for SKB s experiments on engineered barrier system components. i

7 The review of SKB s experiments covered the long term test of buffer material (LOT), the large scale gas injection test (LASGIT), the temperature buffer test (TBT), and the Prototype Repository Experiment (PRE), which are being conducted at SKB s Hard Rock Laboratory (HRL) in Äspö, and tests on bentonite swelling pressure, hydraulic conductivity, and resaturation that are being conducted on behalf of SKB at Clay Technology s laboratories in Lund. To facilitate the reviews, a checklist of quality-affecting issues was devised which proved an effective tool for structuring and recording findings. The checklist covered the framework, design, conduct, analysis and reporting of experiments, and the use of experimental results. Key review findings are: The requirements and planning of experiments do not appear to have been coordinated with the planning and requirements of the repository licence application. Some experiments do support relevant function indicator criteria in the SR-Can report and some data may be available for use in the SR-Site safety assessment. However, most of the ongoing experiments have had little impact on the SR-Can safety assessment because the results are not yet available. Furthermore, it is unclear exactly how and when the results of longterm experiments might influence the repository development programme and licence application. It is not clear whether or how data acquired before SKB s current QA system was introduced have been formally qualified, or whether the QA system includes procedures for such data qualification. SKB s QA programme requires that contractors working on research projects produce their own, or use SKB s, quality plan. Clay Technology appears only recently to have implemented a formal QA system for work in its laboratories. Some of the experiments at the HRL involve collaborations of several radioactive waste management organisations. It is unclear how SKB s QA requirements are implemented through all components of the work on these experiments that might be relevant to the SKB repository programme. SKB has undertaken pilot studies, over-specified controls and instrumentation, and installed alarmed monitoring systems in order to mitigate risks of experiment failure. These measures are proving invaluable in ensuring the success of the experiments because numerous equipment failures have occurred. Work on experiments at the HRL is recorded on logs and field notes. There has been no systematic use of scientific notebooks for the experiments undertaken at the Clay Technology laboratories, although project information and data are stored in spreadsheets. ii

8 SKB maintains lists of documents produced for experiments at the HRL, including information on the review and approval status of each document. Reports of experiments undertaken at the Clay Technology laboratories are published by SKB, but this project was unable to identify any systematic formal document review process. Little information has been obtained with regard to the usability of results from ongoing experiments. Generally, there appear to be no firm plans on how to abstract data from such experiments for use in repository safety assessments. iii

9

10 Contents 1 Introduction Background Approach Report Structure Approaches to QA in Repository Research Experiments QA for the Drigg LLW Repository Post-closure Safety Case QA in the WIPP Repository Research Programme QA in the YMP Repository Research Programme QA in SKB s Repository Research Programme The Project Process Quality Plans Data Acquisition and Data Management Discussion Review of QA in a Selection of SKB s Experiments QA Checklist Review Findings Framework of Experiment Design of Experiment Conduct of Experiment Analysis and Reporting of Experiment Usability of Results Conclusions References...23 Appendix A QA Reviews of SKB's Experiments...25 iv

11

12 1 Introduction 1.1 Background Review of Quality Assurance in SKB s Repository Research Experiments SKB is preparing licence applications for a spent nuclear fuel encapsulation plant and repository which will be supported by the SR-Site safety report. A separate safety report, SR-Can (SKB, 2006), has recently been produced by SKB in preparation for the SR-Site report. SR-Can is based on a safety assessment for the underground disposal of spent fuel in Sweden according to the KBS-3 repository concept. The assessment involves quantitative analyses aimed at developing an understanding of how the repository system will evolve and the potential risks of spent fuel disposal. Therefore, any demonstration of regulatory compliance must be underpinned by assurances that the development and application of models and estimates of parameter values and uncertainties are of appropriate quality. To this end, the safety assessment must be developed within the framework of a quality assurance (QA) programme. SKI is in the process of reviewing the SR-Can safety report. In preparation for this review, and with a view to building confidence in SKB s research activities and understanding SKB s handling of data and other information, SKI has examined SKB s application of QA measures in the management of repository research and development projects that support the SR-Can safety assessment. These preliminary investigations will serve to support the preparation of more detailed quality and technical audits of SKB s repository safety assessment after the submission of a licence application. SKI s approach to this QA study is based on the review of quality-affecting aspects of a selection of SKB s research and development activities. The first stage of this study focused on the numerous computer codes used by SKB in the evaluation of the longterm safety of a repository either as components in the overall risk assessment or in the development of supporting safety arguments, such as in evaluations of engineered barrier system performance. On behalf of SKI, Hicks (2005) provided a review of the quality of the documentation and testing of a selection of these codes. Subsequently, SKI identified the need to review quality-related aspects of some of the many field and experimental investigations instigated by SKB to obtain the data that are used to abstract conceptual understandings of repository evolution and to evaluate the parameters represented in the computer codes. This report provides the results of such a review undertaken by Galson Sciences Ltd on behalf of SKI. 1

13 1.2 Approach In order to establish a comprehensive understanding of QA requirements for repository scientific investigations, the relevant QA procedures adopted in a number of repository programmes were studied. This stage of the project focused on the QA programmes implemented in the management of safety assessments and the supporting research and development activities for the low-level radioactive waste (LLW) repository near Drigg in the UK, and the Waste Isolation Pilot Plant (WIPP) and Yucca Mountain Project (YMP) in the US. The QA procedures undertaken in these projects were compared with those implemented by SKB for the SR-Can safety assessment and for experiments and tests. The main part of the project aimed to review QA issues for a selection of the experiments that might provide data for use in safety assessments and support an understanding of repository behaviour. The project focused on experiments concerned with gaining an understanding of the properties of repository near-field materials and the evolution of the near field after repository closure because of the significance of these factors to repository performance. Thus, after a preliminary consideration of the range of experiments and tests being undertaken as part of the KBS-3 repository development programme, the following experiments were selected for detailed review: the long term test of buffer material (LOT), the large scale gas injection test (LASGIT), the temperature buffer test (TBT), and the Prototype Repository Experiment (PRE) that are being conducted at SKB s Hard Rock Laboratory (HRL) in Äspö, and tests on bentonite swelling pressure, hydraulic conductivity, and resaturation that are being conducted on behalf of SKB at Clay Technology s laboratories in Lund. The QA review included meetings with SKB staff and contractors at the Clay Technology Laboratories in Lund on 15 th November 2005 and at SKB s HRL at Äspö on 16 th November A further meeting took place at the HRL a year later (17 th November 2006). In order to facilitate the discussions at these meetings and the documentation of review findings, a checklist of quality-affecting issues was prepared covering the framework, design, conduct, analysis and reporting of experiments, and the use of experimental results in the KBS-3 repository research programme. The findings of the review were documented on forms based on the above-mentioned checklist. These completed forms are included and discussed in this report. 1.3 Report Structure Section 2 of the report presents a discussion and comparison of QA programmes from a selection of repository research and development programmes. The review of quality-related aspects of experiments undertaken as part of SKB s repository research programme is presented in Section 3. Conclusions are presented in Section 4. Appendix A comprises the completed QA review forms for each experiment. 2

14 2 Approaches to QA in Repository Research Experiments In order to gain a comprehensive understanding of QA requirements for repository scientific investigations, QA procedures adopted in several national repository programmes have been studied. This part of the project focused on the QA programmes implemented in the management of safety assessments and, in particular, research and development activities for the LLW repository near Drigg in the UK, and the WIPP repository and YMP in the US. Detailed and prescriptive approaches to project quality management have been adopted in these projects. The QA procedures for experiments, tests, and data management undertaken in these projects have been compared with the approach to quality management taken by SKB and its contractors for similar types of activity. 2.1 QA for the Drigg LLW Repository Post-closure Safety Case The British Nuclear Group (BNG) - a BNFL business group formed in currently manages and operates the low-level radioactive waste disposal site near Drigg in the UK. In prior to formation of the BNG - BNFL produced a Post-Closure Safety Case (PCSC) for the Drigg site as a regulatory requirement to facilitate a review of the authorisation for waste disposal at the site. The Environment Agency requires that a comprehensive and systematic QA programme is established to cover all activities affecting the safety case, including supporting activities such as research and assessment (Requirement R11 of Environment Agency et al., 1997). This section considers the QA system adopted by BNFL in its preparation of the Drigg PCSC. BNFL s technical services are carried out as Research and Technology (R&T) projects; the scientific and technical work to develop the 2002 Drigg PCSC was undertaken as an R&T project termed the Drigg Technical Programme (DTP). All R&T projects are subject to QA arrangements according to the R&T Integrated Management System (RIMS), which was developed within the framework of international standards (including ISO ), BNFL company policies, and BNFL facility management and capability group systems (BNFL, 2002). RIMS has numerous procedures and instructions for carrying out work on R&T projects, including an instruction to produce a project plan (R&T_I_012, preparation and control of project/task plans) and an instruction to produce a project-specific QA programme (RIMS instruction R&T_I_014). The DTP project plan specified and periodically updated information on numerous activities for each of eight DTP task areas (inventory studies, site characterisation, coastal erosion, safety case preparation, near-field studies, far-field geochemistry, assessment codes, and post-closure radiological safety assessment). Details of activity schedules, document schedules, resources, and links between activities were included in the DTP project plan. The DTP QA programme was included in the DTP project manual (BNFL, 2004) and defined the organisational structure, responsibilities, data and document controls, and QA procedures and instructions for the DTP. The DTP organisational structure 3

15 included a project manager, a quality manager, and a technical auditor, as well as DTP task managers. The quality manager s responsibilities included the co-ordination of audits as a means of checking for the effectiveness of the quality management system. The technical auditor was an external consultant whose responsibilities included ensuring that the work was of an appropriate technical standard. The BNFL QA report (BNFL, 2002) includes reports of several self-audits. Contractors were commissioned to perform some of the technical work on the DTP. These contractors were generally selected based on their specialist knowledge and experience. Contractors working on the DTP were required to have an acceptable quality management system or to agree to conform with RIMS and the requirements of the DTP project manual. BNFL audited its contractors to check for compliance with the QA system. The DTP QA procedures and instructions were applicable to all the task areas and activities specified in the project plan. The DTP project manual lists 28 RIMS procedures and instructions that were of particular relevance to the DTP, including many pertaining to the operation of the Drigg facility and BNFL s Waste Management & Decommissioning (WM&D) Capability Group. In addition, the DTP project manual lists 11 working instructions that were developed specifically for the DTP. Of particular interest to this project are the procedures and instructions that relate directly to the conduct of experiments and field studies, which are listed in Table 2.1, and those relating to data acquisition and data management, which are listed in Table 2.2. However, these procedures and instructions have not been obtained for review under this project. Table 2.1 Procedure R&T_P_404 Procedures and instructions relating to experiments and field studies in support of the DTP. Title R&T_WM&D_P_006 R&T_WM&D_P_007 R&T_WM&D_P_009 R&T_WM&D_P_011 Calibration Pre-work documentation Field investigation Experimental design Experimental studies Instruction R&T_I_021 R&T_WM&D_I_034 R&T_WM&D_I_035 R&T_WM&D_I_038 Title Use of log books Technical specification Preliminary design of field and experimental studies Calibration of analytical equipment 4

16 Table 2.2 Procedure R&T_P_304 R&T_P_311 Procedures and instructions relating to data acquisition and data management in the DTP. Title R&T_WM&D_P_004 Document and data control Records management Database management and design Instruction DTP/WI/006 Title DTP central directory 2.2 QA in the WIPP Repository Research Programme The US Department of Energy s (DOE) Carlsbad Field Office (CBFO) is responsible for operating the WIPP as a repository for the safe disposal of transuranic (TRU) waste. The US Environmental Protection Agency (EPA) requires that the US DOE establishes and implements a QA programme for activities that are important to the containment of TRU waste in the WIPP disposal system. This QA programme must implement the applicable requirements of specific Nuclear Quality Assurance (NQA) standards issued by the American Society of Mechanical Engineers (ASME). The CBFO established these QA requirements in the Quality Assurance Program Document (QAPD), which is the QA plan for the WIPP project (US DOE, 2005). All organisations associated with TRU waste disposal at the WIPP must implement QA programmes that establish and implement the applicable requirements of the QAPD. The areas covered by the QAPD are listed in Table 2.3. Requirements in each of the areas listed in Table 2.3 are discussed in detail in the QAPD. Of particular interest to this project are the scientific investigation requirements. Broadly, these cover the areas listed in Table 2.4. The implementation of the QAPD requirements by participants in the WIPP programme is described in the WIPP Compliance Recertification Application (US DOE, 2004a). The CBFO conducts audits to verify the adequacy, implementation, and effectiveness of the QA programmes adopted by the programme participants. The QA programme adopted by Sandia National Laboratories (SNL) is of particular relevance to this QA review project; SNL is responsible for acquiring data from experimental programmes to support WIPP compliance applications and the SNL QA programme has been verified to be compliant with the requirements of the QAPD (US DOE, 2004a). The SNL QA programme is based on a system of nuclear waste management QA procedures (NPs). The NPs that address experiments and data collection are listed in Table 2.5. For instance, procedure NP 20-1 sets out requirements for preparing and implementing test plans for laboratory and field investigations and procedure NP 20-2 provides instructions on the use of scientific notebooks where considered necessary to record information for such investigations. 5

17 Table 2.3 Requirement Management Performance Assessment Sample control Scientific investigation Software Requirements of the QAPD for the WIPP. Description Organisation, implementation, and management of the QA programme. QA in work processes, design processes, service procurement, and in inspection and testing of processes and equipment. Management and independent assessment of the effectiveness of the QA programme. Control of samples of waste and environmental media. Planning and performance of investigations, and documentation, control, and validation of data. QA of software that is important to compliance application and waste characterisation. Table 2.4 Requirement Planning Performance Documentation, control, and validation of data Scientific investigation requirements in the QAPD for the WIPP. Areas addressed Identification and control of variables. Intended use of data. Compatibility of data with models used. Review and approval of technical procedures for conducting investigations. Documentation of new methods or procedures. Coordination with other organisations that provide input to or use the results of the investigation. Acceptance criteria for data quality evaluation (precision, accuracy, representativeness, comparability, and completeness). Identification of known sources of error or uncertainty. Use of test plans and procedures. Use of scientific notebooks to record the objectives, details of methods used, the work performed, results, and uncertainties. Periodic independent review to confirm results and check for traceability. Verification and control of methods and equipment. Control of data collection to an extent that allows the process to be repeated. Characterisation and control of test media. Identification, traceability, recording, and storage of data using controlled methods and review of data before use. Data validation by independent review of technical adequacy, adequacy of the QA record, and suitability for intended use. Qualification of existing data by consideration of the adequacy of the QA programme under which the data were collected, use of corroborating data, confirmatory testing, and/or peer review. 6

18 Table 2.5 Requirement NP 9-2 NP 12-1 NP 13-1 NP 19-1 NP 20-1 NP 20-2 SNL procedures that are applicable to experiments and data collection for the WIPP project. Description Parameters Control of Measuring and Test Equipment Control of Samples and Standards Software Requirements Test Plans Scientific Notebooks 2.3 QA in the YMP Repository Research Programme The US DOE is investigating a site at Yucca Mountain, Nevada, as a potential location for a geological repository for commercial and defence spent nuclear fuel and high-level radioactive waste. The US Nuclear Regulatory Commission (NRC) requires that work on the YMP relating to radiological safety or waste isolation must be performed in accordance with a QA programme that complies with relevant regulatory requirements. The Quality Assurance Requirements and Description (QARD) (US DOE, 2004b) is the principal QA document for the YMP. It establishes the minimum requirements for the YMP QA programme and has been designed to meet regulatory requirements. The requirements set out in the QARD are summarised in Table 2.6. The QARD includes supplements that contain requirements for specialised activities. These supplements are summarised in Table 2.7. Activities required to collect data (such as for siting or design input) are performed in accordance with the scientific investigation supplement, which is summarised in Table 2.8. Organisations performing work on the YMP are subject to the QARD requirements and must develop implementing documents that translate applicable QARD requirements into work processes. Therefore, the US DOE and its contractors have developed individual procedures that must be followed to implement a project QA programme that addresses the requirements of the QARD (US DOE, 1998). For example, SNL, the newly designated lead agency to coordinate science and technical work for the YMP, has developed quality assurance implementation procedures (QAIPs), Technical Procedures (TPs), and project-level implementing documents such as Administrative Procedures (APs) for its work on the YMP. TPs are generally prepared for scientific investigations involving operations or activities that are repetitive. These QAIPs are listed in Table

19 Table 2.6 Requirement Organisation Summary of requirements in the QARD for the YMP project. Description Creating and maintaining an organisational structure to implement the YMP QA programme. Quality assurance programme Design control Procurement document control Implementing documents Document control Control of purchased items and services Identification and control of items Control of special processes Inspection Test control Control of measuring and test equipment Handling, storage, and shipping Inspection, test, and operating status Non-conformances Corrective action Quality assurance records Audits Planning, implementing, and maintaining the QA programme. Definition, control, and verification of designs. Ensuring that service procurement documents contain appropriate technical and QA requirements. Ensuring that work is prescribed by, and performed in accordance with, written implementing documents. Ensuring that documents are reviewed for adequacy, approved for release, and distributed to and used at the location where the work is being performed. Planning and executing procurements to ensure that purchased items and services meet specified requirements. Ensuring that only correct and accepted items are used or installed. Control of special processes (such as welding, weld overlay, heat treating, chemical cleaning, and non-destructive examinations). Planning and executing inspections. Planning and executing tests that are used to verify conformance of an item to specified requirements, or to demonstrate satisfactory performance for service. Ensuring measuring and test equipment is properly controlled, calibrated, and maintained. Handling, storage, cleaning, packaging, shipping, and preservation of items to prevent damage or loss and to minimize deterioration. Identifying the inspection, test, and operating status of items. Control of items that do not conform to requirements in order to prevent inadvertent installation or use of the item. Ensuring conditions adverse to quality are promptly identified and corrected as soon as practical. Ensuring that QA records are specified, prepared, and maintained. Performing internal and external QA audits to verify compliance with, and to determine the effectiveness of, the QA programme. 8

20 Table 2.7 Supplement Software Sample control Scientific investigation Field surveying Control of the electronic management of data Summary of supplements in the QARD for the YMP project. Description Requirements for the acquisition, development, modification, control, and use of software. Requirements for the control of physical samples. Requirements for scientific investigations, including data identification, data reduction, and model development and use. Requirements for field surveying that might be undertaken during, for example, site characterization, explorations, and installations. The processes and controls for the management of data that either exist or are used in an electronic format. Table 2.8 Supplement Planning Performance Data identification Data review, adequacy, and usage Technical report review Model development and use Summary of the scientific investigation supplement in the QARD. Description General QA planning requirements. Coordination with organisations that provide input to or use the results of the investigation. Provisions for determining the accuracy, precision, and representativeness of results, Use of scientific notebooks to include the objectives, and description of work performed or references to documents that contain such information, methods and computer programs to be used, samples and measuring and test equipment, results, and information on individuals performing the work. Independent review to confirm results and check for traceability. Data should be clearly identified and traceable. Data should be independently reproducible. Data directly relied upon to address safety and waste isolation issues should be qualified, involving independent review for technical correctness. Unqualified data should be qualified by one or more of the following: considering adequacy of the controls under which the data were collected, use of corroborating data, confirmatory testing, peer review, and independent technical assessment. Requirements for document review should be followed. Requirements for planning, control, and documentation of model development and approaches to validation. Computer software should be qualified. Requirements for models to be validated to levels of confidence appropriate to their importance in repository performance assessment. 9

21 Table 2.9 Procedure QAIP 1-2 QAIP 5-1 QAIP QAIP QAIPs implemented by SNL for work on the YMP project. Title Organization and Quality Assurance Program. Preparing and Approving Quality Assurance Implementing Procedures. Technical Procedures. Sample Control. 2.4 QA in SKB s Repository Research Programme SKI (2002) and SKI (2004) present regulations and recommendations pertaining to the safe disposal of spent nuclear fuel and radioactive waste in Sweden. SKI (2004) includes the requirement that activities carried out at nuclear facilities are managed, controlled, evaluated and developed with the support of a management system that is designed to ensure that safety requirements are met. Furthermore, the application, suitability and effectiveness of the management system should be systematically and periodically audited. These requirements are applicable to spent fuel and radioactive waste disposal facilities prior to their closure. SKI (2002) requires that measures implemented to comply with quality assurance requirements for pre-closure safety are also adequate for post-closure safety. Furthermore, international experience and best practice calls for SKB s repository research to be developed under a suitable and audited management system that covers all aspects of QA. SKB has implemented a QA programme that includes a series of procedures for establishing and managing the research and development projects that are undertaken to support the safe management and disposal of spent nuclear fuel and radioactive wastes. No document specifically aimed at presenting SKB s overall approach to and standards for QA has been identified under this QA review project, but SKB s quality system is outlined in, for example, SKB s geoscientific programme for investigation and evaluation of repository sites (SKB, 2000a). The preparation of safety assessments, such as SR-Can, and the experiments undertaken as part of the KBS-3 repository research programme, such as those conducted at the Äspö HRL (see Section 3), are subject to this project management process. The following discussions of SKB s project process and data management system are based on presentations made by SKB staff during the meeting at Äspö in November 2005 (see Section 3) The Project Process The framework for initiating, implementing, and completing projects is set out in SKB s procedure for the project process (Procedure SD-002). Each project is initiated on the basis of a project decision, which may require a project feasibility study. Once the requirement for a project has been established, the project is organised and implemented according to a project model, which has planning, reporting, and evaluation phases as indicated in the flowchart in Figure

22 Project Decision Initiation Implementation Completion Project Plan (sub-project plans, activity plans) Project Documents (results, final report) Evaluation Report Figure 2.1 SKB s project model implemented as part of the project process. Each phase of the project model is closed by a project document; separate procedures have been established for preparing these documents. Details of all documents produced for each project, including information on responsibility for document preparation and the review and approval process, are recorded on a project document list. In the first phase of the project model, a project manager establishes the project organisation and prepares a project plan. The project plan defines project objectives, sets out a strategy for meeting these objectives, and addresses quality and environmental control issues. The requirements for quality and environmental management are described separately in Procedure SD-025. A quality plan is required to define responsibilities for quality planning and quality assessment, and to address document control issues, reporting of preventative and corrective actions, and risk analysis. The quality plan may form an integral part of the project plan or may be a separate document referred to by the project plan. Subcontractors working on SKB s projects must produce their own, or use SKB s, quality plan. Some projects may require more detailed planning and control. Activity plans are prepared for such projects. Each activity plan defines the methods and responsibilities for completing the activity and may include documents such as drawings, manuals, and technical specifications. All projects undertaken at the Äspö HRL require activity plans. Risk management, including the identification and analysis of technical, economic, environmental, and organisational risks, must be addressed for all stages of the project according to Procedure SD-020. Actions must be taken for reducing or eliminating risks. In the final phase of the project model, the project manager is required to prepare an evaluation report, which reflects on the implementation and results of the project, evaluates risk management in the project, and considers experiences that may lead to improvements in other projects. 11

23 2.4.2 Quality Plans As noted above, quality plans are required as part of the project process. For example, the SR-Can safety assessment is subject to the project process and SKB has prepared a draft QA plan which represents one of several project steering documents for the SR-Can project (Hedin, 2005). The broad purpose of the draft QA plan for SR-Can is stated as being to aid in assuring that all relevant factors for long-term safety have been included and appropriately handled in the safety assessment. Therefore, the QA plan focuses on quality assurance in the methods for identifying the features, events, and processes (FEPs) that are of potential importance to the safety assessment and in the methods for accounting for those FEPs in the safety assessment. The QA plan addresses the management and documentation of the FEP database, decisions concerning the treatment of the initial state of the repository, and decisions concerning the assessment of repository evolution (scenario analysis). The draft QA plan was reviewed as part of the present study and SKI provided SKB with review findings during a meeting in May Of particular relevance to the focus of the present study on QA in repository research experiments, it was suggested that the QA plan should provide more information on: the QA of data from past and ongoing research studies (such as field investigations, experimental studies, and modelling studies); the arrangements made for managing contractors and ensuring QA requirements are met by contractors; and provisions for internal and independent audit. QA plans are also required for the experimental projects undertaken as part of SKB s repository research programme. Such QA plans have not been reviewed as part of the present study. Instead, an independent checklist of quality-affecting issues has been prepared to facilitate discussions of QA in SKB s experiments as described in Section Data Acquisition and Data Management Specific SKB procedures for data acquisition and data management have not been identified under this project. However, the data management system implemented by SKB for experiments conducted at the Äspö HRL was described by SKB at the project meeting at Äspö in November Measured data from ongoing experiments are recorded continuously and may be viewed internally or remotely via a secure internet connection. Experiments may also be controlled remotely by SKB s contractors. Field work is recorded on daily activity logs and field notes and data are submitted for storage in SKB s SICADA data management system on a daily basis. The SICADA system was introduced by SKB in 1995 to replace and combine the features of existing databases. SICADA contains data acquired since 1974, which includes data gathered since construction of the Äspö HRL began. Field notes and data (paper or electronic media) are delivered to a SICADA operator, who registers each submission with a unique identifier as well as descriptive data and information relating to the relevant activity plan. Template Microsoft Excel spreadsheets are used 12

24 to import data into SICADA. The SICADA operator also archives the field notes and data. Traceability is maintained between entries in the SICADA database, associated file and data archives, and activity reports for each activity plan. SKB and its contractors may access the database using the SICADA/Diary application and data may be delivered to external organisations on request. 2.5 Discussion The QA systems for the repository programmes considered in this study have been developed in the context of regulatory frameworks and requirements as follows: Comprehensive and detailed requirements for ensuring that work carried out in the US WIPP and YMP repository programmes is of an acceptable quality have been set out in top-level programme QA documents (the QAPD for the WIPP project and the QARD for the YMP) that have been developed in response to respective EPA and NRC requirements. All organisations working on these repository programmes have been required to establish their own QA programmes that implement the top-level requirements. In the UK, the environment agencies require that any application to dispose of radioactive waste should be supported by a safety case that has been developed under a comprehensive and systematic QA programme. In response, BNFL produced a QA programme for all work performed in support of the Drigg LLW disposal site PCSC. SKI s requirements for QA are expressed in terms of a requirement for nuclear activities to be undertaken with the support of a management system. Although this requirement is applicable to spent fuel and radioactive waste disposal facilities, it is limited to pre-closure operations. However, SKI also requires that adequate quality assurance measures are taken with regard to post-closure safety. SKB has in place a process and procedures for establishing and managing projects such as the SR-Can safety assessment and the research and development projects that support such safety assessments. In all of these repository programmes, the need for regular audits of the application, suitability and effectiveness of QA systems has been stressed in regulations and toplevel QA requirements. BNFL presented evidence of several such audits in support of the Drigg PCSC. Checks of the compliance of QA programmes implemented by organisations working on the US WIPP and YMP repository programmes with project QA requirements have been reported in safety assessment documentation. At the meeting at Äspö in November 2006, SKB staff reported that recent checks had found that Clay Technology had adopted and was applying an acceptable QA system in its work for SKB. SKB is undertaking similar audits of its other contractors QA systems. Of particular interest to this project are the QA requirements for scientific investigations. Both the QAPD and QARD include similar explicit detailed requirements relating to the conduct of field and laboratory investigations. These 13

25 requirements address the planning and performance of investigations as well as data management. SNL, a key participant in both the WIPP project and the YMP, has produced procedures for work on experiments and data collection that facilitate the implementation of these requirements. Similarly, BNFL applied written procedures and instructions for field experiments, experimental investigations, and data management under its QA arrangements for the Drigg PCSC. SKB has implemented a project process that addresses experiments and investigations. The project process includes requirements for project planning (including quality plans for activities) and implementation phases. Detailed descriptions of requirements of the type presented in the QARD and QAPD have not been identified for SKB s scientific investigations. The QAPD and QARD include the requirement that data acquired prior to the implementation of the QA programme are qualified for use in repository safety assessment work. Both documents listed possible data qualification methods. This project has not identified any such requirements or methods for qualifying old data (perhaps generated under previous QA systems) for use in SKB s current repository research and development programme. 14

26 3 Review of QA in a Selection of SKB s Experiments SKI is undertaking preliminary reviews of the QA process in SKB s repository research and development projects in preparation for detailed quality and technical audits of SKB s repository safety assessment after SKB s submission of a licence application for a spent nuclear fuel encapsulation plant and repository. As part of this QA review, SKI has identified the need to review quality-related aspects of some of the many field and experimental investigations instigated by SKB to obtain the data that are used to abstract conceptual understandings of repository evolution and to evaluate the parameters represented in the computer codes. The main part of this project aimed to review QA issues for a selection of the experiments that might provide data both for use in repository safety assessments and to support an understanding of repository behaviour. The project focused on experiments concerned with gaining an understanding of the properties of repository near-field materials and the evolution of the near field after repository closure because of the significance of these factors to repository performance. After a preliminary consideration of the range of experiments and tests being undertaken as part of the KBS-3 repository development programme, a number of experiments carried out at the Clay Technology laboratories in Lund and at SKB s HRL in Äspö (SKB, 2005a) were selected for review. The QA review was centred on meetings in Lund (15 th November 2005) to discuss tests on: - bentonite swelling pressure; - hydraulic conductivity; and - resaturation; in Äspö (16 th November 2005) to discuss the following tests on bentonite buffer performance: - LOT (long term test of buffer material); - LASGIT (large scale gas injection test); and - TBT (temperature buffer test); and finally in Äspö (17 th November 2006) to discuss the following experiment: - PRE (Prototype Repository Experiment). The first meeting at Äspö commenced with presentations of SKB s approach to QA in scientific investigations, including document and data management. These presentations form the basis of the discussion of SKB s QA programme for experiments provided in Section 2.4. The three meetings otherwise followed broadly similar formats. Review of each experiment involved a presentation of the background to, and status of, the experiment by an appropriate member of Clay Technology or SKB staff, followed by a discussion of specific quality-affecting issues. A checklist of questions covering key areas of QA was prepared prior to the meetings to facilitate discussions. This checklist is presented in Section 3.1 and the findings of the QA review of experiments are presented in Section 3.2. The first two meetings concluded with visits to the Clay Technology laboratories and the SKB HRL, respectively, to view the experiments. 15

27 3.1 QA Checklist In order to facilitate the discussions of the QA processes applied to the experiments reviewed in this project, a checklist was prepared comprising a series of questions covering five areas of quality-affecting issues in scientific investigations. The checklist was devised based on consideration of the QA requirements for scientific investigations associated with the repository development programmes discussed in Section 2, as well as a broad understanding of issues relating to traceability and quality control of data. The checklist is as follows: 1. Framework of Experiment 1.1 Purpose and objectives What is being investigated? What experiment is being undertaken? Why is the experiment being undertaken? What is the role of the experiment in the repository programme? 1.2 Resources and schedule Where is the experiment being conducted? Who is conducting the What is the schedule for the When will results be available? What constraints do resources such as cost and timing place on experimental planning and design? 1.3 Quality assurance What QA system and standards are used in the planning, design, execution, analysis, and reporting of the How is the expert team selected/trained for the 2. Design of Experiment 2.1 Variables What are the dependent variables (i.e. those being observed)? What are the independent variables (i.e. those that are varied to cause change in the dependent variables) and how are their values selected? What are the control variables (i.e. those that are held constant) and how are their values selected? 2.2 Experimental techniques What experimental techniques and instruments are being used? Are they standard techniques? Are acceleration methods used? Have the techniques been validated and documented? Are the techniques being used under normal conditions? Has equipment been calibrated and checked? 2.3 Uncertainty What are the key uncertainties in the 2.4 Risks to success of experiment What are the risks to the success of the experiment and how are they mitigated? What are the critical decisions in the Is there duplication in the 3. Conduct of Experiment 16

28 3.1 Data collection and quality control How are data collected? How are data stored (e.g. filing, indexing)? How are data checked (e.g. independently)? How are data backed-up? What quality control procedures are used? 3.2 Records of experiment Are notebooks being used for the experiments? Are notebooks checked independently? Are planning, execution and analysis correspondences kept (e.g. s)? Are copies of records kept? 3.3 Equipment Is equipment tested, inspected, and maintained? 4. Analysis and Reporting of Experiment 4.1 Data interpretation What data interpretation methods are being used (models, software packages, model simplifications)? How are uncertainties and sensitivities analysed? 4.2 Reporting and review How are data and observations reported? How are interpretations reported? How are limitations on the use of data and results reported? How are reports reviewed (e.g. independently)? How are review results managed/responded to? 5. Usability of Results 5.1 Verification How are experimental outcomes checked against requirements of the How are experimental results verified? 5.2 Use of results How are results abstracted for use in the repository programme? Are results extrapolated for use on repository length and time scales? What checks are made that data and results are used appropriately and within prescribed limitations? 3.2 Review Findings The checklist was used as the basis of a form for documenting the results of the QA review. Forms containing reviews of the experiments on bentonite swelling pressure, hydraulic conductivity, and bentonite resaturation undertaken at the Clay Technology laboratories in Lund, and the LOT, LASGIT, TBT, and PRE experiments undertaken at the Äspö HRL, are provided in Appendix A. The following sub-sections summarise key review findings in terms of issues presented on the review checklist. 17