Canadian Less Lethal Technology Approval Process

Transcription

1 Canadian Less Lethal Technology Approval Process A Structured Approach to the Selection and Implementation of Less Lethal Technologies for Canadian Law Enforcement Donna Wood DRDC CSS Len Goodman DRDC Toronto Defence R&D Canada CSS Technical Report DRDC CSS TR (E) December 2012

2 Canadian Less Lethal Technology Approval Process A Structured Approach to the Selection and Implementation of Less Lethal Technologies for Canadian Law Enforcement Donna Wood DRDC CSS Len Goodman DRDC Toronto Defence R&D Canada CSS Technical Report DRDC CSS TR (E) December 2012

3 Principal Author Original signed by Donna Wood Donna Wood Project Manager CEWSI Approved by Original signed by Steve Palmer Steve Palmer Director Approved for release by Original signed by Mark Williamson Mark Williamson Chair DRDC CSS DRP This work was done for the Conducted Energy Weapons Strategic Initiative (CEWSI), project number 32bj, a project managed by Defence Research and Development Canada Centre for Security Science (DRDC CSS) in cooperation with Public Safety Canada. Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2012 Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense nationale, 2012 DRDC CSS TR (E)

4 Abstract.. One of the objectives of the Conducted Energy Weapons Strategic Initiative project is to develop a less lethal technology (LLT) approval process suitable for Canadian law enforcement. A focus group, with representatives from federal and provincial policy-makers, police officers, a technical advisor, medical advisor, legal advisor, public affairs advisor and a police training advisor, was established to guide the development. This report presents the Inputs, Outputs, Stakeholders, Sub-activities and Methods for each of the major activities in the proposed approval process: Identify Operational Requirements, Conduct Options Analysis, Conduct Subject Matter Expert Evaluations, Conduct End-User Evaluation, Implement, and Validate In-Service Performance. The activities in this approval process will support the decision-makers by providing the information needed to make evidence-based decisions before the selection and implementation of a new LLT capability. Résumé... L un des objectifs de l Initiative stratégique sur les armes à impulsions consiste à élaborer un processus d approbation des technologies de neutralisation momentanée (TNM) adapté aux organismes canadiens d application de la loi. Un groupe de concertation composé de représentants des décideurs fédéraux et provinciaux, d agents de police, d un conseiller technique, d un conseiller médical, d un conseiller juridique, d un conseiller en matière d affaires publiques et d un conseiller en matière de formation des agents de police a été créé pour orienter l élaboration du processus. Le présent rapport comprend les observations, les résultats, les intervenants, les sous-activités et les méthodes liés à chacune des activités importantes faisant partie du processus d approbation proposé : déterminer les besoins opérationnels, procéder à l analyse des options, effectuer des évaluations d experts en la matière, effectuer une évaluation de l utilisateur final, mettre en œuvre et valider la performance d usage. Les activités menées dans le cadre du présent processus d approbation permettront de fournir aux décideurs les renseignements dont ils ont besoin pour prendre des décisions reposant sur des preuves avant de choisir et de mettre en œuvre une nouvelle capacité en matière de TNM. DRDC CSS TR (E) i

5 Executive summary Canadian Less Lethal Technology Approval Process: A Structured Approach to the Selection and Implementation of Less Lethal Technologies for Canadian Law Enforcement Donna Wood; Len Goodman; DRDC CSS TR (E); Defence R&D Canada CSS; December Introduction or background: Recent cases involving the use of conducted energy weapons by Canadian law enforcement have resulted in a number of inquiries and reviews that have recommended that law enforcement depend less on manufacturer information and more on independent technical and medical advice in selecting appropriate less lethal technologies (LLT). In Nov 2010, the Ministers Responsible for Justice endorsed a research agenda that included the development of a less lethal weapons approval processs that would take advantage of independent expertise. To support the development of a LLT Approval Process suitable for Canadian law enforcement, a focus group was established with participation from federal and provincial policy makers, police, medical, legal and public affairs advisors, as well as technical and medical advisors. The Conducted Energy Weapons Strategic Initiative conducted a study to look at how other technologies are approved in Canada and how the United Kingdom and the United States approve less lethal weapons. This study resulted in the identification of a number of elements that should be included in a LLT Approval Process. Results: Through a series of distributed workshops, the focus group members provided input on the major activities: Identify Operational Requirements, Conduct Options Analysis, Conduct Subject Matter Expert Evaluations, Conduct End-User Evaluations, Implement, and Validate In- Service Performance. For each of these activities, the Stakeholders, Inputs, Outputs, Sub- Activities, Methods and Templates are presented. Significance: The process recommended in this report relies on more than just the information provided by manufacturers or the subjective input of police. Independent objective advice is to be sought from both technical and medical experts and approaches the problem from a risk mitigation perspective. It also specifically links the LLT to a documented set of operational requirements and provides law enforcement with a structured method to conduct an Options Analysis and end-user Evalutions. Future plans: This report will be presented to the Federal Provincial Territorial Conducted Energy Weapons Working Group for consideration for implementation as a process to guide the selection and implementation of LLT for Canadian law enforcement. ii DRDC CSS TR (E)

6 Sommaire... Processus canadien d approbation des technologies de neutralisation momentanée : Une approche structurée pour la sélection et l utilisation de technologies de neutralisation momentanée destinée aux organismes d application de la loi au Canada Donna Wood; Len Goodman; Rapport technique CSS DRDC (E) ; R & D pour la défense Canada CSS; décembre Introduction ou contexte : De récents incidents liés à l utilisation d armes à impulsions par les organismes canadiens chargés de l application de la loi ont donné lieu à un certain nombre d enquêtes et d examens au terme desquels on a recommandé aux organismes chargés de l application de la loi de s en remettre davantage aux conseils techniques et médicaux indépendants qu aux renseignements fournis par les fabricants au moment de choisir des technologies de neutralisation momentanée (TNM) appropriées. En novembre 2010, les ministres responsables de la Justice ont approuvé un programme de recherche comprenant l élaboration d un processus d approbation d armes de neutralisation momentanée (ANM) qui reposerait sur les conseils de spécialistes indépendants. Afin d appuyer l élaboration d un processus d approbation de TNM adapté aux organismes canadiens chargés de l application de la loi, on a créé un groupe de concertation auquel participent des décideurs fédéraux et provinciaux, des conseillers des forces policières, des domaines médicaux, juridiques et des affaires publiques, ainsi que des conseillers techniques et médicaux. Dans le cadre de l Initiative stratégique sur les armes à impulsions, on a mené une étude permettant d examiner la façon dont d autres technologies sont approuvées au Canada et la façon dont le Royaume-Uni et les États-Unis approuvent les ANM. Cette étude a permis d établir un certain nombre d éléments qui devraient figurer dans le processus d approbation des TNM. Résultats : Dans le cadre d un ensemble d ateliers précis, les membres du groupe de concertation ont fourni des observations en ce qui concerne les principales activités : déterminer les besoins opérationnels, procéder à l analyse des options, effectuer des évaluations d experts en la matière, effectuer une évaluation de l utilisateur final, mettre en œuvre et valider la performance d usage. Pour chacune de ces activités, les intervenants, les observations, les résultats, les sous-activités, les méthodes et les modèles sont présentés. Importance : Le processus recommandé dans le présent rapport s appuie sur un plus grand nombre d éléments que les seuls renseignements fournis par les fabricants ou les observations subjectives de la police. Dans le cadre de ce processus, on cherche à obtenir des conseils indépendants et objectifs de la part de spécialistes techniques et médicaux et à adopter une approche reposant sur l atténuation des risques. Le processus établit également un lien précis entre les TNM et un ensemble d exigences opérationnelles documentées, et fournit aux organismes chargés de l application de la loi une méthode structurée permettant de mener une analyse des options et des évaluations sur l utilisateur final. DRDC CSS TR (E) iii

7 Perspectives : Le présent rapport sera présenté au Groupe de travail fédéral, provincial et territorial sur les armes à impulsions afin que celui-ci en considère la mise en œuvre en tant que processus visant à orienter la sélection et la mise en œuvre de TNM destinées aux organismes canadiens chargés de l application de la loi. iv DRDC CSS TR (E)

8 Table of contents Abstract..... i Résumé i Executive summary... ii Sommaire... iii Table of contents... v List of figures... ix List of tables... x Acknowledgements... xi 1 Introduction Background CEWSI Project Objectives Aim Related Documents Limitations Definitions Document Outline Governance Legislative Framework Relationship to the FPT CEW WG LLT Approval Process Focus Group Review of Other Approval Processes United Kingdom (UK) Approval Process High Level Process Guiding Principles Application of the LLT Approval Process Benefits Understanding the Process Notation Trigger Activity End Activity High Level Processes Identify Operational Requirements Conduct Options Analysis Conduct Subject Matter Expert Evaluations Conduct End-User Evaluation Implement Validate In-service Performance Stakeholders DRDC CSS TR (E) v

9 2.9 Relationship to Procurement Process Record-Keeping Identify Operational Requirements Description Stakeholders Inputs Outputs Sub-Activities Assemble Requirements Team Document Operational Requirements Validate Requirements Endorse Requirements Methods and Templates Conduct Options Analysis Description Stakeholders Inputs Outputs Sub-Activities Identify Evaluation Criteria Identify sources of information Obtain information (Conduct market research) Assess results Recommend a Preferred Option Methods Decision Points Templates Conduct Subject Matter Expert Evaluations Description Stakeholders Inputs Outputs Sub-Activities Identify SMEs Conduct Quick Look Analysis and Demonstration Perform Individual SME Analysis/Testing/Studies Convene Medical Panel Identify actions/issues for implementation Decision to proceed to End-User Evaluations Methods Technical Evaluation vi DRDC CSS TR (E)

10 5.6.2 Medical Evaluations Examples of Technical and Medical Evaluations Conduct End-User Evaluation Description Stakeholders Inputs Output Sub-Activities Conduct Initial Evaluation Plannning Conduct Detailed Evaluation Planning Execute Evaluation Plan Analyze Evaluation Results Decision- Approve for Implementation? Methods Implement Description Stakeholders Inputs Outputs Sub-Activities Identify Pilot Implementation Site Finalize Procurement Finalize Tactics, Procedures, Policy, Training Determine In-Service Validation Strategy Conduct Training Implement at Pilot Location Monitor Pilot Implementation Complete Implementation Validate In-Service Performance Description Stakeholders Inputs Outputs Sub-Activities Collect Data Analyze Data Implement Changes Methods Template Conclusion Value of a Cooperative Approach to LLT Approval DRDC CSS TR (E) vii

11 9.2 Next Steps References Annex A Template for a Less Lethal Weapons Statement of Operational Requirement Annex B Detailed LLT Approval Process Annex C Framework for Technical and Medical Evaluation Annex D Sample End-User Evaluation Framework List of symbols/abbreviations/acronyms/initialisms viii DRDC CSS TR (E)

12 List of figures Figure 1: CACP National Use of Force Framework... 1 Figure 2: High Level Activities in the LLT Approval Process Figure 3: Identify Operational Requirements Sub-Activities Figure 4: Conduct Options Analysis Figure 5: Sample Spreadsheet for Collecting Options Information Figure 6: Indicative Example of a Qualitative Assessment Figure 7: Indicative Example of a Quantitative Assessment Figure 8: Conduct Subject Matter Expert Evaluations Sub-Activities Figure 9: Conduct End-User Evaluation Sub-Activities Figure 10: Implement Sub-Activities Figure 11: Validate In-Service Performance DRDC CSS TR (E) ix

13 List of tables Table 1: Process Modeling Notation x DRDC CSS TR (E)

14 Acknowledgements The development of this approval process was guided by a special purpose focus group comprised of: Gabi Hoffmann, British Columbia Ministry of Public Safety and Solicitor General Peter Layden, Alberta Solicitor General and Public Security Lisa Sabourin, Ontario Ministry of Community Safety & Correctional Services Vesna Knezevic, Public Safety Canada Perry Kelly, Public Safety Canada Deputy Chief Mike Federico, Toronto Police Service Steve Palmer, DRDC Centre for Security Science Cpl Thomas Walker, Royal Canadian Mounted Police (RCMP) Cpl Ernesto Angeles, RCMP Nancy Leclair, RCMP Greg Muller, RCMP Dr. Len Goodman, DRDC Toronto Dr. Andy Reed, Ontario Provincial Police Chris Lawrence, Ontario Police College. Focus Group members participated in seven workshops and provided advice and input on the various elements of the approval process. The author would also like to acknowledge the advice and guidance provided by the following people from the United Kingdom (UK): Bob Sheridan, Defence Science & Technology Laboratory (Dstl) Graham Smith, Home Office Centre for Applied Science and Technology (CAST) John A. MacDonald, Association of Chief of Police Officers (ACPO) Gary Wedge, National Policing Improvement Agency (NPIA) Andy Gray, National Policing Improvement Agency (NPIA). DRDC CSS TR (E) xi

15 xii DRDC CSS TR (E)

16 1 Introduction 1.1 Background While a primary objective of any police response is to ensure the safety of the public and the police officer, there are many circumstances where dialogue alone cannot resolve a situation and the use of force may be required. The National Use of Force Framework (NUFF) [1] that was produced in November 2000 under the leadership of the Canadian Association of Chiefs of Police (CACP), provides police with the framework needed to assess situations and take appropriate action to reduce the risk of injury to both police and subjects. This graphical framework (Figure 1) takes into consideration the situation, subject s behaviour, the officer s perception and tactical considerations and allows police to determine an appropriate course of action. Use of Force options range from verbal and non-verbal communication through to the use of lethal force. The NUFF identifies a use of force role for intermediate weapons or less-lethal weapons. Figure 1: CACP National Use of Force Framework The Royal Canadian Mounted Police (RCMP) use the Incident Management Intervention Model (IMIM) which has similar elements to the NUFF, providing a framework for the officer to DRDC CSS (E) 1

17 conduct a risk assessment considering situational factors, the subject s behaviour, officer s perceptions and tactical considerations. Police and other law enforcement agencies currently have a number of less-lethal technologies available for use including conducted energy weapons (CEW), chemical incapacitants, batons, kinetic rounds and noise flash diversion devices. Many use of force options have the potential to cause injury or death even though that may not have been the intent. Recent high-profile cases involving the use of conducted energy weapons by police have resulted in additional public scrutiny on the police use of force decisions as well as their selection of specific devices. The Kiedrowski report [2] criticised the decision by several police agencies to adopt conducted energy weapons. In particular, John Kiedrowski noted that since pilot studies were conducted without policy and research assistance, this resulted in incomplete literature reviews, failure to consult national medical associations, too much reliance on information provided by the manufacturer, and an over-emphasis on anecdotal information from police officers. Justice Braidwood [3] observed that the Police Act and the Use of Force Regulation in British Columbia left the choice of LLT to the Director of Police Services or Chief Constable. He noted that this absence of direction lacked transparency and could easily result in inconsistent application across the province. He further advised law enforcement agencies to rely less on manufacturer s materials particularly in areas related to policy and medical risks and specifically called on provincial authorities to exercise its authority in approving specific weapons. In response to these and other high profile observations, the Federal/Provincial/Territorial (FPT) Ministers Responsible for Justice issued a joint statement in Oct 2010 endorsing national guidelines for CEWs and supporting a CEW research agenda that included the Conducted Energy Weapons Strategic Initiative (CEWSI) a collaborative project with Defence Research and Development Canada and Public Safety Canada [4]. 1.2 CEWSI Project Objectives In spring of 2010 Defence Research and Development Canada (DRDC) initiated the CEWSI research project to address priority questions related specifically to conducted energy weapons as well as less lethal weapons in general. The high level objectives of the CEWSI project are to: a. Develop a CEW test procedure and performance measures for current models in use in Canada as an immediate and interim measure to ensure CEWs are meeting manufacturer s technical specifications; b. Recommend a CEW test procedure and develop comprehensive performance measures for possible inclusion in Canadian national guidelines for CEW employment in Canada as part of an enduring capability; c. Convene a panel of medical experts to conduct an independent evaluation of existing research to examine the physiological impact of CEWs, to identify gaps in the research and to recommend steps to address those gaps, and 2 DRDC CSS TR (E)

18 d. Develop a Less Lethal Weapons approval process that could be applied to emerging less lethal technologies. 1.3 Aim The aim of this paper is to outline the major activities and interdependencies that should be included to support the decision to put in service a LLT. 1.4 Related Documents The content of this report is targeted to address the needs of the last CEWSI objective identified above and is the culmination of several pieces of work. The following additional references are relevant to this discussion: a. Goodman, L., Wood, D., (2011) Toward the Development of a Canadian Less Lethal Weapon Approval Process: A Study of Contemporary Process Models. Defence R&D Canada. This document was a compilation of several technology approval frameworks that appear in government, industry and internationally, and was designed to uncover commonalities and elements that could be adopted for a future LLT national approval framework. b. Wood, D., (2011) Statement of Operational Requirements (SOR): Guidance for Creating an SOR for Less Lethal Weapons. Defence R&D Canada Centre for Security Science. This document outlines a systematic methodology for creating a coherent statement of operational requirements in the context of LLTs. c. Roy, R., Sprague, K.B., (2011) Non-Conventional Weapons Assessment: An Evaluation of Potentially Disruptive Technologies. Defence R&D Canada Center for Operational Research and Analysis. This comprehensive document is a review of many LLTs, but from a military perspective. d. Baines, D., Anctil, B. (2011) Classification of Less Lethal Device Technologies- DRAFT. This documents classifies many LLTs into an organized structure based on physiological effect, and may be useful for the law enforcement community to identify technical and medical expertise required. 1.5 Limitations This paper is not a policy statement. It is intended as advice to policy makers on the elements that should be considered in an approval process of LLTs. Implementation is a policy function and is beyond the scope of this paper. The primary purpose of this approval process is to provide law enforcement agencies with a framework that will ensure that decisions made to introduce new less lethal capabilities are informed decisions. The ultimate decision to adopt a new LLT is the responsibility of law enforcement (both operators and policy makers) based on the assessment of operational needs and DRDC CSS (E) 3

19 risks. The adoption of the recommended practices outlined in this report will enable decisionmakers to demonstrate due diligence in gathering a complete picture and obtaining input from all relevant areas. Canadian citizens trust law enforcement agencies to decide when to use force based on the assessment of the balance of individual circumstances. This report makes no attempt to comment on the operational employment of less lethal technologies and should not be deemed to support the use any specific technology or use of force option. By necessity, the use of any level of force by law enforcement carries a risk of injury to either a subject or an officer and the use of this approval process is not a guarantee that an approved device will not cause injury or death. The structure and governance of law enforcement in Canada is complex and the content of this report is not intended to cover all specific circumstances and relationships. The authors have attempted to focus on generic elements that can be adapted for a variety of arrangements, with the objective that the process can be tailored and implemented to meet the needs of individual jurisdictions. The LLT Approval Process presented here is not intended to replace any internal requirements for competitive procurement but rather is intended to support internal processes. This is discussed in more detail in Section 2.7. In addition, although this is not a policy document, it is intended to be used by policy-makers and end users to advise on recommended practices for the approval of LLT in Canada. It will be delivered to the Federal Provincial Territorial Conducted Energy Weapons Working Group (FPT CEW WG) members. Any decision to implement all or part of the elements described here is a decision to be taken by the appropriate jurisdictions. Unlike the military or the police forces in the UK, Canadian law enforcement does not possess the resources to develop new LLTs. For this reason, an assumption is made in the development of this approval process that only commercially-available technologies will be considered. 1.6 Definitions There are many terms used to describe intermediate weapons that are referenced to in the NUFF. For the purpose of this paper, less lethal is used to describe the application of tactics and technologies that are less likely to result in death or serious injury than conventional firearms and/or munitions [5]. Less lethal technologies are also commonly referred to as: less lethal weapons, non-lethal weapons (NLW), less-than-lethal weapons, non-lethal effect, non-lethal capability, minimal force options, intermediate weapons, less lethal devices or less lethal options. For consistency, this paper will use the terminology Less Lethal Technologies (LLT). The Criminal Code of Canada [7] defines a weapon as anything used, designed to be used or intended for use a. in causing death or injury to any person, or b. for the purpose of threatening or intimidating any person and, without restricting the generality of the foregoing, includes a firearm. 4 DRDC CSS TR (E)

20 The scope of this paper is not intended to be restricted to those technologies or devices that meet the definition of weapon under the Criminal Code of Canada. Further, the application of this approval process for the selection and implementation of a less lethal technology should not imply in any way that the technology meets the definition of weapon under the Criminal Code of Canada. Thus to avoid confusion, the terminology of Less Lethal Technology will be used in this report. More specifically, this approval process is meant to be applied to technologies that are designed to have physical effects against humans. This is explained further in Section 2.2. While the primary users of LLTs in Canada are police officers, other peace officers such as border security agents, corrections officers, or sheriffs also have a requirement for weapons that provide options other than lethal. The term law enforcement is used throughout this paper to indicate that this approval process could be easily applied by agencies other than police. 1.7 Document Outline This paper provides the context and high level description of elements that should be included in an approval process for LLTs suitable for Canadian law enforcement. Each activity is described in detail by providing the Purpose, Inputs, Outputs, Stakeholders as well as Methods and/or Templates applicable. Section 2 introduces the reader to the high level activities in the LLT Approval Process as well as the principal stakeholders. Sections 3 to 8 provide specific details related to each of the major activities in the process by outlining the Inputs, Outputs, Stakeholders, Sub-Activities and Methods or Techniques. Section 9 contains the Conclusion. 1.8 Governance Legislative Framework The use of force by peace officers is regulated in the Criminal Code of Canada [7] and must be in compliance with the Canadian Charter of Rights and Freedoms [6]. Under the Canadian Constitution, provinces are responsible for the administration of justice and law enforcement in their jurisdiction, including providing direction on the types of force employed by police. Provincial Police Acts assign various responsibilities and authorities to the provincial and municipal governments as well as police commissions and boards. These responsibilities may include the selection of weapons in the police inventories. In Canada, there are more than 200 police agencies, including the Royal Canadian Mounted Police (RCMP) which serves as Canada s national police force, several provincial police forces, and many municipal and First Nation s police services. The RCMP plays a unique role in Canada in that it is the contracted police force for a number of provinces, the territories and many municipalities. In these cases, the RCMP works with the Provincial, Territorial and Municipal governments in setting the objectives, priorities and goals of the Policing Service. The Provincial Police Services Agreements set out the basic rules on how FPTs will consult when considering new or updated professional policing standards/procedures. With respect to the RCMP, decisions fall under the authority of the Commissioner, including for the selection and approval of weapons, and ensuring the RCMP operates in conformity with the law and in a manner that can ensure national professional standards and deployability across the DRDC CSS (E) 5

21 country. In determining the professional police standards or procedures in respect of the Provincial Police Service, the Commissioner will harmonize those standards or procedures to be substantially similar to or exceed the comparable professional police standard or procedure applicable to all other police services in the Province unless the Commissioner is of the opinion that to do so would: be contradictory to a requirement imposed by law; negatively affect the RCMP s ability to deliver effective or efficient police services; or, negatively affect public or officer safety. Based on these considerations, if the Commissioner is of the opinion that the Service should not adopt the PT standards and procedures, a written explanation is provided to the PT Minister. Any dispute is subject to an established dispute resolution process. Under the Police Service Agreements, all parties recognize Canada s interest in pursuing harmonized police standards and procedures across Canada on the basis of established policing best practices. Notwithstanding this unified policing model, differences among provincial requirements may introduce specific challenges related to information sharing, interoperability and consistency with respect to decisions to adopt new weapons technologies for use by law enforcement. Only British Columbia has a formal process in place to approve LLTs. The other jurisdictions have the authority to implement processes, although none are currently in place. Public Safety Canada has statutory responsibility for law enforcement respecting the RCMP and Canada Border Services Agency (CBSA) and exercises national leadership in policing with FPT colleagues in Justice and Public Safety. Public Safety Canada also develops legislation, policies and national strategies that enhance police efforts to combat serious and organized crime in Canadian communities, improve border security, contribute to joint anti-crime efforts with international partners and promote secure, interoperable information-sharing among federal departments and agencies within the public safety sector respecting criminal justice information. Justice Braidwood made the observation that CEWs, for example, are sold to law enforcement without any approval under the Hazardous Product Act. This act is specifically designed to control consumer products that contain a poisonous, toxic, flammable, explosive, corrosive, infectious, oxidizing or reactive product [8]. This act provides controls over the import or sale of certain products ranging from banning them outright to simply indicating they are items of concern. While LLTs are not consumer products, precedents have been established to ensure sufficient review and due diligence prior to the import or purchase of devices that because of their design, could result in injury Relationship to the FPT CEW WG In 2007 FPT Ministers responsible for Justice directed jurisdictions to collaborate to share information on CEW policies and practices. An FPT CEW WG was created to facilitate this information exchange. At their October 2009 meeting, FPT Ministers Responsible for Justice requested that the WG develop national guidelines for the use of CEWs and a research agenda that would provide scientifically credible, evidence-based research pertaining to the testing and use of CEWs to guide operational policing policy development. The FPT CEW WG is the primary stakeholder group for the CEWSI project at the policy level as it was this forum that identified the priorities to be addressed under the scope of the project. They provide valuable expertise for the project team and access to the policing and the policing policy communities. The primary means of communicating to this group is through Public Safety 6 DRDC CSS TR (E)

22 Canada which chairs the WG. Three members of the FPT CEW WG have participated actively in the focus group used to develop individual elements of the LLT Approval Process. The remaining members were asked to provide a peer review of this report. Parallel to the development of this detailed LLT Approval Process, members of the FPT CEW WG have been working to develop high level guidelines for the approval of LLTs. The intent of these guidelines is to achieve consensus from all jurisdictions on the major elements of the LLT Approval Process LLT Approval Process Focus Group A special purpose focus group was established as the primary source of input to the authors. A series of seven workshops were held between December 2011 and April 2012 to address various elements of the approval process. The following individuals participated in the LLT Approval Process Focus Group: a. Public Safety Canada: Vesna Knezevic, Perry Kelly b. FPT CEW WG Members: Lisa Sabourin (Ontario), Peter Layden (Alberta), Gabi Hoffmann (British Columbia) c. Operational Advisors: Deputy Chief Mike Federico (Toronto Police Service), Corporal Thomas Walker (RCMP), Corporal Ernesto Angeles (RCMP) d. Technical Advisor: Steve Palmer (DRDC) e. Medical Advisors: Dr. Andy Reed (Ontario Provincial Police), Dr. Len Goodman (DRDC Toronto) f. Legal Advisor: Nancy LeClair (RCMP) g. Public Affairs Advisor: Greg Muller (RCMP) h. Training Advisor: Chris Lawrence (Ontario Police College) 1.9 Review of Other Approval Processes A study was conducted [9] of a number of other approval process in Canada as well as how LLTs are approved in the UK and the US. The approval of medical devices, telecommunications devices, unmanned aerial vehicles (UAVs), as well as the Canadian Standards Association (CSA) process and the approval of the Laser Dazzler for the Canadian Forces (CF) were each reviewed with the objective of identifying common elements that could translate into best practices for the approval of LLTs in Canada. Devices can be divided into two classes: those where formal technical standards exist and those where no technical standards exist. The existence of technical standards allows for the objective evaluation of a device, while the absence of a formal technical standard would dictate the need for a more subjective approach based upon a risk assessment. DRDC CSS (E) 7

23 LLTs used by law enforcement are most often not governed by formal technical specifications, and therefore fall primarily into the second class. Two examples of devices from the study that used an advice-based approach were the approval of UAV and CF Laser Dazzler. These approval processes had the following elements in common: a. Functional and approval authorities were identified. Functional authorities provided oversight on the process while the approval authority was the decision-maker; b. A broad selection of stakeholders were engaged including end-users, manufacturers, and subject matter experts; c. In the absence of technical standards, a risk mitigation approach through the investigation of multiple factors was taken. This also included the generation of specific information to address risks (e.g. technical studies and medical evaluations); and d. Documentation to support the decision process was clearly identified United Kingdom (UK) Approval Process Following the study of the various approval processes described above, the author accompanied by a member of the focus group visited with the stakeholders engaged in the UK LLT Approval Process. This visit was particularly rewarding in achieving a better understanding of how medical advice is incorporated into their approval process. The authors believe that there are several components of the UK approval process that could be valuable to a Canadian LLT Approval Process. Within the UK, approval of LLTs is made centrally while the decision to implement is a distributed one. This affords leveraging of federal funding and pooled resources. Through a spirit of cooperation, police services (through the Association of Chief Police Officers (ACPO) and the National Policing Improvement Agency (NPIA)) work closely with the Home Office and the Defence Science and Technology Laboratory (DSTL). Medical advice is provided by a standing Scientific Advisory Committee on the Medical Implications of Less Lethal Weapons (SACMILL). While not specifically entrenched in formal legislation, the cooperation has resulted out of necessity and is valued by the police because of the obvious benefits to having one voice speak for all services.. 8 DRDC CSS TR (E)

24 2 High Level Process 2.1 Guiding Principles The LLT Approval Process was developed keeping in mind the following guiding principles [9]: a. Transparency the approval process needs to be seen as transparent by the Canadian public in order to engender public confidence; b. Independence the approval process needs to be independent of undue influence from product manufacturers and should incorporate scrutiny of candidate technologies by independent experts; c. Flexibility the approval process needs to be flexible enough that steps/requirements can be added or deleted as needed, and d. Operational relevance the approval process needs to consider the operational requirements of the end user in order to ensure effective and efficient approval. 2.2 Application of the LLT Approval Process When to apply the recommendations in this LLT Approval Process raises some challenging questions due to different interpretations of terminology and as reinforces the need for clear guidance to the law enforcement community. The basic activities recommended here are not unique to either the law enforcement community or to the selection of less lethal technologies but are based on best practices in systems engineering and therefore can be easily applied to support the selection of technologies to fill any capability gap (i.e. not just a less lethal capability gap). This process was developed to support the selection and implementation of technologies that are specifically designed to encourage compliance of a subject through some form of physical effect. Intended effects could be through distraction, incapacitation or repelling. Less lethal technologies are designed to affect one or more senses (e.g. sight, sound, smell), to inflict pain, control muscle response or to have some other such physical effect on a subject. The key point here is that the technology has a specific design purpose as intended by the manufacturer. Technologies that are specifically designed to encourage compliance of a subject, through some form of physical effect, introduce particular challenges to decision-makers because of the potential for either injury or criticism in its application. The following paragraphs describe several less lethal technologies and explain the intended physical effect and the value of applying the LLT Approval Process described in this report. The examples provided are illustrative only to guide law enforcement in deciding how to tailor the approval process to meet their needs. a. Conducted Energy Weapons. CEWs are designed with a primary physical effect of electro-muscular disruption of a subject. A secondary physical effect is the infliction of pain through either the charge or, when used in probe mode, the penetration of the DRDC CSS (E) 9

25 probes into the subject s body. CEWs are complex electronic devices with many components, both hardware and software. The design leads to some variability in performance as well as variability in effect depending on characteristics of the subject such as size, weight, physical condition or the presence of other complicating factors (drugs, alcohol). There are inherent risks of unintended injury as advised by the manufacturer. The complexity of both the weapon and its intended use provides a good justification for the use of the full LLT Approval Process including sufficient investment in technical, medical and operational advice. It is noted that the limited competition in the marketplace would indicate that an options analysis would require less effort. b. Impact Rounds. Impact rounds are an example of non-lethal munitions that have an intended physical effect of inflicting pain in order to gain subject compliance. The rounds themselves and the delivery mechanisms are not complex however there are many options available in the marketplace with a large variance in performance. Because of the large number of options available, a good SOR and a comprehensive Options Analysis is important to support the selection. Consistency of performance should be verified through testing but because much of this data already exists it may not be necessary to conduct separate tests or trials; c. Noise Flash Diversion Devices. Noise Flash Diversion Devices are sometimes referred to as flash bangs and act as distraction devices through the use of intense light and sound. Secondary effects due to the explosive charge could also include compression, fragmentation or even fire. As with less lethal munitions, there are many to choose from. Unfortunately, this also means that there is a large variation in performance among the options available once again making the SOR and Options Analysis important. The variation in performance can be a result of the chemical composition of the explosive charge, the shape and composition of the casing or the nature and design of the fuse. Technical testing would need to be conducted to ensure only the desired effects are present and also that performance is consistent and repeatable. Medical advice should be sought specifically to ensure safe operating parameters which would be unique for the device selected, and d. Road Belt These devices are intended to stop the movement of vehicles and are not intended to have a physiological effect directly against a subject. Testing could be conducted to ensure no unexpected secondary risk is introduced; however this is an example where a medical panel would be able to provide little advice other than an evaluation of potential accidents. This process and the above examples are intended to support the selection and implementation of new less lethal technologies for Canadian law enforcement. The process is not meant to be prescriptive, but rather to provide recommendations based on best practices in systems engineering as well as other approval processes. The content of this report should be taken as advice and can be tailored as needed based on the complexity of the technology and the availability of independent and reliable supporting information. The authors recommend not skipping steps completely, but rather tailoring the process through a dutiful consideration of the depth of investigation that would be required to meet the needs of specific circumstances. For example if other law enforcement agencies have conducted an in-depth technical assessment of a 10 DRDC CSS TR (E)

26 LLT, steps in the approval process may be accomplished more quickly and without the need for duplicating testing. 2.3 Benefits There are numerous benefits to following a consistent approval process: a. Information is available to the decision-makers so the decision will be evidencebased, b. The process is repeatable and therefore can provide clarity as to the amount of rigour that should go into the investigation, c. Both the operator and the public will have confidence that due diligence has gone into the selection of a LLT, d. There will be confidence that the decision taken was an informed decision, e. Data obtained during trials and deployment provide a feedback mechanism that will support the validation of initial assumptions, f. The process provides for the proper documentation of the major decision points, and g. Permits the early identification of issues and risks and provides a mechanism to develop strategies to mitigate those risks. 2.4 Understanding the Process Notation Process models use symbols to visually represent the ideas presented. The process model described in this report was developed using Microsoft Visio and uses the notation indicated in Table 1 below: Symbol Interpretation Preparation. This symbol is used to represent what needs to occur as a prerequisite (trigger activity) for the approval process. Predefined Process. This symbol is used to identify the highlevel activities in the LLT Approval Process that precede and follow the activity being discussed in detail. Process. This symbol is used to represent specific sub-activities that comprise the LLT Approval Process. DRDC CSS (E) 11

27 Document. This symbol represents a physical document that is either an input or an output from a specific activity. Only major documents that are generated within the approval process are identified. Decision. This symbol represents a major decision point (i.e. whether or not to proceed to the next step). There will normally be at least two options to exit in this case, Yes or No. Connector. Solid straight lines are used to join individual symbols and represent the direction of flow in the process. Terminator. This symbol is used to indicate the end of the process. Border. This symbol is used to visually group sub-activities into the major activities of the process. 2.5 Trigger Activity Table 1: Process Modeling Notation Capabilities represent the general potential or wherewithal to handle not only a single welldefined problem but a variety of potential challenges [10]. A capability deficiency or gap exists when law enforcement lack the proper equipment, training, policies or other resources to deal with situations within the scope of their responsibilities. Economic and other resource constraints mean that capability gaps need to be prioritized and options considered that will meet both the operational needs as well as the fiscal realities faced by police services. Capability gaps may be identified in a number of ways: a. Formal Capability and Gap analysis. This is a periodic activity that requires the analysis of threats and risks in order to determine the appropriate set of capabilities that are required. The required capabilities are then compared against the existing capabilities to identify the gaps. A formal analysis is normally done at a very high level and often is part of a long term effort; b. Incidents. There are many instances where capability gaps only become known as a result of an unexpected and unsatisfactory outcome to an event. Law enforcement agencies need to continually assess the outcomes of events in order to evolve at a pace greater than that of the subjects. In these circumstances, a more urgent response may be needed to address the gap, and 12 DRDC CSS TR (E)

28 c. Technology evolution and advancement. While not common in the area of LLT, the speed of technology evolution may result in a capability gap that needs to be addressed. Technology advancement could also introduce a device that could substantially reduce the risk of injury to officers, subjects or bystanders. Often, the law enforcement community does not recognize a capability gap until they are exposed at trade shows or by manufacturers to solutions that may offer improvements over what they currently have. The discovery of a new technology should not lead to skipping steps in the approval process. It is important that law enforcement return to the beginning and properly identify the capability gap and what would be required to fill the gap. If a capability gap cannot be articulated, it will be difficult to justify the addition of a new less lethal technology. In an ideal situation, a law enforcement agency will undertake a strategic review of its capabilities in terms of the emerging threats. A proactive approach will ensure sufficient time to properly study the problem, obtain necessary resources and ensure the selected option is the best decision. A reactive approach, often in response to a specific incident where a subject, officer or bystander has been injured or killed results in increased pressure to resolve the capability gap as quickly as possible often with an increased cost and a reduction in confidence that all aspects of the preferred solution have been considered. The third situation that could result in the initiation of the LLT Approval Process would be technology evolution. In this case, the manufacturer of a product currently in service may indicate that it will no longer be produced. This means police services will need to begin the process to look for a replacement. The fact that the same manufacturer is offering a new technology to replace one they are retiring is not justification to not follow the LLT Approval Process. Even minor changes in a LLT configuration can result in a difference in performance and injury potential. 2.6 End Activity The LLT Approval Process is intended to be flexible in order to meet operational needs. It should be used as a guideline of the main activities that will assist the law enforcement community in ensuring sufficient information has been obtained to inform the decision makers. There are several major decision points in the LLT Approval Process. These are defined to allow specific decisions and a commitment to move forward. These decision points often require the commitment of resources for subsequent steps and therefore also can serve as logical points to end the process or to at least pause until the decision can be revisited in the future. While the proposed process does not specifically include details about redoing work, it may be necessary to pause the process and to re-initiate earlier activity. Nothing presented here should be construed as not supporting this flexibility. It is often difficult to know when the approval process is completely finished. While the approval process is effectively over once the device has been successfully implemented, the in-service validation activity is designed to continue throughout the life cycle of the LLT. DRDC CSS (E) 13

29 2.7 High Level Processes The complete LLT Approval Process can be found at Annex B. Figure 2 below presents a high level view of the LLT Approval Process. Each of the major activities is introduced below, but is described in full detail in the other sections of this report. Identify Capability Gap Identify Operational Requirements Conduct Options Analysis Conduct SME Evaluations Conduct End User Evalution Implement Validate In- Service Performance Figure 2: High Level Activities in the LLT Approval Process Identify Operational Requirements The principal objective of this activity is to capture and formalize the needs of the law enforcement community. This will ensure that prior to the purchase and deployment of a technical solution, there is some objective way to validate that the most effective and suitable option has been selected. Through the identification of operational requirements, law enforcement can ensure that operational needs are the priority. More detail on this activity is provided in Section Conduct Options Analysis The Options Analysis activity provides a structured means to collect information about potential solutions and to evaluate them in an objective manner. The methods of options analysis can be used to evaluate high level options/approaches as well as specific technical solutions. More detail on this activity is provided in Section Conduct Subject Matter Expert Evaluations The approval of LLTs requires input from a number of stakeholders early in the process. Multiple Subject Matter Experts (SMEs) should be consulted to get input on a variety of factors such as medical risk to the subject, the officer and bystanders, the technical operation and verification of performance, the legal implications of the proposed solution as well as training and operational issues. In addition to input from the SMEs, consultation with public affairs representatives and where appropriate, public interest groups help to inform the decision-making process. More detail on this activity is provided in Section Conduct End-User Evaluation End-user Evaluations differ from a pilot implementation in that the performance of a device under consideration is validated prior to its use by law enforcement against subjects in real-life. A formal evaluation by end-users will be designed to objectively validate the effectiveness and 14 DRDC CSS TR (E)

30 suitability of a device in specific circumstances. More detail on this activity is provided in Section Implement Once the decision is made to approve a LLT for implementation, there are a number of steps required to ensure a smooth implementation. This is the first time the LLT will be used in reallife situations. Law enforcement agencies need to be confident with the information obtained in previous steps. A staged implementation will provide an opportunity to review and evolve training, policy, procedures, support, etc. before involving the larger law enforcement community. More detail on this activity is provided in Section Validate In-service Performance The objective of this activity is to identify problems and to take appropriate corrective action. As part of the implementation activity, a plan should be developed to allow validation of the inservice performance of the device chosen to ensure it continues to behave as expected. There are a number of options available and they do not need to be resource intensive. More detail on this activity is provided in Section 8, 2.8 Stakeholders In the absence of technical specifications against which to objectively evaluate LLTs, a broad selection of stakeholders should be consulted in order to properly assess and evaluate a new technology. The following paragraphs outline the main groups that should be engaged in the approval process. a. Police Operations/End Users. This group is in the best position to contribute to the identification of capability gaps and must take responsibility to identify and document the operational requirements for LLTs. Members may play a role in conducting options analysis especially if they fill specialist technical positions. End user evaluations require active engagement of end-users that have experience with other use of force options in order to provide a realistic evaluation of the effectiveness and suitability of the proposed solution. Finally, end-users have an obligation to collect data during the in-service phase to ensure technical and operational problems are identified and reported for either immediate or later analysis; b. Police Management. Officers in management positions will play a role in validating capability gaps and operational requirements and will assign the resources necessary to kick-off the approval process by developing the SOR. Once the options analysis phase is complete management will need to review the results and validate the recommendation. It may be necessary for police management to represent the requirement to police boards in order to obtain sufficient resources to conduct technical and medical evaluations. It is expected that senior police staff will fulfill the role of operational SME as a member of the SME panel and will also ensure DRDC CSS (E) 15

31 sufficient resources are made available to properly conduct both an end-user evaluation of the proposed technologies and in-service validation of the performance of the approved device; c. Federal and Provincial Policing Policy. FPT policy-makers play a role in providing functional oversight to the approval process and in some jurisdictions may be the approval authority at various stages of the process. This group is primarily responsible for identifying policy gaps and for generating the policies needed to the fill the gaps. During the end-user evaluation, it is expected that policy-makers would be observers as the information gleaned will ensure policy is informed. The policymaker at the appropriate level is in the best position to chair/lead the panel of Subject Matter Experts that are providing individual and collective evaluations; d. Medical advisors. There is currently a gap in the provision of advice to law enforcement on the medical implications of LLT s. There is no allied health or medical organization in Canada specifically mandated to provide such advice. At the same time individual medical practitioners are very unlikely to be willing to provide a medical opinion due to concerns for legal repercussions. Taking into consideration the example of the UK model of a standing independent medical panel, options will need to be developed to facilitate access to similar expertise in Canada. The features and issues regarding formation, and composition of a medical panel within the LLT Approval Process is discussed in greater detail in Section of this paper; e. Technical advisors. While there may be informed and skilled members of law enforcement agencies to provide advice, this category of stakeholders refers to independent technical advisors i.e. independent of the police service and independent of the manufacturer. It is this stakeholder that will conduct formal technical evaluations of the LLT to validate performance claims. These skills can be found in academia, independent test labs and some industry. The specific skills and capabilities required will depend on the LLT being evaluated; f. Training Staff. Training staff can be expected to contribute to the validation of operational requirements and may be consulted in the Options Analysis. Their principal role however will be to assist in the development of tactics and procedures, the development and conduct of training and the provision of facilities and advice during the end user evaluations. Training staff are primary participants during the Implementation phase. g. Legal advisors. The nature of LLTs means that there will continue to be a risk of injury to subjects, officers and bystanders. Legal advisors should be consulted throughout the process. In particular, they should be consulted during the Operational Requirements (OR) stage to ensure that any legislative obligations are properly considered and included where necessary in the Statement of Operational Requirements (SOR). Legal advisors will also be able to confirm that identified requirements are lawful and appropriate. During Options Analysis, a legal advisor will be able to provide input to minimize the possibility of litigation from manufacturers. During the subject-matter expert (SME) Evaluations phase, the risk of litigation is a concern of the SMEs and specific agreements may be needed to 16 DRDC CSS TR (E)

32 protect the advisors. During implementation and in-service, legal advisors will provide advice to individual police services to ensure actions taken adequately protect all parties. Police services and provincial policing policy representatives have access to legal advice and advice should be sought internally to ensure that any specific nuances are noted; h. Public Affairs. Public Affairs advisors play a supporting role and should be involved in the approval process at any point where information will need to be communicated to the public. This could include the point where a capability gap is recognized, once a product is recommended for SME evaluation, when the decision is made to conduct end-user evaluations and most particularly when the decision is made to proceed to a pilot implementation. Including a Public Affairs advisor in the quick-look activity as part of the SME panel will provide the advisor with sufficient context to identify potential issues that will be of interest and importance to the public and to prepare an appropriate communications strategy to support the introduction of the new device. It should be emphasized that each agency has its own public affairs support function, but police agencies may collaborate to provide one unified message; i. Special Interest Groups. Some police agencies already engage with special interest groups such as First Nations advocacy groups, Tourette Syndrome Foundation, Canadian Mental Health Association, local community advocates, etc. There is a particular desire by most police forces to identify potential strategies when working with vulnerable communities. Depending on the specific situation, special interest groups may be able to provide advice on approaches to avoid or to consider. The best time for this engagement is right at the beginning with the recognition of the capability gap and the work to identify the operational requirements. Their input to the SOR can be incorporated to ensure the unique characteristics of specific scenarios are factored in. j. Industry. Law enforcement needs to be cautioned not to be overly reliant on the information provided by manufacturers however industry does play a key role in providing initial information especially during options analysis. It is critical that manufacturers not play a role during SME evaluations or during the end-user evaluations although often the technical advisors may need to get additional specification details that are not available in marketing brochures and out of necessity will need to be in contact with the manufacturers. 2.9 Relationship to Procurement Process It is difficult if not impossible for most jurisdictions to purchase LLT without going through some type of competitive procurement process. This may be as simple as obtaining several price quotes or as complex as a multi-level evaluation strategy. The purpose of a competitive procurement process is to ensure transparency, due diligence, as well as maintaining a fair and open selection process not at all dissimilar to the purposes of the LLT Approval Process. The elements of this LLT Approval Process can be used to support a competitive procurement process to meet the objectives of both. DRDC CSS (E) 17

33 The absolute minimum requirement for both processes is an SOR. Formalization of this is evidence that careful consideration has gone into documenting what is really needed by the endusers. The Options Analysis activity could be easily integrated into the procurement process. Through a Request for Information (RFI), industry can be asked to self-identify and to selfevaluate against the SOR. Proposals that do not meet the mandatory requirements identified in the SOR would be screened out while those that do meet the mandatory requirements could be evaluated further. Other activities in the LLT approval process can be used to progressively eliminate options and to substantiate the final selection. While not directly related to this approval process, there may be significant financial benefits to proceeding with a common or collaborative procurement effort in order to minimize upfront costs, achieve economies of scale, and ensure satisfactory service and maintenance agreements Record-Keeping An important aspect of substantiating decisions made regarding the selection and implementation of LLTs is the ability to regenerate the material used to support the decision-making process. Throughout the LLT Approval Process, it is critical to retain copies of key documents as part of an official record. Individual incumbents in positions may change however the document trail will allow new personnel to understand the actions taken and will minimize delays or unnecessary repetition of steps. It is not expected that the content of the key documents will remain unchanged just the opposite. As additional information becomes available, it is reasonable for changes to be made to all documents. For this reason, numbers should be assigned to major versions of key documents especially those that are sent for approvals/endorsement (e.g. 2.0 to 3.0). Documents that are sent for review could be assigned a minor version number (e.g. 2.1 to 2.2). Documents that involve specific decisions should be recorded with the resultant decision and retained as part of the official record. 18 DRDC CSS TR (E)

34 3 Identify Operational Requirements 3.1 Description It is the responsibility of the law enforcement community to take the first step in the LLT Approval Process by recognizing that a capability gap exists and by articulating what it will take to close this gap. The Identify Operational Requirements activity in the LLT Approval Process begins with a commitment from the law enforcement community that a capability gap is sufficiently critical that a concerted effort is needed to look for a means to address the gap. In some cases, a change in tactics or training may be sufficient to reduce the impact of the gap. For the purposes of the LLT Approval Process, it is assumed that the capability gap will require the addition of a LLT. The goal of this activity is for the law enforcement community to capture the physical and functional characteristic of a LLT, to describe the operational context where it will be used and the desired effect. The operational requirements should be solution independent in order to ensure a fair, open and comprehensive evaluation. More detailed information on how to capture operational requirements is available in Statement of Operational Requirements (SOR): Guidance for Creating an SOR for Less Lethal Weapons.[11] 3.2 Stakeholders This activity is led by the operational community because no other stakeholder has the context or detailed insight to completely capture the requirement and the effect needed. While it is possible for one jurisdiction or even one agency to produce the SOR independently, there is significant value in gaining consensus from the broader law enforcement community. Participants in this activity are expected to be those with the greatest experience and knowledge of the capability gap and the environment where the new LLT will be used. The identification of operational requirements may be done by a small group that takes responsibility for the work, but it should be circulated to the larger community to ensure maximum input. Depending on the nature of the capability gap, there will be value in consulting with specific special interest groups as they may possess a unique insight on the best methods to address certain situations. Early in the development of the SOR, a thorough assessment of all vulnerable groups and the potential impact of the technology will highlight items for consideration in the SOR including some preferred approached. Some jurisdictions already consult regularly with special interest groups and this context will be a very important element of the finalized SOR. If the identification of the capability gap has emerged as a result of a specific incident, then the law enforcement community may wish to engage with public affairs and legal advisors to ensure that any unique aspects of the incident are considered and factored into the production of the SOR and to ensure that the identified requirements are lawful and appropriate. Management will be expected to endorse the SOR to confirm that the content accurately reflects the desired effects and the planned concepts of operations. This is particularly important to justify the assignment of personnel and financial resources during the rest of the approval process. For a commitment of federal resources, it will be important to demonstrate that the SOR outlines DRDC CSS (E) 19

35 a broad-based capability deficiency and that the operational requirements represent the needs of the larger law enforcement community. 3.3 Inputs Because Identify Operational Requirements is the activity that marks the initiation of the LLT Approval Process, the primary input to the activity involves bringing experience to the table. Initial work has been done to identify the context and circumstances for a number of operational scenarios where LLT could reasonably be expected to be used [11] however there is no expectation that this list is al-inclusive or that they will not evolve over time or differ between jurisdictions. They are nevertheless a good starting point for the operational community to consider the context within which LLTs will be used. A detailed understanding of these scenarios is particularly important. An early step will be to obtain consensus on scenarios which are being considered since the scenarios will drive much of the discussion to fully understand the operational and environmental context. It is reasonable for the descriptions of the scenarios to be expanded or fine-tuned to meet the need based on the capability gap being addressed. Note that if there is no consensus on the capability gap to be addressed, it will be very difficult to come up to a consensus on what it would take to fill the gap. Consensus gathering in understanding the capability gap is a critical success factor to the development of a useful SOR. 3.4 Outputs The output from the Identify Operational Requirements activity will be the Statement of Operational Requirement. This is a formal document that will be used throughout the approval process and for this reason should be version-controlled. Endorsement of the content of the SOR should be clearly indicated in the document and this should be retained as part of the official record. Because this is the cornerstone document for the LLT Approval Process, the SOR should receive a very wide distribution. Regardless of at which level the SOR is produced, the result should be shared with the broader law enforcement community so that others may benefit from the work that has been done. There is also value in sharing the SOR with industry. This may sometimes introduce challenges, since depending upon the nature of the requirement, making SORs public will require discretion balanced against fairness and transparency. Recognizing that Canadian law enforcement does not have the capacity to develop new products, providing the SOR to industry will give them the information they need to improve their products to better meet Canadian needs. A template and guidelines for creating an SOR are contained in Statement of Operational Requirements (SOR): Guidance for Creating an SOR for Less Lethal Weapons. [11] This report is not meant to be prescriptive, but rather is to be used as a guide as to the type of information that should be included in an SOR. The law enforcement community is free to produce an SOR in any format and is not obliged to use the template provided - it is the content that is most important. 20 DRDC CSS TR (E)

36 3.5 Sub-Activities Figure 3 below shows the sub-activities involved in activity Identify Operational Requirements. These sub-activities are described further in the following paragraphs. Figure 3: Identify Operational Requirements Sub-Activities Assemble Requirements Team Once a capability gap has been recognized by management, the decision will be made to initiate the LLT Approval Process. The first step is to appoint a team lead who will have the responsibility to put together the SOR. The skills required to do this include a first-hand understanding of the operational environment including an appreciation of the capability gap, the ability to gain consensus from a group of peers as well as moderate writing skills to be able to put together the SOR. While the requirements team could be assisted by a consultant, the responsibility for the document will rest with the law enforcement community and they will need to make sure that the SOR completely represents what is required. The requirements team does not need to be large, but should be diverse enough to capture unique aspects of both the problem and solutions spaces. As with the team lead, the rest of the requirements team need to be experienced end-users. If the requirements team is drawn locally, then there will not likely be a need for travel. If however, participants are drawn from a broader base, the team lead will need to determine whether face-to-face meetings will be required or whether the work can be accomplished through distributed means such as , teleconference, videoconference or web-conference Document Operational Requirements Regardless of whether the requirements team will be meeting in person or using distributed means, sufficient time will be needed to ensure all participants have a common understanding of the capability gap and the applicable scenarios. Time invested on achieving consensus on the scope will prevent the need to backtrack later because one team member had a different perspective. The SOR should describe the capability gap in detail including alternatives DRDC CSS (E) 21

37 available and should pay particular attention to the impact of options including the option to do nothing. The requirements team should ensure that they are not focusing on any specific technologies, but should consider only the operational effect they would like to achieve. The team should try to visualize how they would use a new technology and where it would fit into their concept of operations. They should also be clear in identifying the scope and limitations to the requirements. It is important to capture the environmental considerations (i.e. the time and space where the LLT would be used) as well as the mandatory and desirable physical and functional characteristics. Finally, the team should describe the planned impacts on the personnel, training and support infrastructure available in the target organizations. Examples of the kinds of requirements that could be included are provided in Statement of Operational Requirements (SOR): Guidance for Creating an SOR for Less Lethal Weapons.[11] Validate Requirements The draft SOR will likely undergo multiple versions and it may be necessary to achieve consensus with the requirements team over several sessions. While it may be difficult to satisfy the perspectives of every member of the requirements team, the team lead should make every effort to achieve consensus. This will prevent problems at future stages. Once the requirements team agrees that the SOR reflects their requirements to the best of their ability, the SOR should be sent for validation by management Endorse Requirements The endorsement of the SOR should be done at the level where decisions are made by the law enforcement organization. This is the level of accountability. If the SOR was developed by a small team in a single police service, then endorsement could be provided by the police chief. If the SOR was developed by a broader team in a province for example, then it may be desirable to receive endorsement by the decision-makers for each of the participant organizations. If there is an appropriate professional body that represents the community then it would also be appropriate for this organization to also endorse the SOR. An endorsement of the SOR does not mean approval of specific LLTs. It is simply an endorsement that the SOR adequately identifies the capability gap and the operational requirements that would satisfy the gap. This endorsement cannot come from the policy, technical, or other such levels. Once the SOR is endorsed, it can be distributed to other stakeholders. It is the endorsed version that will be used at other stages in the approval process. 3.6 Methods and Templates The report Statement of Operational Requirements (SOR): Guidance for Creating an SOR for Less Lethal Weapons outlines a number of methods for obtaining operational requirements and provides a template that can be used to create an SOR. The technique which is most likely to be 22 DRDC CSS TR (E)

38 used will be the focus group approach; however the other techniques will also be valuable to supplement the work of the focus group. A suggested template for an SOR is reproduced in Annex A. DRDC CSS (E) 23

39 4 Conduct Options Analysis 4.1 Description The depth and breadth of the Options Analysis activity will vary from one jurisdiction to another. For example, a smaller police service may have more flexibility to use the Options Analysis to reduce the potential products to one or two preferred devices while a larger police service may have additional procurement constraints imposed that require the Options Analysis activity be performed internal to the procurement process. The objective of the Options Analysis activity is to survey the product landscape to identify one or more solutions that appear to satisfy the Statement of Operational Requirements. It is during the Options Analysis activity that non-operational criteria may also be considered (for example costs related to purchase and on-going maintenance would be key factors to consider). The Options Analysis activity is used to limit the number and breadth of solutions that will be given further consideration and should not be confused with the Subject Matter Expert Evaluation. 4.2 Stakeholders The options analysis activity is driven by the law enforcement agency that is primarily interested in obtaining additional capability. While the Statement of Operational Requirements must be generated by the end-users, the Options Analysis activity can be led by members of the service with more experience in the technologies under consideration (e.g. armourer) or it could be conducted by an independent organization. The individual conducting the Options Analysis will need to engage primarily with industry to identify products that could potentially meet the SOR and to obtain information that is not available in marketing documentation. This should not cause a conflict as long as the process of engagement with industry is properly documented. Other sources of information include other law enforcement agencies (both domestic and international). A wealth of information can also be obtained through attendance at trade shows or review of trade magazines. For those organizations that are required to follow a formal procurement process, new potential products and vendors can be identified through the issuance of a letter of interest or Request for Information (as suits the local procurement processes). 4.3 Inputs The primary input to the Options Analysis activity is the SOR document. Other inputs will include technical and marketing information provided by the manufactures, advice and input from other jurisdictions or law enforcement agencies. Those conducting the Options Analysis should not constrain their search for input and recommendations. A broad search will result in the largest selection of devices that potentially could meet the needs identified in the Statement of Operational Requirements. 24 DRDC CSS TR (E)

40 It is recognized that devices may not live up to all of the claims made by manufacturers. At this stage of the process, this is not a significant issue as all claims need to be validated through later steps in the process such as Technical and Medical Evaluation and during an Operational Trial. Canadian law enforcement agencies do not have the funding or other resources to enter into developmental efforts therefore only solutions that are commercially available should be considered. It is equally important that the products under consideration be technologically mature and free from frequent changes that are indicators of an immature product. 4.4 Outputs The primary output of the Options Analysis activity is the Options Analysis Report. This report provides a summary of the process undertaken, the factors that were considered, and any rating criteria that was applied during analysis, information about the products that were reviewed as well as all results and recommendations. It is not necessary for the final report to recommend a specific make and model of product although some police services may prefer to reach this level of precision early in the approval process. Other police services may be constrained by formal procurement processes that require the Options Analysis elements to be incorporated into a formal procurement selection process. The recommended method of evaluating multiple options can still be applied within a formal procurement process and in fact will support an open competitive process. 4.5 Sub-Activities The Options Analysis activity follows directly after the 'Identify Operational Requirements' activity and precedes the 'Conduct SME Evaluation' activity. Figure 4 below shows the various sub-activities that comprise 'Conduct Options Analysis'. The following paragraphs provide additional detail. Figure 4: Conduct Options Analysis DRDC CSS (E) 25

41 4.5.1 Identify Evaluation Criteria The first step requires the identification of the criteria that will be used to evaluate potential products. The main source of this information is the SOR however additional guidance or constraints may be provided by management or organizational considerations that are not directly related to the SOR. Each criterion should be assigned a weighting factor (as a portion of 100%) relevant to its importance in the evaluation. This will ensure that the information collected is given consideration relevant to its value. The source of these weighting criteria is often management. Some of the criteria that could be considered include: i. Product meets all mandatory Design and System Effectiveness Requirements (indicated by the use of 'shall' in the SOR); ii. Degree to which the product meets desirable requirements (indicated by the use of 'should' in the SOR). For example, the device should be able to function in a temperature range between -20C and +35C, or should have a belt clip of no less than 52 cm width, and be able to withstand torque of 15N before separating from the belt,etc.; iii. Degree to which the product is intended to work within the planned concept of operation; iv. Degree to which tactics will need to be modified; v. Degree to which personnel and training would be affected; vi. Degree to which the product can be supported with existing law enforcement resources; vii. Maturity of the vendor and the product with respect to quality systems, warranties and ongoing support, and viii. Affordability. For many organizations, affordability will be a major factor for the evaluation. In addition to the up-front purchase cost, the full life-cycle costs should be considered including: maintenance, consumables, disposal, in-service support, etc Identify sources of information The next step in the Options Analysis activity is to identify the sources of information that will be used to gather the required information. For those required to enter into a formal procurement process, this information could be requested as part of a letter of interest or request for information/proposal. Other sources may include: i. Trade shows ii. Trade journals 26 DRDC CSS TR (E)

42 iii. Industry associations iv. Military and other law enforcement agencies (domestic and international) v. Dialogue with industry Obtain information (Conduct market research) Once potential sources of information are identified, the collected information should be tracked against the criteria already established. A common way of summarizing this information is in a spreadsheet that summarizes the information provided. It is important that criteria be specific and comparable. Criteria Options Option 1 Option 2 Option 3 Option 4 Criterion 1 Criterion 2 Criterion 2 Figure 5: Sample Spreadsheet for Collecting Options Information Assess results All potential options should be evaluated. Those that do not meet all of the mandatory requirements identified in the SOR should be screened out at this stage. Careful thought must go into the SOR to avoid the situation where a suitable product is excluded from further consideration because of over-restrictive mandatory requirements. The goal at the end of the day is to have at least one product emerge from the Options Analysis Recommend a Preferred Option The end result of the Options Analysis activity may be a recommendation for a specific solution, but not necessarily a specific make or model. This is the ideal situation that would allow a comparison based on performance and cost of multiple products during the SME activity. 4.6 Methods There are multiple methods that can be used to support an Options Analysis. The most common can be grouped into either a qualitative or a quantitative assessment. For a qualitative assessment, where no scores are attached to the options, recommendations are made on the balance of DRDC CSS (E) 27

43 evidence however they are often difficult to substantiate when proof is required. This method is most useful if the options involve high level approaches as opposed to specific technical solutions. A quantitative assessment requires a significantly greater level of effort to ensure the criteria are adequately identified, weights are appropriately distributed and values are accurately assigned. A method of ranking may then be used to determine the recommended option(s). At the end of the day, a well-structured quantitative assessment provides the best defence of a recommendation and fits well within a formalized procurement framework. Indicative examples of both a qualitative and quantitative assessment framework are shown below: Figure 6: Indicative Example of a Qualitative Assessment 28 DRDC CSS TR (E)

44 Figure 7: Indicative Example of a Quantitative Assessment The law enforcement community in Canada will only be considering mature technologies as opposed to products still in development. Therefore the products should be well supported by complete documentation and experiences of other agencies. The collection of supporting information should be relatively easy to obtain. For most solutions, there are multiple vendors of products that well position law enforcement agencies to demonstrate that care and consideration have gone into the Options Analysis activity. 4.7 Decision Points There are a number of points where decisions should be made within the Options Analysis activity. The first is to obtain buy-in for the evaluation criteria and the weighting values assigned. Support for this is needed to prevent criticism once the data collection work begins. The second is once the analysis is done to ensure there is support for the results of the analysis. Once it is clear that a recommendation can be made to consider further one or more solutions, a decision is needed to proceed to conduct a SME evaluation. The results of the Options Analysis should be presented to the organization that will make the decision to proceed to the next step. More often than not, this will be driven by who will fund the SME evaluations (or in the case of nationally funded evaluations, who will be substantiating the request or prioritization for evaluation). 4.8 Templates The Options Analysis Report can be developed using any template, but should include the following sections: DRDC CSS (E) 29