Program Performance: Evaluations and Measurement [Session105] Tom Schumacher, University of Minnesota Ken Zeko, KPMG LLP Forensic John Stoxen, 3M Company Overview for Program Assessment Background The foundation and culture Commonly used metrics and tools Examples Questions and comments 1
Background: Why Do a Compliance Program Effectiveness Evaluation? Required under Federal Sentencing Compliance Program Guidelines Identifies gaps and weaknesses within and across your various programs Tells you the big picture How are your doing as an organization? Creates leadership support Results matter. Period. Guidelines Standard 8B2.1(b): (b) Due diligence and the promotion of an organizational culture that encourages a commitment to compliance with the law within the meaning of subsection (a) minimally require the following steps: (5) The organization shall take reasonable steps (B) to evaluate periodically the effectiveness of the organization s compliance and ethics program 2
Due Diligence Diligence: Vigilant activity; attentiveness Due Diligence: Such a measure of prudence, activity, or assiduity, as is properly to be expected from, and ordinarily exercised by, a reasonable and prudent man under the particular circumstances. What does due diligence mean for your organization? What is reasonable and prudent? Is there an industry standard? Is it the same degree of care used for other management priorities within a prudent organization? What are the hallmarks of due care for management priorities? Resources to do it right Metrics + Trending to measure progress Accountability for results (owned within operational management structure, not HR, Compliance, Counsel, Audits, etc.) Rewards/incentives Responsive action to improve points of weakness 3
Quick Quiz. How is your organization doing with compliance effectiveness evaluation? Completed Evaluation; In good shape 4
Completed Evaluation. Would rather not discuss results File 13ed the idea; in other words 5
Results of File 13 If you find that a person had a strong suspicion that things were not what they seemed or that someone had withheld some important facts, yet shut his eyes for fear of what he would learn, you may conclude that he acted knowingly, as I have used that word. Quiz #2 Write down the approximate number of employees in your organization Divide that number in half THAT IS THE NUMBER OF PEOPLE, ON AVERAGE, WHO HAVE WITNESSED SIGNIFICANT MISCONDUCT IN THE LAST YEAR AT YOUR ORGANIZATION Source: cross industry studies (two separate studies) 6
What to measure Your program itself Look to the Guidelines elements E.g. do you have training, is it working, do you have a Code, is it useful, is it followed, etc., do you have policies, are they accessible, do people follow them, etc. Just track the Guidelines. Outcomes: Is the dang thing working? How Compliance and Ethics Metrics Delivery Options Many options, may depend upon level of reliability sought Surveys (as part of a broader survey, as an independent survey, from a selected subset, etc.), interviews, consultants, peers, etc. Common Compliance and Ethics Metrics Observations of perceived misconduct Willingness to report misconduct/violations and violations in fact are reported when they are perceived to occur Perceptions about the organization s responsiveness to misconduct Fear of retaliation for reporting concerns Willingness to seek help within the organization for ethical issues Supervisors demonstrate/pay attention to ethics Leadership demonstrates/pays attention to ethics Open discussion of ethics in the workplace encouraged Ethical behavior rewarded at all levels Unethical behavior punished at all levels (management accountability) Perceptions of fair treatment in the workplace Employee willingness to deliver bad news to management Employee knowledge of workplace rules Employee commitment to the organization Confidence in preparedness to respond to ethical situations --May have many related/sub-questions organized in different compliance domains; may just ask a few questions in a broad employee survey. You have lots of options. 7
Resources (samples only) Academic Research E.g. Managing Ethics and Legal Compliance: What works and what hurts, Trevino et al, 41 California Management Review, No. 2 1999 Consultants e.g. Executive Branch Employee Ethics Survey 2000, http://www.usoge.gov/pages/forms_pubs_otherdocs/fpo_files/surveys_ques/srvyemp_if_00.p df E.g. United Nations Organizational Integrity Survey 2004, http://www.un.org/news/ossg/sg/integritysurvey.pdf Reports E.g. National Business Ethics Survey, How Employees View Ethics in their Organizations, 1994-2005, available from www.ethics.org KPMG Forensic, Integrity Survey 2005-06, www.us.kpmg.com/news/index.asp?cid=2051 Benchmarking ERC Benchmarking Initiative (7 Questions for licensed use), more at http://www.ethics.org/ecoa-benchmark/ Example: University of Minnesota Process Embedded within broader employee satisfaction survey 6 +1 question 6 culture 1 do they know about the hotline Observation/lesson Will look harder at external benchmarking metrics next time. 8
Survey Questions I have experienced or observed significant misconduct (violation of law, workplace rules, or significant University policy) in my unit/department within the last twelve months? Yes No If Yes, If the misconduct was not known by responsible University officials, did you or someone else report it to responsible University officials or the University s confidential reporting service? Yes, Yes, No, Don t I reported itothers reported it it was not reported know If Yes, Do you believe responsible University officials took appropriate corrective action? Strongly Disagree Disagree to Some Extent Uncertain Agree to Some Extent Strongly Agree I know where to report violations of law or policy (such as the University's confidential reporting line.) 1 2 3 4 5 I believe I would be protected from retaliation if I report a suspected violation. 1 2 3 4 5 University leadership demonstrates integrity and ethical behavior. 1 2 3 4 5 9
Example: Bi-Annual Business Conduct Self-Assessment Process 3 Business Conduct Program Structure Board Oversight Audit Committee CEO Executive Direction Business Conduct Committee Exec VP Div. VP/ Sub. MD Operational Initiatives Central Compliance Department Compliance Contact Compliance Contact 10
3 3 REPORT 2007 BUSINESS CONDUCT SELF-ASSESSMENT DIVISION Revised June 2007 BACKGROUND BUSINESS CONDUCT SELF-ASSESSMENT PROCESS Revised March 2007 Section 1: Division Overview 3M Confidential 3M Confidential A. Division Structure 1. Name and job title of the 3M employee with ultimate accountability for legal and ethical compliance in your division. 2. Compliance Contact (name, regular position and effective date of Compliance Contact appointment). 3. Number of U.S.-based, current full-time equivalent 3M employee positions in your division. 4. List operating subsidiaries, joint ventures, etc. B. Business and Industry 3M 1. List your key customers (by industry segment, if appropriate) in the market channel(s) in which you participate. 2. If you have government customers outside the United States, briefly list which governments and types of products sold. INSTRUCTIONS FOR DIVISION COMPLIANCE CONTACTS 2007 BUSINESS CONDUCT SELF-ASSESSMENT Revised June 2007 3M Confidential These Instructions are for use by division and Big B Compliance Contacts in completing the 2007 Business Conduct Self-Assessment ( Self-Assessment ). Introduction Compliance with law is only the beginning of 3M s business conduct program. We require compliance with the law, but we expect much more of ourselves. We have worked hard to build a culture where employees not only follow the letter of the law, but also the spirit of the law. The Self-Assessment process is intended to help 3M business leaders measure and improve upon the culture of legal compliance and ethical business conduct in their part of 3M s worldwide operations. At the direction of the 3M Business Conduct Committee, all 3M operating divisions complete this version of the Self-Assessment. The Self-Assessment is not intended to assess business or financial risk. It is not an audit or an investigation. Instead, the Self-Assessment is designed to help your division identify existing and emerging business conduct issues in its business activities and to think about how those issues can be dealt with most effectively, recognizing that risk can never be eliminated entirely. This type of periodic analysis is needed to ensure your organization has an effective compliance and ethics program. For more on the importance of an effective compliance program and why 3M uses the Self-Assessment, read the Background on the Self- Assessment Process document. A. Process for Completing the Self-Assessment To complete the Self-Assessment, you will need the following documents, all of which are available in the Compliance Contact TeamRoom in the Business Conduct Self-Assessment category, in the folder titled 2007 Self-Assessment Materials for Divisions : This document provides background information on 3M s Business Conduct Self-Assessment (Self-Assessment) process and on 3M s tradition of legal and ethical behavior. It can be shared with any 3M employee who will be involved in completing a Self-Assessment. 3M s Tradition of Legal and Ethical Compliance 3M has a long history of decentralized leadership in its worldwide operations. Senior management of business units, international subsidiaries and staff groups have been given the freedom and the responsibility to design and implement programs needed to succeed in their particular operations. This freedom has been a key driver of 3M s success. This responsibility has always included creation of an environment in which all employees act in accordance with 3M s values of uncompromising honesty and integrity. 3M Over the years, 3M has learned that some aspects of an effective compliance and ethics program can most efficiently handled on a global basis. 3M first published business conduct policies in 1988, to provide direction on the universal compliance and ethics principles that guide 3M s global operations. The Business Conduct Committee was created in 1991 and was given responsibility for putting together a program to carry out the global aspects of 3M s business conduct program, including: COMPLIANCE CONTACT TIPS Creating business conduct policies 2007 BUSINESS CONDUCT SELF-ASSESSMENT Communicating policies and expectations to 3M s global workforce Providing training to help employees apply business conduct policies Revised to June 2007 their jobs 3M Confidential Ensuring employees have a confidential and trusted means by which they can report suspected business conduct violations These suggestions should help Division and Big B Compliance Contacts efficiently complete the Business Fully and fairly investigating all reports of suspected violations Conduct Self Assessment ("Self-Assessment"). Each organization in the company is unique, so use your judgment as to whether these ideas will work in your business unit. Those of you who were Compliance Contacts in 2005 should notice a number of changes throughout the Self- Assessment that have been made to make the process more focused and streamlined. Divisions which have already completed the 2007 Self-Assessment have noticed a substantial reduction in the time required, compared with 2005. If questions come up while you complete your Self-Assessment, please call or e-mail the Central Compliance Department resources listed in the Background document. Tips for Division Compliance Contacts 1. Read the materials. Start by reading the Background, Instructions and Report documents in the Compliance Contact TeamRoom in the 2007 Division Self-Assessment Materials folder. You cannot map out an efficient strategy for completing the Self-Assessment until you visualize the whole process. 2. Seek assistance early. As you read the Instructions and Report, note who in your organization has the information you need to complete various sections. By asking for assistance on these sections early, you can give people weeks, rather than days, to respond. 3. Keep your Operating Committee informed. Only Division Vice Presidents/General Managers and Compliance Contacts received the kickoff e-mail notice from the Central Compliance Department about the upcoming Self-Assessment. It is your responsibility to inform the other Operating Committee members about the Self-Assessment process. You may forward the original kickoff e-mail to them if you Compliance Contact Tips for 2007 Business Conduct Self-Assessment 11
Section 1: Overview Follow Up on Action Items from Previous Self-Assessment Business Environment Operational Change Risk Areas Business Conduct Culture and Ethical Leadership Training and Evaluation Section 2: Risk Assessment Senior management discusses 26 business conduct risk areas on C&E matrix Consensus rankings assigned for Likelihood of violation Severity of violation Adequacy of training C&E matrix prioritizes risks 12
13
Section 3: Objective Metrics Online Training Completion 3M Standard Opinion Survey Ethics Questions Audit Findings Related to Ethics Business Conduct Violations 14
Section 4: Identification of Action Priorities Operating Committee & Compliance Contact: Review Self-Assessment Report Reach Consensus on Top 3+ Opportunities to Improve Compliance Culture Create Specific Action Plans For Each Priority Item with Person Responsible and Due Date Business Conduct Self-Assessment Report Widgets Business Business Conduct Committee Presentation March 1, 2007 15
2006 Online Training Completion Percentage Division What You Need to Know about E-mail Employment Law for Leaders Understanding the FCPA Business Conduct Preventing Harassme nt (Employee Edition) Preventing Harassme nt (Supervisor Edition) Advertisin g Division 1 Division 2 Division 3 99 99 95 98 99 97 94 Division 4 Division 5 98 98 99 97 98 Division 6 82 94 93 88 92 88 87 Division 7 Division 8 96 93 94 97 96 92 90 Business-Wide Average 90 96 94 92 95 95 93 All Division Average 93 95.9 92.6 92 89 91 88.3 I can report unethical practices without fear of reprisal. 3M Avg. 2006 Division 1 Division 2 Division 3 Division 4 Division 5 Division 6 71 18 11 83 16 1 90 5 0 81 15 4 82 14 4 90 10 0 89 11 0 0% 20% 40% 60% 80% % % Fav % Neutral % Unfav 16
Priority 1: Confidential Information Division 1: Employees have access to many types of confidential information and this was determined the highest risk area. Legal counsel will conduct a 30 minute group training on the importance of confidentiality with reference to the appropriate sections in the Business Conduct Policies before July 1. Division 2: Extensive external interactions in all functions and the division is exposed to risk if these are not handled properly. Ensure % compliance with corporate training and review key confidential risk areas by function. Owned by operating committee functional leaders and will be completed by end of April. Division 3: The organization s strategic direction and growth requires significant M&A and alliance activity with many types of confidential information being handled. Legal counsel to develop a specialized briefing for Steering Committee to be presented by April 30. HR to review and enhance module for new employee and refresher training by April 30. Division 4: Confidential information is shared with customers at multiple levels and by multiple functions. Develop Division Confidentiality Policy and Communications Plan by Compliance Contact and a designated team with efforts initiated by May 1. K P M G F O R E N S I C Integrity Survey 2005-2006 A D V I S O R Y 17
Objectives Provide a behind-the-scenes look at corporate fraud and misconduct in the post-sarbanes-oxley era Offer organizations insights as they consider: Their exposures to fraud and misconduct risks The effectiveness of programs and controls relied on to mitigate fraud and misconduct risks Compare findings to previous KPMG Integrity Survey Methodology Blind national survey of pre-screened working adults that fell into demographics spanning: All levels of job responsibility 16 job functions 11 industry sectors 4 thresholds of organization size Paper based survey conducted between Nov. 04 and March 05 Benchmarked results where possible against findings in 2000 survey 4,056 respondents Confidence level 95%; Precision level (margin of error) +/- 2% 18
Key Findings Level of misconduct remains unchanged 74% reported that they have observed misconduct in the prior 12 month period compared to 76% in 2000 Half reported observing serious misconduct Conditions that facilitate Management s ability to prevent, detect, and respond to fraud and misconduct have improved since 2000 Pressure to engage in misconduct is down Confidence in reporting concerns to management is up Key Findings Employees in companies with comprehensive ethics and compliance programs reported more favorable results across the board than those employees at companies without such programs At companies with ethics and compliance programs: Fewer observations of misconduct Higher levels of confidence in management s commitment to integrity 19
Prevalence of Misconduct During the Prior 12 Months 2005 74 2000 76 0 20 40 60 80 Prevalence of Misconduct That Could Cause a Significant Loss of Public Trust if Discovered 2005 50 2000 49 0 20 40 60 80 20
Root Causes of Misconduct Feel pressure to do whatever it takes to meet business targets Lack understanding of the standards that apply to their jobs Believe that their code of conduct is not taken seriously Lack resources to get the job done without cutting corners Believe they will be rewarded for results, not the means used to achieve them Believe policies or procedures are easy to bypass or override Fear losing their job if they do not meet targets otherwise 57 55 52 49 49 47 46 Are seeking to bend the rules or steal for their own personal gain 33 0 20 40 60 80 Propensity to Report Misconduct Notify supervisor or another manager 81 Try resolving the matter directly 53 Call the ethics or compliance hotline 38 Notify someone outside the organization 10 Look the other way or do nothing 6 0 20 40 60 80 21
Channels Employees Feel Comfortable Using to Report Misconduct Supervisor 78 9 12 Local managers 62 20 19 Peers or colleagues 57 20 22 Human resources department 56 23 21 Ethics or compliance hotline 53 28 18 Legal department 52 28 20 Senior executives 44 29 27 Internal audit department 39 37 24 Board of directors or audit committee 32 35 33 0 20 40 60 80 Note: Chart does not foot to % due to rounding. Agree Unsure Disagree Channels Employee Feel Comfortable Using for Advice and Counsel Supervisor 78 8 13 Peers or colleagues 75 14 11 Local managers 64 19 17 Legal department 61 24 15 Human resources department 61 21 18 Ethics or compliance hotline 54 28 17 Senior executives 48 27 25 Internal audit department 40 37 23 0 20 40 60 80 Note: Chart does not foot to % due to rounding. Agree Unsure Disagree 22
Perceived Outcomes of Reporting Misconduct Appropriate action would be taken 67 22 11 My report would be handled confidentially 64 22 14 I would be protected from retaliation 52 31 17 Those involved would be disciplined fairly regardless of their position 47 32 21 I would be satisfied with the outcome 39 47 15 I would be doing the right thing 89 10 1 0 20 40 60 80 Note: Chart does not foot to % due to rounding. Agree Unsure Disagree Tone at the Top: Perceptions About the CEO and Other Senior Executives Are positive role models for the organization 65 21 15 Know what type of behavior really goes on inside the organization 48 25 27 Are approachable if employees have questions about ethics or need to deliver bad news 55 21 23 Value ethics and integrity over short-term business goals 57 24 18 Set achievable targets without violating my organization s code of conduct 67 21 12 Would respond appropriately if they became aware of misconduct 70 19 10 Set the right tone at the top on the importance of ethics and integrity 65 22 14 0 20 40 60 80 Note: Chart does not foot to % due to rounding. Agree Unsure Disagree 23
Local Tone: Perceptions of Local Managers and Supervisors Are positive role models for the organization 72 13 15 Know what type of behavior really goes on inside the organization 72 15 13 Are approachable if employees have questions about ethics or need to deliver bad news 77 12 11 Value ethics and integrity over short-term business goals 67 19 14 Set achievable targets without violating my organization s code of conduct 73 16 11 Would respond appropriately if they became aware of misconduct 73 15 12 Set the right tone at the top on the importance of ethics and integrity 72 16 12 0 20 40 60 80 Note: Chart does not foot to % due to rounding. Agree Unsure Disagree Team Culture and Environment: Perceptions of Individual Teams and Work Units People feel motivated and empowered to do the right thing 78 9 13 People feel comfortable raising ethics concerns 64 18 18 People apply the right values to their decisions and behaviors 77 13 10 People share a high commitment to integrity 75 14 11 The opportunity to engage in misconduct is minimal 62 17 22 The ability to conceal misconduct is minimal 57 21 22 The willingness to tolerate misconduct is minimal 64 17 19 Adequate checks are carried out to detect misconduct 55 24 20 0 20 40 60 80 Note: Chart does not foot to % due to rounding. Agree Unsure Disagree 24
Presenter s contact details Kenneth Zeko KPMG LLP (214) 840-6497 kzeko@kpmg.com www.kpmg.com The information contained herein is of a general nature and is not intended to address the circumstances of any particular individual or entity. Although we endeavor to provide accurate and timely information, there can be no guarantee that such information is accurate as of the date it is received or that it will continue to be accurate in the future. No one should act on such information without appropriate professional advice after a thorough examination of the particular situation. 25