Auditors Use of Brainstorming in the Consideration of Fraud: Evidence from the Field

Transcription

1 Auditors Use of Brainstorming in the Consideration of Fraud: Evidence from the Field Joseph F. Brazel* Department of Accounting College of Management North Carolina State University Campus Box 8113 Nelson Hall Raleigh, NC Tina D. Carpenter J.M. Tull School of Accounting Terry College of Business University of Georgia Athens, GA J. Gregory Jenkins Department of Accounting and Information Systems Pamplin College of Business Virginia Polytechnic Institute and State University 3007 Pamplin Hall Blacksburg, VA October 29, 2008 We are grateful for the helpful comments provided by Michael Bamber, Chris Agoglia, Mark Beasley, Jodi Bellovary, Grey Burnett, Ann Gamble, Rick Hatfield, Rich Houston, Kathy Hurtt, Karla Johnstone, Linda Parsons, Robin Pennington, Dave Piercey, Velina Popova, Bob Ramsay, Steve Salterio, Mark Zimbelman, and workshop participants at Baylor University, North Carolina State University, University of Alabama, the 2007 AAA Annual Meeting, the Association of Certified Fraud Examiners, the North Carolina Office of the State Controller, First Citizens Bank, and the 2008 Auditing Section Midyear Conference. We also thank Deloitte & Touche, Ernst & Young, Grant Thornton, KPMG, and PricewaterhouseCoopers for their support and participation in this project. * Corresponding Author

2 Auditors Use of Brainstorming in the Consideration of Fraud: Evidence from the Field ABSTRACT Audit standards require auditors to conduct brainstorming sessions on each audit so that engagement team members can discuss the potential for fraud and how the team might respond to the risk of fraud. Little is known about how these brainstorming sessions take place or how the quality of these sessions affect auditors consideration of fraud. This paper reports the results of a field survey of auditors actual brainstorming sessions and fraud-related judgments for 179 audit engagements. This approach allows us to examine how fraud risk factors identified by auditors influence their fraud risk assessments, how auditors respond to fraud risk assessments (i.e., the scope of their testing), and whether the quality of the brainstorming sessions improve these relationships. We find considerable variation in the quality of brainstorming sessions conducted in practice. Further, our results suggest that identified risk factors are positively correlated with auditors risk assessments and, for debt incentives; this relationship is contingent on the quality of brainstorming. We also find that the quality of brainstorming sessions positively moderates the relationships between risk assessments and the staffing, timing, and extent of fraud-related audit procedures. When session quality is examined as a dichotomous (i.e., high and low) variable, we observe a similar effect for the nature of testing. These new findings highlight the importance of higher quality brainstorming sessions in improving the consideration of fraud in the audit process. Keywords: brainstorming quality; fraud; fraud risk assessments; fraud risk response Data availability: Contact the authors

3 I. INTRODUCTION Fraud detection is among the highest priorities for the accounting profession and standard setters (PCAOB 2007; PCAOB 2003; Elliott 2002). Glass Lewis & Co. (2005) estimated that a number of high profile frauds led to the loss of almost $900 billion in market capitalization from 1997 to Consequently, efforts to improve the detection of fraud have been met with great interest. The current audit guidance on fraud was issued largely due to the perception that new guidance could aid the fraud detection process. Statement on Auditing Standards (SAS) No. 99 (AIPCA 2002), Consideration of Fraud in a Financial Statement Audit, introduced several new requirements including brainstorming sessions to discuss the risk of fraud and how the engagement team might respond to that risk. A recent Public Company Accounting Oversight Board (PCAOB) report indicates that inspectors have found substantial variation in the conduct of brainstorming sessions, and that some sessions fail to meet the requirements of SAS No. 99 (PCAOB 2007). Because holding these sessions is a relatively new requirement, little is known about how they occur in practice and how they influence fraud-related judgments. Accordingly, our objective is to examine how the quality of these brainstorming sessions affects auditors consideration of fraud, including the incorporation of fraud risk factors into fraud risk assessments and the responses to these risk assessments. We therefore provide initial descriptive evidence and a basis for the further modeling of and empirical research in auditor brainstorming. This investigation is important for several reasons. First, prior archival research has provided evidence of a link between fraud and financial and nonfinancial measures, internal control effectiveness, corporate governance data, and other fraud indicators (Beasley 1996; Dechow et al. 1996; Bell and Carcello 2000; Marquardt and Wiedman 2004; Brazel et al. 2008).

4 However, these studies typically focused on associations in publicly available data (e.g., Securities and Exchange Commission (SEC) Accounting and Auditing Enforcement Releases (AAERs)) and were not designed to evaluate auditors fraud judgments. Conversely, prior experimental research has examined auditors reactions to hypothetical fraud cases by measuring their identification of fraud risk factors (e.g., Carpenter 2007), their formation of risk assessments (e.g., Asare and Wright 2004), and responses to risk assessments (e.g., Zimbelman 1997). While this prior research has provided some evidence on auditor fraud judgments, there is no evidence on whether the relationships among these judgments are improved by the quality of brainstorming sessions required by SAS No. 99. We contribute to the literature by being the first to use a field survey to collect auditors client-specific brainstorming session data and related judgments (see Gibbins and Newton 1994; Gibbins et al. 2001; Nelson et al for other examples of the use of field-based surveys in accounting research). Because this methodology allows us to collect the results of auditors actual brainstorming sessions, including client risk factors identified, assessments of fraud risk, and related responses, we are able to examine multiple fraud-related decisions and relationships across a number of diverse audit engagements. Second, prior psychology research suggests that effectiveness of judgments and decisions likely depends on the quality of a team s interaction (e.g., Stasser 1999). Because we collect data about session quality, we are able to develop and test a model to examine how the quality of these sessions affects auditors consideration of fraud. Our measure of brainstorming session quality is multi-faceted (i.e., 21-item scale) and developed from a review of the psychology literature on brainstorming and team decision-making, the accounting literature on auditors judgments and fraud, recent PCAOB inspection reports, and professional guidance. To our 2

5 knowledge, this is the first study in accounting or elsewhere to measure and evaluate the effects of brainstorming quality. Third, our investigation is important to auditors, standard setters, and researchers. PCAOB inspection reports indicate that there is considerable variation among audit teams brainstorming sessions (PCAOB 2007). Our study empirically assesses the PCAOB s findings by investigating the role that brainstorming quality plays in auditors fraud investigations. Thus, this study s findings may inform standard setters as they reconsider the professional guidance on auditors consideration of fraud (PCAOB 2004, 2007; Hogan et al. 2008). Further, our descriptive data related to brainstorming practices fills the void left by fairly vague and limited PCAOB inspection reports related to brainstorming sessions (e.g., PCAOB 2007). 1 SAS No. 99 requires auditors to brainstorm about risk factors to arrive at a fraud risk assessment which is used as a basis to plan and perform audit procedures (AICPA 2002). We hypothesize that the quality of brainstorming sessions positively moderates the relationships between (a) fraud risk factors and related fraud risk assessments and (b) fraud risk assessments and subsequent testing. We examine our hypotheses with data obtained data from 179 audit engagements for which partners and managers from the Big 4 firms and a national firm completed a field survey. Prior experimental research has typically found that, when fraud risk factors are present, fraud risk assessments tend to be higher (e.g., Nieschwietz et al. 2000). However, in these studies, risk assessments were made by individual auditors who were provided with a simplified 1 For example, while the PCAOB (2007) reports that inspection teams have identified audits in which brainstorming sessions occurred after planning, they provide no additional details regarding the issue. Therefore, it is difficult for individuals outside of the PCAOB to discern if issues identified by the PCAOB are widespread or perhaps isolated to only a few audit engagements. Also, our study captures brainstorming data that may not be documented by auditors (e.g., participation levels of engagement team members) and thus not available for PCAOB inspection teams to review. 3

6 view of a hypothetical fraud client. Actual audit engagements are rich in context, and the fraud risk assessment is typically determined after engagement team deliberation. Such an environment is difficult to create in an experimental setting. 2 Therefore, there is a need to investigate fraud risk assessments using non-experimental methods. Complementing this experimental work, our field survey results indicate that fraud risk factors identified for a diverse sample of clients were positively correlated with auditor fraud risk assessments, and that the relationship between debt incentives and risk assessment is marginally improved by the quality of brainstorming sessions. Recent PCAOB inspection findings suggest that some audit teams have not responded to their fraud risk assessments with appropriate audit testing (PCAOB 2007). Further, prior experimental research has also documented that auditors have difficulty responding to fraud risk assessments by altering the nature, staffing, timing, and extent of their testing (e.g., Zimbelman 1997). This suggests that responding to a fraud risk assessment may be a more complex task than developing an initial risk assessment. These prior studies were conducted prior to SAS No. 99 and did not consider the role of brainstorming quality. Based on our theoretical model, we predict and find that auditors responses to fraud risk assessments are contingent on the quality of brainstorming sessions. As session quality increases, the expected positive relationship between fraud risk assessments and responses becomes stronger, with increased fraud risks being more positively related to the degree to which engagement teams modified the staffing, timing, and extent of fraud-related audit procedures. If session quality is dichotomized as either high or low, we observe a similar result for the nature of testing. Overall, when session quality is low, there appears to be little relationship between risk assessments and related testing (i.e., 2 For example, there may be a substantial, negative effect on engagement profitability if fraud risk is assessed at high and subsequent audit testing is substantially expanded. Also, in practice, risk factors such as the extent to which management is able to rationalize fraud may be difficult for auditors to measure and incorporate into their fraud risk assessments. 4

7 responses). Our results should be informative to standard setters, especially the PCAOB as they suggest that there is variation in the way auditors are conducting SAS No. 99 brainstorming sessions in practice and that the quality of these sessions is important in order for the benefits to be realized. The remainder of the paper is organized as follows. Section II describes the background and hypotheses development. Sections III and IV provide the methods and results of the study, respectively. Section V concludes the paper. II. BACKGROUND AND HYPOTHESES DEVELOPMENT Model of Audit Teams Use of Brainstorming in their Fraud Judgments Based on the SAS No. 99 framework, as well as prior accounting research and psychology theory, we developed a model of audit teams use of brainstorming. The model, shown in Figure 1, describes how the relationships among fraud risk factors, fraud risk assessments, and risk responses are positively moderated by the quality of the brainstorming session. [Insert Figure 1 here] Overview of SAS No. 99 SAS No. 99 provides guidance to improve the likelihood that auditors will detect fraudulent financial reporting (AICPA 2002; AICPA 2003), and supplies more structure around the consideration of fraud than the previous fraud standards, SAS No. 53 and SAS No. 82 (AICPA 1988, 1997). In essence, auditors consider fraud in a multi-phase manner. First, auditors obtain information to identify risks of material misstatement due to fraud (e.g., incentives). Using such information, auditors brainstorm to identify fraud risk factors and to synthesize this information to develop a fraud risk assessment. Finally, auditors develop a 5

8 response to their fraud risk assessments such as assigning personnel with more experience or specialized skills to the audit or modifying the nature, timing, and extent of audit procedures. Fraud Risk Factors Because frauds are difficult to find and are hidden by individuals who perpetrate them, auditors attempt to identify risk factors that are associated with the occurrence of fraud. These risk factors are commonly categorized along three dimensions of the fraud triangle: incentives, opportunities, and rationalization (AICPA 2002). Auditors synthesize their understanding of these risk factors with other information (e.g., preliminary analytical procedures) to develop a fraud risk assessment (AICPA 2002). Prior research on fraud risk factors using archival methods has found evidence of a relationship between fraudulent financial reporting and financial statement elements, corporate governance quality, suspicious accounting, and other observable data (e.g., Beasley 1996; Beneish 1997; McVay 2006). Fraud Risk Assessments Experimental studies have examined auditors fraud judgments, specifically the relationship between fraud risk factors and auditors fraud risk assessments. While auditors risk assessments are generally higher when fraud risk factors are present (Nieschwietz et al. 2000), more recent studies have found that context has an important influence on auditors judgments. Wilks and Zimbelman (2004) found that auditors who made decomposed (i.e., based upon the fraud triangle) fraud risk assessments were more sensitive to incentive and opportunity risks than auditors who made holistic fraud risk assessments, but only in a low fraud risk context. Asare and Wright (2004) found that, given a case where fraud was perpetrated, auditors who used a fraud risk checklist made less effective fraud risk assessments than those who used no checklist. Since the context of the experimental setting was important in these studies, and these studies 6

9 were conducted before the implementation of SAS No. 99, it is important to re-examine the relationship between fraud risk factors and fraud risk assessments using current empirical data obtained from a diverse sample of audit engagements. Fraud Risk Responses SAS No. 99 requires auditors to react to higher fraud risk assessments by designing audit procedures to obtain more reliable evidence. Prior research has not consistently found a positive relationship between risk assessments and planned audit procedures. Zimbelman (1997), under the guidance of SAS No. 82 (AICPA 1997), found limited evidence of an increase in the extent of audit procedures and no evidence of a change in the nature of procedures in response to fraud risk assessments. Asare and Wright (2004) found no association between fraud risk assessments and planned audit procedures, consistent with recent PCAOB inspection findings that some audit teams do not respond to fraud risk assessments with appropriate testing (PCAOB 2007). Contrary to the results of these studies, Mock and Turner (2005) found that auditors modified the nature, timing and extent of procedures, and assigned more experienced personnel to the engagement as risk assessments increased. All of these studies were conducted prior to SAS No. 99 and exclude the possibility that brainstorming sessions could improve the sensitivity of fraud risk responses to fraud risk assessments. Brainstorming Quality There is little evidence on how brainstorming sessions influence auditor judgments. Carpenter (2007) examined hierarchical teams of audit staff, seniors and managers who brainstormed and found that the teams generated lists with more quality fraud risks than individual auditors. Further, these teams provided more effective fraud risk assessments (i.e., higher when fraud was present at a hypothetical client than when fraud was not present) than 7

10 auditors working alone. Hoffman and Zimbelman (2008) investigated the role that brainstorming plays with regard to audit managers planned audit procedures. They document that brainstorming (vs. no-brainstorming) leads to more effective modification of standard audit procedures. While both of these studies provide important insights, neither study captures the collective fraud process (i.e., the relationships among fraud risk factors, fraud risk assessments, and fraud risk responses), nor do they examine actual teams where partners judgments are likely very important. 3 Furthermore, these studies compare the effects of brainstorming vs. nobrainstorming even though such sessions are now required. We examine the influence of brainstorming quality, a variable that does vary between engagements (PCAOB 2007), on auditors fraud judgments. There is a substantial body of psychology literature on the role of brainstorming. In general, this research has produced mixed findings with some studies reporting that brainstorming leads to the generation of a greater quantity of ideas, while others report no such benefit (e.g., Hill 1982). Most of these studies were conducted with undergraduate students who had no specific expertise or knowledge and were asked to brainstorm about relatively mundane hypothetical problems (Diehl and Stroebe 1987). However, SAS No. 99 requires auditors with specific training and experience to conduct brainstorming sessions, a task with substantially more at risk than tasks typically included in psychology research. Past studies of brainstorming also did not include hierarchical teams, eliminating the potential benefits or disadvantages of normal hierarchies (cf., Hinsz et al. 1997). Hierarchical teams of auditors are likely to develop synergies which contribute different experiences and enable them to build on one another s ideas 3 Neither Carpenter (2007) nor Hoffman and Zimbelman (2008) include audit partners in their brainstorming teams. Given the authority level assigned to audit partners, it is likely that they wield substantial influence over fraudrelated judgments. Indeed, in our sample, partners attended every brainstorming session and were the largest contributors to the sessions (see Table 3). Our use of survey data from actual audit engagements allows us to include the role of audit partners in our examination of auditors consideration of fraud. 8

11 (Carpenter 2007). Conversely, the hierarchies of an audit team might exacerbate the following problems that can occur during brainstorming: evaluation apprehension, group think, and production blocking (e.g., Beasley and Jenkins 2003). Psychology research has documented a link between cooperative outcome interdependence, the effectiveness of team communications, and resultant decisions. Under cooperative outcome interdependence, team members succeed or fail together, directly benefiting from each other s performance (De Dreu 2007). Audit teams have cooperative outcome interdependence because their team interaction in a brainstorming session is dependent on cooperation as opposed to competition. Cooperative outcome interdependence in a team discussion is dependent on the teams ability to discuss shared information and unshared information (Stasser 1999). This literature suggests that the quality of information sharing among team members is important to the ultimate effectiveness of the team discussion. Building on professional guidance and the accounting and psychology literatures, our model, illustrated in Figure 1, posits that the quality of the brainstorming session (a) positively moderates the relationships between auditors fraud judgments that are prescribed by SAS No. 99 and (b) is based on three important elements discussed below: attendance and communication, brainstorming structure and timing, and engagement team effort. Attendance and Communication Attendance and communication affect the quality of team interaction and discussion. For example, Postmes et al. (2001) suggested that teams which promote independence and critical thought are more likely to ensure careful evaluation of information, and that the quality of teams decisions was improved when team members collaborated to perform tasks. Schippers et al. (2003) found that the quality of team communication was enhanced when teams were more 9

12 diverse and when they encouraged open communication. Stasser (1999) illustrated that openness to ideas improves decision quality as it encourages the sharing of unshared information. In line with this research, we expect that as more members of the engagement team attend and participate in the brainstorming session, there will be greater diversity and more sharing of information leading to an improvement in the overall quality of the session and the responsiveness of fraud judgments. Brainstorming Structure and Timing The structure and timing of team discussions are important to the overall quality of team judgments. For example, increased time pressure exacerbates the tendency of teams to seek cognitive closure, reduces motivation, and increases reliance on heuristics (e.g., Kruglanski and Freund 1983; De Dreu 2003). Further, De Dreu (2003) suggested that such pressure reduces team motivation to process information in a systematic and deliberate way leading to reduced team effectiveness. Although time pressure is ever-present in the audit environment, auditors may potentially mitigate its negative effects by, for example, conducting brainstorming sessions early in the planning phase of the audit rather than later. Brainstorming sessions held earlier in the planning process are expected to positively influence auditors fraud judgments as the engagement team will have more time to respond to identified risks. Engagement Team Effort Based on the psychology literature, we propose that engagement team effort is also an important determinant of the quality of teams brainstorming sessions. There are two qualitatively different mechanisms of information processing: shallow, less critical, heuristic information processing and more argument-based, effortful systematic information processing (Chaiken and Trope 1999). The literature distinguishes teams based on which information 10

13 processing mechanism is in place. If the goal is to discover popular beliefs and preferences, team interaction tends to be preference-driven, leaving little opportunity for innovative decision making. In contrast, information-driven teams employ effortful systematic processing, communicating and integrating relevant information, to arrive at higher quality decisions (Stasser and Birchmeier 2003). Such an effortful process focus is evident in SAS No. 99-related guidance which suggests that, for example, auditors individually identify risks and potential audit responses to those risks prior to the brainstorming session (AICPA 2003). Such a requirement should enhance each team member s involvement in the process of considering fraud, augment their client-specific knowledgebase, and improve their contribution to the brainstorming session. Therefore, we expect that the more effort auditors exert prior to and during the brainstorming session the higher the quality of team interaction and resultant fraud-related judgments. In summary, according to SAS No. 99, brainstorming sessions should provide an opportunity for audit teams to identify fraud risk factors, synthesize this information to develop a fraud risk assessment, and consider how they might respond to their fraud risk assessments with audit testing (AICPA 2002). The PCAOB has found variation in the quality of auditors brainstorming sessions and has found auditors fraud risk assessments and related responses to be inadequate (PCAOB 2007). Based on the above discussion, we expect higher quality brainstorming sessions will improve the responsiveness of auditors fraud judgments. Specifically, we propose the following interactive hypotheses: H1: Fraud risk factors become more positively related to fraud risk assessments as the quality of fraud brainstorming sessions increase. H2: Fraud risk assessments become more positively related to fraud risk responses as the quality of fraud brainstorming sessions increase. 11

14 III. METHOD Participants One hundred seventy-nine practicing auditors completed a web-based survey related to the fraud audit process for this study. Participants were auditors from the Big 4 firms and one national public accounting firm and held positions of partner (56), director (2), senior manager (60), and manager (61). 4 Research Instrument Similar to the methodology used in Gibbins et al. (2001), participants completed a field survey which required them to select one recently completed audit for which a fraud brainstorming session was held. 5 They then responded to questions related to the consideration of fraud based upon their experiences related to that one year s audit engagement. Among the data collected from the participants were measures of fraud risk factors for the audit, the overall fraud risk assessment for the audit engagement, measures of audit responses to the fraud risk assessment, and objective and subjective data regarding the quality of the audit s fraud brainstorming session(s). 6 4 According to agreements with their representative firms, we informed participants that the survey would require approximately 20 minutes to complete. Fifteen participants stopped and restarted the survey and it is impossible to determine the time they actually spent completing the survey. The mean (SD) time spent by the remaining 164 participants was (15.51) minutes. Whether participants stopped and restarted the survey and participants time taken to complete the survey is controlled for in our analyses. Discussions with professionals and the authors own review of workpapers indicate that the majority of the data used in this study (e.g., brainstorming session details, client risks, fraud risk assessment/responses) are typically documented in one section of the workpapers as suggested by SAS No The survey was initially evaluated by scholars in the audit / fraud domain. It was subsequently pre-tested with five partners and two senior managers from two international accounting firms to ensure that the survey fully covered the constructs/relationships we wished to examine and that the survey questions and potential responses were clear. Because no substantive changes were made to the survey based on this pre-test, we were able to use the data from these participants in our analyses. 6 We obtained data for a diverse group of audit engagements. Approximately 18% of the audit clients had revenues < $100M, 23% had revenues between $100-$500M, 13% had $500M-$1B, 25% had $1B-$5B, and 21% had >$5B. We classified the audit clients into 8 distinct industries: manufacturing (n=36), retail (n=19), government/not-forprofit (n=34), energy (n=15), high tech/communications (n=25), healthcare/pharmaceutical (n=13), financial services (n=12), and miscellaneous (n=25). Sixty-seven percent of our observations were audits of publicly traded companies. 12

15 Fraud Risk Factors Fraud risk factors elicited from participants for their audit clients were measured according to SAS No. 99 and prior research (e.g., Wilks and Zimbelman 2004) which define fraud risk factors as events or conditions that indicate incentives to perpetrate fraud, opportunities to carry out fraud, or rationalizations to justify a fraudulent action. Previous studies have examined and documented evidence about fraud risk factors on these dimensions. For example, Erickson et al. (2006) posit (but do not find) that one form of incentive for managers to perpetrate fraud is equity-related compensation. With respect to opportunities, items such as corporate governance quality, internal control strength, and the level of accruals have been found to be correlated with fraudulent activity (e.g., Beasley 1996). We measured incentive with seven questions related to market and debt incentives and opportunity with five measures of corporate governance quality, internal control strength, and accrual levels. We found no research to indicate that rationalization to commit fraud could be measured with multiple items, and thus measured this factor with one direct measure with wording consistent with SAS No. 99. In order to compare responses across participants, we measured all but one of these items on a scale where 1 = extremely low and 10 = extremely high. 7 Table 1 provides descriptive statistics on the items used to measure the fraud risk factors of incentive, opportunity, and rationalization. [Insert Table 1] Factor analysis performed on the seven items used to measure incentive indicated that the items satisfactorily loaded in excess of.50 (Nunnally 1978) on two factors (63.89% of variance explained). Items associated with market-related incentives and debt-related incentives loaded on separate factors. Because participants were allowed to provide data on audits of both publicly 7 The item that was not measured on the scale was related to market incentives. The item was coded based on whether the client chosen was publicly traded (coded 1) or private (coded 0). Results are qualitatively similar when this item is removed from our analyses. 13

16 traded and privately-held companies, we decomposed the incentive construct into market incentive and debt incentive. Four of the five items related to opportunity satisfactorily loaded on one factor (63.39% of variance explained). The factor loading for a measure of total accruals (.283) was unacceptable and therefore the total accruals measure was excluded from our opportunity measure in all subsequent analyses. Tests of measurement reliability for the items composing market incentive, debt incentive, and opportunity provided Cronbach s alpha levels approaching or exceeding the generally accepted threshold of.70 (Nunnally 1978). 8 As the construct of rationalization was measured with one item, no factor or reliability analyses were performed. Thus, for the purposes of this study, fraud risk factors are decomposed into: 1) market incentive (five items), 2) debt incentive (two items), 3) opportunity (four items), and 4) rationalization (one item). For each engagement in our sample, the four risk factors are measured as the mean of the participant s responses to the items classified under each factor. Fraud Risk Assessment and Fraud Risk Responses Similar to Carpenter (2007), participants provided an overall fraud risk assessment for the engagement via a scale ranging from 1 ( extremely low ) to 10 ( extremely high ). The nontabulated mean fraud risk assessment (standard deviation) for the sample was 4.92 (2.23). After providing their risk assessments, participants were asked to provide data on how the engagement team responded to the fraud risk assessment. As described by SAS No. 107 (AICPA 2006) and Bedard et al. (1999), the survey allowed participants to describe their response to the fraud risk assessment by the degree to which they modified the nature, staffing, timing, and extent of subsequent audit procedures. Similar to Bedard et al. (1999), Low (2004), and Brazel and 8 The four measures of opportunity provide a Cronbach s alpha =.62. Such a level has been deemed by prior research to denote moderate reliabilities and is acceptable for testing relations (e.g., Bamber et al. 1989; Rudolph and Welker 1998; Viator 2001). Omitting one or more of the four items does not increase the Cronbach s alpha for opportunity. 14

17 Agoglia (2007), the nature of participants responses was measured with two items: 1) the degree to which the number of testing procedures was increased and 2) the degree to which the types of procedures were changed. Staffing was determined by both the extent to which more experienced staff or forensic specialists were used on the engagement as suggested by O Keefe et al. (1994) and Low (2004). Consistent with SAS No. 107 (AICPA 2006), the timing of audit responses was measured with the extent to which the timing of audit procedures was changed on the engagement (i.e., one item). The extent of auditors responses was measured with three items: the extent to which 1) budgeted audit hours, 2) sample sizes, and 3) audit documentation review were increased (e.g., Bedard et al. 1999). All eight items were measured on a scale where 1 = none and 10 = extensively. Table 2 provides descriptive statistics on the eight items used to measure the fraud risk responses of nature, staffing, timing, and extent. [Insert Table 2] Factor analysis performed on the eight response items provided satisfactory loadings in excess of.50 on their respective fraud risk response factors (Nunnally 1978). Percent of variance explained for nature, staffing, and extent were 87.43, 63.41, and 71.40%, respectively. Tests of measurement reliability for the items composing nature and extent provided Cronbach s alpha levels exceeding the generally accepted threshold of.70 (Nunnally 1978). 9 For the purposes of this study, fraud risk responses are decomposed into: 1) nature (two items), 2) staffing (two items), 3) timing (one item), and 4) extent (3 items). The four fraud risk responses were measured as the mean of the participant s responses to the items classified under each factor. 9 The Cronbach s alpha level for staffing was.423. Given this low level of measurement reliability, results for staffing will be provided with the two staffing items combined and separated. 15

18 Quality of the Fraud Brainstorming Session To our knowledge, our study is the first in accounting or elsewhere to use a field survey to study brainstorming and, in turn, empirically examine brainstorming quality as a multi-faceted construct that can influence the effectiveness of team decision-making. We developed a 21 item measure to determine the quality of brainstorming sessions on each engagement in our sample. Participants provided both objective and subjective data concerning their engagement s fraud brainstorming session(s). Responses for each of the 21 items were coded either 1 (high quality) or 0 (low quality) based upon the participant s response (see below for specific coding criteria for each item). Similar to Defond et al. s (2005) and Carcello et al. s (2006) multiple measure of corporate governance quality, each item was equally weighted to avoid adding additional subjectivity to the measure. Thus, the quality of the engagement s brainstorming was rated on a scale from The non-tabulated mean quality score for all of the engagements in our sample was (SD = 3.85), with a minimum score of 3 and a maximum score of 18. Figure 2 provides a frequency distribution of the brainstorming session quality scores. It appears that, for our measure of brainstorming quality, our sample is relatively normally distributed and contains substantial variation. As suggested in our hypotheses development, brainstorming quality can be divided into three constructs: attendance and communication, brainstorming structure and timing, and engagement team effort. Accordingly, we describe each of our 21 items as components of one of these three constructs. [Insert Figure 2] Attendance and Communication Table 3 provides descriptive statistics in relation to the 21 items used to measure brainstorming quality. Included in our measure were ten items related to which professionals 16

19 led/attended the brainstorming sessions and the extent of their contribution. Proper tone at the top and a highly qualified person leading the session are important to the success of the brainstorming process (AICPA 2003; Landis et al. 2008). Further, having members at all levels of the engagement team actively participate in the brainstorming session should encourage those who are less experienced, but have recent firsthand knowledge of client processes, to provide their insights to senior members of the audit team. Also, whole engagement team participation allows more experienced auditors to share their fraud insights/experiences with less experienced team members and enhances the dissemination of ideas produced by the brainstorming session to the entire engagement team (e.g., AICPA 2003). The level of openness to ideas during a brainstorming session should be maximized to promote the free exchange of ideas and opinions amongst the entire engagement team and encourage the sharing of unshared information (e.g., Smith 1993). SAS No. 99 (AICPA 2002) and a recent PCAOB release (PCAOB 2007) encourage the inclusion of specialists in the brainstorming session, and prior research (e.g., Brazel and Agoglia 2007) suggests that audit judgment performance is enhanced when appropriate specialists are utilized on audit engagements. [Insert Table 3] Therefore, with respect to our field survey, if the brainstorming session was led by either the engagement partner or a fraud specialist, for this item, the engagement received a coding of one (i.e., high quality). If all levels of the engagement team (e.g., staff, seniors, managers, and partners) attended the brainstorming session, the session was deemed high quality (coded one). We also rated brainstorming quality to be high when fraud specialists, IT audit specialists, or tax specialists participated in the brainstorming session. Lastly, we measured the respondent s perceived level of contribution to the brainstorming session from staff, seniors, managers, 17

20 partners, and fraud specialists, as well the level of the engagement management s openness to all ideas submitted during the brainstorming session. These items were measured on a scale from 1 ( extremely low ) to 10 ( extremely high ) and responses related to these items were coded high if they exceeded the sample mean for the item (means provided in Table 3). Brainstorming Structure and Timing Data was collected on the structure and timing of the brainstorming session. Practitioner guidance (e.g., Beasley and Jenkins 2003) suggest that the use of agendas should increase the effectiveness and efficiency of brainstorming sessions. Conversely, Asare and Wright (2004) find that the use of fraud checklists may impair auditor performance. Therefore, sessions were coded as high quality if participants indicated that an agenda (checklist) was (was not) used. When a brainstorming session is held may impact its effectiveness. McGlynn et al. (2004) find that brainstorming earlier (vs. later) in a task improves team performance. In relation to fraud brainstorming, a recent PCAOB report indicates that sessions held earlier in the planning process are likely to have a more pervasive effect on the consideration of fraud in the audit process (PCAOB 2007). In addition, SAS No. 99 and Hogan et al. (2008) suggest holding a brainstorming session at the end of each audit, when the completed fraud audit process can be formally assessed by the engagement team, to enhance the effectiveness of the next year s brainstorming session. Thus, if the initial current year brainstorming session was held pre- or early in planning (vs. later in the audit process) or the engagement team held a brainstorming session at the end of the prior year s audit, it was deemed high quality. Engagement Team Effort In general, the audit literature has shown that when auditors are induced to exert more cognitive effort, the quality of their performance increases (e.g., Kennedy 1993). Braun (2000) 18

21 finds a positive relationship between time spent and fraud detection performance. Thus, our survey captured measures of the extent of brainstorming conducted by the engagement team. Brainstorming quality was deemed high if the time spent by the engagement team preparing for the brainstorming session exceeded the mean for our sample, if the engagement had more than one brainstorming session, or if the total length of time for the brainstorming session(s) exceeded the mean for the sample. SAS No. 99 (AICPA 2002) and Beasley and Jenkins (2003) point toward specific efforts, such as requiring auditors to individually identify risks prior to the session and explicit discussion of fraud risk responses, which should improve the quality of the brainstorming session. Therefore, if participants indicated that the engagement team was asked to identify potential fraud risks prior to the session, the session was classified as high quality. Participants, on a scale from 1 ( extremely low ) to 10 ( extremely high ), provided data on the extent of discussion during the brainstorming sessions about how management might perpetrate fraud and possible audit responses to fraud risks at the client. Responses related to these two items were coded high if they exceeded the mean for the sample. 10 Factor analysis performed on the 21 categorical response items provided significant factor loadings (p <.05) for 20 of the 21 items on their respective constructs of attendance and communication, brainstorming structure and timing, and engagement team effort. Further, each of these 20 response items provided significant factor loadings (p <.05) on the overall construct of brainstorming quality as well. Both the Comparative Fit Indice of 0.93, and the Tucker-Lewis 10 Two items potentially related to brainstorming quality were measured in our survey, but not included in our 21- item measure due to a lack of variation in responses. Beasley and Jenkins (2003) describe the advantages and disadvantages of alternative fraud brainstorming techniques (e.g., open brainstorming, round-robin brainstorming) and anonymous submission of ideas. However, we found that a preponderance of our sample (163/179 engagements) used open brainstorming. Round robin and nominal group techniques were used on twelve and four engagements, respectively. Also, in our sample, only 2% of engagements employed anonymous submission of ideas. 19

22 Indice of 0.93 are above 0.90, which is indicative of a good fit for our overall construct of brainstorming quality (Kline 2004). 11 Control Variables Prior research has found that a client s size, industry, and complexity may impact auditor risk assessment and testing decisions (e.g., O Keefe et al. 1994). We control for size by including a measure of the client s revenues in our analyses. We asked respondents to provide the client s industry (see footnote 5) and, similar to Johnstone and Bedard (2003), created dichotomous industry control variables in order to control for industry effects. As suggested by the audit fee and resource allocation literatures (e.g., Hackenbrack and Knechel 1997), client complexity is measured by the number of financial statement auditors assigned to the engagement divided by our size measure. We also control for the effects of numerous auditor or engagement teamrelated variables which prior research has shown to influence audit judgments (e.g., Bonner 1990). Specifically, we control for the specific audit firm which performed the audit and the survey respondent s general audit experience, position in the firm, experience with the client, experience with the client s industry, extent of fraud training, experience level with actual financial statement fraud, and their perception of the entire engagement team s expertise level. As noted previously, we will control for whether the respondent stopped and restarted the survey and their time taken to complete the survey. Our statistical models are presented in Appendix A. Multicollinearity For our tabulated regression results, we mean-centered all independent variables that constituted an interaction term to mitigate the potential threat of multicollinearity and increase 11 Only one item, % of engagements where all levels of engagement team attended sessions, did not load satisfactorily (p = 0.70). Omitting this item from the factor analysis does not significantly change the factor analysis results. Omitting this item from our session quality measure in our regression analyses does not qualitatively change our results. 20

23 the reliability of our regression coefficients (e.g., the Fraud Risk Assessment (FR) for an observation = scale (1-10) measure for the observation mean FR scale measure for the sample (4.92)). Mean-centering also does not adversely change the interpretability of these variables (Dillon and Goldstein 1984; Cronbach 1987; Jaccard et al. 1990; Aiken and West 1991; Neter et al. 1996; Shankur and Tan 2006; Dikolli et al. 2007; Labro and Vanhoucke 2007; Peecher and Piercey 2008). In tabulated regression analyses of our statistical models, mean-centering our interactive variables provides variance inflation factors (VIFs) for all of our variables that are substantially below the standard threshold of 10 (Belsley et al. 1980; Neter et al. 1996; Kennedy 1998; Kutner et al. 2004). Not mean-centering these variables leads to some cases of VIFs exceeding 10. Non-tabulated results of the full models below with non-centered data provide interactive (hypothesized) effects that are qualitatively similar to the mean-centered results tabulated in this study. However, with non-centered variables, incidences of multicollinearity render many main effects (e.g., market incentives, fraud risk assessment) insignificant. While a less ideal solution, multicollinearity can be mitigated by removing correlated variables from the model (Dillon and Goldstein 1984; Neter et al. 1996). Again, results related to our statistical models with only interactive effects and control variables (i.e., main effects of hypothesized variables removed) are also qualitatively similar to our tabulated regression results. IV. RESULTS Correlations between Variables A correlation matrix is presented in Table 4. Initially the correlation matrix contained all four fraud risk factors, the fraud risk assessment, the four response items, the 21 brainstorming quality items, the brainstorming quality measure, and all eighteen control variables (49 variables in total). However, to present a parsimonious correlation matrix, brainstorming quality items and 21

24 control variables were eliminated from presentation in Table 4 if they were not significantly correlated (p-value <.05) with at least four of the hypothesized variables (e.g., fraud risk factors, fraud risk assessment). Reducing this constraint to three led to a substantially larger correlation matrix. Due to the potential impact of client size on auditor judgments (e.g., O Keefe et al. 1994), we include client size in Table 4 even though it failed to meet the aforementioned criterion. [Insert Table 4] Descriptive Statistics of Brainstorming Practices Given our sample of 179 recently completed audit engagements across five public accounting firms, Table 3 provides rich descriptive evidence regarding how the average fraud brainstorming session is being performed in practice. First, a partner or forensic specialist only led the session 60% of the time, pointing towards the important role audit managers and seniors probably play in determining the quality of the session. Although not tabulated, while forensic specialists attended 56 of the brainstorming sessions, they only led the session on four engagements in our sample. On 27% of the engagements, not all members of the engagement team attended the session. In such cases, it is possible that important information regarding fraud risk factors, assessments, and responses were not conveyed to audit staff and seniors and were thus lost (or conveyed secondhand). Second, consistent with recent PCAOB inspection team reports (PCAOB 2007), we find evidence that specialist (fraud, tax, IT) attendance in sessions varies. It is also interesting to note that, when a fraud specialist was included in the session, their level of contribution was fairly high (7.01), but less than engagement partners (8.41) or managers (7.93). Third, our results are consistent with the hierarchical structure of engagement teams. Specifically, the level of contribution to the session increases with seniority with staff (4.32), 22

25 seniors (6.03), managers (7.93) and partners (8.41) contributing according to their level of experience in the firm. Fourth, consistent with PCAOB inspection team findings (PCAOB 2007), for 35% of engagements, sessions were held after the early stages of planning. Further, 84% of engagements did not perform a final wrap-up session in the prior year. Not holding the session early in the engagement process and not holding a review of the fraud process at the end of the audit may lead to a failure to incorporate fraud risk assessments into substantive testing and an impairment of engagement team learning in relation to their client vis-à-vis fraud, respectively. Last, engagement team preparation for the sessions tended to be substantial, fraud risk sessions lasted, on average, 1.5 hours and 49% percent of engagements had more than one session. Hypotheses Testing Fraud Risk Factors, Fraud Risk Assessments, and the Quality of Brainstorming Sessions Table 5 provides the results of Hypothesis 1 testing. Hypothesis 1 posits that the relationship between fraud risk factors and fraud risk assessments is positively moderated by the quality of the brainstorming session. Thus, Hypothesis 1 is supported by regression results that provide positive and significant interactions between fraud risk factors and the quality of brainstorming sessions on fraud risk assessments. For presentation purposes, only effects related to our control variables with p-values <.10 are presented in our tables. As posited by Hypothesis 1, we find a marginally significant and positive interaction between debt incentive and the moderating variable of session quality on fraud risk assessments (p-value =.06). While the main effects of market incentive, opportunity, and rationalization are all at least marginally significant in Table 5, their interactions with the quality of the brainstorming session are not. Thus, our results provide limited support for Hypothesis 1. [Insert Table 5] 23