More-Advanced Statistical Sampling Concepts for Tests of Controls and Tests of Balances

Transcription

1 APPENDIX 10B More-Advanced Statistical Sampling Concepts for Tests of Controls and Tests of Balances Appendix 10B contains more mathematical and statistical details related to the test of controls sampling introduced in Chapter 10. In fact, it is like a technical appendix on test of controls sampling and substantive testing. In this appendix you will find more-specific explanations of how to do statistical sampling in the test of controls phase of the control risk assessment work, and for substantive tests of details. After studying this appendix in conjunction with Chapter 10, you should be familiar with and able to perform the listed topic items. LO8 Demonstrate that you can apply statistical sampling concepts for tests of controls and tests of balances using the audit risk model. Appendix Topics Part I: Some More Theory on the Behaviour of Sampling Risks Part II: Test of Controls with Attribute Sampling Risk and Rate Quantifications Explain how risks behave under the hypothesis testing approach in statistical sampling. Explain the role of professional judgment in assigning numbers to risk of assessing control risk too low, risk of assessing control risk too high (efficiency risk), and tolerable deviation rate. Use statistical tables or calculations to determine test of controls sample sizes when there is more than one population. Use tables and calculations to compute statistical results (the computed upper error limit (UEL), which is the same as achieved P in Chapter 10) for evidence obtained with detail test of controls procedures. Use the discovery sampling evaluation table for assessment of audit evidence. Choose a test of controls sample size from among several equally acceptable alternatives. 10B-1

2 10B-2 PART 2 Basic Auditing Concepts and Techniques Part III: Audit of an Account Balance Calculate a risk of incorrect acceptance, given judgments about inherent risk, control risk, and analytical procedures risk, using the audit risk model, such as in the CPA Canada Handbook Section 5095, Guideline on Materiality (AuG-41, paragraphs 41 and 42). Explain the considerations in controlling the risk of incorrect rejection. Explain the characteristics of monetary-unit sampling (MUS) and its relationship to attribute sampling. Calculate a monetary-unit sample size for the audit of the details of an account balance. Describe a method for selecting a monetary unit, define a logical unit, and explain the stratification effect of MUS. Calculate a UEL for the evaluation of monetary-value evidence, and discuss the relative merits of alternatives for determining an amount by which a monetary balance should be adjusted. Use your critical thinking skills to evaluate newspaper articles about auditor applications of statistical sampling. Part IV: Other Topics in Statistical Auditing: Regression for Analytical Review, Monetary-Unit Sampling for Tests of Controls, and the Audit Risk Model Explain how linear regression can be used as an analytical review procedure. Describe the advantages of using MUS to test controls. Explain how the audit risk model evolved from the adoption of statistical sampling in auditing. Part I: Some More Theory on the Behaviour of Sampling Risks This appendix continues the discussion that began in Chapter 10 regarding statistical sampling in auditing. As you will recall, Chapter 10 ends with some of the mechanics of using statistical formulas and tables in auditing. Here, in this appendix, you will find more details on the theory and mechanics of statistical auditing. We begin with an overview of some theory about how risks are controlled with all statistical audit procedures, using the concept of hypothesis testing. We then review applications to attribute sampling in tests of controls, followed by statistical applications of substantive testing of details on balances. The appendix ends with statistical applications of regression models to analytical review procedures. Under the negative approach, confidence level equals one minus effectiveness risk, while under the positive approach, confidence level equals one minus efficiency risk. Recall that the negative approach is the more important and common approach in auditing. In particular, it underlies all attribute sampling tables and formulas. MUS always uses the negative approach. The positive approach frequently underlies formulas using the normal distribution assumption; this is briefly discussed at the end of the appendix. Because of its simplicity and straightforward relationship to audit objectives, the negative approach is used throughout the appendix. However, it should be noted that the positive approach is much more important in the sciences, and, in particular, the efficiency risk when associated with the null hypothesis of no difference is the more important risk of the sciences. In contrast, the effectiveness or effectiveness risk is more important in auditing because it is related to the more important hypothesis of material misstatement or significant difference, which it is the auditor s job to detect. In fact, some have characterized the purpose of the auditing profession as controlling effectiveness risk associated with financial statements. For this reason, effectiveness risk is associated with audit effectiveness.

3 APPENDIX 10B More-Advanced Statistical Sampling Concepts for Tests of Controls and Tests of Balances 10B-3 How Sampling Risks Are Controlled in Statistical Auditing When using statistical sampling concepts in auditing, we need to develop a decision rule, which must be used consistently if we are to control risks objectively. It is the assumption of the consistent use of a strict decision rule that allows the risks to be predicted and thus controlled via the sample size. The decision rule is frequently referred to as a hypothesis test, and the auditor is typically interested in distinguishing between two hypotheses: Hypothesis 1: There exists a material misstatement in the total amount recorded for the accounting population. Hypothesis 2: There exists no misstatement in the amount recorded for the accounting population. The decision rule the auditor uses in statistical auditing is to select one of these two hypotheses based on the results of the statistical sample. The mechanics of this will be discussed later in this appendix. For now, we are interested in depicting what happens to sampling risks (risks that arise when testing only a portion of the population statistically) when a consistent decision rule is used. This is when the summary concept of a probability of acceptance curve becomes useful. This curve can be used to represent all the possibilities of sampling risk about a sample result. Probability of Acceptance Curve (or Acceptance Curve, for Short) Consider Exhibit 10B 1, which plots the probability of acceptance of the recorded amount against total misstatement within the recorded amount of some accounting population, such as aggregate accounts receivable. The horizontal axis reflects total misstatement, while the vertical axis reflects probability. In a perfect world of no uncertainty, auditors would want to have a zero probability of accepting a recorded amount having material misstatement (MM in the exhibit). However, the concept of testing or sampling only a part of a population requires the auditor to be willing to accept some uncertainty concerning the total population value. This is reflected in the fact that the probability of acceptance cannot be zero at MM with sampling. However, the auditor can design his or her audit so that this probability is appropriately low (see Exhibit 10B 1). An auditor using a consistent decision rule selecting one of the hypotheses discussed above with a given sample size over a range of possible errors will experience varying probabilities of acceptance, as indicated in Exhibit 10B 1. This is what we mean by a probability of acceptance curve. Note an important feature of this curve: as the error amount increases (going from left to right), the probability of acceptance goes down (as one would expect, because as the amount of error in a population increases, the chances that a sample of a given fixed size will accept the population will decrease). How fast EXHIBIT 10B 1 Probability of Acceptance versus Total Misstatement Probability representing efficiency risk = incorrect decision Probability of correct decision Efficiency risk range MM Probability Probability representing = of incorrect effectiveness risk decision Effectiveness risk range

4 10B-4 PART 2 Basic Auditing Concepts and Techniques the curve drops depends on a variety of factors, including the sample size, the statistical model, and to a lesser extent the error pattern in the population. It is important to remember that each sample size will have a different acceptance curve. Also, as the sample size increases, the curve shifts upward, until it looks like a rectangle, with a value of exactly one from zero errors to materially misstated errors. At MM, the perfect acceptance curve drops to zero, and it stays at zero as the amount of error increases to way above materiality. Note that a perfect acceptance curve results from doing a 100% examination of the population, and also that there is no efficiency or effectiveness risk with a perfect acceptance curve. You can think of the perfect curve as the ideal knowledge state for the auditor. See Exhibit 10B 2 and the discussion following. Concepts of Sampling Risk When there is a less than perfect (i.e., less than 100%) examination with a given test, the probability of acceptance curve is useful for depicting the full range of sampling risks that the auditor may experience. This is summarized in Exhibit 10B 1. To understand these risks, let us consider some scenarios. First, assume there is an immaterial amount of misstatement (i.e., left of MM in Exhibit 10B 1), say, half MM. The probability of acceptance curve in Exhibit 10B 1 tells us the probability of accepting any given amount of misstatement (including half MM). Is acceptance the correct decision given this amount of misstatement? The answer is yes, because the amount of misstatement is less than material. Thus, the probability of acceptance gives us the probability of making the correct decision at half MM. Since the only other alternative in this simple framework is to reject the reported amount, an incorrect decision, this probability of making the incorrect decision must then be one minus the probability of acceptance. This risk of incorrect decision when there is less than material misstatement (i.e., to the left of MM) is the efficiency risk in auditing. A completely different error is possible when there happens to be a material misstatement, that is, to the right of MM in Exhibit 10B 1. Now, accepting the reported amount is the incorrect decision and the probability of accepting the recorded amount is, thus, the risk of accepting a material misstatement. This risk of accepting a material misstatement is the effectiveness risk. The correct decision is to reject the reported amount when there is a material misstatement, and this equals one minus the probability of accepting the reported amount when there is a material misstatement, or one minus effectiveness risk. To recap, efficiency risk can only occur when there is less than material misstatement, and efficiency risk equals one minus the probability of acceptance when there is less than material misstatement. On the other hand, effectiveness risk can only take place when there is a material misstatement, and effectiveness risk equals the probability of acceptance of the recorded amount when there is material misstatement. (Due to rounding errors in calculating values for the tables given later, it is more convenient to say accept up to and including exactly material errors, and reject anything, even if just a penny, above it. That is the mathematical treatment, and the auditor should of course qualitatively re-evaluate all such situations.) Another important thing to note is that as error increases, the probability of acceptance decreases, and, therefore, effectiveness risk decreases. The maximum effectiveness risk is thus at the smallest amount of material misstatement, which is at the point MM itself; that is, maximum effectiveness risk is at MM. Hence, if the auditor controls effectiveness risk at a specified level at MM, he or she automatically controls it at a lower level for errors greater than MM. This is not true for efficiency risk, however. An analysis of Exhibit 10B 1 should make clear that as the probability of acceptance decreases with increasing errors in the immaterial error range, efficiency risk is increasing. Maximum efficiency risk therefore occurs at just below MM and equals (at the limit) one minus effectiveness risk at MM. For a numerical example, assume effectiveness risk is controlled at Then we know that maximum efficiency risk is = In practice, efficiency risk is usually controlled only at zero errors by controlling efficiency risk at its minimum level. Note that this concept of risk control is completely different from that of effectiveness risk control, which is always done via the effectiveness risk s maximum value.

5 APPENDIX 10B More-Advanced Statistical Sampling Concepts for Tests of Controls and Tests of Balances 10B-5 The difference between behaviours of the two sampling risks and the different concepts of control of efficiency and effectiveness risks have important implications for auditors. First, note that it is a very good thing that we can control effectiveness risk at its maximum! This is because effectiveness risk is the more important risk for auditors. It is so important that some people characterize the reason for the existence of the audit profession as being to control effectiveness risk. If auditors do not detect material misstatements in financial statements, then who will? Thus, effectiveness risk can be said to relate to audit effectiveness. Note also that effectiveness risk is very treacherous. This is so because the auditor has no idea that an effectiveness-type incorrect decision is being made. The sample evidence gives no indication that the amount of misstatement is material. The only way to control effectiveness risk is by planning for it in advance during the sample planning stage of the audit. In statistical sampling formulas, this is done through the calculation of the sample size using a planned precision for the test, similar to the polling example discussed in the chapter. More illustrations will be provided later in this appendix. The consequences of an efficiency-type error are much less dire for the auditor but could be important nevertheless. The auditor is aware that there may be a problem because by rejecting a sample result, there is either a material misstatement or an efficiency error. At this point, auditors have several options: (a) review evidence of related tests, (b) expand audit work by increasing the sample size of the test and related procedures (the audit risk model can assist in this task), (c) ask for an adjustment of the recorded amount of the population so that the estimated probability of material misstatement is reduced to an acceptable amount, or (d) perhaps modify the audit opinion based on the test results. The appropriate action, again, depends on professional judgment and the circumstances. The auditor should, however, be prepared to give good reasons for the decision. Generally, an efficiency-type incorrect decision increases the amount of audit work unnecessarily, and thus it is characterized as an audit efficiency error. There is more complete discussion of auditor options at various stages of the audit later in the appendix. Positive and Negative Approaches and Confidence Level In the statistical literature, there is frequent reference to the concept of the confidence level of a statistical test. How does this relate to efficiency and effectiveness risks as used in auditing? The answer depends on a number of things, such as the way the hypothesis tests are constructed. Confidence level is related to the primary or null hypothesis of the test. Specifically, confidence level equals one minus the risk of rejecting the null hypothesis when it is true. Thus, confidence level depends on the null hypothesis used. In auditing, a distinctive statistical terminology has evolved over the years: if hypothesis 1 is the null hypothesis, then we are using the negative approach to hypothesis testing, and if hypothesis 2 is the null hypothesis, then we are using the positive approach to hypothesis testing. Under the negative approach, confidence level equals one minus effectiveness risk, while under the positive approach, confidence level equals one minus efficiency risk. The negative approach is the more important and common approach in auditing. Most important is that the effectiveness risk is controlled by selecting the appropriate confidence level, since confidence level equals one minus effectiveness risk. This also means that the confidence level is aligned with the assurance level provided by the test. This can be seen by noting that effectiveness risk is very similar to the definitions of the audit risk model and its various component risks. This aspect of the audit risk model is further discussed in the Application Case for this appendix. The negative approach is normally used to statistically test internal controls. In particular, it underlies all attribute sampling tables and formulas. MUS, discussed later in the appendix, always uses the negative approach. On the other hand, the positive approach frequently underlies formulas using the normal distribution assumption. The consequences of this are briefly discussed at the end of the appendix. Because of its simplicity and straightforward relationship to audit objectives, the negative approach is used throughout the remainder of the appendix. However, it should be noted that the positive approach is much more important in the

6 10B-6 PART 2 Basic Auditing Concepts and Techniques sciences, and, in particular, the efficiency risk, when associated with the null hypothesis of no difference, is the more important risk of the sciences. Thus, your statistics course may have used a positive approach, and so within that course confidence level equals one minus efficiency risk. Under the positive approach, there is a less-straightforward relationship between assurance, as the auditor uses that term, and the confidence level. Hence, under the positive approach the auditor needs to make adjustments to the confidence level in determining the assurance level provided by the statistical test. These adjustments are particularly important if the statistical test is to be used with the audit risk model. If we use the negative approach, then the effectiveness risk is controlled directly through the confidence level, as discussed above. But then, how is efficiency risk controlled using the negative approach? The first thing to remember is that efficiency risk cannot be controlled the same way that effectiveness risk is controlled (at effectiveness risk s maximum level). As discussed above, you can only control efficiency risk at its minimum value. This is true regardless of whether the positive or negative approach is used. With the formulas we use, this minimum efficiency risk is zero. Under the negative approach, the way to reduce efficiency risk throughout its range is to use a sample size that is bigger than the minimum for the confidence level of the test. Remember that under the negative approach, confidence level equals one minus effectiveness risk, so you are already controlling effectiveness risk once you set the confidence level. Any audit plan that wants to reduce efficiency risk across the entire efficiency risk range needs to do so by increasing the sample size beyond that of the minimum for the stated confidence level. This is illustrated in the end-ofappendix discussion case and discussed in more detail later in this appendix. Effect of Changing the Sample Size under the Negative Approach Under the negative approach, confidence level (CL) = 1 effectiveness risk. Under some interpretations, auditors can equate the confidence level with audit assurance, and so these auditors work with assurance or confidence factors rather than risk factors. The underlying principles remain essentially the same, however, whether the auditor works with risk or confidence levels. What if the auditor varies the sample size while keeping the confidence level constant? Exhibit 10B 2 illustrates what happens: curve A is the probability of acceptance with the smallest sample size possible for given effectiveness risk, curve B results from using twice the curve A sample size, and curve C results from four times the curve A sample size, whereas curve D represents a perfect acceptance curve with no sampling risk. Curve D occurs when a population is examined 100%. From Exhibit 10B 2, it is evident that if confidence level (and thus effectiveness risk, under the negative approach) and material misstatement (MM) are held constant while changing the sample size, it is the efficiency risk that changes with the sample size specifically, efficiency risk is reduced throughout its range when the sample size is increased, and vice versa. This illustrates that if the auditor wishes to reduce efficiency risk while keeping effectiveness risk and MM unchanged, then a larger sample size needs to be used. In fact, if any one of efficiency risk, effectiveness risk, and MM were to be reduced, sample size would need to be increased, and vice versa. Curve A reflects the acceptance curve with the smallest sample size possible with MUS for a given effectiveness risk (1 CL). This smallest sample size is called a discovery sample. If the auditor wishes to control efficiency risk to a lower level than indicated by curve A, the sample size should be increased, while keeping the planned precision and confidence level fixed. Sample size increases in practice are normally implemented through formulas by one of two major approaches: (1) increase the number of errors to be accepted by the sample, or (2) increase the planned expected error rate in computing the sample size. The second approach is the one suggested in CAS 530. This approach also helps explain the concept of performance materiality in Chapter 5 and CAS 530. If the smaller performance materiality is used in planning a sample size but the overall (or specific) materiality of Chapter 5 is used to evaluate the sample results, then the effect is

7 APPENDIX 10B More-Advanced Statistical Sampling Concepts for Tests of Controls and Tests of Balances 10B-7 EXHIBIT 10B 2 Acceptance Curves for Monetary-Unit Sampling MUS curve D Probability of accepting book value MUS curve A MUS curve C MUS curve B MM 0.50MM 0.75MM 1.00MM 1.25MM 1.5MM Efficiency risk range Effectiveness risk range to reduce efficiency risk over its entire range. This is illustrated in Exhibit 10B 1. The auditor gets something for the increased sample size (increased work), and since confidence level (and, therefore, effectiveness risk, under the negative approach) is held fixed, the increased sample size reduces efficiency risk over most of its range. The calculations are illustrated in the Application Case in Chapter 10. Sometimes, a combination of the two approaches is used to compute the sample size. More-rigorous methods are available for controlling efficiency risk for an explicit number of expected errors, but we will not cover these methods here, because the additional complexity does not change the nature of the basic judgments that must be made and the necessary cost-benefit trade-offs that are required. The concepts of effectiveness risk, efficiency risk, and their relationship to confidence level described here apply with some modification to all statistical procedures. In this sense they are very general. Auditors have developed their own terminology for the risks associated with different audit procedures. As we review various statistical procedures, we will identify new risk terms associated with these procedures, but we will see that for the most part these risks relate to the more important sampling risk in auditing, that of effectiveness risk. It should also be noted that these acceptance curves are not the same as probability distributions that you may be familiar with from your statistics courses. Probability distributions measure the probability of various risks by calculating areas under the curve (e.g., the normal probability distribution). The acceptance curves that we have discussed here represent the probabilities as the vertical distance from the curve to the horizontal axis (representing the probability of one or zero, respectively). Acceptance curves make it easier to visualize the sampling risks for different amounts of errors. Acceptance curves are a type of what statisticians call power curve, where the shape of the power curve varies, like the confidence level, with the null hypothesis. Finally, to better appreciate how uncertainty about sampling risk can be eliminated by 100% testing, take a close look at Exhibit 10B 2. In particular notice the rectangle in that figure. Under 100% sampling

8 10B-8 PART 2 Basic Auditing Concepts and Techniques the vertical part of the rectangle drops at exactly 1.00MM. This mean that even one penny above MM the probability of acceptance drops to zero, meaning the effectiveness risk is zero at MM. The rectangle also means that just to the left of (i.e., below) MM the probability of acceptance is exactly one (meaning there is no efficiency risk). Thus, with no sampling risk the auditor always accepts the recorded amount at or below MM and rejects just above MM. In other words, the auditor can perfectly distinguish between the material and immaterial amount of error with 100% testing. This is something one would expect with perfect knowledge of the population. But, logically, the perfect rectangular acceptance curve occurs only with the negative testing approach to hypothesis testing. Under the positive approach to hypothesis testing, perfect knowledge represented by 100% testing would mean the auditor accepts only the zero errors null hypothesis and rejects all non-zero errors. This follows from the different logics underlying the two testing approaches and illustrates that the negative approach is more consistent with the logic of auditing. Part II: Test of Controls with Attribute Sampling Risk and Rate Quantifications The quantification of sampling risk is an exercise of professional judgment. When using statistical sampling methods, auditors must quantify the two risks of decision error. The risk of assessing control risk too low is generally considered more important than the risk of assessing control risk too high. Auditors must also exercise professional judgment to determine the extent of deviation allowable (tolerable rate) in assessing control risk. Tolerable Deviation Rate: A Professional Judgment Auditors should have an idea about the correspondence of rates of deviation in the population with control risk assessments. Perfect control compliance is not necessary, so the question is what rate of deviation in the population signals control risk of 10%, 20%, or 30%, and so forth, up to 100%? To answer this question, you need to relate material misstatement of an account to a tolerable deviation rate for the control procedures affecting that account. Assume that a material dollar misstatement of $30,000 is used in planning the audit of accounts receivable for possible overstatement. This amount is relevant to the audit of control over sales transactions because uncorrected errors in sales transactions misstate the financial statements by remaining uncorrected in the accounts receivable balance. In other words, we test the controls over sales transaction processing in order to determine the control risk relevant to our audit of the accounts receivable balance. Therefore, if sales transactions are in error by $30,000 or more, the accounts receivable balance may be materially misstated. However, a deviation in a sales transaction (e.g., one unsupported by shipping documents) does not necessarily mean the transaction amount is totally in error. After all, missing paperwork may be the only problem. Perhaps a better example is a mathematical accuracy deviation; if a sales invoice is computed incorrectly to charge the customer $2,000 instead of $1,800, there is a 100% control deviation (the inaccuracy), but it does not describe a 100% dollar error. Therefore, more than $30,000 in sales transactions can be exposed to control deviation without generating a $30,000 error in the sales and the accounts receivable balances. This exposure is sometimes called the smoke/fire concept, meaning that there can be more exposure to error (smoke) than actual error (fire), just as in a conflagration.

9 APPENDIX 10B More-Advanced Statistical Sampling Concepts for Tests of Controls and Tests of Balances 10B-9 Smoke/fire thinking produces a multiplier to apply to the tolerable dollar misstatement assigned to the account balance, $30,000 in our example. We know that a multiplier of 1 is not reasonable (i.e., $30,000 on invoices with deviations produces $30,000 of misstatement in the accounts receivable), and a multiplier of 100 or 200 is probably also not reasonable (too large). Some auditors say a multiplier of 3 is reasonable, having no other basis than a mild conservatism. Some public accounting firms have sampling policies with implicit multipliers that range from 3 to 14. A multiplier of 3 has the practical effect of producing the conclusion that $90,000 on sales invoices could be exposed to control deviations. If the total gross sales on all invoices is $8.5 million, the implied tolerable deviation rate is $90,000/$8.5 million = , or approximately 1%. This 1% tolerable deviation rate now represents a theoretical anchor. It is the deviation rate that marks low control risk (say, 0.05 control risk). When this tolerable deviation rate seems too low for practical audit work, auditors can accept a higher tolerable rate. The only thing that happens is that auditors are implicitly saying that a higher control risk is ultimately satisfactory for the audit of the account balance. Higher tolerable rates signal greater control risk in this scheme of thinking, like the example in Exhibit 10B 3. The point in this demonstration of smoke/fire and tolerable-rate thinking is to show that tolerable rate is a decision criterion that helps auditors assess a control risk. The association of tolerable rates with different control risks is the important point. To achieve a low control risk assessment (e.g., 10%), the sample size of transactions will be very large, but the sample size required to obtain a moderate control risk assessment (e.g., 50%) can be much smaller. Therefore, sample size varies inversely with the tolerable rate the lower the rate (and the lower the desired control risk assessment ), the larger the sample size. Some auditors express the tolerable rate as a number (necessary for statistical calculations of sample size), while others do not put a number on it. The audit strategy is to think about the audit plan, including a consideration of the sample size of balances the team wants to audit. For example, Exhibit 10B 4, column 3, shows various numbers of customer accounts. Suppose the audit manager plans to select 107 for confirmation and other procedures. This decision suggests that control risk needs to be as good as 40% (Exhibit 10B 4, column 1), and the tolerable rate for this assessment is 8% (see Exhibit 10B 3). Thus, the auditors need to select a sample of sales transactions for test of controls audit sufficient to justify a decision about an 8% tolerable rate. EXHIBIT 10B 3 Illustrative Control Risk and Tolerable Rate Relationships TOLERABLE RATE CONTROL RISK CONTROL RISK 1% (anchor) Low control risk Moderate control risk Control risk slightly below maximum Maximum control risk Note: The tolerable rate increases 1 percentage point for each additional 0.05 control risk in this example.

10 10B-10 PART 2 Basic Auditing Concepts and Techniques EXHIBIT 10B 4 Control Risk Influence on Substantive Balance-Audit Sample Size CONTROL RISK CATEGORIES (1) POSSIBLE CONTROL RISK ASSESSMENTS (CR) (2) RELATED RISK OF INCORRECT ACCEPTANCE (RIA)* (3) NUMBER OF BALANCE ITEMS TO SELECT FOR SUBSTANTIVE AUDIT Low control risk Moderate control risk Control risk slightly below the maximum Maximum control risk *Assuming AR = 0.05, IR = 1.0, AP = 1.0. Therefore, RIA = 0.05/(1.0 CR 1.0). Risk of Erroneous Control Risk Assessments: Another Professional Judgment Assessing control risk too low causes auditors to rely on control too much (overreliance) and audit the related account balances less than is necessary. The risk in risk of assessing control risk too low relates to the effect of the erroneous control evaluation. This effect is produced in the substantive audit by influencing the sample size for auditing the account balances related to the controls being evaluated. Internal control is evaluated, and control risk is assessed on the environment, the accounting system, and the client control procedures related to particular account balances. For example, auditors will evaluate control over the processing of sales and cash receipts transactions because these are the transactions that produce the debits and credits to the customer accounts receivable. Some other examples are shown in Exhibit 10B 5. The ultimate purpose of the control risk assessment is to decide how much work to do when auditing the general ledger accounts for example, cash accounts receivable, inventory, sales revenue, and expenses. The question of how much work relates directly to the sample size of the general ledger account details to audit for example, how many bank accounts to reconcile, how many customer accounts receivable to confirm, and how many inventory items to count and recalculate for correct costing. The control risk assessment provides supporting information for the balance-audit work. We will proceed, using the audit of accounts receivable as an occasional example. When planning the audit of the accounts receivable balance, auditors make judgments and estimates of the overall risk of failing to detect material misstatements in the balance (AR, audit risk, related to the receivables audit), the probability that errors entered the accounts (IR, inherent risk), and the effectiveness of their analytical procedures for detecting material errors in the receivables (APR, analytical procedures risk). At this stage, the remaining elements of the risk model are the control risk (CR) and the substantive sample risk of incorrect acceptance (RIA). The internal control evaluation task is directed at assessing the control risk, and the risk of incorrect acceptance is then derived using the expanded risk model RIA = AR/(IR CR APR). As the acronym indicates, RIA is the same as effectiveness risk, discussed earlier and in Chapter 10.

11 APPENDIX 10B More-Advanced Statistical Sampling Concepts for Tests of Controls and Tests of Balances 10B-11 EXHIBIT 10B 5 Examples of Classes of Transactions Flowing into General Ledger Balances Classes of transactions: Test of controls populations Transaction processing General ledger accounts: Substantive balance auditing Cash payments Cash receipts Credit sales Purchases Internal control structure Environment Accounting system Control procedures Cash Accounts receivable Sales revenue Accounts payable Inventory Expenses Since control risk is the probability that the client s controls will fail to detect material misstatements, provided any enter the accounting system in the first place, control risk itself can take on values ranging from very low probability (say, 0.10) to maximum probability (1.0). Using the audit risk model (AuG-7: Applying Materiality and Audit Risk Concepts in Conducting an Audit), there exists a risk of incorrect acceptance (also known in the literature as test of details risk ) for every possible control risk assessment. These risks of incorrect acceptance affect the sample sizes for the substantive audit work on the account balances. Exhibit 10B 4 shows a range of possible control risk assessments in column 1. To their left are some labels commonly used in public accounting practice. (Public accounting firms deal with a few control risk categories instead of a full range of control risk probabilities. 1 ) Column 2 contains the risks of incorrect acceptance (RIA) derived from the audit risk model for the balance-audit substantive sample, and column 3 shows the substantive sample sizes based on these risks of incorrect acceptance. The sample sizes in column 3 are the substantive balance-audit sample sizes (e.g., number of customer accounts, number of inventory items), not the test of controls samples. (The actual calculation of these substantive sample sizes is explained later in this appendix.) These relationships are evident in Exhibit 10B 4: (1) For higher control risk, the related risk of incorrect acceptance is lower; (2) for lower risks of incorrect acceptance, the substantive samples are larger; and (3) therefore, the higher the control risk, the larger the substantive sample size required for the audit of the related balance sheet account. These are the relationships suggested by the second examination standard that is, the control structure and the assessment of control risk need to be understood to plan the nature, timing, and extent of substantive tests to be performed. Erroneous decisions leading to assessing control risk too high should also be avoided in the interest of audit efficiency. For the following explanation, you need to be introduced to the concept of the UEL. The UEL is a statistical estimate of the population deviation rate computed from the test of controls sample evidence. It consists of the actual sample deviation rate (number of deviations found in the sample divided by the test of controls sample size) plus a statistical allowance for sampling error. The UEL is similar to the upper confidence limit of a statistical confidence interval. (You probably studied confidence intervals in your statistics course.) It is used in statistical evaluation of test of controls sample results as the estimate of the population deviation rate. It is compared with the tolerable deviation rate when auditors assess the control risk.

12 10B-12 PART 2 Basic Auditing Concepts and Techniques If the number of deviations in a test of controls sample causes the UEL to be higher than the tolerable deviation rate, when the population deviation rate is actually equal to or lower than the tolerable deviation rate, the auditors may assess control risk too high. This causes them to perform more-substantive audit work on the related account balance than they would have performed had they obtained better information about the control risk. Conversely, if achieved sample UEL is too low (specifically, below tolerable error), the auditor may assess control risk too low. This causes the auditor to do less-substantive audit work on the related account balance than would have been performed had he or she obtained better information on the control risk. This is a much more serious risk for the auditor because it ultimately relates to audit effectiveness. Note also that since this risk adversely affects the auditor s ability to detect material misstatement, assessing control risk too low is the effectiveness risk associated with tests of control. Public accounting firms frequently use a risk of assessing control risk too low of 10%. This is a rather arbitrary policy that eases the burden of the number of judgments auditors need to make. The implication is that auditors are willing to take a 1 in 10 chance of assessing control risk too low and suffering the consequences of auditing a substantive sample smaller than they would have audited had the decision error not been made. However, less arbitrary methods can be devised that link control risk misassessments to the extent of substantive testing of details. These refined methods are not discussed here. Review Checkpoints 10B-1 If inherent risk (IR) is assessed as 0.90 and the detection risk (DR) implicit in an audit plan is 0.10, what audit risk (AR) is implied when the assessed level of control risk is each of 0.10, 0.50, 0.70, 0.90, and 1.0? 10B-2 What general considerations are important when an auditor decides on an acceptable risk of assessing control too high? 10B-3 What general considerations are important when an auditor decides on an acceptable risk of assessing control risk too low? 10B-4 What is the probability of finding one or more deviations in a sample of 100 if the population deviation rate is actually 2%? 10B-5 What is the probability of finding one or more deviations in a sample of 100 units if the deviation rate in the population is only 0.5%? 10B-6 What is the connection between possible assessments of control risk and a judgment about tolerable rate, both considered prior to performing test of controls audit procedures? 10B-7 What is the connection between material dollar misstatement assigned for the substantive audit of a balance and tolerable deviation rate used in a test of controls sample? 10B-8 What professional judgment and estimation decisions must be made by auditors when applying statistical sampling in test of controls audit work? Sample Size Determination As noted in the introduction to this appendix, we will use the simplest formulas and tables to plan and evaluate statistical sampling in auditing. These formulas are based on MUS theory. One of the advantages of MUS is that the same formula and tables can be used for planning sample sizes for tests of

13 APPENDIX 10B More-Advanced Statistical Sampling Concepts for Tests of Controls and Tests of Balances 10B-13 controls (attribute sampling) and substantive tests of details. The key formula is R = np. Solving for n in this formula, n = R/P, yields the sample size that would be used with MUS. In this formula, R is the (effectiveness) risk factor or, equivalently, the confidence level factor; P is the precision; and n is the sample size. This same formula is used for sample planning and evaluation if the formula is used to solve for n, then it is being used for sample planning; if the formula is used to solve for P, then it is being used for sample evaluation. The risk level factor, R, is unique for each combination of risk level (or, equivalently, confidence level) and number of errors. Since we are using the negative approach, confidence level = 1 effectiveness risk. The less-serious efficiency risks are frequently controlled indirectly in practice through the number of errors that are considered acceptable. For a given RIA and MM, the greater the number of errors to be considered acceptable by a sample, the greater the sample size required (and, as discussed earlier, the lower the efficiency risk). In case you are interested, R is the mean of a Poisson distribution with a Poisson process that captures the essence of the attributes sampling distribution underlying MUS theory. Here we are using the Poisson distribution to approximate the binomial or hypergeometric distributions that underlie the attribute sampling theory that forms the basis for statistical tests of controls. This approximation is quite good for most audit applications, so we can exploit the simplifications possible with the Poisson distribution. P stands for precision and is referred to as planned precision at the sample determination stage. In tests of controls, P is tolerable error or some fraction of it at the sample determination stage. P is also referred to as the planned UEL at the sample determination stage. The auditor must use professional judgment in specifying the number of errors, RIA, and P in planning sample sizes for tests of controls using the formula and the R value tables at the back of this appendix. Note that the R value tables are based on effectiveness risk values = 1 confidence levels. To illustrate the calculation of sample size for tests of controls, assume the following: 1. Your risk of assessing control risk too low is 1%. That is, effectiveness risk for tests of controls is 1%. 2. Your tolerable rate is 6%. That is, the P value in the formula is 6% = Your expected population deviation rate is zero, so we expect zero errors in the sample. 4. Your sample size is 77 calculated using the formula R = np and the R value tables as follows (to be conservative, always round up to the nearest integer): Sample size = n = R P = Poisson risk factor for zero errors and effectiveness risk of 0.01 tolerable deviation rate = = 77 The sample sizes using the formula are reliable in that they closely approximate but do not understate the sample sizes necessary to achieve audit objectives. (This is because the Poisson is a conservative approximation of the binomial and hypergeometric distributions on which attribute sampling is based.) However, since it is much easier to work with the Poisson table of R values, we will base all our calculations on the R value tables. We can now also illustrate control of efficiency risk. If we wish to reduce efficiency risk, one method is to use a higher number of errors that is, let errors = 1 in the R value tables and all else in the illustration remain unchanged. Then sample size (n) = 6.64/0.06 = 111. Be sure you can confirm this using the formula and the R value tables. Since effectiveness risk is still the same (1%) and so is tolerable deviation rate (6%), what we get for the increased sample size is a reduction in efficiency risk (as illustrated in Exhibit 10B 2). The auditor can make this efficiency risk control explicit by using more-complex formulas, but most firms follow a policy of controlling efficiency risk indirectly, based on subjectively estimating the number of errors or by using a planned precision P that is less than the tolerable deviation rate.

14 10B-14 PART 2 Basic Auditing Concepts and Techniques The effect of using a lower planned precision in sample determination can also be illustrated. Let us assume the auditor uses a planned precision of 3% rather than the tolerable error rate of 6%. Now we get a sample size n = 4.61/0.03 = 154. Again, effectiveness risk is 1% and tolerable error rate is still 6% (it will be used in the sample evaluation decision rule for deciding if the sample error rate is acceptable), so that what the auditor obtains with the increased sample size is, again, a reduction in efficiency risk. A common way to reduce the planned precision from what is material or tolerable is to subtract the expected error rate from the material or tolerable error rate. For example, if the tolerable rate is 0.06 and the expected error rate is 0.03, then planned precision is = This is the main reason for considering the expected error rate in the audit under MUS. The expected error rate reduces the planned precision, thereby increasing the sample size and reducing the efficiency risks. This is true for either tests of controls or substantive tests of balances. Since sample size can be increased to virtually any amount, most audit firms put a restriction on planned precision by not letting it get below one half the tolerable deviation rate. Review Checkpoints 10B-9 What facts, estimates, and judgments do you need to figure a test of controls sample size using the R value tables? What other relevant judgment is not used? 10B-10 Test yourself to see whether you can get the sample size of 87 from the R value table with these specifications: effectiveness risk = 5%, tolerable deviation rate = 9%, expected errors = 3. 10B-11 What facts, estimates, and judgments do you need to figure a test of controls sample size using the R value tables and Poisson risk factors? What other relevant judgment is not used? 10B-12 Test yourself to see whether you can get the sample size of 70 using the Poisson risk factor equation with these specifications: effectiveness risk = 5%, tolerable deviation rate = 9%, expected errors = 2. More about Defining Populations: Stratification Auditors can be flexible when defining populations. Accounting populations are often skewed, meaning that much of the dollar value in a population is in a small number of population units. For example, the 80/20 rule is that 80% of the value tends to be in 20% of the units. Many inventory and accounts receivable populations have this skewness. Sales invoice, cash receipt, and cash payment populations may be skewed, but usually not as much as inventory and receivables balances. Theoretically, a company s control procedures should apply to small-dollar as well as to large-dollar transactions. Nevertheless, many auditors believe that evidential matter is better when more dollars are covered in the test of controls part of the audit. This inclination can be accommodated in a sampling plan by subdividing (stratifying) a population according to a relevant characteristic of interest. For example, sales transactions might be subdivided into foreign and domestic, accounts receivable might be subdivided into sets of customers with balances of $5,000 or more and those with smaller balances, and payroll transactions might be subdivided into supervisory (salaried) and hourly payroll. Nothing is wrong with this kind of stratification. However, you must remember that (1) an audit conclusion based on a sample applies only to the population from which the sample was drawn and (2) the sample should be representative random for statistical sampling. If 10,000 invoices represent 2,000 foreign sales and 8,000 domestic sales, and you want to subdivide the population this way, you will have two