The use of a pen-size optimization workbook for experiment research design using the Visual Basic for Applications in Excel for poultry 1

Transcription

1 2014 Poultry Science Association, Inc. The use of a pen-size optimization workbook for experiment research design using the Visual Basic for Applications in Excel for poultry 1 2 M. Y. Shim and G. M. Pesti Department of Poultry Science, University of Georgia, Athens Primary Audience: Researchers DESCRIPTION OF PROBLEM A fundamental question for poultry nutrition researchers is how many replicates to use in their experiments. The question is complicated by the choice of how many birds to place in each replicate. Each replicate, the experimental unit, may be individual birds, but more often pens of 5 to more than 1,000 birds are used. Grouping birds into pens generally saves on labor. For instance, having 2 pens of 50 birds each may require less labor and record keeping than 10 pens of 10 birds each. Likewise, the variation in the averages of pens should increase as the number of birds in the pens decreases. That is, a pen of 50 birds should provide a more reliable sample of the genetic diversity in a pen than SUMMARY Researchers often seem to choose the number of birds per pen and replications per treatment somewhat arbitrarily on the basis of cost, availability of animals, housing considerations, convenience, tradition, and so on. Statistical simulation, performed while designing an experiment, will provide a researcher with an objective estimate of the number of birds per pen and replicates needed for an experiment of known Type I and II errors. In most cases, the time for making power calculations should be before, not after, an experiment is conducted. Here we present a Microsoft Excel workbook to explore the statistical and economic ramifications of different combinations of birds per replicate and replicates per treatment for research with poultry. Key words: replication, statistical power, poultry, sample size 2014 J. Appl. Poult. Res. 23 : /japr say, 10 birds. For a constant number of birds in an experiment (and total feed costs), having more birds per pen decreases the total number of pens. Fewer pens means fewer error degrees of freedom and reduced statistical power just because of the nature of F- and t- distributions used to evaluate the results. When evaluating experiments, most researchers are primarily concerned with Type I error (α): the probability that they will declare a difference significant when none really exists. Most researchers are satisfied if they only declare differences to be real when they are not, 1 chance in 20, or P < Poultry nutritionists are more often concerned with another type of error, type ii error (β). this error occurs when we say something is not different when it re- 1 Presented as a part of the informal nutrition Symposium From Research measurements to Application: Bridging the Gap at the Poultry Science Association s annual meeting in San diego, California, July 22 25, Corresponding author: gpesti@uga.edu

2 316 JAPR: Symposium Figure 1. Tool overview. ally is. Poultry nutritionists want to know how much of a nutrient they can add before any significant increase in response ceases to exist, or how much of an alternative ingredient they can add before no significant decrease in response can be detected. Poultry nutritionists rarely determine or consider Type II error. The normal standard is to be correct only 4 times out of 5, or P > Paradoxically, if the chance of committing a Type I error, declaring something different when it is not, is decreased by increasing the critical probability value, the chance of committing a Type II error, not declaring a real difference, is increased [1]. The workbook (Figure 1) using the Visual Basic for Applications in Excel [2] was developed to calculate and simulate statistical and economical results by choosing different numbers of birds per pen and different numbers of replicate pens per treatment on the Type I and II errors of a simple experiment comparing 2 treatments. MATERIALS AND METHODS Pen-size optimization workbook for experiment research design (POWER) for Poultry Experiments is posted on the Poultry Nutrition web page [3] (Poultry Nutritionist s Tool Kit under Poultry Science: Extension and Outreach in the College of Agricultural and Environmental Sciences at the University of Georgia). The Dashboard spreadsheet displays a summary of inputs and outputs. The Simulation (Single run) spreadsheet reports results of a simulated experiment based on the specified inputs. The Simulation (Multiple runs) spreadsheet presents the output summaries from many simulated experiments with a normal distribution assumption of the population mean. The Cost_Estimation spreadsheet calculates expected costs (Experiment Cost.xls) from experiments of with different numbers of birds per pen and different numbers of pens per treatment (replicates per experiment). The TC_vs_DD spreadsheet has tables and charts about relationships between (1) total cost and detectable difference and (2) between birds per pen and detectable difference. The Chart 1 and Chart 2 spreadsheets display the same tables as in the TC_vs_DD spreadsheet, and these were included for printing. The Dashboard spreadsheet contains input, run, and output sections. All inputs should be entered on the Dashboard spreadsheet (Figure

3 Shim and Pesti: INFORMAL NUTRITION SYMPOSIUM 317 Figure 2. Setup inputs. 2). The population mean, SD, Type I error, and Type II errors are input there. The example values of population mean and population SD are for a flock of female broilers when they were 48-d-old [4]. The number of birds per pen and replications per treatment entered are those that define the experiment that will be simulated. The current costs to conduct experiments also have to be entered. Appropriate historical or expected data should be entered under Feed Consumption Data. A graph is automatically adjusted and the coefficients are used to generate costs for different weights of birds. Lastly, a feed-planning table must be modified. This table is used to allocate feed charges for different times during the birds life. Cost calculations will automatically be adjusted without any manual corrections to the other spreadsheets. For example, if a researcher plans to use starter, grower, finisher, and withdrawal feeds, he or she should fill in the cells with the days that each feed will be fed and its cost. If the researcher does not want to use grower feed, he or she should empty the Grower column and only enter values into the Starter, Finisher, and Withdrawal feed columns. The number of simulations to be run and the mean for a treated group should be entered just above the Run button (Figure 3). Each time you want to simulate a new experiment, just click on the Run button. The results of one simulated experiment are displayed on the Dashboard under Outputs (Figure 4). The Simulation (Single run) spreadsheet shows a simulation to compare 2 treatments with each replication (Figure 5). In the example in Figure 5, each replicate contains 15 birds and each treatment contains 5 replicates. For example, cell B3 contains a randomly generated individual bird weight from a normal distribution with the mean and SD input. The Simulation (Multiple runs) spreadsheet shows multiple simulations (Figure 6). Each row shows one experiment. The cell B9 presents the average weight of one pen of randomly generated birds. The G9 and O9 cells present average weights of 5 pens each for the control and treated groups, respectively. The H9 and P9 cells present the corresponding SD. These cells will change based on the number of replications. This spreadsheet also contains the average mean difference, t-statistic, and P-values based on the number of replications. The last column is used to tabulate the number of individual simulations meeting the rejection rate probability, which is summed at the bottom of the page; it was an assumption for the simulation to see how accurate the result will be based on the input settings. If the mean values for the control and treated

4 318 JAPR: Symposium Figure 3. Run button. groups are identical, then the hypothesis rejection rate should be 5.0%. In the example in Figure 4, it is 5.0%. By clicking on the simulation, it is clear the P-values vary based on the assumption of a normal distribution for the population mean, such as in a Monte-Carlo simulation. The Cost_Estimation spreadsheet contains the calculation details for experimental costs (Figure 7). Every input value was from the Dashboard; values should not be overwritten on this sheet. The TC_vs_DD spreadsheet displays total costs ($ ~$3, ) and detectable differences against various numbers of birds per pen and replications, which range from 120 to 1,920 total birds (Figure 8). This spreadsheet contains the same graph as in the Dashboard, and a graph of birds per pen (5~80) and detectable differences. The Chart 1 and Chart 2 spreadsheets display the detectable difference versus total costs (Figure 9) and the detectable difference versus birds per pen (Figure 10) that are also displayed on the TC_vs_DD spreadsheet, which were included for printing. RESULTS AND DISCUSSION An easy test exists to demonstrate that the simulation model is working properly: When the control and treatments are input with identical means and standard errors, the null hypothesis that no differences in means exist should have a 5% rejection rate (P < 0.05) [5]. The example Simulation (Multiple runs) spreadsheet shows 5% rejection (Figure 6). Similar tests can be demonstrated for Type II error by running the Figure 4. Reading outputs.

5 Shim and Pesti: INFORMAL NUTRITION SYMPOSIUM 319 Figure 5. Simulation (single run) spreadsheet. problem with real differences between the control and treated groups and different levels of Type II error. The POWER analysis can be used to calculate the minimum sample size required so that an effect of a given size is reasonably likely to be detected. In general, the larger the sample size, the smaller the sampling error tends to be. If sample size is too small, gathering the data is not worthwhile because the results tend to be too imprecise to be of much use. However, a point of diminishing returns exists beyond which increasing the sample size provides little benefit. Once the sample size is large enough to produce a reasonable level of accuracy, making it larger simply wastes time and money. Of course each researcher has to decide what they think reasonable is and then accept that a certain percentage of the time their experiment will not yield the correct conclusions. The POWER calculations can be made whenever one performs a statistical test of a hypothesis and one obtains a statistically nonsignificant result. Because of the very large amount of money at risk from declaring no significant differences in an experiment, more information should be presented than the chance of declaring a significant difference when none exists. The detectable difference displayed on the Dashboard sheet is from Zar [6]. Figure 6. Simulation (multiple runs) spreadsheet.

6 320 JAPR: Symposium Figure 7. Cost estimation spreadsheet. The number of replications required depends on the experimental design employed, the desired difference from the mean that one would care to be able to detect, and a specific probability level and type II error. The number of replications may be estimated using procedures described by Zar [6], Cochran and Cox [7], or Berndtson [8]. When it is assumed that the same numbers of birds are used in an experiment, for example, 600 Figure 8. TC_vs_DD spreadsheet.

7 Shim and Pesti: INFORMAL NUTRITION SYMPOSIUM 321 Figure 9. Chart 1 spreadsheet. total birds, the detectable differences decreased from to g by increasing from 2 (300 birds/pen) to 12 (50 birds/pen) replications. This means that more replication numbers generally give better results than more birds per pen. To estimate about 50 g of detectable differences in 12 replications, 15 birds are needed. However, when replications were decreased from 6 to 2, numbers of birds were increased from 40 to 860 birds per pen to estimate about 50 g of detectable difference. With 2 replications per treatment, increasing the number of birds decreased the detectable differences greatly; but with large numbers of birds per pen, the minimum detectable differences were quite large (Figure 8). With 12 replications, the increase in the number of birds did not affect detectable differences as much as with 2 reps, but caused a steep increase in total costs. To have a relatively small detectable difference with reasonable cost, a proper trade-off study is necessary. This program compares only 2 treatments. When treatments differences are added to the program, it will be more powerful. The POWER for Poultry design should be helpful for making the appropriate calculations and decisions. CONCLUSIONS AND APPLICATIONS 1. An Excel (Microsoft) spreadsheet was effectively developed to calculate and simulate statistical and economical results by choosing different numbers of birds per pen and different numbers of replicate pens per treatment on the Type I and Type II errors of a simple experiment comparing 2 treatments. 2. The developed program allows the researcher to determine the minimum sample size required so that one can be

8 322 JAPR: Symposium Figure 10. Chart 2 spreadsheet. reasonably detect an effect of a given size. 3. More replication numbers generally give better results than more birds per pen. 4. With 12 replications, the increase in the number of birds did not affect detectable differences as much as with 2 reps, but caused a steep increase in total costs. REFERENCES AND NOTES 1. Aaron, D. K., and V. W. Hays How many pigs? Statistical power considerations in swine nutrition experiments. J. Anim. Sci. 82:E245 E Microsoft Corp., Redmond, WA. 3. Shim, M. Y., and G. M. Pesti Pen-size optimization workbook of experimental research design (POWER for Poultry). University of Georgia College of Agricultural and Environmental Sciences Extension Bulletin Accessed September Publications/pubDetail.cfm?pk_id= Shim, M. Y., G. M. Pesti, L. Jackson, M. Tahir, and T. D. Pringle How many chickens? Statistical power considerations in experiments with poultry. The University of Georgia, Athens, GA. Unpublished. 5. Pesti, G. M., and M. Y. Shim A spreadsheet to construct power curves and clarify the meaning of the word equivalent in evaluating experiments with poultry. Poult. Sci. 91: Zar, J. H Power of statistical testing: Hypothesis about means. Am. Lab. 13: Cochran, W. G., and G. M. Cox Experimental Designs. 2nd ed. John Wiley & Sons, New York, NY. 8. Berndtson, W. E A simple and reliable method for selecting or assessing the number of replicates for animal experiments. J. Anim. Sci. 69:67 76.