Anesthesia and Pain Management Drug Cost Reduction while. Maintaining Adequate Patient Care

Transcription

1 Anesthesia and Pain Management Drug Cost Reduction while Maintaining Adequate Patient Care BY JOSHUA ADAM REESE B.S., University of Illinois at Chicago, 2011 THESIS Submitted as partial fulfillment of the requirements for the degree of Master of Science in Industrial Engineering in the Graduate College of the University of Illinois at Chicago, 2013 Chicago, Illinois Defense Committee: Lin Li, PhD, Chair and Advisor Houshang Darabi, PhD Guy Edelman, Anesthesiology

2 This thesis is dedicated to my parents who instilled in me an exceptional work ethic which has allowed me to achieve the academic goals I set for myself. Without their high expectations I would have never accomplished this. This thesis is also dedicated to my fiancé who has strengthened me and allowed me to become the man I am today. ii

3 ACKNOWLEDGMENTS I would like to thank my advisor, Dr. Lin Li, for providing me the opportunity to achieve my thesis based masters degree. Without his support and guidance, this achievement would not have been possible. At times, Dr. Li saw capabilities in me I did not know I had; hence, I have grown during my time working with him and I will be forever grateful for that. He also ensured I would be successful outside of academia, even when I had questioned myself; so not only have I grown, I am graduating with a higher sense of confidence and pride. I also thank Dr. Houshang Darabi and Dr. Guy Edelman for collaborating with me in this effort. Without the joining of the College of Engineering and the College of Medicine, my research would not have been possible. I feel this initial collaborative effort may create a potential new direction for research in the College of Engineering and the College of Medicine at the University of Illinois at Chicago. I thank Dr. Mehmet Ozcan and Dr. Verna Baughman for their valuable insight in determining desirable research areas in healthcare. Without their insight, the first publication would have never been completed and for this I am truly thankful. Lastly, I thank Dr. Yong Wang, Mr. Zeyi Sun, Mr. Erik Osland, Mr. Chongye Wang, Mr. Haoxiang Yang, Mr. Zhichao Zhou, Ms. Mayela Fernandez, and Mr. Adres Bego for creating a friendly atmosphere in the lab and for their help in my times of need. I will be graduating with these friends I would have never made if it weren t for the opportunity to work in Dr. Li s research lab. iii

4 TABLE OF CONTENTS CHAPTER PAGES INTRODUCTION... 1 BISPECTRAL INDEX... 6 Background... 6 Equipment... 7 Software... 9 Data Preparation... 9 Methods... 9 Linear Regression and Auto Regressive Moving Average... 9 Parameters... 9 F Test Parameters General ARMA Procedure One-Minute Interval Modeling Five-Minute Interval Modeling Moving Average Parameters Model Moving Average Forecasting Model Comparison Discussion Limitations PAIN MANAGEMENT Background Method Model Parameters Algorithm Constraint Discussion User Interface Sensitivity Analysis Synergistic Analysis Additive Analysis Antagonistic Analysis Limitations and Assumptions CONCLUSIONS Bispectral Index Intravenous Pain Management iv

5 FUTURE WORK Bispectral Index Intravenous Pain Management APPENDIX REFERENCES VITAE v

6 LIST OF TABLES TABLE TABLE I TABLE II TABLE III TABLE IV TABLE V TABLE VI TABLE VII TABLE VIII TABLE IX TABLE X TABLE XI PAGE POLYNOMIAL FITTING PROCEDURE FOR ONE-MINUTE DATA LINEAR REGRESSION COEFFICIENTS FOR POLYNOMIALS FOR ONE MINUTE INTERVAL F-TEST PROCEDURE FOR ONE MINUTE ARMA MODEL...19 ARMA PARAMETERS FOR ONE MINUTE INTERVAL ONE MINUTE INTERVAL AUTOREGRESSIVE MOVING AVERAGE ROOTS.. 21 PARAMETER VALUES FOR ONE MINUTE INTERVAL...23 ACTUAL AND FORECASTED VALUE COMPARISON FOR ONE MINUTE INTERVAL POLYNOMIAL FITTING FOR THE FIVE MINUTE INTERVAL DATA...27 LINEAR REGRESSION COEFFICIENTS FOR POLYNOMIALS FOR FIVE MINUTE INTERVAL F-TEST PROCEDURE FOR FIVE MINUTE ARMA MODEL..30 ARMA PARAMETERS FOR FIVE MINUTE INTERVAL 31 TABLE XII FIVE MINUTE INTERVAL AUTO REGRESSIVE MOVING AVERAGE ROOTS..32 TABLE XIII TABLE XIV TABLE XV TABLE XVI PARAMETER VALUES FOR FIVE MINUTE INTERVAL.. 33 ACTUAL AND FORECASTED VALUE COMPARISON.35 ACTUAL AND FORECASTED VALUE COMPARISON FOR ONE MINUTE INTERVAL...39 ACTUAL AND FORECASTED VALUE COMPARISON FOR FIVE- MINUTE INTERVAL..40 vi

7 TABLE XVII TABLE XVIII TABLE XIX TABLE XX TABLE XXI DRUG COMBINATION RELATIONSHIPS 45 THE EFFECT OF THE PROBABILITY OF NO RESPONSE TO PAIN ON THE TOTAL COST WITH SYNERGISM THE EFFECT ON DOSE WITH RESPECT TO THE PROBABILITY OF NO RESPONSE TO PAIN WITH SYNERGISM. 54 THE EFFECT OF THE PROBABILITY OF NO RESPONSE TO PAIN ON THE TOTAL COST WITHOUT SYNERGISM THE EFFECT ON DOSE WITH RESPECT TO THE PROBABILITY OF NO RESPONSE TO PAIN WITHOUT SYNERGISM. 55 vii

8 LIST OF FIGURES FIGURE PAGE Figure 1: Projected percentage of gross domestic product for health care and other areas... 2 Figure 2: A single inpatient s cost breakdown per day... 2 Figure 3: Datex-Ohmeda anesthesia monitor courtesy of GE healthcare reference manual... 7 Figure 4: Anesthesia monitor port diagram courtesy of GE healthcare reference manual... 8 Figure 5: Autoregressive moving average modeling procedure Figure 6: Sixth-order polynomial plotted against the one minute interval data Figure 7: Residual plot of the fitted sixth order polynomial Figure 8: Optimized model versus original one minute interval BIS data Figure 9: Autocorrelation function of the residuals for the optimized one minute interval model Figure 10: Fourth order polynomial plotted against the original five minute data Figure 11: Stationary residual plot from the fitted fourth order polynomial Figure 12: Optimized model versus original five minute interval BIS data Figure 13: Autocorrelation function of the residuals for the optimized five minute interval model Figure 14: Moving average plot of fitted values and actual values for one-minute interval Figure 15: Moving average plot of fitted values and actual values for one-minute interval viii

9 LIST OF ABBREVIATIONS ACF ARMA BIS EEG IV MA PACU PTSD RSS Auto Correlation Function Auto Regressive Moving Average Bispectral Index Electroencephalogram Intravenous Moving Average Post Anesthesia Care Unit Post traumatic stress disorder Residuals sum squared ix

10 SUMMARY Drug costs in both anesthesia and cancer pain management contribute to the overall patient cost on an average of 5% of the total healthcare cost. With millions of people receiving surgery and battling cancer pain, the cost of anesthesia and pain management drug delivery becomes a significant way to reduce overall healthcare costs. No means of reducing drug costs currently exists for these two healthcare areas and are the subject of this thesis. Two separate studies are conducted to analyze potential drug cost savings in the healthcare field. During surgery the bispectral index (BIS) represents a patient s depth of anesthesia and is calculated using real time patient electroencephalographic (EEG) data. The BIS is an output on the anesthesia monitor and is a normalized and unit less value between 0 and 100, with 40 to 60 being ideal sedation during surgery. Currently, anesthesiologists lack the timely and accurate information about a patient s BIS status to uphold enhanced steady state maintenance of the BIS. As such, these doctors must overcompensate in dosing because by the time the deviation from optimal sedation occurs, the patient is already distant from the desired level. In this study, BIS prognosis is one method utilized in an attempt to reduce operating room drug costs. Two forecasting methods, auto regressive moving average (ARMA) and moving average (MA), were fit to a single patient s BIS data and the respective forecast accuracies were compared. By applying forecasting models to previous BIS values, a future BIS value from the model output can be displayed on the anesthesia monitor. In doing so, the anesthesiologist has the ability to make possibly smaller dose adjustments without first witnessing the BIS deviation. The purpose of the forecasted BIS value is to provide an early warning output to anesthesiologists and allow them to modify the current dosing regimen by a smaller amount because they will now be x

11 combating smaller deviations. In doing so, an enhanced steady state maintenance program is provided for the anesthesiologists and the patient. Improved steady state maintenance and cost reduction of pain management drugs is the second area of study. Currently, healthcare professionals use passed experiences to develop generic dosing regimens when initiating a pain management procedure for a patient. These generalizations can lead to initial inaccurate doses which result in patient discomfort and inadequate pain management. By forming a non-linear programming model to minimize dosing costs, subject to constraints to assure patient safety, a cost-minimized dosing regimen is developed. The benefit of this study is a suggested cost-minimized dosing regimen may be determined while incorporating patient safety and comfort. xi

12 INTRODUCTION With health care related costs projected to increase beyond 12% of the United States gross domestic product (GDP) by 2051 (Figure 1) [1], methods to maintain patient satisfaction while reducing health care costs are desirable. More specifically, a Salem, Oregon hospital provided a breakdown of an inpatient s per day cost (Figure 2) and determined 5% of its average $2,318 daily cost per inpatient was due to medication [2]. With anesthetics and pain medication being a significant part of those medication costs, there is great potential to produce noteworthy medication cost savings. Patient satisfaction and safety are among the most crucial factors in a hospital s patient care plan. Creating potential cost savings without impeding the hospital s priorities can increase the likelihood of the hospital adopting certain procedures. With the ability to better understand the effects of specific drugs, or combinations of drugs, on the human body, cost-minimization analysis can be performed while still considering the main priorities of a hospital. 1

13 2 Figure 1: Projected percentage of gross domestic product for health care and other areas Figure 2: A single inpatient s cost breakdown per day

14 3 For over forty years pharmacodynamics 1 has been studied in an attempt to mathematically predict and describe the effect drugs have on the human body. One of the current pharmacodynamic equations used today [3] provides the ability to describe the combined effect of two separate drugs on the body. Since pain management can use three medications simultaneously, the equation used in [3] was expanded so it could handle three medications. By using an equation such as the one used by Mertens et al. [3], one may ascertain the probability of no response of an undesired outcome. For this study, pain is the undesired outcome. In the operating room, anesthetics are combined with analgesics 2 to adequately sedate the patient and keep the patient non-responsive to pain, respectively. Calculating the overall effect of multiple drugs using pharmacodynamics allows for a more accurate dose and a better understanding of the dosing requirements needed to accomplish the different goals certain drugs attempt to achieve. In 1996, Aspect Medical Systems introduced the bispectral index (BIS) to the healthcare field. The algorithm used to calculate the BIS, which is owned by Aspect Medical Systems, uses real time patient electroencephalographic (EEG) data to generate a normalized number between 0 and 100 which numerically describes a patient s depth of anesthesia. A value of 0 indicates no brain activity and a BIS value of 100 indicates the patient is not sedated at all. Generally, a BIS value between 40 and 60 deems adequate sedation. The goal of the BIS is to aid the anesthesiologist in determining adequate sedation during surgery. 1 Pharmacodynamics studies the effects drugs have on the human body. 2 Anesthetics are used for sedation and analgesics are used for pain management.

15 4 Since its introduction, studies on the benefits of the BIS have been conducted. Some studies found BIS to reduce the instances of intra-operative awareness [4-7]. Intra-operative awareness can seriously affect patients with symptoms similar to post traumatic stress disorder (PTSD) [8-10]. Even though intra-operative awareness is rare, patients are still concerned about experiencing it and suffering from the complications that may arise from it [11, 12]. Another study found steady-state maintenance of the BIS resulted in lower incidences of post-operative nausea and vomiting [13]. Additionally, other research discusses maintaining a proper steadystate of the patient s BIS can reduce the overall amount of anesthetic administered, the time needed for the recovery of consciousness, and the likelihood of the patient arriving to the postanesthesia care unit (PACU) disoriented from the anesthesia [14]. More rapid emergence times can lead to less time spent in the PACU and further reduce health care costs per patient. Currently, to the best of the author s knowledge, there is no predictive analysis of the BIS. With the proposed benefits of steady-state maintenance, the first part of this research focuses on time series forecasting models to be used to predict future BIS values. Displaying a future BIS value should help anesthesiologists because it will allow the anesthesiologists to see a potential upward or downward shift. Displaying a potential deviation from the desired BIS enables the anesthesiologist to make a possibly smaller corrective dose in order to retain steadystate maintenance of the BIS. This may promote a less volatile steady-state BIS throughout surgery and could also permit drug cost savings through smaller doses. Reducing drug costs in another area of the healthcare field is the analysis for the second part of this research.

16 5 With the number of Americans suffering from cancer at an estimated 13.7 million people in 2012 and over 1.5 million projected to be diagnosed in 2013 [15], cost optimized pain management medication dosing regimens could be very beneficial. An estimated $77.4 billion in direct medical costs was spent on patients battling cancer in 2008 [16]. With approximately 13.7 million people affected by cancer in 2012 and approximately 36% 61% of those people suffering from cancer related pain [17-20], there is great potential for drug cost savings. By utilizing a popular pharmacodynamic model and applying a non-linear optimization technique, a proposed method to minimize drug costs while providing adequate pain relief is analyzed in this research. To the best of the author s knowledge, there is no current research being conducting to minimize steady-state cancer pain and post-surgical pain management drug costs with adequate drug effect and maximum doses still being considered. There are two objectives in this thesis: 1) to aid in the steady-state maintenance of the BIS during surgery to promote patient satisfaction and possibly reduce drug costs, and 2) to develop a non-linear programming model to output a cost-minimized dosing regimen for one, two, or three pain medications while maintaining a desired probability of no response to pain and respecting the maximum allowed dose for each drug used.

17 BISPECTRAL INDEX Background In lieu of the BIS, anesthesiologists used the patient s sympathetic nervous system which provided the necessary information to determine adequate sedation. For example, if an anesthesiologist notices an elevated heart rate and sweat on the patient s forehead, this was a signal the patient was inadequately sedated because the patient s nervous system was displaying signs of stress. Inadequate sedation not only has physical consequences, it can lead to intraoperative awareness. A severe consequence of intra-operative awareness is patients can develop symptoms similar to PTSD [8-10]. Some studies have concluded steady-state maintenance of the BIS has proven to reduce intra-operative incidences [7]. One complication of the BIS is it cannot reduce or account for inter-patient variability. Inter-patient variability arises from the unique physical characteristics each patient has which make it difficult to create generalizations when attempting to link or group patients. Aspect Medical Systems, Inc. decided a BIS value between 40 and 60 is considered adequate sedation during surgery and this has been proven reasonably effective through anesthesiologists experience; however, inter-patient variability creates complications because some patients have an increased (decreased) tolerance to anesthesia and require a larger (smaller) dose to achieve adequate sedation. This can lead the BIS to misrepresent the true patient sedation level even when within the tolerance limit. Despite BIS s inability to account for inter-patient variability, the measure still provides a useful guideline to anesthesiologists. 6

18 7 Many studies discovered potential benefits to the BIS. For instance, steady-state maintenance of the BIS can reduce the overall amount of anesthetic administered, the time to recover consciousness after surgery, and the likelihood of a patient arriving to the PACU disoriented [14]. Decreased overall drug volume dosed during surgery and fast dismissal times from the PACU both can result in cost reductions for a hospital. Equipment The Datex-Ohmeda S/5 anesthesia monitor is one piece of equipment used at the University of Illinois Medical Center (Figure 3). The monitor was connected with an RS232 cable to a laptop computer which was able to transfer the data to the S/5 collection software where it was stored. 3 Figure 3: Datex-Ohmeda anesthesia monitor courtesy of GE healthcare reference manual 3 The data had already been collected prior to the study; therefore, no IRB approval was necessary.

19 8 The location of the RS232 cable port is displayed in Figure 4. The real time data was transferred from the RS232 port on the monitor to the laptop and was recorded using the S/5 collection software. Figure 3 and Figure 4 are courtesy of the GE healthcare reference manual provided with the purchase of the monitor. Figure 4: Anesthesia monitor port diagram courtesy of GE healthcare reference manual

20 9 Software S/5 data collection software was used to collect and store the data from the Datex Ohmeda S/5 anesthesia monitor for real time data collection. The data was stored as a trends.drc file on the laptop after it was successfully transferred to the computer. The operating system used by the laptop is Windows XP. Data Preparation The file type the S/5 collect software saved the data files as was not able to be directly opened in Microsoft Excel. So, the.drc file was opened in Notepad and then cut and pasted into a Microsoft Excel file. Once the data was in Microsoft Excel format, the missing data points were addressed. Missing data points were calculated using the average of the data point directly before and directly after the missing data point location. The average was used for simplicity and was able to be used for all missing data points because there was no case of two or more consecutive missing data points. Methods Linear Regression and Auto Regressive Moving Average Parameters coefficients for the polynomial model Xt residuals of the deterministic model

21 10 Ф coefficient for the auto regressive portion of the ARMA model coefficient for the moving average portion of the ARMA model Yt model output, predicted BIS value t time (minutes), t = 1, 2, 3,... F Test Parameters A1 residual sum of squares (RSS) of the lower order model A0 RSS of the higher order model s number of additional parameters the larger model has compared to the smaller model N total number of data points in the data set r total number of parameters being estimated in the larger model F0 F statistic calculated from the model parameters F F-distribution table value used to determine model adequacy General ARMA Procedure The general ARMA model was chosen for this study and applied to one and five minute intervals for the same data set because these intervals should result in the most accurate forecasts. Each data set is from a single patient during a brain surgery. The time series data collected for this study is non stationary and therefore, must be represented by two components. The first component describes the deterministic portion, trend, of the data and the residuals to

22 11 this model must be stationary, independent and identically distributed in order to permit the ARMA model to be applied. A polynomial model is fit to the data for the deterministic component. This piece of the model will be strictly dependent on time, which is represented by t in (2.1). Xt is the series of residuals of the deterministic model and is fit using the ARMA model depicted as (2.2). Yt is the predicted BIS value determined through the combination of the linear regression and ARMA model. Yt = 0 + 1t + 2t 2 + 3t Xt (2.1) Xt = Ф1Xt 1 + Ф2Xt at 1at 1 2at 2.. (2.2) To determine the number of coefficients necessary to adequately describe the deterministic portion of the data the F test procedure was utilized to ascertain if the additional information provided by the larger model was significant enough to keep the additional parameters. The Chi square distribution fits normal and independent variables and is able to describe the RSS of the models because they are assumed to be normal and independent. Hence, the ratio of the RSSs for the two models in question can be tested using the F-statistic because the F-distribution is defined as the ratio of two independent Chi square distributions. The F test is used as a stopping criterion to avoid over-fitting the model. Most often, additional parameters increase the descriptive nature of a model; however, the F-test determines if the additional descriptive nature justifies the additional parameters. The equation used to calculate the F-statistic is represented in (2.3) and will be compared to the hypothesis testing criteria [21]. F0 = [(A1 A0)/s]/[A0/(N r)] (2.3)

23 12 If F0 from (2.3) is less than the F statistic provided in the F-distribution table value, the model with fewer parameters is determined to be adequate and the procedure is concluded. However, if F0 is greater than the F-value from the table, that is the explanatory power of the additional model parameters is sufficient to justify their inclusion in the model, the larger model is accepted. This procedure continues until additional parameters are not deemed significant. Figure 5 displays the procedure used in this study to properly determine the adequate size of the model [21]. Figure 5: Autoregressive moving average modeling procedure

24 13 Once the number of parameters and the values for those parameters is determined, the deterministic model is fit to the data set. The residuals of the deterministic model must be a stationary stochastic process in order to satisfy the necessary criteria and be fit with an ARMA model. The ARMA (2n, 2n 1) modeling procedure, [21], is used to determine the order and coefficients of the model and is dependent on the residuals being stochastic and stationary. Once again, the F-test is employed to ascertain the adequate number of parameters necessary to properly fit the model to the data. Once the coefficients of the deterministic portion and the parameters for the ARMA model have been determined, they can be combined. The final model is represented by (2.4). Yt = 0 + 1t + 2t 2 + 3t Ф1Xt 1 + Ф2Xt at 1at 1 2at 2.. (2.4) The last step is to use all the estimated parameter values as starting points for an unconstrained joint nonlinear optimization technique. The unconstrained joint nonlinear optimization was conducted using the optimtool function in the MATLAB software. 4 The optimization used the RSS to determine new parameter values and locate a local minimum. This was repeated until the software could not find another set of parameter values that permitted for a reduction in the RSS. One limitation of the nonlinear joint optimization algorithm is it may not find the global minimum. 4 MATLAB R2011b, version , released in 2011

25 14 The final step in the modeling procedure is to determine if the residuals of the integrated model (2.4) are independent of each other. If the residuals of the model are correlated, this violates the assumption of them being independent and identically distributed and the model is flawed. The model would then need to be modified to eliminate the dependency between the residuals by refitting a model which results in independently and identically distributed residuals. A five-minute notification was decided by the anesthesiologists to be a sufficient warning for dose adjustments to be made. One and five minute intervals were decided to be used because the resulting forecasts would be five steps ahead and one step ahead in order to accomplish a five- minute forecast, respectively. If an interval under one-minute was used, this would require the model to predict more steps ahead and would result in a less accurate prediction or a prediction with a very large confidence interval. The further into the future the model must predict, the less accurate the model will be. This is due to the amount of data and changes located between the last data point with which the model is fitted and the desired data point the model must predict. One-Minute Interval Modeling Since the original data was collected every second, every sixtieth data point was used in order to generate a data set with an interval of one-minute. Since the data has a one-minute interval, a five-minute forecast will result in the prediction of five data points. Five-minute and ten-minute forecasts were analyzed for the one-minute data. Table I illustrates the F-statistic procedure and how the sixth degree polynomial was chosen and Table II lists the coefficients for the models which were considered. The first four polynomials were fit to the data and deemed inadequate as well; they were not included in the table for simplicity.

26 15 TABLE I POLYNOMIAL FITTING PROCEDURE FOR ONE-MINUTE DATA F Values Results 5 th Order 6 th Order 6 th Order 7 th Order F5 = > F(1, ) = 3.84 F6 = < F(1, ) = 3.84 F5 Significant, fit 7 th order polynomial F6 is insignificant. 6 th order of polynomial is adequate. TABLE II LINEAR REGRESSION COEFFICIENTS FOR POLYNOMIALS FOR ONE MINUTE INTERVAL 1** 2** β0 β ± ± ± ± ± ± ± ± ± ± ± ± ± ± β * ±5.451*10-5 ±2.98*10-4 ±9.52* ±2.5383*10-3 ±5.65* ± β *10-6 ±5.53* *10-5 ±4.03* *10-6 ±1.8091* *10-4 ±5967* *10-4 ±1.6472*10-4 β4 β5 β6 β * * * *10-6 ±5.63*10-9 ±5.616*10-8 ±3.053*10-7 ±1.225* * * * *10-14 ±7.46*10-10 ±48634* *10-12 ±6.98* *10-12 ±9.799* *10-15 ±7.866*10-15 RSS

27 16 The following equation (2.5) represents the fitted polynomial after using linear regression and the F-test procedure. Figure 6 displays the sixth-order polynomial plotted against the raw BIS data. Yt = t t t t t t 6 + Xt (2.5) Figure 6: Sixth-order polynomial plotted against the one minute interval data The overall trends of the polynomial fit the raw data well and only a few of the data points fall outside the confidence interval. Figure 7 is a plot of the residuals from the fitted polynomial. It illustrates the residuals are stationary and verifies an ARMA model is able to be applied to them. The large variance among the residuals within the first half of the data set, when compared to the

28 17 last half, suggests heteroskedasticity. Heteroskedasticity is the existence of non-constant variance among a data set. The large spikes in the beginning of the residual plot are due to the large fluctuations in the actual data; once the BIS settles, the residuals are reduced. Figure 7: Residual plot of the fitted sixth order polynomial The ARMA (2n, 2n 1) modeling procedure was applied to the residuals plotted in Figure 4. Table III illustrates the F-test procedure and the resulting F-statistics for each model comparison and Table IV lists the chosen model and the other models it was compared with.

29 18 TABLE III F-TEST PROCEDURE FOR ONE MINUTE ARMA MODEL Models F Values Results ARMA(2,1) ARMA(4,3) ARMA(4,3) ARMA(6,5) F = > F (4, ) = 2.37 F = > F (4, ) = 2.37 F is significant, so we should select ARMA (4,3) and keep doing the F-test. F is significant, so we should select ARMA (6, 5) and keep doing the F-test. ARMA(6,5) F is significant, so we should select F = > F (4, ) = 2.37 ARMA(8,7) ARMA (8, 7) and keep doing the F-test. ARMA(8,7) ARMA(10,9) F = > F(4, ) = 2.37 F is significant, so we should select ARMA (10, 9) and keep doing the F-test. ARMA(10,9) ARMA(12,11) ARMA(12,11) ARMA(14,13) ARMA(14,13) ARMA(16,15) ARMA(13,12) ARMA(14,13) ARMA(14,12) ARMA(14,13) F = > F(4, ) = 2.37 F = > F(4, ) = 2.37 F = < F(4, ) = 2.37 F = > F(4, ) = 2.37 F = > F(4, ) = 2.37 F is significant, so we should select ARMA (12, 11) and keep doing the F-test. F is significant, so we should select ARMA (14, 13) and keep doing the F-test. F is insignificant. ARMA (14, 13) is adequate, but the confidence interval for 13 includes zero. So F is significant, so we should select ARMA (14,13) and do the F test with ARMA (14, 12). F is significant, so the ARMA (14, 13) model is the adequate model.

30 19 TABLE IV ARMA PARAMETERS FOR ONE MINUTE INTERVAL Parameters ARMA(13,12) ARMA(14,12) ARMA(14,13) ARMA(16,15) ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ±0.113 RSS

31 20 The equation for the ARMA(14,13) model is represented as (2.6). Xt = Xt Xt Xt Xt Xt Xt Xt Xt Xt Xt Xt Xt Xt Xt 14 + at at at at at at at at at at at at at at 13 (2.6) The above equation is solved for Xt allowing it to be inserted into (2.5) to complete the model. In order to achieve an ideal fit, the roots of the ARMA(14,13) model were analyzed (see Table V). TABLE V ONE MINUTE INTERVAL AUTOREGRESSIVE MOVING AVERAGE ROOTS λ1,2 λ3.4 λ5,6 λ7,8 λ9,10 λ11,12 λ ±0.1836i = ± i = ± i = ± i = ± i = ± i = ± i = After calculating the distances from the origin (bold), λ3 and λ4 are congruent to one, resulting in a possible parsimonious representation of the adequate ARMA(14,13) model. Upon fitting the data to an ARMA(13,12) model and completing the F-test with the ARMA(14,13) model it is realized the ARMA(14,13) model is still adequate. The ARMA(14,13) model is then compared

32 21 to an ARMA(14,12) model and the ARMA(14,13) model is still the model chosen(see Table IV). Comparing the roots to the roots in the table in the Appendix enables the conclusion of no periodicity to be determined. The initial polynomial and ARMA coefficients and corresponding RSS are listed in the second column of TABLE VI. The initial parameter estimates from modeling equations 2.1 and 2.2 were used as initial estimates for equation 2.4 and the MATLAB optimtool was used to find new coefficients for the integrated model based on minimization of the RSS for the final model. The third column in TABLE VI lists the updated coefficients and the corresponding RSS for the optimized model. The RSS located at the bottom of each column in the following table corresponds to the RSS for both the polynomial and the ARMA model combined.

33 22 TABLE VI PARAMETER VALUES FOR ONE MINUTE INTERVAL Initial Coefficients Joint Optimization Coefficients β β β β β β β Ф Ф Ф Ф Ф Ф Ф Ф Ф Ф Ф Ф Ф Ф θ θ θ θ θ θ θ θ θ θ θ θ θ RSS

34 23 A plot of the optimized model versus the original data is illustrated in Figure 8. It can be observed the fitted model sufficiently represents the original data and provides an adequate fit. Figure 8: Optimized model versus original one minute interval BIS data Now that the final model has been determined, the adequacy of this model must be determined. Analysis of the auto correlation function (ACF) is used to determine the adequacy of the model. The two blue lines are used to create a limit in which the lags must remain inside in order to suggest an adequate model. If the lag exceeds the limit, the residual is considered to have dependence which violates the assumption of independent residuals. The lags in the ACF (Figure 9) demonstrate no dependency among the residuals. Lags two and fifteen suggest a possibility of dependence but it can be expected some random cases of possible dependence occur. Lag two suggests dependence among the residuals; however, following the ARMA(2n,

35 24 2n-1) procedure suggests this is the adequate model. Some random cases of dependence can arise, lag fifteen is one of those cases. Figure 9: Autocorrelation function of the residuals for the optimized one minute interval model In order to effectively determine the accuracy of the model, the last ten data points of the original data set were excluded from the fitting procedure. The fifth and tenth data points are used to test the accuracy of the model because these two points represent the five-minute and ten-minute forecasts. The five and ten-minute forecasts are equivalently described as the five and ten-step ahead forecasts, respectively. Table VII illustrates the accuracy of the fitted model by displaying the actual values, model estimates, residuals and percent error for both forecasts.

36 25 TABLE VII ACTUAL AND FORECASTED VALUE COMPARISON FOR ONE MINUTE INTERVAL t Joint Actual Error Percent Optimization Values Error Estimates % % The residuals of the two forecasted values are fairly small. The largest percent error of the two residuals is 2.1%. Five-Minute Interval Modeling For the model fitting of the five minute interval, every three hundredth data point was selected from the original BIS data. Following the same procedure used for the one-minute interval, a polynomial was fit to the original data. Using the F test to determine the adequate number of parameters, a fourth-order polynomial was determined to be sufficient. Table VIII illustrates the F-statistic procedure and how the sixth degree polynomial was chosen. Table IX lists the coefficients for the models which were considered.

37 26 TABLE VIII POLYNOMIAL FITTING FOR THE FIVE MINUTE INTERVAL DATA F Values Results 1 st Order 2 nd Order 2 nd Order 3 rd Order 3 rd Order 4 th Order 4 th Order 5 th Order F1 = < F(1,68) = 3.13 F2 = > F(1,67) = 3.14 F3 = > F(1,66) = 3.14 F4 = < F(1,65) = 3.14 *the coefficients of the 1 st and the 2 nd order polynomials are basically the same, therefore we should keep doing the F test. F2 Significant F3 Significant F4 Insignificant, so 4 th order polynomial is adequate. TABLE IX LINEAR REGRESSION COEFFICIENTS FOR POLYNOMIALS FOR FIVE MINUTE INTERVAL β ± ± ± ± ±6.6 β ± ± ± ± ±1.807 β *10-4 ±3.0371* ± ± ± β * *10-4 ±5.907* ±1.162*10-3 ± β * *10-5 ±8.009*10-6 ±8.152*10-5 β *10-7 ±4.507*10-7 RSS

38 27 The fourth order polynomial is represented as (2.7). Yt = t t t t 4 + Xt (2.7) A graphical representation of the polynomial is presented in Figure 10. The polynomial fits the data well and produces stationary residuals. Figure 11 displays the stationary residuals corresponding to the fourth-order polynomial. Figure 10: Fourth order polynomial plotted against the original five minute data

39 28 Figure 11: Stationary residual plot from the fitted fourth order polynomial Concerning the large spike at time eighteen, this may suggest an outlying data point considering how substantial the residual is when compared to the other residuals in the plot. Patient movement can cause erroneous readings as well, the patient may have been resituated at this point in time which caused the electrode to misread. Again, the residuals at the beginning, times zero through eighteen, are large due to the sporadic nature of the original BIS data. Applying the ARMA(2n, 2n 1) and F test procedures to the residuals results in an ARMA(10,8) model. Table X illustrates the F-test procedure and the resulting F-statistics for each model comparison and Table XI lists the chosen model and the other models it was compared with.

40 29 TABLE X F-TEST PROCEDURE FOR FIVE MINUTE ARMA MODEL Models F Values Results ARMA(2,1) ARMA(4,3) ARMA(4,3) ARMA(6,5) ARMA(6,5) ARMA(8,7) ARMA(8,7) ARMA(10,9) ARMA(10,9) ARMA(12,11) ARMA(9,8) ARMA(10, 9) ARMA(10, 8) ARMA(10, 9) F = > F (4, 63) = 2.4 F = > F (4, 59) = 2.6 F = > F (4, 55) = 2.6 F = > F(4, 51) = 2.6 F = < F(4, 47) = 2.6 F = 3.44 > F(2, 51) = 3.20 F = <F(1, 51) = 4.04 F is significant, so we should select ARMA (4,3) and keep doing F-test. F is significant, so we should select ARMA (6, 5) and keep doing F-test. F is significant, so we should select ARMA (8, 7) and keep doing F-test. F is significant, so we should select ARMA (10, 9) and keep doing F-test. F is insignificant. ARMA (10, 9) 10includes F is significant, so we should select ARMA (10, 9) and keep doing F-test. F is insignificant. The adequate model is ARMA (10, 8).

41 30 TABLE XI ARMA PARAMETERS FOR FIVE-MINUTE INTERVAL Parameters (8,7) (9, 8) (10,8) ( 10, 9) ( 12,11) ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ±0.33 Ф ±0.30 Ф ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ±0.26 RSS The ARMA(10,8) model is represented in (2.8). Xt= Xt Xt Xt Xt Xt Xt Xt Xt Xt Xt 10 + at at at at at at at at at 8 (2.8)

42 31 Analysis of the roots permitted the conclusion of no possibility of a parsimonious representation of the model existing or seasonality. Table XII lists all of the roots from the ARMA (10,8) model and none of them are close to one, this is why no parsimonious representation is possible. When the roots were compared to the roots in the table in the Appendix, it was concluded none of the roots matched and verified periodicity was non-existent. TABLE XII FIVE-MINUTE INTERVAL AUTO REGRESSIVE MOVING AVERAGE ROOTS λ1,2 λ3.4 λ5,6 λ7,8 λ9, ±0.3061i ±0.7716i ±0.7515i = = = ± i = ±0.6891i = Combining the ARMA model and the polynomial generates the model with the initial parameters to be used for the nonlinear joint optimization. Again, MATLAB s optimtool was used to determine the new coefficients which result in the local minimum RSS for the suggested model parameters. Table XIII shows the initial coefficients and the optimized coefficients for the five minute interval model. Figure 12 is a plot of the optimized model versus the original data. Note that starting approximately at time forty six the model never fully responds to the leveling off of the data. One possible implication may describe why this has occurred and it is the possibility of the model being slightly over fit; therefore, it cannot respond quickly enough to the change. The unresponsiveness could come from the model overly relying on the time variable as

43 32 well. The ACF for the five-minute interval (Figure 13) implies independency among the residuals and suggests the model is within the model assumptions. TABLE XIII PARAMETER VALUES FOR FIVE MINUTE INTERVAL Initial Joint Optimization Coefficients Coefficients β β β β β Ф Ф Ф Ф Ф Ф Ф Ф Ф Ф θ θ θ θ θ θ θ θ RSS

44 33 Figure 12: Optimized model versus original five minute interval BIS data Figure 13: Autocorrelation function of the residuals for the optimized five minute interval model The comparison of the predicted values with the last two data points is displayed in TABLE XIV. The one-step ahead forecast is accurate; however, the two-step ahead forecast is not quite as accurate. Since the desired forecast is five minutes, the ten-minute forecast, even though it is inaccurate, is of little concern.

45 34 TABLE XIV ACTUAL AND FORECASTED VALUE COMPARISON FOR FIVE-MINUTE INTERVAL Joint Actual Percent t Optimization Error Values Error Estimates % % Moving Average A general moving average (MA) technique was chosen once it was discerned the amount of historical data points necessary for the linear regression and ARMA process. The MA technique is considerably more simplistic when compared to the ARMA technique. It takes the average of a predetermined number of historical data points in order to generate a forecasted value. It does not develop an overall model which incorporates generated coefficients and it has great opportunity to require less historical data to generate a forecast. This permits the model to start forecasting considerably sooner than the linear regression and ARMA model which needed ten historical data points for the five-minute model and fourteen for the one-minute model. The historical data points, ten and fourteen, necessary before forecasting can be performed result in a fifty minute and fourteen minute delay before the linear regression and ARMA model can start forecasting for the five-minute and one-minute intervals, respectively. The MA technique may only require as few as two historical data observations which results in a ten minute delay for the five-minute data and a two-minute delay for the one-minute data.

46 35 Parameters Yt model output, predicted BIS value t time (minutes), t = 1, 2, 3,... Xt previous BIS values τ time value of estimated parameter m number of previous BIS observations Model The general MA procedure uses the average of a determined number of previous data points and is represented in (2.9). Since the forecast is generated based on the average of the previous observed values, its forecasted values are all the same. This means the two-step ahead is equivalent to the one-step ahead forecasted value because they are both generated from the same previous data points. The potential this model has, and the reason it was analyzed, is based on its ability to require only a few previous data observations to generate a forecast. Y t X t m 1 t (2.9) m Moving Average Forecasting The MA technique was applied to the same data sets as the linear regression and ARMA model. Since the MA technique is highly dependent on previous data points, it is expected the MA technique is better suited for the one-minute interval. As more time elapses between each

47 36 data point, the data points become less dependent on the previous data point. This is why the five-minute interval is assumed to result in less accurate forecasts. Forecasts for both the oneminute and five-minute data intervals were conducted using the Minitab 16 software. Figure 14 illustrates the overall fit of the moving average model using two previous data points on the one-minute interval data. The model appears to fit the data well considering it is only using two previous data points to generate the model fitting. BIS Moving Average Plot for BIS Variable Actual Fits Forecasts 95.0% PI Moving Average Length 2 Accuracy Measures MAPE MAD MSD Index Figure 14: Moving average plot of fitted values and actual values for one-minute interval

48 37 The following table displays the forecasted values and the actual values for the one- minute data interval. It can be noted the overall accuracy of the MA technique is promising. Both forecasts resulted in a percent error of 2.5%. TABLE XV ACTUAL AND FORECASTED VALUE COMPARISON FOR ONE MINUTE INTERVAL t Joint Actual Error Percent Optimization Values Error Estimates % % Figure 15 is a plot of the MA model against the actual data for the five-minute interval data. It appears to lag behind the actual data set s fluctuations. This could be a result of the amount of time elapsed between each data point. The larger amount of time elapsed in between each data point, the less dependent each data point is on the previous one.

49 Moving Average Plot for BIS Variable Actual Fits Forecasts 95.0% PI Moving Average Length 2 BIS 60 Accuracy Measures MAPE MAD MSD Index Figure 15: Moving average plot of fitted values and actual values for one-minute interval The following table displays the forecasted values and the actual values for the one and two-step ahead forecasts. The forecasts corresponding to the five-minute data are not as accurate as the forecasts for the one-minute data. As mentioned before, this is most likely due to the data points being less dependent on each other since there is more time in between each data point. TABLE XVI ACTUAL AND FORECASTED VALUE COMPARISON FOR FIVE-MINUTE INTERVAL Joint Actual Percent t Optimization Error Values Error Estimates % %

50 39 Model Comparison The linear regression model seems to better forecast the BIS data for the five-minute interval for this particular data set; however, the MA model and the ARMA model have equivalent predictive power for the one-minute interval data set in this study. The MA model gains descriptive power with the one-minute data set because the one-minute data points are more dependent on each other than the five-minute data points. This is because more time has passed in between the five-minute data points than the one-minute data points. Discussion The MA technique tends to be more accurate and predominantly used with less stochastic data; however, it may provide a way to avoid the large amount of time necessary for the linear regression and ARMA model to gather enough data points to generate forecasts. Since the MA technique only needs two previous data points to generate a forecast it could be used two or ten minutes, depending on the time interval, after the data has started to be collected. It is conceptualized the MA technique could be used to provide forecasts in the beginning of the surgery and the ARMA model can replace the MA forecast once it has enough previous data points to start generating forecasted values. For example, the five-minute interval ARMA model requires ten previous data points, fifty minutes of data collection, before it can generate forecasts. The MA technique could be used for the interval of ten minutes to fifty minutes and then the ARMA model begins generating forecasts and the MA model is no longer used for that surgery. The accuracy of the MA is not as accurate as the ARMA model; however, in order to fill the idle time at the beginning of the surgery the MA technique seems applicable. The first hour