H.I. Farag 1, A.H. Hashish 2, M.M. Mourad 3, M.M.M. El-Nasharty 1

Transcription

1 A Novel Technique Evaluating RIA Analyses Of (Total & Free) And IGF-1 With Respect to ROC And Hartz s OI Performance To Avoid Unnecessary Radiation Exposure to Cancer Prostate Patients H.I. Farag 1, A.H. Hashish 2, M.M. Mourad 3, M.M.M. El-Nasharty 1 1 Radiotherapy & Nuclear Medicine Dept. National Cancer Institute, Cairo University 2 Physics Dept., Faculty of Science, Mansoura University 3 Urology Dept., Faculty of Medicine, Alazhar University Introduction ROC curve provides a good tool that determines the best threshold between two states of health. Also it evaluates test performance and compare different tests by means of the area under the curve. The lager the area, the greater the performance and the higher the accuracy of the test. When the area under the curve is less than 0.5, previous works (3) suggested to reverse the criterion of positively from greater than to less than or vise versa. Decision matrix could logically relate the results of a diagnostic test to the clinical or pathologic outcome. Consider the following typical decision matrix, D+ D- T+ TP FP T- FN TN Where: - D- indicate non-diseased group T+ +ve test result group T- -ve test result group TP (True positive) sensitivity FN (False negative) FP (False positive) TN (True negative) Specificity Also, Sensitivity: is the number of diseased subjects that are correctly identified as patients by test results (TP). Specificity: is the number of normal subjects that are correctly identified as non-patients by test results (TN). Methodology I- A new idea is introduced for the above criterion summarized in the following two steps: - 1-In case of very low sensitivity of the test. ROC curve for specificity is drawn by plotting false negative (FN) versus true negative (TN), so we get a better representation to the specificity of the test. 2-In General conditions, the draw of (TP) versus (FP) will restore an area larger than 0.5. At any point on the curve we can get a good estimate of sensitivity & specificity. (1) II-The method of overlap index introduced by was used in addition a more accurate OI is represented. III-A new method for analysis had been developed named as Elnasharty s method, to eliminate the disadvantages of ROC and OI. Elnasharty s Method We introduce a new technique called Elnasharty which is more powerful than ROC and Hartz OI analysis. This technique uses a new parameter, the separation power of the test. The technique (table 6-A) performs according to the following action: 1- Arrange the raw data (Test results) in an ascending or descending manner. 2- Mark each group of subjects by a serial code. 3- Count the percent of each group independently that precede (or ) the current test result. This is done in ascending for normal and in descending order for the other(s) or vise versa.

2 4- At 100% or zero of any group, no subject from that group in the next (or previous) region will be detected. 5- Then, calculate the percent of each group that is not mixed with any subject of the other group. Bayes Theorem: Equations from Bayes theorem for posttest specificity (PTSpec.), post test sensitivity (PTSen.), specificity (Spec. Gain) and sensitivity (Sen. Gain) were used. Results and Discussion It is essential to consider the following disadvantages for sensitivity and specificity: 1- Different normal ranges considered by different laboratories would alter sensitivity and specificity depending on how the normal range is defined (1). 2- When more than one disease affects the test result, the degree of elevation is important, and specificity does not take this into account (1). 3- The difficulty of comparing laboratory tests on the basis of two close numbers for specificity and sensitivity (1). Figs. (1- A - F) show the new idea for ROC curves for F/T and IGF-1, associated with tables (1,ab) to show that there is no change in information. Hartz s Overlap Index (OI): OI overcomes the previous mentioned disadvantages. It evaluates and compares tests by a single statistic with no arbitrary parameters that can be used to quantify the utility of a laboratory test. It is a non-parametric index, which is used to evaluate the test s ability to differentiate between two diseases that cause the test s result to elevate (Hartz A.J., 1984). According to Hartz s hypothesis, while OI of zero indicates complete separation, the value of one indicates complete overlap between the populations. The OI for the utilized analyses is shown in table (2). From overlap index it could be concluded that, total (RIA) is superior to that of ELISA, in case of BPH and cancer, because it has less overlap index. Also, (total, free and free/total) and IGF-1 are not good indicators to differentiate among prostate diseases. This is clearly indicated by the overlap index i.e. all calculated values according to Hartz s equation are nearer to one than to zero. Table (2) shows values of OI and the separation power between BPH & Cancer for each analysis. Test Hartz s OI Separation power of the test by Elnasharty s method T (RIA) T (ELISA) F F/T IGF It is noticed from table (2) that, In case of total, Elnasharty s OI method succeeded to evaluate the real value of overlap between diseases. Although, Hartz s OI results are not very accurate but they gave good estimation. On the other hand, Hartz s OI could not evaluate correctly the real values of Free, F/T or IGF-1. Which reflects that there is a range of overlap that could not be detected by Hartz s OI method. Area Under ROC Curve, Hartz OI and Elnasharty s Method The area under an ROC curve can be obtained indirectly from Hartz OI by the relation, given by (3). Area = 1 (OI/2) (1) Referring to Elnasharty s method, mentioned before, one can get OI value, from Elnasharty s method by the relation,

3 Elnasharty s OI = 1 Separation power (2) It is noticed from Table (3) that OI obtained by Elnasharty s technique is more accurate than that obtained by Hartz, as the value of OI, obtained by Elnasharty s method, showed complete unity with the obtained separation power. In other words, unlike the Hartz s OI, it counts all the real overlap because relation (2) can sense all data. From table (3), it can be concluded that Elnasharty s area estimation is less accurate than that of the OI. When the area is less than 0.5, Elnasharty s estimation becomes closer to the axes area. Contrary to the OI s area of Hartz tends to be closer to the conditions area. The overlap depending on Elnasharty s method is much more precise than the overlap presented by Hartz OI is the important factor to determine the ability of the test to differentiate between two diseases. As shown in table (3), the comparison between Elnasharty s estimated area and the ROC estimated area has been done to confirm the ability of the Elnasharty s method to perform different tasks more simply than the ROC and OI. Also, the variation coefficient of Elnasharty s method is found to be within the acceptable range in most cases, except free, whereas OI of Hartz is far away from the real overlap between diseases, except total. Table (3) shows the comparison between Hartz s and Elnasharty s estimated areas with reference to the ROC area. Test Total RIA Total ELISA Free F/T Hartz s OI Hartz s Area Separation power % Elnasharty s OI Elnasharty s area ROC s area Variation % of Elnasharty * ** * ** -0.95** IGF * ** * ** -4.75** * Conditions ** axes Elnasharty s method simply overcomes disadvantages of sensitivity, specificity, and overlap index. - It determines the normal range that contains the total number of normal subjects. - It determines accurately the best thresholds. These thresholds have the largest possible sensitivity and specificity in the region of aggregation of abnormal subjects. It excludes non-patient subjects with specificity that approaches infinity according to the post test probability, Bayes theorem (2). - The degree of overlap between different diseases, i.e. the more accurate Elnasharty s OI = 1 Separation power of the test. When two tests, methodologies separate the same percent of subjects. This will lead to the same area for both tests. Consequently, one has to determine the more accurate test by another means. Which is, the lesser the number of ties and the lesser the subjects having the same test result, the more accurate the test is. - It needs to be conjugated with the ROC curve only in one case, in which two analyses or methods separate the same percent of subjects from the sample. In such a case, one has to compare ROC curves, area under each curve. The lesser the area, the greater the performance

4 of the test or method and vice versa. This is shown in table (4, a-c) which combine raw data with ROC, posttest probability, sensitivity and specificity s, and Elnasharty s method, that shows the separation power and consequently the accurate OI. Also, area could be estimated with a comparable degree of accuracy to that estimated from Hartz s OI, but unfortunately the area of free could not be estimated accurately. This is illustrated in table (3). Comment on the Hartz s Equations: Hart s derived the followings in 1984 Hartz OI = 1 {[2(T - T)]/ n 1 n 2 } (3), and T = n 1 [(n 1 + n 2 +1)/2] (4) Where, n 1 and n 2 are number of subjects within each group. T: is the sum of ranks of n 1 or n 2 group. Underestimation or overestimation of the real overlap between different groups in the sample is attributed to the dependence of T on number of subjects in each group. Unfortunately, these numbers (n 1 & n 2 ) are independent of the ability of the specified test to differentiate between diseases. The latter relies on the test result for each subject. Also, fixed number of groups in the sample will lead to obtain constant value, of T, for the different tests, which is not the actual situation for different analyses. In other words, it is not logically acceptable that whatever (regardless) the test that the researcher is planning to use, it will always have the same T value. Consequently, it can be concluded that Hartz OI is not strongly related to the ability of the analysis to differentiate between diseases. Comparison between Values Determined by RIA and ELISA Methods This comparison, Fig (2, A- H), revealed that: - ELISA under estimates the threshold value, 2.2 ng/ml. While RIA does not, ng/ml. - ELISA is superior to RIA in separating control from BPH and Cancer groups. - RIA is superior to ELISA in separating BPH from cancer. - RIA separates the same percent of subjects for the entire sample, as ELISA. And RIA has the lesser area. - RIA did not record the same test result for two subjects. While in case of ELISA, many patients had the same test result, also, it had many ties i.e. a normal subject had the same test result as a patient one. Theoretically, if measurements were accurate enough, no two individuals would have the same result on a continuous scale (3). From the previous reasons RIA is superior to ELISA because: -Normal and patient subjects are mixed in only two categories, (a) Screening situation, contains all subjects together (control, BPH, and cancer). (b) Complaining groups i.e. abnormal subjects that are isolated from the previous step. RIA is superior to ELISA for both types. as a Not Adequate Biomarket for Prostate Cancer and BPH: Now, we can say that, is not as good as we think it is. It has low maximum sensitivity and specificity and the two maxima are often not at the same threshold, as shown in table (5 - A).

5 Table (5-A): Maximum sensitivity and specificity s from Total (RIA). Total (RIA) Maximum sensitivi ty Maximum specificity Control & BPH Control & cancer BPH& cancer All groups Table (5-B): Maximum sensitivity and specificity s from Free (RIA). Free Maximum sensitivity Maximum specificity Control & Control & BPH & All groups BPH Cancer Cancer Table (5-C): Maximum sensitivity and specificity s from F/T (RIA). Free/Total Control & BPH Control & Cancer BPH & Cancer All groups Maximum sensitivity Maximum specificity Also, maximum specificity by Total by ELISA is 3.6 folds for all groups. And that of sensitivity is for control and BPH. While the specificity of IGF 1 reaches 7 folds (table 6-B). In some cases, which strongly suggest that, IGF-1 is more specific than when considering the non-cancer group (BPH & control). But it is not that good when it is used to separate the BPH from cancer group as it separates only % only. Bone Scintigraphy: In this work we preferred to use a more apparent phenomenon, the sequence of spread of metastases through the body, instead of the histopathological staging and grading which depends on the personal estimation consequently differs in some cases from a lab. to another. The spread occurs as follows, 0) Free of bone metastases (may be organ confined, or spread in pelvic lymph node and both don t appear in bone scan film). 1) Lumbar vertebrae 2) Proximal femur 3) Pelvis 4) Thoracic vertebrae 5) Ribs 6) Skull 7) Cervical vertebrae Unfortunately, subjects of this study have not enough numbers in each stage that prevents the estimation of thresholds among different stages. However, such thresholds can be determined easily when enough numbers are available in each stage. These thresholds are to be estimated for each patient before being subjected to bone scan. If the patient doesn t exceed the estimated threshold then we can deduce that no further progression has occurred.

6 Comparison between Elnasharty s Technique and other Methods: Table (4, a-c) confirms that all methods gave the same thresholds. Analysis done using the new method, Elnasharty s method, shows that it could overcome disadvantages of sensitivity, specificity, and Hartz s OI. Elnasharty s method is a more powerful method to differentiate among different diseases, regardless of their number, with respect to one analysis. New s from Elnasharty s method: 1- Either normal or patient range that contains 100% of subjects. 2- Ability to determine patients with the highest possible sensitivity, consequently, with the highest possible sensitivity. 3- Ability to determine normal subjects with the highest possible specificity, consequently, with the highest possible specificity, that may approaches infinity i.e. beyond doubt. From 2&3 we are able to diagnose correctly from a blood sample, with no need of Ultrasonography, TRUS, or Biopsy in considering prostate cancer, 55.6 % of all sample. And another 55.6 % of the complaining groups. This means that, a new level of diagnosis of diseases is achieved with a new level of certainty about the presence or absence of disease in certain percent of subjects. Which will greatly reduce the costs of diagnosis of diseases. 4- Develop a more powerful technique, Elnasharty s method that overcomes disadvantages of sensitivity, specificity and OI. 5- Elnasharty s method is able to replace ROC and Hartz s OI in Laboratory medicine. And it is simpler and more accurate. 6- Unlike ROC and Hartz s OI, Elnasharty s method is able to perform on more than two diseases or two states of health, no limits, with the same degree of accuracy. 7- Elnasharty s method is the only technique that detected defects of ELISA method. 8- It is the only method that can tell the presence or absence of a disease in a certain subject as long as the test is able to sense that. Conclusion The new method (Elnasharty s method) can replace ROC and Hartz s OI easily. It was used to diagnosis about 55.6% of subjects through RIA of biomarker analysis only. This will save time and costs for pelvic ultra-sonography, TRUS guided biopsy for a certain percent of investigated subjects depending on the analysis used, which even causes spread of cancer metastases, and also pathologic examination of biopsies for a large number of subjects. The method could define the real normal (or the disease) range that includes 100% of such subjects. It can state the presence or absence of a disease in a certain subject or states the inability of the test to determine this fact if the test result lies in the overlap range. References 1- Hartz A. J. (1984): Overlap Index: An Alternative to Sensitivity and Specificity in Comparing Laboratory Test. Arch. Pathol. Lab. Med. Vol. 108, the Utility of 2- Mc Neil B. J., Keeber Emmett, and Adelstein S. James(1975): Primer on elements of medical decision making. N. Engl. J. Med.; 293: Zweig Mark H. and Campbell Gregory(1993): Receiver operating characteristic(roc) plots: A fundamental evaluation tool in clinical medicine. Clin. Chem., 394: