** A. BACKGROUND (EARLY PAPERS AND TEXTBOOKS).**

- J.P. Egan. Signal Detection Theory and ROC Analysis, Academic Press,New York, 1975.
- D.M. Green and J.A. Swets. Signal Detection Theory and Psychophysics. Robert E. Krieger Publishing Co., Huntington,New York, 1974.
- D. Jalihal and L.W. Nolte. Signal detection theory and reconstruction algorithms-performance for images in noise. IEEE Transactions on Biomedical Engineering, 41:501-4, 1994.
- L.B. Lusted. Signal detectability and medical decision-making. Science, 171:1217-1219, 1971.
- N.A. Macmillan and C.D. Creelman. Detection Theory: A User's Guide, Cambridge University Press, Cambridge, 1991.
- B.J. McNeil and S.J. Adelstein. Determining the value of diagnostic and screening tests. Journal of Nuclear Medicine, 17:439-448, 1976.
- B.J. McNeil, E.Keeler, and S.J. Adelstein. Primer on certain elements of medical decision making. New England Journal of Medicine, 293:211-215, 1975.
- W.W. Peterson, T.G. Birdsall, and W.C. Fox. The theory of signal detectibility. Transactions of the IRE Professional Group in Information Theory, PGIT, 2-4:171-212, 1954.
- J.A. Swets and R.M. Pickett. Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. Academic Press,New York, 1982.
- J.A. Swets. Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers. Lawrence Erlbaum Associates, Publishers, Mahwah, New Jersy, 1995.
- M. Treisman and A. Faulkner. The effect of signal probability on the slope of the receiver operating characteristic given by the rating procedure British J. Math. Statist. Psych, 37:199,215,1984.

- C.B. Begg. Statistical methods in medical diagnosis. CRC Critical Reviews in Medical Informatics, 1:1-22, 1986.
- C.B. Begg. Advances in statistical methodology for diagnostic medicine in the 1980's. Statistics in Medicine, 10:1887-1895, 1991.
- G. Champbell. Advances in statistical methodology for the evaluation of diagnostic and laboratory tests. Statistics in Medicine, 13 :499-508,1994.
- R.M. Centor. Signal detectability: The use of ROC curves and their analyses. Medical Decision Making, 11:102-6, 1991.
- L.S. Erdreich and E.T. Lee. Use of relative operating characteristic analysis in epidemiology: A method for dealing with subjective judgement. American Journal of Epidemiology, 114:649-62, 1981.
- D.J. Goodenough, K.Rossmann, and L.B. Lusted. Radiographic applications of receiver operating characteristic (ROC) curves. Radiology, 110:89-95, 1974.
- J.A. Hanley. Receiver operating characteristic (ROC) methodology: The state of the art. Critical Reviews in Diagnostic Imaging, 29:307-35, 1989.
- A.R. Henderson. Assessing test accuracy and its clinical consequences: a primer for receiver operating characteristic curve analysis. [Review] Annals of Clinical Biochemistry,30:521-39, 1993.
- J.K. Hsiao, J.J. Bartko, and W.Z. Potter. Diagnosing diagnoses: Receiver operating characteristic methods and psychiatry. Archives of General Psychiatry, 46:664-7, 1989.
- C.E. Metz. Basic principles of ROC analysis. Seminars In Nuclear Medicine, 8:283-98, 1978.
- C.E. Metz. ROC methodology in radiologic imaging. Investigative Radiology, 21:720-733, 1986.
- C.E. Phelps. The methodologic foundations of studies of the appropriateness of medical care [see comments]. New England Journal of Medicine, 329(17):1241-5, 1993.
- Y.T. van der Schouw, A.L. Verbeek, and S.H.Ruijs. Guidelines for the assessment of new diagnostic tests. Invest Radiol, 30:334-40,1995.
- M. Schulzer. Diagnostic tests: a statistical review. [Review] Muscle and Nerve, 17(7):815-9, 1994.
- J.A. Swets. ROC analysis applied to the evaluation of medical imaging techniques. Investigative Radiology, 14:109-21, 1979.
- J.A. Swets. Measuring the accuracy of diagnostic systems. Science, 240:1285-93, 1988.
- D.A. Turner. An intuitive approach to receiver operating characteristic curve analysis. Journal of Nuclear Medicine, 19:213-20, 1978.

- C.B. Begg and R.A. Greenes. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics, 39:207-215, 1983.
- C.B. Begg. Biases in the assessment of diagnostic tests. Statistics in Medicine, 6:411-423, 1987.
- C.B. Begg. Experimental design of medical imaging trials: Issues and options. Investigative Radiology, 24:934-936, 1989.
- C.B. Begg and B.J. McNeil. Assessment of radiologic tests: Control of bias and other design considerations. Radiology, 167:565-9, 1988.
- K.S. Berbaum, D.D. Dorfman, and E.A. Franken, Jr. Measuring observer performance by ROC analysis: Indications and complications. Investigative Radiology, 24:228-33, 1989.
- G.A. Diamond. ROC steady: A receiver operating characteristic curve that is invariant relative to selection bias. Medical Decision Making, 7:238-43, 1987.
- G.A. Diamond. ROCky III. Medical Decision Making, 7:247-9, 1987.
- G.A. Diamond. Can the discriminant accuracy of a test be determined in the face of selection bias? Medical Decision Making,11:48-56, 1991.
- G.A. Diamond. Scotched on the ROCs. Medical Decision Making, 11:198-200, 1991.
- G.A. Diamond. What is the effect of sampling error on ROC analysis in the face of verification bias? Medical Decision Making, 12:155-6, 1992.
- C.A. Gatsonis and X.H. Zhou. Group sequential designs for comparative ROC studies in diagnostic medicine, Technical report, Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts, 1993.
- W.F. Good, D.Gur, W.H. Straub, and J.H. Feist. Comparing imaging systems by ROC studies: Detection versus interpretation. Investigative Radiology, 24:932-3, 1989.
- R.Gray, C.B. Begg, and R.A. Greenes. Construction of receiver operating characteristic curves when disease verification is subject to selection bias. Medical Decision Making, 4:151-64, 1984.
- R.A. Greenes and C.B. Begg. Assessment of diagnostic technologies: Methodology for unbiased estimation from samples of selectively-verified patients. Investigative Radiology, 20:751-756, 1985.
- D.Gur, J.L. King, H.E. Rockette, C.A. Britton, F.L. Thaete, and R.J. Hoy. Practical issues of experimental ROC analysis: Selection of controls. Investigative Radiology, 25:583-6, 1990.
- D.Gur, H.E. Rockette, W.F. Good, B.S. Slasky, L.A. Cooperstein, W.H. Straub, N.A. Obuchowski, and C.Metz. Effect of observer instruction on ROC study of chest images. Investigative Radiology, 25:230-4, 1990.
- J.A. Hanley and C.B. Begg. Response to ROC steady. Medical Decision Making, 7:244-6, 1987.
- J.A. Hanley. Verification bias and the one-parameter logistic ROC curve-some clarifications. [Comment on Diamond 1991]. Medical Decision Making, 11:203-7, 1991.
- M.G. Hunink and C.B. Begg. Diamond's correction method-a real gem or just cubic zirconium? [Comment on Diamond 1991]. Medical Decision Making, 11:201-3, 1991.
- J.L. King, C.A. Britton, G.D., H.E. Rockette, and P.L. Davis. On the validity of the continuous and discrete confidence rating scales in receiver operating characteristic studies. Investigative Radiology,28:962-963, 1993.
- C.E. Metz. Some practical issues of experimental design and data analysis in radiological ROC studies. Investigative Radiology, 24:234-45, 1989.
- C.E. Metz and J.H. Shen. Gains in accuracy from replicated readings of diagnostic images: Prediction and assessment in terms of ROC analysis. Medical Decision Making, 12:60-75, 1992.
- N.A. Obuchowski. Computing sample size for receiver operating characteristic studies. Investigative Radiology, 29:238-243, 1994.
- D.F. Ransohoff and A.R. Feinstein. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. New England Journal of Medicine, 299:926-930, 1978.
- H.E. Rockette, D.Gur, L.A. Cooperstein, N.A. Obuchowski, J.L. King, C.R. Fuhrman, E.K. Tabor, and C.E. Metz. Effect of two rating formats in multi-disease ROC study of chest images. Investigative Radiology, 25:225-9, 1990.
- H.E. Rockette, D.Gur, and C.E. Metz. The use of continuous and discrete confidence judgments in receiver operating characteristic studies of diagnostic imaging techniques. Investigative Radiology, 27:169-172, 1992.

- I.G. Abrahamson and H.Levitt. Statistical analysis of data from experiments in human signal detection. [Maximum likelihood ROC curve - general location-scale family]. Journal of Mathematical Psychology, 6:391-417, 1969.
- G. Campbell and M.V. Ratnaparkhi. An application of Lomax distributions in receiver operating characteristic (ROC) curve analysis. Comm.Statist. A--Theory Methods :1681-1697,1993.
- D.D. Dorfman and E.Alf, Jr. Maximum likelihood estimation of parameters of signal detection theory-a direct solution.[Binormal ROC curve-dichotomous diagnostic test]. Psychometrika, 33:117-124, 1968.
- D.D. Dorfman and K.S. Berbum. Degeneracy and discrete receiver operating characteristic rating data. Academic Radiology,2:907-915,1995.
- D.D. Dorfman and E.Alf, Jr. Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals-rating method data. [Binormal ROC curve-ordinal data]. Journal of Mathematical Psychology, 6:487-496, 1969.
- D.R. Grey and B.J.T. Morgan. Some aspects of ROC curve-fitting: Normal and logistic models. [Corrections and improvements to Dorfman-Alf; minimum chi-square bilogistic]. Journal of Mathematical Psychology, 9:128-139, 1972.
- J.A. Hanley. The robustness of the ``binormal'' assumptions used in fitting ROC curves. Medical Decision Making, 8:197-203, 1988.
- J.A. Hanley. The use of the 'binormal' model for parametric roc analysis of quantitative diagnostic tests. Statistics in Medicine, 15 : 1575-1585, 1996.
- H.L. Kundel, D.D. Dorfman and K.S. Berbaum. Degeneracy and discrete receiver operating characteristic rating data, 2 :907-15,1995.
- C.E. Metz and X. Pan. "Proper" binormal ROC curves: theory and maximum-likelihood estimation, unpublished manuscript, University of Chicago.
- D. Mossman. Resampling techniques in the analysis of non-binormal ROC data. Med Decis Making , 15:358-66,1995.
- J.C. Ogilvie and C.D. Creelman. Maximum-likelihood estimation of receiver operating characteristic curve parameters. Journal of Mathematical Psychology, 5:377-391, 1968.
- E. Somoza. Eccentric Diagnostic Tests. Medical Decision Making, 16:15-23, 1996.
- J.A. Swets. Form of empirical ROCs in discrimination and diagnostic tasks: Implications for theory and measurement of performance. Psychological Bulletin, 99:181-198, 1986.
- A.N. Tosteson and C.B. Begg. A general regression methodology for ROC curve estimation. Medical Decision Making, 8:204-15, 1988.
- A.N. Tosteson, J. Wittenberg, and C.B. Begg. ROC curve regression analysis: the use of ordinal regression models for diagnostic test assessment. Environmental Health Perspectives, 8:73-8, 1994.
- X.H. Zhou. Testing an underlying assumption on a ROC curve based on rating data. Med Decis Making, 15:276-82,1995.

- R.M. Centor and J.S. Schwartz. An evaluation of methods for estimating the area under the receiver operating characteristic (ROC) curve. Medical Decision Making, 5:149-56, 1985.
- C. Cox. Location-scale cumulative odds models for ordinal data: a generalized non-linear model approach. Statistics in Medicine,14:1191-203, 1995.
- J.A. Hanley and B.J. McNeil. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143:29-36, 1982.
- R.D. Hays. ROC: Estimation of the area under a receiver operating characteristic curve. Appl. Psych. Meas, 14 :208,1990.
- J.Hilden. The area under the ROC curve and its competitors. Medical Decision Making, 11:95-101, 1991.
- D.Katz and B.Foxman. How well do prediction equations predict? Using receiver operating characteristic curves and accuracy curves to compare validity and generalizability. Epidemiology,4(4):319-26, 1993.
- D.K. McClish. Analyzing a portion of the ROC curve. Medical Decision Making, 9:190-5, 1989.
- T.O. Nelson. ROC curves and measures of discrimination accuracy: A reply to Swets. [Comment on Swets 1986a]. Psychological Bulletin, 100:128-132, 1986.
- J.J. Riera-Diaz. The estimation of event related potentials affected by random shifts and scalings. International Journal of Bio-Medical Computing,38:109-20,1995.
- A.J. Simpson and M.J. Fitter. What is the best index of detectability? [Recommends the area under the binormal curve]. Psychological Bulletin, 80:481-488, 1973.
- J.A. Swets. Indices of discrimination or diagnostic accuracy: Their ROCs and implied models. Psychological Bulletin, 99:100-117, 1986.

- A.Agresti. A survey of models for repeated ordered categorical response data. Statistics in Medicine, 8:1209-1224, 1989.
- D.Bamber. The area above the ordinal dominance graph and the area below the receiver operating graph. Journal of Mathematical Psychology, 12:387-415, 1975.
- C.A. Beam. Random-effects models in the receiver operating characteristic curve-based assessment of the effectiveness of diagnostic imaging technology: concepts, approaches, and issues. Academic Radiology, 2: supple: 4-13, 1995.
- C.A. Beam and H.S. Wieand. A statistical method for the comparison of a discrete diagnostic test with several continuous diagnostic tests. Biometrics, 47:907-919, 1991.
- D.A. Block. Comparing two diagnostic tests against the same "gold standard" in the same sample. Biometrics, 53: 73-85, 1997.
- C. Cox. Location-scale cumulative odds models for ordinal data: a generalized non-linear model approach. Statistics in Medicine, 15 :1191-203, 1995.
- E.R. DeLong, W.B. Vernon, and R.R. Bollinger. Sensitivity and specificity of a monitoring test. Biometrics, 41:947-958, 1985.
- E.R. DeLong, D.M. DeLong, and D.L. Clarke-Pearson. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44:837-45, 1988.
- D.D. Dorfman, K.S. Berbaum, and C.E. Metz. Receiver operating characteristic rating analysis: Generalization to the population of readers and patients with the jackknife method. Investigative Radiology, 27:723-31, 1992.
- B. Emir, S.Wieand, J.Q. Su, and S. Cha. Analysis of repeated markers used to predict progression of cancer. Statistics in Medicine, 17: 2563-78, 1998.
- B. Emir, S. Wieand, S.-H. Jung, and Z. Ying. Comparison of diagnostic markers with repeated measurements: A non-parametric ROC curve approach. Staistics in Medicine, 19: 511-23, 2000.
- C.A. Gatsonis. Random-effects models for diagnostic accuracy data. Academic Radiology, 2:suppl:14-21, 1995.
- J.A. Hanley. Alternative approaches to receiver operating characteristic analyses. Radiology, 168:568-70, 1988.
- J.A. Hanley and B.J. McNeil. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology, 148:839-43, 1983.
- R.A. Hilgers. Distribution-free confidence bounds for ROC curves. Methods of Information In Medicine, 30:96-101, 1991.
- F. Hsieh and B.W. Turnbull. Nonparametric and semiparametric estimation of the receiver operating characteristic curve. Annals of Statistics, 24: 25-40, 1996.
- K. Jensen, H.-H. Muller and H. Schafer. Regional confidence bands for ROC curves. Statistics in Medicine, 19: 493-509, 2000.
- V. Kairisto and A. Poola. Software for illustrative presentation of basic clinical characteristics of laboratory tests--GraphROC for Windows. Scandinavian Journal of Clinical & Laboratory Investigation, 222(suppl):43-60, 1995.
- K. Kim. A bivariate cumulative probit regression model for ordered categorical data. Statistics in Medicine, 14 :1341-1352, 1995.
- G.Ma and W.J. Hall. Confidence bands for receiver operating characteristic curves. Medical Decision Making, 13:191-197, 1993.
- D.K. McClish. Comparing the areas under more than two independent ROC curves. Medical Decision Making, 7:149-55, 1987.
- D.K. McClish. Determining a range of false-positive rates for which ROC curves differ. Medical Decision Making, 10:283-7, 1990.
- B.J. McNeil and J.A. Hanley. Statistical approaches to the analysis of receiver operating characteristic (ROC) curves. Medical Decision Making, 4:137-50, 1984.
- A. Moise, B. Clement, M. Raissis and P. Manopoulos. A test for crossing receiver operating characteristic (ROC) curves. Comm. Statist. A -- Theory Methods, 17:1985-2004,1988.
- C.E. Metz. Statistical analysis of ROC data in evaluating diagnostic performance. In D.E. Herbert and R.H. Myers, editors, Multiple regression analysis: Applications in the health sciences, pages 365-384,American Institute of Physics, New York,1986.
- C.E. Metz. Quantification of failure to demonstrate statistical significance: The usefulness of confidence intervals. Investigative Radiology, 28:59-63, 1993.
- C.E. Metz and H.B. Kronman. Statistical significance tests for binormal ROC curves. Journal of Mathematical Psychology, 22:218-243, 1980.
- C.E. Metz, J.H. Shen, and B.A. Herman. New methods for estimating a binormal ROC curve from continuously-distributed test results. Unfinished manuscript; presented at the 1990 Joint Statistical Meetings in Anaheim, California.
- C.E. Metz, P.-L. Wang, and H.B. Kronman. A new approach for testing the significance of differences between ROC curves measured from correlated data. In F.Deconinck, editor, Information Processing in Medical Imaging: Proceedings of the Eighth Conference, 432-445.Martinus Nijhoff Publishers, The Hague, 1984.
- A.Moise, B.Clement, and M.Raissis. A test for crossing receiver operating characteristic (ROC) curves. Communications in Statistics-Theory and Methods, 17:1985-2003, 1988.
- P.A. Murtaugh. ROC curves with multiple marker measurements. Biometrics, 51 :1514-22, 1995.
- N.A. Obuchowski. Multireader, multimodality receiver operating characteristic curve studies: hypothesis testing and sample size estimation using an analysis of variance approach with dependent observations. Academic Radiology, 2: supple: 22-29,1995.
- N.A. Obuchowski Nonparametric analysis of clustered ROC curve data. Unpublished manuscript, 1996 ENAR meeting.
- H.E. Rockette, N.A. Obuchowski, and D.Gur. Nonparametric estimation of degenerate ROC data sets used for comparison of imaging systems. Investigative Radiology, 25:835-7, 1990.
- H.E. Rockette, N.Obuchowski, C.E. Metz, and D.Gur. Statistical issues in ROC curve analysis. Proceedings of the SPIE, 1234:111-119, 1990.
- D.E. Shapiro. The interpretation of diagnostic tests. Statistical Methods in Medical Research, 8: 113-134, 1999.
- E.K. Shultz. Multivariate receiver-operating characteristic curve analysis: prostate cancer screening as an example.[Review] Clinical Chemistry, 41:1248-55, 1995.
- P.J. Smith, T.J. Thompson, M.M. Engelgau and W.H. Herman. A generalized linear model for analysing receiver operating characteristic curves. Statistics in Medicine, 15 :323-33, 1996.
- E. Svensson, and S. Holm. Separation of systematic and random differences in ordinal rating scales. Statistics in Medicine,13:2437-53, 1994.
- M. Swaving, H. Van Houwelingen, F.P. Ottes, T. Steerneman. Statistical comparison of ROC curves from multiple readers. Medical Decision Making, 16 : 143-153, 1996.
- M.L. Thompson and W.Zucchini. On the statistical analysis of ROC curves. Statistics In Medicine, 8:1277-90, 1989.
- A.L. Toledano and C.A. Gatsonis. Regression analysis of correlated receiver operating characteristic data. Academic Radiology,2:supple:30--36, 1995.
- A.Y. Toledano and C.A. Gatsonis. Ordinal regression methodology for roc curves derived from correlated data. Statistics in Medicine,15:1807-26,1996.
- S. Wieand, M.H. Gail, B.R. James, and K.L. James. A family of nonparametric statistics for comparing diagnostic markers with paired or unpaired data. Biometrika , 76:585-592, 1989.
- X.H. Zhou. Testing an underlying assumption on a ROC curve based on rating data. Medical Decision Making,15:276-282,1995.
- X.H. Zhou Testing an underlying assumption on a ROC curve based on rating data. Medical Decision Making, 15 :276-82, 1995.
- X.H. Zhou and C.A. Gatsonis. A smple method for comparing correlated roc curves using incomplete data. Statistics in Medicine, 15 :1687-1693, 1996.
- K.H. Zou, C. M Tempany, J. R. Fielding, and S. G. Silverman. Original smooth receiver operating characteristic curve estimation from continous data: Statistical methods for analyzing the predictive value of spiral CT of ureteral stones. Academic Radiology, 5: 680-687, 1998.
- K.H. Zou, W.J. Hall and D.E.Shapiro. Smooth nonparametric receiver operating characteristic curves for continuous diagnostic tests. Statistics in Medicine, 16: 2143-2156, 1997.
- K.H. Zou and W. J. Hall. Two transformation models for estimating an ROC curve derived from continous data. Journal of Applied Statistics, 26: 621-631, 2000.
- K.H. Zou. Comparison of correlated ROC curves derived from repeated diagnostic test data. Academic Radiology, 8: 225-233, 2001.

- T.A. Alonzo and M.S. Pepe. Using a combination of reference tests to assess the accuracy of a new diagnostic test. Statistics in Medicine, 18: 2987-3003, 1999.
- G.Baker, S. Evaluating a new test using a reference test with estimated sensitivity and specificity. Communication in Statistics, Part A-Theory and Methods, 20:2739-2752, 1991.
- C.B. Begg and C.E. Metz. Consensus diagnoses and ``gold standards''. [Comment on Henkelman, Kay and Bronskill 1990]. Medical Decision Making, 10:29-30, 1990.
- G. Campbell and J.M. Deleo. Fundamentals of fuzzy receiver operating characteristic (ROC) functions. Comput. Sci. Statist.: Proc. 21st Symp. Interface (Kenneth Berk and Linda Malone, eds.), Amer. Statist. Assoc. (Alexandria, VA) :543-548,1989.
- P.Deneef. Evaluating rapid test for streptococcal pharyngitis: The apparent accuracy of a diagnostic test when there are errors in the standard of comparison. Medical Decision Making, 7:92-96, 1987.
- S.V. Faraone and M.T. Tsuang. Measuring diagnostic accuracy in the absence of a "gold standard".[Review] American Journal of Psychiatry, 151(5):650-7, 1994.
- R.A. Greenberg and J.F. Jekel. Some problems in the determination of the false positive and false negative rates of tuberculin tests. American Review of Respiratory Disease, 100:645-650, 1969.
- J.J. Gart and A.A. Buck. Comparison of a screening test and a reference test in epidemiological studies: II. a probabilistic model for the comparison of diagnostic tests. American Journal of Epidemiology, 83:593-602, 1966.
- D.A. Grayson. Statistical diagnosis and the influence of diagnostic error. Biometrics, 43:975-984, 1987.
- W.J. Hall & D.E. Shapiro A receiver operating characteristic curve for subjective probability of disease assessments in the absence of a gold standard (abstract only). Medical Decision Making, 14:430, 1994.
- R.M. Henkelman, I.Kay, and M.J. Bronskill. Receiver operator characteristic (ROC) analysis without truth. Medical Decision Making, 10:24-9, 1990.
- S.L. Hui and S.D. Walter. Estimating the error rates of diagnostic tests. Biometrics, 36:167-171, 1980.
- S.L. Hui and X.H. Zhou. Evaluation of diagnostic tests without gold standards. Statistical methods in medical research, 7: 354-370, 1998.
- H.C. Kraemer. The robustness of common measures of 2X2 association to bias due to misclassifications. American Statistician, 39:286-290, 1985.
- N.J.D. Nagelkerke, V.Fidler, and M.Buwalda. Instrumental variables in the evaluation of diagnostic test procedures when the true disease state is unknown. Statistics in Medicine, 7:739-744, 1988.
- C.E. Phelps and A.Hutson. Estimating diagnostic test accuracy using a `fuzzy' gold standard. Medical Decision Making, 15:44-57, 1995.
- R.M. Poses, R.D. Cebul, and R.M. Centor. Evaluating physicians' probabilistic judgements. Medical Decision Making, 8:233-240, 1988.
- R.M. Poses, C.B. Bekes, R.L. Winkler, W.E. Scott, and F.J. Copare. Are two (inexperienced) heads better than one (experienced) head? Archives of Internal Medicine, 150:1874-1878, 1990.
- M.Staquet, M.Rozencweig, Y.J. Lee, and F.M. Muggia. Methodology for the assessment of new dichotomous diagnostic tests. Journal of Chronic Disease, 34:599-610, 1981.
- V. Torrance-Rynard and S. Walter. Effects of dependent errors in the assessment of diagnostic test performance. Statistics in Medicine, 16: 2157-2175, 1997.
- J.S. Uebersax. Statistical modeling of expert ratings on medical treatment appropriateness. Journal of the American Statistical Association, 88:421-427, 1993.
- J.S. Uebersax and W.M. Grove. Latent class analysis of diagnostic agreement. Statistics in Medicine, 9:559-572, 1990.
- J.S. Uebersax and W.M. Grove. A latent trait finite mixture model for the analysis of rating agreement. Biometrics, 49:823-835, 1993.
- P.M. Vacek. The effect of conditional dependence on the evaluation of diagnostic tests. Biometrics, 41:959-968, 1985.
- P.N. Valenstein. Evaluating diagnostic tests with imperfect standards. American Journal of Clinical Pathology, 93:252-258, 1990.
- S.D. Walter and L.M. Irwig. Estimation of test error rates, disease prevalence, and relative risk from misclassified data. Journal of Clinical Epidemiology, 41:923-38, 1988.
- X.H. Zhou and R. Higgs. Assessing the relative accuracies of two screening tests in the presence of verification bias. Statistics in Medicine, 19: 1697-1705, 2000.
- X.H. Zhou and R. Higgs. COMPROC and CHECKNORM: computer programs for comparing accuracies of diagnostic tests using ROC curves in the presence of verification bias. Computer Methods and Programs in Biomedicine, 57: 179-186,1998.
- X.H. Zhou and C.A. Rodenberg. Estimating a ROC curve in the presence of verification bias. Communication in Statistics, Ther., 27: 635-657.
- X.H. Zhou. Maximum likelihood estimators of sensitivity and specificity corrected for verification bias. Commun. Statist. Theor. Meth, 22: 3177-3198, 1993.
- X.H. Zhou. Effect of verification bias on positive and negative predictive values. Statistics in Medicine, 13: 1737-1745, 1994.
- X.H. Zhou. Nonparametric ML estimate of a ROC area corrected for verification bias. Biometrics, 52: 310-316, 1996.
- X.H. Zhou. Comparing the correlated areas under the ROC curves of two diagnostic tests in the presence of verification bias. Biometrics, 54, 453-470, 1998.
- X.H. Zhou. Comparing accuracies of two screening tests in the presence of verication bias. JRSS, Ser C, 47: 135-147, 1998.
- X.H. Zhou. Correction for verificatin bias in studies of a diagnostic test's accuracy. Statistics Methods in Medical Research, 7: 337-353, 1998.

- C.. Berkey, J.J. Anderson and D.C. Hoaglin. Multiple-outcome meta-analysis of clinical trials. Statistics in Medicine, 15 :537-557, 1996.
- C.S. Berkey, D.C. Hoaglin, F. Mosteller and G.A. Colditz. A random-effects regression model for meta-analysis. Statistics in Medicine, 14 :195-411, 1995.
- W.J. Hall. Efficiency of weighted averages, Technical Report 92/02, Department of Biostatistics, University of Rochester, Rochester, New York, Feb. 1992.
- M.Helfand. Meta-analysis in deriving summary estimates of test performance. Medical Decision Making, 13:182-3, 1993.
- L.Irwig, A.N.A. Tosteson, C.Gatsonis, J.Lau, G.Colditz, T.Chalmers, and F.Mosteller. Guidelines for meta-analyses evaluating diagnostic tests. Annals of Internal Medicine, 120:667-676, 1994.
- J.W. P.F. Kardaun and O.J. W.F. Kardaun. Comparative diagnostic performance of three radiological procedures for the detection of lumbar disk herniation. Methods of Information in Medicine, 29:12-22, 1990.
- J.W. P.F. Kardaun, J.Schipper, and R.Braakman. CT, myelography, and phlebography in the detection of lumbar disk herniation: An analysis of the literature. American Journal of Neuroradiology, 10:1111-1122, 1989.
- D.L. Kent, D.R. Haynor, E.B. Larson, and R.A. Deyo. Diagnosis of lumbar spinal stenosis in adults: A meta-analysis of the accuracy of CT, MR, and myelography. American Journal of Roentgenology, 158:1135-44, 1992.
- B.Littenberg, A.I. Mushlin, and the Diagnostic Technology AssessmentConsortium. Technetium bone scanning in the diagnosis of osteomyelitis: A meta-analysis of test performance. Journal of General Internal Medicine, 7:158-163, 1992.
- B.Littenberg and L.E. Moses. Estimating diagnostic accuracy from multiple conflicting reports: A new meta-analytic method. Medical Decision Making, 13:313-321, 1993.
- D.K. McClish. Combining and comparing area estimates across studies or strata. Medical Decision Making, 12:274-9, 1992.
- A.S. Midgette, T.A. Stukel, and B.Littenberg. A meta-analytic method for summarizing diagnostic test performances. Medical Decision Making, 13:253-257, 1993.
- L.E. Moses, D.E. Shapiro, and B.Littenberg. Combining independent studies of a diagnostic test into a summary ROC curve: Data-analytic approaches and some additional considerations. Statistics in Medicine, 12:1293-1316, 1993.
- H.E. Rockett, W.L. Campbell, C.A. Britton, J.M. Holbert, J.L. King, D. Gur. Empiric assessment of parameters that affect the design of multireader receiver operating characteristic studies. Academic Radiology, 6: 723-729.
- C.M. Rutter and C.Gatsonis. Regression methods for meta-analysis of diagnostic test data. Academic Radiology,2:supple:48-56,1995.
- D.E. Shapiro. Meta-analysis in ROC space: Combining independent studies of a diagnostic test, Technical Report 421, [Ph.D. Dissertation]. Department of Statistics, Stanford University, March 1993.
- D.E. Shapiro. Issues in combining independent estimates of sensitivity and specificity of a diagnostic test. Academic Radiology,2:Suppl:37-47,1995.
- J.A. Swets. Sensitivities and specificities of diagnostic tests. Journal of the American Medical Association, 248:548-549, 1982.
- X.H. Zhou Empirical Bayes combination of estimated areas under ROC curves using estimating equations. Medical Decision Making, 16 :24-28, 1996.
- X.H. Zhou Methods for combining rates from several studies. Statistics in Medicine, 18: 557-566, 1999.

- G.Campbell and J.M. Deleo. Fundamentals of fuzzy receiver operating characteristic (ROC) functions. Computer Science and Statistics: Proceedings of the 21st Symposium on the Inference, pages 543-548, 1989.
- G.Campbell, D.Levy, A.Lausier, M.R. Horton, and J.J. Bailey. Nonparametric comparison of entire ROC curves for computerized ECG left ventricular hypertrophy algorithms using data from the Framingham Heart Study. Journal of Electrocardiology, 22 Suppl:152-7, 1989.
- G.Campbell, D.Levy, and J.J. Bailey. Bootstrap comparison of fuzzy ROC curves for ECG-LVH algorithms using data from the Framingham Heart Study. Journal of Electrocardiology, 23 Suppl:132-7, 1990.
- G.Campbell. Advances in statistical methodology for the evaluation of diagnostic and laboratory tests. Statistics in Medicine, 13:499-508, 1994.
- M.J. Goddard and I.Hinberg. Receiver operator characteristic (ROC) curves and non-normal data: An empirical study. Statistics In Medicine, 9:325-37, 1990.
- S.W. Greenhouse and N.Mantel. The evaluation of diagnostic tests. Biometrics, 6:399-412, 1950.
- M.Jaraiedi and G.D. Herrin. A non-parametric method for fitting ROC curves for evaluation of industrial inspection performance. Applied Stochastic Models and Data Analysis, 4:79-88, 1988.
- V. Kairisto. Optimal bin widths for frequency histograms and ROC curves [letter]. Clinical Chemistry, 41:766-7, 1995.
- V. Kairisto. Software for illustrative presentation of basic clinical characteristics of laboratory tests--GraphROC for Windows. Scandinavian Journal of Clinical & Laboratory Investigation, 222(suppl):43-60,1995.
- K.Linnet. Comparison of quantitative diagnostic tests: Type I error, power, and sample size. Statistics in Medicine, 6:147-158, 1987.
- K.Linnet. A review on the methodology for assessing diagnostic tests. Clinical Chemistry, 34:1379-1386, 1988.
- K.Linnet. Assessing diagnostic tests by a strictly proper scoring rule. Statistics in Medicine, 8:609-618, 1989.
- E.A. Robertson, M.H. Zweig, and A.C. VanSteirtghem. Evaluating the clinical efficacy of laboratory tests. American Journal of Clinical Pathology, 79:78-86, 1983.
- S.Sukhatme and C.A. Beam. Stratification in nonparametric ROC studies. Biometrics, 50(1):149-63, 1994.
- Y.T. van der Schouw, H.Straatman, and A.L. Verbeek. ROC curves and the areas under them for dichotomized tests: empirical findings for logistically and normally distributed diagnostic test results. Medical Decision Making, 14(4):374-81, 1994.
- M.H. Zweig. Evaluation of the clinical accuracy of laboratory tests. Archives of Pathology Laboratory Medicine, 112:383-6, 1988.
- M.H. Zweig and G.Campbell. Receiver operating characteristic (ROC) plots: A fundamental evaluation tool in clinical medicine. Clinical Chemistry, 39:561-577, 1993.
- H. Zou, D.E. Shapiro and W.J. Hall. Smooth nonparametric receiver operating characteristic curves for continuous diagnostic tests (abstract only). Medical Decision Making, 14:431, 1994.

- D.P. Chakraborty, E.S. Breatnach, M.V. Yester, B.Soto, and G.T. Barnes. Digital and conventional chest imaging: A modified ROC study of observer performance using simulated nodules. Radiology, 158:35-9, 1986.
- D.P. Chakraborty. Maximum likelihood analysis of free-response receiver operating characteristic (FROC) data. Medical Physics, 16:561-8, 1989.
- D.P. Chakraborty and L.H. Winter. Free-response methodology: Alternate analysis and a new observer-performance experiment. Radiology, 174:873-81, 1990.
- P. DeNeef and D.L. Kent. Using treatment-tradeoff preferences to select diagnostic strategies: Linking the ROC curve to threshold analysis. Medical Decision Making, 13 :126-132,1993.
- S. Dreiseitl, L. Ohno-Machado, M. Binder. Comparing three-class diagnostic tests by three-way ROC analysis. Medical Decision Making, 20: 323-331, 2000.
- O.Gefeller and H.Brenner. How to correct for chance agreement in the estimation of sensitivity and specificity of diagnostic tests. Methods of Information in Medicine, 33:180-186, 1994.
- C.Herrmann, E.Buhr, D.Hoeschen and S.Y. Fan SYnewblock. Comparison of ROC and AFC methods in a visual detection task. Medical Physics, 20(3):805-11, 1993.
- K. Hnatkova, J.D. Poloniecki, A.J. Camm, and M.Malik Computation of multifactorial receiver operator and predictive accuracy characteristics. Computer Methods and Programs in Biomedicine, 42(3):147-56, 1994.
- W. Leisenring , M.S. Pepe G. Longton. A marginal regression modelling framework for evaluating medical diagnostic tests. 1996 ENAR meeting. Unpublished manuscript.
- P.A. Murtaugh ROC curves with multiple marker measurements. Biometrics, 51 (4):1514-22,1995.
- B. Reiser, D. Faraggi. Confidence intervals for the generalized ROC criterion. Biometrics, 53: 644-652, 2000.
- H.E. Rockett, J.L. King, J.L. Medina, H.B. Eisen, M.L. Brown and D. Gur. Imaging systems evaluation: effect of subtle cases on the design and analysis of receiver operating characteristic studies. American Journal of Roentgenology, 165 :679-83,1995.
- E.K. Shultz. Multivariate receiver-operating characteristic curve analysis: prostate cancer screening as an example. [Review] Clinical Chemistry,41:1248-55, 1995.
- S.J. Starr, C.E. Metz, L.B. Lusted, and D.J. Goodenough. Visual detection and localization of radiographic images. Radiology, 116:533-8, 1975.
- S.J. Starr, C.E. Metz, and L.B. Lusted. Comments on the generalization of receiver operating characteristic analysis to detection and localization tasks. Physics In Medicine and Biology, 22:376-81, 1977.
- W.R. Steinbach and K.Richter. Multiple classification and receiver operating characteristic (ROC) analysis. Medical Decision Making, 7:234-7, 1987.
- A. Taube. To judge the judges--kappa, ROC or what?. Arch Anat Cytol Pathol,43:227-33,1995.
- L.L. Wheeless, R.D. Robinson, O.P. Lapets, C.Cox, A.Rubio, M.Weintraub, and L.J. Benjamin. Classification of red blood cells as normal, sickle, or other abnormal, using a single image analysis feature. Cytometry, 17(2):159-66, 1994.

- R.Bairagi and C.M. Suchindran. An estimator of the cutoff point maximizing sum of sensitivity and specificity. Sankhya, Series B, Indian Journal of Statistics, 51:263-269, 1989.
- G.R. Bergus. When is a test positive? The use of decision analysis to optimize test interpretation. Family Medicine, 25(10):656-60, 1993.
- D.Borowiak and J.F. Reed, III. Utility of combining two diagnostic tests. Computer Methods and Programs in Biomedicine, 35:171-175, 1991.
- F.A. Connell and T.D. Koepsell. Measures of gain in certainty from a diagnostic test. American Journal of Epidemiology, 121:744-53, 1985.
- P.DeNeef and D.L. Kent. Using treatment-tradeoff preferences to select diagnostic strategies: Linking the ROC curve to threshold analysis. Medical Decision Making, 13:126-32, 1993.
- W.L. England. An exponential model used for optimal threshold selection on ROC curves. Medical Decision Making, 8:120-31, 1988.
- A.Fajardo-Gutierrez, L.Yamamoto-Kimura, L.Yanez-Velasco, J.Garduno-Espinosa, and M.C. Martinez-Garcia. The utility of joint sensitivity and specificity curves applied to a diagnostic test. Salud Publica de Mexico, 36(3):311-7, 1994.
- D.G. Fryback and J.R. Thornbury. The efficacy of diagnostic imaging. Medical Decision Making, 11:95-101, 1991.
- P.F. Griner, R.J. Mayewski, A.I. Mushlin, and P.Greenland. Selection and interpretation of diagnostic tests and procedures: Principles and applications. Annals of Internal Medicine, 94:553-592, 1981.
- W.J. Hall and C.E. Phelps. A clinically meaningful measure of the difference between diagnostic technologies. Medical Decision Making, 7:286, 1987.
- H.C. Kraemer. Evaluating Medical Tests: Objective and Quantitative Guidelines. Sage Publications, Newbury Park, California, 1992.
- C.E. Metz, D.J. Goodenough, and K.Rossmann. Evaluation of receiver operating characteristic curve data in terms of information theory, with applications in radiography. Radiology, 109:297-303, 1973.
- D.Mossman and E.Somoza. Maximizing diagnostic information from the dexamethasone suppression test: An approach to criterion selection using receiver operating characteristic analysis. Archives of General Psychiatry, 46:653-60, 1989.
- D.Mossman and E.Somoza. Assessing improvements in the dexamethasone suppression test using receiver operating characteristics analysis. Biological Psychiatry, 25:159-73, 1989.
- D.A. Noe. Selecting a diagnostic study's cutoff value by using its receiver operating characteristic curve. Clinical Chemistry, 29:571-2, 1983.
- J.C. Peirce and R.G. Cornell. Integrating stratum-specific likelihood ratios with the analysis of ROC curves. Medical Decision Making, 13:141-51, 1993.
- C.E. Phelps and A.I. Mushlin. Focusing technology assessment using medical decision theory. Medical Decision Making, 8:279-289, 1988.
- F.Sainfort. Evaluation of medical technologies: A generalized ROC analysis. Medical Decision Making, 11:208-20, 1991.
- H.Schafer. Constructing a cut-off point for a quantitative diagnostic test. Statistics in Medicine, 8:1381-1391, 1989.
- E.Somoza Classifying binormal diagnostic tests using separation-asymmetry diagrams with constant-performance curves. Medical Decision Making, 14:157-68, 1994.
- E.Somoza Classification of diagnostic tests. International Journal of Bio-Medical Computing, 37 :41-55, 1994.
- E.Somoza and D.Mossman. ``Biological markers'' and psychiatric diagnosis: Risk-benefit balancing using ROC analysis. Biological Psychiatry, 29:811-26, 1991.
- E.Somoza and D.Mossman. Comparing and optimizing diagnostic tests: An information-theoretical approach. Medical Decision Making, 12:179-88, 1992.
- E.Somoza and D.Mossman. Comparing diagnostic tests using information theory: The INFO-ROC technique. Journal of Neuropsychiatry and Clinical Neurosciences, 4:214-9, 1992.
- E.Somoza, L.Soutullo-Esperon, and D.Mossman. Evaluation and optimization of diagnostic tests using receiver operating characteristic analysis and information theory. International Journal of Bio-Medical Computing, 24:153-89, 1989.
- J.Vermont, J.L. Bosson, P.Franccois, C.Robert, A.Rueff, and J.Demongeot. Strategies for graphical threshold determination. Computer Methods and Programs In Biomedicine, 35:141-50, 1991.
- C.D. Ward. The differential positive rate, a derivative of receiver operating characteristic curves useful in comparing tests and determining decision levels. Clinical Chemistry, 32:1428-9, 1986.

- R.F. Raubertas, L.E. Rodewald, S.G. Humiston, and P.G. Szilagyi. ROC curves for classification trees. Medical Decision Making, 14:169-174, 1994.

- J.K. Gohagan, E.L. Spitznagel, M.M. McCrate, and T.B. Frank. ROC analysis of mammography and palpation for breast screening. Investigative Radiology, 19:587-92, 1984.
- J.E. Goin. ROC curve estimation and hypothesis testing: Applications to breast cancer detection. Pattern Recognition, 15:263-269, 1982.
- X.M. Tu, J. Kowalski, G. Jia. Bayesian analysis of prevalence with covariates using simulation-based techniques: Applications to HIV screening. Statistics in Medicine, 18: 3059-3073, 1999.
- G. Tusch. Evaluation of partial classification algorithms using ROC curves. Medinfo, 8 :904-8,1995.

- N.A. Obuchowski and D.K. McClish. Sample size determination for diagnostic accuracy studies involving binormal ROC curve parameters. Statistics in Medicine, 16: 1529-1542, 1997.
- N.A. Obuchowski. Sample size tables for receiver operating characteristic studies. AJR, 175: 603-108, 2000.

- D. Faraggi. THe effect of random measurement error on receiver operating characteristic (ROC) curves. Statistics in Medicine, 19: 61-70, 2000.
- M. Coffin, S. Sukhatme. Receiver opearating characteristic studies and measurement errors. Biometrics, 53: 823-837, 1997.

- T.A. Alonzo, M.S. Pepe. Distribution-free ROC analysis using binary regression techniques. Biostatistics, 3(3):421-432, 2002.
- W.S. Andrus and K.T. Bird. Editorial: Radiology and the receiver operating characteristic (ROC) curve. Chest, 67:378-9, 1975.
- J.Ashton and M.L. Moeschberger. A SAS macro for estimating the error rates of two diagnostic tests, neither being a gold standard. Proceedings of the SAS Users Group International Conference, 995-996, 1988.
- S.R. Aylward. Continuous mixture modeling via goodness-of-fit ridges. Pattern Recognition, 35(9):1821-1833, 2002.
- S.V. Beiden, M.A. Maloof, R.F. Wagner. A general model for finite-sample effects in training and testing of competing classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12):1561-1569, 2003.
- S.V. Beiden, R.F. Wagner, G. Campbell. Components-of-variance models and multiple-bootstrap experiments: An alternative method for random-effects, receiver operating characteristic analysis. Academic Radiology, 7(5):341-349, 2000.
- S.V. Beiden, R.F. Wagner, G. Campbell et al. Components-of-variance models for random-effects ROC analysis: The case of unequal variance structures across modalities. Academic Radiology, 8(7):605-615, 2001.
- S.V. Beiden, R.F. Wagner, G. Campbell et al. Analysis of uncertainties in estimates of components of variance in multivariate ROC analysis. Academic Radiology, 8(7):616-622, 2001.
- B.M. Bennett. On tests for equality of predictive values for t diagnostic procedures. Statistics in Medicine, 4:535-539, 1985.
- K. Bernstein, K. Matthews. Correlation of system performance parameters to ROC analysis of PET/CT images. Medical Physics, 31(6):1819-1819, 2004.
- J.Brismar. Understanding receiver-operating-characteristic curves: A graphic approach. American Journal of Roentgenology, 157:1119-21, 1991.
- W.S. Browner and T.B. Newman. Are all significant p values created equal? the analogy between diagnostic tests and clinical research. Journal of the American Medical Association, 257:2459-2463, 1987.
- D.P. Chakraborty, K.S. Berbaum Observer studies involving detection and localization: Modeling, analysis, and validation. Medical Physics, 31(8):2313-2330, 2004.
- H.P. Chan, B. Sahiner, R.F. Wagner, et al. Classifier design for computer-aided diagnosis: Effects of finite sample size on the mean performance of classical and neural network classifiers. Medical Physics, 26(12):2654-2668, 1999.
- Y.Charpak, C.Blery, and C.Chastang. Designing a study for evaluating a protocol for the selective performance of preoperative tests. Statistics in Medicine, 6:813-822, 1987.
- J. Chen J, K.M. Conigrave, P. Macaskill, et al. Combining carbohydrate-deficient transferrin and gamma-glutamyltransferase to increase diagnostic accuracy for problem drinking. Alcohol and Alcoholism, 38(6):574-582, 2003.
- M.S. Chester. Human visual perception and ROC methodology in medical imaging. Physics In Medicine and Biology, 37:1433-76, 1992.
- T.E. Cohn, D.G. Green, and W.P. Tanner, Jr. Receiver operating characteristic analysis: Application to the study of quantum fluctuation effects in optic nerve of rana pipiens. Journal of General Physiology, 66:583-616, 1975.
- R. Dahyot, P. Charbonnier, F. Heits A Bayesian approach to object detection using probabilistic appearance-based models. Pattern Analysis and Applications, 7(3):317-332, 2004.
- R.Detrano. Accuracy curves: An alternative graphical representation of probability data. Journal of Clinical Epidemiology, 42:983-6, 1989.
- S. Dreiseitl, L. Ohno-Machado, M. Binder. Comparing three-class diagnostic tests by three-way ROC analysis. Medical Decision Making, 20(3):323-331, 2000.
- D.C. Edwards, C.E. Metz, R.W. Nishikawa. The hypervolume under the ROC hypersurface of "near-guessing" and "near-perfect" observers in N-class classification tasks. IEEE Transactions of Medical Imaging, 24(3):293-299, 2005.
- D.C. Edwards, C.E. Metz, M.A. Kupinski. Ideal observers and optimal ROC hypersurfaces in N-class classification. IEEE Transactions on Medical Imaging, 23(7):891-895, 2004.
- D. Faraggi, B. Reiser, E.F. Schisterman. ROC curve analysis for biomarkers based on pooled assessments. Statistics in Medicine, 22(15):2515-2527, 2003.
- T. Fawcett, P.A. Flach A response to Webb and Ting's On the application of ROC analysis to predict classification performance under varying class distributions. Machine Learning, 58(1):33-38, 2005.
- A.R. Feinstein. On the sensitivity, specificity, and discrimination of diagnostic tests. Clinical Pharmacology and Therapeutics, 17:104-118, 1975.
- C. Ferri, J. Hernandez-Orallo, M.A. Salido. Volume under the ROC surface for multi-class problems. Lecture Notes in Artificial Intelligence, 2837:108-120, 2003.
- J.Fuentes and M.Martinez. An alternative to receiver-operating characteristic curves. Clinical Chemistry, 29:1445, 1983.
- J. Furnkranz, P.A. Flach. ROC 'n' rule learning - Towards a better understanding of covering algorithms. Machine Learning, 58(1):39-77, 2005.
- M.H. Gail and S.B. Green. A generalization of the one-sided two-sample Kolmogorov-Smirnov statistic for evaluating diagnostic tests. Biometrics, 32:561-570, 1976.
- O. Gefeller and H. Brenner. How to correct for chance agreement in the estimation of sensitivity and specificiety of diagnostic tests. Meth. Inform. in Med, :180-6,1994.
- A.J. Girling. Rank statistics expressible as integrals under P-P-plots and receiver operating characteristic curves. Journal of the Royal Statistical Society Series B-Statistical Methodology, 62:367-382, 2000.
- P.Glasziou and J.Hilden. Decision tables and logic in decision analysis. Medical Decision Making, 6:154-60, 1986.
- G. Gorog. An Excel program for calculating and plotting receiver-operator characteristic (ROC) curves, histograms and descriptive statistics. Computers in Biology and Medicine, 24(3):167-9, 1994.
- M. Greiner, D. Pfeiffer, R.D. Smith. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Preventive Veterinary Medicine, 45(1-2):23-41, 2000.
- D.H. Gustafson, F. Sainfort, M. Eichler, et al. Developing and testing a model to predict outcomes of organizational change. Health Services Research, 38(2):751-776, 2003.
- K.O. Hajian-Tilaki, J.A. Hanley. Comparison of three methods for estimating the standard error of the area under the curve in ROC analysis of quantitative data. Academic Radiology, 9(11):1278-1285, 2002.
- A.P. Hallstrom and G.B. Trobaugh. Specificity, sensitivity, and prevalence in the design of randomized trials: A univariate analysis. Controlled Clinical Trials, 6:128-135, 1985.
- T. Hansen, H. Neumann. A biologically motivated scheme for robust junction detection. Lecture Notes in Computer Science, 2525:16-26, 2002
- M.B. Harrington. Some methodological questions concerning receiver operating characteristic (ROC) analysis as a method for assessing image quality in radiology. Journal of Digital Imaging, 3:211-8, 1990.
- R.D. Hays. ROC: Estimation of the area under a receiver operating characteristic curve. Applied Psychological Measurement Inc., 14:208-208, 1990.
- J.W. Hoppin, M.A. Kupinski, G.A. Kastis, et al. Objective comparison of quantitative imaging modalities without the use of a gold standard. IEEE Transactions on Medical Imaging, 21(5):441-449, 2002.
- J.M. Irvine. Assessing target search performance: the free-response operator characteristic model. Optical Engineering, 43(12):2926-2934,2004.
- H. Ishwaran, C.A. Gatsonis. A general class of hierarchical ordinal regression models with applications to correlated ROC analysis. Canadian Journal of Statistics-Revue Canadienne de Statistique, 28(4):731-750, 2000.
- R.H. Jones and M.W. McClatchey. Beyond sensitivity, specificity and statistical independence. Statistics in Medicine, 7:1289-1295, 1988.
- V. Kairisto and A. Poola. Software for illustrative presentation of basic clinical characteristics of laboratory tests--GraphROC for Windows. Scandinavian Journal of Clinical & Laboratory Investigation, 222 (supple):43-60,1995.
- L.Kuncheva and K.Andreeva. DREAM: a shell-like software system for medical data analysis and decision support. Computer Methods and Programs in Biomedicine, 40(2):73-81, 1993.
- M.A. Kupinski,D.C. Edwards, M.L. Giger, et al. Ideal observer approximation using Bayesian classification neural networks. IEEE Transactions on Medical Imaging, 20(9):886-899, 2001.
- P.A. Lachenbruch. Multiple reading procedures: The performance of diagnostic tests. Statistics in Medicine, 7:549-557, 1988.
- A.R. Laird, B.P. Rogers, M.E. Meyerand. Comparison of Fourier and wavelet resampling methods. Magnetic Resonance in Medicine, 51(2):418-422, 2004.
- R.A. Lew and P.S. Levy. Estimation of prevalence on the basis of screening tests. Statistics in Medicine, 8:1225-1230, 1989.
- R.B. Lirio, I.C. Donderiz, and M.C. PerezAbalo. Maximum likelihood estimation of signal detection model parameters for the assessment of two-stage diagnostic strategies. International Journal of Bio-Medical Computing, 31:117-26, 1992.
- B. Liu, C.E. Metz, Y.L. Jiang. An ROC comparison of four methods of combining information from multiple images of the same patient. Medical Physics, 31(9):2552-2563, 2004.
- A.Y. Liu, E.F. Schisterman, E. Teoh. Sample size and power calculation in comparing diagnostic accuracy of biomarkers with pooled assessments. Journal of Applied Statistics 31(1):49-59, 2004.
- A.Y. Liu, E.F. Schisterman, Y. Zhu. On linear combinations of biomarkers to improve diagnostic accuracy. Statistics in Medicine, 24(1):37-47, 2005.
- Y. Lu, D.N. Heller, S.J. Zhao. Receiver operating characteristic (ROC) analysis for diagnostic examinations with uninterpretable cases. Statistics in Medicine, 21(13):1849-1865, 2002.
- K. Lui. A discussion on the conventional estimator of sensitivity and specificity in multiple tests. Statistics in Medicine, 8:1231-1240, 1989.
- P. Macaskill. Empirical Bayes estimates generated in a hierarchical summary ROC analysis agreed closely with those of a full Bayesian analysis. Journal of Clinical Epidemiology, 57(9):925-932, 2004.
- F.A. Mann, C.F. Hildebolt, and A.J. Wilson. Statistical analysis with receiver operating characteristic curves. Radiology, 184:37-8, 1992.
- R.J. Marshall. The predictive value of simple rules for combining two diagnostic tests. Biometrics, 45:1213-1222, 1989.
- C. Metz. Models in medicine IV. ROC analysis: Methodology and practical applications. Medical Physics, 30(6):1442-1442, 2003.
- D.K. McClish and D.Quade. Improving estimates of prevalence by repeated testing. Biometrics, 41:81-89, 1985.
- M.J. Monsour, A.T. Evans, and L.L. Kupper. Confidence intervals for post-test probability. Statistics in Medicine, 10:443-456, 1991.
- A.I. Mushlin, A.S. Detsky, C.E. Phelps, P.W. O'Connor D.K. Kido, W.Kucharczyk, D.W. Giang, C.Mooney, C.M. Tansey and W.J. Hall. The accuracy of magnetic resonance imaging in patients with suspected multiple sclerosis. The Rochester-Toronto Magnetic Resonance Imaging Study Group. JAMA, 269(24):3146-51, 1993.
- D.B. Mumford. ROC analysis. British Journal of Psychiatry, 157:301, 1990.
- C.T. Nakas, C.T. Yannoutsos. Ordered multiple-class ROC analysis with continuous measurements. Statistics in Medicine, 23(22):3437-3449, 2004.
- N.A. Obuchowski. How many observers are needed in clinical studies of medical imaging? American Journal of Roentgenology, 182(4):867-869, 2004.
- N.A. Obuchowski. Fundamentals of clinical research for radiologists - ROC analysis American Journal of Roentgenology, 184(2):364-372, 2005.
- W.Palmas, T.A. Denton, A.P. Morise, and G.A. Diamond. Afterimages: integration of diagnostic information through Bayesian enhancement of scintigraphic images American Heart Journal, 128(2):281-7, 1994 Aug.
- J. Paetz. Finding optimal decision scores by evolutionary strategies. Artificial Intelligence in Medicine, 32(2):85-95, 2004.
- S.H. Park, J.M. Goo, C.H. Jo. Receiver operating characteristic (ROC) curve: Practical review for radiologists. Korean Journal of Radiology, 5(1):11-18, 2004.
- G. Parker, D. Hadzi-Pavlovic, L. Both, et al. Measuring disordered personality functioning: to love and to work reprised. Acta Psychiatrica Scandinavica, 110(3):230-239, 2004.
- J.L. Pater and A.R. Willan. Clinical trials as diagnostic tests. Controlled Clinical Trials, 5:107-113, 1984.
- P.E. Politser. Explanations of statistical concepts: Can they penetrate the haze of Bayes? Methods of Information in Medicine, 23:99-108, 1984.
- C. Perlich, F. Provost, J.S. Simonoff. Tree induction vs. logistic regression: A learning-curve analysis. Journal of Machine Learning Research, 4(2):211-255, 2004.
- G. Regehr, J. Colliver. On the equivalence of classic ROC analysis and the loss-function model to set cut points in sequential testing. Academic Medicine, 78(4):361-364, 2003.
- M.Rodriguez, I.Moussa, J.Froning, M.Kochumian, and V.F. Froelicher. Improved exercise test accuracy using discriminant function analysis and "recovery ST slope". Journal of Electrocardiology, 26(3):207-18, 1993.
- H.E. Rockett, J.L. KIng, J.L. Medina, H.B. Eisen, M.L. Brown, D. Gur. Imaging systems evaluation: effect of subtle cases on the design and analysis of receiver operating characteristic studies. American Journal of Roentgenology,165:679-83, 1995.
- B. Sahiner, H.P. Chan, N. Petrick, et al. Feature selection and classifier performance in computer-aided diagnosis: The effect of finite sample size. Medical Physics, 27(7):1509-1522, 2000.
- M.S. Scher, B.L. Jones, D.A. Steppe, et al. Functional brain maturation in neonates as measured by EEG-sleep analyses. Clinical Neurophysiology, 114(5):875-882, 2003.
- E.F. Schisterman, D. Faraggi, B.Reiser. Adjusting the generalized ROC curve for covariates. Statistics in Medicine, 23(21):3319-3331, 2004.
- P.M. Shankar, V.A. Dumane, T. George, et al. Classification of breast masses in ultrasonic B scans using Nakagami and K distributions. Physics in Medicine and Biology, 48(14):2229-2240, 2003.
- P.M. Shankar, V.A. Dumane, C.W. Piccoli, et al. Computer-aided classification of breast masses in ultrasonic B-scans using a multiparameter approach. IEEE Transactions on Ultrasonics Ferroelectrics and Frequency Control 50(8):1002-1009, 2003.
- H.C. Sox, Jr. Probability theory in the use of diagnostic tests. Annals of Internal Medicine, 104:60-66, 1986.
- C. Stephan, S. Wesseling, T. Schink, et al. Comparison of eight computer programs for receiver-operating characteristic analysis. Clinical Chemistry, 49(3):433-439, 2003.
- R.G. Swensson, J.L. King, D. Gur. A constrained formulation for the receiver operating characteristic (ROC) curve based on probability summation. Medical Physics, 28(8):1597-1609, 2001.
- Y. Takahashi, K. Murase, H. Higashino, et al. Receiver operating characteristic (ROC) analysis of images reconstructed with iterative expectation maximization algorithms. Annals of Nuclear Medicine, 15(6):521-525, 2001.
- A.Taube. Sensitivity, specificity and predictive values: A graphical approach. Statistics in Medicine, 5:585-591, 1986.
- L.A. Thibodeau. Evaluating diagnostic tests. Biometrics, 37:801-804, 1981.
- L.A. Thibodeau. Sensitivity and specificity. Encyclopedia of Statistical Sciences, 8:370-372, 1988.
- M.L. Thompson. Assessing the diagnostic accuracy of a sequence of tests. Biostatistics, 4(3):341-351, 2003.
- M.Treisman and A.Faulkner. The effect of signal probability on the slope of the receiver operating characteristic given by the rating procedure. British Journal of Mathematical and Statistical Psychology, 37199-215, 1984.
- P.M. Vacek. The effect of correlation between two diagnostic tests on maximum likelihood estimates of test error rates and disease prevalences. In ASA Proceedings of the Statistical Computing Section, 271-276, 1984.
- P.M. Vacek, M.Kresnow, and R.M. Potter. The sampling distributions of estimators for the error rates of diagnostic tests. ASA Proceedings of the Statistical Computing Section, 438-440, 1985.
- R. Wagner. Multiple-reader, multiple-case (MRMC) ROC analysis in diagnostic imaging, computer-aided diagnosis, and statistical pattern recognition. Medical Physics, 29(6):1331-1331, 2002.
- R.F. Wagner, S.V. Beiden, G. Campbell, et al. Assessment of medical imaging and computer-assist systems: Lessons from recent experience. Academic Radiology, 9(11):1264-1277, 2002.
- R.F. Wagner, S.V. Beiden, C.E. Metz. Continuous versus categorical data for ROC analysis: Some quantitative considerations. Academic Radiology, 8(4):328-334, 2001.
- J.J. Wang, Z. Wang, G.K. Aguirre, et al. To smooth or not to smooth? - ROC analysis of perfusion fMRI data. Magnetic Resonance Imaging, 23(1):75-81, 2005.
- G.I. Webb, K.M. Ting. On the application of ROC analysis to predict classification performance under varying class distributions. Machine Learning, 58(1):25-32, 2005.
- M. Weiner, R. Centor. Web-based, interactive receiver operating characteristics (ROC) analysis. Journal of the American Medical Informatics Association, Suppl. S:1058-1058, 2001.
- T. Wigren, S. Remle, B. Wahlberg. Analysis of a low-complexity change detection scheme. International Journal of Adaptive Control and SIgnal Processing, 14(5):481-503, 2000.
- A.A. White and J.R. Landis. A general categorical data methodology for evaluating medical diagnostic tests. Communication in Statistics, Part A-Theory and Methods, 117:567-605, 1982.
- T.Yanagawa and B.C. Gladen. Estimating disease rates from a diagnostic test. American Journal of Epidemiology, 119:1015-1023, 1984.
- T.Yanagawa and F.Kasagi. Estimating prevalence and incidence of disease from a diagnostic test. Statistical Theory and Data Analysis, 783-801, 1985.
- K.H. Zou, S.K. Warfield, J.R. Fielding, et al.
Statistical validation based on parametric receiver operating characteristic analysis
of continuous classification data.
Academic Radiology, 10(12):1359-1368, 2003.