Metz CE. ROC methodology in radiologic imaging. Invest Radiol 1986; 21:720 --733.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....characterised with an obvious increase of false positive cases. The network performance is measured using the area A z under a ROC curve generated by plotting the true positive fraction (sensitivity) against the false positive fraction (1 specificity) of the cases for various threshold values [29]. Alternatively, the free receiver operating characteristic (FROC) curve may be used which considers the number of false positive clusters per image instead of the specificity value [8] The finally selected network (the one with the best cross validation performance) for the Nijmegen database has ....

Metz CE. ROC methodology in radiologic imaging. Invest Radiol 1986;21(9):720--33.

....problem where the quantity to be maximized is the performance of the classifier on an independent dataset. There are, however, numerous problems with representing classifier performance by a single (scalar) objective function, which is needed so that one can use a scalar optimizer [6] [7]. Binary classifiers [4] have, in essence, two implicit objective functions: one describing how well they classify the abnormal cases (sensitivity) and one describing how well they classify the normal cases (specificity) These two objective functions are noncommensurable, implying that it may not ....

....fraction of class observations that are correctly classified is used as an estimate of . Likewise, the fraction of class observations that are correctly classified is used as an estimate of . A popular construct used for describing the performance of a diagnostic classifier is the ROC curve [6] [7], 20] 21] A ROC curve is generated by varying the value of one (or more) of the components of the parameter vector , and plotting the corresponding and values. For example, the Fig. 2. The two ROC curves have equal Az values, but, depending upon the relative preferences concerning the ....

[Article contains additional citation context not shown here]

C. E. Metz, "ROC methodology in radiologic imaging," Investigative Radiol., vol. 21, pp. 720--733, 1986.

....maximal values. However, in this paper we do not have an accurate R a for our data sets (although it is still very interesting to compare R a with R r ) hence we omit the derivation of the theoretical SDR and SNR. In addition, we can perform Receiver Operating Characteristic (ROC) curve analysis [27 28], which involves computing the True Positive Rate (TPR) and the False Positive Rate (FPR) while varying algorithm parameters, such as the BT threshold. TPR indicates the sensitivity of the method, and the False Negative Rate (FNR=1 FPR) indicates the specificity of the method. We can write TPR and ....

Metz, C.E., "ROC methodology in radiologic imaging," Investigative Radiology, vol.21, pp.720-733, 1986.

....feature analysis tool. 10 24] The representation obtained by appropriate wavelets provides a simple hierarchical framework for interpreting the image at dif y The terms sensitivity and false positive detection rate are widely used for evaluating a diagnostic medical imaging system. [33] ferent resolutions, namely, the information of an image is characterized by different structures in the image. The framework can separate small objects such as microcalcifications from large objects such as large background structures. In our approach, we emphasize preserving image detail during ....

....10 bits with a pixel size of 0:1 Theta 0:1 mm 2 using a Fuji drum scanner system. 5. Performance evaluation method There are several different methods currently employed to evaluate the performance of a detection scheme. Currently, we are using receiver operating characteristic (ROC) analysis [33] and other associated measures such as Academic Reports of Tokyo Institute of Polytechnics 16: 24 37, 1994 8 free response receiver operating characteristic (FROC) analysis. 34 36] We believe that these are appropriate measures, because there will generally be a tradeoff between sensitivity ....

C. E. Metz, "ROC methodology in radiologic imaging," Investigative Radiology 21, 720-733 (1986).

....= 9. When combining the scores for each region into an aggregate score for the complete image, we used the product rule but weighted regions in which the classification gave good results more heavily. This was done using A z , the area under the ROC curve, as measure of classification performance [9]. We did not use automatic feature selection techniques, but evaluated the performance of various subsets of the computed features. Results are given in Table I. The ROC curves for the best performing feature sets for both databases are shown in Figure 2. The results are satisfactory for the TB ....

C.E. Metz. ROC methodology in radiologic imaging. Investigative Radiology, 21(9):720--733, 1986.

....of our system is to provide a rich set of primitives and tools to enable an anatomist to develop yet more realistic models of the human anatomy. 4. 2 Evaluation of new medical display technologies The evaluation of new medical display technologies by the receiver operating characteristics (ROC) [24] methodology requires the rating of many images whose normal abnormal states are known. Geometric modeling and rendering can provide an inexpensive source of images for this purpose. All abnormalities are userdefined and precisely known. For subjects participating in an ROC experiment, feedback ....

C. E. Metz, "ROC methodology in radiologic imaging," Investigative Radiology, vol. 21, pp. 720--733, 1986.

....Receiver Operating Curve Figure 3: In this operating curve, the maximin operating point is at the point A, which is the intersection point of the ROC curve with a line passing through the origin. The slope of the line is computed from the economic gain matrix (see text) Radiology community (see [3], and literature cited therein) and automatic target recognition (ATR) community make use of ROC methodology. For a discussion of quantitative performance evaluation for line detection algorithms where optimal operating points can be used see [4, 5] 2. PRACTICAL ISSUES Although the theory ....

C. E. Metz. ROC methodology in radiologic imaging. Investigative Radiology, 21(9), 1986.

....characteristic (ROC) curve. The ROC is an analytical technique used to provide both a desired accuracy index, and the desired basis for a description of utility in terms of cost and benefit [88] The advantages of ROC analysis over alternative methods of analysis have been clearly established [89, 90]. The ROC curve is a plot of the classifier s true positive detection rate versus its false positive rate. The false positive (FP) rate is the probability of incorrectly classifying a nontarget object (e.g. a normal tissue region) as a target object (e.g. a tumor region) Similarly, the true ....

....It would then be possible to trade a lower (higher) FP rate for a higher (lower) TP detection rate by choosing appropriate value(s) for the parameter(s) in question. A typical ROC curve is shown in Figure 13. The Area Under the ROC Curve (AUC) is an accepted way of comparing classifier performance [88, 89]. A perfect classifier would have a TP rate of 1.0 (or 100 ) and a FP rate of 0.0, and therefore would have an AUC of 1.0. Random guessing would result in an AUC of 0.5. When the different possible errors that can be made (false positive and false negative) by the classifier have different ....

[Article contains additional citation context not shown here]

C. E. Metz, "ROC methodology in radiologic imaging," Investigative Radiology, vol. 21, pp. 720--733, 1986.

....and to characterize the spatial resolution requirement and the effect of unsharp mask filtering on the detectability of subtle micro calcifications in digital mammography. ROC analysis has also been utilized in a study on the effect of attention cueing on breast cancer detection performance [13]. According to the ROC model, a radiologist or an observer decides to render a positive or negative diagnosis by comparing his or her confidence concerning each image with an internal confidence criterion. If confidence in a positive diagnosis exceeds this confidence criterion, the image is read ....

....ROC curve for the experiments: a) Arrangement Experiment; b) Density Experiment. Legend : thick lines (stereo viewing) thin lines (mono viewing) For our experiments, observer response data are evaluated by ROC analysis using standard techniques for five category ratings as described by Metz [13]. A maximum likelihood curve fit for data from each subject is computed. The Hotelling T 2 test (multivariate counterpart of student t test) is then applied to test the significance of the difference between performances of observers when using the stereo and mono viewing modes [14] In order to ....

[Article contains additional citation context not shown here]

Charles E. Metz. ROC methodology in radiologic imaging. Investigative Radiology, 21:720--733, 1986.

....4 indicates an abnormality is probably present. A response of 5 indicates an abnormality is definitely present. These categories are intended to motivate the subject to generate combinations of true positive and false positive fractions that are distributed more or less evenly along the ROC curve[22]. If desired, the subject can enter the response in less than 30 seconds by pressing the n key for next . Subjects are given feedback after each response. A tone is sounded if the image that has just been rated contains an abnormality. In each session the subject evaluates 60 images, with the ....

....less complicated and easier to learn and also help subjects keep response criteria constant throughout the session. 6. DATA ANALYSIS Observer response data are evaluated by receiver operating characteristic (ROC) analysis using standard techniques for five category ratings as described by Metz [22]. This analysis involves construction of ROC curves in which the correct rate for detection of a particular abnormality is plotted as a function of the false alarm rate. This type of analysis has been accepted as the most rigorous and objective means of comparing diagnostic imaging modalities in ....

[Article contains additional citation context not shown here]

C. E. Metz, "ROC methodology in radiologic imaging," Investigative Radiology, vol. 21, pp. 720--733, 1986.

.... An example of such a task is the analysis and interpretation of Single Photon Emission Computed Tomography (SPECT) or Positron Emission Tomography (PET) brain images in Alzheimer s disease (AD) This task has been studied in a few reports [4, 5, 6, 7] where it was found that the ROC performance [8] of the ANN was superior to that of a linear or quadratic discriminant and comparable or even superior to that of a human expert. This success may partly be due to the non parametric nature of ANN modeling: a functional form of the discriminant function (f. ex. linear or quadratic) is not assumed ....

....defined the right ANN architecture as the smallest ANN capable of learning the data set, this was repeated until no more parameters could be zeroed without introducing misclassifications into the data set. The performance of the smallest found ANN was estimated through the area under the ROC curve [8]. A ROC curve is generated by testing the ANN on new subjects using different decision thresholds. For each decision threshold two performance characteristics are obtained, namely the false positive ratio and the true positive ratio, which are plotted against each other. Using the area under this ....

Charles E. Metz. Roc methodology in radiologic imaging. Invest Radiol, 21:720--733, 1986.

....and simple. However, in practice, several subtle issues relating to experimental design and data analysis must be confronted [27] Although a discussion of some issues relating to stereoscopic color displays and receiver operating characteristic analysis can be found in the literature [24] [28], no comprehensive discussion of the design of psychophysical studies from an interdisciplinary perspective has been reported. In sections II and III, we bring together a discussion of various technical and psychophysical issues which may be encountered while designing and conducting stereoscopic ....

....III. ISSUES IN DATA ANALYSIS Receiver operating characteristic (ROC) analysis has been used extensively with studies on human perception and decision making [61] 62] ROC analysis has also been accepted as a rigorous and objective means of comparing diagnostic imaging modalities in radiology [28], 63] To compare the performance of observers using a modified or innovative technology with the performance of the same observers using conventional technology, it is necessary to gather performance data for a series of images in which the truth state is known and then construct ROC curves in ....

Charles E. Metz, "ROC methodology in radiologic imaging", Investigative Radiology, vol. 21, pp. 720--733, 1986.

....evaluate 60 images, with the knowledge that exactly half of the images contained a particular abnormality. V. DATA ANALYSIS Receiver operating characteristic (ROC) analysis has been accepted as the most rigorous and objective means of comparing diagnostic imaging modalities in radiology [26] [29], 30] In mammography related research, ROC analysis has been used to characterize the accuracy of mammography [31] to compare the performance of mammography and palpation [7] and to characterize the spatial resolution requirement and the effect of unsharp mask filtering on the detectability of ....

.... conventional technology, it is necessary to gather performance data for a series of images in which diagnostic truth is known and then to construct the ROC curves (see Figure 6) 26] 35] In the domain of medical imaging, ROC curves are most commonly assumed to have the binormal functional form [29]. The two adjustable parameters of binormal ROC curves can be fitted from the ROC data by using the maximum likelihood parameter estimation scheme [30] If for the same observers, the curve for the new technology lies above the curve for the conventional technology, there is objective evidence ....

[Article contains additional citation context not shown here]

Charles E. Metz, "ROC methodology in radiologic imaging", Investigative Radiology, vol. 21, pp. 720--733, 1986.

....1. INTRODUCTION Over the years, many different algorithms have been designed for the automated detection of microcalcifications. These techniques are typically evaluated using a ROC or FROC curve which displays the trade off between some measure of sensitivity and a measure of specificity [1,2]. In order to do this, a definition of a target is required. A target is simply any area of an image containing microcalcifications which we would wish the system to detect and prompt. The system s sensitivity is then a measure of its ability to select these targets and the specificity is ....

C.E. Metz, ROC Methodology in Radiologic Imaging. Investigative Radiology, 21(9), 1986, pp 720--733.

....during the experimental sessions. Such rests are likely to reduce variability in subject s responses. 4. DATA ANALYSIS AND RESULTS Observer response data are evaluated by receiver operating characteristic (ROC) analysis using standard techniques for five category ratings as described by Metz [23]. This type of analysis has been used extensively with studies on human perception and decision making [24, 25] In particular, ROC analysis is accepted as the most rigorous and objective means of comparing diagnostic imaging modalities in radiology [13, 23] To compare the performance of ....

.... five category ratings as described by Metz [23] This type of analysis has been used extensively with studies on human perception and decision making [24, 25] In particular, ROC analysis is accepted as the most rigorous and objective means of comparing diagnostic imaging modalities in radiology [13, 23]. To compare the performance of observers using a modified or innovative technology with the performance of the same observers using conventional technology, it is necessary to gather performance data for a series of images in which the truth state is known and then construct ROC curves in which ....

C. E. Metz, "ROC methodology in radiologic imaging," Investigative Radiology, vol. 21, pp. 720--733, 1986.

....clinical, and mammographic features. For example, case #32 from the atlas described a known malignancy in a 65 year old woman with an irregular, low density mass, no halo sign, and no calcifications; MammoNet calculated a probability of breast cancer of 99.6 . Analysis using the LABROC1 program [15] yielded an estimated area under the receiver operating characteristic (ROC) curve of 0.881 0.045 (Figure 2) At a probability threshold for breast cancer of 15 (which approximates the positive predictive value of mammographic suspicion) MammoNet correctly identified 23 of the 25 actually ....

Metz CE. ROC methodology in radiologic imaging. Invest Radiol 1986; 21:720-733.

No context found.

Metz CE. ROC methodology in radiologic imaging. Invest Radiol 1986; 21:720 --733.

No context found.

C.E. Metz, "ROC methodology in radiologic imaging," Investigative Radiology, vol. 21, pp. 720--733, 1986.

No context found.

C. E. Metz. ROC methodology in radiologic imaging. Investigative Radiology, 21(9):720{ 732, September 1986. 214

No context found.

Metz CE. ROC methodology in radiologic imaging. Invest Radiol 1986;21:720--733.

No context found.

C. E. Metz. ROC Methodology in Radiologic Imaging. In: Investigative Radiology, 21(9), 1986, pp 720-733.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC