Total Operating Characteristic (TOC) Curve and ROC Curve

Description

Construction of the Total Operating Characteristic (TOC) Curve and the Receiver (aka Relative) Operating Characteristic (ROC) Curve for spatial and non-spatial data. The TOC method is a modification of the ROC method which measures the ability of an index variable to diagnose either presence or absence of a characteristic. The diagnosis depends on whether the value of an index variable is above a threshold. Each threshold generates a two-by-two contingency table, which contains four entries: hits (H), misses (M), false alarms (FA), and correct rejections (CR). While ROC shows for each threshold only two ratios, H/(H + M) and FA/(FA + CR), TOC reveals the size of every entry in the contingency table for each threshold (Pontius Jr., R.G., Si, K. 2014. The total operating characteristic to measure diagnostic ability for multiple thresholds. Int. J. Geogr. Inf. Sci. 28 (3), 570-583 <doi:10.1080/13658816.2013.862623>).

Details

Author(s)

Robert G. Pontius, Ali Santacruz, Amin Tayyebi, Benoit Parmentier, Kangping Si

Maintainer: Ali Santacruz

References

Pontius Jr., R.G., Kangpin, Si. 2014. The total operating characteristic to measure diagnostic ability for multiple thresholds. International Journal of Geographical Information Science 28 (3): 570-583. <doi:10.1080/13658816.2013.862623>

Pontius, G., Parmentier, B. 2014. Recommendations for using the Relative Operating Characteristic (ROC). Landscape Ecology 29 (3): 367-382. <doi:10.1007/s10980-013-9984-8>

See Also

TOC, plot

Construct the table for the ROC curve

Description

Construct the table for the Relative Operating Characteristic (ROC) curve for spatial or non-spatial data

Usage

## S4 method for signature 'numeric,numeric'
ROC(index, boolean, mask = NULL, nthres = NULL, thres = NULL, 
NAval = 0, progress = FALSE)

## S4 method for signature 'SpatRaster,SpatRaster'
ROC(index, boolean, mask = NULL, nthres = NULL, thres = NULL, 
NAval = 0, progress = FALSE)

Arguments

Details

thresholds are calculated as the unique values of the index object after masking out NA values (default option), if neither nthres nor thres is provided. The default option can be time-consuming if the amount of unique values in the index object (after masking out NA values) is large (e.g., greater than 1000). In the latter case, the user may prefer to enter specified thresholds (with the thres argument), or to indicate the number of equal-interval thresholds to be evaluated for the ROC curve (with the nthres argument)

Value

an object of class Roc containing the ROC table, the area under the curve (AUC), maximum AUC and minimum AUC

References

Pontius Jr., R.G., Kangpin, Si. 2014. The total operating characteristic to measure diagnostic ability for multiple thresholds. International Journal of Geographical Information Science 28 (3): 570-583. <doi:10.1080/13658816.2013.862623>

Pontius, G., Parmentier, B. 2014. Recommendations for using the Relative Operating Characteristic (ROC). Landscape Ecology 29 (3): 367-382. <doi:10.1007/s10980-013-9984-8>

See Also

plot

Examples

index <- rast(system.file("external/Prob_Map2.rst", package = "TOC"))
boolean <- rast(system.file("external/Change_Map2b.rst", package = "TOC"))
mask <- rast(system.file("external/MASK4.rst", package = "TOC"))

## thresholds can be defined by indicating the number of equal-interval thresholds 
rocd <- ROC(index, boolean, mask, nthres = 100)
rocd

## a vector of thresholds can also be used to define the thresholds
thresholds <- seq(min(unique(index)), max(unique(index)) + 1, 
                  by = ceiling(max(unique(index))/10))
rocd <- ROC(index, boolean, mask, thres = thresholds)
rocd

## all the unique values of the index object can be evaluated as thresholds 
## (default option)
## Not run: 
rocd <- ROC(index, boolean, mask, progress = TRUE)
rocd

## End(Not run)

## generate the ROC curve using non-spatial data (i.e., an object of class numeric)
## Not run: 
index <- rast(system.file("external/Prob_Map2.rst", package = "TOC"))
boolean <- rast(system.file("external/Change_Map2b.rst", package = "TOC"))
mask <- rast(system.file("external/MASK4.rst", package = "TOC"))

index <- values(index, mat = FALSE)
boolean <- values(boolean, mat = FALSE)
mask <- values(mask, mat = FALSE)
rocd <- ROC(index, boolean, mask, nthres = 100)
rocd

## End(Not run)

Construct the table for the TOC curve

Description

Construct the table for the Total Operating Characteristic (TOC) curve for spatial or non-spatial data. The TOC method is a modification of the ROC method which measures the ability of an index variable to diagnose either presence or absence of a characteristic. The diagnosis depends on whether the value of an index variable is above a threshold. Each threshold generates a two-by-two contingency table, which contains four entries: hits (H), misses (M), false alarms (FA), and correct rejections (CR). While ROC shows for each threshold only two ratios, H/(H + M) and FA/(FA + CR), TOC reveals the size of every entry in the contingency table for each threshold (Pontius Jr., R.G., Si, K. 2014. <doi:10.1080/13658816.2013.862623>).

Usage

## S4 method for signature 'numeric,numeric'
TOC(index, boolean, mask = NULL, nthres = NULL, thres = NULL, 
NAval = 0, P = NA, Q = NA, progress = FALSE, units = character(0))
## S4 method for signature 'SpatRaster,SpatRaster'
TOC(index, boolean, mask = NULL, nthres = NULL, thres = NULL, 
NAval = 0, P = NA, Q = NA, progress = FALSE)

Arguments

Details

thresholds are calculated as the unique values of the index object after masking out NA values (default option), if neither nthres nor thres is provided. The default option can be time-consuming if the amount of unique values in the index object (after masking out NA values) is large (e.g., greater than 1000). In the latter case, the user may prefer to enter specified thresholds (with the thres argument), or to indicate the number of equal-interval thresholds to be evaluated for the TOC curve (with the nthres argument)

Value

an object of class Toc containing the TOC table, the area under the curve (AUC), maximum AUC and minimum AUC, the prevalence, the population and the data units (for data in the TOC table slot, and the prevalence and population slots)

References

Pontius Jr., R.G., Kangpin, Si. 2014. The total operating characteristic to measure diagnostic ability for multiple thresholds. International Journal of Geographical Information Science 28 (3): 570-583. <doi:10.1080/13658816.2013.862623>

Pontius, G., Parmentier, B. 2014. Recommendations for using the Relative Operating Characteristic (ROC). Landscape Ecology 29 (3): 367-382. <doi:10.1007/s10980-013-9984-8>

See Also

plot

Examples

index <- rast(system.file("external/Prob_Map2.rst", package = "TOC"))
boolean <- rast(system.file("external/Change_Map2b.rst", package = "TOC"))
mask <- rast(system.file("external/MASK4.rst", package = "TOC"))

## thresholds can be defined by indicating the number of equal-interval thresholds 
tocd <- TOC(index, boolean, mask, nthres = 100)
tocd

## a vector of thresholds can also be used to define the thresholds
thresholds <- seq(min(unique(index)), max(unique(index)) + 1, 
                  by = ceiling(max(unique(index))/10))
tocd <- TOC(index, boolean, mask, thres = thresholds)
tocd

## all the unique values of the index object can be evaluated as thresholds 
## (default option)
## Not run: 
tocd <- TOC(index, boolean, mask, progress = TRUE)
tocd

## End(Not run)

## generate the TOC curve using non-spatial data (i.e., an object of class numeric)
## Not run: 
index <- values(index, mat = FALSE)
boolean <- values(boolean, mat = FALSE)
mask <- values(mask, mat = FALSE)
tocd <- TOC(index, boolean, mask, nthres = 100)

## End(Not run)

Toc and Roc classes

Description

Toc and Roc classes

Objects from the Class

Objects can be created by calls of the form new("Toc", ...), or with the helper functions such as Toc.

Slots

Slots for Roc and Toc objects

table:

data.frame

AUC:

numeric; Area Under the Curve

maxAUC:

numeric; maximum AUC

minAUC:

numeric; minimum AUC

prevalence:

numeric; prevalence

population:

numeric; population

units:

character; units for data in the TOC table, prevalence and population

Examples

showClass("Toc")

Plot an object of class Toc or Roc

Description

Plot a Total Operating Characteristic (TOC) curve or a Relative Operating Characteristic (ROC) curve

Usage

## S4 method for signature 'Toc'
plot(x, labelThres = FALSE, modelLeg = "Model", digits = 3, 
nticks = 5, digitsL = 1, posL = NULL, offsetL = 0.5, ...)
   

## S4 method for signature 'Roc'
plot(x, labelThres = FALSE, modelLeg = "Model", digits = 3, 
nticks = 5, digitsL = 1, posL = NULL, offsetL = 0.5, ...)

Arguments

Value

a plot showing the TOC or the ROC curve

References

Pontius Jr., R.G., Kangpin, Si. 2014. The total operating characteristic to measure diagnostic ability for multiple thresholds. International Journal of Geographical Information Science 28 (3): 570-583. <doi:10.1080/13658816.2013.862623>

Pontius, G., Parmentier, B. 2014. Recommendations for using the Relative Operating Characteristic (ROC). Landscape Ecology 29 (3): 367-382. <doi:10.1007/s10980-013-9984-8>

See Also

TOC, ROC

Examples

index <- rast(system.file("external/Prob_Map2.rst", package = "TOC"))
boolean <- rast(system.file("external/Change_Map2b.rst", package = "TOC"))
mask <- rast(system.file("external/MASK4.rst", package = "TOC"))

## create and plot the TOC curve
tocd <- TOC(index, boolean, mask, nthres = 100)
plot(tocd, main = "TOC curve")

## create and plot the ROC curve
rocd <- ROC(index, boolean, mask, nthres = 100)
plot(rocd, main = "ROC curve")

## label the thresholds in the plot
tocd <- TOC(index, boolean, mask, nthres = 10)
plot(tocd, labelThres = TRUE, cex = 0.8, posL = 4)

Construct a basic ROC table

Description

TOC internal function. Construct a basic ROC table

Usage

roctable(indval, boolval, maskval = NULL, nthres = NULL, 
         thres = NULL, NAval = 0, progress = FALSE, 
         ones.bool = NULL, zeros.bool = NULL)

Arguments

Value

a data.frame with a basic ROC table and a numeric value for minimum value in the index vector

Note

This function is not meant to be called by users directly

scale the output TOC values and change units

Description

scale the 'Hits' and 'Hits+FalseAlarms' values in the TOC output table, as well as the prevalence and population, using a scaling factor. Labels for the modified units in the TOC object are changed to newUnits

Usage

## S4 method for signature 'Toc'
scaling(x, scalingFactor, newUnits)
   

Arguments

Value

an object of class TOC

See Also

TOC, ROC

Examples

index <- rast(system.file("external/Prob_Map2.rst", package = "TOC"))
boolean <- rast(system.file("external/Change_Map2b.rst", package = "TOC"))
mask <- rast(system.file("external/MASK4.rst", package = "TOC"))
tocd <- TOC(index, boolean, mask, nthres = 100)
plot(tocd)

## scale units from square m to square km
tocd_sqkm <- scaling(tocd, scalingFactor = 1000000, newUnits = "square km")
plot(tocd_sqkm) 

Uncertainty in AUC calculation

Description

TOC internal function. It calculates uncertainty in AUC calculation

Usage

uncertainty(index, tocd)

Arguments

Value

a numeric value representing uncertainty in AUC calculation

Note

This function is not meant to be called by users directly