Clustering the classification trees in a forest based on similarities

Description

A function to get insight into a forest of classification trees by clustering the trees in a forest using Partitioning Around Medoids (PAM, Kaufman & Rousseeuw, 2009), based on user provided similarities, or based on similarities calculated by the package using a similarity measure chosen by the user (see Sies & Van Mechelen, 2020).

Usage

clusterforest(
  observeddata,
  treedata = NULL,
  trees,
  simmatrix = NULL,
  m = NULL,
  tol = NULL,
  weight = NULL,
  fromclus = 1,
  toclus = 1,
  treecov = NULL,
  sameobs = FALSE,
  seed = NULL,
  no_cores = detectCores(logical = FALSE)
)

Arguments

Details

The user should provide the number of clusters that the solution should contain, or a range of numbers that should be explored. In the latter case, the resulting clusterforest object will contain clustering results for each solution. On this clusterforest object, several methods, such as plot, print and summary, can be used.

Value

The function returns an object of class clusterforest, with attributes:

References

Kaufman, L., & Rousseeuw, P. J. (2009). Finding groups in data: an introduction to cluster analysis (Vol. 344). John Wiley & Sons.

Sies, A. & Van Mechelen I. (2020). C443: An R-package to see a forest for the trees. Journal of Classification.

Shannon, W. D., & Banks, D. (1999). Combining classification trees using MLE. Statistics in medicine, 18(6), 727-740.

Chipman, H. A., George, E. I., & McCulloh, R. E. (1998). Making sense of a forest of trees. Computing Science and Statistics, 84-92.

Examples

require(MASS)
require(ranger)
require(rpart)
#Function to draw a bootstrap sample from a dataset
DrawBoots <- function(dataset, i){
set.seed(2394 + i)
Boot <- dataset[sample(1:nrow(dataset), size = nrow(dataset), replace = TRUE),]
return(Boot)
}

#Function to grow a tree using rpart on a dataset
GrowTree <- function(x,y,BootsSample, minsplit = 40, minbucket = 20, maxdepth =3){
 controlrpart <- rpart.control(minsplit = minsplit, minbucket = minbucket, maxdepth = maxdepth,
  maxsurrogate = 0, maxcompete = 0)
 tree <- rpart(as.formula(paste(noquote(paste(y, "~")), noquote(paste(x, collapse="+")))),
 data = BootsSample, control = controlrpart)
 return(tree)
}

#Use functions to draw 10 boostrapsamples and grow a tree on each sample
Boots<- lapply(1:10, function(k) DrawBoots(Pima.tr ,k))
Trees <- lapply(1:10, function (i) GrowTree(x=c("npreg", "glu",  "bp",  "skin",
"bmi", "ped", "age"), y="type", Boots[[i]] ))

#Clustering the trees in this forest
ClusterForest<- clusterforest(observeddata=Pima.tr,treedata=Boots,trees=Trees,m=1,
fromclus=1, toclus=2, sameobs=FALSE, no_cores=2)

#Example RandomForest
Pima.tr.ranger <- ranger(type ~ ., data = Pima.tr, keep.inbag = TRUE, num.trees=20,
max.depth=3)

ClusterForest<- clusterforest(observeddata=Pima.tr,trees=Pima.tr.ranger,m=5,
                           fromclus=1, toclus=2, sameobs=FALSE, no_cores=2)

Get the cluster assignments for a solution of a clusterforest object

Description

A function to get the cluster assignments for a given solution of a clusterforest object.

Usage

clusters(clusterforest, solution)

Arguments

Get the cluster assignments for a solution of a clusterforest object

Description

A function to get the cluster assignments for a given solution of a clusterforest object.

Usage

## S3 method for class 'clusterforest'
clusters(clusterforest, solution = 1)

Arguments

Get the cluster assignments for a solution of a clusterforest object

Description

A function to get the cluster assignments for a given solution of a clusterforest object.

Usage

## Default S3 method:
clusters(clusterforest, solution)

Arguments

Drug consumption data set

Description

A dataset collected by Fehrman et al. (2017), freely available on the UCI Machine Learning Repository (Lichman, 2013) containing records of 1885 respondents regarding their use of 18 types of drugs, and their measurements on 12 predictors. #' All predictors were originally categorical and were quantified by Fehrman et al. (2017). The meaning of the values can be found on https://archive.ics.uci.edu/dataset/373/drug+consumption+quantified. The original response categories for each drug were: never used the drug, used it over a decade ago, or in the last decade, year, month, week, or day. We transformed these into binary response categories, where 0 (non-user) consists of the categories never used the drug and used it over a decade ago and 1 (user) consists of all other categories.

Usage

drugs

Format

A data frame with 1185 rows and 32 variables:

ID

Respondent ID

Age

Age of respondent

Gender

Gender of respondent, where 0.48 denotes female and -0.48 denotes male

Edu

Level of education of participant

Country

Country of current residence of participant

Ethn

Ethnicity of participant

Neuro

NEO-FFI-R Neuroticism score

Extr

NEO-FFI-R Extraversion score

Open

NEO-FFI-R Openness to experience score

Agree

NEO-FFI-R Agreeableness score

Consc

NEO-FFI-R Conscientiousness score

Impul

Impulsiveness score measured by BIS-11

Sensat

Sensation seeking score measured by ImpSS

Alc

Alcohol user (1) or non-user (0)

Amphet

Amphetamine user (1) or non-user (0)

Amyl

Amyl nitrite user (1) or non-user (0)

Benzos

Benzodiazepine user (1) or non-user (0)

Caff

Caffeine user (1) or non-user (0)

Can

Cannabis user (1) or non-user (0)

Choco

Chocolate user (1) or non-user (0)

Coke

Coke user (1) or non-user (0)

Crack

Crack user (1) or non-user (0)

Ecst

Ecstacy user (1) or non-user (0)

Her

Heroin user (1) or non-user (0)

Ket

Ketamine user (1) or non-user (0)

Leghighs

Legal Highs user (1) or non-user (0)

LSD

LSD user (1) or non-user (0)

Meth

Methadone user (1) or non-user (0)

Mush

Magical Mushroom user (1) or non-user (0)

Nico

Nicotine user (1) or non-user (0)

Semeron

Semeron user (1) or non-user (0), fictitious drug to identify over-claimers

VSA

volatile substance abuse user(1) or non-user (0)

Source

https://archive.ics.uci.edu/dataset/373/drug+consumption+quantified

References

Fehrman, E., Muhammad, A. K., Mirkes, E. M., Egan, V., & Gorban, A. N. (2017). The Five Factor Model of personality and evaluation of drug consumption risk. In Data Science (pp. 231-242). Springer, Cham. Lichman, M. (2013). UCI machine learning repository.

Get the medoid trees for a solution of a clusterforest object

Description

A function to get the medoid trees for a given solution of a clusterforest object.

Usage

medoidtrees(clusterforest, solution)

Arguments

Get the medoid trees for a solution of a clusterforest object

Description

A function to get the medoid trees for a given solution of a clusterforest object.

Usage

## S3 method for class 'clusterforest'
medoidtrees(clusterforest, solution = 1)

Arguments

Get the medoid trees for a solution of a clusterforest object

Description

A function to get the medoid trees for a given solution of a clusterforest object.

Usage

## Default S3 method:
medoidtrees(clusterforest, solution)

Arguments

Plot a clusterforest object

Description

A function that can be used to plot a clusterforest object, either by returning plots with information such as average silhouette width and within cluster siiliarity on the cluster solutions, or plots of the medoid trees of each solution.

Usage

## S3 method for class 'clusterforest'
plot(x, solution = NULL, predictive_plots = FALSE, ...)

Arguments

Details

This function can be used to plot a clusterforest object in two ways. If it's used without specifying a solution, then the average silhouette width, and within cluster similarity measures are plotted for each solution. If additionally, predictive_plots=TRUE, two more plots are returned, namely a plot showing for each solution the predictive accuracy when making predictions based on the medoid trees, and a plot showing for each solution the agreement between the class label for each object predicted on the basis of the random forest as a whole versus based on the medoid trees. These plots may be helpful in deciding how many clusters are needed to summarize the forest (see Sies & Van Mechelen, 2020).

If the function is used with the clusterforest object and the number of the solution, then the medoid tree(s) of that solution are plotted.

References

Sies, A. & Van Mechelen I. (2020). C443: An R-package to see a forest for the trees. Journal of Classification.

Examples

require(MASS)
require(rpart)
#Function to draw a bootstrap sample from a dataset
DrawBoots <- function(dataset, i){
set.seed(2394 + i)
Boot <- dataset[sample(1:nrow(dataset), size = nrow(dataset), replace = TRUE),]
return(Boot)
}

#Function to grow a tree using rpart on a dataset
GrowTree <- function(x,y,BootsSample, minsplit = 40, minbucket = 20, maxdepth =3){
 controlrpart <- rpart.control(minsplit = minsplit, minbucket = minbucket,
 maxdepth = maxdepth, maxsurrogate = 0, maxcompete = 0)
 tree <- rpart(as.formula(paste(noquote(paste(y, "~")),
 noquote(paste(x, collapse="+")))), data = BootsSample,
 control = controlrpart)
 return(tree)
}

#Use functions to draw 20 boostrapsamples and grow a tree on each sample
Boots<- lapply(1:10, function(k) DrawBoots(Pima.tr ,k))
Trees <- lapply(1:10, function (i) GrowTree(x=c("npreg", "glu",  "bp",
 "skin",  "bmi", "ped", "age"), y="type",
Boots[[i]] ))

ClusterForest<- clusterforest(observeddata=Pima.tr,treedata=Boots,trees=Trees,m=1,
fromclus=1, toclus=5, sameobs=FALSE, no_cores=2)
plot(ClusterForest)
plot(ClusterForest,2)

Print a clusterforest object

Description

A function that can be used to print a clusterforest object.

Usage

## S3 method for class 'clusterforest'
print(x, solution = 1, ...)

Arguments

Summarize a clusterforest object

Description

A function to summarize a clusterforest object.

Usage

## S3 method for class 'clusterforest'
summary(object, ...)

Arguments

Get the similarity matrix that wast used to create a clusterforest object

Description

A function to get the similarity matrix used to obtain a clusterforest object.

Usage

treesimilarities(clusterforest)

Arguments

Get the similarity matrix that wast used to create a clusterforest object

Description

A function to get the similarity matrix used to obtain a clusterforest object.

Usage

## S3 method for class 'clusterforest'
treesimilarities(clusterforest)

Arguments

Get the similarity matrix that wast used to create a clusterforest object

Description

A function to get the similarity matrix used to obtain a clusterforest object.

Usage

## Default S3 method:
treesimilarities(clusterforest)

Arguments

Mapping the tree clustering solution to a known source of variation underlying the forest

Description

A function that can be used to get insight into a clusterforest solution, in the case that there are known sources of variation underlying the forest. These known sources of variation must be included in the clusterforest object (and thus must be defined when running the clusterforest function) In case of a categorical covariate, it visualizes the number of trees from each value of the covariate that belong to each cluster. In case of a continuous covariate, it returns the mean and standard deviation of the covariate in each cluster.

Usage

treesource(clusterforest, solution)

Arguments

Value

Examples

require(rpart)
data_Amphet <-drugs[,c ("Amphet","Age", "Gender", "Edu", "Neuro", "Extr", "Open", "Agree",
"Consc", "Impul","Sensat")]
data_cocaine <-drugs[,c ("Coke","Age", "Gender", "Edu", "Neuro", "Extr", "Open", "Agree",
                         "Consc", "Impul","Sensat")]


#Function to draw a bootstrap sample from a dataset
DrawBoots <- function(dataset, i){
set.seed(2394 + i)
Boot <- dataset[sample(1:nrow(dataset), size = nrow(dataset), replace = TRUE),]
return(Boot)
}

#Function to grow a tree using rpart on a dataset
GrowTree <- function(x,y,BootsSample, minsplit = 40, minbucket = 20, maxdepth =3){

 controlrpart <- rpart.control(minsplit = minsplit, minbucket = minbucket, maxdepth = maxdepth,
 maxsurrogate = 0, maxcompete = 0)
 tree <- rpart(as.formula(paste(noquote(paste(y, "~")), noquote(paste(x, collapse="+")))),
  data = BootsSample, control = controlrpart)
 return(tree)
}

#Draw bootstrap samples and grow trees
BootsA<- lapply(1:5, function(k) DrawBoots(data_Amphet,k))
BootsC<- lapply(1:5, function(k) DrawBoots(data_cocaine,k))
Boots = c(BootsA,BootsC)

TreesA <- lapply(1:5, function (i) GrowTree(x=c ("Age", "Gender", "Edu", "Neuro",
"Extr", "Open", "Agree","Consc", "Impul","Sensat"), y="Amphet", BootsA[[i]] ))
TreesC <- lapply(1:5, function (i) GrowTree(x=c ( "Age", "Gender", "Edu", "Neuro",
"Extr", "Open", "Agree", "Consc", "Impul","Sensat"), y="Coke", BootsC[[i]] ))
Trees=c(TreesA,TreesC)

#Cluster the trees
ClusterForest<- clusterforest(observeddata=drugs,treedata=Boots,trees=Trees,m=1,
fromclus=2, toclus=2, treecov=rep(c("Amphet","Coke"),each=5), sameobs=FALSE, no_cores=2)

#Link cluster result to known source of variation
treesource(ClusterForest, 2)

Mapping the tree clustering solution to a known source of variation underlying the forest

Description

A function that can be used to get insight into a clusterforest solution, in the case that there is a known source of variation underlying the forest. It visualizes the number of trees from each source that belong to each cluster.

Usage

## S3 method for class 'clusterforest'
treesource(clusterforest, solution)

Arguments

Mapping the tree clustering solution to a known source of variation underlying the forest

Description

A function that can be used to get insight into a clusterforest solution, in the case that there is a known source of variation underlying the forest. It visualizes the number of trees from each source that belong to each cluster.

Usage

## Default S3 method:
treesource(clusterforest, solution)

Arguments