Title: Adaptive Trial Simulator
Version: 1.4.0
Date: 2024-05-03
Description: Package that simulates adaptive (multi-arm, multi-stage) clinical trials using adaptive stopping, adaptive arm dropping, and/or adaptive randomisation. Developed as part of the INCEPT (Intensive Care Platform Trial) project (https://incept.dk/), primarily supported by a grant from Sygeforsikringen "danmark" (https://www.sygeforsikring.dk/).
License: GPL (≥ 3)
Imports: stats, parallel, utils
Encoding: UTF-8
Language: en-GB
NeedsCompilation: no
URL: https://inceptdk.github.io/adaptr/, https://github.com/INCEPTdk/adaptr/, https://incept.dk/
BugReports: https://github.com/INCEPTdk/adaptr/issues/
RoxygenNote: 7.3.1
Suggests: ggplot2, covr, rmarkdown, knitr, testthat, vdiffr
VignetteBuilder: knitr
Config/testthat/edition: 3
Packaged: 2024-05-03 11:21:53 UTC; agra0037
Author: Anders Granholm ORCID iD [aut, cre], Benjamin Skov Kaas-Hansen ORCID iD [aut], Aksel Karl Georg Jensen ORCID iD [ctb], Theis Lange ORCID iD [ctb]
Maintainer: Anders Granholm <andersgran@gmail.com>
Repository: CRAN
Date/Publication: 2024-05-03 12:10:02 UTC

adaptr: Adaptive Trial Simulator

Description

logo Adaptive Trial Simulator

The adaptr package simulates adaptive (multi-arm, multi-stage) randomised clinical trials using adaptive stopping, adaptive arm dropping and/or response-adaptive randomisation. The package is developed as part of the INCEPT (Intensive Care Platform Trial) project, funded primarily by a grant from Sygeforsikringen "danmark".

Details

The adaptr package contains the following primary functions (in order of typical use):

  1. The setup_cluster() initiates a parallel computation cluster that can be used to run simulations and post-processing in parallel, increasing speed. Details on parallelisation and other options for running adaptr functions in parallel are described in the setup_cluster() documentation.

  2. The setup_trial() function is the general function that sets up a trial specification. The simpler, special-case functions setup_trial_binom() and setup_trial_norm() may be used for easier specification of trial designs using binary, binomially distributed or continuous, normally distributed outcomes, respectively, with some limitations in flexibility.

  3. The calibrate_trial() function calibrates a trial specification to obtain a certain value for a performance metric (typically used to calibrate the Bayesian type 1 error rate in a scenario with no between-arm differences), using the functions below.

  4. The run_trial() and run_trials() functions are used to conduct single or multiple simulations, respectively, according to a trial specification setup as described in #2.

  5. The extract_results(), check_performance() and summary() functions are used to extract results from multiple trial simulations, calculate performance metrics, and summarise results. The plot_convergence() function assesses stability of performance metrics according to the number of simulations conducted. The plot_metrics_ecdf() function plots empirical cumulative distribution functions for numerical performance metrics. The check_remaining_arms() function summarises all combinations of remaining arms across multiple trials simulations.

  6. The plot_status() and plot_history() functions are used to plot the overall trial/arm statuses for multiple simulated trials or the history of trial metrics over time for single/multiple simulated trials, respectively.

For further information see the documentation of each function or the Overview vignette (vignette("Overview", package = "adaptr")) for an example of how the functions work in combination. For further examples and guidance on setting up trial specifications, see the setup_trial() documentation, the Basic examples vignette (vignette("Basic-examples", package = "adaptr")) and the Advanced example vignette (vignette("Advanced-example", package = "adaptr")).

If using the package, please consider citing it using citation(package = "adaptr").

Author(s)

Maintainer: Anders Granholm andersgran@gmail.com (ORCID)

Authors:

Other contributors:

References

Granholm A, Jensen AKG, Lange T, Kaas-Hansen BS (2022). adaptr: an R package for simulating and comparing adaptive clinical trials. Journal of Open Source Software, 7(72), 4284. doi:10.21105/joss.04284

Granholm A, Kaas-Hansen BS, Lange T, Schjørring OL, Andersen LW, Perner A, Jensen AKG, Møller MH (2022). An overview of methodological considerations regarding adaptive stopping, arm dropping and randomisation in clinical trials. J Clin Epidemiol. doi:10.1016/j.jclinepi.2022.11.002

Website/manual

GitHub repository

Examples of studies using adaptr:

Granholm A, Lange T, Harhay MO, Jensen AKG, Perner A, Møller MH, Kaas-Hansen BS (2023). Effects of duration of follow-up and lag in data collection on the performance of adaptive clinical trials. Pharm Stat. doi:10.1002/pst.2342

Granholm A, Lange T, Harhay MO, Perner A, Møller MH, Kaas-Hansen BS (2024). Effects of sceptical priors on the performance of adaptive clinical trials with binary outcomes. Pharm Stat. doi:10.1002/pst.2387

See Also

setup_cluster(), setup_trial(), setup_trial_binom(), setup_trial_norm(), calibrate_trial(), run_trial(), run_trials(), extract_results(), check_performance(), summary(), check_remaining_arms(), plot_convergence(), plot_metrics_ecdf(), print(), plot_status(), plot_history().


Check availability of required packages

Description

Used internally, helper function to check if SUGGESTED packages are available. Will halt execution if any of the queried packages are not available and provide installation instructions.

Usage

assert_pkgs(pkgs = NULL)

Arguments

pkgs

character vector with name(s) of package(s) to check.

Value

TRUE if all packages available, otherwise execution is halted with an error.


Calculate the ideal design percentage

Description

Used internally by check_performance(), calculates the ideal design percentage as described in that function's documentation.

Usage

calculate_idp(sels, arms, true_ys, highest_is_best)

Arguments

sels

a character vector specifying the selected arms (according to the selection strategies described in extract_results()).

arms

character vector with unique names for the trial arms.

true_ys

numeric vector specifying true outcomes (e.g., event probabilities, mean values, etc.) for all trial arms.

highest_is_best

single logical, specifies whether larger estimates of the outcome are favourable or not; defaults to FALSE, corresponding to, e.g., an undesirable binary outcomes (e.g., mortality) or a continuous outcome where lower numbers are preferred (e.g., hospital length of stay).

Value

A single numeric value between 0 and 100 corresponding to the ideal design percentage.


Calibrate trial specification

Description

This function calibrates a trial specification using a Gaussian process-based Bayesian optimisation algorithm. The function calibrates an input trial specification object (using repeated calls to run_trials() while adjusting the trial specification) to a target value within a search_range in a single input dimension (x) in order to find an optimal value (y).
The default (and expectedly most common use case) is to calibrate a trial specification to adjust the superiority and inferiority thresholds to obtain a certain probability of superiority; if used with a trial specification with identical underlying outcomes (no between-arm differences), this probability is an estimate of the Bayesian analogue of the total type-1 error rate for the outcome driving the adaptations, and if between-arm differences are present, this corresponds to an estimate of the Bayesian analogue of the power.
The default is to perform the calibration while varying single, constant, symmetric thresholds for superiority / inferiority throughout a trial design, as described in Details, and the default values have been chosen to function well in this case.
Advanced users may use the function to calibrate trial specifications according to other metrics - see Details for how to specify a custom function used to modify (or recreate) a trial specification object during the calibration process.
The underlying Gaussian process model and its control hyperparameters are described under Details, and the model is partially based on code from Gramacy 2020 (with permission; see References).

Usage

calibrate_trial(
  trial_spec,
  n_rep = 1000,
  cores = NULL,
  base_seed = NULL,
  fun = NULL,
  target = 0.05,
  search_range = c(0.9, 1),
  tol = target/10,
  dir = 0,
  init_n = 2,
  iter_max = 25,
  resolution = 5000,
  kappa = 0.5,
  pow = 1.95,
  lengthscale = 1,
  scale_x = TRUE,
  noisy = is.null(base_seed),
  narrow = !noisy & !is.null(base_seed),
  prev_x = NULL,
  prev_y = NULL,
  path = NULL,
  overwrite = FALSE,
  version = NULL,
  compress = TRUE,
  sparse = TRUE,
  progress = NULL,
  export = NULL,
  export_envir = parent.frame(),
  verbose = FALSE,
  plot = FALSE
)

Arguments

trial_spec

trial_spec object, generated and validated by the setup_trial(), setup_trial_binom() or setup_trial_norm() function.

n_rep

single integer, the number of simulations to run at each evaluation. Values ⁠< 100⁠ are not permitted; values ⁠< 1000⁠ are permitted but recommended against.

cores

NULL or single integer. If NULL, a default value/cluster set by setup_cluster() will be used to control whether simulations are run in parallel on a default cluster or sequentially in the main process; if a cluster/value has not been specified by setup_cluster(), cores will then be set to the value stored in the global "mc.cores" option (if previously set by ⁠options(mc.cores = <number of cores>⁠), and 1 if that option has not been specified.
If the resulting number of cores = 1, computations will be run sequentially in the primary process, and if cores > 1, a new parallel cluster will be setup using the parallel library and removed once the function completes. See setup_cluster() for details.

base_seed

single integer or NULL (default); the random seed used as the basis for all simulation runs (see run_trials()) and random number generation within the rest of the calibration process; if used, the global random seed will be restored after the function has been run.
Note: providing a base_seed is highly recommended, as this will generally lead to faster and more stable calibration.

fun

NULL (the default), in which case the trial specification will be calibrated using the default process described above and further in Details; otherwise a user-supplied function used during the calibration process, which should have a structure as described in Details.

target

single finite numeric value (defaults to 0.05); the target value for y to calibrate the trial_spec object to.

search_range

finite numeric vector of length 2; the lower and upper boundaries in which to search for the best x. Defaults to c(0.9, 1.0).

tol

single finite numeric value (defaults to target / 10); the accepted tolerance (in the direction(s) specified by dir) accepted; when a y-value within the accepted tolerance of the target is obtained, the calibration stops.
Note: tol should be specified to be sensible considering n_rep; e.g., if the probability of superiority is targeted with n_rep == 1000, a tol of 0.01 will correspond to 10 simulated trials.
A too low tol relative to n_rep may lead to very slow calibration or calibration that cannot succeed regardless of the number of iterations.
Important: even when a large number of simulations are conducted, using a very low tol may lead to calibration not succeeding as it may also be affected by other factors, e.g., the total number of simulated patients, the possible maximum differences in simulated outcomes, and the number of posterior draws (n_draws in the setup_trial() family of functions), which affects the minimum differences in posterior probabilities when simulating trials and thus can affect calibration, including when using the default calibration function. Increasing the number of posterior draws or the number of repetitions should be attempted if the desired tolerance cannot be achieved with lower numbers.

dir

single numeric value; specifies the direction(s) of the tolerance range. If 0 (the default) the tolerance range will be target - tol to target + tol. If ⁠< 0⁠, the range will be target - tol to target, and if ⁠> 0⁠, the range will be target to target + tol.

init_n

single integer ⁠>= 2⁠. The number of initial evaluations evenly spread over the search_range, with one evaluation at each boundary (thus, the default value of 2 is the minimum permitted value; if calibrating according to a different target than the default, a higher value may be sensible).

iter_max

single integer ⁠> 0⁠ (default 25). The maximum number of new evaluations after the initial grid (with size specified by init_n) has been set up. If calibration is unsuccessful after the maximum number of iterations, the prev_x and prev_y arguments (described below) may be used to to start a new calibration process re-using previous evaluations.

resolution

single integer (defaults to 5000), size of the grid at which the predictions used to select the next value to evaluate at are made.
Note: memory use will substantially increase with higher values. See also the narrow argument below.

kappa

single numeric value ⁠> 0⁠ (default 0.5); corresponding to the width of the uncertainty bounds used to find the next target to evaluate. See Details.

pow

single numerical value in the ⁠[1, 2]⁠ range (default 1.95), controlling the smoothness of the Gaussian process. See Details.

lengthscale

single numerical value (defaults to 1) or numerical vector of length 2; values must be finite and non-negative. If a single value is provided, this will be used as the lengthscale hyperparameter; if a numerical vector of length 2 is provided, the second value must be higher than the first and the optimal lengthscale in this range will be found using an optimisation algorithm. If any value is 0, a small amount of noise will be added as lengthscales must be ⁠> 0⁠. Controls smoothness in combination with pow. See Details.

scale_x

single logical value; if TRUE (the default) the x-values will be scaled to the ⁠[0, 1]⁠ range according to the minimum/maximum values provided. If FALSE, the model will use the original scale. If distances on the original scale are small, scaling may be preferred. The returned values will always be on the original scale. See Details.

noisy

single logical value; if FALSE, a noiseless process is assumed, and interpolation between values is performed (i.e., with no uncertainty at the x-values assumed). If TRUE, the y-values are assumed to come from a noisy process, and regression is performed (i.e., some uncertainty at the evaluated x-values will be assumed and included in the predictions). Specifying FALSE requires a base_seed supplied, and is generally recommended, as this will usually lead to faster and more stable calibration. If a low n_rep is used (or if trials are calibrated to other metrics other than the default), specifying TRUE may be necessary even when using a valid base_seed. Defaults to TRUE if a base_seed is supplied and FALSE if not.

narrow

single logical value. If FALSE, predictions are evenly spread over the full x-range. If TRUE, the prediction grid will be spread evenly over an interval consisting of the two x-values with corresponding y-values closest to the target in opposite directions. Can only be TRUE when a base_seed is provided and noisy is FALSE (the default value is TRUE in that case, otherwise it is FALSE), and only if the function can safely be assumed to be only monotonically increasing or decreasing (which is generally reasonable if the default is used for fun), in which case this will lead to a faster search and a smoother prediction grid in the relevant region without increasing memory use.

prev_x, prev_y

numeric vectors of equal lengths, corresponding to previous evaluations. If provided, these will be used in the calibration process (added before the initial grid is setup, with values in the grid matching values in prev_x leading to those evaluations being skipped).

path

single character string or NULL (the default); if a valid file path is provided, the calibration results will either be saved to this path (if the file does not exist or if overwrite is TRUE, see below) or the previous results will be loaded and returned (if the file exists, overwrite is FALSE, and if the input trial_spec and central control settings are identical to the previous run, otherwise an error is produced). Results are saved/loaded using the saveRDS() / readRDS() functions.

overwrite

single logical, defaults to FALSE, in which case previous results are loaded if a valid file path is provided in path and the object in path contains the same input trial_spec and the previous calibration used the same central control settings (otherwise, the function errors). If TRUE and a valid file path is provided in path, the complete calibration function will be run with results saved using saveRDS(), regardless of whether or not a previous result was saved in path.

version

passed to saveRDS() when saving calibration results, defaults to NULL (as in saveRDS()), which means that the current default version is used. Ignored if calibration results are not saved.

compress

passed to saveRDS() when saving calibration results, defaults to TRUE (as in saveRDS()), see saveRDS() for other options. Ignored if calibration results are not saved.

sparse, progress, export, export_envir

passed to run_trials(), see description there.

verbose

single logical, defaults to FALSE. If TRUE, the function will print details on calibration progress.

plot

single logical, defaults to FALSE. If TRUE, the function will print plots of the Gaussian process model predictions and return them as part of the final object; requires the ggplot2 package installed.

Details

Default calibration

If fun is NULL (as default), the default calibration strategy will be employed. Here, the target y is the probability of superiority (as described in check_performance() and summary()), and the function will calibrate constant stopping thresholds for superiority and inferiority (as described in setup_trial(), setup_trial_binom(), and setup_trial_norm()), which corresponds to the Bayesian analogues of the type 1 error rate if there are no differences between arms in the trial specification, which we expect to be the most common use case, or the power, if there are differences between arms in the trial specification.

The stopping calibration process will, in the default case, use the input x as the stopping threshold for superiority and 1 - x as the stopping threshold for inferiority, respectively, i.e., stopping thresholds will be constant and symmetric.

The underlying default function calibrated is typically essentially noiseless if a high enough number of simulations are used with an appropriate random base_seed, and generally monotonically decreasing. The default values for the control hyperparameters have been set to normally work well in this case (including init_n, kappa, pow, lengthscale, narrow, scale_x, etc.). Thus, few initial grid evaluations are used in this case, and if a base_seed is provided, a noiseless process is assumed and narrowing of the search range with each iteration is performed, and the uncertainty bounds used in the acquisition function (corresponding to quantiles from the posterior predictive distribution) are relatively narrow.

Specifying calibration functions

A user-specified calibration function should have the following structure:

# The function must take the arguments x and trial_spec
# trial_spec is the original trial_spec object which should be modified
# (alternatively, it may be re-specified, but the argument should still
# be included, even if ignored)
function(x, trial_spec) {
  # Calibrate trial_spec, here as in the default function
  trial_spec$superiority <- x
  trial_spec$inferiority <- 1 - x

  # If relevant, known y values corresponding to specific x values may be
  # returned without running simulations (here done as in the default
  # function). In that case, a code block line the one below can be included,
  # with changed x/y values - of note, the other return values should not be
  # changed
  if (x == 1) {
    return(list(sims = NULL, trial_spec = trial_spec, y = 0))
  }

  # Run simulations - this block should be included unchanged
  sims <- run_trials(trial_spec, n_rep = n_rep, cores = cores,
                     base_seed = base_seed, sparse = sparse,
                     progress = progress, export = export,
                     export_envir = export_envir)

 # Return results - only the y value here should be changed
 # summary() or check_performance() will often be used here
 list(sims = sims, trial_spec = trial_spec,
      y = summary(sims)$prob_superior)
}

Note: changes to the trial specification are not validated; users who define their own calibration function need to ensure that changes to calibrated trial specifications does not lead to invalid values; otherwise, the procedure is prone to error when simulations are run. Especially, users should be aware that changing true_ys in a trial specification generated using the simplified setup_trial_binom() and setup_trial_norm() functions requires changes in multiple places in the object, including in the functions used to generate random outcomes, and in these cases (and otherwise if in doubt) re-generating the trial_spec instead of modifying should be preferred as this is safer and leads to proper validation.

Note: if the y values corresponding to certain x values are known, then the user may directly return these values without running simulations (e.g., in the default case an x of 1 will require ⁠>100%⁠ or ⁠<0%⁠ probabilities for stopping rules, which is impossible, and hence the y value in this case is by definition 1).

Gaussian process optimisation function and control hyperparameters

The calibration function uses a relatively simple Gaussian optimisation function with settings that should work well for the default calibration function, but can be changed as required, which should be considered if calibrating according to other targets (effects of using other settings may be evaluated in greater detail by setting verbose and plot to TRUE).
The function may perform both interpolation (i.e., assuming a noiseless, deterministic process with no uncertainty at the values already evaluated) or regression (i.e., assuming a noisy, stochastic process), controlled by the noisy argument.

The covariance matrix (or kernel) is defined as:

⁠exp(-||x - x'||^pow / lengthscale)⁠

with ⁠||x -x'||⁠ corresponding to a matrix containing the absolute Euclidean distances of values of x (and values on the prediction grid), scaled to the ⁠[0, 1]⁠ range if scale_x is TRUE and on their original scale if FALSE. Scaling i generally recommended (as this leads to more comparable and predictable effects of pow and lengthscale, regardless of the true scale), and also recommended if the range of values is smaller than this range. The absolute distances are raised to the power pow, which must be a value in the ⁠[1, 2]⁠ range. Together with lengthscale, pow controls the smoothness of the Gaussian process model, with 1 corresponding to less smoothing (i.e., piecewise straight lines between all evaluations if lengthscale is 1) and values ⁠> 1⁠ corresponding to more smoothing. After raising the absolute distances to the chosen power pow, the resulting matrix is divided by lengthscale. The default is 1 (no change), and values ⁠< 1⁠ leads to faster decay in correlations and thus less smoothing (more wiggly fits), and values ⁠> 1⁠ leads to more smoothing (less wiggly fits). If a single specific value is supplied for lengthscale this is used; if a range of values is provided, a secondary optimisation process determines the value to use within that range.

Some minimal noise ("jitter") is always added to the diagonals of the matrices where relevant to ensure numerical stability; if noisy is TRUE, a "nugget" value will be determined using a secondary optimisation process

Predictions will be made over an equally spaced grid of x values of size resolution; if narrow is TRUE, this grid will only be spread out between the x values with corresponding y values closest to and below and closes to and above target, respectively, leading to a finer grid in the range of relevance (as described above, this should only be used for processes that are assumed to be noiseless and should only be used if the process can safely be assumed to be monotonically increasing or decreasing within the search_range). To suggest the next x value for evaluations, the function uses an acquisition function based on bi-directional uncertainty bounds (posterior predictive distributions) with widths controlled by the kappa hyperparameter. Higher kappa/wider uncertainty bounds leads to increased exploration (i.e., the algorithm is more prone to select values with high uncertainty, relatively far from existing evaluations), while lower kappa/narrower uncertainty bounds leads to increased exploitation (i.e., the algorithm is more prone to select values with less uncertainty, closer to the best predicted mean values). The value in the x grid leading with one of the boundaries having the smallest absolute distance to the target is chosen (within the narrowed range, if narrow is TRUE). See Greenhill et al, 2020 under References for a general description of acquisition functions.

IMPORTANT: we recommend that control hyperparameters are explicitly specified, even for the default calibration function. Although the default values should be sensible for the default calibration function, these may change in the future. Further, we generally recommend users to perform small-scale comparisons (i.e., with fewer simulations than in the final calibration) of the calibration process with different hyperparameters for specific use cases beyond the default (possibly guided by setting the verbose and plot options to TRUE) before running a substantial number of calibrations or simulations, as the exact choices may have important influence on the speed and likelihood of success of the calibration process.
It is the responsibility of the user to specify sensible values for the settings and hyperparameters.

Value

A list of special class "trial_calibration", which contains the following elements that can be extracted using $ or [[:

References

Gramacy RB (2020). Chapter 5: Gaussian Process Regression. In: Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences. Chapman Hall/CRC, Boca Raton, Florida, USA. Available online.

Greenhill S, Rana S, Gupta S, Vellanki P, Venkatesh S (2020). Bayesian Optimization for Adaptive Experimental Design: A Review. IEEE Access, 8, 13937-13948. doi:10.1109/ACCESS.2020.2966228

Examples

## Not run: 
# Setup a trial specification to calibrate
# This trial specification has similar event rates in all arms
# and as the default calibration settings are used, this corresponds to
# assessing the Bayesian type 1 error rate for this design and scenario
binom_trial <- setup_trial_binom(arms = c("A", "B"),
                                 true_ys = c(0.25, 0.25),
                                 data_looks = 1:5 * 200)

# Run calibration using default settings for most parameters
res <- calibrate_trial(binom_trial, n_rep = 1000, base_seed = 23)

# Print calibration summary result
res

## End(Not run)


cat() with sep = ""

Description

Used internally. Passes everything on to cat() but enforces sep = "". Relates to cat() as paste0() relates to paste().

Usage

cat0(...)

Arguments

...

strings to be concatenated and printed.


Check performance metrics for trial simulations

Description

Calculates performance metrics for a trial specification based on simulation results from the run_trials() function, with bootstrapped uncertainty measures if requested. Uses extract_results(), which may be used directly to extract key trial results without summarising. This function is also used by summary() to calculate the performance metrics presented by that function.

Usage

check_performance(
  object,
  select_strategy = "control if available",
  select_last_arm = FALSE,
  select_preferences = NULL,
  te_comp = NULL,
  raw_ests = FALSE,
  final_ests = NULL,
  restrict = NULL,
  uncertainty = FALSE,
  n_boot = 5000,
  ci_width = 0.95,
  boot_seed = NULL,
  cores = NULL
)

Arguments

object

trial_results object, output from the run_trials() function.

select_strategy

single character string. If a trial was not stopped due to superiority (or had only 1 arm remaining, if select_last_arm is set to TRUE in trial designs with a common control arm; see below), this parameter specifies which arm will be considered selected when calculating trial design performance metrics, as described below; this corresponds to the consequence of an inconclusive trial, i.e., which arm would then be used in practice.
The following options are available and must be written exactly as below (case sensitive, cannot be abbreviated):

  • "control if available" (default): selects the first control arm for trials with a common control arm if this arm is active at end-of-trial, otherwise no arm will be selected. For trial designs without a common control, no arm will be selected.

  • "none": selects no arm in trials not ending with superiority.

  • "control": similar to "control if available", but will throw an error if used for trial designs without a common control arm.

  • "final control": selects the final control arm regardless of whether the trial was stopped for practical equivalence, futility, or at the maximum sample size; this strategy can only be specified for trial designs with a common control arm.

  • "control or best": selects the first control arm if still active at end-of-trial, otherwise selects the best remaining arm (defined as the remaining arm with the highest probability of being the best in the last adaptive analysis conducted). Only works for trial designs with a common control arm.

  • "best": selects the best remaining arm (as described under "control or best").

  • "list or best": selects the first remaining arm from a specified list (specified using select_preferences, technically a character vector). If none of these arms are are active at end-of-trial, the best remaining arm will be selected (as described above).

  • "list": as specified above, but if no arms on the provided list remain active at end-of-trial, no arm is selected.

select_last_arm

single logical, defaults to FALSE. If TRUE, the only remaining active arm (the last control) will be selected in trials with a common control arm ending with equivalence or futility, before considering the options specified in select_strategy. Must be FALSE for trial designs without a common control arm.

select_preferences

character vector specifying a number of arms used for selection if one of the "list or best" or "list" options are specified for select_strategy. Can only contain valid arms available in the trial.

te_comp

character string, treatment-effect comparator. Can be either NULL (the default) in which case the first control arm is used for trial designs with a common control arm, or a string naming a single trial arm. Will be used when calculating err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described below).

raw_ests

single logical. If FALSE (default), the posterior estimates (post_ests or post_ests_all, see setup_trial() and run_trial()) will be used to calculate err and sq_err (the error and the squared error of the estimated compared to the specified effect in the selected arm) and err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described for te_comp and below). If TRUE, the raw estimates (raw_ests or raw_ests_all, see setup_trial() and run_trial()) will be used instead of the posterior estimates.

final_ests

single logical. If TRUE (recommended) the final estimates calculated using outcome data from all patients randomised when trials are stopped are used (post_ests_all or raw_ests_all, see setup_trial() and run_trial()); if FALSE, the estimates calculated for each arm when an arm is stopped (or at the last adaptive analysis if not before) using data from patients having reach followed up at this time point and not all patients randomised are used (post_ests or raw_ests, see setup_trial() and run_trial()). If NULL (the default), this argument will be set to FALSE if outcome data are available immediate after randomisation for all patients (for backwards compatibility, as final posterior estimates may vary slightly in this situation, even if using the same data); otherwise it will be said to TRUE. See setup_trial() for more details on how these estimates are calculated.

restrict

single character string or NULL. If NULL (default), results are summarised for all simulations; if "superior", results are summarised for simulations ending with superiority only; if "selected", results are summarised for simulations ending with a selected arm only (according to the specified arm selection strategy for simulations not ending with superiority). Some summary measures (e.g., prob_conclusive) have substantially different interpretations if restricted, but are calculated nonetheless.

uncertainty

single logical; if FALSE (default) uncertainty measures are not calculated, if TRUE, non-parametric bootstrapping is used to calculate uncertainty measures.

n_boot

single integer (default 5000); the number of bootstrap samples to use if uncertainty = TRUE. Values ⁠< 100⁠ are not allowed and values ⁠< 1000⁠ will lead to a warning, as results are likely to be unstable in those cases.

ci_width

single numeric ⁠>= 0⁠ and ⁠< 1⁠, the width of the percentile-based bootstrapped confidence intervals. Defaults to 0.95, corresponding to 95% confidence intervals.

boot_seed

single integer, NULL (default), or "base". If a value is provided, this value will be used to initiate random seeds when bootstrapping with the global random seed restored after the function has run. If "base" is specified, the base_seed specified in run_trials() is used. Regardless of whether simulations are run sequentially or in parallel, bootstrapped results will be identical if a boot_seed is specified.

cores

NULL or single integer. If NULL, a default value set by setup_cluster() will be used to control whether extractions of simulation results are done in parallel on a default cluster or sequentially in the main process; if a value has not been specified by setup_cluster(), cores will then be set to the value stored in the global "mc.cores" option (if previously set by ⁠options(mc.cores = <number of cores>⁠), and 1 if that option has not been specified.
If cores = 1, computations will be run sequentially in the primary process, and if cores > 1, a new parallel cluster will be setup using the parallel library and removed once the function completes. See setup_cluster() for details.

Details

The ideal design percentage (IDP) returned is based on Viele et al, 2020 doi:10.1177/1740774519877836 (and also described in Granholm et al, 2022 doi:10.1016/j.jclinepi.2022.11.002, which also describes the other performance measures) and has been adapted to work for trials with both desirable/undesirable outcomes and non-binary outcomes. Briefly, the expected outcome is calculated as the sum of the true outcomes in each arm multiplied by the corresponding selection probabilities (ignoring simulations with no selected arm). The IDP is then calculated as:

Value

A tidy data.frame with added class trial_performance (to control the number of digits printed, see print()), with the columns "metric" (described below), "est" (estimate of each metric), and the following four columns if uncertainty = TRUE: "err_sd"(bootstrapped SDs), "err_mad" (bootstrapped MAD-SDs, as described in setup_trial() and stats::mad()), "lo_ci", and "hi_ci", the latter two corresponding to the lower/upper limits of the percentile-based bootstrapped confidence intervals. Bootstrap estimates are not calculated for the minimum (⁠_p0⁠) and maximum values (⁠_p100⁠) of size, sum_ys, and ratio_ys, as non-parametric bootstrapping for minimum/maximum values is not sensible - bootstrap estimates for these values will be NA.
The following performance metrics are calculated:

See Also

extract_results(), summary(), plot_convergence(), plot_metrics_ecdf(), check_remaining_arms().

Examples

# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
                                 control = "A",
                                 true_ys = c(0.20, 0.18, 0.22, 0.24),
                                 data_looks = 1:20 * 100)

# Run 10 simulations with a specified random base seed
res <- run_trials(binom_trial, n_rep = 10, base_seed = 12345)

# Check performance measures, without assuming that any arm is selected in
# the inconclusive simulations, with bootstrapped uncertainty measures
# (unstable in this example due to the very low number of simulations
# summarised):
check_performance(res, select_strategy = "none", uncertainty = TRUE,
n_boot = 1000, boot_seed = "base")


Check remaining arm combinations

Description

This function summarises the numbers and proportions of all combinations of remaining arms (i.e., excluding arms dropped for inferiority or futility at any analysis, and arms dropped for equivalence at earlier analyses in trials with a common control) across multiple simulated trial results. The function supplements the extract_results(), check_performance(), and summary() functions, and is especially useful for designs with ⁠> 2⁠ arms, where it provides details that the other functions mentioned do not.

Usage

check_remaining_arms(object, ci_width = 0.95)

Arguments

object

trial_results object, output from the run_trials() function.

ci_width

single numeric ⁠>= 0⁠ and ⁠< 1⁠, the width of the approximate confidence intervals for the proportions of combinations (calculated analytically). Defaults to 0.95, corresponding to 95% confidence intervals.

Value

a data.frame containing the combinations of remaining arms, sorted in descending order of, with the following columns:

See Also

extract_results(), check_performance(), summary(), plot_convergence(), plot_metrics_ecdf().

Examples

# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
                                 control = "A",
                                 true_ys = c(0.20, 0.18, 0.22, 0.24),
                                 data_looks = 1:20 * 200,
                                 equivalence_prob = 0.7,
                                 equivalence_diff = 0.03,
                                 equivalence_only_first = FALSE)

# Run 35 simulations with a specified random base seed
res <- run_trials(binom_trial, n_rep = 25, base_seed = 12345)

# Check remaining arms (printed with fewer digits)
print(check_remaining_arms(res), digits = 3)


Estimates covariance matrices used by Gaussian process optimisation

Description

Used internally, estimates covariance matrices used by the Gaussian process optimisation function. Calculates pairwise absolute distances raised to a power (which defaults to 2) using the pow_abs_dist() function, divides the result by a lengthscale hyperparameter (which defaults to 1, i.e., no changes due to division), and subsequently returns the inverse exponentiation of the resulting matrix.

Usage

cov_mat(x1, x2 = x1, g = NULL, pow = 2, lengthscale = 1)

Arguments

x1

numeric vector, with length corresponding to the number of rows in the returned matrix.

x2

numeric vector, with length corresponding to the number of columns in the returned matrix. If not specified, x1 will be used for x2.

g

single numerical value; jitter/nugget value added to the diagonal if not NULL (the default); should be supplied if x1 is the same as x2, to avoid potentially negative values in the matrix diagonal due to numerical instability.

pow

single numeric value, the power that all distances are raised to. Defaults to 2, corresponding to pairwise, squared, Euclidean distances.

lengthscale

single numerical value; lengthscale hyperparameter that the matrix returned from pow_abs_dist() is divided by before the inverse exponentiation is done.

Value

Covariance matrix with length(x1) rows and length(x2) columns used by the Gaussian process optimiser.


Simulate single trial after setting seed

Description

Helper function to dispatch the running of several trials to lapply() or parallel::parLapply(), setting seeds correctly if a base_seed was used when calling run_trials(). Used internally in calls by the run_trials() function.

Usage

dispatch_trial_runs(is, trial_spec, seeds, sparse, cores, cl = NULL)

Arguments

is

vector of integers, the simulation numbers/indices.

trial_spec

trial specification as provided by setup_trial(), setup_trial_binom() or setup_trial_norm().

sparse

single logical, as described in run_trial(); defaults to TRUE when running multiple simulations, in which case only the data necessary to summarise all simulations are saved for each simulation. If FALSE, more detailed data for each simulation is saved, allowing more detailed printing of individual trial results and plotting using plot_history() (plot_status() does not require non-sparse results).

cores

NULL or single integer. If NULL, a default value/cluster set by setup_cluster() will be used to control whether simulations are run in parallel on a default cluster or sequentially in the main process; if a cluster/value has not been specified by setup_cluster(), cores will then be set to the value stored in the global "mc.cores" option (if previously set by ⁠options(mc.cores = <number of cores>⁠), and 1 if that option has not been specified.
If the resulting number of cores = 1, computations will be run sequentially in the primary process, and if cores > 1, a new parallel cluster will be setup using the parallel library and removed once the function completes. See setup_cluster() for details.

cl

NULL (default) for running sequentially, otherwise a parallel cluster for parallel computation if cores > 1.

Value

Single trial simulation object, as described in run_trial().


Assert equivalent functions

Description

Used internally. Compares the definitions of two functions (ignoring environments, bytecodes, etc., by only comparing function arguments and bodies, using deparse()).

Usage

equivalent_funs(fun1, fun2)

Arguments

fun1, fun2

functions to compare.

Value

Single logical.


Extract history

Description

Used internally. Extracts relevant parameters at each conducted adaptive analysis from a single trial.

Usage

extract_history(object, metric = "prob")

Arguments

object

single trial_result from run_trial(), only works if run with argument sparse = FALSE.

metric

either "prob" (default), in which case allocation probabilities at each adaptive analysis are returned; "n"/"n all", in which case the total number of patients with available follow-up data ("n") or allocated ("n all") to each arm during each adaptive analysis are returned; "pct"/"pct all" in which case the proportions of of patients allocated and having available follow-up data ("pct") or allocated in total ("pct all") to each arm out of the total number of patients are returned; "sum ys"/"sum ys all", in which case the total summed available outcome data ("sum ys") or total summed outcome data including outcomes of patients randomised that have not necessarily reached follow-up yet ("sum ys all") in each arm after each adaptive analysis are returned; or "ratio ys"/"ratio ys all", in which case the total summed outcomes as specified for "sum ys"/"sum ys all" divided by the number of patients after each analysis adaptive are returned.

Value

A tidy data.frame (one row per arm per look) containing the following columns:


Extract simulation results

Description

This function extracts relevant information from multiple simulations of the same trial specification in a tidy data.frame (1 simulation per row). See also the check_performance() and summary() functions, that uses the output from this function to further summarise simulation results.

Usage

extract_results(
  object,
  select_strategy = "control if available",
  select_last_arm = FALSE,
  select_preferences = NULL,
  te_comp = NULL,
  raw_ests = FALSE,
  final_ests = NULL,
  cores = NULL
)

Arguments

object

trial_results object, output from the run_trials() function.

select_strategy

single character string. If a trial was not stopped due to superiority (or had only 1 arm remaining, if select_last_arm is set to TRUE in trial designs with a common control arm; see below), this parameter specifies which arm will be considered selected when calculating trial design performance metrics, as described below; this corresponds to the consequence of an inconclusive trial, i.e., which arm would then be used in practice.
The following options are available and must be written exactly as below (case sensitive, cannot be abbreviated):

  • "control if available" (default): selects the first control arm for trials with a common control arm if this arm is active at end-of-trial, otherwise no arm will be selected. For trial designs without a common control, no arm will be selected.

  • "none": selects no arm in trials not ending with superiority.

  • "control": similar to "control if available", but will throw an error if used for trial designs without a common control arm.

  • "final control": selects the final control arm regardless of whether the trial was stopped for practical equivalence, futility, or at the maximum sample size; this strategy can only be specified for trial designs with a common control arm.

  • "control or best": selects the first control arm if still active at end-of-trial, otherwise selects the best remaining arm (defined as the remaining arm with the highest probability of being the best in the last adaptive analysis conducted). Only works for trial designs with a common control arm.

  • "best": selects the best remaining arm (as described under "control or best").

  • "list or best": selects the first remaining arm from a specified list (specified using select_preferences, technically a character vector). If none of these arms are are active at end-of-trial, the best remaining arm will be selected (as described above).

  • "list": as specified above, but if no arms on the provided list remain active at end-of-trial, no arm is selected.

select_last_arm

single logical, defaults to FALSE. If TRUE, the only remaining active arm (the last control) will be selected in trials with a common control arm ending with equivalence or futility, before considering the options specified in select_strategy. Must be FALSE for trial designs without a common control arm.

select_preferences

character vector specifying a number of arms used for selection if one of the "list or best" or "list" options are specified for select_strategy. Can only contain valid arms available in the trial.

te_comp

character string, treatment-effect comparator. Can be either NULL (the default) in which case the first control arm is used for trial designs with a common control arm, or a string naming a single trial arm. Will be used when calculating err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described below).

raw_ests

single logical. If FALSE (default), the posterior estimates (post_ests or post_ests_all, see setup_trial() and run_trial()) will be used to calculate err and sq_err (the error and the squared error of the estimated compared to the specified effect in the selected arm) and err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described for te_comp and below). If TRUE, the raw estimates (raw_ests or raw_ests_all, see setup_trial() and run_trial()) will be used instead of the posterior estimates.

final_ests

single logical. If TRUE (recommended) the final estimates calculated using outcome data from all patients randomised when trials are stopped are used (post_ests_all or raw_ests_all, see setup_trial() and run_trial()); if FALSE, the estimates calculated for each arm when an arm is stopped (or at the last adaptive analysis if not before) using data from patients having reach followed up at this time point and not all patients randomised are used (post_ests or raw_ests, see setup_trial() and run_trial()). If NULL (the default), this argument will be set to FALSE if outcome data are available immediate after randomisation for all patients (for backwards compatibility, as final posterior estimates may vary slightly in this situation, even if using the same data); otherwise it will be said to TRUE. See setup_trial() for more details on how these estimates are calculated.

cores

NULL or single integer. If NULL, a default value set by setup_cluster() will be used to control whether extractions of simulation results are done in parallel on a default cluster or sequentially in the main process; if a value has not been specified by setup_cluster(), cores will then be set to the value stored in the global "mc.cores" option (if previously set by ⁠options(mc.cores = <number of cores>⁠), and 1 if that option has not been specified.
If cores = 1, computations will be run sequentially in the primary process, and if cores > 1, a new parallel cluster will be setup using the parallel library and removed once the function completes. See setup_cluster() for details.

Value

A data.frame containing the following columns:

See Also

check_performance(), summary(), plot_convergence(), plot_metrics_ecdf(), check_remaining_arms().

Examples

# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
                                 control = "A",
                                 true_ys = c(0.20, 0.18, 0.22, 0.24),
                                 data_looks = 1:20 * 100)

# Run 10 simulations with a specified random base seed
res <- run_trials(binom_trial, n_rep = 10, base_seed = 12345)

# Extract results and Select the control arm if available
# in simulations not ending with superiority
extract_results(res, select_strategy = "control")


Extract results from a batch of trials from an object with multiple trials

Description

Used internally by extract_results(). Extracts results from a batch of simulations from a simulation object with multiple simulation results returned by run_trials(), used to facilitate parallelisation.

Usage

extract_results_batch(
  trial_results,
  control = control,
  select_strategy = select_strategy,
  select_last_arm = select_last_arm,
  select_preferences = select_preferences,
  te_comp = te_comp,
  which_ests = which_ests,
  te_comp_index = te_comp_index,
  te_comp_true_y = te_comp_true_y
)

Arguments

trial_results

list of trial results to summarise, the current batch.

control

single character string, the common control arm from the trial specification (NULL if none).

select_strategy

single character string. If a trial was not stopped due to superiority (or had only 1 arm remaining, if select_last_arm is set to TRUE in trial designs with a common control arm; see below), this parameter specifies which arm will be considered selected when calculating trial design performance metrics, as described below; this corresponds to the consequence of an inconclusive trial, i.e., which arm would then be used in practice.
The following options are available and must be written exactly as below (case sensitive, cannot be abbreviated):

  • "control if available" (default): selects the first control arm for trials with a common control arm if this arm is active at end-of-trial, otherwise no arm will be selected. For trial designs without a common control, no arm will be selected.

  • "none": selects no arm in trials not ending with superiority.

  • "control": similar to "control if available", but will throw an error if used for trial designs without a common control arm.

  • "final control": selects the final control arm regardless of whether the trial was stopped for practical equivalence, futility, or at the maximum sample size; this strategy can only be specified for trial designs with a common control arm.

  • "control or best": selects the first control arm if still active at end-of-trial, otherwise selects the best remaining arm (defined as the remaining arm with the highest probability of being the best in the last adaptive analysis conducted). Only works for trial designs with a common control arm.

  • "best": selects the best remaining arm (as described under "control or best").

  • "list or best": selects the first remaining arm from a specified list (specified using select_preferences, technically a character vector). If none of these arms are are active at end-of-trial, the best remaining arm will be selected (as described above).

  • "list": as specified above, but if no arms on the provided list remain active at end-of-trial, no arm is selected.

select_last_arm

single logical, defaults to FALSE. If TRUE, the only remaining active arm (the last control) will be selected in trials with a common control arm ending with equivalence or futility, before considering the options specified in select_strategy. Must be FALSE for trial designs without a common control arm.

select_preferences

character vector specifying a number of arms used for selection if one of the "list or best" or "list" options are specified for select_strategy. Can only contain valid arms available in the trial.

te_comp

character string, treatment-effect comparator. Can be either NULL (the default) in which case the first control arm is used for trial designs with a common control arm, or a string naming a single trial arm. Will be used when calculating err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described below).

which_ests

single character string, a combination of the raw_ests and final_ests arguments from extract_results().

te_comp_index

single integer, index of the treatment effect comparator arm (NULL if none).

te_comp_true_y

single numeric value, true y value in the treatment effect comparator arm (NULL if none).

Value

A data.frame containing all columns returned by extract_results() and described in that function (sim will start from 1, but this is changed where relevant by extract_results()).


Extract statuses

Description

Used internally. Extracts overall trial statuses or statuses from a single arm from multiple trial simulations. Works with sparse results.

Usage

extract_statuses(object, x_value, arm = NULL)

Arguments

object

trial_results object from run_trials().

x_value

single character string, determining whether the number of adaptive analysis looks ("look", default), the total cumulated number of patients randomised ("total n") or the total cumulated number of patients with outcome data available at each adaptive analysis ("followed n") are plotted on the x-axis.

arm

character vector containing one or more unique, valid arm names, NA, or NULL (default). If NULL, the overall trial statuses are plotted, otherwise the specified arms or all arms (if NA is specified) are plotted.

Value

A tidy data.frame (one row possible status per look) containing the following columns:


Find beta distribution parameters from thresholds

Description

Helper function to find a beta distribution with parameters corresponding to the fewest possible patients with events/non-events and a specified event proportion. Used in the Advanced example vignette (vignette("Advanced-example", "adaptr")) to derive beta prior distributions for use in beta-binomial conjugate models, based on a belief that the true event probability lies within a specified percentile-based interval (defaults to ⁠95%⁠). May similarly be used by users to derive other beta priors.

Usage

find_beta_params(
  theta = NULL,
  boundary_target = NULL,
  boundary = "lower",
  interval_width = 0.95,
  n_dec = 0,
  max_n = 10000
)

Arguments

theta

single numeric ⁠> 0⁠ and ⁠< 1⁠, expected true event probability.

boundary_target

single numeric ⁠> 0⁠ and ⁠< 1⁠, target lower or upper boundary of the interval.

boundary

single character string, either "lower" (default) or "upper", used to select which boundary to use when finding appropriate parameters for the beta distribution.

interval_width

width of the credible interval whose lower/upper boundary should be used (see boundary_target); must be ⁠> 0⁠ and ⁠< 1⁠; defaults to 0.95.

n_dec

single non-negative integer; the returned parameters are rounded to this number of decimals. Defaults to 0, in which case the parameters will correspond to whole number of patients.

max_n

single integer ⁠> 0⁠ (default 10000), the maximum total sum of the parameters, corresponding to the maximum total number of patients that will be considered by the function when finding the optimal parameter values. Corresponds to the maximum number of patients contributing information to a beta prior; more than the default number of patients are unlikely to be used in a beta prior.

Value

A single-row data.frame with five columns: the two shape parameters of the beta distribution (alpha, beta), rounded according to n_dec, and the actual lower and upper boundaries of the interval and the median (with appropriate names, e.g. p2.5, p50, and p97.5 for a ⁠95%⁠ interval), when using those rounded values.


Format digits before printing

Description

Used internally.

Usage

fmt_dig(x, dig)

Arguments

x

numeric, the numeric value(s) to format.

dig

single integer, the number of digits.

Value

Formatted character string.


Create formatted label with absolute and relative frequencies (percentages)

Description

Used internally.

Usage

fmt_pct(e, n, dec = 1)

Arguments

e

integer, the numerator (e.g., the number of events).

n

integer, the denominator (e.g., the total number of patients).

dec

integer, the number of decimals for the percentage.

Value

Formatted character string.


Generate draws from posterior beta-binomial distributions

Description

Used internally. This function generates draws from posterior distributions using separate beta-binomial models (binomial outcome, conjugate beta prior) for each arm, with flat (beta(1, 1)) priors.

Usage

get_draws_binom(arms, allocs, ys, control, n_draws)

Arguments

arms

character vector, currently active arms as specified in setup_trial() / setup_trial_binom() / setup_trial_norm().

allocs

character vector, allocations of all patients (including allocations to currently inactive arms).

ys

numeric vector, outcomes of all patients in the same order as alloc (including outcomes of patients in currently inactive arms).

control

unused argument in the built-in functions for setup_trial_binom() and setup_trial_norm, but required as this argument is supplied by the run_trial() function, and may be used in user-defined functions used to generate posterior draws.

n_draws

single integer, number of posterior draws.

Value

A matrix (with numeric values) with length(arms) columns and n_draws rows, with arms as column names.


Generic documentation for get_draws_* functions

Description

Used internally. See the setup_trial() function documentation for additional details on how to specify functions to generate posterior draws.

Arguments

arms

character vector, currently active arms as specified in setup_trial() / setup_trial_binom() / setup_trial_norm().

allocs

character vector, allocations of all patients (including allocations to currently inactive arms).

ys

numeric vector, outcomes of all patients in the same order as alloc (including outcomes of patients in currently inactive arms).

control

unused argument in the built-in functions for setup_trial_binom() and setup_trial_norm, but required as this argument is supplied by the run_trial() function, and may be used in user-defined functions used to generate posterior draws.

n_draws

single integer, number of posterior draws.

Value

A matrix (with numeric values) with length(arms) columns and n_draws rows, with arms as column names.


Generate draws from posterior normal distributions

Description

Used internally. This function generates draws from posterior, normal distributions for continuous outcomes. Technically, these posteriors use no priors (for simulation speed), corresponding to the use of improper flat priors. These posteriors correspond (and give similar results) to using normal-normal models (normally distributed outcome, conjugate normal prior) for each arm, assuming that a non-informative, flat prior is used. Thus, the posteriors directly correspond to normal distributions with each groups' mean as the mean and each groups' standard error as the standard deviation. As it is necessary to always return valid draws, in cases where ⁠< 2⁠ patients have been randomised to an arm, posterior draws will come from an extremely wide normal distribution with mean corresponding to the mean of all included patients with outcome data and a standard deviation corresponding to the difference between the highest and lowest recorded outcomes for all patients with available outcome data multiplied by 1000.

Usage

get_draws_norm(arms, allocs, ys, control, n_draws)

Arguments

arms

character vector, currently active arms as specified in setup_trial() / setup_trial_binom() / setup_trial_norm().

allocs

character vector, allocations of all patients (including allocations to currently inactive arms).

ys

numeric vector, outcomes of all patients in the same order as alloc (including outcomes of patients in currently inactive arms).

control

unused argument in the built-in functions for setup_trial_binom() and setup_trial_norm, but required as this argument is supplied by the run_trial() function, and may be used in user-defined functions used to generate posterior draws.

n_draws

single integer, number of posterior draws.

Value

A matrix (with numeric values) with length(arms) columns and n_draws rows, with arms as column names.


Generate binary outcomes from binomial distributions

Description

Used internally. Function factory used to generate a function that generates binary outcomes from binomial distributions.

Usage

get_ys_binom(arms, event_probs)

Arguments

arms

character vector of arms as specified in setup_trial_binom().

event_probs

numeric vector of true event probabilities in all arms as specified in setup_trial_binom().

Value

A function which takes the argument allocs (a character vector with the allocations) and returns a numeric vector of similar length with the corresponding, randomly generated outcomes (0 or 1, from binomial distribution).


Generate normally distributed continuous outcomes

Description

Used internally. Function factory used to generate a function that generates outcomes from normal distributions.

Usage

get_ys_norm(arms, means, sds)

Arguments

arms

character vector, arms as specified in setup_trial_norm().

means

numeric vector, true means in all arms as specified in setup_trial_norm().

sds

numeric vector, true standard deviations (sds) in all arms as specified in setup_trial_norm().

Value

A function which takes the argument allocs (a character vector with the allocations) and returns a numeric vector of the same length with the corresponding, randomly generated outcomes (from normal distributions).


Gaussian process-based optimisation

Description

Used internally. Simple Gaussian process-based Bayesian optimisation function, used to find the next value to evaluate (as x) in the calibrate_trial() function. Uses only a single input dimension, which may be rescaled to the ⁠[0, 1]⁠ range by the function, and a covariance structure based on absolute distances between values, raised to a power (pow) and subsequently divided by lengthscale before the inverse exponentiation of the resulting matrix is used. The pow and lengthscale hyperparameters consequently control the smoothness by controlling the rate of decay between correlations with distance.
The optimisation algorithm uses bi-directional uncertainty bounds in an acquisition function that suggests the next target to evaluate, with wider uncertainty bounds (higher kappa) leading to increased 'exploration' (i.e., the function is more prone to suggest new target values where the uncertainty is high and often further from the best evaluation so far) and narrower uncertainty bounds leading to increased 'exploitation' (i.e., the function is more prone to suggest new target values relatively close to the mean predictions from the model).
The dir argument controls whether the suggested value (based on both uncertainty bounds) should be the value closest to target in either direction (dir = 0), at or above target (dir > 0), or at or below target (dir < 0), if any, are preferred.
When the function being evaluated is noise-free and monotonically increasing or decreasing, the optimisation function can narrow the range of predictions based on the input evaluations (narrow = TRUE), leading to a finer grid of potential new targets to suggest compared to when predictions are spaced over the full range.
If the new value at which to evaluate the function suggested has already been evaluated, random noise will be added to ensure evaluation at a new value (if narrow is FALSE, noise will be based on a random draw from a normal distribution with the current suggested value as mean and the standard deviation of the x values as SD, truncated to the range of x-values; if narrow is TRUE, a new value drawn from a uniform distribution within the current narrowed range will be suggested. For both strategies, the process will be repeated until the suggested value is 'new').
The Gaussian process model used is partially based on code from Gramacy 2020 (with permission), see References.

Usage

gp_opt(
  x,
  y,
  target,
  dir = 0,
  resolution = 5000,
  kappa = 1.96,
  pow = 1.95,
  lengthscale = 1,
  scale_x = TRUE,
  noisy = FALSE,
  narrow = FALSE
)

Arguments

x

numeric vector, the previous values where the function being calibrated was evaluated.

y

numeric vector, the corresponding results of the previous evaluations at the x values (must be of the same length as x).

target

single numeric value, the desired target value for the calibration process.

dir

single numeric value (default 0), used when selecting the next value to evaluate at. See which_nearest() for further description.

resolution

single integer (default 5000), size of the grid at which the predictions used to select the next value to evaluate at are made.
Note: memory use and time will substantially increase with higher values.

kappa

single numeric value ⁠> 0⁠ (default 1.96), used for the width of uncertainty bounds (based on the Gaussian process posterior predictive distribution), which are used to select the next value to evaluate at.

pow

single numerical value, passed to cov_mat() and controls the smoothness of the Gaussian process. Should be between 1 (no smoothness, piecewise straight lines between each subsequent x/y-coordinate if lengthscale described below is 1) and 2; defaults to 1.95, which leads to slightly faster decay of correlations when x values are internally scaled to the ⁠[0, 1]⁠-range compared to 2.

lengthscale

single numerical value (default 1) or numerical vector of length 2; all values must be finite and non-negative. If a single value is provided, this will be used as the lengthscale hyperparameter and passed directly to cov_mat(). If a numerical vector of length 2 is provided, the second value must be higher than the first and the optimal lengthscale in this range will be found using an optimisation algorithm. If any value is 0, a minimum amount of noise will be added as lengthscales must be ⁠> 0⁠. Controls smoothness/decay in combination with pow.

scale_x

single logical value; if TRUE (the default) the x-values will be scaled to the ⁠[0, 1]⁠ range according to the minimum/maximum values provided. If FALSE, the model will use the original scale. If distances on the original scale are small, scaling may be preferred. The returned values will always be on the original scale.

noisy

single logical value. If FALSE (the default), a noiseless process is assumed, and interpolation between values is performed (i.e., with no uncertainty at the evaluated x-values); if TRUE, the y-values are assumed to come from a noisy process, and regression is performed (i.e., some uncertainty at the evaluated x-values will be included in the predictions, with the amount estimated using an optimisation algorithm).

narrow

single logical value. If FALSE (the default), predictions are evenly spread over the full x-range. If TRUE, the prediction grid will be spread evenly over an interval consisting of the two x-values with corresponding y-values closest to the target in opposite directions. This setting should only be used if noisy is FALSE and only if the function can safely be assumed to be only monotonically increasing or decreasing, in which case this will lead to a faster search and a smoother prediction grid in the relevant region without increasing memory use.

Value

List containing two elements, next_x, a single numerical value, the suggested next x value at which to evaluate the function, and predictions, a data.frame with resolution rows and the four columns: x, the x grid values where predictions are made; y_hat, the predicted means, and lub and uub, the lower and upper uncertainty bounds of the predictions according to kappa.

References

Gramacy RB (2020). Chapter 5: Gaussian Process Regression. In: Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences. Chapman Hall/CRC, Boca Raton, Florida, USA. Available online.

Greenhill S, Rana S, Gupta S, Vellanki P, Venkatesh S (2020). Bayesian Optimization for Adaptive Experimental Design: A Review. IEEE Access, 8, 13937-13948. doi:10.1109/ACCESS.2020.2966228


Make x-axis scale for history/status plots

Description

Used internally. Prepares the x-axis scale for history/status plots. Requires the ggplot2 package installed.

Usage

make_x_scale(x_value)

Arguments

x_value

single character string, determining whether the number of adaptive analysis looks ("look", default), the total cumulated number of patients randomised ("total n") or the total cumulated number of patients with outcome data available at each adaptive analysis ("followed n") are plotted on the x-axis.

Value

An appropriate scale for the ggplot2 plot x-axis according to the value specified in x_value.


Make y-axis scale for history/status plots

Description

Used internally. Prepares the y-axis scale for history/status plots. Requires the ggplot2 package installed.

Usage

make_y_scale(y_value)

Arguments

y_value

single character string, determining which values are plotted on the y-axis. The following options are available: allocation probabilities ("prob", default), the total number of patients with outcome data available ("n") or randomised ("n all") to each arm, the percentage of patients with outcome data available ("pct") or randomised ("pct all") to each arm out of the current total, the sum of all available ("sum ys") outcome data or all outcome data for randomised patients including outcome data not available at the time of the current adaptive analysis ("sum ys all"), the ratio of outcomes as defined for "sum ys"/"sum ys all" divided by the corresponding number of patients in each arm.

Value

An appropriate scale for the ggplot2 plot y-axis according to the value specified in y_value.


Plot convergence of performance metrics

Description

Plots performance metrics according to the number of simulations conducted for multiple simulated trials. The simulated trial results may be split into a number of batches to illustrate stability of performance metrics across different simulations. Calculations are done according to specified selection and restriction strategies as described in extract_results() and check_performance(). Requires the ggplot2 package installed.

Usage

plot_convergence(
  object,
  metrics = "size mean",
  resolution = 100,
  select_strategy = "control if available",
  select_last_arm = FALSE,
  select_preferences = NULL,
  te_comp = NULL,
  raw_ests = FALSE,
  final_ests = NULL,
  restrict = NULL,
  n_split = 1,
  nrow = NULL,
  ncol = NULL,
  cores = NULL
)

Arguments

object

trial_results object, output from the run_trials() function.

metrics

the performance metrics to plot, as described in check_performance(). Multiple metrics may be plotted at the same time. Valid metrics include: size_mean, size_sd, size_median, size_p25, size_p75, size_p0, size_p100, sum_ys_mean, sum_ys_sd, sum_ys_median, sum_ys_p25, sum_ys_p75, sum_ys_p0, sum_ys_p100, ratio_ys_mean, ratio_ys_sd, ratio_ys_median, ratio_ys_p25, ratio_ys_p75, ratio_ys_p0, ratio_ys_p100, prob_conclusive, prob_superior, prob_equivalence, prob_futility, prob_max, ⁠prob_select_*⁠ (with * being either "⁠arm_<name>⁠ for all arm names or none), rmse, rmse_te, mae, mae_te, and idp. All may be specified as above, case sensitive, but with either spaces or underlines. Defaults to "size mean".

resolution

single positive integer, the number of points calculated and plotted, defaults to 100 and must be ⁠>= 10⁠. Higher numbers lead to smoother plots, but increases computation time. If the value specified is higher than the number of simulations (or simulations per split), the maximum possible value will be used instead.

select_strategy

single character string. If a trial was not stopped due to superiority (or had only 1 arm remaining, if select_last_arm is set to TRUE in trial designs with a common control arm; see below), this parameter specifies which arm will be considered selected when calculating trial design performance metrics, as described below; this corresponds to the consequence of an inconclusive trial, i.e., which arm would then be used in practice.
The following options are available and must be written exactly as below (case sensitive, cannot be abbreviated):

  • "control if available" (default): selects the first control arm for trials with a common control arm if this arm is active at end-of-trial, otherwise no arm will be selected. For trial designs without a common control, no arm will be selected.

  • "none": selects no arm in trials not ending with superiority.

  • "control": similar to "control if available", but will throw an error if used for trial designs without a common control arm.

  • "final control": selects the final control arm regardless of whether the trial was stopped for practical equivalence, futility, or at the maximum sample size; this strategy can only be specified for trial designs with a common control arm.

  • "control or best": selects the first control arm if still active at end-of-trial, otherwise selects the best remaining arm (defined as the remaining arm with the highest probability of being the best in the last adaptive analysis conducted). Only works for trial designs with a common control arm.

  • "best": selects the best remaining arm (as described under "control or best").

  • "list or best": selects the first remaining arm from a specified list (specified using select_preferences, technically a character vector). If none of these arms are are active at end-of-trial, the best remaining arm will be selected (as described above).

  • "list": as specified above, but if no arms on the provided list remain active at end-of-trial, no arm is selected.

select_last_arm

single logical, defaults to FALSE. If TRUE, the only remaining active arm (the last control) will be selected in trials with a common control arm ending with equivalence or futility, before considering the options specified in select_strategy. Must be FALSE for trial designs without a common control arm.

select_preferences

character vector specifying a number of arms used for selection if one of the "list or best" or "list" options are specified for select_strategy. Can only contain valid arms available in the trial.

te_comp

character string, treatment-effect comparator. Can be either NULL (the default) in which case the first control arm is used for trial designs with a common control arm, or a string naming a single trial arm. Will be used when calculating err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described below).

raw_ests

single logical. If FALSE (default), the posterior estimates (post_ests or post_ests_all, see setup_trial() and run_trial()) will be used to calculate err and sq_err (the error and the squared error of the estimated compared to the specified effect in the selected arm) and err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described for te_comp and below). If TRUE, the raw estimates (raw_ests or raw_ests_all, see setup_trial() and run_trial()) will be used instead of the posterior estimates.

final_ests

single logical. If TRUE (recommended) the final estimates calculated using outcome data from all patients randomised when trials are stopped are used (post_ests_all or raw_ests_all, see setup_trial() and run_trial()); if FALSE, the estimates calculated for each arm when an arm is stopped (or at the last adaptive analysis if not before) using data from patients having reach followed up at this time point and not all patients randomised are used (post_ests or raw_ests, see setup_trial() and run_trial()). If NULL (the default), this argument will be set to FALSE if outcome data are available immediate after randomisation for all patients (for backwards compatibility, as final posterior estimates may vary slightly in this situation, even if using the same data); otherwise it will be said to TRUE. See setup_trial() for more details on how these estimates are calculated.

restrict

single character string or NULL. If NULL (default), results are summarised for all simulations; if "superior", results are summarised for simulations ending with superiority only; if "selected", results are summarised for simulations ending with a selected arm only (according to the specified arm selection strategy for simulations not ending with superiority). Some summary measures (e.g., prob_conclusive) have substantially different interpretations if restricted, but are calculated nonetheless.

n_split

single positive integer, the number of consecutive batches the simulation results will be split into, which will be plotted separately. Default is 1 (no splitting); maximum value is the number of simulations summarised (after restrictions) divided by 10.

nrow, ncol

the number of rows and columns when plotting multiple metrics in the same plot (using faceting in ggplot2). Defaults to NULL, in which case this will be determined automatically.

cores

NULL or single integer. If NULL, a default value set by setup_cluster() will be used to control whether extractions of simulation results are done in parallel on a default cluster or sequentially in the main process; if a value has not been specified by setup_cluster(), cores will then be set to the value stored in the global "mc.cores" option (if previously set by ⁠options(mc.cores = <number of cores>⁠), and 1 if that option has not been specified.
If cores = 1, computations will be run sequentially in the primary process, and if cores > 1, a new parallel cluster will be setup using the parallel library and removed once the function completes. See setup_cluster() for details.

Value

A ggplot2 plot object.

See Also

check_performance(), summary(), extract_results(), check_remaining_arms().

Examples

#### Only run examples if ggplot2 is installed ####
if (requireNamespace("ggplot2", quietly = TRUE)){

  # Setup a trial specification
  binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
                                   control = "A",
                                   true_ys = c(0.20, 0.18, 0.22, 0.24),
                                   data_looks = 1:20 * 100)

  # Run multiple simulation with a fixed random base seed
  res_mult <- run_trials(binom_trial, n_rep = 25, base_seed = 678)

  # NOTE: the number of simulations in this example is smaller than
  # recommended - the plots reflect that, and show that performance metrics
  # are not stable and have likely not converged yet

  # Convergence plot of mean sample sizes
  plot_convergence(res_mult, metrics = "size mean")

}

if (requireNamespace("ggplot2", quietly = TRUE)){

  # Convergence plot of mean sample sizes and ideal design percentages,
  # with simulations split in 2 batches
  plot_convergence(res_mult, metrics = c("size mean", "idp"), n_split = 2)

}


Plot trial metric history

Description

Plots the history of relevant metrics over the progress of a single or multiple trial simulations. Simulated trials only contribute until the time they are stopped, i.e., if some trials are stopped earlier than others, they will not contribute to the summary statistics at later adaptive looks. Data from individual arms in a trial contribute until the complete trial is stopped.
These history plots require non-sparse results (sparse set to FALSE; see run_trial() and run_trials()) and the ggplot2 package installed.

Usage

plot_history(object, x_value = "look", y_value = "prob", line = NULL, ...)

## S3 method for class 'trial_result'
plot_history(object, x_value = "look", y_value = "prob", line = NULL, ...)

## S3 method for class 'trial_results'
plot_history(
  object,
  x_value = "look",
  y_value = "prob",
  line = NULL,
  ribbon = list(width = 0.5, alpha = 0.2),
  cores = NULL,
  ...
)

Arguments

object

trial_results object, output from the run_trials() function.

x_value

single character string, determining whether the number of adaptive analysis looks ("look", default), the total cumulated number of patients randomised ("total n") or the total cumulated number of patients with outcome data available at each adaptive analysis ("followed n") are plotted on the x-axis.

y_value

single character string, determining which values are plotted on the y-axis. The following options are available: allocation probabilities ("prob", default), the total number of patients with outcome data available ("n") or randomised ("n all") to each arm, the percentage of patients with outcome data available ("pct") or randomised ("pct all") to each arm out of the current total, the sum of all available ("sum ys") outcome data or all outcome data for randomised patients including outcome data not available at the time of the current adaptive analysis ("sum ys all"), the ratio of outcomes as defined for "sum ys"/"sum ys all" divided by the corresponding number of patients in each arm.

line

list styling the lines as per ggplot2 conventions (e.g., linetype, linewidth).

...

additional arguments, not used.

ribbon

list, as line but only appropriate for trial_results objects (i.e., when multiple simulations are run). Also allows to specify the width of the interval: must be between 0 and 1, with 0.5 (default) showing the inter-quartile ranges.

cores

NULL or single integer. If NULL, a default value set by setup_cluster() will be used to control whether extractions of simulation results are done in parallel on a default cluster or sequentially in the main process; if a value has not been specified by setup_cluster(), cores will then be set to the value stored in the global "mc.cores" option (if previously set by ⁠options(mc.cores = <number of cores>⁠), and 1 if that option has not been specified.
If cores = 1, computations will be run sequentially in the primary process, and if cores > 1, a new parallel cluster will be setup using the parallel library and removed once the function completes. See setup_cluster() for details.

Value

A ggplot2 plot object.

See Also

plot_status().

Examples

#### Only run examples if ggplot2 is installed ####
if (requireNamespace("ggplot2", quietly = TRUE)){

  # Setup a trial specification
  binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
                                   control = "A",
                                   true_ys = c(0.20, 0.18, 0.22, 0.24),
                                   data_looks = 1:20 * 100)



  # Run a single simulation with a fixed random seed
  res <- run_trial(binom_trial, seed = 12345)

  # Plot total allocations to each arm according to overall total allocations
  plot_history(res, x_value = "total n", y_value = "n")

}

if (requireNamespace("ggplot2", quietly = TRUE)){

  # Run multiple simulation with a fixed random base seed
  # Notice that sparse = FALSE is required
  res_mult <- run_trials(binom_trial, n_rep = 15, base_seed = 12345, sparse = FALSE)

  # Plot allocation probabilities at each look
  plot_history(res_mult, x_value = "look", y_value = "prob")

  # Other y_value options are available but not shown in these examples

}


Plot empirical cumulative distribution functions of performance metrics

Description

Plots empirical cumulative distribution functions (ECDFs) of numerical performance metrics across multiple simulations from a "trial_results" object returned by run_trials(). Requires the ggplot2 package installed.

Usage

plot_metrics_ecdf(
  object,
  metrics = c("size", "sum_ys", "ratio_ys"),
  select_strategy = "control if available",
  select_last_arm = FALSE,
  select_preferences = NULL,
  te_comp = NULL,
  raw_ests = FALSE,
  final_ests = NULL,
  restrict = NULL,
  nrow = NULL,
  ncol = NULL,
  cores = NULL
)

Arguments

object

trial_results object, output from the run_trials() function.

metrics

the performance metrics to plot, as described in extract_results(). Multiple metrics may be plotted at the same time. Valid metrics include: size, sum_ys, ratio_ys_mean, sq_err, sq_err_te, err, err_te, abs_err, abs_err_te, (as described in extract_results(), with the addition of abs_err and abs_err_te, which are the absolute errors, i.e., abs(err) and abs(err_te)). All may be specified using either spaces or underlines (case sensitive). Defaults to plotting size, sum_ys, and ratio_ys_mean.

select_strategy

single character string. If a trial was not stopped due to superiority (or had only 1 arm remaining, if select_last_arm is set to TRUE in trial designs with a common control arm; see below), this parameter specifies which arm will be considered selected when calculating trial design performance metrics, as described below; this corresponds to the consequence of an inconclusive trial, i.e., which arm would then be used in practice.
The following options are available and must be written exactly as below (case sensitive, cannot be abbreviated):

  • "control if available" (default): selects the first control arm for trials with a common control arm if this arm is active at end-of-trial, otherwise no arm will be selected. For trial designs without a common control, no arm will be selected.

  • "none": selects no arm in trials not ending with superiority.

  • "control": similar to "control if available", but will throw an error if used for trial designs without a common control arm.

  • "final control": selects the final control arm regardless of whether the trial was stopped for practical equivalence, futility, or at the maximum sample size; this strategy can only be specified for trial designs with a common control arm.

  • "control or best": selects the first control arm if still active at end-of-trial, otherwise selects the best remaining arm (defined as the remaining arm with the highest probability of being the best in the last adaptive analysis conducted). Only works for trial designs with a common control arm.

  • "best": selects the best remaining arm (as described under "control or best").

  • "list or best": selects the first remaining arm from a specified list (specified using select_preferences, technically a character vector). If none of these arms are are active at end-of-trial, the best remaining arm will be selected (as described above).

  • "list": as specified above, but if no arms on the provided list remain active at end-of-trial, no arm is selected.

select_last_arm

single logical, defaults to FALSE. If TRUE, the only remaining active arm (the last control) will be selected in trials with a common control arm ending with equivalence or futility, before considering the options specified in select_strategy. Must be FALSE for trial designs without a common control arm.

select_preferences

character vector specifying a number of arms used for selection if one of the "list or best" or "list" options are specified for select_strategy. Can only contain valid arms available in the trial.

te_comp

character string, treatment-effect comparator. Can be either NULL (the default) in which case the first control arm is used for trial designs with a common control arm, or a string naming a single trial arm. Will be used when calculating err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described below).

raw_ests

single logical. If FALSE (default), the posterior estimates (post_ests or post_ests_all, see setup_trial() and run_trial()) will be used to calculate err and sq_err (the error and the squared error of the estimated compared to the specified effect in the selected arm) and err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described for te_comp and below). If TRUE, the raw estimates (raw_ests or raw_ests_all, see setup_trial() and run_trial()) will be used instead of the posterior estimates.

final_ests

single logical. If TRUE (recommended) the final estimates calculated using outcome data from all patients randomised when trials are stopped are used (post_ests_all or raw_ests_all, see setup_trial() and run_trial()); if FALSE, the estimates calculated for each arm when an arm is stopped (or at the last adaptive analysis if not before) using data from patients having reach followed up at this time point and not all patients randomised are used (post_ests or raw_ests, see setup_trial() and run_trial()). If NULL (the default), this argument will be set to FALSE if outcome data are available immediate after randomisation for all patients (for backwards compatibility, as final posterior estimates may vary slightly in this situation, even if using the same data); otherwise it will be said to TRUE. See setup_trial() for more details on how these estimates are calculated.

restrict

single character string or NULL. If NULL (default), results are summarised for all simulations; if "superior", results are summarised for simulations ending with superiority only; if "selected", results are summarised for simulations ending with a selected arm only (according to the specified arm selection strategy for simulations not ending with superiority). Some summary measures (e.g., prob_conclusive) have substantially different interpretations if restricted, but are calculated nonetheless.

nrow, ncol

the number of rows and columns when plotting multiple metrics in the same plot (using faceting in ggplot2). Defaults to NULL, in which case this will be determined automatically.

cores

NULL or single integer. If NULL, a default value set by setup_cluster() will be used to control whether extractions of simulation results are done in parallel on a default cluster or sequentially in the main process; if a value has not been specified by setup_cluster(), cores will then be set to the value stored in the global "mc.cores" option (if previously set by ⁠options(mc.cores = <number of cores>⁠), and 1 if that option has not been specified.
If cores = 1, computations will be run sequentially in the primary process, and if cores > 1, a new parallel cluster will be setup using the parallel library and removed once the function completes. See setup_cluster() for details.

Details

Note that the arguments related to arm selection and error calculation are only relevant if errors are visualised.

Value

A ggplot2 plot object.

See Also

check_performance(), summary(), extract_results(), plot_convergence(), check_remaining_arms().

Examples

#### Only run examples if ggplot2 is installed ####
if (requireNamespace("ggplot2", quietly = TRUE)){

  # Setup a trial specification
  binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
                                   control = "A",
                                   true_ys = c(0.20, 0.18, 0.22, 0.24),
                                   data_looks = 1:20 * 100)

  # Run multiple simulation with a fixed random base seed
  res_mult <- run_trials(binom_trial, n_rep = 25, base_seed = 678)

  # NOTE: the number of simulations in this example is smaller than
  # recommended - the plots reflect that, and would likely be smoother if
  # a larger number of trials had been simulated

  # Plot ECDFs of continuous performance metrics
  plot_metrics_ecdf(res_mult)

}


Plot statuses

Description

Plots the statuses over time of multiple simulated trials (overall or for one or more specific arms). Requires the ggplot2 package installed.

Usage

plot_status(
  object,
  x_value = "look",
  arm = NULL,
  area = list(alpha = 0.5),
  nrow = NULL,
  ncol = NULL
)

## S3 method for class 'trial_results'
plot_status(
  object,
  x_value = "look",
  arm = NULL,
  area = list(alpha = 0.5),
  nrow = NULL,
  ncol = NULL
)

Arguments

object

trial_results object, output from the run_trials() function.

x_value

single character string, determining whether the number of adaptive analysis looks ("look", default), the total cumulated number of patients randomised ("total n") or the total cumulated number of patients with outcome data available at each adaptive analysis ("followed n") are plotted on the x-axis.

arm

character vector containing one or more unique, valid arm names, NA, or NULL (default). If NULL, the overall trial statuses are plotted, otherwise the specified arms or all arms (if NA is specified) are plotted.

area

list of styling settings for the area as per ggplot2 conventions (e.g., alpha, linewidth). The default (list(alpha = 0.5)) sets the transparency to 50% so overlain shaded areas are visible.

nrow, ncol

the number of rows and columns when plotting statuses for multiple arms in the same plot (using faceting in ggplot2). Defaults to NULL, in which case this will be determined automatically where relevant.

Value

A ggplot2 plot object.

See Also

plot_history().

Examples

#### Only run examples if ggplot2 is installed ####
if (requireNamespace("ggplot2", quietly = TRUE)){

  # Setup a trial specification
  binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
                                   control = "A",
                                   true_ys = c(0.20, 0.18, 0.22, 0.24),
                                   data_looks = 1:20 * 100)

  # Run multiple simulation with a fixed random base seed
  res_mult <- run_trials(binom_trial, n_rep = 25, base_seed = 12345)

  # Plot trial statuses at each look according to total allocations
  plot_status(res_mult, x_value = "total n")

}

if (requireNamespace("ggplot2", quietly = TRUE)){

  # Plot trial statuses for all arms
  plot_status(res_mult, arm = NA)

}


Calculates matrix of absolute distances raised to a power

Description

Used internally, calculates the absolute distances for values in a matrix with possibly unequal dimensions, and raises these to a power.

Usage

pow_abs_dist(x1, x2 = x1, pow = 2)

Arguments

x1

numeric vector, with length corresponding to the number of rows in the returned matrix.

x2

numeric vector, with length corresponding to the number of columns in the returned matrix. If not specified, x1 will be used for x2.

pow

single numeric value, the power that all distances are raised to. Defaults to 2, corresponding to pairwise, squared, Euclidean distances.

Value

Matrix with length(x1) rows and length(x2) columns including the calculated absolute pairwise distances raised to pow.


Print methods for adaptive trial objects

Description

Prints contents of the first input x in a human-friendly way, see Details for more information.

Usage

## S3 method for class 'trial_spec'
print(x, prob_digits = 3, ...)

## S3 method for class 'trial_result'
print(x, prob_digits = 3, ...)

## S3 method for class 'trial_performance'
print(x, digits = 3, ...)

## S3 method for class 'trial_results'
print(
  x,
  select_strategy = "control if available",
  select_last_arm = FALSE,
  select_preferences = NULL,
  te_comp = NULL,
  raw_ests = FALSE,
  final_ests = NULL,
  restrict = NULL,
  digits = 1,
  cores = NULL,
  ...
)

## S3 method for class 'trial_results_summary'
print(x, digits = 1, ...)

## S3 method for class 'trial_calibration'
print(x, ...)

Arguments

x

object to print, see Details.

prob_digits

single integer (default is 3), the number of digits used when printing probabilities, allocation probabilities and softening powers (with 2 extra digits added for stopping rule probability thresholds in trial specifications and for outcome rates in summarised results from multiple simulations).

...

additional arguments, not used.

digits

single integer, the number of digits used when printing the numeric results. Default is 3 for outputs from check_performance() and 1 for outputs from run_trials() and the accompanying summary() method.

select_strategy

single character string. If a trial was not stopped due to superiority (or had only 1 arm remaining, if select_last_arm is set to TRUE in trial designs with a common control arm; see below), this parameter specifies which arm will be considered selected when calculating trial design performance metrics, as described below; this corresponds to the consequence of an inconclusive trial, i.e., which arm would then be used in practice.
The following options are available and must be written exactly as below (case sensitive, cannot be abbreviated):

  • "control if available" (default): selects the first control arm for trials with a common control arm if this arm is active at end-of-trial, otherwise no arm will be selected. For trial designs without a common control, no arm will be selected.

  • "none": selects no arm in trials not ending with superiority.

  • "control": similar to "control if available", but will throw an error if used for trial designs without a common control arm.

  • "final control": selects the final control arm regardless of whether the trial was stopped for practical equivalence, futility, or at the maximum sample size; this strategy can only be specified for trial designs with a common control arm.

  • "control or best": selects the first control arm if still active at end-of-trial, otherwise selects the best remaining arm (defined as the remaining arm with the highest probability of being the best in the last adaptive analysis conducted). Only works for trial designs with a common control arm.

  • "best": selects the best remaining arm (as described under "control or best").

  • "list or best": selects the first remaining arm from a specified list (specified using select_preferences, technically a character vector). If none of these arms are are active at end-of-trial, the best remaining arm will be selected (as described above).

  • "list": as specified above, but if no arms on the provided list remain active at end-of-trial, no arm is selected.

select_last_arm

single logical, defaults to FALSE. If TRUE, the only remaining active arm (the last control) will be selected in trials with a common control arm ending with equivalence or futility, before considering the options specified in select_strategy. Must be FALSE for trial designs without a common control arm.

select_preferences

character vector specifying a number of arms used for selection if one of the "list or best" or "list" options are specified for select_strategy. Can only contain valid arms available in the trial.

te_comp

character string, treatment-effect comparator. Can be either NULL (the default) in which case the first control arm is used for trial designs with a common control arm, or a string naming a single trial arm. Will be used when calculating err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described below).

raw_ests

single logical. If FALSE (default), the posterior estimates (post_ests or post_ests_all, see setup_trial() and run_trial()) will be used to calculate err and sq_err (the error and the squared error of the estimated compared to the specified effect in the selected arm) and err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described for te_comp and below). If TRUE, the raw estimates (raw_ests or raw_ests_all, see setup_trial() and run_trial()) will be used instead of the posterior estimates.

final_ests

single logical. If TRUE (recommended) the final estimates calculated using outcome data from all patients randomised when trials are stopped are used (post_ests_all or raw_ests_all, see setup_trial() and run_trial()); if FALSE, the estimates calculated for each arm when an arm is stopped (or at the last adaptive analysis if not before) using data from patients having reach followed up at this time point and not all patients randomised are used (post_ests or raw_ests, see setup_trial() and run_trial()). If NULL (the default), this argument will be set to FALSE if outcome data are available immediate after randomisation for all patients (for backwards compatibility, as final posterior estimates may vary slightly in this situation, even if using the same data); otherwise it will be said to TRUE. See setup_trial() for more details on how these estimates are calculated.

restrict

single character string or NULL. If NULL (default), results are summarised for all simulations; if "superior", results are summarised for simulations ending with superiority only; if "selected", results are summarised for simulations ending with a selected arm only (according to the specified arm selection strategy for simulations not ending with superiority). Some summary measures (e.g., prob_conclusive) have substantially different interpretations if restricted, but are calculated nonetheless.

cores

NULL or single integer. If NULL, a default value set by setup_cluster() will be used to control whether extractions of simulation results are done in parallel on a default cluster or sequentially in the main process; if a value has not been specified by setup_cluster(), cores will then be set to the value stored in the global "mc.cores" option (if previously set by ⁠options(mc.cores = <number of cores>⁠), and 1 if that option has not been specified.
If cores = 1, computations will be run sequentially in the primary process, and if cores > 1, a new parallel cluster will be setup using the parallel library and removed once the function completes. See setup_cluster() for details.

Details

The behaviour depends on the class of x:

Value

Invisibly returns x.

Methods (by class)


Calculate the probability that all arms are practically equivalent

Description

Used internally. This function takes a matrix as calculated by the get_draws_binom(), get_draws_norm() or a corresponding custom function (specified using the fun_draws argument in setup_trial(); see get_draws_generic()), and an equivalence difference, and calculates the probability of all arms being equivalent (absolute differences between highest and lowest value in the same set of posterior draws being less than the difference considered practically equivalent).

Usage

prob_all_equi(m, equivalence_diff = NULL)

Arguments

m

a matrix with one column per trial arm (named as the arms) and one row for each draw from the posterior distributions.

equivalence_diff

single numeric value (⁠> 0⁠) or NULL (default, corresponding to no equivalence assessment). If a numeric value is specified, estimated absolute differences smaller than this threshold will be considered equivalent. For designs with a common control arm, the differences between each non-control arm and the control arm is used, and for trials without a common control arm, the difference between the highest and lowest estimated outcome rates are used and the trial is only stopped for equivalence if all remaining arms are equivalent.

Value

A single numeric value corresponding to the probability of all arms being practically equivalent.


Calculate the probabilities of each arm being the best

Description

Used internally. This function takes a matrix as calculated by the get_draws_binom(), get_draws_norm() or a corresponding custom function (as specified using the fun_draws argument in setup_trial(); see get_draws_generic()) and calculates the probabilities of each arm being the best (defined as either the highest or the lowest value, as specified by the highest_is_best argument in setup_trial(), setup_trial_binom() or setup_trial_norm()).

Usage

prob_best(m, highest_is_best = FALSE)

Arguments

m

a matrix with one column per trial arm (named as the arms) and one row for each draw from the posterior distributions.

highest_is_best

single logical, specifies whether larger estimates of the outcome are favourable or not; defaults to FALSE, corresponding to, e.g., an undesirable binary outcomes (e.g., mortality) or a continuous outcome where lower numbers are preferred (e.g., hospital length of stay).

Value

A named numeric vector of probabilities (names corresponding to arms).


Calculate probabilities of comparisons of arms against with common control

Description

Used internally. This function takes a matrix as calculated by the get_draws_binom(), get_draws_norm() or a corresponding custom function (as specified using the fun_draws argument in setup_trial(); see get_draws_generic()) and a single character specifying the control arm, and calculates the probabilities of each arm being better than a common control (defined as either higher or lower than the control, as specified by the highest_is_best argument in setup_trial(), setup_trial_binom() or setup_trial_norm()). This function also calculates equivalence and futility probabilities compared to the common control arm, as specified in setup_trial(), setup_trial_binom() or setup_trial_norm(), unless equivalence_diff or futility_diff, respectively, are set to NULL (the default).

Usage

prob_better(
  m,
  control = NULL,
  highest_is_best = FALSE,
  equivalence_diff = NULL,
  futility_diff = NULL
)

Arguments

m

a matrix with one column per trial arm (named as the arms) and one row for each draw from the posterior distributions.

control

a single character string specifying the common control arm.

highest_is_best

single logical, specifies whether larger estimates of the outcome are favourable or not; defaults to FALSE, corresponding to, e.g., an undesirable binary outcomes (e.g., mortality) or a continuous outcome where lower numbers are preferred (e.g., hospital length of stay).

equivalence_diff

single numeric value (⁠> 0⁠) or NULL (default, corresponding to no equivalence assessment). If a numeric value is specified, estimated absolute differences smaller than this threshold will be considered equivalent. For designs with a common control arm, the differences between each non-control arm and the control arm is used, and for trials without a common control arm, the difference between the highest and lowest estimated outcome rates are used and the trial is only stopped for equivalence if all remaining arms are equivalent.

futility_diff

single numeric value (⁠> 0⁠) or NULL (default, corresponding to no futility assessment). If a numeric value is specified, estimated differences below this threshold in the beneficial direction (as specified in highest_is_best) will be considered futile when assessing futility in designs with a common control arm. If only 1 arm remains after dropping arms for futility, the trial will be stopped without declaring the last arm superior.

Value

A named (row names corresponding to the trial arms) matrix containing 1-3 columns: probs_better, probs_equivalence (if equivalence_diff is specified), and probs_futile (if futility_diff is specified). All columns will contain NA for the control arm.


Generate breakpoints and other values for printing progress

Description

Used internally. Generates breakpoints, messages, and 'batches' of trial numbers to simulate when using run_trials() with the progress argument in use. Breaks will be multiples of the number of cores, and repeated use of the same values for breaks is avoided (if, e.g., the number of breaks times the number of cores is not possible if few new trials are to be run). Inputs are validated by run_trials().

Usage

prog_breaks(progress, prev_n_rep, n_rep_new, cores)

Arguments

progress

single numeric ⁠> 0⁠ and ⁠<= 1⁠ or NULL. If NULL (default), no progress is printed to the console. Otherwise, progress messages are printed to the control at intervals proportional to the value specified by progress.
Note: as printing is not possible from within clusters on multiple cores, the function conducts batches of simulations on multiple cores (if specified), with intermittent printing of statuses. Thus, all cores have to finish running their current assigned batches before the other cores may proceed with the next batch. If there are substantial differences in the simulation speeds across cores, using progress may thus increase total run time (especially with small values).

prev_n_rep

single integer, the previous number of simulations run (to add to the indices generated and used).

n_rep_new

single integers, number of new simulations to run (i.e., n_rep as supplied to run_trials() minus the number of previously run simulations if grow is used in run_trials()).

cores

NULL or single integer. If NULL, a default value/cluster set by setup_cluster() will be used to control whether simulations are run in parallel on a default cluster or sequentially in the main process; if a cluster/value has not been specified by setup_cluster(), cores will then be set to the value stored in the global "mc.cores" option (if previously set by ⁠options(mc.cores = <number of cores>⁠), and 1 if that option has not been specified.
If the resulting number of cores = 1, computations will be run sequentially in the primary process, and if cores > 1, a new parallel cluster will be setup using the parallel library and removed once the function completes. See setup_cluster() for details.

Value

List containing breaks (the number of patients at each break), start_mess and prog_mess (the basis of the first and subsequent progress messages), and batches (a list with each entry corresponding to the simulation numbers in each batch).


Update allocation probabilities

Description

Used internally. This function calculates new allocation probabilities for each arm, based on the information specified in setup_trial(), setup_trial_binom() or setup_trial_norm() and the calculated probabilities of each arm being the best by prob_best().

Usage

reallocate_probs(
  probs_best,
  fixed_probs,
  min_probs,
  max_probs,
  soften_power = 1,
  match_arm = NULL,
  rescale_fixed = FALSE,
  rescale_limits = FALSE,
  rescale_factor = 1,
  rescale_ignore = NULL
)

Arguments

probs_best

a resulting named vector from the prob_best() function.

fixed_probs

numeric vector, fixed allocation probabilities for each arm. Must be either a numeric vector with NA for arms without fixed probabilities and values between 0 and 1 for the other arms or NULL (default), if adaptive randomisation is used for all arms or if one of the special settings ("sqrt-based", "sqrt-based start", "sqrt-based fixed", or "match") is specified for control_prob_fixed (described below).

min_probs

numeric vector, lower threshold for adaptive allocation probabilities; lower probabilities will be rounded up to these values. Must be NA (default for all arms) if no lower threshold is wanted and for arms using fixed allocation probabilities.

max_probs

numeric vector, upper threshold for adaptive allocation probabilities; higher probabilities will be rounded down to these values. Must be NA (default for all arms) if no threshold is wanted and for arms using fixed allocation probabilities.

soften_power

either a single numeric value or a numeric vector of exactly the same length as the maximum number of looks/adaptive analyses. Values must be between 0 and 1 (default); if ⁠< 1⁠, then re-allocated non-fixed allocation probabilities are all raised to this power (followed by rescaling to sum to 1) to make adaptive allocation probabilities less extreme, in turn used to redistribute remaining probability while respecting limits when defined by min_probs and/or max_probs. If 1, then no softening is applied.

match_arm

index of the control arm. If not NULL (default), the control arm allocation probability will be similar to that of the best non-control arm. Must be NULL in designs without a common control arm.

rescale_fixed

logical indicating whether fixed_probs should be rescaled following arm dropping.

rescale_limits

logical indicating whether min/max_probs should be rescaled following arm dropping.

rescale_factor

numerical, rescale factor defined as ⁠initial number of arms/number of active arms⁠.

rescale_ignore

NULL or index of an arm that will be ignored by the rescale_fixed and rescale_limits arguments.

Value

A named (according to the arms) numeric vector with updated allocation probabilities.


Replace non-finite values with other value (finite-OR-operator)

Description

Used internally, helper function that replaces non-finite (i.e., NA, NaN, Inf, and -Inf) values according to is.finite(), primarily used to replace NaN/Inf/-Inf with NA.

Usage

a %f|% b

Arguments

a

atomic vector of any type.

b

single value to replace non-finite values with.

Value

If values in a are non-finite, they are replaced with b, otherwise they are left unchanged.


Replace NULL with other value (NULL-OR-operator)

Description

Used internally, primarily when working with list arguments, because, e.g., list_name$element_name yields NULL when unspecified.

Usage

a %||% b

Arguments

a, b

atomic values of any type.

Value

If a is NULL, b is returned. Otherwise a is returned.


Rescale numeric vector to sum to 1

Description

Used internally.

Usage

rescale(x)

Arguments

x

numeric vector.

Value

Numeric vector, x rescaled to sum to a total of 1.


Simulate a single trial

Description

This function conducts a single trial simulation using a trial specification as specified by setup_trial(), setup_trial_binom() or setup_trial_norm().
During simulation, the function randomises "patients", randomly generates outcomes, calculates the probabilities that each arm is the best (and better than the control, if any). This is followed by checking inferiority, superiority, equivalence and/or futility as desired; dropping arms, and re-adjusting allocation probabilities according to the criteria specified in the trial specification. If there is no common control arm, the trial simulation will be stopped at the final specified adaptive analysis, when 1 arm is superior to the others, or when all arms are considered equivalent (if equivalence is assessed). If a common control arm is specified, all other arms will be compared to that, and if 1 of these pairwise comparisons crosses the applicable superiority threshold at an adaptive analysis, that arm will become the new control and the old control will be considered inferior and dropped. If multiple non-control arms cross the applicable superiority threshold in the same adaptive analysis, the one with the highest probability of being the overall best will become the new control. Equivalence/futility will also be checked if specified, and equivalent or futile arms will be dropped in designs with a common control arm and the entire trial will be stopped if all remaining arms are equivalent in designs without a common control arm. The trial simulation will be stopped when only 1 arm is left, when the final arms are all equivalent, or after the final specified adaptive analysis.
After stopping (regardless of reason), a final analysis including outcome data from all patients randomised to all arms will be conducted (with the final control arm, if any, used as the control in this analysis). Results from this analysis will be saved, but not used with regards to the adaptive stopping rules. This is particularly relevant if less patients have available outcome data at the last adaptive analyses than the total number of patients randomised (as specified in setup_trial(), setup_trial_binom(), or setup_trial_norm()), as the final analysis will then include all patients randomised, which may be more than in the last adaptive analysis conducted.

Usage

run_trial(trial_spec, seed = NULL, sparse = FALSE)

Arguments

trial_spec

trial_spec object, generated and validated by the setup_trial(), setup_trial_binom() or setup_trial_norm() function.

seed

single integer or NULL (default). If a value is provided, this value will be used as the random seed when running and the global random seed will be restored after the function has run, so it is not affected.

sparse

single logical; if FALSE (default) everything listed below is included in the returned object. If TRUE, only a limited amount of data are included in the returned object. This can be practical when running many simulations and saving the results using the run_trials() function (which relies on this function), as the output file will thus be substantially smaller. However, printing of individual trial results will be substantially less detailed for sparse results and non-sparse results are required by plot_history().

Value

A trial_result object containing everything listed below if sparse (as described above) is FALSE. Otherwise only final_status, final_n, followed_n, trial_res, seed, and sparse are included.

Examples

# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
                                 true_ys = c(0.20, 0.18, 0.22, 0.24),
                                 data_looks = 1:20 * 100)

# Run trial with a specified random seed
res <- run_trial(binom_trial, seed = 12345)

# Print results with 3 decimals
print(res, digits = 3)


Simulate multiple trials

Description

This function conducts multiple simulations using a trial specification as specified by setup_trial(), setup_trial_binom() or setup_trial_norm(). This function essentially manages random seeds and runs multiple simulation using run_trial() - additional details on individual simulations are provided in that function's description. This function allows simulating trials in parallel using multiple cores, automatically saving and re-loading saved objects, and "growing" already saved simulation files (i.e., appending additional simulations to the same file).

Usage

run_trials(
  trial_spec,
  n_rep,
  path = NULL,
  overwrite = FALSE,
  grow = FALSE,
  cores = NULL,
  base_seed = NULL,
  sparse = TRUE,
  progress = NULL,
  version = NULL,
  compress = TRUE,
  export = NULL,
  export_envir = parent.frame()
)

Arguments

trial_spec

trial_spec object, generated and validated by the setup_trial(), setup_trial_binom() or setup_trial_norm() function.

n_rep

single integer; the number of simulations to run.

path

single character string; if specified (defaults to NULL), files will be written to and loaded from this path using the saveRDS() / readRDS() functions.

overwrite

single logical; defaults to FALSE, in which case previous simulations saved in the same path will be re-loaded (if the same trial specification was used). If TRUE, the previous file is overwritten (even if the the same trial specification was not used). If grow is TRUE, this argument must be set to FALSE.

grow

single logical; defaults to FALSE. If TRUE and a valid path to a valid previous file containing less simulations than n_rep, the additional number of simulations will be run (appropriately re-using the same base_seed, if specified) and appended to the same file.

cores

NULL or single integer. If NULL, a default value/cluster set by setup_cluster() will be used to control whether simulations are run in parallel on a default cluster or sequentially in the main process; if a cluster/value has not been specified by setup_cluster(), cores will then be set to the value stored in the global "mc.cores" option (if previously set by ⁠options(mc.cores = <number of cores>⁠), and 1 if that option has not been specified.
If the resulting number of cores = 1, computations will be run sequentially in the primary process, and if cores > 1, a new parallel cluster will be setup using the parallel library and removed once the function completes. See setup_cluster() for details.

base_seed

single integer or NULL (default); a random seed used as the basis for simulations. Regardless of whether simulations are run sequentially or in parallel, random number streams will be identical and appropriate (see setup_cluster() for details).

sparse

single logical, as described in run_trial(); defaults to TRUE when running multiple simulations, in which case only the data necessary to summarise all simulations are saved for each simulation. If FALSE, more detailed data for each simulation is saved, allowing more detailed printing of individual trial results and plotting using plot_history() (plot_status() does not require non-sparse results).

progress

single numeric ⁠> 0⁠ and ⁠<= 1⁠ or NULL. If NULL (default), no progress is printed to the console. Otherwise, progress messages are printed to the control at intervals proportional to the value specified by progress.
Note: as printing is not possible from within clusters on multiple cores, the function conducts batches of simulations on multiple cores (if specified), with intermittent printing of statuses. Thus, all cores have to finish running their current assigned batches before the other cores may proceed with the next batch. If there are substantial differences in the simulation speeds across cores, using progress may thus increase total run time (especially with small values).

version

passed to saveRDS() when saving simulations, defaults to NULL (as in saveRDS()), which means that the current default version is used. Ignored if simulations are not saved.

compress

passed to saveRDS() when saving simulations, defaults to TRUE (as in saveRDS()), see saveRDS() for other options. Ignored if simulations are not saved.

export

character vector of names of objects to export to each parallel core when running in parallel; passed as the varlist argument to parallel::clusterExport(). Defaults to NULL (no objects exported), ignored if cores == 1. See Details below.

export_envir

environment where to look for the objects defined in export when running in parallel and export is not NULL. Defaults to the environment from where the function is called.

Details

Exporting objects when using multiple cores

If setup_trial() is used to define a trial specification with custom functions (in the fun_y_gen, fun_draws, and fun_raw_est arguments of setup_trial()) and run_trials() is run with cores > 1, it is necessary to export additional functions or objects used by these functions and defined by the user outside the function definitions provided. Similarly, functions from external packages loaded using library() or require() must be exported or called prefixed with the namespace, i.e., ⁠package::function⁠. The export and export_envir arguments are used to export objects calling the parallel::clusterExport()-function. See also setup_cluster(), which may be used to setup a cluster and export required objects only once per session.

Value

A list of a special class "trial_results", which contains the trial_results (results from all simulations; note that seed will be NULL in the individual simulations), trial_spec (the trial specification), n_rep, base_seed, elapsed_time (the total simulation run time), sparse (as described above) and adaptr_version (the version of the adaptr package used to run the simulations). These results may be extracted, summarised, and plotted using the extract_results(), check_performance(), summary(), print.trial_results(), plot_convergence(), check_remaining_arms(), plot_status(), and plot_history() functions. See the definitions of these functions for additional details and details on additional arguments used to select arms in simulations not ending in superiority and other summary choices.

Examples

# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
                                 true_ys = c(0.20, 0.18, 0.22, 0.24),
                                 data_looks = 1:20 * 100)

# Run 10 simulations with a specified random base seed
res <- run_trials(binom_trial, n_rep = 10, base_seed = 12345)

# See ?extract_results, ?check_performance, ?summary and ?print for details
# on extracting resutls, summarising and printing


Setup default cluster for use in parallelised adaptr functions

Description

This function setups (or removes) a default cluster for use in all parallelised functions in adaptr using the parallel package. The function also exports objects that should be available on the cluster and sets the random number generator appropriately. See Details for further info on how adaptr handles sequential/parallel computation.

Usage

setup_cluster(cores, export = NULL, export_envir = parent.frame())

Arguments

cores

can be either unspecified, NULL, or a single integer ⁠> 0⁠. If NULL or 1, an existing default cluster is removed (if any), and the default will subsequently be to run functions sequentially in the main process if cores = 1, and according to getOption("mc.cores") if NULL (unless otherwise specified in individual functions calls). The parallel::detectCores() function may be used to see the number of available cores, although this comes with some caveats (as described in the function documentation), including that the number of cores may not always be returned and may not match the number of cores that are available for use. In general, using less cores than available may be preferable if other processes are run on the machine at the same time.

export

character vector of names of objects to export to each parallel core when running in parallel; passed as the varlist argument to parallel::clusterExport(). Defaults to NULL (no objects exported), ignored if cores == 1. See Details below.

export_envir

environment where to look for the objects defined in export when running in parallel and export is not NULL. Defaults to the environment from where the function is called.

Details

Using sequential or parallel computing in adaptr

All parallelised adaptr functions have a cores argument that defaults to NULL. If a non-NULL integer ⁠> 0⁠ is provided to the cores argument in any of those (except setup_cluster()), the package will run calculations sequentially in the main process if cores = 1, and otherwise initiate a new cluster of size cores that will be removed once the function completes, regardless of whether or not a default cluster or the global "mc.cores" option have been specified.

If cores is NULL in any adaptr function (except setup_cluster()), the package will use a default cluster if one exists or run computations sequentially if setup_cluster() has last been called with cores = 1. If setup_cluster() has not been called or last called with cores = NULL, then the package will check if the global "mc.cores" option has been specified (using ⁠options(mc.cores = <number of cores>)⁠). If this option has been set with a value ⁠> 1⁠, then a new, temporary cluster of that size is setup, used, and removed once the function completes. If this option has not been set or has been set to 1, then computations will be run sequentially in the main process.

Generally, we recommend using the setup_cluster() function as this avoids the overhead of re-initiating new clusters with every call to one of the parallelised adaptr functions. This is especially important when exporting many or large objects to a parallel cluster, as this can then be done only once (with the option to export further objects to the same cluster when calling run_trials()).

Type of clusters used and random number generation

The adaptr package solely uses parallel socket clusters (using parallel::makePSOCKcluster()) and thus does not use forking (as this is not available on all operating systems and may cause crashes in some situations). As such, user-defined objects that should be used by the adaptr functions when run in parallel need to be exported using either setup_cluster() or run_trials(), if not included in the generated trial_spec object.

The adaptr package uses the "L'Ecuyer-CMRG" kind (see RNGkind()) for safe random number generation for all parallelised functions. This is also the case when running adaptr functions sequentially with a seed provided, to ensure that the same results are obtained regardless of whether sequential or parallel computation is used. All functions restore both the random number generator kind and the global random seed after use if called with a seed.

Value

Invisibly returns the default parallel cluster or NULL, as appropriate. This may be used with other functions from the parallel package by advanced users, for example to load certain libraries on the cluster prior to calling run_trials().

Examples


# Setup a cluster using 2 cores
setup_cluster(cores = 2)

# Get existing default cluster (printed here as invisibly returned)
print(setup_cluster())

# Remove existing default cluster
setup_cluster(cores = NULL)

# Specify preference for running computations sequentially
setup_cluster(cores = 1)

# Remove default cluster preference
setup_cluster(cores = NULL)

# Set global option to default to using 2 new clusters each time
# (only used if no default cluster preference is specified)
options(mc.cores = 2)


Setup a generic trial specification

Description

Specifies the design of an adaptive trial with any type of outcome and validates all inputs. Use calibrate_trial() to calibrate the trial specification to obtain a specific value for a certain performance metric (e.g., the Bayesian type 1 error rate). Use run_trial() or run_trials() to conduct single/multiple simulations of the specified trial, respectively.
See setup_trial_binom() and setup_trial_norm() for simplified setup of trial designs for common outcome types. For additional trial specification examples, see the the Basic examples vignette (vignette("Basic-examples", package = "adaptr")) and the Advanced example vignette (vignette("Advanced-example", package = "adaptr")).

Usage

setup_trial(
  arms,
  true_ys,
  fun_y_gen = NULL,
  fun_draws = NULL,
  start_probs = NULL,
  fixed_probs = NULL,
  min_probs = rep(NA, length(arms)),
  max_probs = rep(NA, length(arms)),
  rescale_probs = NULL,
  data_looks = NULL,
  max_n = NULL,
  look_after_every = NULL,
  randomised_at_looks = NULL,
  control = NULL,
  control_prob_fixed = NULL,
  inferiority = 0.01,
  superiority = 0.99,
  equivalence_prob = NULL,
  equivalence_diff = NULL,
  equivalence_only_first = NULL,
  futility_prob = NULL,
  futility_diff = NULL,
  futility_only_first = NULL,
  highest_is_best = FALSE,
  soften_power = 1,
  fun_raw_est = mean,
  cri_width = 0.95,
  n_draws = 5000,
  robust = TRUE,
  description = NULL,
  add_info = NULL
)

Arguments

arms

character vector with unique names for the trial arms.

true_ys

numeric vector specifying true outcomes (e.g., event probabilities, mean values, etc.) for all trial arms.

fun_y_gen

function, generates outcomes. See setup_trial() Details for information on how to specify this function.
Note: this function is called once during setup to validate its output (with the global random seed restored afterwards).

fun_draws

function, generates posterior draws. See setup_trial() Details for information on how to specify this function.
Note: this function is called up to three times during setup to validate its output (with the global random seed restored afterwards).

start_probs

numeric vector, allocation probabilities for each arm at the beginning of the trial. The default (NULL) automatically generates equal randomisation probabilities for each arm.

fixed_probs

numeric vector, fixed allocation probabilities for each arm. Must be either a numeric vector with NA for arms without fixed probabilities and values between 0 and 1 for the other arms or NULL (default), if adaptive randomisation is used for all arms or if one of the special settings ("sqrt-based", "sqrt-based start", "sqrt-based fixed", or "match") is specified for control_prob_fixed (described below).

min_probs

numeric vector, lower threshold for adaptive allocation probabilities; lower probabilities will be rounded up to these values. Must be NA (default for all arms) if no lower threshold is wanted and for arms using fixed allocation probabilities.

max_probs

numeric vector, upper threshold for adaptive allocation probabilities; higher probabilities will be rounded down to these values. Must be NA (default for all arms) if no threshold is wanted and for arms using fixed allocation probabilities.

rescale_probs

NULL (default) or one of either "fixed", "limits", or "both". Rescales fixed_probs (if "fixed" or "both") and min_probs/max_probs (if "limits" or "both") after arm dropping in trial specifications with ⁠>2 arms⁠ using a rescale_factor defined as ⁠initial number of arms/number of active arms⁠. ⁠"fixed_probs⁠ and min_probs are rescaled as ⁠initial value * rescale factor⁠, except for fixed_probs controlled by the control_prob_fixed argument, which are never rescaled. max_probs are rescaled as ⁠1 - ( (1 - initial value) * rescale_factor)⁠.
Must be NULL if there are only ⁠2 arms⁠ or if control_prob_fixed is "sqrt-based fixed". If not NULL, one or more valid non-NA values must be specified for either min_probs/max_probs or fixed_probs (not counting a fixed value for the original control if control_prob_fixed is "sqrt-based"/"sqrt-based start"/"sqrt-based fixed").
Note: using this argument and specific combinations of values in the other arguments may lead to invalid combined (total) allocation probabilities after arm dropping, in which case all probabilities will ultimately be rescaled to sum to 1. It is the responsibility of the user to ensure that rescaling fixed allocation probabilities and minimum/maximum allocation probability limits will not lead to invalid or unexpected allocation probabilities after arm dropping.
Finally, any initial values that are overwritten by the control_prob_fixed argument after arm dropping will not be rescaled.

data_looks

vector of increasing integers, specifies when to conduct adaptive analyses (= the total number of patients with available outcome data at each adaptive analysis). The last number in the vector represents the final adaptive analysis, i.e., the final analysis where superiority, inferiority, practical equivalence, or futility can be claimed. Instead of specifying data_looks, the max_n and look_after_every arguments can be used in combination (in which case data_looks must be NULL, the default value).

max_n

single integer, number of patients with available outcome data at the last possible adaptive analysis (defaults to NULL). Must only be specified if data_looks is NULL. Requires specification of the look_after_every argument.

look_after_every

single integer, specified together with max_n. Adaptive analyses will be conducted after every look_after_every patients have available outcome data, and at the total sample size as specified by max_n (max_n does not need to be a multiple of look_after_every). If specified, data_looks must be NULL (default).

randomised_at_looks

vector of increasing integers or NULL, specifying the number of patients randomised at the time of each adaptive analysis, with new patients randomised using the current allocation probabilities at said analysis. If NULL (the default), the number of patients randomised at each analysis will match the number of patients with available outcome data at said analysis, as specified by data_looks or max_n and look_after_every, i.e., outcome data will be available immediately after randomisation for all patients.
If not NULL, the vector must be of the same length as the number of adaptive analyses specified by data_looks or max_n and look_after_every, and all values must be larger than or equal to the number of patients with available outcome data at each analysis.

control

single character string, name of one of the arms or NULL (default). If specified, this arm will serve as a common control arm, to which all other arms will be compared and the inferiority/superiority/equivalence thresholds (see below) will be for those comparisons. See setup_trial() Details for information on behaviour with respect to these comparisons.

control_prob_fixed

if a common control arm is specified, this can be set NULL (the default), in which case the control arm allocation probability will not be fixed if control arms change (the allocation probability for the first control arm may still be fixed using fixed_probs, but will not be 'reused' for the new control arm).
If not NULL, a vector of probabilities of either length 1 or ⁠number of arms - 1⁠ can be provided, or one of the special arguments "sqrt-based", "sqrt-based start", "sqrt-based fixed" or "match".
See setup_trial() Details for details on how this affects trial behaviour.

inferiority

single numeric value or vector of numeric values of the same length as the maximum number of possible adaptive analyses, specifying the probability threshold(s) for inferiority (default is 0.01). All values must be ⁠>= 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no values may be lower than the preceding value. If a common controlis not used, all values must be ⁠< 1 / number of arms⁠. An arm will be considered inferior and dropped if the probability that it is best (when comparing all arms) or better than the control arm (when a common control is used) drops below the inferiority threshold at an adaptive analysis.

superiority

single numeric value or vector of numeric values of the same length as the maximum number of possible adaptive analyses, specifying the probability threshold(s) for superiority (default is 0.99). All values must be ⁠>= 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no values may be higher than the preceding value. If the probability that an arm is best (when comparing all arms) or better than the control arm (when a common control is used) exceeds the superiority threshold at an adaptive analysis, said arm will be declared the winner and the trial will be stopped (if no common control is used or if the last comparator is dropped in a design with a common control) or become the new control and the trial will continue (if a common control is specified).

equivalence_prob

single numeric value, vector of numeric values of the same length as the maximum number of possible adaptive analyses or NULL (default, corresponding to no equivalence assessment), specifying the probability threshold(s) for equivalence. If not NULL, all values must be ⁠> 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no value may be higher than the preceding value. If not NULL, arms will be dropped for equivalence if the probability of either (a) equivalence compared to a common control or (b) equivalence between all arms remaining (designs without a common control) exceeds the equivalence threshold at an adaptive analysis. Requires specification of equivalence_diff and equivalence_only_first.

equivalence_diff

single numeric value (⁠> 0⁠) or NULL (default, corresponding to no equivalence assessment). If a numeric value is specified, estimated absolute differences smaller than this threshold will be considered equivalent. For designs with a common control arm, the differences between each non-control arm and the control arm is used, and for trials without a common control arm, the difference between the highest and lowest estimated outcome rates are used and the trial is only stopped for equivalence if all remaining arms are equivalent.

equivalence_only_first

single logical in trial specifications where equivalence_prob and equivalence_diff are specified and a common control arm is included, otherwise NULL (default). If a common control arm is used, this specifies whether equivalence will only be assessed for the first control (if TRUE) or also for subsequent control arms (if FALSE) if one arm is superior to the first control and becomes the new control.

futility_prob

single numeric value, vector of numeric values of the same length as the maximum number of possible adaptive analyses or NULL (default, corresponding to no futility assessment), specifying the probability threshold(s) for futility. All values must be ⁠> 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no value may be higher than the preceding value. If not NULL, arms will be dropped for futility if the probability for futility compared to the common control exceeds the futility threshold at an adaptive analysis. Requires a common control arm (otherwise this argument must be NULL), specification of futility_diff, and futility_only_first.

futility_diff

single numeric value (⁠> 0⁠) or NULL (default, corresponding to no futility assessment). If a numeric value is specified, estimated differences below this threshold in the beneficial direction (as specified in highest_is_best) will be considered futile when assessing futility in designs with a common control arm. If only 1 arm remains after dropping arms for futility, the trial will be stopped without declaring the last arm superior.

futility_only_first

single logical in trial specifications designs where futility_prob and futility_diff are specified, otherwise NULL (default and required in designs without a common control arm). Specifies whether futility will only be assessed against the first control (if TRUE) or also for subsequent control arms (if FALSE) if one arm is superior to the first control and becomes the new control.

highest_is_best

single logical, specifies whether larger estimates of the outcome are favourable or not; defaults to FALSE, corresponding to, e.g., an undesirable binary outcomes (e.g., mortality) or a continuous outcome where lower numbers are preferred (e.g., hospital length of stay).

soften_power

either a single numeric value or a numeric vector of exactly the same length as the maximum number of looks/adaptive analyses. Values must be between 0 and 1 (default); if ⁠< 1⁠, then re-allocated non-fixed allocation probabilities are all raised to this power (followed by rescaling to sum to 1) to make adaptive allocation probabilities less extreme, in turn used to redistribute remaining probability while respecting limits when defined by min_probs and/or max_probs. If 1, then no softening is applied.

fun_raw_est

function that takes a numeric vector and returns a single numeric value, used to calculate a raw summary estimate of the outcomes in each arm. Defaults to mean(), which is always used in the setup_trial_binom() and setup_trial_norm() functions.
Note: the function is called one time per arm during setup to validate the output structure.

cri_width

single numeric ⁠>= 0⁠ and ⁠< 1⁠, the width of the percentile-based credible intervals used when summarising individual trial results. Defaults to 0.95, corresponding to 95% credible intervals.

n_draws

single integer, the number of draws from the posterior distributions for each arm used when running the trial. Defaults to 5000; can be reduced for a speed gain (at the potential loss of stability of results if too low) or increased for increased precision (increasing simulation time). Values ⁠< 100⁠ are not allowed and values ⁠< 1000⁠ are not recommended and warned against.

robust

single logical, if TRUE (default) the medians and median absolute deviations (scaled to be comparable to the standard deviation for normal distributions; MAD_SDs, see stats::mad()) are used to summarise the posterior distributions; if FALSE, the means and standard deviations (SDs) are used instead (slightly faster, but may be less appropriate for posteriors skewed on the natural scale).

description

optional single character string describing the trial design, will only be used in print functions if not NULL (the default).

add_info

optional single string containing additional information regarding the trial design or specifications, will only be used in print functions if not NULL (the default).

Details

How to specify the fun_y_gen function

The function must take the following arguments:

The function must return a single numeric vector, corresponding to the outcomes for all patients allocated since the last adaptive analysis, in the same order as allocs.
See the Advanced example vignette (vignette("Advanced-example", package = "adaptr")) for an example with further details.

How to specify the fun_draws function

The function must take the following arguments:

The function must return a matrix (containing numeric values) with arms named columns and n_draws rows. The matrix must have columns only for currently active arms (when called). Each row should contain a single posterior draw for each arm on the original outcome scale: if they are estimated as, e.g., the log(odds), these estimates must be transformed to probabilities and similarly for other measures.
Important: the matrix cannot contain NAs, even if no patients have been randomised to an arm yet. See the provided example for one way to alleviate this.
See the Advanced examples vignette (vignette("Advanced-example", package = "adaptr")) for an example with further details.

Notes

Using additional custom or functions from loaded packages in the custom functions

If the fun_y_gen, fun_draws, or fun_raw_est functions calls other user-specified functions (or uses objects defined by the user outside these functions or the setup_trial()-call) or functions from external packages and simulations are conducted on multiple cores, these objects or functions must be prefixed with their namespaces (i.e., ⁠package::function()⁠) or exported, as described in setup_cluster() and run_trials().

More information on arguments

Superiority and inferiority

In trial designs without a common control arm, superiority and inferiority are assessed by comparing all currently active groups. This means that if a "final" analysis of a trial without a common control and ⁠> 2 arms⁠ is conducted including all arms (as will often be done in practice) after an adaptive trial has stopped, the final probabilities of the best arm being superior may differ slightly.
For example, in a trial with three arms and no common control arm, one arm may be dropped early for inferiority defined as ⁠< 1%⁠ probability of being the overall best arm. The trial may then continue with the two remaining arms, and stopped when one is declared superior to the other defined as ⁠> 99%⁠ probability of being the overall best arm. If a final analysis is then conducted including all arms, the final probability of the best arm being overall superior will generally be slightly lower as the probability of the first dropped arm being the best will often be ⁠> 0%⁠, even if very low and below the inferiority threshold.
This is less relevant trial designs with a common control, as pairwise assessments of superiority/inferiority compared to the common control will not be influenced similarly by previously dropped arms (and previously dropped arms may be included in the analyses, even if posterior distributions are not returned for those). Similarly, in actual clinical trials and when randomised_at_looks is specified with numbers higher than the number of patients with available outcome data at each analysis, final probabilities may change somewhat when the all patients are have completed follow-up and are included in a final analysis.

Equivalence

Equivalence is assessed after both inferiority and superiority have been assessed (and in case of superiority, it will be assessed against the new control arm in designs with a common control, if specified - see above).

Futility

Futility is assessed after inferiority, superiority, and equivalence have been assessed (and in case of superiority, it will be assessed against the new control arm in designs with a common control, if specified - see above). Arms will thus be dropped for equivalence before futility.

Varying probability thresholds

Different probability thresholds (for superiority, inferiority, equivalence, and futility) may be specified for different adaptive analyses. This may be used, e.g., to apply more strict probability thresholds at earlier analyses (or make one or more stopping rules not apply at earlier analyses), similar to the use of monitoring boundaries with different thresholds used for interim analyses in conventional, frequentist group sequential trial designs. See the Basic examples vignette (vignette("Basic-examples", package = "adaptr")) for an example.

Value

A trial_spec object used to run simulations by run_trial() or run_trials(). The output is essentially a list containing the input values (some combined in a data.frame called trial_arms), but its class signals that these inputs have been validated and inappropriate combinations and settings have been ruled out. Also contains best_arm, holding the arm(s) with the best value(s) in true_ys. Use str() to peruse the actual content of the returned object.

Examples

# Setup a custom trial specification with right-skewed, log-normally
# distributed continuous outcomes (higher values are worse)

# Define the function that will generate the outcomes in each arm
# Notice: contents should match arms/true_ys in the setup_trial() call below
get_ys_lognorm <- function(allocs) {
  y <- numeric(length(allocs))
  # arms (names and order) and values (except for exponentiation) should match
  # those used in setup_trial (below)
  means <- c("Control" = 2.2, "Experimental A" = 2.1, "Experimental B" = 2.3)
  for (arm in names(means)) {
    ii <- which(allocs == arm)
    y[ii] <- rlnorm(length(ii), means[arm], 1.5)
  }
  y
}

# Define the function that will generate posterior draws
# In this example, the function uses no priors (corresponding to improper
# flat priors) and calculates results on the log-scale, before exponentiating
# back to the natural scale, which is required for assessments of
# equivalence, futility and general interpretation
get_draws_lognorm <- function(arms, allocs, ys, control, n_draws) {
  draws <- list()
  logys <- log(ys)
  for (arm in arms){
    ii <- which(allocs == arm)
    n <- length(ii)
    if (n > 1) {
      # Necessary to avoid errors if too few patients randomised to this arm
      draws[[arm]] <- exp(rnorm(n_draws, mean = mean(logys[ii]), sd = sd(logys[ii])/sqrt(n - 1)))
    } else {
      # Too few patients randomised to this arm - extreme uncertainty
      draws[[arm]] <- exp(rnorm(n_draws, mean = mean(logys), sd = 1000 * (max(logys) - min(logys))))
    }
  }
  do.call(cbind, draws)
}

# The actual trial specification is then defined
lognorm_trial <- setup_trial(
  # arms should match those above
  arms = c("Control", "Experimental A", "Experimental B"),
  # true_ys should match those above
  true_ys = exp(c(2.2, 2.1, 2.3)),
  fun_y_gen = get_ys_lognorm, # as specified above
  fun_draws = get_draws_lognorm, # as specified above
  max_n = 5000,
  look_after_every = 200,
  control = "Control",
  # Square-root-based, fixed control group allocation ratio
  # and response-adaptive randomisation for other arms
  control_prob_fixed = "sqrt-based",
  # Equivalence assessment
  equivalence_prob = 0.9,
  equivalence_diff = 0.5,
  equivalence_only_first = TRUE,
  highest_is_best = FALSE,
  # Summarise raw results by taking the mean on the
  # log scale and back-transforming
  fun_raw_est = function(x) exp(mean(log(x))) ,
  # Summarise posteriors using medians with MAD-SDs,
  # as distributions will not be normal on the actual scale
  robust = TRUE,
  # Description/additional info used when printing
  description = "continuous, log-normally distributed outcome",
  add_info = "SD on the log scale for all arms: 1.5"
)

# Print trial specification with 3 digits for all probabilities
print(lognorm_trial, prob_digits = 3)


Setup a trial specification using a binary, binomially distributed outcome

Description

Specifies the design of an adaptive trial with a binary, binomially distributed outcome and validates all inputs. Uses beta-binomial conjugate models with beta(1, 1) prior distributions, corresponding to a uniform prior (or the addition of 2 patients, 1 with an event and 1 without, in each arm) to the trial. Use calibrate_trial() to calibrate the trial specification to obtain a specific value for a certain performance metric (e.g., the Bayesian type 1 error rate). Use run_trial() or run_trials() to conduct single/multiple simulations of the specified trial, respectively.
Note: add_info as specified in setup_trial() is set to NULL for trial specifications setup by this function.
Further details: please see setup_trial(). See setup_trial_norm() for simplified setup of trials with a normally distributed continuous outcome.
For additional trial specification examples, see the the Basic examples vignette (vignette("Basic-examples", package = "adaptr")) and the Advanced example vignette (vignette("Advanced-example", package = "adaptr")).

Usage

setup_trial_binom(
  arms,
  true_ys,
  start_probs = NULL,
  fixed_probs = NULL,
  min_probs = rep(NA, length(arms)),
  max_probs = rep(NA, length(arms)),
  rescale_probs = NULL,
  data_looks = NULL,
  max_n = NULL,
  look_after_every = NULL,
  randomised_at_looks = NULL,
  control = NULL,
  control_prob_fixed = NULL,
  inferiority = 0.01,
  superiority = 0.99,
  equivalence_prob = NULL,
  equivalence_diff = NULL,
  equivalence_only_first = NULL,
  futility_prob = NULL,
  futility_diff = NULL,
  futility_only_first = NULL,
  highest_is_best = FALSE,
  soften_power = 1,
  cri_width = 0.95,
  n_draws = 5000,
  robust = TRUE,
  description = "generic binomially distributed outcome trial"
)

Arguments

arms

character vector with unique names for the trial arms.

true_ys

numeric vector, true probabilities (between 0 and 1) of outcomes in all trial arms.

start_probs

numeric vector, allocation probabilities for each arm at the beginning of the trial. The default (NULL) automatically generates equal randomisation probabilities for each arm.

fixed_probs

numeric vector, fixed allocation probabilities for each arm. Must be either a numeric vector with NA for arms without fixed probabilities and values between 0 and 1 for the other arms or NULL (default), if adaptive randomisation is used for all arms or if one of the special settings ("sqrt-based", "sqrt-based start", "sqrt-based fixed", or "match") is specified for control_prob_fixed (described below).

min_probs

numeric vector, lower threshold for adaptive allocation probabilities; lower probabilities will be rounded up to these values. Must be NA (default for all arms) if no lower threshold is wanted and for arms using fixed allocation probabilities.

max_probs

numeric vector, upper threshold for adaptive allocation probabilities; higher probabilities will be rounded down to these values. Must be NA (default for all arms) if no threshold is wanted and for arms using fixed allocation probabilities.

rescale_probs

NULL (default) or one of either "fixed", "limits", or "both". Rescales fixed_probs (if "fixed" or "both") and min_probs/max_probs (if "limits" or "both") after arm dropping in trial specifications with ⁠>2 arms⁠ using a rescale_factor defined as ⁠initial number of arms/number of active arms⁠. ⁠"fixed_probs⁠ and min_probs are rescaled as ⁠initial value * rescale factor⁠, except for fixed_probs controlled by the control_prob_fixed argument, which are never rescaled. max_probs are rescaled as ⁠1 - ( (1 - initial value) * rescale_factor)⁠.
Must be NULL if there are only ⁠2 arms⁠ or if control_prob_fixed is "sqrt-based fixed". If not NULL, one or more valid non-NA values must be specified for either min_probs/max_probs or fixed_probs (not counting a fixed value for the original control if control_prob_fixed is "sqrt-based"/"sqrt-based start"/"sqrt-based fixed").
Note: using this argument and specific combinations of values in the other arguments may lead to invalid combined (total) allocation probabilities after arm dropping, in which case all probabilities will ultimately be rescaled to sum to 1. It is the responsibility of the user to ensure that rescaling fixed allocation probabilities and minimum/maximum allocation probability limits will not lead to invalid or unexpected allocation probabilities after arm dropping.
Finally, any initial values that are overwritten by the control_prob_fixed argument after arm dropping will not be rescaled.

data_looks

vector of increasing integers, specifies when to conduct adaptive analyses (= the total number of patients with available outcome data at each adaptive analysis). The last number in the vector represents the final adaptive analysis, i.e., the final analysis where superiority, inferiority, practical equivalence, or futility can be claimed. Instead of specifying data_looks, the max_n and look_after_every arguments can be used in combination (in which case data_looks must be NULL, the default value).

max_n

single integer, number of patients with available outcome data at the last possible adaptive analysis (defaults to NULL). Must only be specified if data_looks is NULL. Requires specification of the look_after_every argument.

look_after_every

single integer, specified together with max_n. Adaptive analyses will be conducted after every look_after_every patients have available outcome data, and at the total sample size as specified by max_n (max_n does not need to be a multiple of look_after_every). If specified, data_looks must be NULL (default).

randomised_at_looks

vector of increasing integers or NULL, specifying the number of patients randomised at the time of each adaptive analysis, with new patients randomised using the current allocation probabilities at said analysis. If NULL (the default), the number of patients randomised at each analysis will match the number of patients with available outcome data at said analysis, as specified by data_looks or max_n and look_after_every, i.e., outcome data will be available immediately after randomisation for all patients.
If not NULL, the vector must be of the same length as the number of adaptive analyses specified by data_looks or max_n and look_after_every, and all values must be larger than or equal to the number of patients with available outcome data at each analysis.

control

single character string, name of one of the arms or NULL (default). If specified, this arm will serve as a common control arm, to which all other arms will be compared and the inferiority/superiority/equivalence thresholds (see below) will be for those comparisons. See setup_trial() Details for information on behaviour with respect to these comparisons.

control_prob_fixed

if a common control arm is specified, this can be set NULL (the default), in which case the control arm allocation probability will not be fixed if control arms change (the allocation probability for the first control arm may still be fixed using fixed_probs, but will not be 'reused' for the new control arm).
If not NULL, a vector of probabilities of either length 1 or ⁠number of arms - 1⁠ can be provided, or one of the special arguments "sqrt-based", "sqrt-based start", "sqrt-based fixed" or "match".
See setup_trial() Details for details on how this affects trial behaviour.

inferiority

single numeric value or vector of numeric values of the same length as the maximum number of possible adaptive analyses, specifying the probability threshold(s) for inferiority (default is 0.01). All values must be ⁠>= 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no values may be lower than the preceding value. If a common controlis not used, all values must be ⁠< 1 / number of arms⁠. An arm will be considered inferior and dropped if the probability that it is best (when comparing all arms) or better than the control arm (when a common control is used) drops below the inferiority threshold at an adaptive analysis.

superiority

single numeric value or vector of numeric values of the same length as the maximum number of possible adaptive analyses, specifying the probability threshold(s) for superiority (default is 0.99). All values must be ⁠>= 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no values may be higher than the preceding value. If the probability that an arm is best (when comparing all arms) or better than the control arm (when a common control is used) exceeds the superiority threshold at an adaptive analysis, said arm will be declared the winner and the trial will be stopped (if no common control is used or if the last comparator is dropped in a design with a common control) or become the new control and the trial will continue (if a common control is specified).

equivalence_prob

single numeric value, vector of numeric values of the same length as the maximum number of possible adaptive analyses or NULL (default, corresponding to no equivalence assessment), specifying the probability threshold(s) for equivalence. If not NULL, all values must be ⁠> 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no value may be higher than the preceding value. If not NULL, arms will be dropped for equivalence if the probability of either (a) equivalence compared to a common control or (b) equivalence between all arms remaining (designs without a common control) exceeds the equivalence threshold at an adaptive analysis. Requires specification of equivalence_diff and equivalence_only_first.

equivalence_diff

single numeric value (⁠> 0⁠) or NULL (default, corresponding to no equivalence assessment). If a numeric value is specified, estimated absolute differences smaller than this threshold will be considered equivalent. For designs with a common control arm, the differences between each non-control arm and the control arm is used, and for trials without a common control arm, the difference between the highest and lowest estimated outcome rates are used and the trial is only stopped for equivalence if all remaining arms are equivalent.

equivalence_only_first

single logical in trial specifications where equivalence_prob and equivalence_diff are specified and a common control arm is included, otherwise NULL (default). If a common control arm is used, this specifies whether equivalence will only be assessed for the first control (if TRUE) or also for subsequent control arms (if FALSE) if one arm is superior to the first control and becomes the new control.

futility_prob

single numeric value, vector of numeric values of the same length as the maximum number of possible adaptive analyses or NULL (default, corresponding to no futility assessment), specifying the probability threshold(s) for futility. All values must be ⁠> 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no value may be higher than the preceding value. If not NULL, arms will be dropped for futility if the probability for futility compared to the common control exceeds the futility threshold at an adaptive analysis. Requires a common control arm (otherwise this argument must be NULL), specification of futility_diff, and futility_only_first.

futility_diff

single numeric value (⁠> 0⁠) or NULL (default, corresponding to no futility assessment). If a numeric value is specified, estimated differences below this threshold in the beneficial direction (as specified in highest_is_best) will be considered futile when assessing futility in designs with a common control arm. If only 1 arm remains after dropping arms for futility, the trial will be stopped without declaring the last arm superior.

futility_only_first

single logical in trial specifications designs where futility_prob and futility_diff are specified, otherwise NULL (default and required in designs without a common control arm). Specifies whether futility will only be assessed against the first control (if TRUE) or also for subsequent control arms (if FALSE) if one arm is superior to the first control and becomes the new control.

highest_is_best

single logical, specifies whether larger estimates of the outcome are favourable or not; defaults to FALSE, corresponding to, e.g., an undesirable binary outcomes (e.g., mortality) or a continuous outcome where lower numbers are preferred (e.g., hospital length of stay).

soften_power

either a single numeric value or a numeric vector of exactly the same length as the maximum number of looks/adaptive analyses. Values must be between 0 and 1 (default); if ⁠< 1⁠, then re-allocated non-fixed allocation probabilities are all raised to this power (followed by rescaling to sum to 1) to make adaptive allocation probabilities less extreme, in turn used to redistribute remaining probability while respecting limits when defined by min_probs and/or max_probs. If 1, then no softening is applied.

cri_width

single numeric ⁠>= 0⁠ and ⁠< 1⁠, the width of the percentile-based credible intervals used when summarising individual trial results. Defaults to 0.95, corresponding to 95% credible intervals.

n_draws

single integer, the number of draws from the posterior distributions for each arm used when running the trial. Defaults to 5000; can be reduced for a speed gain (at the potential loss of stability of results if too low) or increased for increased precision (increasing simulation time). Values ⁠< 100⁠ are not allowed and values ⁠< 1000⁠ are not recommended and warned against.

robust

single logical, if TRUE (default) the medians and median absolute deviations (scaled to be comparable to the standard deviation for normal distributions; MAD_SDs, see stats::mad()) are used to summarise the posterior distributions; if FALSE, the means and standard deviations (SDs) are used instead (slightly faster, but may be less appropriate for posteriors skewed on the natural scale).

description

character string, default is "generic binomially distributed outcome trial". See arguments of setup_trial().

Value

A trial_spec object used to run simulations by run_trial() or run_trials(). The output is essentially a list containing the input values (some combined in a data.frame called trial_arms), but its class signals that these inputs have been validated and inappropriate combinations and settings have been ruled out. Also contains best_arm, holding the arm(s) with the best value(s) in true_ys. Use str() to peruse the actual content of the returned object.

Examples

# Setup a trial specification using a binary, binomially
# distributed, undesirable outcome
binom_trial <- setup_trial_binom(
  arms = c("Arm A", "Arm B", "Arm C"),
  true_ys = c(0.25, 0.20, 0.30),
  # Minimum allocation of 15% in all arms
  min_probs = rep(0.15, 3),
  data_looks = seq(from = 300, to = 2000, by = 100),
  # Stop for equivalence if > 90% probability of
  # absolute differences < 5 percentage points
  equivalence_prob = 0.9,
  equivalence_diff = 0.05,
  soften_power = 0.5 # Limit extreme allocation ratios
)

# Print using 3 digits for probabilities
print(binom_trial, prob_digits = 3)


Setup a trial specification using a continuous, normally distributed outcome

Description

Specifies the design of an adaptive trial with a continuous, normally distributed outcome and validates all inputs. Uses normally distributed posterior distributions for the mean values in each trial arm; technically, no priors are used (as using normal-normal conjugate prior models with extremely wide or uniform priors gives similar results for these simple, unadjusted estimates). This corresponds to the use of improper, flat priors, although not explicitly specified as such. Use calibrate_trial() to calibrate the trial specification to obtain a specific value for a certain performance metric (e.g., the Bayesian type 1 error rate). Use run_trial() or run_trials() to conduct single/multiple simulations of the specified trial, respectively.
Note: add_info as specified in setup_trial() is set to the arms and standard deviations used for trials specified using this function.
Further details: please see setup_trial(). See setup_trial_binom() for simplified setup of trials with binomially distributed binary outcomes.
For additional trial specification examples, see the the Basic examples vignette (vignette("Basic-examples", package = "adaptr")) and the Advanced example vignette (vignette("Advanced-example", package = "adaptr")).

Usage

setup_trial_norm(
  arms,
  true_ys,
  sds,
  start_probs = NULL,
  fixed_probs = NULL,
  min_probs = rep(NA, length(arms)),
  max_probs = rep(NA, length(arms)),
  rescale_probs = NULL,
  data_looks = NULL,
  max_n = NULL,
  look_after_every = NULL,
  randomised_at_looks = NULL,
  control = NULL,
  control_prob_fixed = NULL,
  inferiority = 0.01,
  superiority = 0.99,
  equivalence_prob = NULL,
  equivalence_diff = NULL,
  equivalence_only_first = NULL,
  futility_prob = NULL,
  futility_diff = NULL,
  futility_only_first = NULL,
  highest_is_best = FALSE,
  soften_power = 1,
  cri_width = 0.95,
  n_draws = 5000,
  robust = FALSE,
  description = "generic normally distributed outcome trial"
)

Arguments

arms

character vector with unique names for the trial arms.

true_ys

numeric vector, simulated means of the outcome in all trial arms.

sds

numeric vector, true standard deviations (must be ⁠> 0⁠) of the outcome in all trial arms.

start_probs

numeric vector, allocation probabilities for each arm at the beginning of the trial. The default (NULL) automatically generates equal randomisation probabilities for each arm.

fixed_probs

numeric vector, fixed allocation probabilities for each arm. Must be either a numeric vector with NA for arms without fixed probabilities and values between 0 and 1 for the other arms or NULL (default), if adaptive randomisation is used for all arms or if one of the special settings ("sqrt-based", "sqrt-based start", "sqrt-based fixed", or "match") is specified for control_prob_fixed (described below).

min_probs

numeric vector, lower threshold for adaptive allocation probabilities; lower probabilities will be rounded up to these values. Must be NA (default for all arms) if no lower threshold is wanted and for arms using fixed allocation probabilities.

max_probs

numeric vector, upper threshold for adaptive allocation probabilities; higher probabilities will be rounded down to these values. Must be NA (default for all arms) if no threshold is wanted and for arms using fixed allocation probabilities.

rescale_probs

NULL (default) or one of either "fixed", "limits", or "both". Rescales fixed_probs (if "fixed" or "both") and min_probs/max_probs (if "limits" or "both") after arm dropping in trial specifications with ⁠>2 arms⁠ using a rescale_factor defined as ⁠initial number of arms/number of active arms⁠. ⁠"fixed_probs⁠ and min_probs are rescaled as ⁠initial value * rescale factor⁠, except for fixed_probs controlled by the control_prob_fixed argument, which are never rescaled. max_probs are rescaled as ⁠1 - ( (1 - initial value) * rescale_factor)⁠.
Must be NULL if there are only ⁠2 arms⁠ or if control_prob_fixed is "sqrt-based fixed". If not NULL, one or more valid non-NA values must be specified for either min_probs/max_probs or fixed_probs (not counting a fixed value for the original control if control_prob_fixed is "sqrt-based"/"sqrt-based start"/"sqrt-based fixed").
Note: using this argument and specific combinations of values in the other arguments may lead to invalid combined (total) allocation probabilities after arm dropping, in which case all probabilities will ultimately be rescaled to sum to 1. It is the responsibility of the user to ensure that rescaling fixed allocation probabilities and minimum/maximum allocation probability limits will not lead to invalid or unexpected allocation probabilities after arm dropping.
Finally, any initial values that are overwritten by the control_prob_fixed argument after arm dropping will not be rescaled.

data_looks

vector of increasing integers, specifies when to conduct adaptive analyses (= the total number of patients with available outcome data at each adaptive analysis). The last number in the vector represents the final adaptive analysis, i.e., the final analysis where superiority, inferiority, practical equivalence, or futility can be claimed. Instead of specifying data_looks, the max_n and look_after_every arguments can be used in combination (in which case data_looks must be NULL, the default value).

max_n

single integer, number of patients with available outcome data at the last possible adaptive analysis (defaults to NULL). Must only be specified if data_looks is NULL. Requires specification of the look_after_every argument.

look_after_every

single integer, specified together with max_n. Adaptive analyses will be conducted after every look_after_every patients have available outcome data, and at the total sample size as specified by max_n (max_n does not need to be a multiple of look_after_every). If specified, data_looks must be NULL (default).

randomised_at_looks

vector of increasing integers or NULL, specifying the number of patients randomised at the time of each adaptive analysis, with new patients randomised using the current allocation probabilities at said analysis. If NULL (the default), the number of patients randomised at each analysis will match the number of patients with available outcome data at said analysis, as specified by data_looks or max_n and look_after_every, i.e., outcome data will be available immediately after randomisation for all patients.
If not NULL, the vector must be of the same length as the number of adaptive analyses specified by data_looks or max_n and look_after_every, and all values must be larger than or equal to the number of patients with available outcome data at each analysis.

control

single character string, name of one of the arms or NULL (default). If specified, this arm will serve as a common control arm, to which all other arms will be compared and the inferiority/superiority/equivalence thresholds (see below) will be for those comparisons. See setup_trial() Details for information on behaviour with respect to these comparisons.

control_prob_fixed

if a common control arm is specified, this can be set NULL (the default), in which case the control arm allocation probability will not be fixed if control arms change (the allocation probability for the first control arm may still be fixed using fixed_probs, but will not be 'reused' for the new control arm).
If not NULL, a vector of probabilities of either length 1 or ⁠number of arms - 1⁠ can be provided, or one of the special arguments "sqrt-based", "sqrt-based start", "sqrt-based fixed" or "match".
See setup_trial() Details for details on how this affects trial behaviour.

inferiority

single numeric value or vector of numeric values of the same length as the maximum number of possible adaptive analyses, specifying the probability threshold(s) for inferiority (default is 0.01). All values must be ⁠>= 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no values may be lower than the preceding value. If a common controlis not used, all values must be ⁠< 1 / number of arms⁠. An arm will be considered inferior and dropped if the probability that it is best (when comparing all arms) or better than the control arm (when a common control is used) drops below the inferiority threshold at an adaptive analysis.

superiority

single numeric value or vector of numeric values of the same length as the maximum number of possible adaptive analyses, specifying the probability threshold(s) for superiority (default is 0.99). All values must be ⁠>= 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no values may be higher than the preceding value. If the probability that an arm is best (when comparing all arms) or better than the control arm (when a common control is used) exceeds the superiority threshold at an adaptive analysis, said arm will be declared the winner and the trial will be stopped (if no common control is used or if the last comparator is dropped in a design with a common control) or become the new control and the trial will continue (if a common control is specified).

equivalence_prob

single numeric value, vector of numeric values of the same length as the maximum number of possible adaptive analyses or NULL (default, corresponding to no equivalence assessment), specifying the probability threshold(s) for equivalence. If not NULL, all values must be ⁠> 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no value may be higher than the preceding value. If not NULL, arms will be dropped for equivalence if the probability of either (a) equivalence compared to a common control or (b) equivalence between all arms remaining (designs without a common control) exceeds the equivalence threshold at an adaptive analysis. Requires specification of equivalence_diff and equivalence_only_first.

equivalence_diff

single numeric value (⁠> 0⁠) or NULL (default, corresponding to no equivalence assessment). If a numeric value is specified, estimated absolute differences smaller than this threshold will be considered equivalent. For designs with a common control arm, the differences between each non-control arm and the control arm is used, and for trials without a common control arm, the difference between the highest and lowest estimated outcome rates are used and the trial is only stopped for equivalence if all remaining arms are equivalent.

equivalence_only_first

single logical in trial specifications where equivalence_prob and equivalence_diff are specified and a common control arm is included, otherwise NULL (default). If a common control arm is used, this specifies whether equivalence will only be assessed for the first control (if TRUE) or also for subsequent control arms (if FALSE) if one arm is superior to the first control and becomes the new control.

futility_prob

single numeric value, vector of numeric values of the same length as the maximum number of possible adaptive analyses or NULL (default, corresponding to no futility assessment), specifying the probability threshold(s) for futility. All values must be ⁠> 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no value may be higher than the preceding value. If not NULL, arms will be dropped for futility if the probability for futility compared to the common control exceeds the futility threshold at an adaptive analysis. Requires a common control arm (otherwise this argument must be NULL), specification of futility_diff, and futility_only_first.

futility_diff

single numeric value (⁠> 0⁠) or NULL (default, corresponding to no futility assessment). If a numeric value is specified, estimated differences below this threshold in the beneficial direction (as specified in highest_is_best) will be considered futile when assessing futility in designs with a common control arm. If only 1 arm remains after dropping arms for futility, the trial will be stopped without declaring the last arm superior.

futility_only_first

single logical in trial specifications designs where futility_prob and futility_diff are specified, otherwise NULL (default and required in designs without a common control arm). Specifies whether futility will only be assessed against the first control (if TRUE) or also for subsequent control arms (if FALSE) if one arm is superior to the first control and becomes the new control.

highest_is_best

single logical, specifies whether larger estimates of the outcome are favourable or not; defaults to FALSE, corresponding to, e.g., an undesirable binary outcomes (e.g., mortality) or a continuous outcome where lower numbers are preferred (e.g., hospital length of stay).

soften_power

either a single numeric value or a numeric vector of exactly the same length as the maximum number of looks/adaptive analyses. Values must be between 0 and 1 (default); if ⁠< 1⁠, then re-allocated non-fixed allocation probabilities are all raised to this power (followed by rescaling to sum to 1) to make adaptive allocation probabilities less extreme, in turn used to redistribute remaining probability while respecting limits when defined by min_probs and/or max_probs. If 1, then no softening is applied.

cri_width

single numeric ⁠>= 0⁠ and ⁠< 1⁠, the width of the percentile-based credible intervals used when summarising individual trial results. Defaults to 0.95, corresponding to 95% credible intervals.

n_draws

single integer, the number of draws from the posterior distributions for each arm used when running the trial. Defaults to 5000; can be reduced for a speed gain (at the potential loss of stability of results if too low) or increased for increased precision (increasing simulation time). Values ⁠< 100⁠ are not allowed and values ⁠< 1000⁠ are not recommended and warned against.

robust

single logical, if TRUE (default) the medians and median absolute deviations (scaled to be comparable to the standard deviation for normal distributions; MAD_SDs, see stats::mad()) are used to summarise the posterior distributions; if FALSE, the means and standard deviations (SDs) are used instead (slightly faster, but may be less appropriate for posteriors skewed on the natural scale).

description

character string, default is "generic normally distributed outcome trial". See arguments of setup_trial().

Details

Because the posteriors used in this type of trial (with a generic, continuous, normally distributed outcome) are by definition normally distributed, FALSE is used as the default value for the robust argument.

Value

A trial_spec object used to run simulations by run_trial() or run_trials(). The output is essentially a list containing the input values (some combined in a data.frame called trial_arms), but its class signals that these inputs have been validated and inappropriate combinations and settings have been ruled out. Also contains best_arm, holding the arm(s) with the best value(s) in true_ys. Use str() to peruse the actual content of the returned object.

Examples

# Setup a trial specification using a continuous, normally distributed, desirable outcome
norm_trial <- setup_trial_norm(
  arms = c("Control", "New A", "New B", "New C"),
  true_ys = c(15, 20, 14, 13),
  sds = c(2, 2.5, 1.9, 1.8), # SDs in each arm
  max_n = 500,
  look_after_every = 50,
  control = "Control", # Common control arm
  # Square-root-based, fixed control group allocation ratios
  control_prob_fixed = "sqrt-based fixed",
  # Desirable outcome
  highest_is_best = TRUE,
  soften_power = 0.5 # Limit extreme allocation ratios
)

# Print using 3 digits for probabilities
print(norm_trial, prob_digits = 3)


stop() and warning() with call. = FALSE

Description

Used internally. Calls stop0() or warning() but enforces call. = FALSE, to suppress the call from the error/warning.

Usage

stop0(...)

warning0(...)

Arguments

...

zero or more objects which can be coerced to character (and which are pasted together with no separator) or a single condition object.


Summarise distribution

Description

Used internally, to summarise posterior distributions, but the logic does apply to any distribution (thus, the name).

Usage

summarise_dist(x, robust = TRUE, interval_width = 0.95)

Arguments

x

a numeric vector of posterior draws.

robust

single logical. if TRUE (default) the median and median absolute deviation (MAD-SD; scaled to be comparable to the standard deviation for normal distributions) are used to summarise the distribution; if FALSE, the mean and standard deviation (SD) are used instead (slightly faster, but may be less appropriate for skewed distribution).

interval_width

single numeric value (⁠> 0⁠ and ⁠<1⁠); the width of the interval; default is 0.95, corresponding to 95% percentile-base credible intervals for posterior distributions.

Details

MAD-SDs are scaled to correspond to SDs if distributions are normal, similarly to the stats::mad() function; see details regarding calculation in that function's description.

Value

A numeric vector with four named elements: est (the median/mean), err (the MAD-SD/SD), lo and hi (the lower and upper boundaries of the interval).


Summarise numeric vector

Description

Used internally, to summarise numeric vectors.

Usage

summarise_num(x)

Arguments

x

a numeric vector.

Value

A numeric vector with seven named elements: mean, sd, median, p25, p75, p0, and p100 corresponding to the mean, standard deviation, median, and 25-/75-/0-/100-percentiles.


Summary of simulated trial results

Description

Summarises simulation results from the run_trials() function. Uses extract_results() and check_performance(), which may be used directly to extract key trial results without summarising or to calculate performance metrics (with uncertainty measures if desired) and return them in a tidy data.frame.

Usage

## S3 method for class 'trial_results'
summary(
  object,
  select_strategy = "control if available",
  select_last_arm = FALSE,
  select_preferences = NULL,
  te_comp = NULL,
  raw_ests = FALSE,
  final_ests = NULL,
  restrict = NULL,
  cores = NULL,
  ...
)

Arguments

object

trial_results object, output from the run_trials() function.

select_strategy

single character string. If a trial was not stopped due to superiority (or had only 1 arm remaining, if select_last_arm is set to TRUE in trial designs with a common control arm; see below), this parameter specifies which arm will be considered selected when calculating trial design performance metrics, as described below; this corresponds to the consequence of an inconclusive trial, i.e., which arm would then be used in practice.
The following options are available and must be written exactly as below (case sensitive, cannot be abbreviated):

  • "control if available" (default): selects the first control arm for trials with a common control arm if this arm is active at end-of-trial, otherwise no arm will be selected. For trial designs without a common control, no arm will be selected.

  • "none": selects no arm in trials not ending with superiority.

  • "control": similar to "control if available", but will throw an error if used for trial designs without a common control arm.

  • "final control": selects the final control arm regardless of whether the trial was stopped for practical equivalence, futility, or at the maximum sample size; this strategy can only be specified for trial designs with a common control arm.

  • "control or best": selects the first control arm if still active at end-of-trial, otherwise selects the best remaining arm (defined as the remaining arm with the highest probability of being the best in the last adaptive analysis conducted). Only works for trial designs with a common control arm.

  • "best": selects the best remaining arm (as described under "control or best").

  • "list or best": selects the first remaining arm from a specified list (specified using select_preferences, technically a character vector). If none of these arms are are active at end-of-trial, the best remaining arm will be selected (as described above).

  • "list": as specified above, but if no arms on the provided list remain active at end-of-trial, no arm is selected.

select_last_arm

single logical, defaults to FALSE. If TRUE, the only remaining active arm (the last control) will be selected in trials with a common control arm ending with equivalence or futility, before considering the options specified in select_strategy. Must be FALSE for trial designs without a common control arm.

select_preferences

character vector specifying a number of arms used for selection if one of the "list or best" or "list" options are specified for select_strategy. Can only contain valid arms available in the trial.

te_comp

character string, treatment-effect comparator. Can be either NULL (the default) in which case the first control arm is used for trial designs with a common control arm, or a string naming a single trial arm. Will be used when calculating err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described below).

raw_ests

single logical. If FALSE (default), the posterior estimates (post_ests or post_ests_all, see setup_trial() and run_trial()) will be used to calculate err and sq_err (the error and the squared error of the estimated compared to the specified effect in the selected arm) and err_te and sq_err_te (the error and the squared error of the treatment effect comparing the selected arm to the comparator arm, as described for te_comp and below). If TRUE, the raw estimates (raw_ests or raw_ests_all, see setup_trial() and run_trial()) will be used instead of the posterior estimates.

final_ests

single logical. If TRUE (recommended) the final estimates calculated using outcome data from all patients randomised when trials are stopped are used (post_ests_all or raw_ests_all, see setup_trial() and run_trial()); if FALSE, the estimates calculated for each arm when an arm is stopped (or at the last adaptive analysis if not before) using data from patients having reach followed up at this time point and not all patients randomised are used (post_ests or raw_ests, see setup_trial() and run_trial()). If NULL (the default), this argument will be set to FALSE if outcome data are available immediate after randomisation for all patients (for backwards compatibility, as final posterior estimates may vary slightly in this situation, even if using the same data); otherwise it will be said to TRUE. See setup_trial() for more details on how these estimates are calculated.

restrict

single character string or NULL. If NULL (default), results are summarised for all simulations; if "superior", results are summarised for simulations ending with superiority only; if "selected", results are summarised for simulations ending with a selected arm only (according to the specified arm selection strategy for simulations not ending with superiority). Some summary measures (e.g., prob_conclusive) have substantially different interpretations if restricted, but are calculated nonetheless.

cores

NULL or single integer. If NULL, a default value set by setup_cluster() will be used to control whether extractions of simulation results are done in parallel on a default cluster or sequentially in the main process; if a value has not been specified by setup_cluster(), cores will then be set to the value stored in the global "mc.cores" option (if previously set by ⁠options(mc.cores = <number of cores>⁠), and 1 if that option has not been specified.
If cores = 1, computations will be run sequentially in the primary process, and if cores > 1, a new parallel cluster will be setup using the parallel library and removed once the function completes. See setup_cluster() for details.

...

additional arguments, not used.

Value

A "trial_results_summary" object containing the following values:

See Also

extract_results(), check_performance(), plot_convergence(), plot_metrics_ecdf(), check_remaining_arms().

Examples

# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
                                 control = "A",
                                 true_ys = c(0.20, 0.18, 0.22, 0.24),
                                 data_looks = 1:20 * 100)

# Run 10 simulations with a specified random base seed
res <- run_trials(binom_trial, n_rep = 10, base_seed = 12345)

# Summarise simulations - select the control arm if available in trials not
# ending with a superiority decision
res_sum <- summary(res, select_strategy = "control")

# Print summary
print(res_sum, digits = 1)


Update previously saved calibration result

Description

This function updates a previously saved "trial_calibration"-object created and saved by calibrate_trial() using a previous version of adaptr, including the embedded trial specification and trial results objects (internally using the update_saved_trials() function). This allows the use of calibration results, including the calibrated trial specification and the best simulations results from the calibration process, to be used without errors by this version of the package. The function should be run only once per saved simulation object and will issue a warning if the object is already up to date. And overview of the changes made according to the adaptr package version used to generate the original object is provided in Details.

Usage

update_saved_calibration(path, version = NULL, compress = TRUE)

Arguments

path

single character; the path to the saved "trial_calibration"-object containing the calibration result saved by calibrate_trial().

version

passed to saveRDS() when saving the updated object, defaults to NULL (as in saveRDS()), which means that the current default version is used.

compress

passed to saveRDS() when saving the updated object, defaults to TRUE (as in saveRDS()), see saveRDS() for other options.

Details

The following changes are made according to the version of adaptr used to generate the original "trial_calibration" object:

Value

Invisibly returns the updated "trial_calibration"-object.

See Also

run_trials().


Update previously saved simulation results

Description

This function updates a previously saved "trial_results" object created and saved by run_trials() using a previous version of adaptr, allowing the results from these previous simulations to be post-processed (including performance metric calculation, printing and plotting) without errors by this version of the package. The function should be run only once per saved simulation object and will issue a warning if the object is already up to date. And overview of the changes made according to the adaptr package version used to generate the original object is provided in Details.
NOTE: some values cannot be updated and will be set to NA (the posterior estimates from the 'final' analysis conducted after the last adaptive analysis and including outcome data for all patients), and thus using both raw_ests = TRUE and final_ests = TRUE in the extract_results() and summary() functions will lead to missing values for some of the values calculated for updated simulation objects.
NOTE: other objects created by the adaptr package, i.e., trial specifications generated by setup_trial() / setup_trial_binom() / setup_trial_norm() and single simulation results from run_trials() when not included in as part of the returned output from run_trials() should be re-created by re-running the relevant code using the updated version of adaptr; if manually re-loaded from previous sessions, they may cause errors and problems with the updated version of the package.

Usage

update_saved_trials(path, version = NULL, compress = TRUE)

Arguments

path

single character; the path to the saved "trial_results"-object containing the simulations saved by run_trials().

version

passed to saveRDS() when saving the updated object, defaults to NULL (as in saveRDS()), which means that the current default version is used.

compress

passed to saveRDS() when saving the updated object, defaults to TRUE (as in saveRDS()), see saveRDS() for other options.

Details

The following changes are made according to the version of adaptr used to generate the original "trial_results" object:

Value

Invisibly returns the updated "trial_results"-object.

See Also

run_trials().


Validate trial specification

Description

Used internally. Validates the inputs common to all trial specifications, as specified in setup_trial(), setup_trial_binom() and setup_trial_norm().

Usage

validate_trial(
  arms,
  true_ys,
  start_probs = NULL,
  fixed_probs = NULL,
  min_probs = rep(NA, length(arms)),
  max_probs = rep(NA, length(arms)),
  rescale_probs = NULL,
  data_looks = NULL,
  max_n = NULL,
  look_after_every = NULL,
  randomised_at_looks = NULL,
  control = NULL,
  control_prob_fixed = NULL,
  inferiority = 0.01,
  superiority = 0.99,
  equivalence_prob = NULL,
  equivalence_diff = NULL,
  equivalence_only_first = NULL,
  futility_prob = NULL,
  futility_diff = NULL,
  futility_only_first = NULL,
  highest_is_best = FALSE,
  soften_power = 1,
  cri_width = 0.95,
  n_draws = 5000,
  robust = FALSE,
  description = NULL,
  add_info = NULL,
  fun_y_gen,
  fun_draws,
  fun_raw_est
)

Arguments

arms

character vector with unique names for the trial arms.

true_ys

numeric vector specifying true outcomes (e.g., event probabilities, mean values, etc.) for all trial arms.

start_probs

numeric vector, allocation probabilities for each arm at the beginning of the trial. The default (NULL) automatically generates equal randomisation probabilities for each arm.

fixed_probs

numeric vector, fixed allocation probabilities for each arm. Must be either a numeric vector with NA for arms without fixed probabilities and values between 0 and 1 for the other arms or NULL (default), if adaptive randomisation is used for all arms or if one of the special settings ("sqrt-based", "sqrt-based start", "sqrt-based fixed", or "match") is specified for control_prob_fixed (described below).

min_probs

numeric vector, lower threshold for adaptive allocation probabilities; lower probabilities will be rounded up to these values. Must be NA (default for all arms) if no lower threshold is wanted and for arms using fixed allocation probabilities.

max_probs

numeric vector, upper threshold for adaptive allocation probabilities; higher probabilities will be rounded down to these values. Must be NA (default for all arms) if no threshold is wanted and for arms using fixed allocation probabilities.

rescale_probs

NULL (default) or one of either "fixed", "limits", or "both". Rescales fixed_probs (if "fixed" or "both") and min_probs/max_probs (if "limits" or "both") after arm dropping in trial specifications with ⁠>2 arms⁠ using a rescale_factor defined as ⁠initial number of arms/number of active arms⁠. ⁠"fixed_probs⁠ and min_probs are rescaled as ⁠initial value * rescale factor⁠, except for fixed_probs controlled by the control_prob_fixed argument, which are never rescaled. max_probs are rescaled as ⁠1 - ( (1 - initial value) * rescale_factor)⁠.
Must be NULL if there are only ⁠2 arms⁠ or if control_prob_fixed is "sqrt-based fixed". If not NULL, one or more valid non-NA values must be specified for either min_probs/max_probs or fixed_probs (not counting a fixed value for the original control if control_prob_fixed is "sqrt-based"/"sqrt-based start"/"sqrt-based fixed").
Note: using this argument and specific combinations of values in the other arguments may lead to invalid combined (total) allocation probabilities after arm dropping, in which case all probabilities will ultimately be rescaled to sum to 1. It is the responsibility of the user to ensure that rescaling fixed allocation probabilities and minimum/maximum allocation probability limits will not lead to invalid or unexpected allocation probabilities after arm dropping.
Finally, any initial values that are overwritten by the control_prob_fixed argument after arm dropping will not be rescaled.

data_looks

vector of increasing integers, specifies when to conduct adaptive analyses (= the total number of patients with available outcome data at each adaptive analysis). The last number in the vector represents the final adaptive analysis, i.e., the final analysis where superiority, inferiority, practical equivalence, or futility can be claimed. Instead of specifying data_looks, the max_n and look_after_every arguments can be used in combination (in which case data_looks must be NULL, the default value).

max_n

single integer, number of patients with available outcome data at the last possible adaptive analysis (defaults to NULL). Must only be specified if data_looks is NULL. Requires specification of the look_after_every argument.

look_after_every

single integer, specified together with max_n. Adaptive analyses will be conducted after every look_after_every patients have available outcome data, and at the total sample size as specified by max_n (max_n does not need to be a multiple of look_after_every). If specified, data_looks must be NULL (default).

randomised_at_looks

vector of increasing integers or NULL, specifying the number of patients randomised at the time of each adaptive analysis, with new patients randomised using the current allocation probabilities at said analysis. If NULL (the default), the number of patients randomised at each analysis will match the number of patients with available outcome data at said analysis, as specified by data_looks or max_n and look_after_every, i.e., outcome data will be available immediately after randomisation for all patients.
If not NULL, the vector must be of the same length as the number of adaptive analyses specified by data_looks or max_n and look_after_every, and all values must be larger than or equal to the number of patients with available outcome data at each analysis.

control

single character string, name of one of the arms or NULL (default). If specified, this arm will serve as a common control arm, to which all other arms will be compared and the inferiority/superiority/equivalence thresholds (see below) will be for those comparisons. See setup_trial() Details for information on behaviour with respect to these comparisons.

control_prob_fixed

if a common control arm is specified, this can be set NULL (the default), in which case the control arm allocation probability will not be fixed if control arms change (the allocation probability for the first control arm may still be fixed using fixed_probs, but will not be 'reused' for the new control arm).
If not NULL, a vector of probabilities of either length 1 or ⁠number of arms - 1⁠ can be provided, or one of the special arguments "sqrt-based", "sqrt-based start", "sqrt-based fixed" or "match".
See setup_trial() Details for details on how this affects trial behaviour.

inferiority

single numeric value or vector of numeric values of the same length as the maximum number of possible adaptive analyses, specifying the probability threshold(s) for inferiority (default is 0.01). All values must be ⁠>= 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no values may be lower than the preceding value. If a common controlis not used, all values must be ⁠< 1 / number of arms⁠. An arm will be considered inferior and dropped if the probability that it is best (when comparing all arms) or better than the control arm (when a common control is used) drops below the inferiority threshold at an adaptive analysis.

superiority

single numeric value or vector of numeric values of the same length as the maximum number of possible adaptive analyses, specifying the probability threshold(s) for superiority (default is 0.99). All values must be ⁠>= 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no values may be higher than the preceding value. If the probability that an arm is best (when comparing all arms) or better than the control arm (when a common control is used) exceeds the superiority threshold at an adaptive analysis, said arm will be declared the winner and the trial will be stopped (if no common control is used or if the last comparator is dropped in a design with a common control) or become the new control and the trial will continue (if a common control is specified).

equivalence_prob

single numeric value, vector of numeric values of the same length as the maximum number of possible adaptive analyses or NULL (default, corresponding to no equivalence assessment), specifying the probability threshold(s) for equivalence. If not NULL, all values must be ⁠> 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no value may be higher than the preceding value. If not NULL, arms will be dropped for equivalence if the probability of either (a) equivalence compared to a common control or (b) equivalence between all arms remaining (designs without a common control) exceeds the equivalence threshold at an adaptive analysis. Requires specification of equivalence_diff and equivalence_only_first.

equivalence_diff

single numeric value (⁠> 0⁠) or NULL (default, corresponding to no equivalence assessment). If a numeric value is specified, estimated absolute differences smaller than this threshold will be considered equivalent. For designs with a common control arm, the differences between each non-control arm and the control arm is used, and for trials without a common control arm, the difference between the highest and lowest estimated outcome rates are used and the trial is only stopped for equivalence if all remaining arms are equivalent.

equivalence_only_first

single logical in trial specifications where equivalence_prob and equivalence_diff are specified and a common control arm is included, otherwise NULL (default). If a common control arm is used, this specifies whether equivalence will only be assessed for the first control (if TRUE) or also for subsequent control arms (if FALSE) if one arm is superior to the first control and becomes the new control.

futility_prob

single numeric value, vector of numeric values of the same length as the maximum number of possible adaptive analyses or NULL (default, corresponding to no futility assessment), specifying the probability threshold(s) for futility. All values must be ⁠> 0⁠ and ⁠<= 1⁠, and if multiple values are supplied, no value may be higher than the preceding value. If not NULL, arms will be dropped for futility if the probability for futility compared to the common control exceeds the futility threshold at an adaptive analysis. Requires a common control arm (otherwise this argument must be NULL), specification of futility_diff, and futility_only_first.

futility_diff

single numeric value (⁠> 0⁠) or NULL (default, corresponding to no futility assessment). If a numeric value is specified, estimated differences below this threshold in the beneficial direction (as specified in highest_is_best) will be considered futile when assessing futility in designs with a common control arm. If only 1 arm remains after dropping arms for futility, the trial will be stopped without declaring the last arm superior.

futility_only_first

single logical in trial specifications designs where futility_prob and futility_diff are specified, otherwise NULL (default and required in designs without a common control arm). Specifies whether futility will only be assessed against the first control (if TRUE) or also for subsequent control arms (if FALSE) if one arm is superior to the first control and becomes the new control.

highest_is_best

single logical, specifies whether larger estimates of the outcome are favourable or not; defaults to FALSE, corresponding to, e.g., an undesirable binary outcomes (e.g., mortality) or a continuous outcome where lower numbers are preferred (e.g., hospital length of stay).

soften_power

either a single numeric value or a numeric vector of exactly the same length as the maximum number of looks/adaptive analyses. Values must be between 0 and 1 (default); if ⁠< 1⁠, then re-allocated non-fixed allocation probabilities are all raised to this power (followed by rescaling to sum to 1) to make adaptive allocation probabilities less extreme, in turn used to redistribute remaining probability while respecting limits when defined by min_probs and/or max_probs. If 1, then no softening is applied.

cri_width

single numeric ⁠>= 0⁠ and ⁠< 1⁠, the width of the percentile-based credible intervals used when summarising individual trial results. Defaults to 0.95, corresponding to 95% credible intervals.

n_draws

single integer, the number of draws from the posterior distributions for each arm used when running the trial. Defaults to 5000; can be reduced for a speed gain (at the potential loss of stability of results if too low) or increased for increased precision (increasing simulation time). Values ⁠< 100⁠ are not allowed and values ⁠< 1000⁠ are not recommended and warned against.

robust

single logical, if TRUE (default) the medians and median absolute deviations (scaled to be comparable to the standard deviation for normal distributions; MAD_SDs, see stats::mad()) are used to summarise the posterior distributions; if FALSE, the means and standard deviations (SDs) are used instead (slightly faster, but may be less appropriate for posteriors skewed on the natural scale).

description

optional single character string describing the trial design, will only be used in print functions if not NULL (the default).

add_info

optional single string containing additional information regarding the trial design or specifications, will only be used in print functions if not NULL (the default).

fun_y_gen

function, generates outcomes. See setup_trial() Details for information on how to specify this function.
Note: this function is called once during setup to validate its output (with the global random seed restored afterwards).

fun_draws

function, generates posterior draws. See setup_trial() Details for information on how to specify this function.
Note: this function is called up to three times during setup to validate its output (with the global random seed restored afterwards).

fun_raw_est

function that takes a numeric vector and returns a single numeric value, used to calculate a raw summary estimate of the outcomes in each arm. Defaults to mean(), which is always used in the setup_trial_binom() and setup_trial_norm() functions.
Note: the function is called one time per arm during setup to validate the output structure.

Value

An object of class trial_spec containing the validated trial specification.


vapply helpers

Description

Used internally. Helpers for simplifying code invoking vapply().

Usage

vapply_num(X, FUN, ...)

vapply_int(X, FUN, ...)

vapply_str(X, FUN, ...)

vapply_lgl(X, FUN, ...)

Arguments

X

a vector (atomic or list) or an expression object. Other objects (including classed objects) will be coerced by base::as.list.

FUN

the function to be applied to each element of X: see ‘Details’. In the case of functions like +, %*%, the function name must be backquoted or quoted.

...

optional arguments to FUN.


Verify input is single integer (potentially within range)

Description

Used internally.

Usage

verify_int(x, min_value = -Inf, max_value = Inf, open = "no")

Arguments

x

value to check.

min_value, max_value

single integers (each), lower and upper bounds between which x should lie.

open

single character, determines whether min_value and max_value are excluded or not. Valid values: "no" (= closed interval; min_value and max_value included; default value), "right", "left", "yes" (= open interval, min_value and max_value excluded).

Value

Single logical.


Find the index of value nearest to a target value

Description

Used internally, to find the index of the value in a vector nearest to a target value, possibly in a specific preferred direction.

Usage

which_nearest(values, target, dir)

Arguments

values

numeric vector, the values considered.

target

single numeric value, the target to find the value closest to.

dir

single numeric value. If 0 (the default), finds the index of the value closest to the target, regardless of the direction. If ⁠< 0⁠ or ⁠> 0⁠, finds the index of the value closest to the target, but only considers values at or below/above target, respectfully, if any (otherwise returns the closest value regardless of direction).

Value

Single integer, the index of the value closest to target according to dir.