Title: | Adaptive Trial Simulator |
Version: | 1.4.0 |
Date: | 2024-05-03 |
Description: | Package that simulates adaptive (multi-arm, multi-stage) clinical trials using adaptive stopping, adaptive arm dropping, and/or adaptive randomisation. Developed as part of the INCEPT (Intensive Care Platform Trial) project (https://incept.dk/), primarily supported by a grant from Sygeforsikringen "danmark" (https://www.sygeforsikring.dk/). |
License: | GPL (≥ 3) |
Imports: | stats, parallel, utils |
Encoding: | UTF-8 |
Language: | en-GB |
NeedsCompilation: | no |
URL: | https://inceptdk.github.io/adaptr/, https://github.com/INCEPTdk/adaptr/, https://incept.dk/ |
BugReports: | https://github.com/INCEPTdk/adaptr/issues/ |
RoxygenNote: | 7.3.1 |
Suggests: | ggplot2, covr, rmarkdown, knitr, testthat, vdiffr |
VignetteBuilder: | knitr |
Config/testthat/edition: | 3 |
Packaged: | 2024-05-03 11:21:53 UTC; agra0037 |
Author: | Anders Granholm |
Maintainer: | Anders Granholm <andersgran@gmail.com> |
Repository: | CRAN |
Date/Publication: | 2024-05-03 12:10:02 UTC |
adaptr: Adaptive Trial Simulator
Description
Adaptive Trial Simulator
The adaptr
package simulates adaptive (multi-arm, multi-stage) randomised
clinical trials using adaptive stopping, adaptive arm dropping and/or
response-adaptive randomisation. The package is developed as part of the
INCEPT (Intensive Care Platform Trial) project,
funded primarily by a grant from
Sygeforsikringen "danmark".
Details
The adaptr
package contains the following primary functions (in order of
typical use):
The
setup_cluster()
initiates a parallel computation cluster that can be used to run simulations and post-processing in parallel, increasing speed. Details on parallelisation and other options for runningadaptr
functions in parallel are described in thesetup_cluster()
documentation.The
setup_trial()
function is the general function that sets up a trial specification. The simpler, special-case functionssetup_trial_binom()
andsetup_trial_norm()
may be used for easier specification of trial designs using binary, binomially distributed or continuous, normally distributed outcomes, respectively, with some limitations in flexibility.The
calibrate_trial()
function calibrates a trial specification to obtain a certain value for a performance metric (typically used to calibrate the Bayesian type 1 error rate in a scenario with no between-arm differences), using the functions below.The
run_trial()
andrun_trials()
functions are used to conduct single or multiple simulations, respectively, according to a trial specification setup as described in #2.The
extract_results()
,check_performance()
andsummary()
functions are used to extract results from multiple trial simulations, calculate performance metrics, and summarise results. Theplot_convergence()
function assesses stability of performance metrics according to the number of simulations conducted. Theplot_metrics_ecdf()
function plots empirical cumulative distribution functions for numerical performance metrics. Thecheck_remaining_arms()
function summarises all combinations of remaining arms across multiple trials simulations.The
plot_status()
andplot_history()
functions are used to plot the overall trial/arm statuses for multiple simulated trials or the history of trial metrics over time for single/multiple simulated trials, respectively.
For further information see the documentation of each function or the
Overview vignette (vignette("Overview", package = "adaptr")
) for an
example of how the functions work in combination.
For further examples and guidance on setting up trial specifications, see the
setup_trial()
documentation, the Basic examples vignette
(vignette("Basic-examples", package = "adaptr")
) and the
Advanced example vignette
(vignette("Advanced-example", package = "adaptr")
).
If using the package, please consider citing it using
citation(package = "adaptr")
.
Author(s)
Maintainer: Anders Granholm andersgran@gmail.com (ORCID)
Authors:
Benjamin Skov Kaas-Hansen epiben@hey.com (ORCID)
Other contributors:
Aksel Karl Georg Jensen akje@sund.ku.dk (ORCID) [contributor]
Theis Lange thlan@sund.ku.dk (ORCID) [contributor]
References
Granholm A, Jensen AKG, Lange T, Kaas-Hansen BS (2022). adaptr: an R package for simulating and comparing adaptive clinical trials. Journal of Open Source Software, 7(72), 4284. doi:10.21105/joss.04284
Granholm A, Kaas-Hansen BS, Lange T, Schjørring OL, Andersen LW, Perner A, Jensen AKG, Møller MH (2022). An overview of methodological considerations regarding adaptive stopping, arm dropping and randomisation in clinical trials. J Clin Epidemiol. doi:10.1016/j.jclinepi.2022.11.002
Examples of studies using adaptr
:
Granholm A, Lange T, Harhay MO, Jensen AKG, Perner A, Møller MH, Kaas-Hansen BS (2023). Effects of duration of follow-up and lag in data collection on the performance of adaptive clinical trials. Pharm Stat. doi:10.1002/pst.2342
Granholm A, Lange T, Harhay MO, Perner A, Møller MH, Kaas-Hansen BS (2024). Effects of sceptical priors on the performance of adaptive clinical trials with binary outcomes. Pharm Stat. doi:10.1002/pst.2387
See Also
setup_cluster()
, setup_trial()
, setup_trial_binom()
,
setup_trial_norm()
, calibrate_trial()
, run_trial()
, run_trials()
,
extract_results()
, check_performance()
, summary()
,
check_remaining_arms()
, plot_convergence()
, plot_metrics_ecdf()
,
print()
, plot_status()
, plot_history()
.
Check availability of required packages
Description
Used internally, helper function to check if SUGGESTED packages are available. Will halt execution if any of the queried packages are not available and provide installation instructions.
Usage
assert_pkgs(pkgs = NULL)
Arguments
pkgs |
character vector with name(s) of package(s) to check. |
Value
TRUE
if all packages available, otherwise execution is halted with
an error.
Calculate the ideal design percentage
Description
Used internally by check_performance()
, calculates the ideal design
percentage as described in that function's documentation.
Usage
calculate_idp(sels, arms, true_ys, highest_is_best)
Arguments
sels |
a character vector specifying the selected arms (according to the
selection strategies described in |
arms |
character vector with unique names for the trial arms. |
true_ys |
numeric vector specifying true outcomes (e.g., event
probabilities, mean values, etc.) for all trial |
highest_is_best |
single logical, specifies whether larger estimates of
the outcome are favourable or not; defaults to |
Value
A single numeric value between 0
and 100
corresponding to the
ideal design percentage.
Calibrate trial specification
Description
This function calibrates a trial specification using a Gaussian process-based
Bayesian optimisation algorithm.
The function calibrates an input trial specification object (using repeated
calls to run_trials()
while adjusting the trial specification) to a
target
value within a search_range
in a single input dimension (x
) in
order to find an optimal value (y
).
The default (and expectedly most common use case) is to calibrate a trial
specification to adjust the superiority
and inferiority
thresholds to
obtain a certain probability of superiority; if used with a trial
specification with identical underlying outcomes (no between-arm
differences), this probability is an estimate of the Bayesian analogue of the
total type-1 error rate for the outcome driving the adaptations, and if
between-arm differences are present, this corresponds to an estimate of the
Bayesian analogue of the power.
The default is to perform the calibration while varying single, constant,
symmetric thresholds for superiority
/ inferiority
throughout a trial
design, as described in Details, and the default values have been chosen
to function well in this case.
Advanced users may use the function to calibrate trial specifications
according to other metrics - see Details for how to specify a custom
function used to modify (or recreate) a trial specification object during
the calibration process.
The underlying Gaussian process model and its control hyperparameters are
described under Details, and the model is partially based on code from
Gramacy 2020 (with permission; see References).
Usage
calibrate_trial(
trial_spec,
n_rep = 1000,
cores = NULL,
base_seed = NULL,
fun = NULL,
target = 0.05,
search_range = c(0.9, 1),
tol = target/10,
dir = 0,
init_n = 2,
iter_max = 25,
resolution = 5000,
kappa = 0.5,
pow = 1.95,
lengthscale = 1,
scale_x = TRUE,
noisy = is.null(base_seed),
narrow = !noisy & !is.null(base_seed),
prev_x = NULL,
prev_y = NULL,
path = NULL,
overwrite = FALSE,
version = NULL,
compress = TRUE,
sparse = TRUE,
progress = NULL,
export = NULL,
export_envir = parent.frame(),
verbose = FALSE,
plot = FALSE
)
Arguments
trial_spec |
|
n_rep |
single integer, the number of simulations to run at each
evaluation. Values |
cores |
|
base_seed |
single integer or |
fun |
|
target |
single finite numeric value (defaults to |
search_range |
finite numeric vector of length |
tol |
single finite numeric value (defaults to |
dir |
single numeric value; specifies the direction(s) of the tolerance
range. If |
init_n |
single integer |
iter_max |
single integer |
resolution |
single integer (defaults to |
kappa |
single numeric value |
pow |
single numerical value in the |
lengthscale |
single numerical value (defaults to |
scale_x |
single logical value; if |
noisy |
single logical value; if |
narrow |
single logical value. If |
prev_x , prev_y |
numeric vectors of equal lengths, corresponding to
previous evaluations. If provided, these will be used in the calibration
process (added before the initial grid is setup, with values in the grid
matching values in |
path |
single character string or |
overwrite |
single logical, defaults to |
version |
passed to |
compress |
passed to |
sparse , progress , export , export_envir |
passed to |
verbose |
single logical, defaults to |
plot |
single logical, defaults to |
Details
Default calibration
If fun
is NULL
(as default), the default calibration strategy will be
employed. Here, the target y
is the probability of superiority (as
described in check_performance()
and summary()
), and the function will
calibrate constant stopping thresholds for superiority and inferiority (as
described in setup_trial()
, setup_trial_binom()
, and
setup_trial_norm()
), which corresponds to the Bayesian analogues of the
type 1 error rate if there are no differences between arms in the trial
specification, which we expect to be the most common use case, or the power,
if there are differences between arms in the trial specification.
The stopping calibration process will, in the default case, use the input x
as the stopping threshold for superiority and 1 - x
as the stopping
threshold for inferiority, respectively, i.e., stopping thresholds will be
constant and symmetric.
The underlying default function calibrated is typically essentially
noiseless if a high enough number of simulations are used with an
appropriate random base_seed
, and generally monotonically decreasing. The
default values for the control hyperparameters have been set to normally
work well in this case (including init_n
, kappa
, pow
, lengthscale
,
narrow
, scale_x
, etc.). Thus, few initial grid evaluations are used in
this case, and if a base_seed
is provided, a noiseless process is assumed
and narrowing of the search range with each iteration is performed, and the
uncertainty bounds used in the acquisition function (corresponding to
quantiles from the posterior predictive distribution) are relatively narrow.
Specifying calibration functions
A user-specified calibration function should have the following structure:
# The function must take the arguments x and trial_spec # trial_spec is the original trial_spec object which should be modified # (alternatively, it may be re-specified, but the argument should still # be included, even if ignored) function(x, trial_spec) { # Calibrate trial_spec, here as in the default function trial_spec$superiority <- x trial_spec$inferiority <- 1 - x # If relevant, known y values corresponding to specific x values may be # returned without running simulations (here done as in the default # function). In that case, a code block line the one below can be included, # with changed x/y values - of note, the other return values should not be # changed if (x == 1) { return(list(sims = NULL, trial_spec = trial_spec, y = 0)) } # Run simulations - this block should be included unchanged sims <- run_trials(trial_spec, n_rep = n_rep, cores = cores, base_seed = base_seed, sparse = sparse, progress = progress, export = export, export_envir = export_envir) # Return results - only the y value here should be changed # summary() or check_performance() will often be used here list(sims = sims, trial_spec = trial_spec, y = summary(sims)$prob_superior) }
Note: changes to the trial specification are not validated; users who
define their own calibration function need to ensure that changes to
calibrated trial specifications does not lead to invalid values; otherwise,
the procedure is prone to error when simulations are run. Especially, users
should be aware that changing true_ys
in a trial specification generated
using the simplified setup_trial_binom()
and setup_trial_norm()
functions
requires changes in multiple places in the object, including in the functions
used to generate random outcomes, and in these cases (and otherwise if in
doubt) re-generating the trial_spec
instead of modifying should be
preferred as this is safer and leads to proper validation.
Note: if the y
values corresponding to certain x
values are known,
then the user may directly return these values without running simulations
(e.g., in the default case an x
of 1
will require >100%
or <0%
probabilities for stopping rules, which is impossible, and hence the y
value in this case is by definition 1
).
Gaussian process optimisation function and control hyperparameters
The calibration function uses a relatively simple Gaussian optimisation
function with settings that should work well for the default calibration
function, but can be changed as required, which should be considered if
calibrating according to other targets (effects of using other settings may
be evaluated in greater detail by setting verbose
and plot
to TRUE
).
The function may perform both interpolation (i.e., assuming a noiseless,
deterministic process with no uncertainty at the values already evaluated) or
regression (i.e., assuming a noisy, stochastic process), controlled by the
noisy
argument.
The covariance matrix (or kernel) is defined as:
exp(-||x - x'||^pow / lengthscale)
with ||x -x'||
corresponding to a matrix containing the absolute Euclidean
distances of values of x
(and values on the prediction grid), scaled to
the [0, 1]
range if scale_x
is TRUE
and on their original scale if
FALSE
. Scaling i generally recommended (as this leads to more comparable
and predictable effects of pow
and lengthscale
, regardless of the true
scale), and also recommended if the range of values is smaller than this
range. The absolute distances are raised to the power pow
, which must be a
value in the [1, 2]
range. Together with lengthscale
, pow
controls the
smoothness of the Gaussian process model, with 1
corresponding to less
smoothing (i.e., piecewise straight lines between all evaluations if
lengthscale
is 1
) and values > 1
corresponding to more smoothing. After
raising the absolute distances to the chosen power pow
, the resulting
matrix is divided by lengthscale
. The default is 1
(no change), and
values < 1
leads to faster decay in correlations and thus less smoothing
(more wiggly fits), and values > 1
leads to more smoothing (less wiggly
fits). If a single specific value is supplied for lengthscale
this is used;
if a range of values is provided, a secondary optimisation process determines
the value to use within that range.
Some minimal noise ("jitter") is always added to the diagonals of the
matrices where relevant to ensure numerical stability; if noisy
is TRUE
,
a "nugget" value will be determined using a secondary optimisation process
Predictions will be made over an equally spaced grid of x
values of size
resolution
; if narrow
is TRUE
, this grid will only be spread out
between the x
values with corresponding y
values closest to and below and
closes to and above target
, respectively, leading to a finer grid in the
range of relevance (as described above, this should only be used for processes
that are assumed to be noiseless and should only be used if the process can
safely be assumed to be monotonically increasing or decreasing within the
search_range
). To suggest the next x
value for evaluations, the function
uses an acquisition function based on bi-directional uncertainty bounds
(posterior predictive distributions) with widths controlled by the kappa
hyperparameter. Higher kappa
/wider uncertainty bounds leads to increased
exploration (i.e., the algorithm is more prone to select values with high
uncertainty, relatively far from existing evaluations), while lower
kappa
/narrower uncertainty bounds leads to increased exploitation (i.e.,
the algorithm is more prone to select values with less uncertainty, closer to
the best predicted mean values). The value in the x
grid leading with one of
the boundaries having the smallest absolute distance to the target
is
chosen (within the narrowed range, if narrow
is TRUE
). See
Greenhill et al, 2020 under References for a general description of
acquisition functions.
IMPORTANT:
we recommend that control hyperparameters are explicitly specified, even
for the default calibration function. Although the default values should be
sensible for the default calibration function, these may change in the
future. Further, we generally recommend users to perform small-scale
comparisons (i.e., with fewer simulations than in the final calibration) of
the calibration process with different hyperparameters for specific use cases
beyond the default (possibly guided by setting the verbose
and plot
options to TRUE
) before running a substantial number of calibrations or
simulations, as the exact choices may have important influence on the speed
and likelihood of success of the calibration process.
It is the responsibility of the user to specify sensible values for the
settings and hyperparameters.
Value
A list of special class "trial_calibration"
, which contains the
following elements that can be extracted using $
or [[
:
-
success
: single logical,TRUE
if the calibration succeeded with the best result being within the tolerance range,FALSE
if the calibration process ended after all allowed iterations without obtaining a result within the tolerance range. -
best_x
: single numerical value, thex
-value (on the original, input scale) at which the besty
-value was found, regardless ofsuccess
. -
best_y
: single numerical value, the besty
-value obtained, regardless ofsuccess
. -
best_trial_spec
: the best calibrated version of the originaltrial_spec
object supplied, regardless ofsuccess
(i.e., the returned trial specification object is only adequately calibrated ifsuccess
isTRUE
). -
best_sims
: the trial simulation results (fromrun_trials()
) leading to the besty
-value, regardless ofsuccess
. If no new simulations have been conducted (e.g., if the besty
-value is from one of theprev_y
-values), this will beNULL
. -
evaluations
: a two-columndata.frame
containing the variablesx
andy
, corresponding to allx
-values andy
-values (including values supplied throughprev_x
/prev_y
). -
input_trial_spec
: the unaltered, uncalibrated, originaltrial_spec
-object provided to the function. -
elapsed_time
: the total run time of the calibration process. -
control
: list of the most central settings provided to the function. -
fun
: the function used for calibration; ifNULL
was supplied when starting the calibration, the default function (described in Details) is returned after being used in the function. -
adaptr_version
: the version of theadaptr
package used to run the calibration process. -
plots
: list containingggplot2
plot objects of each Gaussian process suggestion step, only included ifplot
isTRUE
.
References
Gramacy RB (2020). Chapter 5: Gaussian Process Regression. In: Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences. Chapman Hall/CRC, Boca Raton, Florida, USA. Available online.
Greenhill S, Rana S, Gupta S, Vellanki P, Venkatesh S (2020). Bayesian Optimization for Adaptive Experimental Design: A Review. IEEE Access, 8, 13937-13948. doi:10.1109/ACCESS.2020.2966228
Examples
## Not run:
# Setup a trial specification to calibrate
# This trial specification has similar event rates in all arms
# and as the default calibration settings are used, this corresponds to
# assessing the Bayesian type 1 error rate for this design and scenario
binom_trial <- setup_trial_binom(arms = c("A", "B"),
true_ys = c(0.25, 0.25),
data_looks = 1:5 * 200)
# Run calibration using default settings for most parameters
res <- calibrate_trial(binom_trial, n_rep = 1000, base_seed = 23)
# Print calibration summary result
res
## End(Not run)
cat() with sep = ""
Description
Used internally. Passes everything on to cat()
but enforces sep = ""
.
Relates to cat()
as paste0()
relates to paste()
.
Usage
cat0(...)
Arguments
... |
strings to be concatenated and printed. |
Check performance metrics for trial simulations
Description
Calculates performance metrics for a trial specification based on
simulation results from the run_trials()
function, with bootstrapped
uncertainty measures if requested. Uses extract_results()
, which may be
used directly to extract key trial results without summarising. This function
is also used by summary()
to calculate the performance metrics presented by
that function.
Usage
check_performance(
object,
select_strategy = "control if available",
select_last_arm = FALSE,
select_preferences = NULL,
te_comp = NULL,
raw_ests = FALSE,
final_ests = NULL,
restrict = NULL,
uncertainty = FALSE,
n_boot = 5000,
ci_width = 0.95,
boot_seed = NULL,
cores = NULL
)
Arguments
object |
|
select_strategy |
single character string. If a trial was not stopped
due to superiority (or had only 1 arm remaining, if
|
select_last_arm |
single logical, defaults to |
select_preferences |
character vector specifying a number of arms used
for selection if one of the |
te_comp |
character string, treatment-effect comparator. Can be either
|
raw_ests |
single logical. If |
final_ests |
single logical. If |
restrict |
single character string or |
uncertainty |
single logical; if |
n_boot |
single integer (default |
ci_width |
single numeric |
boot_seed |
single integer, |
cores |
|
Details
The ideal design percentage (IDP) returned is based on Viele et al, 2020 doi:10.1177/1740774519877836 (and also described in Granholm et al, 2022 doi:10.1016/j.jclinepi.2022.11.002, which also describes the other performance measures) and has been adapted to work for trials with both desirable/undesirable outcomes and non-binary outcomes. Briefly, the expected outcome is calculated as the sum of the true outcomes in each arm multiplied by the corresponding selection probabilities (ignoring simulations with no selected arm). The IDP is then calculated as:
For desirable outcomes (
highest_is_best
isTRUE
):
100 * (expected outcome - lowest true outcome) / (highest true outcome - lowest true outcome)
For undesirable outcomes (
highest_is_best
isFALSE
):
100 - IDP calculated for desirable outcomes
Value
A tidy data.frame
with added class trial_performance
(to control
the number of digits printed, see print()
), with the columns
"metric"
(described below), "est"
(estimate of each metric), and the
following four columns if uncertainty = TRUE
: "err_sd"
(bootstrapped
SDs), "err_mad"
(bootstrapped MAD-SDs, as described in setup_trial()
and stats::mad()
), "lo_ci"
, and "hi_ci"
, the latter two corresponding
to the lower/upper limits of the percentile-based bootstrapped confidence
intervals. Bootstrap estimates are not calculated for the minimum
(_p0
) and maximum values (_p100
) of size
, sum_ys
, and ratio_ys
,
as non-parametric bootstrapping for minimum/maximum values is not
sensible - bootstrap estimates for these values will be NA
.
The following performance metrics are calculated:
-
n_summarised
: the number of simulations summarised. -
size_mean
,size_sd
,size_median
,size_p25
,size_p75
,size_p0
,size_p100
: the mean, standard deviation, median as well as 25-, 75-, 0- (min), and 100- (max) percentiles of the sample sizes (number of patients randomised in each simulated trial) of the summarised trial simulations. -
sum_ys_mean
,sum_ys_sd
,sum_ys_median
,sum_ys_p25
,sum_ys_p75
,sum_ys_p0
,sum_ys_p100
: the mean, standard deviation, median as well as 25-, 75-, 0- (min), and 100- (max) percentiles of the totalsum_ys
across all arms in the summarised trial simulations (e.g., the total number of events in trials with a binary outcome, or the sums of continuous values for all patients across all arms in trials with a continuous outcome). Always uses all outcomes from all randomised patients regardless of whether or not all patients had outcome data available at the time of trial stopping (corresponding tosum_ys_all
in results fromrun_trial()
). -
ratio_ys_mean
,ratio_ys_sd
,ratio_ys_median
,ratio_ys_p25
,ratio_ys_p75
,ratio_ys_p0
,ratio_ys_p100
: the mean, standard deviation, median as well as 25-, 75-, 0- (min), and 100- (max) percentiles of the finalratio_ys
(sum_ys
as described above divided by the total number of patients randomised) across all arms in the summarised trial simulations. -
prob_conclusive
: the proportion (0
to1
) of conclusive trial simulations, i.e., simulations not stopped at the maximum sample size without a superiority, equivalence or futility decision. -
prob_superior
,prob_equivalence
,prob_futility
,prob_max
: the proportion (0
to1
) of trial simulations stopped for superiority, equivalence, futility or inconclusive at the maximum allowed sample size, respectively.
Note: Some metrics may not make sense if summarised simulation results arerestricted
. -
prob_select_*
: the selection probabilities for each arm and for no selection, according to the specified selection strategy. Contains one element perarm
, namedprob_select_arm_<arm name>
andprob_select_none
for the probability of selecting no arm. -
rmse
,rmse_te
: the root mean squared errors of the estimates for the selected arm and for the treatment effect, as described inextract_results()
. -
mae
,mae_te
: the median absolute errors of the estimates for the selected arm and for the treatment effect, as described inextract_results()
. -
idp
: the ideal design percentage (IDP; 0-100%), see Details.
See Also
extract_results()
, summary()
, plot_convergence()
,
plot_metrics_ecdf()
, check_remaining_arms()
.
Examples
# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
control = "A",
true_ys = c(0.20, 0.18, 0.22, 0.24),
data_looks = 1:20 * 100)
# Run 10 simulations with a specified random base seed
res <- run_trials(binom_trial, n_rep = 10, base_seed = 12345)
# Check performance measures, without assuming that any arm is selected in
# the inconclusive simulations, with bootstrapped uncertainty measures
# (unstable in this example due to the very low number of simulations
# summarised):
check_performance(res, select_strategy = "none", uncertainty = TRUE,
n_boot = 1000, boot_seed = "base")
Check remaining arm combinations
Description
This function summarises the numbers and proportions of all combinations of
remaining arms (i.e., excluding arms dropped for inferiority or futility at
any analysis, and arms dropped for equivalence at earlier analyses in trials
with a common control
) across multiple simulated trial results. The
function supplements the extract_results()
, check_performance()
, and
summary()
functions, and is especially useful for designs with > 2
arms,
where it provides details that the other functions mentioned do not.
Usage
check_remaining_arms(object, ci_width = 0.95)
Arguments
object |
|
ci_width |
single numeric |
Value
a data.frame
containing the combinations of remaining arms, sorted
in descending order of, with the following columns:
-
arm_*
, one column per arm, each named asarm_<arm name>
. These columns will contain an empty character string""
for dropped arms (including arms dropped at the final analysis), and otherwise be"superior"
,"control"
,"equivalence"
(only if equivalent at the final analysis), or"active"
, as described inrun_trial()
. -
n
integer vector, number of trial simulations ending with the combination of remaining arms as specified by the preceding columns. -
prop
numeric vector, the proportion of trial simulations ending with the combination of remaining arms as specified by the preceding columns. -
se
,lo_ci
,hi_ci
: the standard error ofprop
and the confidence intervals of the width specified byci_width
.
See Also
extract_results()
, check_performance()
, summary()
,
plot_convergence()
, plot_metrics_ecdf()
.
Examples
# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
control = "A",
true_ys = c(0.20, 0.18, 0.22, 0.24),
data_looks = 1:20 * 200,
equivalence_prob = 0.7,
equivalence_diff = 0.03,
equivalence_only_first = FALSE)
# Run 35 simulations with a specified random base seed
res <- run_trials(binom_trial, n_rep = 25, base_seed = 12345)
# Check remaining arms (printed with fewer digits)
print(check_remaining_arms(res), digits = 3)
Estimates covariance matrices used by Gaussian process optimisation
Description
Used internally, estimates covariance matrices used by the Gaussian process
optimisation function. Calculates pairwise absolute distances raised to a
power (which defaults to 2
) using the pow_abs_dist()
function, divides
the result by a lengthscale
hyperparameter (which defaults to 1
, i.e., no
changes due to division), and subsequently returns the inverse exponentiation
of the resulting matrix.
Usage
cov_mat(x1, x2 = x1, g = NULL, pow = 2, lengthscale = 1)
Arguments
x1 |
numeric vector, with length corresponding to the number of rows in the returned matrix. |
x2 |
numeric vector, with length corresponding to the number of columns
in the returned matrix. If not specified, |
g |
single numerical value; jitter/nugget value added to the diagonal
if not |
pow |
single numeric value, the power that all distances are raised to.
Defaults to |
lengthscale |
single numerical value; lengthscale hyperparameter that
the matrix returned from |
Value
Covariance matrix with length(x1)
rows and length(x2)
columns
used by the Gaussian process optimiser.
Simulate single trial after setting seed
Description
Helper function to dispatch the running of several trials to lapply()
or
parallel::parLapply()
, setting seeds correctly if a base_seed
was used
when calling run_trials()
. Used internally in calls by the run_trials()
function.
Usage
dispatch_trial_runs(is, trial_spec, seeds, sparse, cores, cl = NULL)
Arguments
is |
vector of integers, the simulation numbers/indices. |
trial_spec |
trial specification as provided by |
sparse |
single logical, as described in |
cores |
|
cl |
|
Value
Single trial simulation object, as described in run_trial()
.
Assert equivalent functions
Description
Used internally. Compares the definitions of two functions (ignoring
environments, bytecodes, etc., by only comparing function arguments and
bodies, using deparse()
).
Usage
equivalent_funs(fun1, fun2)
Arguments
fun1 , fun2 |
functions to compare. |
Value
Single logical.
Extract history
Description
Used internally. Extracts relevant parameters at each conducted adaptive analysis from a single trial.
Usage
extract_history(object, metric = "prob")
Arguments
object |
single |
metric |
either |
Value
A tidy data.frame
(one row per arm per look) containing the following
columns:
-
look
: consecutive numbers (integers) of each interim look. -
look_ns
: total number of patients (integers) with outcome data available at current adaptive analysis look to all arms in the trial. -
look_ns_all
: total number of patients (integers) randomised at current adaptive analysis look to all arms in the trial. -
arm
: the currentarm
in the trial. -
value
: as described undermetric
.
Extract simulation results
Description
This function extracts relevant information from multiple simulations of the
same trial specification in a tidy data.frame
(1 simulation per row).
See also the check_performance()
and summary()
functions, that uses the
output from this function to further summarise simulation results.
Usage
extract_results(
object,
select_strategy = "control if available",
select_last_arm = FALSE,
select_preferences = NULL,
te_comp = NULL,
raw_ests = FALSE,
final_ests = NULL,
cores = NULL
)
Arguments
object |
|
select_strategy |
single character string. If a trial was not stopped
due to superiority (or had only 1 arm remaining, if
|
select_last_arm |
single logical, defaults to |
select_preferences |
character vector specifying a number of arms used
for selection if one of the |
te_comp |
character string, treatment-effect comparator. Can be either
|
raw_ests |
single logical. If |
final_ests |
single logical. If |
cores |
|
Value
A data.frame
containing the following columns:
-
sim
: the simulation number (from1
to the total number of simulations). -
final_n
: the final sample size in each simulation. -
sum_ys
: the sum of the total counts in all arms, e.g., the total number of events in trials with a binary outcome (setup_trial_binom()
) or the sum of the arm totals in trials with a continuous outcome (setup_trial_norm()
). Always uses all outcome data from all randomised patients regardless of whether or not all patients had outcome data available at the time of trial stopping (corresponding tosum_ys_all
in results fromrun_trial()
). -
ratio_ys
: calculated assum_ys/final_n
(as described above). -
final_status
: the final trial status for each simulation, either"superiority"
,"equivalence"
,"futility"
, or"max"
, as described inrun_trial()
. -
superior_arm
: the final superior arm in simulations stopped for superiority. Will beNA
in simulations not stopped for superiority. -
selected_arm
: the final selected arm (as described above). Will correspond to thesuperior_arm
in simulations stopped for superiority and beNA
if no arm is selected. Seeselect_strategy
above. -
err
: the squared error of the estimate in the selected arm, calculated asestimated effect - true effect
for the selected arm. -
sq_err:
the squared error of the estimate in the selected arm, calculated aserr^2
for the selected arm, witherr
defined above. -
err_te
: the error of the treatment effect comparing the selected arm to the comparator arm (as specified inte_comp
). Calculated as:
(estimated effect in the selected arm - estimated effect in the comparator arm) -
(true effect in the selected arm - true effect in the comparator arm)
Will beNA
for simulations without a selected arm, with no comparator specified (seete_comp
above), and when the selected arm is the comparator arm. -
sq_err_te
: the squared error of the treatment effect comparing the selected arm to the comparator arm (as specified inte_comp
), calculated aserr_te^2
, witherr_te
defined above.
See Also
check_performance()
, summary()
, plot_convergence()
,
plot_metrics_ecdf()
, check_remaining_arms()
.
Examples
# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
control = "A",
true_ys = c(0.20, 0.18, 0.22, 0.24),
data_looks = 1:20 * 100)
# Run 10 simulations with a specified random base seed
res <- run_trials(binom_trial, n_rep = 10, base_seed = 12345)
# Extract results and Select the control arm if available
# in simulations not ending with superiority
extract_results(res, select_strategy = "control")
Extract results from a batch of trials from an object with multiple trials
Description
Used internally by extract_results()
. Extracts results from a batch of
simulations from a simulation object with multiple simulation results
returned by run_trials()
, used to facilitate parallelisation.
Usage
extract_results_batch(
trial_results,
control = control,
select_strategy = select_strategy,
select_last_arm = select_last_arm,
select_preferences = select_preferences,
te_comp = te_comp,
which_ests = which_ests,
te_comp_index = te_comp_index,
te_comp_true_y = te_comp_true_y
)
Arguments
trial_results |
list of trial results to summarise, the current batch. |
control |
single character string, the common |
select_strategy |
single character string. If a trial was not stopped
due to superiority (or had only 1 arm remaining, if
|
select_last_arm |
single logical, defaults to |
select_preferences |
character vector specifying a number of arms used
for selection if one of the |
te_comp |
character string, treatment-effect comparator. Can be either
|
which_ests |
single character string, a combination of the |
te_comp_index |
single integer, index of the treatment effect comparator
arm ( |
te_comp_true_y |
single numeric value, true |
Value
A data.frame
containing all columns returned by extract_results()
and described in that function (sim
will start from 1
, but this is
changed where relevant by extract_results()
).
Extract statuses
Description
Used internally. Extracts overall trial statuses or statuses from a single
arm
from multiple trial simulations. Works with sparse results.
Usage
extract_statuses(object, x_value, arm = NULL)
Arguments
object |
|
x_value |
single character string, determining whether the number of
adaptive analysis looks ( |
arm |
character vector containing one or more unique, valid |
Value
A tidy data.frame
(one row possible status per look) containing the
following columns:
-
x
: the look numbers or total number of patients at each look, as specified inx_value
. -
status
: each possible status ("Recruiting"
,"Inferiority"
(only relevant for individual arms),"Futility"
,"Equivalence"
, and"Superiority"
, as relevant). -
p
: the proportion (0-1
) of patients with eachstatus
at each value ofx
. -
value
: as described undermetric
.
Find beta distribution parameters from thresholds
Description
Helper function to find a beta distribution with parameters corresponding
to the fewest possible patients with events/non-events and a specified event
proportion. Used in the Advanced example vignette
(vignette("Advanced-example", "adaptr")
) to derive beta
prior
distributions for use in beta-binomial conjugate models, based on a belief
that the true event probability lies within a specified percentile-based
interval (defaults to 95%
). May similarly be used by users to derive other
beta
priors.
Usage
find_beta_params(
theta = NULL,
boundary_target = NULL,
boundary = "lower",
interval_width = 0.95,
n_dec = 0,
max_n = 10000
)
Arguments
theta |
single numeric |
boundary_target |
single numeric |
boundary |
single character string, either |
interval_width |
width of the credible interval whose lower/upper
boundary should be used (see |
n_dec |
single non-negative integer; the returned parameters are rounded
to this number of decimals. Defaults to |
max_n |
single integer |
Value
A single-row data.frame
with five columns: the two shape parameters
of the beta distribution (alpha
, beta
), rounded according to n_dec
,
and the actual lower and upper boundaries of the interval and the median
(with appropriate names, e.g. p2.5
, p50
, and p97.5
for a
95%
interval), when using those rounded values.
Format digits before printing
Description
Used internally.
Usage
fmt_dig(x, dig)
Arguments
x |
numeric, the numeric value(s) to format. |
dig |
single integer, the number of digits. |
Value
Formatted character string.
Create formatted label with absolute and relative frequencies (percentages)
Description
Used internally.
Usage
fmt_pct(e, n, dec = 1)
Arguments
e |
integer, the numerator (e.g., the number of events). |
n |
integer, the denominator (e.g., the total number of patients). |
dec |
integer, the number of decimals for the percentage. |
Value
Formatted character string.
Generate draws from posterior beta-binomial distributions
Description
Used internally. This function generates draws from posterior distributions
using separate beta-binomial models (binomial outcome, conjugate beta prior)
for each arm, with flat (beta(1, 1)
) priors.
Usage
get_draws_binom(arms, allocs, ys, control, n_draws)
Arguments
arms |
character vector, currently active |
allocs |
character vector, allocations of all patients (including
allocations to currently inactive |
ys |
numeric vector, outcomes of all patients in the same order
as |
control |
unused argument in the built-in functions for
|
n_draws |
single integer, number of posterior draws. |
Value
A matrix
(with numeric values) with length(arms)
columns and
n_draws
rows, with arms
as column names.
Generic documentation for get_draws_* functions
Description
Used internally. See the setup_trial()
function documentation for
additional details on how to specify functions to generate posterior draws.
Arguments
arms |
character vector, currently active |
allocs |
character vector, allocations of all patients (including
allocations to currently inactive |
ys |
numeric vector, outcomes of all patients in the same order
as |
control |
unused argument in the built-in functions for
|
n_draws |
single integer, number of posterior draws. |
Value
A matrix
(with numeric values) with length(arms)
columns and
n_draws
rows, with arms
as column names.
Generate draws from posterior normal distributions
Description
Used internally. This function generates draws from posterior, normal
distributions for continuous outcomes. Technically, these posteriors use no
priors (for simulation speed), corresponding to the use of improper flat
priors. These posteriors correspond (and give similar results) to using
normal-normal models (normally distributed outcome, conjugate normal prior)
for each arm, assuming that a non-informative, flat prior is used. Thus, the
posteriors directly correspond to normal distributions with each groups' mean
as the mean and each groups' standard error as the standard deviation.
As it is necessary to always return valid draws, in cases where < 2
patients have been randomised to an arm
, posterior draws will come from an
extremely wide normal distribution with mean corresponding to the mean of all
included patients with outcome data and a standard deviation corresponding to
the difference between the highest and lowest recorded outcomes for all
patients with available outcome data multiplied by 1000
.
Usage
get_draws_norm(arms, allocs, ys, control, n_draws)
Arguments
arms |
character vector, currently active |
allocs |
character vector, allocations of all patients (including
allocations to currently inactive |
ys |
numeric vector, outcomes of all patients in the same order
as |
control |
unused argument in the built-in functions for
|
n_draws |
single integer, number of posterior draws. |
Value
A matrix
(with numeric values) with length(arms)
columns and
n_draws
rows, with arms
as column names.
Generate binary outcomes from binomial distributions
Description
Used internally. Function factory used to generate a function that generates binary outcomes from binomial distributions.
Usage
get_ys_binom(arms, event_probs)
Arguments
arms |
character vector of |
event_probs |
numeric vector of true event probabilities in all |
Value
A function which takes the argument allocs
(a character vector
with the allocations) and returns a numeric vector of similar length with
the corresponding, randomly generated outcomes (0 or 1, from binomial
distribution).
Generate normally distributed continuous outcomes
Description
Used internally. Function factory used to generate a function that generates outcomes from normal distributions.
Usage
get_ys_norm(arms, means, sds)
Arguments
arms |
character vector, |
means |
numeric vector, true |
sds |
numeric vector, true standard deviations ( |
Value
A function which takes the argument allocs
(a character vector
with the allocations) and returns a numeric vector of the same length with
the corresponding, randomly generated outcomes (from normal distributions).
Gaussian process-based optimisation
Description
Used internally. Simple Gaussian process-based Bayesian optimisation
function, used to find the next value to evaluate (as x
) in the
calibrate_trial()
function. Uses only a single input dimension, which may
be rescaled to the [0, 1]
range by the function, and a covariance structure
based on absolute distances between values, raised to a power (pow
) and
subsequently divided by lengthscale
before the inverse exponentiation of
the resulting matrix is used. The pow
and lengthscale
hyperparameters
consequently control the smoothness by controlling the rate of decay between
correlations with distance.
The optimisation algorithm uses bi-directional uncertainty bounds in an
acquisition function that suggests the next target to evaluate, with wider
uncertainty bounds (higher kappa
) leading to increased 'exploration' (i.e.,
the function is more prone to suggest new target values where the uncertainty
is high and often further from the best evaluation so far) and narrower
uncertainty bounds leading to increased 'exploitation' (i.e., the function is
more prone to suggest new target values relatively close to the mean
predictions from the model).
The dir
argument controls whether the suggested value (based on both
uncertainty bounds) should be the value closest to target
in either
direction (dir = 0
), at or above target
(dir > 0
), or at or below
target (dir < 0
), if any, are preferred.
When the function being evaluated is noise-free and monotonically increasing
or decreasing, the optimisation function can narrow the range of predictions
based on the input evaluations (narrow = TRUE
), leading to a finer grid of
potential new targets to suggest compared to when predictions are spaced over
the full range.
If the new value at which to evaluate the function suggested has already been
evaluated, random noise will be added to ensure evaluation at a new value
(if narrow
is FALSE
, noise will be based on a random draw from a normal
distribution with the current suggested value as mean and the standard
deviation of the x
values as SD, truncated to the range of x
-values; if
narrow
is TRUE
, a new value drawn from a uniform distribution within the
current narrowed range will be suggested. For both strategies, the process
will be repeated until the suggested value is 'new').
The Gaussian process model used is partially based on code from Gramacy
2020 (with permission), see References.
Usage
gp_opt(
x,
y,
target,
dir = 0,
resolution = 5000,
kappa = 1.96,
pow = 1.95,
lengthscale = 1,
scale_x = TRUE,
noisy = FALSE,
narrow = FALSE
)
Arguments
x |
numeric vector, the previous values where the function being calibrated was evaluated. |
y |
numeric vector, the corresponding results of the previous
evaluations at the |
target |
single numeric value, the desired target value for the calibration process. |
dir |
single numeric value (default |
resolution |
single integer (default |
kappa |
single numeric value |
pow |
single numerical value, passed to |
lengthscale |
single numerical value (default |
scale_x |
single logical value; if |
noisy |
single logical value. If |
narrow |
single logical value. If |
Value
List containing two elements, next_x
, a single numerical value, the
suggested next x
value at which to evaluate the function, and
predictions
, a data.frame
with resolution
rows and the four columns:
x
, the x
grid values where predictions are made; y_hat
, the predicted
means, and lub
and uub
, the lower and upper uncertainty bounds of the
predictions according to kappa
.
References
Gramacy RB (2020). Chapter 5: Gaussian Process Regression. In: Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences. Chapman Hall/CRC, Boca Raton, Florida, USA. Available online.
Greenhill S, Rana S, Gupta S, Vellanki P, Venkatesh S (2020). Bayesian Optimization for Adaptive Experimental Design: A Review. IEEE Access, 8, 13937-13948. doi:10.1109/ACCESS.2020.2966228
Make x-axis scale for history/status plots
Description
Used internally. Prepares the x-axis scale for history/status plots.
Requires the ggplot2
package installed.
Usage
make_x_scale(x_value)
Arguments
x_value |
single character string, determining whether the number of
adaptive analysis looks ( |
Value
An appropriate scale for the ggplot2
plot x-axis according to
the value specified in x_value
.
Make y-axis scale for history/status plots
Description
Used internally. Prepares the y-axis scale for history/status plots.
Requires the ggplot2
package installed.
Usage
make_y_scale(y_value)
Arguments
y_value |
single character string, determining which values are plotted
on the y-axis. The following options are available: allocation
probabilities ( |
Value
An appropriate scale for the ggplot2
plot y-axis according to
the value specified in y_value
.
Plot convergence of performance metrics
Description
Plots performance metrics according to the number of simulations conducted
for multiple simulated trials. The simulated trial results may be split into
a number of batches to illustrate stability of performance metrics across
different simulations. Calculations are done according to specified selection
and restriction strategies as described in extract_results()
and
check_performance()
. Requires the ggplot2
package installed.
Usage
plot_convergence(
object,
metrics = "size mean",
resolution = 100,
select_strategy = "control if available",
select_last_arm = FALSE,
select_preferences = NULL,
te_comp = NULL,
raw_ests = FALSE,
final_ests = NULL,
restrict = NULL,
n_split = 1,
nrow = NULL,
ncol = NULL,
cores = NULL
)
Arguments
object |
|
metrics |
the performance metrics to plot, as described in
|
resolution |
single positive integer, the number of points calculated
and plotted, defaults to |
select_strategy |
single character string. If a trial was not stopped
due to superiority (or had only 1 arm remaining, if
|
select_last_arm |
single logical, defaults to |
select_preferences |
character vector specifying a number of arms used
for selection if one of the |
te_comp |
character string, treatment-effect comparator. Can be either
|
raw_ests |
single logical. If |
final_ests |
single logical. If |
restrict |
single character string or |
n_split |
single positive integer, the number of consecutive batches the
simulation results will be split into, which will be plotted separately.
Default is |
nrow , ncol |
the number of rows and columns when plotting multiple
metrics in the same plot (using faceting in |
cores |
|
Value
A ggplot2
plot object.
See Also
check_performance()
, summary()
, extract_results()
,
check_remaining_arms()
.
Examples
#### Only run examples if ggplot2 is installed ####
if (requireNamespace("ggplot2", quietly = TRUE)){
# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
control = "A",
true_ys = c(0.20, 0.18, 0.22, 0.24),
data_looks = 1:20 * 100)
# Run multiple simulation with a fixed random base seed
res_mult <- run_trials(binom_trial, n_rep = 25, base_seed = 678)
# NOTE: the number of simulations in this example is smaller than
# recommended - the plots reflect that, and show that performance metrics
# are not stable and have likely not converged yet
# Convergence plot of mean sample sizes
plot_convergence(res_mult, metrics = "size mean")
}
if (requireNamespace("ggplot2", quietly = TRUE)){
# Convergence plot of mean sample sizes and ideal design percentages,
# with simulations split in 2 batches
plot_convergence(res_mult, metrics = c("size mean", "idp"), n_split = 2)
}
Plot trial metric history
Description
Plots the history of relevant metrics over the progress of a single or
multiple trial simulations. Simulated trials only contribute until the
time they are stopped, i.e., if some trials are stopped earlier than others,
they will not contribute to the summary statistics at later adaptive looks.
Data from individual arms in a trial contribute until the complete trial is
stopped.
These history plots require non-sparse results (sparse
set to
FALSE
; see run_trial()
and run_trials()
) and the ggplot2
package
installed.
Usage
plot_history(object, x_value = "look", y_value = "prob", line = NULL, ...)
## S3 method for class 'trial_result'
plot_history(object, x_value = "look", y_value = "prob", line = NULL, ...)
## S3 method for class 'trial_results'
plot_history(
object,
x_value = "look",
y_value = "prob",
line = NULL,
ribbon = list(width = 0.5, alpha = 0.2),
cores = NULL,
...
)
Arguments
object |
|
x_value |
single character string, determining whether the number of
adaptive analysis looks ( |
y_value |
single character string, determining which values are plotted
on the y-axis. The following options are available: allocation
probabilities ( |
line |
list styling the lines as per |
... |
additional arguments, not used. |
ribbon |
list, as |
cores |
|
Value
A ggplot2
plot object.
See Also
Examples
#### Only run examples if ggplot2 is installed ####
if (requireNamespace("ggplot2", quietly = TRUE)){
# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
control = "A",
true_ys = c(0.20, 0.18, 0.22, 0.24),
data_looks = 1:20 * 100)
# Run a single simulation with a fixed random seed
res <- run_trial(binom_trial, seed = 12345)
# Plot total allocations to each arm according to overall total allocations
plot_history(res, x_value = "total n", y_value = "n")
}
if (requireNamespace("ggplot2", quietly = TRUE)){
# Run multiple simulation with a fixed random base seed
# Notice that sparse = FALSE is required
res_mult <- run_trials(binom_trial, n_rep = 15, base_seed = 12345, sparse = FALSE)
# Plot allocation probabilities at each look
plot_history(res_mult, x_value = "look", y_value = "prob")
# Other y_value options are available but not shown in these examples
}
Plot empirical cumulative distribution functions of performance metrics
Description
Plots empirical cumulative distribution functions (ECDFs) of numerical
performance metrics across multiple simulations from a "trial_results"
object returned by run_trials()
. Requires the ggplot2
package installed.
Usage
plot_metrics_ecdf(
object,
metrics = c("size", "sum_ys", "ratio_ys"),
select_strategy = "control if available",
select_last_arm = FALSE,
select_preferences = NULL,
te_comp = NULL,
raw_ests = FALSE,
final_ests = NULL,
restrict = NULL,
nrow = NULL,
ncol = NULL,
cores = NULL
)
Arguments
object |
|
metrics |
the performance metrics to plot, as described in
|
select_strategy |
single character string. If a trial was not stopped
due to superiority (or had only 1 arm remaining, if
|
select_last_arm |
single logical, defaults to |
select_preferences |
character vector specifying a number of arms used
for selection if one of the |
te_comp |
character string, treatment-effect comparator. Can be either
|
raw_ests |
single logical. If |
final_ests |
single logical. If |
restrict |
single character string or |
nrow , ncol |
the number of rows and columns when plotting multiple
metrics in the same plot (using faceting in |
cores |
|
Details
Note that the arguments related to arm selection and error calculation are only relevant if errors are visualised.
Value
A ggplot2
plot object.
See Also
check_performance()
, summary()
, extract_results()
,
plot_convergence()
, check_remaining_arms()
.
Examples
#### Only run examples if ggplot2 is installed ####
if (requireNamespace("ggplot2", quietly = TRUE)){
# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
control = "A",
true_ys = c(0.20, 0.18, 0.22, 0.24),
data_looks = 1:20 * 100)
# Run multiple simulation with a fixed random base seed
res_mult <- run_trials(binom_trial, n_rep = 25, base_seed = 678)
# NOTE: the number of simulations in this example is smaller than
# recommended - the plots reflect that, and would likely be smoother if
# a larger number of trials had been simulated
# Plot ECDFs of continuous performance metrics
plot_metrics_ecdf(res_mult)
}
Plot statuses
Description
Plots the statuses over time of multiple simulated trials (overall or for one
or more specific arms). Requires the ggplot2
package installed.
Usage
plot_status(
object,
x_value = "look",
arm = NULL,
area = list(alpha = 0.5),
nrow = NULL,
ncol = NULL
)
## S3 method for class 'trial_results'
plot_status(
object,
x_value = "look",
arm = NULL,
area = list(alpha = 0.5),
nrow = NULL,
ncol = NULL
)
Arguments
object |
|
x_value |
single character string, determining whether the number of
adaptive analysis looks ( |
arm |
character vector containing one or more unique, valid |
area |
list of styling settings for the area as per |
nrow , ncol |
the number of rows and columns when plotting statuses for
multiple arms in the same plot (using faceting in |
Value
A ggplot2
plot object.
See Also
Examples
#### Only run examples if ggplot2 is installed ####
if (requireNamespace("ggplot2", quietly = TRUE)){
# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
control = "A",
true_ys = c(0.20, 0.18, 0.22, 0.24),
data_looks = 1:20 * 100)
# Run multiple simulation with a fixed random base seed
res_mult <- run_trials(binom_trial, n_rep = 25, base_seed = 12345)
# Plot trial statuses at each look according to total allocations
plot_status(res_mult, x_value = "total n")
}
if (requireNamespace("ggplot2", quietly = TRUE)){
# Plot trial statuses for all arms
plot_status(res_mult, arm = NA)
}
Calculates matrix of absolute distances raised to a power
Description
Used internally, calculates the absolute distances for values in a matrix with possibly unequal dimensions, and raises these to a power.
Usage
pow_abs_dist(x1, x2 = x1, pow = 2)
Arguments
x1 |
numeric vector, with length corresponding to the number of rows in the returned matrix. |
x2 |
numeric vector, with length corresponding to the number of columns
in the returned matrix. If not specified, |
pow |
single numeric value, the power that all distances are raised to.
Defaults to |
Value
Matrix with length(x1)
rows and length(x2)
columns including the
calculated absolute pairwise distances raised to pow
.
Print methods for adaptive trial objects
Description
Prints contents of the first input x
in a human-friendly way, see
Details for more information.
Usage
## S3 method for class 'trial_spec'
print(x, prob_digits = 3, ...)
## S3 method for class 'trial_result'
print(x, prob_digits = 3, ...)
## S3 method for class 'trial_performance'
print(x, digits = 3, ...)
## S3 method for class 'trial_results'
print(
x,
select_strategy = "control if available",
select_last_arm = FALSE,
select_preferences = NULL,
te_comp = NULL,
raw_ests = FALSE,
final_ests = NULL,
restrict = NULL,
digits = 1,
cores = NULL,
...
)
## S3 method for class 'trial_results_summary'
print(x, digits = 1, ...)
## S3 method for class 'trial_calibration'
print(x, ...)
Arguments
x |
object to print, see Details. |
prob_digits |
single integer (default is |
... |
additional arguments, not used. |
digits |
single integer, the number of digits used when printing
the numeric results. Default is |
select_strategy |
single character string. If a trial was not stopped
due to superiority (or had only 1 arm remaining, if
|
select_last_arm |
single logical, defaults to |
select_preferences |
character vector specifying a number of arms used
for selection if one of the |
te_comp |
character string, treatment-effect comparator. Can be either
|
raw_ests |
single logical. If |
final_ests |
single logical. If |
restrict |
single character string or |
cores |
|
Details
The behaviour depends on the class of x
:
-
trial_spec
: prints a trial specification setup bysetup_trial()
,setup_trial_binom()
orsetup_trial_norm()
.
-
trial_result
: prints the results of a single trial simulated byrun_trial()
. More details are saved in thetrial_result
object and thus printed if thesparse
argument inrun_trial()
orrun_trials()
is set toFALSE
; ifTRUE
, fewer details are printed, but the omitted details are available by printing thetrial_spec
object created bysetup_trial()
,setup_trial_binom()
orsetup_trial_norm()
.
-
trial_results
: prints the results of multiple simulations generated usingrun_trials()
. Further documentation on how multiple trials are summarised before printing can be found in thesummary()
function documentation.
-
trial_results_summary
: print method for summary of multiple simulations of the same trial specification, generated by using thesummary()
function on an object generated byrun_trials()
.
Value
Invisibly returns x
.
Methods (by class)
-
print(trial_spec)
: Trial specification -
print(trial_result)
: Single trial result -
print(trial_performance)
: Trial performance metrics -
print(trial_results)
: Multiple trial results -
print(trial_results_summary)
: Summary of multiple trial results -
print(trial_calibration)
: Trial calibration
Calculate the probability that all arms are practically equivalent
Description
Used internally. This function takes a matrix
as calculated by the
get_draws_binom()
, get_draws_norm()
or a corresponding custom function
(specified using the fun_draws
argument in setup_trial()
; see
get_draws_generic()
), and an equivalence difference, and calculates the
probability of all arms being equivalent (absolute differences between
highest and lowest value in the same set of posterior draws being less than
the difference considered practically equivalent).
Usage
prob_all_equi(m, equivalence_diff = NULL)
Arguments
m |
a matrix with one column per trial arm (named as the |
equivalence_diff |
single numeric value ( |
Value
A single numeric value corresponding to the probability of all arms being practically equivalent.
Calculate the probabilities of each arm being the best
Description
Used internally. This function takes a matrix
as calculated by the
get_draws_binom()
, get_draws_norm()
or a corresponding custom function
(as specified using the fun_draws
argument in setup_trial()
; see
get_draws_generic()
) and calculates the probabilities of each arm being the
best (defined as either the highest or the lowest value, as specified by the
highest_is_best
argument in setup_trial()
, setup_trial_binom()
or
setup_trial_norm()
).
Usage
prob_best(m, highest_is_best = FALSE)
Arguments
m |
a matrix with one column per trial arm (named as the |
highest_is_best |
single logical, specifies whether larger estimates of
the outcome are favourable or not; defaults to |
Value
A named numeric vector of probabilities (names corresponding to
arms
).
Calculate probabilities of comparisons of arms against with common control
Description
Used internally. This function takes a matrix
as calculated by the
get_draws_binom()
, get_draws_norm()
or a corresponding custom function
(as specified using the fun_draws
argument in setup_trial()
; see
get_draws_generic()
) and a single character specifying the control
arm,
and calculates the probabilities of each arm being better than a common
control
(defined as either higher or lower than the control
, as specified
by the highest_is_best
argument in setup_trial()
, setup_trial_binom()
or setup_trial_norm()
). This function also calculates equivalence and
futility probabilities compared to the common control
arm, as specified in
setup_trial()
, setup_trial_binom()
or setup_trial_norm()
, unless
equivalence_diff
or futility_diff
, respectively, are set to NULL
(the default).
Usage
prob_better(
m,
control = NULL,
highest_is_best = FALSE,
equivalence_diff = NULL,
futility_diff = NULL
)
Arguments
m |
a matrix with one column per trial arm (named as the |
control |
a single character string specifying the common |
highest_is_best |
single logical, specifies whether larger estimates of
the outcome are favourable or not; defaults to |
equivalence_diff |
single numeric value ( |
futility_diff |
single numeric value ( |
Value
A named (row names corresponding to the trial arms
) matrix
containing 1-3 columns: probs_better
, probs_equivalence
(if
equivalence_diff
is specified), and probs_futile
(if futility_diff
is
specified). All columns will contain NA
for the control arm.
Generate breakpoints and other values for printing progress
Description
Used internally. Generates breakpoints, messages, and 'batches' of trial
numbers to simulate when using run_trials()
with the progress
argument in
use. Breaks will be multiples of the number of cores
, and repeated use of
the same values for breaks is avoided (if, e.g., the number of breaks times
the number of cores is not possible if few new trials are to be run). Inputs
are validated by run_trials()
.
Usage
prog_breaks(progress, prev_n_rep, n_rep_new, cores)
Arguments
progress |
single numeric |
prev_n_rep |
single integer, the previous number of simulations run (to add to the indices generated and used). |
n_rep_new |
single integers, number of new simulations to run (i.e.,
|
cores |
|
Value
List containing breaks
(the number of patients at each break),
start_mess
and prog_mess
(the basis of the first and subsequent
progress messages), and batches
(a list with each entry corresponding to
the simulation numbers in each batch).
Update allocation probabilities
Description
Used internally. This function calculates new allocation probabilities for
each arm, based on the information specified in setup_trial()
,
setup_trial_binom()
or setup_trial_norm()
and the calculated
probabilities of each arm being the best by prob_best()
.
Usage
reallocate_probs(
probs_best,
fixed_probs,
min_probs,
max_probs,
soften_power = 1,
match_arm = NULL,
rescale_fixed = FALSE,
rescale_limits = FALSE,
rescale_factor = 1,
rescale_ignore = NULL
)
Arguments
probs_best |
a resulting named vector from the |
fixed_probs |
numeric vector, fixed allocation probabilities for each
arm. Must be either a numeric vector with |
min_probs |
numeric vector, lower threshold for adaptive allocation
probabilities; lower probabilities will be rounded up to these values. Must
be |
max_probs |
numeric vector, upper threshold for adaptive allocation
probabilities; higher probabilities will be rounded down to these values.
Must be |
soften_power |
either a single numeric value or a numeric vector of
exactly the same length as the maximum number of looks/adaptive analyses.
Values must be between |
match_arm |
index of the |
rescale_fixed |
logical indicating whether |
rescale_limits |
logical indicating whether |
rescale_factor |
numerical, rescale factor defined as
|
rescale_ignore |
|
Value
A named (according to the arms
) numeric vector with updated
allocation probabilities.
Replace non-finite values with other value (finite-OR-operator)
Description
Used internally, helper function that replaces non-finite (i.e., NA
, NaN
,
Inf
, and -Inf
) values according to is.finite()
, primarily used to
replace NaN
/Inf
/-Inf
with NA
.
Usage
a %f|% b
Arguments
a |
atomic vector of any type. |
b |
single value to replace non-finite values with. |
Value
If values in a
are non-finite, they are replaced with b
,
otherwise they are left unchanged.
Replace NULL with other value (NULL-OR-operator)
Description
Used internally, primarily when working with list arguments, because, e.g.,
list_name$element_name
yields NULL
when unspecified.
Usage
a %||% b
Arguments
a , b |
atomic values of any type. |
Value
If a
is NULL
, b
is returned. Otherwise a
is returned.
Rescale numeric vector to sum to 1
Description
Used internally.
Usage
rescale(x)
Arguments
x |
numeric vector. |
Value
Numeric vector, x
rescaled to sum to a total of 1.
Simulate a single trial
Description
This function conducts a single trial simulation using a trial specification
as specified by setup_trial()
, setup_trial_binom()
or
setup_trial_norm()
.
During simulation, the function randomises "patients", randomly generates
outcomes, calculates the probabilities that each arm
is the best (and
better than the control, if any). This is followed by checking inferiority,
superiority, equivalence and/or futility as desired; dropping arms, and
re-adjusting allocation probabilities according to the criteria specified in
the trial specification. If there is no common control
arm, the trial
simulation will be stopped at the final specified adaptive analysis, when 1
arm is superior to the others, or when all arms are considered equivalent (if
equivalence is assessed). If a common control
arm is specified, all other
arms will be compared to that, and if 1 of these pairwise comparisons crosses
the applicable superiority threshold at an adaptive analysis, that arm will
become the new control and the old control will be considered inferior and
dropped. If multiple non-control arms cross the applicable superiority
threshold in the same adaptive analysis, the one with the highest probability
of being the overall best will become the new control. Equivalence/futility
will also be checked if specified, and equivalent or futile arms will be
dropped in designs with a common control
arm and the entire trial will be
stopped if all remaining arms are equivalent in designs without a common
control
arm. The trial simulation will be stopped when only 1 arm is left,
when the final arms are all equivalent, or after the final specified adaptive
analysis.
After stopping (regardless of reason), a final analysis including outcome
data from all patients randomised to all arms will be conducted (with the
final control
arm, if any, used as the control
in this analysis).
Results from this analysis will be saved, but not used with regards to the
adaptive stopping rules. This is particularly relevant if less patients have
available outcome data at the last adaptive analyses than the total number of
patients randomised (as specified in setup_trial()
, setup_trial_binom()
,
or setup_trial_norm()
), as the final analysis will then include all
patients randomised, which may be more than in the last adaptive analysis
conducted.
Usage
run_trial(trial_spec, seed = NULL, sparse = FALSE)
Arguments
trial_spec |
|
seed |
single integer or |
sparse |
single logical; if |
Value
A trial_result
object containing everything listed below if
sparse
(as described above) is FALSE
. Otherwise only final_status
,
final_n
, followed_n
, trial_res
, seed
, and sparse
are included.
-
final_status
: either"superiority"
,"equivalence"
,"futility"
, or"max"
(stopped at the last possible adaptive analysis), as calculated during the adaptive analyses. -
final_n
: the total number of patients randomised. -
followed_n
: the total number of patients with available outcome data at the last adaptive analysis conducted. -
max_n
: the pre-specified maximum number of patients with outcome data available at the last possible adaptive analysis. -
max_randomised
: the pre-specified maximum number of patients randomised at the last possible adaptive analysis. -
looks
: numeric vector, the total number of patients with outcome data available at each conducted adaptive analysis. -
planned_looks
: numeric vector, the cumulated number of patients planned to have outcome data available at each adaptive analysis, even those not conducted if the simulation is stopped before the final possible analysis. -
randomised_at_looks
: numeric vector, the cumulated number of patients randomised at each conducted adaptive analysis (only including the relevant numbers for the analyses actually conducted). -
start_control
: character, initial commoncontrol
arm (if specified). -
final_control
: character, final commoncontrol
arm (if relevant). -
control_prob_fixed
: fixed commoncontrol
arm probabilities (if specified; seesetup_trial()
). -
inferiority
,superiority
,equivalence_prob
,equivalence_diff
,equivalence_only_first
,futility_prob
,futility_diff
,futility_only_first
,highest_is_best
, andsoften_power
: as specified insetup_trial()
. -
best_arm
: the bestarm
(s), as described insetup_trial()
. -
trial_res
: adata.frame
containing most of the information specified for each arm insetup_trial()
includingtrue_ys
(true outcomes as specified insetup_trial()
) and for each arm the sum of the outcomes (sum_ys
/sum_ys_all
; i.e., the total number of events for binary outcomes or the totals of continuous outcomes) and sum of patients (ns
/ns_all
), summary statistics for the raw outcome data (raw_ests
/raw_ests_all
, calculated as specified insetup_trial()
, defaults to mean values, i.e., event rates for binary outcomes or means for continuous outcomes) and posterior estimates (post_ests
/post_ests_all
,post_errs
/post_errs_all
,lo_cri
/lo_cri_all
, andhi_cri
/hi_cri_all
, calculated as specified insetup_trial()
),final_status
of each arm ("inferior"
,"superior"
,"equivalence"
,"futile"
,"active"
, or"control"
(currently active control arm, including if the current control when stopped for equivalence)),status_look
(specifying the cumulated number of patients with outcome data available when an adaptive analysis changed thefinal_status
to"superior"
,"inferior"
,"equivalence"
, or"futile"
),status_probs
, the probability (in the last adaptive analysis for each arm) that each arm was the best/better than the common control arm (if any)/equivalent to the common control arm (if any and stopped for equivalence;NA
if the control arm was stopped due to the last remaining other arm(s) being stopped for equivalence)/futile if stopped for futility at the last analysis it was included in,final_alloc
, the final allocation probability for each arm the last time patients were randomised to it, including for arms stopped at the maximum sample size, andprobs_best_last
, the probabilities of each remaining arm being the overall best in the last conducted adaptive analysis (NA
for previously dropped arms).
Note: for the variables in thedata.frame
where a version including the_all
-suffix is included, the versions WITHOUT this suffix are calculated using patients with available outcome data at the time of analysis, while the versions WITH the_all
-suffixes are calculated using outcome data for all patients randomised at the time of analysis, even if they have not reached the time of follow-up yet (seesetup_trial()
). -
all_looks
: a list of lists containing one list per conducted trial look (adaptive analysis). These lists contain the variablesarms
,old_status
(status before the analysis of the current round was conducted),new_status
(as specified above, status after current analysis has been conducted),sum_ys
/sum_ys_all
(as described above),ns
/ns_all
(as described above),old_alloc
(the allocation probability used during this look),probs_best
(the probabilities of each arm being the best in the current adaptive analysis),new_alloc
(the allocation probabilities after updating these in the current adaptive analysis; NA for all arms when the trial is stopped and no further adaptive analyses will be conducted),probs_better_first
(if a common control is provided, specifying the probabilities that each arm was better than the control in the first analysis conducted during that look),probs_better
(asprobs_better_first
, but updated if another arm becomes the new control),probs_equivalence_first
andprobs_equivalence
(as forprobs_better
/probs_better_first
, but for equivalence if equivalence is assessed). The last variables areNA
if the arm was not active in the applicable adaptive analysis or if they would not be included during the next adaptive analysis. -
allocs
: a character vector containing the allocations of all patients in the order of randomization. -
ys
: a numeric vector containing the outcomes of all patients in the order of randomization (0
or1
for binary outcomes). -
seed
: the random seed used, if specified. -
description
,add_info
,cri_width
,n_draws
,robust
: as specified insetup_trial()
,setup_trial_binom()
orsetup_trial_norm()
. -
sparse
: single logical, corresponding to thesparse
input.
Examples
# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
true_ys = c(0.20, 0.18, 0.22, 0.24),
data_looks = 1:20 * 100)
# Run trial with a specified random seed
res <- run_trial(binom_trial, seed = 12345)
# Print results with 3 decimals
print(res, digits = 3)
Simulate multiple trials
Description
This function conducts multiple simulations using a trial specification as
specified by setup_trial()
, setup_trial_binom()
or setup_trial_norm()
.
This function essentially manages random seeds and runs multiple simulation
using run_trial()
- additional details on individual simulations are
provided in that function's description. This function allows simulating
trials in parallel using multiple cores, automatically saving and re-loading
saved objects, and "growing" already saved simulation files (i.e., appending
additional simulations to the same file).
Usage
run_trials(
trial_spec,
n_rep,
path = NULL,
overwrite = FALSE,
grow = FALSE,
cores = NULL,
base_seed = NULL,
sparse = TRUE,
progress = NULL,
version = NULL,
compress = TRUE,
export = NULL,
export_envir = parent.frame()
)
Arguments
trial_spec |
|
n_rep |
single integer; the number of simulations to run. |
path |
single character string; if specified (defaults to |
overwrite |
single logical; defaults to |
grow |
single logical; defaults to |
cores |
|
base_seed |
single integer or |
sparse |
single logical, as described in |
progress |
single numeric |
version |
passed to |
compress |
passed to |
export |
character vector of names of objects to export to each
parallel core when running in parallel; passed as the |
export_envir |
|
Details
Exporting objects when using multiple cores
If setup_trial()
is used to define a trial specification with custom
functions (in the fun_y_gen
, fun_draws
, and fun_raw_est
arguments of
setup_trial()
) and run_trials()
is run with cores > 1
, it is necessary
to export additional functions or objects used by these functions and defined
by the user outside the function definitions provided. Similarly, functions
from external packages loaded using library()
or require()
must be
exported or called prefixed with the namespace, i.e., package::function
.
The export
and export_envir
arguments are used to export objects calling
the parallel::clusterExport()
-function. See also setup_cluster()
, which
may be used to setup a cluster and export required objects only once per
session.
Value
A list of a special class "trial_results"
, which contains the
trial_results
(results from all simulations; note that seed
will be
NULL
in the individual simulations), trial_spec
(the trial
specification), n_rep
, base_seed
, elapsed_time
(the total simulation
run time), sparse
(as described above) and adaptr_version
(the version
of the adaptr
package used to run the simulations). These results may be
extracted, summarised, and plotted using the extract_results()
,
check_performance()
, summary()
, print.trial_results()
,
plot_convergence()
, check_remaining_arms()
, plot_status()
, and
plot_history()
functions. See the definitions of these functions for
additional details and details on additional arguments used to select arms
in simulations not ending in superiority and other summary choices.
Examples
# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
true_ys = c(0.20, 0.18, 0.22, 0.24),
data_looks = 1:20 * 100)
# Run 10 simulations with a specified random base seed
res <- run_trials(binom_trial, n_rep = 10, base_seed = 12345)
# See ?extract_results, ?check_performance, ?summary and ?print for details
# on extracting resutls, summarising and printing
Setup default cluster for use in parallelised adaptr functions
Description
This function setups (or removes) a default cluster for use in all
parallelised functions in adaptr
using the parallel
package. The function
also exports objects that should be available on the cluster and sets the
random number generator appropriately. See Details for further info on
how adaptr
handles sequential/parallel computation.
Usage
setup_cluster(cores, export = NULL, export_envir = parent.frame())
Arguments
cores |
can be either unspecified, |
export |
character vector of names of objects to export to each
parallel core when running in parallel; passed as the |
export_envir |
|
Details
Using sequential or parallel computing in adaptr
All parallelised adaptr
functions have a cores
argument that defaults to
NULL
. If a non-NULL
integer > 0
is provided to the cores
argument in
any of those (except setup_cluster()
), the package will run calculations
sequentially in the main process if cores = 1
, and otherwise initiate a new
cluster of size cores
that will be removed once the function completes,
regardless of whether or not a default cluster or the global "mc.cores"
option have been specified.
If cores
is NULL
in any adaptr
function (except setup_cluster()
), the
package will use a default cluster if one exists or run computations
sequentially if setup_cluster()
has last been called with cores = 1
.
If setup_cluster()
has not been called or last called with cores = NULL
,
then the package will check if the global "mc.cores"
option has been
specified (using options(mc.cores = <number of cores>)
). If this option has
been set with a value > 1
, then a new, temporary cluster of that size is
setup, used, and removed once the function completes. If this option has not
been set or has been set to 1
, then computations will be run sequentially
in the main process.
Generally, we recommend using the setup_cluster()
function as this avoids
the overhead of re-initiating new clusters with every call to one of the
parallelised adaptr
functions. This is especially important when exporting
many or large objects to a parallel
cluster, as this can then be done only
once (with the option to export further objects to the same cluster when
calling run_trials()
).
Type of clusters used and random number generation
The adaptr
package solely uses parallel socket clusters (using
parallel::makePSOCKcluster()
) and thus does not use forking (as this is not
available on all operating systems and may cause crashes in some situations).
As such, user-defined objects that should be used by the adaptr
functions
when run in parallel need to be exported using either setup_cluster()
or
run_trials()
, if not included in the generated trial_spec
object.
The adaptr
package uses the "L'Ecuyer-CMRG"
kind (see RNGkind()
) for
safe random number generation for all parallelised functions. This is also
the case when running adaptr
functions sequentially with a seed provided,
to ensure that the same results are obtained regardless of whether sequential
or parallel computation is used. All functions restore both the random number
generator kind and the global random seed after use if called with a seed.
Value
Invisibly returns the default parallel
cluster or NULL
, as
appropriate. This may be used with other functions from the parallel
package by advanced users, for example to load certain libraries on the
cluster prior to calling run_trials()
.
Examples
# Setup a cluster using 2 cores
setup_cluster(cores = 2)
# Get existing default cluster (printed here as invisibly returned)
print(setup_cluster())
# Remove existing default cluster
setup_cluster(cores = NULL)
# Specify preference for running computations sequentially
setup_cluster(cores = 1)
# Remove default cluster preference
setup_cluster(cores = NULL)
# Set global option to default to using 2 new clusters each time
# (only used if no default cluster preference is specified)
options(mc.cores = 2)
Setup a generic trial specification
Description
Specifies the design of an adaptive trial with any type of outcome and
validates all inputs. Use calibrate_trial()
to calibrate the trial
specification to obtain a specific value for a certain performance metric
(e.g., the Bayesian type 1 error rate). Use run_trial()
or run_trials()
to conduct single/multiple simulations of the specified trial, respectively.
See setup_trial_binom()
and setup_trial_norm()
for simplified setup
of trial designs for common outcome types. For additional trial specification
examples, see the the Basic examples vignette
(vignette("Basic-examples", package = "adaptr")
) and the
Advanced example vignette
(vignette("Advanced-example", package = "adaptr")
).
Usage
setup_trial(
arms,
true_ys,
fun_y_gen = NULL,
fun_draws = NULL,
start_probs = NULL,
fixed_probs = NULL,
min_probs = rep(NA, length(arms)),
max_probs = rep(NA, length(arms)),
rescale_probs = NULL,
data_looks = NULL,
max_n = NULL,
look_after_every = NULL,
randomised_at_looks = NULL,
control = NULL,
control_prob_fixed = NULL,
inferiority = 0.01,
superiority = 0.99,
equivalence_prob = NULL,
equivalence_diff = NULL,
equivalence_only_first = NULL,
futility_prob = NULL,
futility_diff = NULL,
futility_only_first = NULL,
highest_is_best = FALSE,
soften_power = 1,
fun_raw_est = mean,
cri_width = 0.95,
n_draws = 5000,
robust = TRUE,
description = NULL,
add_info = NULL
)
Arguments
arms |
character vector with unique names for the trial arms. |
true_ys |
numeric vector specifying true outcomes (e.g., event
probabilities, mean values, etc.) for all trial |
fun_y_gen |
function, generates outcomes. See |
fun_draws |
function, generates posterior draws. See |
start_probs |
numeric vector, allocation probabilities for each arm at
the beginning of the trial. The default ( |
fixed_probs |
numeric vector, fixed allocation probabilities for each
arm. Must be either a numeric vector with |
min_probs |
numeric vector, lower threshold for adaptive allocation
probabilities; lower probabilities will be rounded up to these values. Must
be |
max_probs |
numeric vector, upper threshold for adaptive allocation
probabilities; higher probabilities will be rounded down to these values.
Must be |
rescale_probs |
|
data_looks |
vector of increasing integers, specifies when to conduct
adaptive analyses (= the total number of patients with available outcome
data at each adaptive analysis). The last number in the vector represents
the final adaptive analysis, i.e., the final analysis where superiority,
inferiority, practical equivalence, or futility can be claimed.
Instead of specifying |
max_n |
single integer, number of patients with available outcome data
at the last possible adaptive analysis (defaults to |
look_after_every |
single integer, specified together with |
randomised_at_looks |
vector of increasing integers or |
control |
single character string, name of one of the |
control_prob_fixed |
if a common |
inferiority |
single numeric value or vector of numeric values of the
same length as the maximum number of possible adaptive analyses, specifying
the probability threshold(s) for inferiority (default is |
superiority |
single numeric value or vector of numeric values of the
same length as the maximum number of possible adaptive analyses, specifying
the probability threshold(s) for superiority (default is |
equivalence_prob |
single numeric value, vector of numeric values of the
same length as the maximum number of possible adaptive analyses or |
equivalence_diff |
single numeric value ( |
equivalence_only_first |
single logical in trial specifications where
|
futility_prob |
single numeric value, vector of numeric values of the
same length as the maximum number of possible adaptive analyses or |
futility_diff |
single numeric value ( |
futility_only_first |
single logical in trial specifications designs
where |
highest_is_best |
single logical, specifies whether larger estimates of
the outcome are favourable or not; defaults to |
soften_power |
either a single numeric value or a numeric vector of
exactly the same length as the maximum number of looks/adaptive analyses.
Values must be between |
fun_raw_est |
function that takes a numeric vector and returns a
single numeric value, used to calculate a raw summary estimate of the
outcomes in each |
cri_width |
single numeric |
n_draws |
single integer, the number of draws from the posterior
distributions for each arm used when running the trial. Defaults to
|
robust |
single logical, if |
description |
optional single character string describing the trial
design, will only be used in print functions if not |
add_info |
optional single string containing additional information
regarding the trial design or specifications, will only be used in print
functions if not |
Details
How to specify the fun_y_gen
function
The function must take the following arguments:
-
allocs
: character vector, the trialarms
that new patients allocated since the last adaptive analysis are randomised to.
The function must return a single numeric vector, corresponding to the
outcomes for all patients allocated since the last adaptive analysis, in the
same order as allocs
.
See the Advanced example vignette
(vignette("Advanced-example", package = "adaptr")
) for an example with
further details.
How to specify the fun_draws
function
The function must take the following arguments:
-
arms
: character vector, the unique trialarms
, in the same order as above, but only the currently active arms are included when the function is called. -
allocs
: a vector of allocations for all patients, corresponding to the trialarms
, including patients allocated to both currently active AND inactivearms
when called. -
ys
: a vector of outcomes for all patients in the same order asallocs
, including outcomes for patients allocated to both currently active AND inactivearms
when called. -
control
: single character, the currentcontrol
arm, will beNULL
for designs without a common control arm, but required regardless as the argument is supplied byrun_trial()
/run_trials()
. -
n_draws
: single integer, the number of posterior draws for each arm.
The function must return a matrix
(containing numeric values) with arms
named columns and n_draws
rows. The matrix
must have columns
only for currently active arms (when called). Each row should contain a
single posterior draw for each arm on the original outcome
scale: if they are estimated as, e.g., the log(odds), these estimates must
be transformed to probabilities and similarly for other measures.
Important: the matrix
cannot contain NA
s, even if no patients have been
randomised to an arm yet. See the provided example for one way to alleviate
this.
See the Advanced examples vignette
(vignette("Advanced-example", package = "adaptr")
) for an example with
further details.
Notes
Different estimation methods and prior distributions may be used; complex functions will lead to slower simulations compared to simpler methods for obtaining posterior draws, including those specified using the
setup_trial_binom()
andsetup_trial_norm()
functions.Technically, using log relative effect measures — e.g. log(odds ratios) or log(risk ratios) - or differences compared to a reference arm (e.g., mean differences or absolute risk differences) instead of absolute values in each arm will work to some extent (be cautious!):
Stopping for superiority/inferiority/max sample sizes will work.
Stopping for equivalence/futility may be used with relative effect measures on the log scale, but thresholds have to be adjusted accordingly.
Several summary statistics from
run_trial()
(sum_ys
and posterior estimates) may be nonsensical if relative effect measures are used (depending on calculation method; see theraw_ests
argument in the relevant functions).In the same vein,
extract_results()
(sum_ys
,sq_err
, andsq_err_te
), andsummary()
(sum_ys_mean/sd/median/q25/q75/q0/q100
,rmse
, andrmse_te
) may be equally nonsensical when calculated on the relative scale (see theraw_ests
argument in the relevant functions.
Using additional custom or functions from loaded packages in the custom functions
If the fun_y_gen
, fun_draws
, or fun_raw_est
functions calls other
user-specified functions (or uses objects defined by the user outside these
functions or the setup_trial()
-call) or functions from external packages
and simulations are conducted on multiple cores, these objects or functions
must be prefixed with their namespaces (i.e., package::function()
) or
exported, as described in setup_cluster()
and run_trials()
.
More information on arguments
-
control
: if one or more treatment arms are superior to the control arm (i.e., passes the superiority threshold as defined above), this arm will become the new control (if multiple arms are superior, the one with the highest probability of being the overall best will become the new control), the previous control will be dropped for inferiority, and all remaining arms will be immediately compared to the new control in the same adaptive analysis and dropped if inferior (or possibly equivalent/futile, see below) compared to this new control arm. Only applies in trials with a commoncontrol
. -
control_prob_fixed
: If the length is 1, then this allocation probability will be used for thecontrol
group (including if a new arm becomes the control and the original control is dropped). If multiple values are specified the first value will be used when all arms are active, the second when one arm has been dropped, and so forth. If 1 or more values are specified, previously setfixed_probs
,min_probs
ormax_probs
for new control arms will be ignored. If all allocation probabilities do not sum to 1 (e.g, due to multiple limits) they will be rescaled to do so.
Can also be set to one of the special arguments"sqrt-based"
,"sqrt-based start"
,"sqrt-based fixed"
or"match"
(written exactly as one of those, case sensitive). This requiresstart_probs
to beNULL
and relevantfixed_probs
to beNULL
(orNA
for the control arm).
If one of the"sqrt-based"/"sqrt-based start"/"sqrt-based fixed"
options are used, the function will set square-root-transformation-based starting allocation probabilities. These are defined as:
square root of number of non-control arms to 1-ratio for other arms
scaled to sum to 1, which will generally increase power for comparisons against the commoncontrol
, as discussed in, e.g., Park et al, 2020 doi:10.1016/j.jclinepi.2020.04.025.
If"sqrt-based"
or"sqrt-based fixed"
, square-root-transformation-based allocation probabilities will be used initially and also for new controls when arms are dropped (with probabilities always calculated based on the number of active non-control arms). If"sqrt-based"
, response-adaptive randomisation will be used for non-control arms, while the non-control arms will use fixed, square-root based allocation probabilities at all times (with probabilities always calculated based on the number of active non-control arms). If"sqrt-based start"
, the control arm allocation probability will be fixed to a square-root based probability at all times calculated according to the initial number of arms (with this probability also being used for new control(s) when the original control is dropped).
If"match"
is specified, the control group allocation probability will always be matched to be similar to the highest non-control arm allocation probability.
Superiority and inferiority
In trial designs without a common control arm, superiority and inferiority
are assessed by comparing all currently active groups. This means that
if a "final" analysis of a trial without a common control and > 2 arms
is
conducted including all arms (as will often be done in practice) after an
adaptive trial has stopped, the final probabilities of the best arm being
superior may differ slightly.
For example, in a trial with three arms and no common control
arm, one arm
may be dropped early for inferiority defined as < 1%
probability of being
the overall best arm
. The trial may then continue with the two remaining
arms, and stopped when one is declared superior to the other defined as
> 99%
probability of being the overall best arm
. If a final analysis is
then conducted including all arms, the final probability of the best arm
being overall superior will generally be slightly lower as the probability
of the first dropped arm being the best will often be > 0%
, even if very
low and below the inferiority threshold.
This is less relevant trial designs with a common control
, as pairwise
assessments of superiority/inferiority compared to the common control
will
not be influenced similarly by previously dropped arms (and previously
dropped arms may be included in the analyses, even if posterior distributions
are not returned for those).
Similarly, in actual clinical trials and when randomised_at_looks
is
specified with numbers higher than the number of patients with available
outcome data at each analysis, final probabilities may change somewhat when
the all patients are have completed follow-up and are included in a final
analysis.
Equivalence
Equivalence is assessed after both inferiority and superiority have
been assessed (and in case of superiority, it will be assessed against the
new control
arm in designs with a common control
, if specified - see
above).
Futility
Futility is assessed after inferiority, superiority, and equivalence have been assessed (and in case of superiority, it will be assessed against the new control arm in designs with a common control, if specified - see above). Arms will thus be dropped for equivalence before futility.
Varying probability thresholds
Different probability thresholds (for superiority, inferiority, equivalence,
and futility) may be specified for different adaptive analyses. This may be
used, e.g., to apply more strict probability thresholds at earlier analyses
(or make one or more stopping rules not apply at earlier analyses), similar
to the use of monitoring boundaries with different thresholds used for
interim analyses in conventional, frequentist group sequential trial designs.
See the Basic examples vignette
(vignette("Basic-examples", package = "adaptr")
) for an example.
Value
A trial_spec
object used to run simulations by run_trial()
or
run_trials()
. The output is essentially a list containing the input
values (some combined in a data.frame
called trial_arms
), but its class
signals that these inputs have been validated and inappropriate
combinations and settings have been ruled out. Also contains best_arm
,
holding the arm(s) with the best value(s) in true_ys
. Use str()
to
peruse the actual content of the returned object.
Examples
# Setup a custom trial specification with right-skewed, log-normally
# distributed continuous outcomes (higher values are worse)
# Define the function that will generate the outcomes in each arm
# Notice: contents should match arms/true_ys in the setup_trial() call below
get_ys_lognorm <- function(allocs) {
y <- numeric(length(allocs))
# arms (names and order) and values (except for exponentiation) should match
# those used in setup_trial (below)
means <- c("Control" = 2.2, "Experimental A" = 2.1, "Experimental B" = 2.3)
for (arm in names(means)) {
ii <- which(allocs == arm)
y[ii] <- rlnorm(length(ii), means[arm], 1.5)
}
y
}
# Define the function that will generate posterior draws
# In this example, the function uses no priors (corresponding to improper
# flat priors) and calculates results on the log-scale, before exponentiating
# back to the natural scale, which is required for assessments of
# equivalence, futility and general interpretation
get_draws_lognorm <- function(arms, allocs, ys, control, n_draws) {
draws <- list()
logys <- log(ys)
for (arm in arms){
ii <- which(allocs == arm)
n <- length(ii)
if (n > 1) {
# Necessary to avoid errors if too few patients randomised to this arm
draws[[arm]] <- exp(rnorm(n_draws, mean = mean(logys[ii]), sd = sd(logys[ii])/sqrt(n - 1)))
} else {
# Too few patients randomised to this arm - extreme uncertainty
draws[[arm]] <- exp(rnorm(n_draws, mean = mean(logys), sd = 1000 * (max(logys) - min(logys))))
}
}
do.call(cbind, draws)
}
# The actual trial specification is then defined
lognorm_trial <- setup_trial(
# arms should match those above
arms = c("Control", "Experimental A", "Experimental B"),
# true_ys should match those above
true_ys = exp(c(2.2, 2.1, 2.3)),
fun_y_gen = get_ys_lognorm, # as specified above
fun_draws = get_draws_lognorm, # as specified above
max_n = 5000,
look_after_every = 200,
control = "Control",
# Square-root-based, fixed control group allocation ratio
# and response-adaptive randomisation for other arms
control_prob_fixed = "sqrt-based",
# Equivalence assessment
equivalence_prob = 0.9,
equivalence_diff = 0.5,
equivalence_only_first = TRUE,
highest_is_best = FALSE,
# Summarise raw results by taking the mean on the
# log scale and back-transforming
fun_raw_est = function(x) exp(mean(log(x))) ,
# Summarise posteriors using medians with MAD-SDs,
# as distributions will not be normal on the actual scale
robust = TRUE,
# Description/additional info used when printing
description = "continuous, log-normally distributed outcome",
add_info = "SD on the log scale for all arms: 1.5"
)
# Print trial specification with 3 digits for all probabilities
print(lognorm_trial, prob_digits = 3)
Setup a trial specification using a binary, binomially distributed outcome
Description
Specifies the design of an adaptive trial with a binary, binomially
distributed outcome and validates all inputs. Uses beta-binomial
conjugate models with beta(1, 1)
prior distributions, corresponding to a
uniform prior (or the addition of 2 patients, 1 with an event and 1 without,
in each arm
) to the trial. Use calibrate_trial()
to calibrate the trial
specification to obtain a specific value for a certain performance metric
(e.g., the Bayesian type 1 error rate). Use run_trial()
or run_trials()
to conduct single/multiple simulations of the specified trial, respectively.
Note: add_info
as specified in setup_trial()
is set to NULL
for
trial specifications setup by this function.
Further details: please see setup_trial()
. See setup_trial_norm()
for
simplified setup of trials with a normally distributed continuous outcome.
For additional trial specification examples, see the the Basic examples
vignette (vignette("Basic-examples", package = "adaptr")
) and the
Advanced example vignette
(vignette("Advanced-example", package = "adaptr")
).
Usage
setup_trial_binom(
arms,
true_ys,
start_probs = NULL,
fixed_probs = NULL,
min_probs = rep(NA, length(arms)),
max_probs = rep(NA, length(arms)),
rescale_probs = NULL,
data_looks = NULL,
max_n = NULL,
look_after_every = NULL,
randomised_at_looks = NULL,
control = NULL,
control_prob_fixed = NULL,
inferiority = 0.01,
superiority = 0.99,
equivalence_prob = NULL,
equivalence_diff = NULL,
equivalence_only_first = NULL,
futility_prob = NULL,
futility_diff = NULL,
futility_only_first = NULL,
highest_is_best = FALSE,
soften_power = 1,
cri_width = 0.95,
n_draws = 5000,
robust = TRUE,
description = "generic binomially distributed outcome trial"
)
Arguments
arms |
character vector with unique names for the trial arms. |
true_ys |
numeric vector, true probabilities (between |
start_probs |
numeric vector, allocation probabilities for each arm at
the beginning of the trial. The default ( |
fixed_probs |
numeric vector, fixed allocation probabilities for each
arm. Must be either a numeric vector with |
min_probs |
numeric vector, lower threshold for adaptive allocation
probabilities; lower probabilities will be rounded up to these values. Must
be |
max_probs |
numeric vector, upper threshold for adaptive allocation
probabilities; higher probabilities will be rounded down to these values.
Must be |
rescale_probs |
|
data_looks |
vector of increasing integers, specifies when to conduct
adaptive analyses (= the total number of patients with available outcome
data at each adaptive analysis). The last number in the vector represents
the final adaptive analysis, i.e., the final analysis where superiority,
inferiority, practical equivalence, or futility can be claimed.
Instead of specifying |
max_n |
single integer, number of patients with available outcome data
at the last possible adaptive analysis (defaults to |
look_after_every |
single integer, specified together with |
randomised_at_looks |
vector of increasing integers or |
control |
single character string, name of one of the |
control_prob_fixed |
if a common |
inferiority |
single numeric value or vector of numeric values of the
same length as the maximum number of possible adaptive analyses, specifying
the probability threshold(s) for inferiority (default is |
superiority |
single numeric value or vector of numeric values of the
same length as the maximum number of possible adaptive analyses, specifying
the probability threshold(s) for superiority (default is |
equivalence_prob |
single numeric value, vector of numeric values of the
same length as the maximum number of possible adaptive analyses or |
equivalence_diff |
single numeric value ( |
equivalence_only_first |
single logical in trial specifications where
|
futility_prob |
single numeric value, vector of numeric values of the
same length as the maximum number of possible adaptive analyses or |
futility_diff |
single numeric value ( |
futility_only_first |
single logical in trial specifications designs
where |
highest_is_best |
single logical, specifies whether larger estimates of
the outcome are favourable or not; defaults to |
soften_power |
either a single numeric value or a numeric vector of
exactly the same length as the maximum number of looks/adaptive analyses.
Values must be between |
cri_width |
single numeric |
n_draws |
single integer, the number of draws from the posterior
distributions for each arm used when running the trial. Defaults to
|
robust |
single logical, if |
description |
character string, default is
|
Value
A trial_spec
object used to run simulations by run_trial()
or
run_trials()
. The output is essentially a list containing the input
values (some combined in a data.frame
called trial_arms
), but its class
signals that these inputs have been validated and inappropriate
combinations and settings have been ruled out. Also contains best_arm
,
holding the arm(s) with the best value(s) in true_ys
. Use str()
to
peruse the actual content of the returned object.
Examples
# Setup a trial specification using a binary, binomially
# distributed, undesirable outcome
binom_trial <- setup_trial_binom(
arms = c("Arm A", "Arm B", "Arm C"),
true_ys = c(0.25, 0.20, 0.30),
# Minimum allocation of 15% in all arms
min_probs = rep(0.15, 3),
data_looks = seq(from = 300, to = 2000, by = 100),
# Stop for equivalence if > 90% probability of
# absolute differences < 5 percentage points
equivalence_prob = 0.9,
equivalence_diff = 0.05,
soften_power = 0.5 # Limit extreme allocation ratios
)
# Print using 3 digits for probabilities
print(binom_trial, prob_digits = 3)
Setup a trial specification using a continuous, normally distributed outcome
Description
Specifies the design of an adaptive trial with a continuous, normally
distributed outcome and validates all inputs. Uses normally distributed
posterior distributions for the mean values in each
trial arm; technically, no priors are used (as using normal-normal
conjugate prior models with extremely wide or uniform priors gives similar
results for these simple, unadjusted estimates). This corresponds to the use
of improper, flat priors, although not explicitly specified as such. Use
calibrate_trial()
to calibrate the trial specification to obtain a specific
value for a certain performance metric (e.g., the Bayesian type 1 error
rate). Use run_trial()
or run_trials()
to conduct single/multiple
simulations of the specified trial, respectively.
Note: add_info
as specified in setup_trial()
is set to the arms and
standard deviations used for trials specified using this function.
Further details: please see setup_trial()
. See setup_trial_binom()
for simplified setup of trials with binomially distributed binary outcomes.
For additional trial specification examples, see the the
Basic examples vignette
(vignette("Basic-examples", package = "adaptr")
) and the
Advanced example vignette
(vignette("Advanced-example", package = "adaptr")
).
Usage
setup_trial_norm(
arms,
true_ys,
sds,
start_probs = NULL,
fixed_probs = NULL,
min_probs = rep(NA, length(arms)),
max_probs = rep(NA, length(arms)),
rescale_probs = NULL,
data_looks = NULL,
max_n = NULL,
look_after_every = NULL,
randomised_at_looks = NULL,
control = NULL,
control_prob_fixed = NULL,
inferiority = 0.01,
superiority = 0.99,
equivalence_prob = NULL,
equivalence_diff = NULL,
equivalence_only_first = NULL,
futility_prob = NULL,
futility_diff = NULL,
futility_only_first = NULL,
highest_is_best = FALSE,
soften_power = 1,
cri_width = 0.95,
n_draws = 5000,
robust = FALSE,
description = "generic normally distributed outcome trial"
)
Arguments
arms |
character vector with unique names for the trial arms. |
true_ys |
numeric vector, simulated means of the outcome in all trial
|
sds |
numeric vector, true standard deviations (must be |
start_probs |
numeric vector, allocation probabilities for each arm at
the beginning of the trial. The default ( |
fixed_probs |
numeric vector, fixed allocation probabilities for each
arm. Must be either a numeric vector with |
min_probs |
numeric vector, lower threshold for adaptive allocation
probabilities; lower probabilities will be rounded up to these values. Must
be |
max_probs |
numeric vector, upper threshold for adaptive allocation
probabilities; higher probabilities will be rounded down to these values.
Must be |
rescale_probs |
|
data_looks |
vector of increasing integers, specifies when to conduct
adaptive analyses (= the total number of patients with available outcome
data at each adaptive analysis). The last number in the vector represents
the final adaptive analysis, i.e., the final analysis where superiority,
inferiority, practical equivalence, or futility can be claimed.
Instead of specifying |
max_n |
single integer, number of patients with available outcome data
at the last possible adaptive analysis (defaults to |
look_after_every |
single integer, specified together with |
randomised_at_looks |
vector of increasing integers or |
control |
single character string, name of one of the |
control_prob_fixed |
if a common |
inferiority |
single numeric value or vector of numeric values of the
same length as the maximum number of possible adaptive analyses, specifying
the probability threshold(s) for inferiority (default is |
superiority |
single numeric value or vector of numeric values of the
same length as the maximum number of possible adaptive analyses, specifying
the probability threshold(s) for superiority (default is |
equivalence_prob |
single numeric value, vector of numeric values of the
same length as the maximum number of possible adaptive analyses or |
equivalence_diff |
single numeric value ( |
equivalence_only_first |
single logical in trial specifications where
|
futility_prob |
single numeric value, vector of numeric values of the
same length as the maximum number of possible adaptive analyses or |
futility_diff |
single numeric value ( |
futility_only_first |
single logical in trial specifications designs
where |
highest_is_best |
single logical, specifies whether larger estimates of
the outcome are favourable or not; defaults to |
soften_power |
either a single numeric value or a numeric vector of
exactly the same length as the maximum number of looks/adaptive analyses.
Values must be between |
cri_width |
single numeric |
n_draws |
single integer, the number of draws from the posterior
distributions for each arm used when running the trial. Defaults to
|
robust |
single logical, if |
description |
character string, default is
|
Details
Because the posteriors used in this type of trial (with a generic,
continuous, normally distributed outcome) are by definition normally
distributed, FALSE
is used as the default value for the robust
argument.
Value
A trial_spec
object used to run simulations by run_trial()
or
run_trials()
. The output is essentially a list containing the input
values (some combined in a data.frame
called trial_arms
), but its class
signals that these inputs have been validated and inappropriate
combinations and settings have been ruled out. Also contains best_arm
,
holding the arm(s) with the best value(s) in true_ys
. Use str()
to
peruse the actual content of the returned object.
Examples
# Setup a trial specification using a continuous, normally distributed, desirable outcome
norm_trial <- setup_trial_norm(
arms = c("Control", "New A", "New B", "New C"),
true_ys = c(15, 20, 14, 13),
sds = c(2, 2.5, 1.9, 1.8), # SDs in each arm
max_n = 500,
look_after_every = 50,
control = "Control", # Common control arm
# Square-root-based, fixed control group allocation ratios
control_prob_fixed = "sqrt-based fixed",
# Desirable outcome
highest_is_best = TRUE,
soften_power = 0.5 # Limit extreme allocation ratios
)
# Print using 3 digits for probabilities
print(norm_trial, prob_digits = 3)
stop() and warning() with call. = FALSE
Description
Used internally. Calls stop0()
or warning()
but enforces call. = FALSE
,
to suppress the call from the error/warning.
Usage
stop0(...)
warning0(...)
Arguments
... |
zero or more objects which can be coerced to character (and which are pasted together with no separator) or a single condition object. |
Summarise distribution
Description
Used internally, to summarise posterior distributions, but the logic does apply to any distribution (thus, the name).
Usage
summarise_dist(x, robust = TRUE, interval_width = 0.95)
Arguments
x |
a numeric vector of posterior draws. |
robust |
single logical. if |
interval_width |
single numeric value ( |
Details
MAD-SDs are scaled to correspond to SDs if distributions are normal,
similarly to the stats::mad()
function; see details regarding calculation
in that function's description.
Value
A numeric vector with four named elements: est
(the median/mean),
err
(the MAD-SD/SD), lo
and hi
(the lower and upper boundaries of the
interval).
Summarise numeric vector
Description
Used internally, to summarise numeric vectors.
Usage
summarise_num(x)
Arguments
x |
a numeric vector. |
Value
A numeric vector with seven named elements: mean
, sd
, median
,
p25
, p75
, p0
, and p100
corresponding to the mean, standard
deviation, median, and 25-/75-/0-/100-percentiles.
Summary of simulated trial results
Description
Summarises simulation results from the run_trials()
function. Uses
extract_results()
and check_performance()
, which may be used directly to
extract key trial results without summarising or to calculate performance
metrics (with uncertainty measures if desired) and return them in a tidy
data.frame
.
Usage
## S3 method for class 'trial_results'
summary(
object,
select_strategy = "control if available",
select_last_arm = FALSE,
select_preferences = NULL,
te_comp = NULL,
raw_ests = FALSE,
final_ests = NULL,
restrict = NULL,
cores = NULL,
...
)
Arguments
object |
|
select_strategy |
single character string. If a trial was not stopped
due to superiority (or had only 1 arm remaining, if
|
select_last_arm |
single logical, defaults to |
select_preferences |
character vector specifying a number of arms used
for selection if one of the |
te_comp |
character string, treatment-effect comparator. Can be either
|
raw_ests |
single logical. If |
final_ests |
single logical. If |
restrict |
single character string or |
cores |
|
... |
additional arguments, not used. |
Value
A "trial_results_summary"
object containing the following values:
-
n_rep
: the number of simulations. -
n_summarised
: as described incheck_performance()
. -
highest_is_best
: as specified insetup_trial()
. -
elapsed_time
: the total simulation time. -
size_mean
,size_sd
,size_median
,size_p25
,size_p75
,size_p0
,size_p100
,sum_ys_mean
,sum_ys_sd
,sum_ys_median
,sum_ys_p25
,sum_ys_p75
,sum_ys_p0
,sum_ys_p100
,ratio_ys_mean
,ratio_ys_sd
,ratio_ys_median
,ratio_ys_p25
,ratio_ys_p75
,ratio_ys_p0
,ratio_ys_p100
,prob_conclusive
,prob_superior
,prob_equivalence
,prob_futility
,prob_max
,prob_select_*
(with*
being either "arm_<name>
for allarm
names ornone
),rmse
,rmse_te
,mae
,mae_te
, andidp
: performance metrics as described incheck_performance()
. Note that allsum_ys_
andratio_ys_
measures use outcome data from all randomised patients, regardless of whether they had outcome data available at the last analysis or not, as described inextract_results()
. -
select_strategy
,select_last_arm
,select_preferences
,te_comp
,raw_ests
,final_ests
,restrict
: as specified above. -
control
: the control arm specified bysetup_trial()
,setup_trial_binom()
orsetup_trial_norm()
;NULL
if no control. -
equivalence_assessed
,futility_assessed
: single logicals, specifies whether the trial design specification includes assessments of equivalence and/or futility. -
base_seed
: as specified inrun_trials()
. -
cri_width
,n_draws
,robust
,description
,add_info
: as specified insetup_trial()
,setup_trial_binom()
orsetup_trial_norm()
.
See Also
extract_results()
, check_performance()
, plot_convergence()
,
plot_metrics_ecdf()
, check_remaining_arms()
.
Examples
# Setup a trial specification
binom_trial <- setup_trial_binom(arms = c("A", "B", "C", "D"),
control = "A",
true_ys = c(0.20, 0.18, 0.22, 0.24),
data_looks = 1:20 * 100)
# Run 10 simulations with a specified random base seed
res <- run_trials(binom_trial, n_rep = 10, base_seed = 12345)
# Summarise simulations - select the control arm if available in trials not
# ending with a superiority decision
res_sum <- summary(res, select_strategy = "control")
# Print summary
print(res_sum, digits = 1)
Update previously saved calibration result
Description
This function updates a previously saved "trial_calibration"
-object created
and saved by calibrate_trial()
using a previous version of adaptr
,
including the embedded trial specification and trial results objects
(internally using the update_saved_trials()
function). This allows the
use of calibration results, including the calibrated trial specification and
the best simulations results from the calibration process, to be used without
errors by this version of the package. The function should be run only once
per saved simulation object and will issue a warning if the object is already
up to date. And overview of the changes made according to the adaptr
package
version used to generate the original object is provided in Details.
Usage
update_saved_calibration(path, version = NULL, compress = TRUE)
Arguments
path |
single character; the path to the saved
|
version |
passed to |
compress |
passed to |
Details
The following changes are made according to the version of adaptr
used to
generate the original "trial_calibration"
object:
-
v1.3.0+
: updates version number of the"trial_calibration"
-object and updates the embedded"trial_results"
-object (saved in$best_sims
, if any) and"trial_spec"
-objects (saved in$input_trial_spec
and$best_trial_spec
) as described inupdate_saved_trials()
.
Value
Invisibly returns the updated "trial_calibration"
-object.
See Also
Update previously saved simulation results
Description
This function updates a previously saved "trial_results"
object created and
saved by run_trials()
using a previous version of adaptr
, allowing the
results from these previous simulations to be post-processed (including
performance metric calculation, printing and plotting) without errors by this
version of the package. The function should be run only once per saved
simulation object and will issue a warning if the object is already up to
date. And overview of the changes made according to the adaptr
package
version used to generate the original object is provided in Details.
NOTE: some values cannot be updated and will be set to NA
(the
posterior estimates from the 'final' analysis conducted after the last
adaptive analysis and including outcome data for all patients), and thus
using both raw_ests = TRUE
and final_ests = TRUE
in the
extract_results()
and summary()
functions will lead to missing values for
some of the values calculated for updated simulation objects.
NOTE: other objects created by the adaptr
package, i.e., trial
specifications generated by
setup_trial()
/ setup_trial_binom()
/ setup_trial_norm()
and single
simulation results from run_trials()
when not included in as part of the
returned output from run_trials()
should be re-created by re-running the
relevant code using the updated version of adaptr
; if manually re-loaded
from previous sessions, they may cause errors and problems with the updated
version of the package.
Usage
update_saved_trials(path, version = NULL, compress = TRUE)
Arguments
path |
single character; the path to the saved |
version |
passed to |
compress |
passed to |
Details
The following changes are made according to the version of adaptr
used to
generate the original "trial_results"
object:
-
v1.2.0+
: updates version number and thereallocate_probs
argument in the embedded trial specification. -
v1.1.1 or earlier
: updates version number and everything related to follow-up and data collection lag (in these versions, therandomised_at_looks
argument in thesetup_trial()
functions did not exist, but for practical purposes was identical to the number of patients with available data at each look) and thereallocate_probs
argument in the embedded trial specification.
Value
Invisibly returns the updated "trial_results"
-object.
See Also
Validate trial specification
Description
Used internally. Validates the inputs common to all trial specifications, as
specified in setup_trial()
, setup_trial_binom()
and setup_trial_norm()
.
Usage
validate_trial(
arms,
true_ys,
start_probs = NULL,
fixed_probs = NULL,
min_probs = rep(NA, length(arms)),
max_probs = rep(NA, length(arms)),
rescale_probs = NULL,
data_looks = NULL,
max_n = NULL,
look_after_every = NULL,
randomised_at_looks = NULL,
control = NULL,
control_prob_fixed = NULL,
inferiority = 0.01,
superiority = 0.99,
equivalence_prob = NULL,
equivalence_diff = NULL,
equivalence_only_first = NULL,
futility_prob = NULL,
futility_diff = NULL,
futility_only_first = NULL,
highest_is_best = FALSE,
soften_power = 1,
cri_width = 0.95,
n_draws = 5000,
robust = FALSE,
description = NULL,
add_info = NULL,
fun_y_gen,
fun_draws,
fun_raw_est
)
Arguments
arms |
character vector with unique names for the trial arms. |
true_ys |
numeric vector specifying true outcomes (e.g., event
probabilities, mean values, etc.) for all trial |
start_probs |
numeric vector, allocation probabilities for each arm at
the beginning of the trial. The default ( |
fixed_probs |
numeric vector, fixed allocation probabilities for each
arm. Must be either a numeric vector with |
min_probs |
numeric vector, lower threshold for adaptive allocation
probabilities; lower probabilities will be rounded up to these values. Must
be |
max_probs |
numeric vector, upper threshold for adaptive allocation
probabilities; higher probabilities will be rounded down to these values.
Must be |
rescale_probs |
|
data_looks |
vector of increasing integers, specifies when to conduct
adaptive analyses (= the total number of patients with available outcome
data at each adaptive analysis). The last number in the vector represents
the final adaptive analysis, i.e., the final analysis where superiority,
inferiority, practical equivalence, or futility can be claimed.
Instead of specifying |
max_n |
single integer, number of patients with available outcome data
at the last possible adaptive analysis (defaults to |
look_after_every |
single integer, specified together with |
randomised_at_looks |
vector of increasing integers or |
control |
single character string, name of one of the |
control_prob_fixed |
if a common |
inferiority |
single numeric value or vector of numeric values of the
same length as the maximum number of possible adaptive analyses, specifying
the probability threshold(s) for inferiority (default is |
superiority |
single numeric value or vector of numeric values of the
same length as the maximum number of possible adaptive analyses, specifying
the probability threshold(s) for superiority (default is |
equivalence_prob |
single numeric value, vector of numeric values of the
same length as the maximum number of possible adaptive analyses or |
equivalence_diff |
single numeric value ( |
equivalence_only_first |
single logical in trial specifications where
|
futility_prob |
single numeric value, vector of numeric values of the
same length as the maximum number of possible adaptive analyses or |
futility_diff |
single numeric value ( |
futility_only_first |
single logical in trial specifications designs
where |
highest_is_best |
single logical, specifies whether larger estimates of
the outcome are favourable or not; defaults to |
soften_power |
either a single numeric value or a numeric vector of
exactly the same length as the maximum number of looks/adaptive analyses.
Values must be between |
cri_width |
single numeric |
n_draws |
single integer, the number of draws from the posterior
distributions for each arm used when running the trial. Defaults to
|
robust |
single logical, if |
description |
optional single character string describing the trial
design, will only be used in print functions if not |
add_info |
optional single string containing additional information
regarding the trial design or specifications, will only be used in print
functions if not |
fun_y_gen |
function, generates outcomes. See |
fun_draws |
function, generates posterior draws. See |
fun_raw_est |
function that takes a numeric vector and returns a
single numeric value, used to calculate a raw summary estimate of the
outcomes in each |
Value
An object of class trial_spec
containing the validated trial
specification.
vapply helpers
Description
Used internally. Helpers for simplifying code invoking vapply().
Usage
vapply_num(X, FUN, ...)
vapply_int(X, FUN, ...)
vapply_str(X, FUN, ...)
vapply_lgl(X, FUN, ...)
Arguments
X |
a vector (atomic or list) or an |
FUN |
the function to be applied to each element of |
... |
optional arguments to |
Verify input is single integer (potentially within range)
Description
Used internally.
Usage
verify_int(x, min_value = -Inf, max_value = Inf, open = "no")
Arguments
x |
value to check. |
min_value , max_value |
single integers (each), lower and upper bounds
between which |
open |
single character, determines whether |
Value
Single logical.
Find the index of value nearest to a target value
Description
Used internally, to find the index of the value in a vector nearest to a target value, possibly in a specific preferred direction.
Usage
which_nearest(values, target, dir)
Arguments
values |
numeric vector, the values considered. |
target |
single numeric value, the target to find the value closest to. |
dir |
single numeric value. If |
Value
Single integer, the index of the value closest to target
according
to dir
.