FixSeqMTP : Tools for Fixed Sequence Multiple Testing Procedures

Description

The FixSeqMTP package provides three categories of functions for generalized/directional fixed sequence mutliple testing procedures:

FWER controlling procedures

FSFWER.arbidept.p.adjust and FSFWER.arbidept.cv

FDR controlling procedures

FSFDR.arbidept.p.adjust and FSFDR.arbidept.cv

FSFDR.indept.p.adjust and FSFDR.indept.cv

mdFWER controlling procedures

FSmdFWER.arbidept.p.adjust and FSmdFWER.arbidept.cv

FSmdFWER.indept.p.adjust and FSmdFWER.indept.cv

Author(s)

Yalin Zhu, Wenge Guo

References

Qiu, Z., Guo, W., & Lynch, G. (2015). On generalized fixed sequence procedures for controlling the FWER. Statistics in medicine, 34(30), 3968-3983.

Lynch, G., Guo, W., Sarkar, S. K., & Finner, H. (2016). The Control of the False Discovery Rate in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1611.03146.

Grandhi, A., Guo, W., & Romano, J. P. (2016). Control of Directional Errors in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1602.02345.

Critical Values for Fixed Sequence FDR Controlling Procedure under Arbitrary Dependence

Description

Given a set of pre-ordered p-values and accuracy for the result, return the corresponding critical values using the generalized fixed sequence FDR controlling procedure under arbitrary dependence (See Theorem 3.1 and 4.1 in Lynch et al. (2016)). The function also provides an option to make decisions given a pre-specified significant level \alpha.

Usage

FSFDR.arbidept.cv(p, k=1, alpha = 0.05, make.decision = TRUE)

Arguments

Value

A numeric vector of the critical values (of the same length as p) if make.decision = FALSE, or a data frame including original p-values, critical values and decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

References

Lynch, G., Guo, W., Sarkar, S. K., & Finner, H. (2016). The Control of the False Discovery Rate in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1611.03146.

See Also

FSFWER.arbidept.cv for fixed sequence FWER controlling procedures.

Examples

## generate a pre-ordered pvalue vector for 50 hypotheses, where 80% are true nulls
set.seed(1234); m <- 50; pi0 <- 0.8; m0 <- m*pi0; m1 <- m-m0
mu <- c(4*0.9^(1:m1), rep(0,m0))
Zstat <- rnorm(n = m, mean = mu)
Pval <- 1-pnorm(Zstat)
## conventional fixed sequence procedure
FSFDR.arbidept.cv(p = Pval, alpha = 0.05)
## generalized fixed sequence procedure allowing stop at 5th acceptance
FSFDR.arbidept.cv(p = Pval, alpha = 0.05, k=5)

Adjusted P-values for Fixed Sequence FDR Controlling Procedure under Arbitrary Dependence

Description

Given a set of pre-ordered p-values and accuracy for the result, returns adjusted p-values using the generalized fixed sequence multiple testing procedures under arbitrary dependence (See Theorem 3.1 and 4.1 in Lynch et al. (2016)). The function also provides an option to make decisions given a pre-specified significant level \alpha.

Usage

FSFDR.arbidept.p.adjust(p, alpha=0.05, k=1, tol = 1e-6, make.decision = TRUE)

Arguments

Details

The generalized fixed sequence FDR controlling procedure stops on the k-th acceptances and automatically accepts the rest of hypotheses, where k is a pre-specified positive integer. When k=1, the generalized procedure becomes conventional one (Theorem 3.1 in Lynch et al. (2016)), which stops testing once one acceptance appears. This method strongly controls FDR under arbitrary dependence.

Value

A numeric vector of the adjusted p-values (of the same length as p) if make.decision = FALSE, or a data frame including original p-values, adjusted p-values and decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

References

Lynch, G., Guo, W., Sarkar, S. K., & Finner, H. (2016). The Control of the False Discovery Rate in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1611.03146.

See Also

FSFWER.arbidept.p.adjust for fixed sequence FWER controlling procedures.

Examples

## generate a pre-ordered pvalue vector for 50 hypotheses, where 80% are true nulls
set.seed(1234); m <- 50; pi0 <- 0.8; m0 <- m*pi0; m1 <- m-m0
mu <- c(4*0.9^(1:m1), rep(0,m0))
Zstat <- rnorm(n = m, mean = mu)
Pval <- 1-pnorm(Zstat)
## conventional fixed sequence procedure
FSFDR.arbidept.p.adjust(p = Pval, alpha = 0.05)
## generalized fixed sequence procedure allowing stop at 5th acceptance
FSFDR.arbidept.p.adjust(p = Pval, alpha = 0.05, k=5)

Critical Values for Fixed Sequence FDR Controlling Procedure under Independence

Description

Given a set of pre-ordered p-values and accuracy for the result, return the corresponding critical values using the generalized fixed sequence FDR controlling procedure under independence for true nulls (See Theorem 3.2 and 4.2 in Lynch et al. (2016)). The function also provides an option to make decisions given a pre-specified significant level \alpha.

Usage

FSFDR.indept.cv(p, k=1, alpha = 0.05, tol = 1e-6, make.decision = TRUE)

Arguments

Value

A numeric vector of the critical values (of the same length as p) if make.decision = FALSE, or a data frame including original p-values, critical values and decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

References

Lynch, G., Guo, W., Sarkar, S. K., & Finner, H. (2016). The Control of the False Discovery Rate in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1611.03146.

See Also

FSFWER.arbidept.cv for fixed sequence FWER controlling procedures.

Examples

## generate a pre-ordered pvalue vector for 50 hypotheses, where 80% are true nulls
set.seed(1234); m <- 50; pi0 <- 0.8; m0 <- m*pi0; m1 <- m-m0
mu <- c(4*0.9^(1:m1), rep(0,m0))
Zstat <- rnorm(n = m, mean = mu)
Pval <- 1-pnorm(Zstat)
## conventional fixed sequence procedure
FSFDR.indept.cv(p = Pval, alpha = 0.05)
## generalized fixed sequence procedure allowing stop at 5th acceptance
FSFDR.indept.cv(p = Pval, alpha = 0.05, k=5)

Adjusted P-values for Fixed Sequence FDR Controlling Procedure under Independence

Description

Given a set of pre-ordered p-values and accuracy for the result, returns adjusted p-values using the generalized fixed sequence multiple testing procedures under independence for true nulls (See Theorem 3.2 and 4.2 in Lynch et al. (2016)). The function also provides an option to make decisions given a pre-specified significant level \alpha.

Usage

FSFDR.indept.p.adjust(p, alpha=0.05, k=1, tol = 1e-6, make.decision = TRUE)

Arguments

Details

The generalized fixed sequence FDR controlling procedure stops on the k-th acceptances and automatically accepts the rest of hypotheses, where k is a pre-specified positive integer. When k=1, the generalized procedure becomes conventional one (Theorem 3.2 in Lynch et al. (2016)), which stops testing once one acceptance appears. This method strongly controls FDR if the true null p-values are mutually independent and are independent of the false null p-values. When k=1, the conventional procedure strongly controls FDR if the p-values are negatively associated on the true null p-values.

Value

A numeric vector of the adjusted p-values (of the same length as p) if make.decision = FALSE, or a data frame including original p-values, adjusted p-values and decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

References

Lynch, G., Guo, W., Sarkar, S. K., & Finner, H. (2016). The Control of the False Discovery Rate in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1611.03146.

See Also

FSFWER.arbidept.p.adjust for fixed sequence FWER controlling procedures.

Examples

## generate a pre-ordered pvalue vector for 50 hypotheses, where 80% are true nulls
set.seed(1234); m <- 50; pi0 <- 0.8; m0 <- m*pi0; m1 <- m-m0
mu <- c(4*0.9^(1:m1), rep(0,m0))
Zstat <- rnorm(n = m, mean = mu)
Pval <- 1-pnorm(Zstat)
## conventional fixed sequence procedure
FSFDR.indept.p.adjust(p = Pval, alpha = 0.05)
## generalized fixed sequence procedure allowing stop at 5th acceptance
FSFDR.indept.p.adjust(p = Pval, alpha = 0.05, k=5)

Critical Values for Fixed Sequence FWER Controlling Procedures under Arbitrary Dependence

Description

Given a set of pre-ordered p-values and accuracy for the result, return the corresponding critical values using one of three generalized fixed sequence FWER controlling procedures. The function also provides an option to make decisions given a pre-specified significant level \alpha.

Usage

FSFWER.arbidept.cv(p, alpha=0.05, beta=0.5, tol = 1e-6,
  method = c("reject","accept","both"), make.decision = TRUE)

Arguments

Details

The critical value calculation methods for Fixed Sequence multiple testing include Procedure A1 only using numbers of rejections ("reject"), Procedure A2 only using numbers of acceptances ("accept") and Procedure A3 using both numbers of rejections and numbers of acceptances ("both"). The three methods strongly control FWER under arbitrary dependence. The constant beta needs to be specified for the Procedure A2 ("accept"), while one can ignore this argument when using other methods.

Value

A numeric vector of the critical values (of the same length as p) if make.decision = FALSE, or a data frame including original p-values, critical values and decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

References

Qiu, Z., Guo, W., & Lynch, G. (2015). On generalized fixed sequence procedures for controlling the FWER. Statistics in medicine, 34(30), 3968-3983.

See Also

FSFDR.arbidept.cv and FSFDR.indept.cv for fixed sequence FDR controlling procedures.

Examples

  ## Clinical trial example in Qiu et al. (2015)
Pval <- c(0.0008, 0.0135, 0.0197, 0.7237, 0.0003, 0.2779, 0.0054, 0.8473)
FSFWER.arbidept.cv(p=Pval, alpha=0.05, method = "reject")
FSFWER.arbidept.cv(p=Pval, alpha=0.05, beta=0.1, method = "accept")
FSFWER.arbidept.cv(p=Pval, alpha=0.05, beta=0.5, method = "accept")
FSFWER.arbidept.cv(p=Pval, alpha=0.05, beta=0.9, method = "accept")
FSFWER.arbidept.cv(p=Pval, alpha=0.05, method = "both")

Adjusted P-values for Fixed Sequence FWER Controlling Procedures under Arbitrary Dependence

Description

Given a set of pre-ordered p-values and accuracy for the result, returns adjusted p-values using one of three generalized fixed sequence FWER controlling procedures. The function also provides an option to make decisions given a pre-specified significant level \alpha.

Usage

FSFWER.arbidept.p.adjust(p, alpha=0.05, beta=0.5, tol = 1e-6,
  method = c("reject","accept","both"), make.decision = TRUE)

Arguments

Details

The adjustment methods for Fixed Sequence multiple testing include Procedure A1 only using numbers of rejections ("reject"), Procedure A2 only using numbers of acceptances ("accept") and Procedure A3 using both numbers of rejections and numbers of acceptances ("both"). The three methods strongly control FWER under arbitrary dependence. The constant beta needs to be specified for the Procedure A2 ("accept"), while one can ignore this argument when using other methods.

Value

A numeric vector of the adjusted p-values (of the same length as p) if make.decision = FALSE, or a data frame including original p-values, adjusted p-values and decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

References

Qiu, Z., Guo, W., & Lynch, G. (2015). On generalized fixed sequence procedures for controlling the FWER. Statistics in medicine, 34(30), 3968-3983.

See Also

FSFDR.arbidept.p.adjust and FSFDR.arbidept.p.adjust for fixed sequence FDR controlling procedures.

Examples

  ## Clinical trial example in Qiu et al. (2015)
Pval <- c(0.0008, 0.0135, 0.0197, 0.7237, 0.0003, 0.2779, 0.0054, 0.8473)
FSFWER.arbidept.p.adjust(p=Pval, alpha=0.05, method = "reject")
FSFWER.arbidept.p.adjust(p=Pval, alpha=0.05, beta=0.1, method = "accept")
FSFWER.arbidept.p.adjust(p=Pval, alpha=0.05, beta=0.5, method = "accept")
FSFWER.arbidept.p.adjust(p=Pval, alpha=0.05, beta=0.9, method = "accept")
FSFWER.arbidept.p.adjust(p=Pval, alpha=0.05, method = "both")

Critical values for Fixed Sequence mdFWER Controlling Procedure under Arbitrary Dependence Along with Directional Decisions Regarding Parameters of Interest

Description

Given a set of pre-ordered test statistics and the corresponding p-values, returns critical values using the directional fixed sequence multiple testing procedures under arbitrary dependence (See Procedure 1 and Theorem 1 in Grandhi et al. (2016)). The function also provides an option to make decisions and determine the sign given a pre-specified significant level \alpha and the test statistics.

Usage

FSmdFWER.arbidept.cv(p, test.stat, alpha=0.05, make.decision = TRUE)

Arguments

Value

A numeric vector of the critical values (of the same length as p) if make.decision = FALSEALSE, or a data frame including original p-values, critical values, test statistics and directional decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

References

Grandhi, A., Guo, W., & Romano, J. P. (2016). Control of Directional Errors in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1602.02345.

See Also

FSmdFWER.indept.cv for fixed sequence mdFWER controlling procedures under independence.

Examples

## Clinical trial example in Grandhi et al. (2016)
Pval <- c(0.0008, 0.0135, 0.0197, 0.7237, 0.0003, 0.2779, 0.0054, 0.8473)
Zstat <- c(3.4434, 2.5085, 2.3642, -0.3543, 3.7651, 1.0900, 2.8340, 0.1930)
FSmdFWER.arbidept.cv(p = Pval, test.stat = Zstat, make.decision = TRUE)

Adjusted P-values for Fixed Sequence mdFWER Controlling Procedure under Arbitrary Dependence Along with Directional Decisions Regarding Parameters of Interest

Description

Given a set of pre-ordered test statistics and the corresponding p-values, returns adjusted p-values using the directional fixed sequence multiple testing procedures under arbitrary dependence (See Procedure 1 and Theorem 1 in Grandhi et al. (2016)). The function also provides an option to make decisions and determine the sign given a pre-specified significant level \alpha and the test statistics.

Usage

FSmdFWER.arbidept.p.adjust(p, test.stat, alpha=0.05, make.decision = TRUE)

Arguments

Value

A numeric vector of the adjusted p-values (of the same length as p) if make.decision = FALSEALSE, or a data frame including original p-values, adjusted p-values, test statistics and directional decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

References

Grandhi, A., Guo, W., & Romano, J. P. (2016). Control of Directional Errors in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1602.02345.

See Also

FSmdFWER.indept.p.adjust for fixed sequence mdFWER controlling procedures under independence.

Examples

## Clinical trial example in Grandhi et al. (2016)
Pval <- c(0.0008, 0.0135, 0.0197, 0.7237, 0.0003, 0.2779, 0.0054, 0.8473)
Zstat <- c(3.4434, 2.5085, 2.3642, -0.3543, 3.7651, 1.0900, 2.8340, 0.1930)
FSmdFWER.arbidept.p.adjust(p = Pval, test.stat = Zstat, make.decision = TRUE)

Critical values for Fixed Sequence mdFWER Controlling Procedure under Independence Along with Directional Decisions Regarding Parameters of Interest

Description

Given a set of pre-ordered test statistics and the corresponding p-values, returns critical values using the directional fixed sequence multiple testing procedures under independence (See Procedure 2 and Theorem 2 in Grandhi et al. (2016)). The function also provides an option to make decisions and determine the sign given a pre-specified significant level \alpha and the test statistics.

Usage

FSmdFWER.indept.cv(p, test.stat, alpha=0.05, make.decision = TRUE)

Arguments

Value

A numeric vector of the critical values (of the same length as p) if make.decision = FALSEALSE, or a data frame including original p-values, critical values, test statistics and directional decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

References

Grandhi, A., Guo, W., & Romano, J. P. (2016). Control of Directional Errors in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1602.02345.

See Also

FSmdFWER.arbidept.cv for fixed sequence mdFWER controlling procedures under arbitrary dependence.

Examples

## Clinical trial example in Grandhi et al. (2016)
Pval <- c(0.0008, 0.0135, 0.0197, 0.7237, 0.0003, 0.2779, 0.0054, 0.8473)
Zstat <- c(3.4434, 2.5085, 2.3642, -0.3543, 3.7651, 1.0900, 2.8340, 0.1930)
FSmdFWER.indept.cv(p = Pval, test.stat = Zstat, make.decision = TRUE)

Adjusted P-values for Fixed Sequence mdFWER Controlling Procedure under Independence Along with Directional Decisions Regarding Parameters of Interest

Description

Given a set of pre-ordered test statistics and the corresponding p-values, returns adjusted p-values using the directional fixed sequence multiple testing procedures under independence (See Procedure 2 and Theorem 2 in Grandhi et al. (2016)). The function also provides an option to make decisions and determine the sign given a pre-specified significant level \alpha and the test statistics.

Usage

FSmdFWER.indept.p.adjust(p, test.stat, alpha=0.05, make.decision = TRUE)

Arguments

Value

A numeric vector of the adjusted p-values (of the same length as p) if make.decision = FALSEALSE, or a data frame including original p-values, adjusted p-values, test statistics and directional decision rules if make.decision = TRUE.

Author(s)

Yalin Zhu

References

Grandhi, A., Guo, W., & Romano, J. P. (2016). Control of Directional Errors in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1602.02345.

See Also

FSmdFWER.indept.p.adjust for fixed sequence mdFWER controlling procedures under independence.

Examples

## Clinical trial example in Grandhi et al. (2015)
Pval <- c(0.0008, 0.0135, 0.0197, 0.7237, 0.0003, 0.2779, 0.0054, 0.8473)
Zstat <- c(3.4434, 2.5085, 2.3642, -0.3543, 3.7651, 1.0900, 2.8340, 0.1930)
FSmdFWER.indept.p.adjust(p = Pval, test.stat = Zstat, make.decision = TRUE)

Bisection algorithm (FDR)

Description

Bisection algorithm to find the solution for the adjusted p-value for FDR controlling g-FSMTPs.

Usage

bisection.FDR(f, a=0, b=1, p, k, j, n = 1000, tol)

Arguments

Value

a solution of the objective function which is between the interval from a to b.

Author(s)

Yalin Zhu

See Also

bisection.FWER

Bisection algorithm (FWER)

Description

Bisection algorithm to find the solution for the adjusted p-value for FWER controlling g-FSMTPs.

Usage

bisection.FWER(f, a=0, b=1, p, beta, j, n = 1000, tol)

Arguments

Value

a solution of the objective function which is between the interval from a to b.

Author(s)

Yalin Zhu

See Also

bisection.FDR

Objective function to be optimized (2)

Description

Objective function to be optimized for the adjusted p-values for FWER controlling g-FSMTP based on the numbers of acceptances only. (See Procedure A2 in Qiu et al. (2015))

Usage

optim.accept.adjp(alpha, p, beta)

Arguments

Value

difference between adjusted p-value and significant level alpha.

Author(s)

Yalin Zhu

References

Qiu, Z., Guo, W., & Lynch, G. (2015). On generalized fixed sequence procedures for controlling the FWER. Statistics in medicine, 34(30), 3968-3983.

Objective function to be optimized (4)

Description

Objective function to be optimized for the adjusted p-values for FDR controlling g-FSMTP under arbitrary dependence. (See Theorem 3.1 and Theorem 4.1 in Lynch et al. (2016))

Usage

optim.arbidept.adjp(alpha, p, k)

Arguments

Value

difference between adjusted p-value and significant level alpha.

Author(s)

Yalin Zhu

References

Lynch, G., Guo, W., Sarkar, S. K., & Finner, H. (2016). The Control of the False Discovery Rate in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1611.03146.

Objective function to be optimized (3)

Description

Objective function to be optimized for the adjusted p-values for FWER controlling g-FSMTP based on the numbers of both rejections and acceptances. (See Procedure A3 in Qiu et al. (2015))

Usage

optim.both.adjp(alpha, p, beta)

Arguments

Value

difference between adjusted p-value and significant level alpha.

Author(s)

Yalin Zhu

References

Qiu, Z., Guo, W., & Lynch, G. (2015). On generalized fixed sequence procedures for controlling the FWER. Statistics in medicine, 34(30), 3968-3983.

Objective function to be optimized (5)

Description

Objective function to be optimized for the adjusted p-values for FDR controlling g-FSMTP under independence. (See Theorem 3.2 and Theorem 4.2 in Lynch et al. (2016))

Usage

optim.indept.adjp(alpha, p, k)

Arguments

Value

difference between adjusted p-value and significant level alpha.

Author(s)

Yalin Zhu

References

Lynch, G., Guo, W., Sarkar, S. K., & Finner, H. (2016). The Control of the False Discovery Rate in Fixed Sequence Multiple Testing. arXiv preprint arXiv:1611.03146.

Objective function to be optimized (1)

Description

Objective function to be optimized for the adjusted p-values for FWER controlling g-FSMTP based on the numbers of rejections only. (See Procedure A1 in Qiu et al. (2015))

Usage

optim.reject.adjp(alpha, p, beta)

Arguments

Value

difference between adjusted p-value and significant level alpha.

Author(s)

Yalin Zhu

References

Qiu, Z., Guo, W., & Lynch, G. (2015). On generalized fixed sequence procedures for controlling the FWER. Statistics in medicine, 34(30), 3968-3983.