Generic Function for Fitting Bayesian Neural Network Models

Description

This is a generic function for fitting Bayesian Neural Network (BNN) models. It dispatches to methods based on the class of the input data.

Usage

bnns(
  formula,
  data,
  L = 1,
  nodes = rep(2, L),
  act_fn = rep(2, L),
  out_act_fn = 1,
  iter = 1000,
  warmup = 200,
  thin = 1,
  chains = 2,
  cores = 2,
  seed = 123,
  prior_weights = NULL,
  prior_bias = NULL,
  prior_sigma = NULL,
  verbose = FALSE,
  refresh = max(iter/10, 1),
  normalize = TRUE,
  ...
)

Arguments

Details

The function serves as a generic interface to different methods of fitting Bayesian Neural Networks. The specific method dispatched depends on the class of the input arguments, allowing for flexibility in the types of inputs supported.

Value

The result of the method dispatched by the class of the input data. Typically, this would be an object of class "bnns" containing the fitted model and associated information.

References

1. Bishop, C.M., 1995. Neural networks for pattern recognition. Oxford university press.

2. Carpenter, B., Gelman, A., Hoffman, M.D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M.A., Guo, J., Li, P. and Riddell, A., 2017. Stan: A probabilistic programming language. Journal of statistical software, 76.

3. Neal, R.M., 2012. Bayesian learning for neural networks (Vol. 118). Springer Science & Business Media.

See Also

bnns.default

Examples

# Example usage with formula interface:
data <- data.frame(x1 = runif(10), x2 = runif(10), y = rnorm(10))
model <- bnns(y ~ -1 + x1 + x2,
  data = data, L = 1, nodes = 2, act_fn = 1,
  iter = 1e1, warmup = 5, chains = 1
)

# See the documentation for bnns.default for more details on the default implementation.

Bayesian Neural Network Model Using Formula(default) Interface

Description

Fits a Bayesian Neural Network (BNN) model using a formula interface. The function parses the formula and data to create the input feature matrix and target vector, then fits the model using bnns.default.

Usage

## Default S3 method:
bnns(
  formula,
  data,
  L = 1,
  nodes = rep(2, L),
  act_fn = rep(2, L),
  out_act_fn = 1,
  iter = 1000,
  warmup = 200,
  thin = 1,
  chains = 2,
  cores = 2,
  seed = 123,
  prior_weights = NULL,
  prior_bias = NULL,
  prior_sigma = NULL,
  verbose = FALSE,
  refresh = max(iter/10, 1),
  normalize = TRUE,
  ...
)

Arguments

Details

The function uses the provided formula and data to generate the design matrix for the predictors and the response vector. It then calls helper function bnns_train to fit the Bayesian Neural Network model.

Value

An object of class "bnns" containing the fitted model and associated information, including:

• fit: The fitted Stan model object.

• data: A list containing the processed training data.

• call: The matched function call.

• formula: The formula used for the model.

Examples

# Example usage:
data <- data.frame(x1 = runif(10), x2 = runif(10), y = rnorm(10))
model <- bnns(y ~ -1 + x1 + x2,
  data = data, L = 1, nodes = 2, act_fn = 3,
  iter = 1e1, warmup = 5, chains = 1
)

Internal function for training the BNN

Description

This function performs the actual fitting of the Bayesian Neural Network. It is called by the exported bnns methods and is not intended for direct use.

Usage

bnns_train(
  train_x,
  train_y,
  L = 1,
  nodes = rep(2, L),
  act_fn = rep(2, L),
  out_act_fn = 1,
  iter = 1000,
  warmup = 200,
  thin = 1,
  chains = 2,
  cores = 2,
  seed = 123,
  prior_weights = NULL,
  prior_bias = NULL,
  prior_sigma = NULL,
  verbose = FALSE,
  refresh = max(iter/10, 1),
  normalize = TRUE,
  ...
)

Arguments

Details

The function uses the generate_stan_code function to dynamically generate Stan code based on the specified number of layers and nodes. Stan is then used to fit the Bayesian Neural Network.

Value

An object of class "bnns" containing the following components:

fit

The fitted Stan model object.

call

The matched call.

data

A list containing the Stan data used in the model.

See Also

stan

Examples

# Example usage:
train_x <- matrix(runif(20), nrow = 10, ncol = 2)
train_y <- rnorm(10)
model <- bnns::bnns_train(train_x, train_y,
  L = 1, nodes = 2, act_fn = 2,
  iter = 1e1, warmup = 5, chains = 1
)

# Access Stan model fit
model$fit

Internal function to generate Stan Code Based on Output Activation Function

Description

This function serves as a wrapper to generate Stan code for Bayesian neural networks tailored to different types of response variables. Based on the specified output activation function (out_act_fn), it delegates the code generation to the appropriate function for continuous, binary, or categorical response models.

Usage

generate_stan_code(num_layers, nodes, out_act_fn = 1)

Arguments

Details

This function dynamically calls one of the following functions based on the value of out_act_fn:

• Continuous response: Calls generate_stan_code_cont.

• Binary response: Calls generate_stan_code_bin.

• Categorical response: Calls generate_stan_code_cat.

If an unsupported value is provided for out_act_fn, the function throws an error. The generated Stan code is adapted for the response type, including appropriate likelihood functions and transformations.

Value

A character string containing the Stan code for the specified Bayesian neural network model. The Stan model includes data, parameters, transformed parameters, and model blocks, adjusted based on the specified response type.

See Also

• generate_stan_code_cont: For continuous response models.

• generate_stan_code_bin: For binary response models.

• generate_stan_code_cat: For categorical response models.

Examples

# Generate Stan code for a continuous response model
stan_code <- generate_stan_code(num_layers = 2, nodes = c(10, 5), out_act_fn = 1)
cat(stan_code)

# Generate Stan code for a binary response model
stan_code <- generate_stan_code(num_layers = 2, nodes = c(10, 5), out_act_fn = 2)
cat(stan_code)

# Generate Stan code for a categorical response model
stan_code <- generate_stan_code(num_layers = 2, nodes = c(10, 5), out_act_fn = 3)
cat(stan_code)

Internal function to generate Stan Code for Binary Response Models

Description

This function generates Stan code for a Bayesian neural network model designed to predict binary response variables. The Stan code is dynamically constructed based on the specified number of hidden layers and nodes per layer. It supports various activation functions for the hidden layers, including tanh, sigmoid, softplus and relu. The model uses a Bernoulli likelihood for binary outcomes.

Usage

generate_stan_code_bin(num_layers, nodes)

Arguments

Details

The generated Stan code models a binary response variable using a neural network. The hidden layers apply the specified activation functions, while the output layer applies the logistic function to predict the probability of the binary outcome.

• For one hidden layer: The function simplifies the Stan code structure.

• For multiple hidden layers: The code dynamically includes additional layers based on the input arguments.

Supported activation functions for the hidden layers:

• 1: Tanh

• 2: Sigmoid

• 3: Softplus

• 4: ReLU

• 5: linear

The output layer uses a logistic transformation (inv_logit) to constrain predictions between 0 and 1, which aligns with the Bernoulli likelihood.

Value

A character string containing the Stan code for the specified Bayesian neural network model. The Stan model includes data, parameters, transformed parameters, and model blocks. The code is adjusted based on whether the network has one or multiple hidden layers.

Examples

# Generate Stan code for a single hidden layer with 10 nodes
stan_code <- generate_stan_code_bin(1, c(10))
cat(stan_code)

# Generate Stan code for two hidden layers with 8 and 4 nodes
stan_code <- generate_stan_code_bin(2, c(8, 4))
cat(stan_code)

Internal function to generate Stan Code for Neural Networks with Categorical Response

Description

This function generates Stan code for modeling a categorical response using neural networks with multiple layers. The generated code supports customizable activation functions for each layer and softmax-based prediction for the categorical output.

Usage

generate_stan_code_cat(num_layers, nodes)

Arguments

Details

The Stan code includes the following components:

• Data Block: Defines inputs, response variable, layer configurations, and activation functions.

• Parameters Block: Declares weights and biases for all layers and the output layer.

• Transformed Parameters Block: Computes intermediate outputs (z and a) for each layer and calculates the final predictions (y_hat) using the softmax function.

• Model Block: Specifies priors for parameters and models the categorical response using categorical_logit.

Supported activation functions for the hidden layers:

• 1: Tanh

• 2: Sigmoid

• 3: Softplus

• 4: ReLU

• 5: linear

The categorical response (y) is assumed to take integer values from 1 to K, where K is the total number of categories.

Value

A string containing the Stan code for the specified neural network architecture and categorical response model.

See Also

generate_stan_code_bin(), generate_stan_code_cont()

Examples

# Generate Stan code for a neural network with 3 layers
num_layers <- 3
nodes <- c(10, 8, 6) # 10 nodes in the first layer, 8 in the second, 6 in the third
stan_code <- generate_stan_code_cat(num_layers, nodes)
cat(stan_code)

Internal function to generate Stan Code for Continuous Response Models

Description

This function generates Stan code for a Bayesian neural network model designed to predict continuous response variables. The Stan code is dynamically constructed based on the specified number of hidden layers and nodes per layer. It supports various activation functions for the hidden layers, including tanh, sigmoid, softplus and relu.

Usage

generate_stan_code_cont(num_layers, nodes)

Arguments

Details

The generated Stan code models a continuous response variable using a neural network. The hidden layers apply the specified activation functions, while the output layer performs a linear transformation to predict the response. The likelihood assumes normally distributed residuals.

• For one hidden layer: The function simplifies the Stan code structure.

• For multiple hidden layers: The code dynamically includes additional layers based on the input arguments.

Supported activation functions for the hidden layers:

• 1: Tanh

• 2: Sigmoid

• 3: Softplus

• 4: ReLU

• 5: linear

Value

A character string containing the Stan code for the specified Bayesian neural network model. The Stan model includes data, parameters, transformed parameters, and model blocks. The code is adjusted based on whether the network has one or multiple hidden layers.

Examples

# Generate Stan code for a single hidden layer with 10 nodes
stan_code <- generate_stan_code_cont(1, c(10))
cat(stan_code)

# Generate Stan code for two hidden layers with 8 and 4 nodes
stan_code <- generate_stan_code_cont(2, c(8, 4))
cat(stan_code)

Measure Performance for Binary Classification Models

Description

Evaluates the performance of a binary classification model using a confusion matrix and accuracy.

Usage

measure_bin(obs, pred, cut = 0.5)

Arguments

Value

A list containing:

conf_mat

A confusion matrix comparing observed and predicted class labels.

accuracy

The proportion of correct predictions.

ROC

ROC generated using pROC::roc

AUC

Area under the ROC curve.

Examples

obs <- c(1, 0, 1, 1, 0)
pred <- c(0.9, 0.4, 0.8, 0.7, 0.3)
cut <- 0.5
measure_bin(obs, pred, cut)
# Returns: list(conf_mat = <confusion matrix>, accuracy = 1, ROC = <ROC>, AUC = 1)

Measure Performance for Multi-Class Classification Models

Description

Evaluates the performance of a multi-class classification model using log loss and multiclass AUC.

Usage

measure_cat(obs, pred)

Arguments

Details

The log loss is calculated as:

-\frac{1}{N} \sum_{i=1}^N \sum_{c=1}^C y_{ic} \log(p_{ic})

where y_{ic} is 1 if observation i belongs to class c, and p_{ic} is the predicted probability for that class.

The AUC is computed using the pROC::multiclass.roc function, which provides an overall measure of model performance for multiclass classification.

Value

A list containing:

log_loss

The negative log-likelihood averaged across observations.

ROC

ROC generated using pROC::roc

AUC

The multiclass Area Under the Curve (AUC) as computed by pROC::multiclass.roc.

Examples

library(pROC)
obs <- factor(c("A", "B", "C"), levels = LETTERS[1:3])
pred <- matrix(
  c(
    0.8, 0.1, 0.1,
    0.2, 0.6, 0.2,
    0.7, 0.2, 0.1
  ),
  nrow = 3, byrow = TRUE
)
measure_cat(obs, pred)
# Returns: list(log_loss = 1.012185, ROC = <ROC>, AUC = 0.75)

Measure Performance for Continuous Response Models

Description

Evaluates the performance of a continuous response model using RMSE and MAE.

Usage

measure_cont(obs, pred)

Arguments

Value

A list containing:

rmse

Root Mean Squared Error.

mae

Mean Absolute Error.

Examples

obs <- c(3.2, 4.1, 5.6)
pred <- c(3.0, 4.3, 5.5)
measure_cont(obs, pred)
# Returns: list(rmse = 0.1732051, mae = 0.1666667)

Predict Method for "bnns" Objects

Description

Generates predictions from a fitted Bayesian Neural Network (BNN) model.

Usage

## S3 method for class 'bnns'
predict(object, newdata = NULL, ...)

Arguments

Details

This function uses the posterior distribution from the Stan model in the bnns object to compute predictions for the provided input data.

Value

A matrix/array of predicted values(regression)/probabilities(classification) where first dimension corresponds to the rows of newdata or the training data if newdata is NULL. Second dimension corresponds to the number of posterior samples. In case of out_act_fn = 3, the third dimension corresponds to the class.

See Also

bnns, print.bnns

Examples

# Example usage:
data <- data.frame(x1 = runif(10), x2 = runif(10), y = rnorm(10))
model <- bnns(y ~ -1 + x1 + x2,
  data = data, L = 1, nodes = 2, act_fn = 2,
  iter = 1e1, warmup = 5, chains = 1
)
new_data <- data.frame(x1 = runif(5), x2 = runif(5))
predictions <- predict(model, newdata = new_data)
print(predictions)

Print Method for "bnns" Objects

Description

Displays a summary of a fitted Bayesian Neural Network (BNN) model, including the function call and the Stan fit details.

Usage

## S3 method for class 'bnns'
print(x, ...)

Arguments

Value

The function is called for its side effects and does not return a value. It prints the following:

• The function call used to generate the "bnns" object.

• A summary of the Stan fit object stored in x$fit.

See Also

bnns, summary.bnns

Examples

# Example usage:
data <- data.frame(x1 = runif(10), x2 = runif(10), y = rnorm(10))
model <- bnns(y ~ -1 + x1 + x2,
  data = data, L = 1, nodes = 2, act_fn = 2,
  iter = 1e1, warmup = 5, chains = 1
)
print(model)

relu transformation

Description

relu transformation

Usage

relu(x)

Arguments

Value

A numeric vector or matrix after relu transformation.

Examples

relu(matrix(1:4, , nrow = 2))

sigmoid transformation

Description

sigmoid transformation

Usage

sigmoid(x)

Arguments

Value

A numeric vector or matrix after sigmoid transformation.

Examples

sigmoid(matrix(1:4, nrow = 2))

Apply Softmax Function to a 3D Array

Description

This function applies the softmax transformation along the third dimension of a 3D array. The softmax function converts raw scores into probabilities such that they sum to 1 for each slice along the third dimension.

Usage

softmax_3d(x)

Arguments

Details

The softmax transformation is computed as:

\text{softmax}(x_{ijk}) = \frac{\exp(x_{ijk})}{\sum_{l} \exp(x_{ijl})}

This is applied for each pair of indices ⁠(i, j)⁠ across the third dimension (k).

The function processes the input array slice-by-slice for the first two dimensions ⁠(i, j)⁠, normalizing the values along the third dimension (k) for each slice.

Value

A 3D array of the same dimensions as x, where the values along the third dimension are transformed using the softmax function.

Examples

# Example: Apply softmax to a 3D array
x <- array(runif(24), dim = c(2, 3, 4)) # Random 3D array (2x3x4)
softmax_result <- softmax_3d(x)

softplus transformation

Description

softplus transformation

Usage

softplus(x)

Arguments

Value

A numeric vector or matrix after softplus transformation.

Examples

softplus(matrix(1:4, nrow = 2))

Summary of a Bayesian Neural Network (BNN) Model

Description

Provides a comprehensive summary of a fitted Bayesian Neural Network (BNN) model, including details about the model call, data, network architecture, posterior distributions, and model fitting information.

Usage

## S3 method for class 'bnns'
summary(object, ...)

Arguments

Details

The function prints the following information:

• Call: The original function call used to fit the model.

• Data Summary: Number of observations and features in the training data.

• Network Architecture: Structure of the BNN including the number of hidden layers, nodes per layer, and activation functions.

• Posterior Summary: Summarized posterior distributions of key parameters (e.g., weights, biases, and noise parameter).

• Model Fit Information: Bayesian sampling details, including the number of iterations, warmup period, thinning, and chains.

• Notes: Remarks and warnings, such as checks for convergence diagnostics.

Value

A list (returned invisibly) containing the following elements:

• "Number of observations": The number of observations in the training data.

• "Number of features": The number of features in the training data.

• "Number of hidden layers": The number of hidden layers in the neural network.

• "Nodes per layer": A comma-separated string representing the number of nodes in each hidden layer.

• "Activation functions": A comma-separated string representing the activation functions used in each hidden layer.

• "Output activation function": The activation function used in the output layer.

• "Stanfit Summary": A summary of the Stan model, including key parameter posterior distributions.

• "Iterations": The total number of iterations used for sampling in the Bayesian model.

• "Warmup": The number of iterations used as warmup in the Bayesian model.

• "Thinning": The thinning interval used in the Bayesian model.

• "Chains": The number of Markov chains used in the Bayesian model.

• "Performance": Predictive performance metrics, which vary based on the output activation function.

The function also prints the summary to the console.

See Also

bnns, print.bnns

Examples

# Fit a Bayesian Neural Network
data <- data.frame(x1 = runif(10), x2 = runif(10), y = rnorm(10))
model <- bnns(y ~ -1 + x1 + x2,
  data = data, L = 1, nodes = 2, act_fn = 2,
  iter = 1e1, warmup = 5, chains = 1
)

# Get a summary of the model
summary(model)