Generating Samples of Gene Expression Data with Variational Autoencoders
Dongmin Jung
December 15, 2021
1 Introduction
A fundamental problem in biomedical research is the low number of observations, mostly due to a lack of available biosamples, prohibitive costs, or ethical reasons. By augmenting a few real observations with artificially generated samples, their analysis could lead to more robust and higher reproducibility. One possible solution to the problem is the use of generative models, which are statistical models of data that attempt to capture the entire probability distribution from the observations. Using the variational autoencoder (VAE), a well-known deep generative model, this package is aimed to generate samples with gene expression data, especially for single-cell RNA-seq data. Furthermore, the VAE can use conditioning to produce specific cell types or subpopulations. The conditional VAE (CVAE) allows us to create targeted samples rather than completely random ones.
Autoencoders are an unsupervised neural networks that perform data compression from multidimensional to a preferred dimensionality. They reconstruct input data using the hidden layer weights calculated by encoding. The basic idea of an autoencoder is to have an output layer with the same dimensionality as the inputs. The idea is to try to reconstruct each dimension exactly by passing it through the network. It is common but not necessary for an autoencoder to have a symmetric architecture between the input and output. The number of units in the middle layer is typically fewer than that in the input or output. After training an autoencoder, it is not necessary to use both the encoder and decoder portions. For example, when using the approach for dimensionality reduction, one can use the encoder portion in order to create the reduced representations of the data. The reconstructions of the decoder might not be required at all. As a result, an autoencoder is capable of performing dimension reduction. The objective function of this neural network encompasses reconstruction loss. The loss function uses the sum of squared differences between the input and the output in order to force the output to be as similar as possible to the input. Also, the cross-entropy can used as a loss function for quantifying the difference between two probability distributions.
Another interesting application of the autoencoder is one in which we use only the decoder portion of the network. Variational autoencoders are based on Bayesian inference in which the compressed representation follows probability distribution. This constraint differentiates the VAE from standard autoencoder. The VAE can generate new data while conventional autoencoders fail. For example, one might add a term to the loss function to enforce the fact that the hidden variables are drawn from a Gaussian distribution. Then, one might repeatedly draw samples from this Gaussian distribution and use only the decoder portion of the network in order to generate samples of the original data. In this autoencoder, bottleneck vector (latent vector) is replaced by two vectors, namely, mean vector and standard deviation vector. The overall loss function \(J = L + \lambda R\) of the VAE is expressed as a weighted sum of the reconstruction loss \(L\) and the regularization loss \(R\), where \(\lambda > 0\) is the regularization parameter. The term “variational” comes from the close relationship between the regularization and the variational inference method in statistics. One can use a variety of choices for the reconstruction error, and we will use the binary cross-entropy loss between the input and output. The regularization loss is simply the Kullback-Leibler divergence measure of the conditional distributions of the hidden representations of particular points with respect to the standard multivariate Gaussian distribution. Small values of \(\lambda\) will favor exact reconstruction, and the approach will behave like a traditional autoencoder.
One can apply conditioning to variational autoencoders in order to obtain some interesting results. The basic idea in the conditional variational autoencoder is to add an additional conditional input. From an implementation perspective, we can encode category information as a one-hot representation, indicating to the model which class is at the input. One can use an autoencoder for embedding multimodal data in a joint latent space. Multimodal data is essentially data in which the input features are heterogeneous. In addition, by separating the samples into different classes, the data points within the same category become more similar, enhancing the modeling capacity and sample quality of the CVAE.
2 Example
2.1 VAE
Consider artificial data. The data consist of 1000 genes and three groups of 100 samples. Each group has 100 differentially expressed genes.
if (keras::is_keras_available() & reticulate::py_available()) {
library(VAExprs)
### simulate differentially expressed genes
set.seed(1)
g <- 3
n <- 100
m <- 1000
mu <- 5
sigma <- 5
mat <- matrix(rnorm(n*m*g, mu, sigma), m, n*g)
rownames(mat) <- paste0("gene", seq_len(m))
colnames(mat) <- paste0("cell", seq_len(n*g))
group <- factor(sapply(seq_len(g), function(x) {
rep(paste0("group", x), n)
}))
names(group) <- colnames(mat)
mu_upreg <- 6
sigma_upreg <- 10
deg <- 100
for (i in seq_len(g)) {
mat[(deg*(i-1) + 1):(deg*i), group == paste0("group", i)] <-
mat[1:deg, group==paste0("group", i)] + rnorm(deg, mu_upreg, sigma_upreg)
}
# positive expression only
mat[mat < 0] <- 0
x_train <- as.matrix(t(mat))
# heatmap
heatmap(mat, Rowv = NA, Colv = NA,
col = colorRampPalette(c('green', 'red'))(100),
scale = "none")
}## Loading required package: keras## Loading required package: mclust## Package 'mclust' version 6.0.0
## Type 'citation("mclust")' for citing this R package in publications.
The VAE model can be built by using the function “fit_vae” with gene expression data and the cell annotation from the object “sce”. The overall loss function of the VAE is expressed as a weighted sum of the reconstruction loss and the regularization loss. The reconstruction loss is the binary cross-entropy loss between the input and output and the regularization loss is simply the Kullback-Leibler divergence measure. Note that the same dataset is used for training and validation.
if (keras::is_keras_available() & reticulate::py_available()) {
# model parameters
batch_size <- 32
original_dim <- 1000
intermediate_dim <- 512
epochs <- 100
# VAE
vae_result <- fit_vae(x_train = x_train, x_val = x_train,
encoder_layers = list(layer_input(shape = c(original_dim)),
layer_dense(units = intermediate_dim,
activation = "relu")),
decoder_layers = list(layer_dense(units = intermediate_dim,
activation = "relu"),
layer_dense(units = original_dim,
activation = "sigmoid")),
epochs = epochs, batch_size = batch_size,
use_generator = FALSE,
callbacks = keras::callback_early_stopping(
monitor = "val_loss",
patience = 10,
restore_best_weights = TRUE))
}## normalizing...## training...## Train on 300 samples, validate on 300 samples
## Epoch 1/100
## 300/300 - 1s - loss: 666.8692 - val_loss: 631.6392 - 608ms/epoch - 2ms/sample
## Epoch 2/100
## 300/300 - 0s - loss: 615.7798 - val_loss: 606.5409 - 136ms/epoch - 453us/sample
## Epoch 3/100
## 300/300 - 0s - loss: 605.7056 - val_loss: 602.0835 - 86ms/epoch - 287us/sample
## Epoch 4/100
## 300/300 - 0s - loss: 601.8386 - val_loss: 600.2693 - 90ms/epoch - 301us/sample
## Epoch 5/100
## 300/300 - 0s - loss: 599.9330 - val_loss: 599.0478 - 119ms/epoch - 397us/sample
## Epoch 6/100
## 300/300 - 0s - loss: 598.7737 - val_loss: 597.4734 - 122ms/epoch - 405us/sample
## Epoch 7/100
## 300/300 - 0s - loss: 596.6468 - val_loss: 595.0594 - 112ms/epoch - 372us/sample
## Epoch 8/100
## 300/300 - 0s - loss: 594.3279 - val_loss: 591.3529 - 108ms/epoch - 361us/sample
## Epoch 9/100
## 300/300 - 0s - loss: 591.4283 - val_loss: 589.4288 - 109ms/epoch - 365us/sample
## Epoch 10/100
## 300/300 - 0s - loss: 589.1440 - val_loss: 588.0930 - 111ms/epoch - 371us/sample
## Epoch 11/100
## 300/300 - 0s - loss: 587.3600 - val_loss: 585.7337 - 106ms/epoch - 352us/sample
## Epoch 12/100
## 300/300 - 0s - loss: 585.4524 - val_loss: 583.9483 - 107ms/epoch - 355us/sample
## Epoch 13/100
## 300/300 - 0s - loss: 583.9553 - val_loss: 583.1240 - 107ms/epoch - 356us/sample
## Epoch 14/100
## 300/300 - 0s - loss: 582.1324 - val_loss: 582.2618 - 104ms/epoch - 348us/sample
## Epoch 15/100
## 300/300 - 0s - loss: 581.6183 - val_loss: 580.7430 - 103ms/epoch - 344us/sample
## Epoch 16/100
## 300/300 - 0s - loss: 581.1014 - val_loss: 580.0137 - 85ms/epoch - 282us/sample
## Epoch 17/100
## 300/300 - 0s - loss: 580.2037 - val_loss: 580.4147 - 92ms/epoch - 305us/sample
## Epoch 18/100
## 300/300 - 0s - loss: 580.5753 - val_loss: 581.0384 - 97ms/epoch - 323us/sample
## Epoch 19/100
## 300/300 - 0s - loss: 581.3382 - val_loss: 581.3032 - 102ms/epoch - 339us/sample
## Epoch 20/100
## 300/300 - 0s - loss: 581.1837 - val_loss: 582.0475 - 97ms/epoch - 324us/sample
## Epoch 21/100
## 300/300 - 0s - loss: 580.9458 - val_loss: 579.8857 - 99ms/epoch - 330us/sample
## Epoch 22/100
## 300/300 - 0s - loss: 580.3290 - val_loss: 579.4973 - 101ms/epoch - 337us/sample
## Epoch 23/100
## 300/300 - 0s - loss: 579.8987 - val_loss: 579.7849 - 102ms/epoch - 339us/sample
## Epoch 24/100
## 300/300 - 0s - loss: 579.6116 - val_loss: 579.1454 - 106ms/epoch - 354us/sample
## Epoch 25/100
## 300/300 - 0s - loss: 579.6967 - val_loss: 579.4654 - 95ms/epoch - 315us/sample
## Epoch 26/100
## 300/300 - 0s - loss: 580.2263 - val_loss: 579.3260 - 85ms/epoch - 284us/sample
## Epoch 27/100
## 300/300 - 0s - loss: 579.2452 - val_loss: 578.7335 - 104ms/epoch - 346us/sample
## Epoch 28/100
## 300/300 - 0s - loss: 579.1614 - val_loss: 578.8248 - 108ms/epoch - 360us/sample
## Epoch 29/100
## 300/300 - 0s - loss: 579.4428 - val_loss: 579.0937 - 110ms/epoch - 365us/sample
## Epoch 30/100
## 300/300 - 0s - loss: 579.2597 - val_loss: 578.6889 - 112ms/epoch - 374us/sample
## Epoch 31/100
## 300/300 - 0s - loss: 579.0898 - val_loss: 578.5281 - 112ms/epoch - 372us/sample
## Epoch 32/100
## 300/300 - 0s - loss: 579.2763 - val_loss: 579.1401 - 111ms/epoch - 371us/sample
## Epoch 33/100
## 300/300 - 0s - loss: 578.8994 - val_loss: 578.3897 - 121ms/epoch - 403us/sample
## Epoch 34/100
## 300/300 - 0s - loss: 579.3108 - val_loss: 578.4510 - 106ms/epoch - 353us/sample
## Epoch 35/100
## 300/300 - 0s - loss: 578.7910 - val_loss: 578.4729 - 98ms/epoch - 327us/sample
## Epoch 36/100
## 300/300 - 0s - loss: 578.6545 - val_loss: 578.2725 - 106ms/epoch - 353us/sample
## Epoch 37/100
## 300/300 - 0s - loss: 579.4405 - val_loss: 579.9574 - 98ms/epoch - 325us/sample
## Epoch 38/100
## 300/300 - 0s - loss: 578.9894 - val_loss: 578.7063 - 94ms/epoch - 312us/sample
## Epoch 39/100
## 300/300 - 0s - loss: 578.6751 - val_loss: 578.2703 - 97ms/epoch - 323us/sample
## Epoch 40/100
## 300/300 - 0s - loss: 578.3203 - val_loss: 578.1939 - 96ms/epoch - 321us/sample
## Epoch 41/100
## 300/300 - 0s - loss: 579.4728 - val_loss: 578.8683 - 94ms/epoch - 313us/sample
## Epoch 42/100
## 300/300 - 0s - loss: 578.7963 - val_loss: 578.2378 - 98ms/epoch - 327us/sample
## Epoch 43/100
## 300/300 - 0s - loss: 578.6837 - val_loss: 579.9054 - 94ms/epoch - 313us/sample
## Epoch 44/100
## 300/300 - 0s - loss: 579.1590 - val_loss: 579.2203 - 88ms/epoch - 294us/sample
## Epoch 45/100
## 300/300 - 0s - loss: 579.9704 - val_loss: 581.3318 - 99ms/epoch - 330us/sample
## Epoch 46/100
## 300/300 - 0s - loss: 579.7921 - val_loss: 579.5184 - 106ms/epoch - 353us/sample
## Epoch 47/100
## 300/300 - 0s - loss: 579.4997 - val_loss: 578.1942 - 106ms/epoch - 354us/sample
## Epoch 48/100
## 300/300 - 0s - loss: 578.8897 - val_loss: 579.3456 - 102ms/epoch - 341us/sample
## Epoch 49/100
## 300/300 - 0s - loss: 578.6443 - val_loss: 579.1740 - 105ms/epoch - 350us/sample
## Epoch 50/100
## 300/300 - 0s - loss: 578.4458 - val_loss: 578.2447 - 174ms/epoch - 580us/sampleThe function “plot_vae” draws the plot for model architecture.
if (keras::is_keras_available() & reticulate::py_available()) {
# model architecture
plot_vae(vae_result$model)
}The function “gen_exprs” can generate samples with expression data by using the trained model.
if (keras::is_keras_available() & reticulate::py_available()) {
# sample generation
set.seed(1)
gen_sample_result <- gen_exprs(vae_result, num_samples = 100)
# heatmap
heatmap(cbind(t(x_train), t(gen_sample_result$x_gen)),
col = colorRampPalette(c('green', 'red'))(100),
Rowv=NA)
}## generating...## post-processing...
The function “plot_aug” uses reduced dimension plots for augmented data visualization.
if (keras::is_keras_available() & reticulate::py_available()) {
# plot for augmented data
plot_aug(gen_sample_result, "PCA")
}
2.2 CVAE
The “yan” data set is single-cell RNA sequencing data with 20214 genes and 90 cells from human preimplantation embryos and embryonic stem cells at different passages. The rows in the dataset correspond to genes and columns correspond to cells. The “SingleCellExperiment” class can be used to store and manipulate single-cell genomics data. It extends the “RangedSummarizedExperiment” class and follows similar conventions. The object “sce” can be created by the data “yan” with cell type annotation “ann”.
if (keras::is_keras_available() & reticulate::py_available()) {
library(VAExprs)
library(SC3)
library(SingleCellExperiment)
# create a SingleCellExperiment object
sce <- SingleCellExperiment::SingleCellExperiment(
assays = list(counts = as.matrix(yan)),
colData = ann
)
# define feature names in feature_symbol column
rowData(sce)$feature_symbol <- rownames(sce)
# remove features with duplicated names
sce <- sce[!duplicated(rowData(sce)$feature_symbol), ]
# remove genes that are not expressed in any samples
sce <- sce[which(rowMeans(assay(sce)) > 0),]
dim(assay(sce))
# model parameters
batch_size <- 32
original_dim <- 19595
intermediate_dim <- 256
epochs <- 100
# model
cvae_result <- fit_vae(object = sce,
encoder_layers = list(layer_input(shape = c(original_dim)),
layer_dense(units = intermediate_dim,
activation = "relu")),
decoder_layers = list(layer_dense(units = intermediate_dim,
activation = "relu"),
layer_dense(units = original_dim,
activation = "sigmoid")),
epochs = epochs, batch_size = batch_size,
use_generator = TRUE,
callbacks = keras::callback_early_stopping(
monitor = "loss",
patience = 20,
restore_best_weights = TRUE))
# model architecture
plot_vae(cvae_result$model)
}## pre-processing...## normalizing...## training...## Epoch 1/100
## 3/3 - 3s - loss: 13122.8688 - 3s/epoch - 847ms/step
## Epoch 2/100
## 3/3 - 0s - loss: 10984.3132 - 262ms/epoch - 87ms/step
## Epoch 3/100
## 3/3 - 0s - loss: 9785.6410 - 211ms/epoch - 70ms/step
## Epoch 4/100
## 3/3 - 0s - loss: 9432.4365 - 209ms/epoch - 70ms/step
## Epoch 5/100
## 3/3 - 0s - loss: 8823.0264 - 233ms/epoch - 78ms/step
## Epoch 6/100
## 3/3 - 0s - loss: 8473.4170 - 228ms/epoch - 76ms/step
## Epoch 7/100
## 3/3 - 0s - loss: 8493.4482 - 202ms/epoch - 67ms/step
## Epoch 8/100
## 3/3 - 0s - loss: 8405.3213 - 212ms/epoch - 71ms/step
## Epoch 9/100
## 3/3 - 1s - loss: 8358.8669 - 1s/epoch - 391ms/step
## Epoch 10/100
## 3/3 - 0s - loss: 8230.8200 - 250ms/epoch - 83ms/step
## Epoch 11/100
## 3/3 - 0s - loss: 8076.1606 - 239ms/epoch - 80ms/step
## Epoch 12/100
## 3/3 - 0s - loss: 7948.1844 - 212ms/epoch - 71ms/step
## Epoch 13/100
## 3/3 - 0s - loss: 8041.3431 - 230ms/epoch - 77ms/step
## Epoch 14/100
## 3/3 - 0s - loss: 7965.7287 - 194ms/epoch - 65ms/step
## Epoch 15/100
## 3/3 - 0s - loss: 7930.0493 - 235ms/epoch - 78ms/step
## Epoch 16/100
## 3/3 - 0s - loss: 8090.8374 - 186ms/epoch - 62ms/step
## Epoch 17/100
## 3/3 - 0s - loss: 7936.9458 - 227ms/epoch - 76ms/step
## Epoch 18/100
## 3/3 - 0s - loss: 7862.8612 - 195ms/epoch - 65ms/step
## Epoch 19/100
## 3/3 - 0s - loss: 7988.3270 - 190ms/epoch - 63ms/step
## Epoch 20/100
## 3/3 - 0s - loss: 7919.2490 - 195ms/epoch - 65ms/step
## Epoch 21/100
## 3/3 - 0s - loss: 7825.9862 - 248ms/epoch - 83ms/step
## Epoch 22/100
## 3/3 - 0s - loss: 8034.5955 - 205ms/epoch - 68ms/step
## Epoch 23/100
## 3/3 - 0s - loss: 7696.9364 - 202ms/epoch - 67ms/step
## Epoch 24/100
## 3/3 - 0s - loss: 7990.6815 - 261ms/epoch - 87ms/step
## Epoch 25/100
## 3/3 - 0s - loss: 7775.3786 - 197ms/epoch - 66ms/step
## Epoch 26/100
## 3/3 - 0s - loss: 7896.3101 - 201ms/epoch - 67ms/step
## Epoch 27/100
## 3/3 - 0s - loss: 7801.4377 - 226ms/epoch - 75ms/step
## Epoch 28/100
## 3/3 - 0s - loss: 7785.1938 - 181ms/epoch - 60ms/step
## Epoch 29/100
## 3/3 - 0s - loss: 7862.7340 - 180ms/epoch - 60ms/step
## Epoch 30/100
## 3/3 - 1s - loss: 7949.6413 - 1s/epoch - 360ms/step
## Epoch 31/100
## 3/3 - 0s - loss: 7913.6917 - 174ms/epoch - 58ms/step
## Epoch 32/100
## 3/3 - 0s - loss: 7590.7404 - 187ms/epoch - 62ms/step
## Epoch 33/100
## 3/3 - 0s - loss: 7641.5249 - 181ms/epoch - 60ms/step
## Epoch 34/100
## 3/3 - 0s - loss: 7862.5508 - 211ms/epoch - 70ms/step
## Epoch 35/100
## 3/3 - 0s - loss: 7643.3433 - 176ms/epoch - 59ms/step
## Epoch 36/100
## 3/3 - 0s - loss: 7783.5820 - 172ms/epoch - 57ms/step
## Epoch 37/100
## 3/3 - 0s - loss: 7750.2274 - 174ms/epoch - 58ms/step
## Epoch 38/100
## 3/3 - 0s - loss: 7670.7352 - 218ms/epoch - 73ms/step
## Epoch 39/100
## 3/3 - 0s - loss: 7686.0210 - 175ms/epoch - 58ms/step
## Epoch 40/100
## 3/3 - 0s - loss: 7661.2762 - 176ms/epoch - 59ms/step
## Epoch 41/100
## 3/3 - 0s - loss: 7737.3252 - 174ms/epoch - 58ms/step
## Epoch 42/100
## 3/3 - 0s - loss: 7592.3491 - 213ms/epoch - 71ms/step
## Epoch 43/100
## 3/3 - 0s - loss: 7717.3524 - 175ms/epoch - 58ms/step
## Epoch 44/100
## 3/3 - 0s - loss: 7799.5942 - 179ms/epoch - 60ms/step
## Epoch 45/100
## 3/3 - 0s - loss: 7587.5928 - 236ms/epoch - 79ms/step
## Epoch 46/100
## 3/3 - 0s - loss: 7670.1410 - 179ms/epoch - 60ms/step
## Epoch 47/100
## 3/3 - 0s - loss: 7649.3981 - 179ms/epoch - 60ms/step
## Epoch 48/100
## 3/3 - 0s - loss: 7691.0939 - 212ms/epoch - 71ms/step
## Epoch 49/100
## 3/3 - 0s - loss: 7722.6826 - 177ms/epoch - 59ms/step
## Epoch 50/100
## 3/3 - 0s - loss: 7636.8942 - 176ms/epoch - 59ms/step
## Epoch 51/100
## 3/3 - 1s - loss: 7519.8094 - 975ms/epoch - 325ms/step
## Epoch 52/100
## 3/3 - 0s - loss: 7584.0046 - 176ms/epoch - 59ms/step
## Epoch 53/100
## 3/3 - 0s - loss: 7647.8070 - 183ms/epoch - 61ms/step
## Epoch 54/100
## 3/3 - 0s - loss: 7614.6276 - 196ms/epoch - 65ms/step
## Epoch 55/100
## 3/3 - 0s - loss: 7706.4045 - 249ms/epoch - 83ms/step
## Epoch 56/100
## 3/3 - 0s - loss: 7612.2952 - 207ms/epoch - 69ms/step
## Epoch 57/100
## 3/3 - 0s - loss: 7683.9886 - 193ms/epoch - 64ms/step
## Epoch 58/100
## 3/3 - 0s - loss: 7682.5438 - 310ms/epoch - 103ms/step
## Epoch 59/100
## 3/3 - 0s - loss: 7726.7248 - 247ms/epoch - 82ms/step
## Epoch 60/100
## 3/3 - 0s - loss: 7591.7581 - 175ms/epoch - 58ms/step
## Epoch 61/100
## 3/3 - 0s - loss: 7571.9910 - 174ms/epoch - 58ms/step
## Epoch 62/100
## 3/3 - 0s - loss: 7690.2210 - 180ms/epoch - 60ms/step
## Epoch 63/100
## 3/3 - 1s - loss: 7727.5555 - 1s/epoch - 354ms/step
## Epoch 64/100
## 3/3 - 0s - loss: 7607.9481 - 173ms/epoch - 58ms/step
## Epoch 65/100
## 3/3 - 0s - loss: 7560.1183 - 173ms/epoch - 58ms/step
## Epoch 66/100
## 3/3 - 0s - loss: 7556.2790 - 180ms/epoch - 60ms/step
## Epoch 67/100
## 3/3 - 0s - loss: 7600.5103 - 213ms/epoch - 71ms/step
## Epoch 68/100
## 3/3 - 0s - loss: 7489.8848 - 188ms/epoch - 63ms/step
## Epoch 69/100
## 3/3 - 0s - loss: 7717.7759 - 175ms/epoch - 58ms/step
## Epoch 70/100
## 3/3 - 0s - loss: 7569.2166 - 172ms/epoch - 57ms/step
## Epoch 71/100
## 3/3 - 0s - loss: 7592.9302 - 210ms/epoch - 70ms/step
## Epoch 72/100
## 3/3 - 0s - loss: 7603.9111 - 175ms/epoch - 58ms/step
## Epoch 73/100
## 3/3 - 0s - loss: 7560.6123 - 169ms/epoch - 56ms/step
## Epoch 74/100
## 3/3 - 0s - loss: 7516.9108 - 168ms/epoch - 56ms/step
## Epoch 75/100
## 3/3 - 1s - loss: 7575.7803 - 973ms/epoch - 324ms/step
## Epoch 76/100
## 3/3 - 0s - loss: 7532.9220 - 168ms/epoch - 56ms/step
## Epoch 77/100
## 3/3 - 0s - loss: 7636.4103 - 170ms/epoch - 57ms/step
## Epoch 78/100
## 3/3 - 0s - loss: 7371.5636 - 189ms/epoch - 63ms/step
## Epoch 79/100
## 3/3 - 0s - loss: 7632.8094 - 211ms/epoch - 70ms/step
## Epoch 80/100
## 3/3 - 0s - loss: 7486.6429 - 173ms/epoch - 58ms/step
## Epoch 81/100
## 3/3 - 0s - loss: 7331.7376 - 196ms/epoch - 65ms/step
## Epoch 82/100
## 3/3 - 0s - loss: 7368.2318 - 180ms/epoch - 60ms/step
## Epoch 83/100
## 3/3 - 0s - loss: 7585.7554 - 223ms/epoch - 74ms/step
## Epoch 84/100
## 3/3 - 0s - loss: 7473.3662 - 178ms/epoch - 59ms/step
## Epoch 85/100
## 3/3 - 0s - loss: 7545.0013 - 174ms/epoch - 58ms/step
## Epoch 86/100
## 3/3 - 0s - loss: 7479.2215 - 175ms/epoch - 58ms/step
## Epoch 87/100
## 3/3 - 0s - loss: 7625.3677 - 211ms/epoch - 70ms/step
## Epoch 88/100
## 3/3 - 0s - loss: 7612.7886 - 182ms/epoch - 61ms/step
## Epoch 89/100
## 3/3 - 0s - loss: 7354.0651 - 191ms/epoch - 64ms/step
## Epoch 90/100
## 3/3 - 0s - loss: 7471.3841 - 208ms/epoch - 69ms/step
## Epoch 91/100
## 3/3 - 0s - loss: 7531.1683 - 175ms/epoch - 58ms/step
## Epoch 92/100
## 3/3 - 0s - loss: 7500.8265 - 179ms/epoch - 60ms/step
## Epoch 93/100
## 3/3 - 0s - loss: 7488.9810 - 216ms/epoch - 72ms/step
## Epoch 94/100
## 3/3 - 0s - loss: 7372.2904 - 172ms/epoch - 57ms/step
## Epoch 95/100
## 3/3 - 0s - loss: 7519.4377 - 172ms/epoch - 57ms/step
## Epoch 96/100
## 3/3 - 1s - loss: 7496.8224 - 960ms/epoch - 320ms/step
## Epoch 97/100
## 3/3 - 0s - loss: 7516.7526 - 177ms/epoch - 59ms/step
## Epoch 98/100
## 3/3 - 0s - loss: 7439.3939 - 167ms/epoch - 56ms/step
## Epoch 99/100
## 3/3 - 0s - loss: 7575.9368 - 172ms/epoch - 57ms/step
## Epoch 100/100
## 3/3 - 0s - loss: 7370.1673 - 203ms/epoch - 68ms/stepif (keras::is_keras_available() & reticulate::py_available()) {
# sample generation
set.seed(1)
gen_sample_result <- gen_exprs(cvae_result, 100,
batch_size, use_generator = TRUE)
# plot for augmented data
plot_aug(gen_sample_result, "PCA")
}## generating...## post-processing...
3 Session information
## R version 4.3.1 (2023-06-16)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 22.04.3 LTS
##
## Matrix products: default
## BLAS: /home/biocbuild/bbs-3.18-bioc/R/lib/libRblas.so
## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0
##
## locale:
## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
## [3] LC_TIME=en_GB LC_COLLATE=C
## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
## [9] LC_ADDRESS=C LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## time zone: America/New_York
## tzcode source: system (glibc)
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] SC3_1.30.0 GenomicRanges_1.54.0
## [3] SummarizedExperiment_1.32.0 SingleCellExperiment_1.24.0
## [5] IRanges_2.36.0 S4Vectors_0.40.0
## [7] VAExprs_1.8.0 mclust_6.0.0
## [9] keras_2.13.0
##
## loaded via a namespace (and not attached):
## [1] sylly.en_0.1-3 sylly_0.1-6
## [3] later_1.3.1 bitops_1.0-7
## [5] textstem_0.1.4 tibble_3.2.1
## [7] matlab_1.0.4 lifecycle_1.0.3
## [9] doParallel_1.0.17 NLP_0.2-1
## [11] lattice_0.22-5 SnowballC_0.7.1
## [13] magrittr_2.0.3 sass_0.4.7
## [15] rmarkdown_2.25 jquerylib_0.1.4
## [17] yaml_2.3.7 httpuv_1.6.12
## [19] text2vec_0.6.3 doRNG_1.8.6
## [21] reticulate_1.34.0 cowplot_1.1.1
## [23] RColorBrewer_1.1-3 abind_1.4-5
## [25] zlibbioc_1.48.0 rvest_1.0.3
## [27] ttgsea_1.10.0 purrr_1.0.2
## [29] BiocGenerics_0.48.0 RCurl_1.98-1.12
## [31] WriteXLS_6.4.0 float_0.3-1
## [33] CatEncoders_0.1.1 GenomeInfoDbData_1.2.11
## [35] data.tree_1.0.0 tm_0.7-11
## [37] ggrepel_0.9.4 irlba_2.3.5.1
## [39] tokenizers_0.3.0 pheatmap_1.0.12
## [41] DelayedMatrixStats_1.24.0 codetools_0.2-19
## [43] DelayedArray_0.28.0 scuttle_1.12.0
## [45] xml2_1.3.5 tidyselect_1.2.0
## [47] PRROC_1.3.1 farver_2.1.1
## [49] ScaledMatrix_1.10.0 viridis_0.6.4
## [51] matrixStats_1.0.0 stats4_4.3.1
## [53] base64enc_0.1-3 jsonlite_1.8.7
## [55] rsparse_0.5.1 BiocNeighbors_1.20.0
## [57] e1071_1.7-13 ellipsis_0.3.2
## [59] scater_1.30.0 iterators_1.0.14
## [61] foreach_1.5.2 tools_4.3.1
## [63] stringdist_0.9.10 Rcpp_1.0.11
## [65] glue_1.6.2 gridExtra_2.3
## [67] tfruns_1.5.1 SparseArray_1.2.0
## [69] xfun_0.40 MatrixGenerics_1.14.0
## [71] GenomeInfoDb_1.38.0 dplyr_1.1.3
## [73] withr_2.5.1 fastmap_1.1.1
## [75] fansi_1.0.5 digest_0.6.33
## [77] rsvd_1.0.5 mime_0.12
## [79] R6_2.5.1 colorspace_2.1-0
## [81] koRpus_0.13-8 koRpus.lang.en_0.1-4
## [83] RhpcBLASctl_0.23-42 DiagrammeR_1.0.10
## [85] utf8_1.2.4 generics_0.1.3
## [87] data.table_1.14.8 robustbase_0.99-0
## [89] class_7.3-22 stopwords_2.3
## [91] httr_1.4.7 htmlwidgets_1.6.2
## [93] S4Arrays_1.2.0 whisker_0.4.1
## [95] pkgconfig_2.0.3 gtable_0.3.4
## [97] tensorflow_2.14.0 XVector_0.42.0
## [99] pcaPP_2.0-3 htmltools_0.5.6.1
## [101] scales_1.2.1 Biobase_2.62.0
## [103] png_0.1-8 lgr_0.4.4
## [105] knitr_1.44 rstudioapi_0.15.0
## [107] visNetwork_2.1.2 proxy_0.4-27
## [109] cachem_1.0.8 stringr_1.5.0
## [111] parallel_4.3.1 vipor_0.4.5
## [113] mlapi_0.1.1 pillar_1.9.0
## [115] grid_4.3.1 vctrs_0.6.4
## [117] promises_1.2.1 slam_0.1-50
## [119] BiocSingular_1.18.0 beachmat_2.18.0
## [121] xtable_1.8-4 cluster_2.1.4
## [123] beeswarm_0.4.0 evaluate_0.22
## [125] zeallot_0.1.0 mvtnorm_1.2-3
## [127] cli_3.6.1 compiler_4.3.1
## [129] rlang_1.1.1 crayon_1.5.2
## [131] rngtools_1.5.2 webchem_1.3.0
## [133] rrcov_1.7-4 labeling_0.4.3
## [135] ggbeeswarm_0.7.2 stringi_1.7.12
## [137] viridisLite_0.4.2 BiocParallel_1.36.0
## [139] DeepPINCS_1.10.0 munsell_0.5.0
## [141] Matrix_1.6-1.1 sparseMatrixStats_1.14.0
## [143] ggplot2_3.4.4 shiny_1.7.5.1
## [145] ROCR_1.0-11 bslib_0.5.1
## [147] DEoptimR_1.1-34 References
Aggarwal, C. C. (2018). Neural Networks and Deep Learning. Springer.
Al-Jabery, K., Obafemi-Ajayi, T., Olbricht, G., & Wunsch, D. (2019). Computational Learning Approaches to Data Analytics in Biomedical Applications. Academic Press.
Cinelli, L. P., Marins, M. A., da Silva, E. A. B., & Netto, S. L. (2021). Variational Methods for Machine Learning with Applications to Deep Networks. Springer.
Das, H., Pradhan, C., & Dey, N. (2020). Deep Learning for Data Analytics: Foundations, Biomedical Applications, and Challenges. Academic Press.
Marouf, M., Machart, P., Bansal, V., Kilian, C., Magruder, D. S., Krebs, C. F., & Bonn, S. (2020). Realistic in silico generation and augmentation of single-cell RNA-seq data using generative adversarial networks. Nature communications, 11(1), 1-12.
Pedrycz, W., & Chen, S. M. (Eds.). (2020). Deep Learning: Concepts and Architectures. Springer.
Yan, W. Q. (2020). Computational Methods for Deep Learning: Theoretic, Practice and Applications. Springer.