---
title: "Introduction to Using AutoTab"
output: rmarkdown::html_vignette
bibliography: references.bib
vignette: >
  %\VignetteIndexEntry{autotab-intro}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---



# Introduction

This vignette walks through a complete workflow using **AutoTab** on
simulated tabular data containing a mix of continuous, binary, and
categorical variables. We cover setting up the required
Python/TensorFlow environment, preprocessing, training a variational
autoencoder (VAE), and sampling synthetic data from the trained model.

The workflow below is organized into five training examples, each
highlighting a different set of hyperparameter options available in
`VAE_train()`. These examples are not intended to represent an optimized
model, they are meant to demonstrate how each hyperparameter is specified
and what its effect looks like during training.

| Example | Description |
|---------|-------------|
| 1 | A VAE executing simple functionalities |
| 2 | A VAE executing a weighted wTAB-ELBO and separate examination of by variable reconstruction loss. |
| 3 | A VAE executing cyclical beta annealing. |
| 4 | A VAE executing temperature annealing. |
| 5 | A VAE executing L2 regularization and Batch Normalization. |

After training, we walk through how to rebuild a trained encoder/decoder
from saved weights and sample synthetic observations from the model.

## Simulated data in AutoTab

``` r
library(autotab)
```

The simulated data contains 15 variables and 10,000 rows. There are
five variables of each type: continuous, binary, and categorical.
This dataset is accessible in the AutoTab package.

| Distribution | Variable Name | Range / Levels |
|---|---|---|
| Continuous | Age | (18,87) |
| Continuous | Height | (59,79) |
| Continuous | Weight | (100,328) |
| Continuous | SBP (Systolic Blood Pressure) | (90,188) |
| Continuous | Cholesterol | (100,361) |
| Binary | Female | 1=Yes; 0=No |
| Binary | History (family history of heart attacks) | |
| Binary | Smoke (current smoker) | |
| Binary | Hyperten (family history of hypertension) | |
| Binary | Heart (personal history of heart attack) | |
| Categorical | Socio (Socioeconomic Status) | Low; Medium; High |
| Categorical | Marital (Marital Status) | Single; Married; Divorced; Widowed |
| Categorical | Employ (Employment Status) | Unemployed; Student; Employed; Retired |
| Categorical | Race | Black; White; Hispanic; Other |
| Categorical | Edu (Education) | Some College; High School; Bachelors; Graduate |



``` r
data("data_example")
str(as.data.frame(data_example))
#> 'data.frame':	10000 obs. of  15 variables:
#>  $ age        : num  40.5 44.5 36.6 26.5 46 ...
#>  $ height     : num  67.8 70.4 68.8 70.7 71.1 ...
#>  $ weight     : num  137 198 229 217 202 ...
#>  $ SBP        : num  110 125 112 114 113 ...
#>  $ cholesterol: num  219 189 275 196 209 ...
#>  $ female     : num  1 0 0 1 1 0 0 0 1 1 ...
#>  $ history    : num  0 0 1 0 0 0 1 1 0 1 ...
#>  $ smoke      : num  1 1 1 0 1 1 1 1 0 1 ...
#>  $ hyperten   : num  0 1 1 1 1 1 1 0 0 1 ...
#>  $ heart      : num  0 0 1 1 1 1 1 0 1 1 ...
#>  $ socio      : chr  "High" "Medium" "Medium" "Low" ...
#>  $ marital    : chr  "Married" "Divorced" "Married" "Married" ...
#>  $ employ     : chr  "Employed" "Employed" "Employed" "Student" ...
#>  $ race       : chr  "White" "Black" "White" "Hispanic" ...
#>  $ edu        : chr  "BS" "HS" "BS" "HS" ...
```


## Preprocessing

Preprocessing of the data includes removing missing values (or using an
imputation method on the dataset), one-hot-coding, variable scaling, and
ensuring all columns are numeric. Some recommended packages for handling
missing data prior to preprocessing include mice and missForest
[@buurenMiceMultivariateImputation2011; @stekhovenMissForestNonParametric2012].
The user may also decide to use the complete case version of the intended
dataset. We use min-max-scaling for all continuous variables, and
one-hot-coding for the categorical variables. Out of the total one-hot-coded
columns for a given variable only one can be true in each row of data. This
column will be 1 and all others receive a 0 for that observation. After
one-hot-coding, socioeconomic status goes from 1 to 3 columns, marital
status goes from 1 to 4 columns, and so on. This increases the number of
columns from 15 to 29.

The code below creates the one-hot-coded categorical variables from the
the categorical variables shown above, using the caret package.


``` r
library("caret")
#> Loading required package: ggplot2
#> Loading required package: lattice
encoded_data = dummyVars(~ socio + marital + employ + race + edu,
                         data = data_example)
one_hot_encoded = as.data.frame(predict(encoded_data,
                                        newdata = data_example))
```

The code below is used to min-max-scale the continuous variables, the
function min_max_scale is used from the AutoTab package.


``` r
data_cont = subset(data_example,
                   select = c(age, height, weight, SBP, cholesterol))
data_scaled = as.data.frame(
  lapply(data_cont, min_max_scale))
```

The code below binds the one-hot-coded, min-max-scaled, and binary
variables together. In doing so, continuous variables are now in the range
of [0,1] and the categorical variables are one-hot-coded. Additionally, all
columns are numeric.


``` r
data_bin = subset(data_example,
                  select = c(female, history, smoke, hyperten, heart))
data = cbind(data_scaled, data_bin, one_hot_encoded)
head(data)
#>         age    height    weight       SBP cholesterol female history smoke hyperten heart socioHigh socioLow
#> 1 0.3281284 0.4440304 0.1619451 0.2011243   0.4554580      1       0     1        0     0         1        0
#> 2 0.3868740 0.5768439 0.4274326 0.3602980   0.3399162      0       0     1        1     0         0        0
#> 3 0.2710934 0.4963822 0.5672755 0.2279109   0.6694594      0       1     1        1     1         0        0
#> 4 0.1237089 0.5940458 0.5119936 0.2475795   0.3671093      1       0     0        1     1         0        1
#> 5 0.4092805 0.6123429 0.4464353 0.2369313   0.4173742      1       0     1        1     1         0        0
#> 6 0.3760423 0.4976137 0.5650082 0.2733449   0.7009214      0       0     1        1     1         0        0
#>   socioMedium maritalDivorced maritalMarried maritalSingle maritalWidowed employEmployed employRetired
#> 1           0               0              1             0              0              1             0
#> 2           1               1              0             0              0              1             0
#> 3           1               0              1             0              0              1             0
#> 4           0               0              1             0              0              0             0
#> 5           1               0              1             0              0              0             0
#> 6           1               0              1             0              0              1             0
#>   employStudent employUnemployed raceBlack raceHispanic raceOther raceWhite eduBS eduGraduate eduHS
#> 1             0                0         0            0         0         1     1           0     0
#> 2             0                0         1            0         0         0     0           0     1
#> 3             0                0         0            0         0         1     1           0     0
#> 4             1                0         0            1         0         0     0           0     1
#> 5             1                0         0            0         0         1     1           0     0
#> 6             0                0         0            1         0         0     0           0     1
#>   eduSome college
#> 1               0
#> 2               0
#> 3               0
#> 4               0
#> 5               0
#> 6               0
```

In the preprocessed dataset there are 29 columns. With this final
preprocessed dataset AutoTab can be applied.


## Setting up the required environment

In order to use the functionalities within AutoTab that are directly
related to training the VAE, a reticulate environment must be set up in
the live R session. The underlying neural network components in AutoTab
are implemented via the Keras and TensorFlow Python libraries. Managing
the version requirements at the system level can lead to conflicts across
Python packages.

The recommended workflow is for the user to establish a Conda environment
that has the dependencies needed for the implementation of AutoTab. A
Conda environment is a self-contained environment that allows the user to
have control over Python and package versions. The package reticulate is
then used to point to the correct Conda environment. AutoTab is built and
tested using Python 3.11 and TensorFlow 2.13.

AutoTab includes a dedicated function, `install_autotab_env()`, that
automatically builds this Conda environment using a bundled environment
specification file. This removes the need for the user to manually
download or create the environment file themselves.


``` r
install_autotab_env()
```

Running this function will create a Conda environment named
`r-autotab-env` containing the exact Python, TensorFlow, and Keras versions
AutoTab was built and tested against. If the environment already exists,
`install_autotab_env()` will detect this and skip rebuilding it.

Once the environment has been created, it must be activated within the R
session using reticulate.


``` r
reticulate::use_condaenv("r-autotab-env", required = TRUE)
reticulate::py_config()
```

The above will not be run when the vingette is ran. The user must run
the above code themselves to activate the conda environment to have the
remaining portions of the vingette run. Without an active conda environment
the vingette will not run portions of the code that require the conda
enviorment.

AutoTab was built using TensorFlow version 2.13 and may be incompatible
with some system configurations, particularly older macOS software
environments. If compatibility issues arise, we recommend using a virtual
environment with the required dependencies or running AutoTab in a
cloud-based environment such as Google Colab or a computing cluster.

## Running an AutoTab training step

Now that the preprocessed data and training environment is ready, the next
step is to build feat_dist and make it available to AutoTab.


``` r
feat_dist = extracting_distribution(data_example)
print(feat_dist)
#>    column_name distribution num_params
#> 1          age     gaussian          2
#> 2       height     gaussian          2
#> 3       weight     gaussian          2
#> 4          SBP     gaussian          2
#> 5  cholesterol     gaussian          2
#> 6       female    bernoulli          1
#> 7      history    bernoulli          1
#> 8        smoke    bernoulli          1
#> 9     hyperten    bernoulli          1
#> 10       heart    bernoulli          1
#> 11       socio  categorical          3
#> 12     marital  categorical          4
#> 13      employ  categorical          4
#> 14        race  categorical          4
#> 15         edu  categorical          4
```

The output above is in the same order, from row 1 to row 15 as the input
data created during preprocessing. Hence, the additional function in
AutoTab, feat_reorder, is not needed in this example.

To show its utility, we will manually reorder feat_dist_temp and then
resolve the mismatch order. The code below swaps the rows of age and
female in feat_dist_temp.


``` r
feat_dist_shuffled = feat_dist[sample(nrow(feat_dist)), ]
```

This results in a new feat_dist that does not match the order of the input
data.

To resolve this mis-ordering and force the same order as the preprocessed
data the code below can be implemented.


``` r
feat_dist_fixed = feat_reorder(feat_dist_shuffled, data)
identical(feat_dist, feat_dist_fixed)
#> [1] TRUE
```

Now, the order of feat_dist_fixed matches that of the preprocessed data and
is ready to use in AutoTab. Given that the original feat_dist already
matched in ordering, we will use this version. To make it accessible to
AutoTab the following code must be run:


``` r
set_feat_dist(feat_dist)
```

With the data preprocessed and feat_dist set within the environment, a
training statement can be run within AutoTab.

### Example one

Example One will run a simple VAE train statement. First, encoder_info and
decoder_info need to be created. These two lists will describe the
structure of the VAE encoder and decoder, along with applying any L2
regularization and BN. Below we show the code to create encoder_info and
decoder_info in this example.


``` r
encoder_info = list(list("dense", 50, "tanh"), list("dense", 100, "tanh"))
decoder_info = list(list("dense", 100, "tanh"), list("dense", 50, "tanh"))
```

The above example is a simple two layer VAE with hyperbolic tangent
activations. Next, we will run a simple VAE train statement. First, we use
the AutoTab function reset_seeds. This ensures reproducibility across R and
Python. The seed 2026 is chosen as an example, the user may choose their
own seed.


``` r
reset_seeds(2026)
#> Random seeds reset
```

The code below will run a VAE step with a beta value of 0.2, a latent
space with 10 dimensions, an early stopping criterion of 10 epochs with the
default min_delta of 0.001, a batch size of 32, a temperature of 1.2, a
learning rate of 0.00001, no spectral normalization, and will train the VAE
across 10 epochs.


``` r
example1 = VAE_train(data, encoder_info = encoder_info,
                     decoder_info = decoder_info, latent_dim = 10,
                     batchsize = 32, epoch = 10, beta = 0.2,
                     temperature = 1.2, Lip_en = 0, lip_dec = 0,
                     wait = 10, lr = 0.00001)
#> Output- Processing feature age with distribution: gaussian and num_params: 2 with index starting at 0
#> Output- Processing feature height with distribution: gaussian and num_params: 2 with index starting at 2
#> Output- Processing feature weight with distribution: gaussian and num_params: 2 with index starting at 4
#> Output- Processing feature SBP with distribution: gaussian and num_params: 2 with index starting at 6
#> Output- Processing feature cholesterol with distribution: gaussian and num_params: 2 with index starting at 8
#> Output- Processing feature female with distribution: bernoulli and num_params: 1 with index starting at 10
#> Output- Processing feature history with distribution: bernoulli and num_params: 1 with index starting at 11
#> Output- Processing feature smoke with distribution: bernoulli and num_params: 1 with index starting at 12
#> Output- Processing feature hyperten with distribution: bernoulli and num_params: 1 with index starting at 13
#> Output- Processing feature heart with distribution: bernoulli and num_params: 1 with index starting at 14
#> Output- Processing feature socio with distribution: categorical and num_params: 3 with index starting at 15
#> Output- Processing feature marital with distribution: categorical and num_params: 4 with index starting at 18
#> Output- Processing feature employ with distribution: categorical and num_params: 4 with index starting at 22
#> Output- Processing feature race with distribution: categorical and num_params: 4 with index starting at 26
#> Output- Processing feature edu with distribution: categorical and num_params: 4 with index starting at 30
#> Loss - Processing feature 1 with distribution: gaussian and num_params: 2 with index starting at 0
#> Loss - Processing feature 2 with distribution: gaussian and num_params: 2 with index starting at 2
#> Loss - Processing feature 3 with distribution: gaussian and num_params: 2 with index starting at 4
#> Loss - Processing feature 4 with distribution: gaussian and num_params: 2 with index starting at 6
#> Loss - Processing feature 5 with distribution: gaussian and num_params: 2 with index starting at 8
#> Loss - Processing feature 6 with distribution: bernoulli and num_params: 1 with index starting at 10
#> Loss - Processing feature 7 with distribution: bernoulli and num_params: 1 with index starting at 11
#> Loss - Processing feature 8 with distribution: bernoulli and num_params: 1 with index starting at 12
#> Loss - Processing feature 9 with distribution: bernoulli and num_params: 1 with index starting at 13
#> Loss - Processing feature 10 with distribution: bernoulli and num_params: 1 with index starting at 14
#> Loss - Processing feature 11 with distribution: categorical and num_params: 3 with index starting at 15
#> Loss - Processing feature 12 with distribution: categorical and num_params: 4 with index starting at 18
#> Loss - Processing feature 13 with distribution: categorical and num_params: 4 with index starting at 22
#> Loss - Processing feature 14 with distribution: categorical and num_params: 4 with index starting at 26
#> Loss - Processing feature 15 with distribution: categorical and num_params: 4 with index starting at 30
#> Loss - total KerasTensor(type_spec=TensorSpec(shape=(), dtype=tf.float32, name=None), name='tf.math.reduce_sum_6/Sum:0', description="created by layer 'tf.math.reduce_sum_6'")
#> Epoch 1/10
#> 250/250 - 7s - loss: 5.1730 - kl_loss: 0.2322 - recon_loss: 4.9409 - val_loss: 4.7679 - val_kl_loss: 0.2145 - val_recon_loss: 4.5534 - 7s/epoch - 28ms/step
#> Epoch 2/10
#> 250/250 - 2s - loss: 4.4627 - kl_loss: 0.2147 - recon_loss: 4.2480 - val_loss: 4.1469 - val_kl_loss: 0.2230 - val_recon_loss: 3.9239 - 2s/epoch - 7ms/step
#> Epoch 3/10
#> 250/250 - 2s - loss: 3.8698 - kl_loss: 0.2352 - recon_loss: 3.6347 - val_loss: 3.6299 - val_kl_loss: 0.2509 - val_recon_loss: 3.3790 - 2s/epoch - 8ms/step
#> Epoch 4/10
#> 250/250 - 2s - loss: 3.4816 - kl_loss: 0.2653 - recon_loss: 3.2163 - val_loss: 3.2958 - val_kl_loss: 0.2797 - val_recon_loss: 3.0161 - 2s/epoch - 7ms/step
#> Epoch 5/10
#> 250/250 - 2s - loss: 3.2085 - kl_loss: 0.2874 - recon_loss: 2.9212 - val_loss: 3.0808 - val_kl_loss: 0.2964 - val_recon_loss: 2.7844 - 2s/epoch - 7ms/step
#> Epoch 6/10
#> 250/250 - 2s - loss: 3.0247 - kl_loss: 0.3000 - recon_loss: 2.7247 - val_loss: 2.8818 - val_kl_loss: 0.3056 - val_recon_loss: 2.5761 - 2s/epoch - 7ms/step
#> Epoch 7/10
#> 250/250 - 2s - loss: 2.8655 - kl_loss: 0.3107 - recon_loss: 2.5548 - val_loss: 2.7858 - val_kl_loss: 0.3151 - val_recon_loss: 2.4707 - 2s/epoch - 7ms/step
#> Epoch 8/10
#> 250/250 - 2s - loss: 2.7531 - kl_loss: 0.3194 - recon_loss: 2.4338 - val_loss: 2.7084 - val_kl_loss: 0.3238 - val_recon_loss: 2.3845 - 2s/epoch - 7ms/step
#> Epoch 9/10
#> 250/250 - 2s - loss: 2.6556 - kl_loss: 0.3279 - recon_loss: 2.3277 - val_loss: 2.6069 - val_kl_loss: 0.3324 - val_recon_loss: 2.2745 - 2s/epoch - 7ms/step
#> Epoch 10/10
#> 250/250 - 2s - loss: 2.5915 - kl_loss: 0.3374 - recon_loss: 2.2541 - val_loss: 2.5001 - val_kl_loss: 0.3443 - val_recon_loss: 2.1558 - 2s/epoch - 6ms/step
```

AutoTab will print summary information at each epoch. When run in an
interactive R session, the user will also see the loss curves update live
within the plot viewer, where the y axis is the loss value. This is not
displayed in this document.

This viewer displays three curves: the top curve is the overall loss,
this is AutoTab's wTAB-ELBO. The middle curve is the regularization
portion of wTAB-ELBO, and the bottom curve is the reconstruction portion.
As training progresses, the reconstruction loss typically decreases while
the regularization pressure increases, indicating that the latent space is
being smoothed towards the prior while the reconstruction of the output
data improves. Within each plot the blue line represents the loss on the
training data and the green line represents the loss on the validation
set.

### Example two

Example two will show how the user can output the loss per variable type
with seperate = 1 added to the training statement and add optional
weighting with weighted = 1. This feature of AutoTab can be helpful in
identifying whether all three variable types are being trained properly
or if any of the variable types are isolated due to gradient imbalance. If
an imbalance is identified, the user can add recon_weights to the
VAE_train statement with weighted = 1.


``` r
reset_seeds(2026)
#> Random seeds reset
recon_weights = c(1, 3, 1)
example2 = VAE_train(data, encoder_info = encoder_info,
                     decoder_info = decoder_info, latent_dim = 10,
                     batchsize = 32, epoch = 10, beta = 0.2,
                     temperature = 1.2, seperate = 1,
                     weighted = 1, recon_weights = recon_weights,
                     Lip_en = 0, lip_dec = 0, wait = 10, lr = 0.00001)
#> Output- Processing feature age with distribution: gaussian and num_params: 2 with index starting at 0
#> Output- Processing feature height with distribution: gaussian and num_params: 2 with index starting at 2
#> Output- Processing feature weight with distribution: gaussian and num_params: 2 with index starting at 4
#> Output- Processing feature SBP with distribution: gaussian and num_params: 2 with index starting at 6
#> Output- Processing feature cholesterol with distribution: gaussian and num_params: 2 with index starting at 8
#> Output- Processing feature female with distribution: bernoulli and num_params: 1 with index starting at 10
#> Output- Processing feature history with distribution: bernoulli and num_params: 1 with index starting at 11
#> Output- Processing feature smoke with distribution: bernoulli and num_params: 1 with index starting at 12
#> Output- Processing feature hyperten with distribution: bernoulli and num_params: 1 with index starting at 13
#> Output- Processing feature heart with distribution: bernoulli and num_params: 1 with index starting at 14
#> Output- Processing feature socio with distribution: categorical and num_params: 3 with index starting at 15
#> Output- Processing feature marital with distribution: categorical and num_params: 4 with index starting at 18
#> Output- Processing feature employ with distribution: categorical and num_params: 4 with index starting at 22
#> Output- Processing feature race with distribution: categorical and num_params: 4 with index starting at 26
#> Output- Processing feature edu with distribution: categorical and num_params: 4 with index starting at 30
#> Loss - Processing feature 1 with distribution: gaussian and num_params: 2 with index starting at 0
#> Loss - Processing feature 2 with distribution: gaussian and num_params: 2 with index starting at 2
#> Loss - Processing feature 3 with distribution: gaussian and num_params: 2 with index starting at 4
#> Loss - Processing feature 4 with distribution: gaussian and num_params: 2 with index starting at 6
#> Loss - Processing feature 5 with distribution: gaussian and num_params: 2 with index starting at 8
#> Loss - Processing feature 6 with distribution: bernoulli and num_params: 1 with index starting at 10
#> Loss - Processing feature 7 with distribution: bernoulli and num_params: 1 with index starting at 11
#> Loss - Processing feature 8 with distribution: bernoulli and num_params: 1 with index starting at 12
#> Loss - Processing feature 9 with distribution: bernoulli and num_params: 1 with index starting at 13
#> Loss - Processing feature 10 with distribution: bernoulli and num_params: 1 with index starting at 14
#> Loss - Processing feature 11 with distribution: categorical and num_params: 3 with index starting at 15
#> Loss - Processing feature 12 with distribution: categorical and num_params: 4 with index starting at 18
#> Loss - Processing feature 13 with distribution: categorical and num_params: 4 with index starting at 22
#> Loss - Processing feature 14 with distribution: categorical and num_params: 4 with index starting at 26
#> Loss - Processing feature 15 with distribution: categorical and num_params: 4 with index starting at 30
#> Loss - total KerasTensor(type_spec=TensorSpec(shape=(), dtype=tf.float32, name=None), name='tf.math.reduce_sum_6/Sum:0', description="created by layer 'tf.math.reduce_sum_6'")
#> Epoch 1/10
#> 250/250 - 9s - loss: 7.1226 - kl_loss: 0.2340 - recon_loss: 6.8887 - cont_loss: 0.5941 - bin_loss: 0.9656 - cat_loss: 3.3977 - val_loss: 6.5549 - val_kl_loss: 0.2181 - val_recon_loss: 6.3368 - val_cont_loss: 0.5768 - val_bin_loss: 0.8772 - val_cat_loss: 3.1284 - 9s/epoch - 35ms/step
#> Epoch 2/10
#> 250/250 - 2s - loss: 6.1246 - kl_loss: 0.2225 - recon_loss: 5.9021 - cont_loss: 0.5547 - bin_loss: 0.8099 - cat_loss: 2.9176 - val_loss: 5.6986 - val_kl_loss: 0.2355 - val_recon_loss: 5.4632 - val_cont_loss: 0.5315 - val_bin_loss: 0.7496 - val_cat_loss: 2.6827 - 2s/epoch - 6ms/step
#> Epoch 3/10
#> 250/250 - 2s - loss: 5.3389 - kl_loss: 0.2561 - recon_loss: 5.0827 - cont_loss: 0.5100 - bin_loss: 0.7086 - cat_loss: 2.4468 - val_loss: 5.0168 - val_kl_loss: 0.2807 - val_recon_loss: 4.7361 - val_cont_loss: 0.4885 - val_bin_loss: 0.6665 - val_cat_loss: 2.2481 - 2s/epoch - 6ms/step
#> Epoch 4/10
#> 250/250 - 2s - loss: 4.8412 - kl_loss: 0.3040 - recon_loss: 4.5371 - cont_loss: 0.4666 - bin_loss: 0.6490 - cat_loss: 2.1235 - val_loss: 4.6268 - val_kl_loss: 0.3255 - val_recon_loss: 4.3013 - val_cont_loss: 0.4380 - val_bin_loss: 0.6322 - val_cat_loss: 1.9666 - 2s/epoch - 6ms/step
#> Epoch 5/10
#> 250/250 - 2s - loss: 4.5095 - kl_loss: 0.3385 - recon_loss: 4.1711 - cont_loss: 0.4224 - bin_loss: 0.6166 - cat_loss: 1.8989 - val_loss: 4.3631 - val_kl_loss: 0.3519 - val_recon_loss: 4.0112 - val_cont_loss: 0.3994 - val_bin_loss: 0.6054 - val_cat_loss: 1.7955 - 2s/epoch - 6ms/step
#> Epoch 6/10
#> 250/250 - 2s - loss: 4.2875 - kl_loss: 0.3570 - recon_loss: 3.9305 - cont_loss: 0.3810 - bin_loss: 0.5951 - cat_loss: 1.7644 - val_loss: 4.1337 - val_kl_loss: 0.3641 - val_recon_loss: 3.7696 - val_cont_loss: 0.3561 - val_bin_loss: 0.5917 - val_cat_loss: 1.6384 - 2s/epoch - 7ms/step
#> Epoch 7/10
#> 250/250 - 2s - loss: 4.0997 - kl_loss: 0.3713 - recon_loss: 3.7285 - cont_loss: 0.3404 - bin_loss: 0.5831 - cat_loss: 1.6387 - val_loss: 4.0039 - val_kl_loss: 0.3763 - val_recon_loss: 3.6276 - val_cont_loss: 0.3231 - val_bin_loss: 0.5757 - val_cat_loss: 1.5774 - 2s/epoch - 7ms/step
#> Epoch 8/10
#> 250/250 - 2s - loss: 3.9643 - kl_loss: 0.3828 - recon_loss: 3.5815 - cont_loss: 0.2997 - bin_loss: 0.5726 - cat_loss: 1.5641 - val_loss: 3.9136 - val_kl_loss: 0.3889 - val_recon_loss: 3.5246 - val_cont_loss: 0.2803 - val_bin_loss: 0.5715 - val_cat_loss: 1.5297 - 2s/epoch - 6ms/step
#> Epoch 9/10
#> 250/250 - 1s - loss: 3.8471 - kl_loss: 0.3947 - recon_loss: 3.4524 - cont_loss: 0.2539 - bin_loss: 0.5644 - cat_loss: 1.5051 - val_loss: 3.7936 - val_kl_loss: 0.4003 - val_recon_loss: 3.3933 - val_cont_loss: 0.2327 - val_bin_loss: 0.5612 - val_cat_loss: 1.4772 - 1s/epoch - 6ms/step
#> Epoch 10/10
#> 250/250 - 2s - loss: 3.7668 - kl_loss: 0.4074 - recon_loss: 3.3594 - cont_loss: 0.2117 - bin_loss: 0.5560 - cat_loss: 1.4797 - val_loss: 3.6628 - val_kl_loss: 0.4157 - val_recon_loss: 3.2471 - val_cont_loss: 0.1874 - val_bin_loss: 0.5501 - val_cat_loss: 1.4094 - 2s/epoch - 6ms/step
```

In this output there are 6 more tracked losses, a training and validation
loss for each variable type. When run interactively, the plot viewer will
also display 3 additional plots (6 in total), one for each variable type.

### Example three

Example Three will show how the user can apply β annealing or cyclical
annealing. The hyperparameters in the VAE_train statement to indicate
cyclical annealing are kl_warm, kl_cyclical, n_cycles, and ratio. The code
below will train the VAE with cyclical annealing across 2 cycles with a
ratio of 0.4.


``` r
reset_seeds(2026)
#> Random seeds reset
example3 = VAE_train(data, encoder_info = encoder_info,
                     decoder_info = decoder_info,
                     latent_dim = 10, batchsize = 32,
                     epoch = 10, beta = 0.2,
                     temperature = 1.2, kl_warm = TRUE,
                     kl_cyclical = TRUE, n_cycles = 2, ratio = 0.4,
                     Lip_en = 0, lip_dec = 0, wait = 10, lr = 0.00001)
#> Output- Processing feature age with distribution: gaussian and num_params: 2 with index starting at 0
#> Output- Processing feature height with distribution: gaussian and num_params: 2 with index starting at 2
#> Output- Processing feature weight with distribution: gaussian and num_params: 2 with index starting at 4
#> Output- Processing feature SBP with distribution: gaussian and num_params: 2 with index starting at 6
#> Output- Processing feature cholesterol with distribution: gaussian and num_params: 2 with index starting at 8
#> Output- Processing feature female with distribution: bernoulli and num_params: 1 with index starting at 10
#> Output- Processing feature history with distribution: bernoulli and num_params: 1 with index starting at 11
#> Output- Processing feature smoke with distribution: bernoulli and num_params: 1 with index starting at 12
#> Output- Processing feature hyperten with distribution: bernoulli and num_params: 1 with index starting at 13
#> Output- Processing feature heart with distribution: bernoulli and num_params: 1 with index starting at 14
#> Output- Processing feature socio with distribution: categorical and num_params: 3 with index starting at 15
#> Output- Processing feature marital with distribution: categorical and num_params: 4 with index starting at 18
#> Output- Processing feature employ with distribution: categorical and num_params: 4 with index starting at 22
#> Output- Processing feature race with distribution: categorical and num_params: 4 with index starting at 26
#> Output- Processing feature edu with distribution: categorical and num_params: 4 with index starting at 30
#> Loss - Processing feature 1 with distribution: gaussian and num_params: 2 with index starting at 0
#> Loss - Processing feature 2 with distribution: gaussian and num_params: 2 with index starting at 2
#> Loss - Processing feature 3 with distribution: gaussian and num_params: 2 with index starting at 4
#> Loss - Processing feature 4 with distribution: gaussian and num_params: 2 with index starting at 6
#> Loss - Processing feature 5 with distribution: gaussian and num_params: 2 with index starting at 8
#> Loss - Processing feature 6 with distribution: bernoulli and num_params: 1 with index starting at 10
#> Loss - Processing feature 7 with distribution: bernoulli and num_params: 1 with index starting at 11
#> Loss - Processing feature 8 with distribution: bernoulli and num_params: 1 with index starting at 12
#> Loss - Processing feature 9 with distribution: bernoulli and num_params: 1 with index starting at 13
#> Loss - Processing feature 10 with distribution: bernoulli and num_params: 1 with index starting at 14
#> Loss - Processing feature 11 with distribution: categorical and num_params: 3 with index starting at 15
#> Loss - Processing feature 12 with distribution: categorical and num_params: 4 with index starting at 18
#> Loss - Processing feature 13 with distribution: categorical and num_params: 4 with index starting at 22
#> Loss - Processing feature 14 with distribution: categorical and num_params: 4 with index starting at 26
#> Loss - Processing feature 15 with distribution: categorical and num_params: 4 with index starting at 30
#> Loss - total KerasTensor(type_spec=TensorSpec(shape=(), dtype=tf.float32, name=None), name='tf.math.reduce_sum_6/Sum:0', description="created by layer 'tf.math.reduce_sum_6'")
#> Cyclical Beta updated to: 0.00000 (epoch 0)
#> Epoch 1/10
#> 250/250 - 9s - loss: 4.9319 - kl_loss: 0.0000e+00 - recon_loss: 4.9319 - val_loss: 4.5290 - val_kl_loss: 0.0000e+00 - val_recon_loss: 4.5290 - 9s/epoch - 35ms/step
#> Cyclical Beta updated to: 0.10000 (epoch 1)
#> Epoch 2/10
#> 250/250 - 1s - loss: 4.3439 - kl_loss: 0.1292 - recon_loss: 4.2147 - val_loss: 4.0182 - val_kl_loss: 0.1368 - val_recon_loss: 3.8815 - 1s/epoch - 6ms/step
#> Cyclical Beta updated to: 0.20000 (epoch 2)
#> Epoch 3/10
#> 250/250 - 2s - loss: 3.8762 - kl_loss: 0.2734 - recon_loss: 3.6028 - val_loss: 3.6342 - val_kl_loss: 0.2788 - val_recon_loss: 3.3554 - 2s/epoch - 7ms/step
#> Cyclical Beta updated to: 0.20000 (epoch 3)
#> Epoch 4/10
#> 250/250 - 2s - loss: 3.4874 - kl_loss: 0.2871 - recon_loss: 3.2003 - val_loss: 3.3016 - val_kl_loss: 0.2965 - val_recon_loss: 3.0050 - 2s/epoch - 7ms/step
#> Cyclical Beta updated to: 0.20000 (epoch 4)
#> Epoch 5/10
#> 250/250 - 2s - loss: 3.2134 - kl_loss: 0.3010 - recon_loss: 2.9124 - val_loss: 3.0846 - val_kl_loss: 0.3074 - val_recon_loss: 2.7771 - 2s/epoch - 7ms/step
#> Cyclical Beta updated to: 0.00000 (epoch 5)
#> Epoch 6/10
#> 250/250 - 1s - loss: 2.6647 - kl_loss: 0.0000e+00 - recon_loss: 2.6647 - val_loss: 2.4679 - val_kl_loss: 0.0000e+00 - val_recon_loss: 2.4679 - 1s/epoch - 5ms/step
#> Cyclical Beta updated to: 0.10000 (epoch 6)
#> Epoch 7/10
#> 250/250 - 2s - loss: 2.6673 - kl_loss: 0.2416 - recon_loss: 2.4257 - val_loss: 2.5749 - val_kl_loss: 0.2510 - val_recon_loss: 2.3239 - 2s/epoch - 6ms/step
#> Cyclical Beta updated to: 0.20000 (epoch 7)
#> Epoch 8/10
#> 250/250 - 2s - loss: 2.7717 - kl_loss: 0.4445 - recon_loss: 2.3273 - val_loss: 2.7143 - val_kl_loss: 0.4066 - val_recon_loss: 2.3077 - 2s/epoch - 7ms/step
#> Cyclical Beta updated to: 0.20000 (epoch 8)
#> Epoch 9/10
#> 250/250 - 2s - loss: 2.6580 - kl_loss: 0.3924 - recon_loss: 2.2656 - val_loss: 2.6063 - val_kl_loss: 0.3844 - val_recon_loss: 2.2219 - 2s/epoch - 7ms/step
#> Cyclical Beta updated to: 0.20000 (epoch 9)
#> Epoch 10/10
#> 250/250 - 2s - loss: 2.5882 - kl_loss: 0.3827 - recon_loss: 2.2055 - val_loss: 2.4955 - val_kl_loss: 0.3843 - val_recon_loss: 2.1112 - 2s/epoch - 7ms/step
```

In this output, the beta value that is cyclically updated is tracked.

### Example four

The next example will focus on temperature warming. The structure from
Example 1 will be used with the addition of two hyperparameters: temp_warm
and temp_epoch.


``` r
reset_seeds(2026)
#> Random seeds reset
example4 = VAE_train(data, encoder_info = encoder_info,
                     decoder_info = decoder_info, latent_dim = 10,
                     batchsize = 32, epoch = 10, beta = 0.2,
                     temperature = 1.2, temp_warm = TRUE,
                     temp_epoch = 10, Lip_en = 0, lip_dec = 0,
                     wait = 10, lr = 0.00001)
#> Output- Processing feature age with distribution: gaussian and num_params: 2 with index starting at 0
#> Output- Processing feature height with distribution: gaussian and num_params: 2 with index starting at 2
#> Output- Processing feature weight with distribution: gaussian and num_params: 2 with index starting at 4
#> Output- Processing feature SBP with distribution: gaussian and num_params: 2 with index starting at 6
#> Output- Processing feature cholesterol with distribution: gaussian and num_params: 2 with index starting at 8
#> Output- Processing feature female with distribution: bernoulli and num_params: 1 with index starting at 10
#> Output- Processing feature history with distribution: bernoulli and num_params: 1 with index starting at 11
#> Output- Processing feature smoke with distribution: bernoulli and num_params: 1 with index starting at 12
#> Output- Processing feature hyperten with distribution: bernoulli and num_params: 1 with index starting at 13
#> Output- Processing feature heart with distribution: bernoulli and num_params: 1 with index starting at 14
#> Output- Processing feature socio with distribution: categorical and num_params: 3 with index starting at 15
#> Output- Processing feature marital with distribution: categorical and num_params: 4 with index starting at 18
#> Output- Processing feature employ with distribution: categorical and num_params: 4 with index starting at 22
#> Output- Processing feature race with distribution: categorical and num_params: 4 with index starting at 26
#> Output- Processing feature edu with distribution: categorical and num_params: 4 with index starting at 30
#> Loss - Processing feature 1 with distribution: gaussian and num_params: 2 with index starting at 0
#> Loss - Processing feature 2 with distribution: gaussian and num_params: 2 with index starting at 2
#> Loss - Processing feature 3 with distribution: gaussian and num_params: 2 with index starting at 4
#> Loss - Processing feature 4 with distribution: gaussian and num_params: 2 with index starting at 6
#> Loss - Processing feature 5 with distribution: gaussian and num_params: 2 with index starting at 8
#> Loss - Processing feature 6 with distribution: bernoulli and num_params: 1 with index starting at 10
#> Loss - Processing feature 7 with distribution: bernoulli and num_params: 1 with index starting at 11
#> Loss - Processing feature 8 with distribution: bernoulli and num_params: 1 with index starting at 12
#> Loss - Processing feature 9 with distribution: bernoulli and num_params: 1 with index starting at 13
#> Loss - Processing feature 10 with distribution: bernoulli and num_params: 1 with index starting at 14
#> Loss - Processing feature 11 with distribution: categorical and num_params: 3 with index starting at 15
#> Loss - Processing feature 12 with distribution: categorical and num_params: 4 with index starting at 18
#> Loss - Processing feature 13 with distribution: categorical and num_params: 4 with index starting at 22
#> Loss - Processing feature 14 with distribution: categorical and num_params: 4 with index starting at 26
#> Loss - Processing feature 15 with distribution: categorical and num_params: 4 with index starting at 30
#> Loss - total KerasTensor(type_spec=TensorSpec(shape=(), dtype=tf.float32, name=None), name='tf.math.reduce_sum_6/Sum:0', description="created by layer 'tf.math.reduce_sum_6'")
#> Temperature updated to: 1.2
#> Epoch 1/10
#> 250/250 - 10s - loss: 5.1730 - kl_loss: 0.2322 - recon_loss: 4.9409 - val_loss: 4.7679 - val_kl_loss: 0.2145 - val_recon_loss: 4.5534 - 10s/epoch - 38ms/step
#> Temperature updated to: 1.08
#> Epoch 2/10
#> 250/250 - 2s - loss: 4.7144 - kl_loss: 0.2164 - recon_loss: 4.4980 - val_loss: 4.3693 - val_kl_loss: 0.2262 - val_recon_loss: 4.1430 - 2s/epoch - 7ms/step
#> Temperature updated to: 0.96
#> Epoch 3/10
#> 250/250 - 2s - loss: 4.3118 - kl_loss: 0.2411 - recon_loss: 4.0706 - val_loss: 4.0221 - val_kl_loss: 0.2591 - val_recon_loss: 3.7630 - 2s/epoch - 6ms/step
#> Temperature updated to: 0.84
#> Epoch 4/10
#> 250/250 - 1s - loss: 4.1006 - kl_loss: 0.2773 - recon_loss: 3.8233 - val_loss: 3.8472 - val_kl_loss: 0.2952 - val_recon_loss: 3.5519 - 1s/epoch - 6ms/step
#> Temperature updated to: 0.72
#> Epoch 5/10
#> 250/250 - 2s - loss: 4.0233 - kl_loss: 0.3091 - recon_loss: 3.7142 - val_loss: 3.8276 - val_kl_loss: 0.3239 - val_recon_loss: 3.5037 - 2s/epoch - 6ms/step
#> Temperature updated to: 0.6
#> Epoch 6/10
#> 250/250 - 2s - loss: 4.1107 - kl_loss: 0.3367 - recon_loss: 3.7740 - val_loss: 3.8605 - val_kl_loss: 0.3516 - val_recon_loss: 3.5089 - 2s/epoch - 7ms/step
#> Temperature updated to: 0.5
#> Epoch 7/10
#> 250/250 - 2s - loss: 4.2286 - kl_loss: 0.3686 - recon_loss: 3.8600 - val_loss: 4.0652 - val_kl_loss: 0.3840 - val_recon_loss: 3.6812 - 2s/epoch - 7ms/step
#> Temperature updated to: 0.5
#> Epoch 8/10
#> 250/250 - 1s - loss: 4.0174 - kl_loss: 0.3969 - recon_loss: 3.6205 - val_loss: 3.9407 - val_kl_loss: 0.4086 - val_recon_loss: 3.5322 - 1s/epoch - 6ms/step
#> Temperature updated to: 0.5
#> Epoch 9/10
#> 250/250 - 2s - loss: 3.8597 - kl_loss: 0.4180 - recon_loss: 3.4417 - val_loss: 3.7813 - val_kl_loss: 0.4267 - val_recon_loss: 3.3546 - 2s/epoch - 7ms/step
#> Temperature updated to: 0.5
#> Epoch 10/10
#> 250/250 - 1s - loss: 3.7730 - kl_loss: 0.4362 - recon_loss: 3.3369 - val_loss: 3.6164 - val_kl_loss: 0.4461 - val_recon_loss: 3.1704 - 1s/epoch - 6ms/step
```

The output now tracks the temperature at each epoch. This value is
decreasing after each additional epoch.

### Example five

The last example shows additional options within the encoder_info and
decoder_info statements.


``` r
encoder_info5 = list(list("dense", 50, "tanh", 0, 0, TRUE),
                     list("dense", 100, "tanh", 0, 0, TRUE))
decoder_info5 = list(list("dense", 100, "tanh", 1),
                     list("dense", 50, "tanh"))
```


``` r
reset_seeds(2026)
#> Random seeds reset
example5 = VAE_train(data, encoder_info = encoder_info5,
                     decoder_info = decoder_info5, latent_dim = 10,
                     batchsize = 32, epoch = 10, beta = 0.2,
                     temperature = 1.2, Lip_en = 0, lip_dec = 0,
                     wait = 10, lr = 0.00001)
#> Output- Processing feature age with distribution: gaussian and num_params: 2 with index starting at 0
#> Output- Processing feature height with distribution: gaussian and num_params: 2 with index starting at 2
#> Output- Processing feature weight with distribution: gaussian and num_params: 2 with index starting at 4
#> Output- Processing feature SBP with distribution: gaussian and num_params: 2 with index starting at 6
#> Output- Processing feature cholesterol with distribution: gaussian and num_params: 2 with index starting at 8
#> Output- Processing feature female with distribution: bernoulli and num_params: 1 with index starting at 10
#> Output- Processing feature history with distribution: bernoulli and num_params: 1 with index starting at 11
#> Output- Processing feature smoke with distribution: bernoulli and num_params: 1 with index starting at 12
#> Output- Processing feature hyperten with distribution: bernoulli and num_params: 1 with index starting at 13
#> Output- Processing feature heart with distribution: bernoulli and num_params: 1 with index starting at 14
#> Output- Processing feature socio with distribution: categorical and num_params: 3 with index starting at 15
#> Output- Processing feature marital with distribution: categorical and num_params: 4 with index starting at 18
#> Output- Processing feature employ with distribution: categorical and num_params: 4 with index starting at 22
#> Output- Processing feature race with distribution: categorical and num_params: 4 with index starting at 26
#> Output- Processing feature edu with distribution: categorical and num_params: 4 with index starting at 30
#> Loss - Processing feature 1 with distribution: gaussian and num_params: 2 with index starting at 0
#> Loss - Processing feature 2 with distribution: gaussian and num_params: 2 with index starting at 2
#> Loss - Processing feature 3 with distribution: gaussian and num_params: 2 with index starting at 4
#> Loss - Processing feature 4 with distribution: gaussian and num_params: 2 with index starting at 6
#> Loss - Processing feature 5 with distribution: gaussian and num_params: 2 with index starting at 8
#> Loss - Processing feature 6 with distribution: bernoulli and num_params: 1 with index starting at 10
#> Loss - Processing feature 7 with distribution: bernoulli and num_params: 1 with index starting at 11
#> Loss - Processing feature 8 with distribution: bernoulli and num_params: 1 with index starting at 12
#> Loss - Processing feature 9 with distribution: bernoulli and num_params: 1 with index starting at 13
#> Loss - Processing feature 10 with distribution: bernoulli and num_params: 1 with index starting at 14
#> Loss - Processing feature 11 with distribution: categorical and num_params: 3 with index starting at 15
#> Loss - Processing feature 12 with distribution: categorical and num_params: 4 with index starting at 18
#> Loss - Processing feature 13 with distribution: categorical and num_params: 4 with index starting at 22
#> Loss - Processing feature 14 with distribution: categorical and num_params: 4 with index starting at 26
#> Loss - Processing feature 15 with distribution: categorical and num_params: 4 with index starting at 30
#> Loss - total KerasTensor(type_spec=TensorSpec(shape=(), dtype=tf.float32, name=None), name='tf.math.reduce_sum_6/Sum:0', description="created by layer 'tf.math.reduce_sum_6'")
#> Epoch 1/10
#> 250/250 - 10s - loss: 9.1136 - kl_loss: 3.1167 - recon_loss: 5.9951 - val_loss: 6.8337 - val_kl_loss: 1.5032 - val_recon_loss: 5.3288 - 10s/epoch - 39ms/step
#> Epoch 2/10
#> 250/250 - 2s - loss: 7.6085 - kl_loss: 2.4993 - recon_loss: 5.1074 - val_loss: 6.7814 - val_kl_loss: 2.1216 - val_recon_loss: 4.6579 - 2s/epoch - 7ms/step
#> Epoch 3/10
#> 250/250 - 2s - loss: 6.3006 - kl_loss: 2.0061 - recon_loss: 4.2928 - val_loss: 5.7638 - val_kl_loss: 1.8045 - val_recon_loss: 3.9576 - 2s/epoch - 7ms/step
#> Epoch 4/10
#> 250/250 - 1s - loss: 5.3509 - kl_loss: 1.6181 - recon_loss: 3.7311 - val_loss: 4.9019 - val_kl_loss: 1.4617 - val_recon_loss: 3.4384 - 1s/epoch - 6ms/step
#> Epoch 5/10
#> 250/250 - 2s - loss: 4.6167 - kl_loss: 1.3125 - recon_loss: 3.3025 - val_loss: 4.2798 - val_kl_loss: 1.1813 - val_recon_loss: 3.0967 - 2s/epoch - 8ms/step
#> Epoch 6/10
#> 250/250 - 1s - loss: 4.0958 - kl_loss: 1.0726 - recon_loss: 3.0214 - val_loss: 3.7911 - val_kl_loss: 0.9642 - val_recon_loss: 2.8251 - 1s/epoch - 6ms/step
#> Epoch 7/10
#> 250/250 - 2s - loss: 3.7213 - kl_loss: 0.8894 - recon_loss: 2.8301 - val_loss: 3.5066 - val_kl_loss: 0.8057 - val_recon_loss: 2.6991 - 2s/epoch - 7ms/step
#> Epoch 8/10
#> 250/250 - 2s - loss: 3.4720 - kl_loss: 0.7574 - recon_loss: 2.7129 - val_loss: 3.3319 - val_kl_loss: 0.6858 - val_recon_loss: 2.6443 - 2s/epoch - 7ms/step
#> Epoch 9/10
#> 250/250 - 2s - loss: 3.2895 - kl_loss: 0.6577 - recon_loss: 2.6300 - val_loss: 3.1610 - val_kl_loss: 0.5995 - val_recon_loss: 2.5598 - 2s/epoch - 7ms/step
#> Epoch 10/10
#> 250/250 - 1s - loss: 3.1677 - kl_loss: 0.5867 - recon_loss: 2.5793 - val_loss: 3.0014 - val_kl_loss: 0.5374 - val_recon_loss: 2.4623 - 1s/epoch - 6ms/step
```

AutoTab has a dedicated help page for applying MoG priors and therefore
these examples are excluded.

## Sampling from a VAE

Once the VAE has been trained, the next step is sampling from the VAE to
understand how well the generated data represents the input data. AutoTab
has three functions that the user can utilize to rebuild a trained VAE and
sample from it: Encoder_weights, encoder_latent and Decoder_weights.

Sampling from the VAE can be done in five steps:

1. Extract the trained model weights and biases.
2. Rebuild the trained encoder.
3. Rebuild the trained decoder.
4. Sample the latent space or prior.
5. Pass the sampled latent vectors through the trained decoder.

After these five steps are executed, the user can determine how the
generated data will be transformed back to its original form (e.g.,
reversing min-max-scaling). Step 5 provides access to the generative
distributional parameters learned by the VAE. At this stage, the user may
choose how to sample from these distributions to produce synthetic
observations. Suggestions are provided in this section; however,
alternative approaches may also be used.

Below we outline the execution from step 1 to 5 using the trained VAE from
Example one. Example one was a simple VAE with a set beta at 0.2, a latent
dimension of 10, and two hidden layers in both the encoder and decoder.

**Step One: Extract the trained model weights and biases.**


``` r
library("dplyr")
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
library("keras")
trained_model = example1$trained_model
```

The above code is pulling the trained_model from the saved training
object, example1. The other saved information in example1 is the loss
history, this is not needed for sampling from a trained VAE.

**Step Two: Rebuild the trained encoder.**


``` r
weights_encoder = Encoder_weights(encoder_layers = 2,
                                  trained_model, lip_enc = 0, pi_enc = 0,
                                  BNenc_layers = 0, learn_BN = 0)
latent_encoder = encoder_latent(encoder_input = data,
                                encoder_info = encoder_info, latent_dim = 10,
                                Lip_en = 0, power_iterations = 0)
print(latent_encoder)
#> Model: "encoder"
#> _____________________________________________________________________________________________________________
#>  Layer (type)                   Output Shape                     Param #    Connected to                     
#> =============================================================================================================
#>  input_3 (InputLayer)           [(None, 29)]                     0          []                               
#>  dense_4 (Dense)                (None, 50)                       1500       ['input_3[0][0]']                
#>  dense_5 (Dense)                (None, 100)                      5100       ['dense_4[0][0]']                
#>  z_mean (Dense)                 (None, 10)                       1010       ['dense_5[0][0]']                
#>  z_log_var (Dense)              (None, 10)                       1010       ['dense_5[0][0]']                
#> =============================================================================================================
#> Total params: 8620 (33.67 KB)
#> Trainable params: 8620 (33.67 KB)
#> Non-trainable params: 0 (0.00 Byte)
#> _____________________________________________________________________________________________________________
```

The output above shows that the encoder has been rebuilt. The input layer
has 29 columns, the two dense layers have 50 and 100 nodes, and the
results of the encoder are z_mean and z_log_var. Latent_encoder can be
thought of as the skeleton for the encoder. Now, we need to apply weights
to the encoder skeleton.


``` r
latent_encoder %>% set_weights(weights_encoder)
```

Now, the encoder is built with the learned weights and biases. These are
the typical weights and biases seen in a neural network.

**Step Three: Rebuild the trained decoder.**


``` r
weights_decoder = Decoder_weights(encoder_layers = 2,
                                  trained_model, lip_enc = 0, pi_enc = 0,
                                  prior_learn = "fixed", BNenc_layers = 0,
                                  learn_BN = 0)
decoder = decoder_model(decoder_input = NULL,
                        decoder_info = decoder_info, latent_dim = 10,
                        feat_dist = feat_dist, lip_dec = 0, pi_dec = 0)
#> Output- Processing feature age with distribution: gaussian and num_params: 2 with index starting at 0
#> Output- Processing feature height with distribution: gaussian and num_params: 2 with index starting at 2
#> Output- Processing feature weight with distribution: gaussian and num_params: 2 with index starting at 4
#> Output- Processing feature SBP with distribution: gaussian and num_params: 2 with index starting at 6
#> Output- Processing feature cholesterol with distribution: gaussian and num_params: 2 with index starting at 8
#> Output- Processing feature female with distribution: bernoulli and num_params: 1 with index starting at 10
#> Output- Processing feature history with distribution: bernoulli and num_params: 1 with index starting at 11
#> Output- Processing feature smoke with distribution: bernoulli and num_params: 1 with index starting at 12
#> Output- Processing feature hyperten with distribution: bernoulli and num_params: 1 with index starting at 13
#> Output- Processing feature heart with distribution: bernoulli and num_params: 1 with index starting at 14
#> Output- Processing feature socio with distribution: categorical and num_params: 3 with index starting at 15
#> Output- Processing feature marital with distribution: categorical and num_params: 4 with index starting at 18
#> Output- Processing feature employ with distribution: categorical and num_params: 4 with index starting at 22
#> Output- Processing feature race with distribution: categorical and num_params: 4 with index starting at 26
#> Output- Processing feature edu with distribution: categorical and num_params: 4 with index starting at 30
print(decoder)
```

<details>
<summary>Click to expand full decoder architecture</summary>

<pre><code>Model: "model_2"
_____________________________________________________________________________________________________________
 Layer (type)                   Output Shape                     Param #    Connected to                     
=============================================================================================================
 input_4 (InputLayer)           [(None, 10)]                     0          []                               
 dense_6 (Dense)                (None, 100)                      1100       ['input_4[0][0]']                
 dense_7 (Dense)                (None, 50)                       5050       ['dense_6[0][0]']                
 decoder_output_linear (Dense)  (None, 34)                       1734       ['dense_7[0][0]']                
 tf.compat.v1.shape_57 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_59 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_61 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_63 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_65 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.__operators__.getitem_55 (  ()                               0          ['tf.compat.v1.shape_57[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.compat.v1.shape_58 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.__operators__.getitem_57 (  ()                               0          ['tf.compat.v1.shape_59[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.compat.v1.shape_60 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.__operators__.getitem_59 (  ()                               0          ['tf.compat.v1.shape_61[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.compat.v1.shape_62 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.__operators__.getitem_61 (  ()                               0          ['tf.compat.v1.shape_63[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.compat.v1.shape_64 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.__operators__.getitem_63 (  ()                               0          ['tf.compat.v1.shape_65[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.compat.v1.shape_66 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_72 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_73 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_74 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_75 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_76 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.slice_55 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_55[0][
                                                                            0]']                             
 tf.__operators__.getitem_56 (  ()                               0          ['tf.compat.v1.shape_58[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.slice_57 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_57[0][
                                                                            0]']                             
 tf.__operators__.getitem_58 (  ()                               0          ['tf.compat.v1.shape_60[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.slice_59 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_59[0][
                                                                            0]']                             
 tf.__operators__.getitem_60 (  ()                               0          ['tf.compat.v1.shape_62[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.slice_61 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_61[0][
                                                                            0]']                             
 tf.__operators__.getitem_62 (  ()                               0          ['tf.compat.v1.shape_64[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.slice_63 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_63[0][
                                                                            0]']                             
 tf.__operators__.getitem_64 (  ()                               0          ['tf.compat.v1.shape_66[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.compat.v1.shape_67 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_68 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_69 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_70 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.compat.v1.shape_71 (TFOpLa  (2,)                             0          ['decoder_output_linear[0][0]']  
 mbda)                                                                                                       
 tf.__operators__.getitem_70 (  ()                               0          ['tf.compat.v1.shape_72[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.__operators__.getitem_71 (  ()                               0          ['tf.compat.v1.shape_73[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.__operators__.getitem_72 (  ()                               0          ['tf.compat.v1.shape_74[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.__operators__.getitem_73 (  ()                               0          ['tf.compat.v1.shape_75[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.__operators__.getitem_74 (  ()                               0          ['tf.compat.v1.shape_76[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.math.tanh_10 (TFOpLambda)   (None, 1)                        0          ['tf.slice_55[0][0]']            
 tf.slice_56 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_56[0][
                                                                            0]']                             
 tf.math.tanh_11 (TFOpLambda)   (None, 1)                        0          ['tf.slice_57[0][0]']            
 tf.slice_58 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_58[0][
                                                                            0]']                             
 tf.math.tanh_12 (TFOpLambda)   (None, 1)                        0          ['tf.slice_59[0][0]']            
 tf.slice_60 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_60[0][
                                                                            0]']                             
 tf.math.tanh_13 (TFOpLambda)   (None, 1)                        0          ['tf.slice_61[0][0]']            
 tf.slice_62 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_62[0][
                                                                            0]']                             
 tf.math.tanh_14 (TFOpLambda)   (None, 1)                        0          ['tf.slice_63[0][0]']            
 tf.slice_64 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_64[0][
                                                                            0]']                             
 tf.__operators__.getitem_65 (  ()                               0          ['tf.compat.v1.shape_67[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.__operators__.getitem_66 (  ()                               0          ['tf.compat.v1.shape_68[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.__operators__.getitem_67 (  ()                               0          ['tf.compat.v1.shape_69[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.__operators__.getitem_68 (  ()                               0          ['tf.compat.v1.shape_70[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.__operators__.getitem_69 (  ()                               0          ['tf.compat.v1.shape_71[0][0]']  
 SlicingOpLambda)                                                                                            
 tf.slice_70 (TFOpLambda)       (None, 3)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_70[0][
                                                                            0]']                             
 tf.slice_71 (TFOpLambda)       (None, 4)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_71[0][
                                                                            0]']                             
 tf.slice_72 (TFOpLambda)       (None, 4)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_72[0][
                                                                            0]']                             
 tf.slice_73 (TFOpLambda)       (None, 4)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_73[0][
                                                                            0]']                             
 tf.slice_74 (TFOpLambda)       (None, 4)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_74[0][
                                                                            0]']                             
 tf.math.add_22 (TFOpLambda)    (None, 1)                        0          ['tf.math.tanh_10[0][0]']        
 tf.math.softplus_5 (TFOpLambd  (None, 1)                        0          ['tf.slice_56[0][0]']            
 a)                                                                                                          
 tf.math.add_23 (TFOpLambda)    (None, 1)                        0          ['tf.math.tanh_11[0][0]']        
 tf.math.softplus_6 (TFOpLambd  (None, 1)                        0          ['tf.slice_58[0][0]']            
 a)                                                                                                          
 tf.math.add_24 (TFOpLambda)    (None, 1)                        0          ['tf.math.tanh_12[0][0]']        
 tf.math.softplus_7 (TFOpLambd  (None, 1)                        0          ['tf.slice_60[0][0]']            
 a)                                                                                                          
 tf.math.add_25 (TFOpLambda)    (None, 1)                        0          ['tf.math.tanh_13[0][0]']        
 tf.math.softplus_8 (TFOpLambd  (None, 1)                        0          ['tf.slice_62[0][0]']            
 a)                                                                                                          
 tf.math.add_26 (TFOpLambda)    (None, 1)                        0          ['tf.math.tanh_14[0][0]']        
 tf.math.softplus_9 (TFOpLambd  (None, 1)                        0          ['tf.slice_64[0][0]']            
 a)                                                                                                          
 tf.slice_65 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_65[0][
                                                                            0]']                             
 tf.slice_66 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_66[0][
                                                                            0]']                             
 tf.slice_67 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_67[0][
                                                                            0]']                             
 tf.slice_68 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_68[0][
                                                                            0]']                             
 tf.slice_69 (TFOpLambda)       (None, 1)                        0          ['decoder_output_linear[0][0]',  
                                                                             'tf.__operators__.getitem_69[0][
                                                                            0]']                             
 tf.math.tanh_15 (TFOpLambda)   (None, 3)                        0          ['tf.slice_70[0][0]']            
 tf.math.tanh_16 (TFOpLambda)   (None, 4)                        0          ['tf.slice_71[0][0]']            
 tf.math.tanh_17 (TFOpLambda)   (None, 4)                        0          ['tf.slice_72[0][0]']            
 tf.math.tanh_18 (TFOpLambda)   (None, 4)                        0          ['tf.slice_73[0][0]']            
 tf.math.tanh_19 (TFOpLambda)   (None, 4)                        0          ['tf.slice_74[0][0]']            
 tf.math.truediv_10 (TFOpLambd  (None, 1)                        0          ['tf.math.add_22[0][0]']         
 a)                                                                                                          
 tf.clip_by_value_17 (TFOpLamb  (None, 1)                        0          ['tf.math.softplus_5[0][0]']     
 da)                                                                                                         
 tf.math.truediv_11 (TFOpLambd  (None, 1)                        0          ['tf.math.add_23[0][0]']         
 a)                                                                                                          
 tf.clip_by_value_18 (TFOpLamb  (None, 1)                        0          ['tf.math.softplus_6[0][0]']     
 da)                                                                                                         
 tf.math.truediv_12 (TFOpLambd  (None, 1)                        0          ['tf.math.add_24[0][0]']         
 a)                                                                                                          
 tf.clip_by_value_19 (TFOpLamb  (None, 1)                        0          ['tf.math.softplus_7[0][0]']     
 da)                                                                                                         
 tf.math.truediv_13 (TFOpLambd  (None, 1)                        0          ['tf.math.add_25[0][0]']         
 a)                                                                                                          
 tf.clip_by_value_20 (TFOpLamb  (None, 1)                        0          ['tf.math.softplus_8[0][0]']     
 da)                                                                                                         
 tf.math.truediv_14 (TFOpLambd  (None, 1)                        0          ['tf.math.add_26[0][0]']         
 a)                                                                                                          
 tf.clip_by_value_21 (TFOpLamb  (None, 1)                        0          ['tf.math.softplus_9[0][0]']     
 da)                                                                                                         
 tf.math.multiply_35 (TFOpLamb  (None, 1)                        0          ['tf.slice_65[0][0]']            
 da)                                                                                                         
 tf.math.multiply_36 (TFOpLamb  (None, 1)                        0          ['tf.slice_66[0][0]']            
 da)                                                                                                         
 tf.math.multiply_37 (TFOpLamb  (None, 1)                        0          ['tf.slice_67[0][0]']            
 da)                                                                                                         
 tf.math.multiply_38 (TFOpLamb  (None, 1)                        0          ['tf.slice_68[0][0]']            
 da)                                                                                                         
 tf.math.multiply_39 (TFOpLamb  (None, 1)                        0          ['tf.slice_69[0][0]']            
 da)                                                                                                         
 tf.math.multiply_40 (TFOpLamb  (None, 3)                        0          ['tf.math.tanh_15[0][0]']        
 da)                                                                                                         
 tf.math.multiply_41 (TFOpLamb  (None, 4)                        0          ['tf.math.tanh_16[0][0]']        
 da)                                                                                                         
 tf.math.multiply_42 (TFOpLamb  (None, 4)                        0          ['tf.math.tanh_17[0][0]']        
 da)                                                                                                         
 tf.math.multiply_43 (TFOpLamb  (None, 4)                        0          ['tf.math.tanh_18[0][0]']        
 da)                                                                                                         
 tf.math.multiply_44 (TFOpLamb  (None, 4)                        0          ['tf.math.tanh_19[0][0]']        
 da)                                                                                                         
 tf.concat_5 (TFOpLambda)       (None, 2)                        0          ['tf.math.truediv_10[0][0]',     
                                                                             'tf.clip_by_value_17[0][0]']    
 tf.concat_6 (TFOpLambda)       (None, 2)                        0          ['tf.math.truediv_11[0][0]',     
                                                                             'tf.clip_by_value_18[0][0]']    
 tf.concat_7 (TFOpLambda)       (None, 2)                        0          ['tf.math.truediv_12[0][0]',     
                                                                             'tf.clip_by_value_19[0][0]']    
 tf.concat_8 (TFOpLambda)       (None, 2)                        0          ['tf.math.truediv_13[0][0]',     
                                                                             'tf.clip_by_value_20[0][0]']    
 tf.concat_9 (TFOpLambda)       (None, 2)                        0          ['tf.math.truediv_14[0][0]',     
                                                                             'tf.clip_by_value_21[0][0]']    
 tf.math.sigmoid_5 (TFOpLambda  (None, 1)                        0          ['tf.math.multiply_35[0][0]']    
 )                                                                                                           
 tf.math.sigmoid_6 (TFOpLambda  (None, 1)                        0          ['tf.math.multiply_36[0][0]']    
 )                                                                                                           
 tf.math.sigmoid_7 (TFOpLambda  (None, 1)                        0          ['tf.math.multiply_37[0][0]']    
 )                                                                                                           
 tf.math.sigmoid_8 (TFOpLambda  (None, 1)                        0          ['tf.math.multiply_38[0][0]']    
 )                                                                                                           
 tf.math.sigmoid_9 (TFOpLambda  (None, 1)                        0          ['tf.math.multiply_39[0][0]']    
 )                                                                                                           
 r_layer_5 (RLayer)             (None, 3)                        0          ['tf.math.multiply_40[0][0]']    
 r_layer_6 (RLayer)             (None, 4)                        0          ['tf.math.multiply_41[0][0]']    
 r_layer_7 (RLayer)             (None, 4)                        0          ['tf.math.multiply_42[0][0]']    
 r_layer_8 (RLayer)             (None, 4)                        0          ['tf.math.multiply_43[0][0]']    
 r_layer_9 (RLayer)             (None, 4)                        0          ['tf.math.multiply_44[0][0]']    
 concatenate_1 (Concatenate)    (None, 34)                       0          ['tf.concat_5[0][0]',            
                                                                             'tf.concat_6[0][0]',            
                                                                             'tf.concat_7[0][0]',            
                                                                             'tf.concat_8[0][0]',            
                                                                             'tf.concat_9[0][0]',            
                                                                             'tf.math.sigmoid_5[0][0]',      
                                                                             'tf.math.sigmoid_6[0][0]',      
                                                                             'tf.math.sigmoid_7[0][0]',      
                                                                             'tf.math.sigmoid_8[0][0]',      
                                                                             'tf.math.sigmoid_9[0][0]',      
                                                                             'r_layer_5[0][0]',              
                                                                             'r_layer_6[0][0]',              
                                                                             'r_layer_7[0][0]',              
                                                                             'r_layer_8[0][0]',              
                                                                             'r_layer_9[0][0]']              
=============================================================================================================
Total params: 7884 (30.80 KB)
Trainable params: 7884 (30.80 KB)
Non-trainable params: 0 (0.00 Byte)
_____________________________________________________________________________________________________________</code></pre>
</details>


Based on the output above it is clear that decoder takes a
sample from the latent space with the correct dimension of 10, followed by
dense hidden layers with 100 and 50 nodes.

Now that the decoder skeleton is built, the learned weights need to be
applied.


``` r
decoder %>% set_weights(weights_decoder)
```

Now, the decoder is built with the learned weights.

**Step Four: Sample the latent space or prior.**

Now that the VAE is built, the user can either sample from the chosen
prior, or sample from the learned posterior (the latent space). In this
example, a sample from the prior will be used, in this case N(0, 1).


``` r
sample_prior <- matrix(rnorm(nrow(data) * 10),
                      nrow = nrow(data), ncol = 10)
```

This latent sample can be used in step 5.

If the user desired to sample directly from the latent space, or to
visualize the latent space from a latent sample, the steps below could be
used.


``` r
input_data <- as.matrix(data)
latent_space = predict(latent_encoder, input_data)
#> 313/313 - 1s - 562ms/epoch - 2ms/step
dim(latent_space[[1]])
#> [1] 10000    10
dim(latent_space[[2]])
#> [1] 10000    10
```

The output in the console shows 313 due to our batch size of 32 over
10,000 samples (10,000/32 ≈ 313). This is an automatic output from Keras by
using the function predict.

The resulting latent_space is a list with two objects, one that represents
the mean and one that represents that log variance that are learned
through the encoder. Each one has a dimension of 10,000 by 10. This is the
size of the data by the chosen latent dimension.


``` r
z_mean = latent_space[[1]]
z_log_var = latent_space[[2]]
```

Now that the latent space is separated into its respective parts the
reparameterization trick can be applied, as implemented in
`Latent_sample()`.


``` r
sample_latent_fromposterior = as.matrix(Latent_sample(z_mean, z_log_var))
```

**Step Five: Pass the sampled latent vectors through the trained decoder.**

This step will use the sample from the latent space and put it through the
trained decoder to create a generative output.


``` r
decoder_sample = predict(decoder, sample_prior)
#> 313/313 - 1s - 923ms/epoch - 3ms/step
```

decoder_sample now has the generative parameters from the trained VAE.

The dimension of this dataset is the total sample size by the decoder
output layer size (in this example 10000 by 34). In our example, the input
data has 29 columns, 5 of which are continuous. The decoder outputs 2
columns for each continuous variable, a mean and SD. Hence, there are 5
additional columns in the decoder output.

## Sampling from the generative results

This section discusses possible options for creating the synthetic dataset
based on the generative results from the decoder. Some recommendations are
provided for AutoTab; however, the final choice is up to the user, and
multiple approaches are possible. If additional preprocessing steps were
applied to the data, they must be reversed accordingly. For this reason,
the generative step is application dependent, and careful consideration
should be given when implementing it.

The order of the generative sample will match the order of feat_dist. Based
on the previous output of feat_dist, the first 5 variables were
continuous. That means the first 10 columns belong to the continuous
variables. To generate a sample from each output we sample from the
following distribution:

$$x \sim N\left(\mu_\theta(z), \sigma^2_\theta(z)\right),$$

where $\theta$ are the parameters of the decoder and $z$ is the latent
sample. The parameters for the distribution are within the R object
decoder_sample created in Step Five.

The generative parameters for a continuous variable are the mean and SD.
The user may decide to sample from a distribution using these values or
first undo the min-max-scaling and sample from that distribution. In this
example, using the variable age, we first undo the min-max-scaling and
then sample.

Let:

$$x = R * x_{scaled} + min,$$

where $R = max - min$, min is the minimum of age in the original data, max
is the maximum of age in the original data, and $x$ is the unscaled
decoder output. Then:

$$E[x] = E[R * x_{scaled} + min] = R \, E[x_{scaled}] + min$$

and

$$Var(x) = Var(R * x_{scaled} + min) = R^2 Var(x_{scaled})$$

so,

$$\sigma = R * \sigma_{scaled}, \quad \mu = R * \mu_{scaled} + min$$

The code to implement this, on the variable age as an illustrative
example, is shown below.


``` r
scaling_min = min(data_example$age)
scaling_max = max(data_example$age)
decoder_sample[,1] = (decoder_sample[,1] * (scaling_max - scaling_min)
                      + scaling_min)
decoder_sample[,2] = decoder_sample[,2] * (scaling_max - scaling_min)
```

Now the first row of age can be sampled from N(mean, SD²), the unscaled
generative Gaussian distribution using the following code:


``` r
decoder_sample[1, 1] = rnorm(1, mean = decoder_sample[1,1],
                               sd = decoder_sample[1,2])
```

For binary variables, the user can decide to either sample from a
Bernoulli distribution or choose the maximum a posteriori (MAP). A MAP
estimate corresponds to the most probable class label given the
probabilistic output [@murphyMachineLearningProbabilistic2012]. Choosing
between these approaches is left to the user. The code below displays how
MAP can be applied to the first row of the 11th column, which is a binary
variable based on feat_dist.


``` r
decoder_sample[1, 11] = ifelse(decoder_sample[1, 11] > 0.5, 1, 0)
decoder_sample[1, 11]
#> [1] 0
```

The code above takes the predicted value and maps it to 1 since the
probability was greater than 0.5.

For categorical variables, the user can choose to sample from a
multinomial distribution with one draw based on the column probabilities
or use another MAP approach.

Choosing between another generative sample from the categorical
distribution versus taking the most probable category from the decoder
output is again a design choice and is up to the user. The code below
displays how argmax can be applied to the first row of columns 16–18,
which represents a categorical variable based on feat_dist.


``` r
decoder_sample[1, 16:18] = as.integer(decoder_sample[1, 16:18] ==
                                            max(decoder_sample[1, 16:18]))
```

After applying argmax, the column will indicate the one-hot-coded vector.

# Conclusion

This vignette walked through a complete AutoTab workflow: simulating and
preprocessing mixed-type tabular data, setting up the required Python/
TensorFlow environment, training a variational autoencoder under five
different hyperparameter configurations, and rebuilding a trained encoder
and decoder to sample synthetic observations. The five training examples
demonstrated core AutoTab features including weighted reconstruction loss,
Beta annealing, Gumbel-softmax temperature annealing, and regularization
options, though they are not intended to represent an optimized model for
this dataset.

For guidance on Mixture-of-Gaussians priors, see the `mog_prior` help
page. For further details on any individual function discussed here,
consult the corresponding function documentation (e.g., `?VAE_train`,
`?Encoder_weights`, `?Latent_sample`).


# References
