A variational autoencoder (VAE) generates synthetic tabular data by compressing each record into a probabilistic latent space, then decoding that representation into a reconstruction or new example. section 2.3.3
Synthetic data generated with VAEs can help teams:
- Expand scarce datasets for analysis or model development. section 2.3.3
- Increase minority-class diversity in unbalanced classification problems. section 2.3.3
- Create rare edge cases for system testing while avoiding direct exposure of represented records. section 2.3.3
The encoder condenses input data into a latent space, and the decoder reconstructs it with high fidelity. Effective reconstruction suggests that the latent representation captures core distributional information, enabling new examples. section 2.3.3
Generating synthetic credit records
With SDV, load a German credit dataset, detect and refine its metadata, fit a TVAESynthesizer for 10,000 epochs, and sample 10,000 synthetic rows. section 2.3.3
Listing 2.5 Generating a synthetic dataset
import pandas as pd
from sdv.metadata import SingleTableMetadata #1
from sdv.single_table import TVAESynthesizer #2
data = pd.read_csv("./german_credit_data.csv", index_col=0) #3
data = data.reset_index()
print(data.shape)
print(data.head)
metadata = SingleTableMetadata() #4
metadata.detect_from_dataframe(data) #5
print(metadata.to_dict()) #6
metadata.update_column(column_name="Saving accounts",
sdtype="categorical") #7
synthesizer = TVAESynthesizer(metadata, epochs=10_000) #8
synthesizer.fit(data) #9
synthetic_data = synthesizer.sample(num_rows=10_000) #10
synthetic_data.head()
Runnable Python listing for fitting a TVAESynthesizer and sampling a synthetic tabular dataset. section 2.3.3
