Part 1

The Big Picture

🎨 Generative Modeling (The "Creator")

Instead of just looking at pictures and learning to slap a label on them, this AI studies thousands of examples to learn the actual "recipe" of the data. Once it knows the recipe, it can generate brand-new, realistic examples that have never existed before.

🕵️ Unsupervised Learning

Learning without an Answer Key! The model does not use human-provided labels to learn. It figures out the hidden rules and structures entirely on its own just by observing the raw data.

The Core Capabilities

  • Density Estimation: Figures out the "hidden ingredients" to understand the overall shape and spread of the data.
  • Sample Generation: Using the learned "recipe" to actually create brand-new, realistic data instances.
  • Outlier Detection: Spotting highly unusual events (like a hazard for a self-driving car) because it knows what "normal" looks like.
  • Data Debiasing: Tells if certain features are under-represented, allowing humans to fix data and build fairer systems.
Model Families

The Branches of Generative Modeling

Click any model box below to jump straight to its detailed architecture!

Generative Modeling
(Unsupervised Learning)
LATENT VARIABLE MODELS ADVERSARIAL MODELS
Focus: Density Estimation & Interpretable Features
Click to View
1. Autoencoder (AE) Deterministic compression into a latent bottleneck.
Goal: Perfect Reconstruction.
Click to View
2. Variational Autoencoder (VAE) Probabilistic latent space (μ, σ).
Goal: Reconstruction + Generating New Samples.
Focus: Pure, High-Fidelity Sample Generation
Click to View
3. Generative Adversarial Network (GAN) Noise mapping via Generator vs. Discriminator.
Goal: Fool the Discriminator.
Click to View
4. Cycle GAN 2x Generators & 2x Discriminators.
Goal: Unpaired Style Translation.
Adds stochasticity (randomness) & regularization
to allow for brand-new data generation
Doubles the architecture and adds Cyclic Loss
to translate directly between two unlinked domains
Shifts focus away from interpretable latent variables toward pure generation from random noise
Part 3

Deep Generative Modeling Use Cases

Hover over any Use Case node (UC) below to view its explanation.

Deep Generative Modeling System Latent Variable Models Adversarial Models Core Applications 👨‍💻 AI Developer 🤖 Downstream System Isolate randomness into a dice roll to allow predictable math for backprop. UC9: 🎲 Reparameterization Trick Keep the latent space organized with continuity and completeness. UC8: 📐 Apply KL Divergence Train by checking how different the redrawn picture is from the original. UC7: 📉 Minimize Recon Loss Compress input data into a tiny bottleneck representation. UC5: 🗜️ Encode to Latent Space Redraw the original data from the compressed latent summary. UC6: 🖼️ Decode from Latent Endless cat-and-mouse game between forger and detective. UC12: ⚔️ Optimize Competing Obj. Look at fake and real pictures to guess which is which. UC11: 🕵️ Classify Real vs. Fake Start with random noise and paint a fake picture. UC10: 🎭 Generate Fake Data Translate data directly from one style to another without matched pairs. UC13: 🔄 Translate Style (Cycle) Spot highly unusual or rare events by knowing what normal looks like. UC3: 🎯 Detect Outliers Identify under-represented features to build fairer AI systems. UC4: ⚖️ Debias Data Learn the overall shape and spread of the data distribution. UC1: 📊 Density Estimation Use the learned recipe to create brand-new, realistic data instances. UC2: 🎨 Generate New Samples
Part 4

The Foundation: Autoencoders

🦇

Latent Variables (Plato's Cave)

Imagine you are in a cave and can only see shadows moving on the wall. The shadows are the data we observe (like pixels in an image), but the latent variables are the actual, hidden 3D objects casting those shadows.

Method 1: Autoencoders (The Summarizer)

The architecture of compressing data into a hidden representation and forcing the network to redraw it perfectly.

1. Autoencoder Architecture
Input Data (X)
 Encoder
Latent Space (Z)
Low-dimensional bottleneck
 Decoder
Reconstructed Output
Reconstruction Loss
Mean Squared Error = (X - Output)²
Part 5

Method 2: Variational Autoencoders (VAEs)

The Flexible Summarizer: A mathematical evolution designed to generate brand new data.

🎲

Stochasticity (Adding "Wiggle Room")

Instead of a single, rigid summary, it learns a flexible range defined by a mean and a standard deviation.

🗂️

Regularization (The Organizer)

Forces the latent space to have Continuity and Completeness (KL Divergence).

🧮

The Reparameterization Trick

Isolates randomness into a "dice roll" constant (epsilon), keeping calculations smooth for backpropagation.

2. Variational Autoencoder (VAE) Architecture
Input Data (X)
Encoder Network
Mean (μ)
Std Dev (σ)
Random Constant
Epsilon (ε)
Latent Variable (Z)
Z = μ + (σ × ε)
Decoder Network
Reconstructed Output
Regularization Term
KL Divergence vs Prior (μ=0, σ=1)
Reconstruction Loss
Total Loss = Recon Loss + Reg Term
Part 6

Method 3: GANs & Cycle GANs

🎭

GANs

The Forger vs. The Detective

  • The Generator

    Starts with completely random noise and tries to paint a realistic fake picture.

  • The Discriminator

    Looks at both fake and real pictures, and tries to classify which is which.

🦓

Cycle GANs

The Unpaired Translators

Two pairs of generators/discriminators working together to translate data directly from one style to another (e.g. horses to zebras).

🐎⇆🦓
3. Generative Adversarial Network (GAN) Architecture
Random Gaussian Noise
Generator Network
Fake Data
Real Data (X)
Discriminator Network
Classification:
Real vs. Fake
Generator Objective:
Minimize fake ID prob
Discriminator Objective:
Maximize correct real/fake
 4. Cycle GAN Architecture (Unpaired Translation)
Domain A
(e.g., Horse)
Generator A→B
Fake Domain B
Generator B→A
Reconstructed
Domain A
Discriminator B
Domain B
(e.g., Zebra)
Generator B→A
Fake Domain A
Generator A→B
Reconstructed
Domain B
Discriminator A
Cyclic Loss
Code Dependency Linking Both Domains
Learning Path

Generative Modeling Sequence Flow

Instructor Core Concepts Autoencoders VAEs GANs 1. Introduce Generative Modeling Goal: Estimate probability distribution P(X) 2. Explain Latent Variables (Z) Plato's Cave Analogy: Observe raw data (X) to uncover hidden underlying features (Z) 3. Introduce Autoencoders (Deterministic) Encoder: maps X → Z (Bottleneck) Decoder: maps Z → Recon X 4. Define Reconstruction Loss Train by minimizing distance: Mean Squared Error = (Input - Reconstructed Output)² 5. Add Stochasticity for Generation (VAEs) Parameterize Z as a probability distribution defined by mean (μ) and standard deviation (σ) 6. Define Full VAE Loss Function Total Loss = Reconstruction Loss + Regularization Term 7. Explain Regularization Compare to a normal prior (μ=0, σ=1) Formula: KL Divergence (Ensures Continuity & Completeness) 8. Address Backprop Roadblock Issue: Backpropagation requires deterministic nodes. 9. Introduce Reparameterization Trick Isolate randomness into constant epsilon (ε). Formula: Z = μ + (σ × ε) 10. Introduce GANs Generator: Maps Noise → Fake Discriminator: Classifies Real vs. Fake X 11. Define Competing Objective Discriminator: Maximize correct classifications. Generator: Minimize fake ID. 12. Expand to Cycle GANs Two sets of Gen & Disc. Linked by Cyclic Loss for unpaired style translation.
History

Evolution of Generative Modeling

📜
Pre-1980s

Foundational Statistics

Latent Variables, Density Estimation, and Outlier Detection. Foundational statistical concepts that have existed for decades.

🗜️
1986

Autoencoders

The concept of using neural networks to mathematically compress and reconstruct data unsupervised, popularized by Geoffrey Hinton.

🎲
2013

Variational Autoencoders (VAEs)

Introduced by Kingma and Welling, bringing the crucial "probabilistic twist" that allowed models to smoothly generate brand new data.

⚔️
2014

Generative Adversarial Networks

The "forger vs. detective" framework famously invented by Ian Goodfellow, kicking off a massive boom in high-quality image generation.

🔄
2017

Cycle GANs

Architecture using pairs of generators and discriminators for unpaired style translation, introduced by Jun-Yan Zhu at UC Berkeley.

Eyes permanently on the wall

Interactive Sandbox

Loading...

Loading...

Loading Example...

💡 Frequently Asked Questions (Generative Modeling)

What is the main difference between supervised and unsupervised learning?▼
Supervised learning relies on a dataset where data comes with explicit human-provided labels. Unsupervised learning uses raw data without any labels. Generative modeling is unsupervised.
How does the "Plato's Cave" analogy explain latent variables?▼
Prisoners can only see shadows moving on a wall (the observable data we see, like pixels). The true 3D objects casting shadows represent the latent variables, the hidden features governing the data distribution.
Why do Variational Autoencoders (VAEs) use the Reparameterization Trick?▼
VAEs introduce randomness. This pure randomness breaks the predictable math required for backpropagation. The trick solves this by isolating randomness into a separate constant called epsilon.
How do Generative Adversarial Networks (GANs) learn to create realistic data?▼
GANs employ a Generator (the "forger") and a Discriminator (the "detective"). The competing objective forces the generator to continuously improve its fakes until it can completely fool the discriminator.
If the GAN "Discriminator" judges real vs. fake, isn't that just supervised learning? How does it get its knowledge?▼
It feels like supervised learning because the Discriminator acts as a classifier, but the key difference is that no human had to manually label the data. The Discriminator gets its knowledge automatically by comparing two distinct buckets: the raw training dataset (which the network automatically tags as "Real") and the output coming from the Generator (which the network automatically tags as "Fake"). It is a clever trick that pits two neural networks against each other to achieve an unsupervised goal without needing a human "Watchman" .
Does training stop when the fake data perfectly matches the real data and fools the Discriminator?▼
Yes! In the lecture, this is referred to as the "Global Optimum". The two networks compete in a loop until the Generator perfectly reproduces the true data distribution. Visually, this means the generated fake data exactly overlaps the locations of the real data. At this point, the Discriminator is completely fooled and can no longer tell the difference, meaning the Generator has successfully learned how to create perfect new samples and training has achieved its goal.