Image generation

Autoencoders, GANs, diffusion: compress, forge, denoise

In this chapter, you will learn to

• Compress and rebuild data (autoencoders)
• Understand how a model generates new images (GANs, diffusion)
• Evaluate what a generative model has learned

Tools: PyTorch

5 exercises

Each exercise is graded automatically, in your browser. The full statements are available once you have an account.

1. What this is for in real life

   Recognizing a cat is one thing; drawing a cat that never existed is another. Generative models learn to produce new, believable data, not just to label it. - Images from text. Stable Diffusion (2022) and DALL·E…

2. The compressor

   You work at a space agency. The probes send back 64-pixel images, but bandwidth is limited: you can only transmit 4 numbers per image. You must compress, then reconstruct. The problem: how do you fit 64 values…

3. The Forger

   Your compressor works. Now, a bold idea: could we invent new data that does not exist? You draw a random vector in the compressed space and decode it. Result: noise. Random vectors fall into the « holes » of the…

4. The duel of the networks

   Your autoencoder compresses and reconstructs. Here is another way to generate: make two networks fight each other (a GAN). - The generator starts from random noise and produces fake samples; - the…

5. Walking back up the noise

   Here is the idea behind the models that generate images today (Stable Diffusion, DALL·E): diffusion. You start from real data that you progressively corrupt with noise in 10 steps, until it is pure noise. Then…