Generative AI spans several distinct model families, each suited to a different kind of content and built on different underlying architectures.
Text Generation — Large Language Models
Transformer-based models trained to predict the next token in a sequence. Powers chatbots, summarization, code generation, and translation. See What Is an LLM?.
Image Generation — Diffusion Models
Trained to reverse a gradual noising process — starting from random noise and iteratively "denoising" toward a coherent image guided by a text prompt. This is the architecture behind most modern text-to-image tools.
Image Generation — GANs (Generative Adversarial Networks)
An older approach (introduced 2014) using two competing networks — a generator that creates images and a discriminator that tries to tell real from fake. GANs were foundational to generative image AI but have largely been superseded by diffusion models for most modern text-to-image tools, mainly due to more stable training and better output diversity.
Audio Generation
Text-to-speech models convert written text into natural-sounding speech; some newer models also generate music or sound effects from text descriptions. See Audio AI.
Video Generation
An extension of image-generation techniques across the time dimension — computationally far more expensive than image generation, and as of today generally more limited in coherence over longer durations. See Video Generation.
Code Generation
Not a separate architecture — typically an LLM trained (or fine-tuned) with a large proportion of source code in its training data, giving it strong performance on programming tasks specifically.
Comparison Table
| Type | Typical Architecture | Example Output |
|---|---|---|
| Text | Transformer (decoder-only) | Chat responses, articles, code |
| Images | Diffusion model | Text-to-image art, product mockups |
| Audio | Specialized transformer/diffusion variants | Speech, music |
| Video | Diffusion model extended over time | Short video clips |
Common Mistakes
- Assuming one architecture (usually "transformers") explains all of generative AI — image and audio generation frequently rely on diffusion-based approaches, not just transformers
- Treating GANs as obsolete in every context — they still see use in some specialized applications, even though diffusion models dominate mainstream text-to-image tools
Interview Relevance
"What's the difference between how an LLM generates text and how a diffusion model generates images?" tests whether a candidate understands generative AI as a family of techniques, not a single algorithm.
Practice Question
A startup wants to generate both marketing copy and product mockup images from a single product description. What two different types of generative models would this likely require?