Most explanations of AI image generation stop at "it learns from millions of pictures," which is true but explains almost nothing about what actually happens when you type a prompt and an image appears seconds later. The real mechanism is stranger and more interesting than that, and understanding it explains most of the quirks people notice, why hands look wrong, why text used to be garbled, why two generations of the same character never quite match.
It starts with pure noise, not a blank canvas
Most image generators, including Stable Diffusion and Midjourney, are diffusion models. During training, the model is shown real images with random noise added to them in small increments, repeated until the image is statistically indistinguishable from static. Generation runs that process backward. It starts from pure random noise and, at each of roughly 20 to 50 steps, a neural network looks at the current noisy image and predicts exactly what noise is present in it, then subtracts a little of that noise out. Repeat that several dozen times and a coherent image emerges from what began as static. The model never learns to draw in one shot, because mapping noise directly to a finished picture is too hard a problem. Mapping slightly noisy to slightly less noisy, over and over, is a much easier one.
The image never touches real pixels until the very last step
Modern systems do not do this noise removal on the full-size image. They first compress it into a much smaller mathematical grid, for example shrinking a 512 by 512 pixel image down to a 64 by 64 grid, using a component called a Variational Autoencoder. Every step of denoising happens in that compressed space, called latent space, which is not a mystical concept, it is simply a shorthand representation that keeps the overall structure and content of the image while discarding the fine pixel-level noise a human eye does not need to identify what is in a picture. Only at the very end does a decoder expand that compressed result back into a full image. This is why it is called latent diffusion, and it is also why it is dramatically cheaper to run than the earliest image generators, which worked in full pixel space the whole time.
How your text prompt actually steers the image
Your prompt does not get typed into the picture, it gets converted into numbers first, through a system called CLIP, originally built by OpenAI. CLIP trains two separate encoders, one for images and one for text, on hundreds of millions of image and caption pairs, using a training method that pulls matching pairs closer together in a shared mathematical space and pushes mismatched pairs apart. The end result is a space where the text "a photo of a dog" and an actual photo of a dog land near each other numerically. During generation, your prompt's position in that space is fed into the denoising network at every single step, nudging each round of noise removal toward whatever would make the image look more like your words. There is a second, less visible mechanism doing a lot of the work too. During training, the text instruction is randomly hidden some of the time, so the model also learns what an image looks like with no guidance at all. At generation time, it runs both versions, guided and unguided, and extrapolates away from the unguided version toward the guided one. That is the actual math behind the "guidance scale" or "CFG" slider you see in most tools, push it too high and the image gets more literal but starts looking artificial, too low and it looks better but starts ignoring your prompt.
Not every image generator is a diffusion model anymore
This is a genuine 2026 architecture shift worth knowing. OpenAI's image generator behind ChatGPT is not a diffusion model at all, it is autoregressive, generating an image token by token in a single pass, architecturally closer to how a language model writes a sentence one word at a time than to how Midjourney paints from noise. That difference is exactly why it renders legible text inside images so much more reliably, a diffusion model treats letters as visual shapes to imitate, while a token-based model treats them as symbols it is actually composing. Separately, most frontier diffusion models themselves changed their internal architecture too, moving away from the convolutional networks used in the original Stable Diffusion toward transformer-based backbones, the same core architecture behind large language models, because they scale better and, in newer designs, let text and image information cross-reference each other directly during generation rather than the text just sitting on the sidelines as a steering signal.
Why hands are still hard, and why it is not really a bug
Hands show up in training photos far less cleanly than faces do, they are frequently small in the frame, partially hidden by objects, or cropped at the edge of a photo, so the model has proportionally much weaker signal to learn hand structure from. On top of that, no mainstream generator does any actual skeletal or biomechanical simulation, a generated hand is a learned visual texture, not a simulated jointed structure, so nothing internally enforces the rule that a hand has exactly five fingers with fixed relative proportions the way a 3D animation rig would. It has genuinely improved as training data and model architectures have gotten better, but it remains a soft, learned pattern rather than a hard, guaranteed constraint, which is the same underlying reason shadows and reflections in AI images often do not physically line up, no mainstream tool performs true light simulation, so a shadow's direction and a reflection's contents are also learned approximations rather than calculated physics.
The other quirk with the same root cause: characters that will not stay the same
Ask most image generators for the same character twice and small details drift, the exact shade of a jacket, the precise face shape, the pattern on a background object. The reason is structural: diffusion models are stateless, every single generation samples fresh from a probability distribution with no memory of what was generated a moment before. The common fixes, feeding the model several reference images of the same subject, or fine-tuning a small custom add-on specifically on one character, are all external workarounds bolted onto a fundamentally memoryless system, not something the architecture does natively.
The takeaway
None of this is magic, and very little of it is a mystery anymore. An AI image generator is a system that learns to remove noise in small, repeated steps, guided by a mathematical translation of your words, running either in a compressed shorthand of the image or, increasingly, as a sequence of tokens generated one at a time. The specific things people notice as "AI-looking," slightly wrong hands, mismatched shadows, a character that will not stay consistent across images, all trace back to the same handful of structural facts about how these systems actually work, not random unpredictable error.
