Model collapse is the failure mode in which generative AI models, trained on data produced by earlier generative models rather than on original human-made content, progressively lose quality and diversity. It was formalized in a 2024 Nature paper, “AI models collapse when trained on recursively generated data,” which found that “indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear.”
The intuition is straightforward. A model is an approximation of the data it was trained on; it slightly over-represents common patterns and under-represents rare ones. Train a new model on the first model’s output, and that bias compounds: the rare cases get sampled even less, errors accumulate, and after several generations the models converge toward a bland, low-variety output that no longer resembles the original distribution. The Nature authors showed this is not unique to language models - it also occurs in variational autoencoders and simple statistical models - so it appears to be a general hazard of recursive training.
The problem is becoming practical rather than theoretical because the open web, the primary source of training data, is filling with AI-generated text and images. A model that scrapes the internet a few years from now will inevitably ingest the output of earlier models, raising the risk of collapse unless developers can identify and prioritize authentic human data. This is why “synthetic data” is double-edged: carefully generated synthetic data can be useful for training, but unlabeled model output leaking back into the training pool is corrosive.
Why business readers should care: model collapse explains why data provenance is becoming a strategic concern, not a compliance afterthought. Organizations sitting on large stores of genuine human-generated data may find that asset appreciating precisely because clean, non-synthetic data is getting scarcer.