Every AI system is hungry for examples, and the real world is a slow, expensive, privacy-encumbered supplier. Synthetic data is the practice of manufacturing those examples instead — and doing it without poisoning the model you are trying to improve.
Data became the bottleneck
For most of the deep learning era, the recipe for a better model was simple: more parameters, more compute, more data. The first two are purchasable. The third has quietly become the binding constraint. The high-quality text of the public internet has largely been consumed by pretraining runs, labeled data for specialized domains was never abundant in the first place, and the data you most want — examples of your exact task, in your exact domain, with your exact edge cases — frequently does not exist at all. You cannot download a corpus of customer conversations about a product you launched last quarter, or a thousand examples of an agent correctly recovering from a tool failure that has only happened twice.
Synthetic data is the response to that constraint: instead of finding examples, you manufacture them. A model, a simulator, or a rule-based generator produces the inputs, the outputs, or both, and those manufactured examples are used to train, tune, or test a system. What was once a niche trick for balancing datasets has become a central technique in how modern AI systems are built, and understanding how to do it well — and where it quietly fails — is now a core engineering skill.
What synthetic data actually is
The term covers a family of techniques that share one property: the examples were generated by a process you control rather than collected from the world. At one end sit fully synthetic datasets, where both inputs and labels are manufactured — a language model inventing customer questions and writing the ideal answers. In the middle sits augmentation, where real examples are transformed into variants: paraphrased, translated, perturbed, recombined. At the other end sits synthetic labeling, where real inputs are collected but a model rather than a human produces the annotations.
The generator can be another AI model, which is now the dominant pattern, but it does not have to be. Physics engines generate synthetic sensor data for robotics. Rendering pipelines generate labeled images for vision. Rule-based programs generate arithmetic problems with guaranteed-correct answers. Test harnesses generate API traffic. What unites them is that the data-generating process is inspectable and adjustable, which is exactly what real-world data collection is not. When your generator is code, you can turn the dials: more edge cases, more of the rare class, more difficulty. When your data comes from the world, you get what the world happens to give you.
A short history, compressed
Synthetic data is older than the current wave of enthusiasm suggests. Statisticians were generating synthetic microdata for census privacy in the early 1990s. SMOTE, the classic technique for oversampling minority classes by interpolating between real examples, dates to 2002. Computer vision spent a decade training on rendered scenes because pixel-perfect labels come free with the renderer. Self-driving programs built entire simulated cities because you cannot ethically collect a million examples of pedestrians stepping into traffic.
What changed with large language models is that the generator became universal. A single model can produce plausible examples of nearly any text-shaped task: questions, answers, conversations, code, reasoning chains, structured records. The marginal cost of an example collapsed from dollars — a human annotator's time — to a fraction of a cent of inference. That collapse in cost is the economic engine behind everything else in this post, and it explains why techniques that were once reserved for well-funded labs are now available to any team with an API key.
The distillation pattern
The most common industrial use of synthetic data is distillation: a large, capable, expensive model generates training examples, and a smaller, cheaper model is fine-tuned on them. The small model learns to imitate the large model's behavior on the distribution of tasks you care about, and for narrow tasks the imitation can be strikingly close at a fraction of the serving cost. A support classifier, an extraction model, a domain-specific summarizer — all of these can often be distilled into a model an order of magnitude smaller than the teacher that generated their training data.
Distillation reframes what you are paying for. Instead of paying annotators by the hour, you pay for teacher-model inference by the token, and the teacher works nights and weekends. The catch is that the student inherits the teacher's mistakes along with its skills. If the teacher is wrong about your domain eight percent of the time, your training set is eight percent poison unless you filter it. Every serious distillation pipeline is therefore really two systems: a generator and a quality gate, and the second one is where the engineering actually lives.
Generating instruction data
Modern assistants are shaped by instruction tuning, and instruction tuning runs on example conversations: a request, and the response the model should have given. Collecting these from humans is slow and expensive, so the field developed bootstrapping techniques in which a model generates its own curriculum. Starting from a small seed set of human-written tasks, the model is prompted to invent new task instructions, then to answer them, and the resulting pairs are filtered and fed back in. Each generation broadens the pool of tasks the next generation draws from.
The technique works because generating a plausible task is easier than being universally competent at tasks, and judging an answer is often easier than producing it. But bootstrapped instruction data has characteristic weaknesses: it clusters around task types the model finds natural to generate, it drifts toward a homogeneous house style, and it inherits every bias of the seed set and the generator. Left unchecked, you get ten thousand variations of the same twenty tasks, phrased with the same politeness, testing nothing new. The remedy is deliberate diversity engineering, which deserves its own section.
Diversity is the actual hard problem
Ask a language model for a customer complaint and you will get a well-formed, medium-length, politely irritated message about a late delivery. Ask it for a thousand and you will get the same message a thousand ways. Generation without diversity pressure collapses to the mode, and a training set that collapses to the mode teaches the model nothing about the tails — which is precisely where production failures live: the furious customer, the message in mixed languages, the question that is actually three questions, the request that looks routine but is a policy trap.
Diversity has to be injected structurally, because asking the generator to please be diverse does not work. The standard tools are conditioning variables: generate this complaint as an elderly customer unfamiliar with the app, as a reseller with fifty affected orders, as someone writing from a phone in a hurry, at three levels of anger, in five registers of formality. Persona libraries, topic taxonomies, difficulty ladders, and length targets turn one prompt into a grid of thousands of distinct cells, and sampling the grid gives you coverage that naive generation never will. The generator supplies fluency; the grid supplies variety. Neither works alone.
Diversity does not emerge from scale — a million samples of the mode are still the mode. It has to be engineered in through explicit conditioning, and measured, not assumed.
Measuring what you generated
Because generated data looks plausible one example at a time, its defects are only visible in aggregate, and you need measurement to see them. Embedding-based clustering reveals whether your hundred thousand examples are actually four hundred examples wearing disguises. N-gram and template analysis catches the generator's verbal tics — the phrases it reaches for constantly, which the student model will then overproduce forever. Length histograms, label balance, topic distribution, and duplication rates are all cheap to compute and routinely surprising. Treat a synthetic dataset the way you would treat scraped data from an untrusted source, because that is what it is: data from a process with unknown biases, requiring audit before use.
The audit should also include the most old-fashioned technique available: reading it. A person who reads two hundred random samples will find failure patterns no metric flags — subtle factual drift, answers that are correct but uselessly generic, questions no real user would ask. An hour of reading per generation batch is among the highest-leverage hours in the whole pipeline.
Filtering: the quality gate
Every synthetic pipeline is generate-then-filter, and the filter matters more than the generator. The cheapest filters are structural: parseable JSON, compilable code, answers that reference the question, length within bounds. Above those sit consistency checks: generate the answer twice and keep it only if the two agree; ask the model to solve its own generated problem and keep the pair only if it succeeds. Above those sits model-based judging, where a strong model grades each example against a rubric and only high scores survive. The strictest and best filters are verifiers grounded in ground truth: unit tests for code, symbolic solvers for math, execution for queries, schema validation for structured output.
The ordering matters economically as well as logically: run the cheap filters first and the expensive ones last, so the strong model only grades candidates that survived the trivial checks. Yields of thirty to sixty percent after filtering are normal and healthy. A pipeline that keeps ninety-five percent of what it generates almost certainly has a filter that is not doing its job, and the defects it passes will surface later, expensively, as model behavior.
The generator sets the ceiling on your dataset's quality, but the filter sets the floor — and models are shaped far more by the floor than the ceiling.
Verification changes everything
The single most important structural fact about synthetic data is this: in some domains, correctness can be checked mechanically, and in those domains synthetic data is almost unreasonably effective. Generated code can be run against generated tests. Generated math can be checked by solvers, or by verifying that a claimed solution actually satisfies the problem. Generated SQL can be executed against a database and compared to a reference result. In these verifiable domains, you can generate a million candidates, discard the ninety percent that fail verification, and be left with a large corpus whose correctness does not depend on the generator's judgment at all.
This is why the most dramatic synthetic-data successes cluster in code and mathematics, and it explains the recent surge of reasoning models trained with verifiable rewards: when a checker can score attempts automatically, the model can generate its own practice problems, attempt them at scale, and learn preferentially from the attempts that verifiably succeeded. The strategic implication for anyone building a pipeline is to push as much of your task as possible into verifiable form. Every property you can check with code is a property you no longer have to trust a model about.
Reasoning traces as training data
A newer and increasingly important species of synthetic data is the reasoning trace: not just the question and the final answer, but the intermediate thinking that connects them. Models fine-tuned on high-quality worked solutions learn not only what to answer but how to decompose problems, and distilling the long, careful reasoning of a large model into a smaller one has proven to transfer a surprising amount of capability. The trace is generated cheaply — the teacher model was going to reason anyway — and filtered by outcome: keep the traces that arrived at verifiably correct answers, discard the rest.
Outcome filtering has a known weakness, though: a trace can reach the right answer through wrong reasoning, and a student trained on lucky traces learns confident nonsense. More careful pipelines score the steps as well as the destination, using process-level judges or by checking that intermediate claims hold. As with everything in this field, the question is never whether the data looks good — it is what process guarantees which properties, and what slips through the gaps between those guarantees.
Synthetic data for agents
Agentic systems raise the stakes because the unit of training data is no longer a single exchange but a trajectory: a sequence of model decisions, tool calls, observations, and recoveries stretching over many steps. Real trajectories are scarce — they require a working agent, real tasks, and often a human watching — so synthetic trajectories have become the workhorse of agent training. The pattern is to build a task generator that produces goals with known solutions inside a controlled environment, let an agent attempt them at scale, and keep the trajectories that verifiably succeeded, optionally having a stronger model repair or annotate the near-misses.
The environment is the crucial ingredient. A mock e-commerce store, a sandboxed filesystem, a synthetic codebase with planted bugs, a fake CRM with consistent records — each is a small world where the agent can act freely, outcomes can be checked programmatically, and failure costs nothing. Building these environments is genuine engineering effort, but it pays twice: the same environment that generates training trajectories also serves as an evaluation harness, replayable forever, with difficulty dials you control.
Simulated users
Half of every conversation is the user, so training and testing conversational systems requires someone to play that role, and humans are too slow and too expensive to play it ten thousand times. User simulators — models prompted or tuned to behave like customers with goals, moods, knowledge gaps, and patience limits — fill the seat. A good simulator does not cooperate: it changes its mind mid-conversation, provides information out of order, misunderstands instructions, abandons threads, and occasionally tries to get the assistant to do something it should not. Those are exactly the behaviors that break assistants in production, and exactly the ones absent from polite synthetic dialogues.
The pairing of an assistant under test with an adversarially configured user simulator, plus a judge scoring the outcome, is one of the most productive patterns in applied AI right now. It generates training data, regression tests, and red-team findings from the same machinery. Its limit is fidelity: simulated users are still a model's idea of users, and they will miss the sheer strangeness of real ones. Simulation narrows the gap to production; it never closes it.
A user simulator that always cooperates is a rubber stamp. The value is in the simulated user who is confused, impatient, ambiguous, and occasionally adversarial — because the real ones are.
Synthetic data for evals
Evaluation may be where synthetic data earns its keep fastest, because eval sets are small enough to verify carefully and leverage is enormous: a few hundred well-constructed test cases steer every subsequent decision about prompts, models, and architecture. Synthetic generation lets you populate the parts of the test matrix that real traffic has not yet produced: the rare intents, the boundary conditions, the adversarial phrasings, the fifty variations of the same question that should all get the same answer. When a bug report arrives, a generator can turn one failure into a family of test cases probing the same weakness from every angle.
Two disciplines keep synthetic evals honest. First, human-review the eval set even if you would never human-review training data at scale — errors in a training set average out, but errors in an eval set systematically distort every decision you make with it. Second, never let the same model family generate both the eval and the system under test without independent verification, or you will measure agreement rather than correctness. An eval a model wrote, took, and graded is a hall of mirrors.
Model collapse, examined calmly
The best-known fear about synthetic data has a dramatic name: model collapse. Train a model on model output, the story goes, and each generation loses a little of the distribution's tails, drifts a little toward the mode, until the descendants know nothing but their own averaged reflection. The phenomenon is real and reproducible in the laboratory setting that produces it: recursive training where each generation's output wholly replaces the previous data, unfiltered, for many generations.
Practical pipelines rarely resemble that setting. Real deployments accumulate data rather than replace it — synthetic examples are added alongside real ones, not swapped in for them — and research on the accumulate regime shows degradation largely arrested. More importantly, production pipelines filter, and filtering changes the mathematics: selection against defects is an injection of information, not a photocopy of a photocopy. The honest summary is that collapse is not a mystical inevitability but an accounting question — each generation, does your process add information through verification, curation, and fresh real data, or does it only recycle? Pipelines that only recycle will degrade. Pipelines anchored by verifiers and real-data admixture have not just avoided degradation but driven some of the strongest recent model improvements.
Keeping the anchor
The practical corollary is that synthetic data works best as an amplifier of real data rather than a replacement for it. Real examples carry information about the world that no generator conditioned on model priors can supply: the actual distribution of user intents, the actual vocabulary of your domain, the actual shape of the problems that arrive on a Tuesday. A common and effective pattern uses real data as seeds — every synthetic example is a variation, extension, or recombination anchored in a real one — and holds the evaluation set strictly real, so the measurement of progress can never be flattered by the same process that manufactured the training set.
Mixing ratios are an empirical matter, but the asymmetry of roles is not: real data defines the target distribution, synthetic data densifies coverage of it. Teams that invert this — synthetic defines the task, a sprinkle of real data legitimizes it — tend to build models that are excellent at the generator's idea of the job and mediocre at the job.
Privacy: the original promise
Long before LLMs, synthetic data was pitched as the answer to a governance problem: how do you let people build on sensitive data — medical records, transactions, messages — without exposing anyone's actual records? Generate a synthetic dataset with the same statistical shape, the pitch goes, and share that instead. The promise is real but narrower than the marketing. A generator trained on sensitive records can memorize and re-emit them; rare individuals in the tails are the most likely to be memorized and the most identifiable when they leak. Plausible-looking synthetic records are not automatically private records.
Rigorous versions of this practice exist — differentially private training of the generator gives mathematical bounds on what any single record can contribute — at a measurable cost in fidelity, particularly in the tails. The engineering guidance is unglamorous: treat privacy claims about synthetic data as claims requiring proof, test empirically whether real records can be extracted or membership inferred, and involve people who think about re-identification professionally before shipping a synthetic dataset outside the boundary that contained the real one.
The economics, plainly
The cost argument for synthetic data is usually summarized as cheap, which misses the actual structure. Human-labeled examples cost dollars each and arrive at a rate measured in weeks; synthetic examples cost fractions of a cent and arrive at a rate measured in hours. But the synthetic pipeline has fixed costs the per-example price hides: designing the generation scheme, building the filters and verifiers, auditing the output, iterating when the first three versions produce subtly wrong data. For a few hundred examples, humans are often cheaper and always simpler. The synthetic advantage compounds with scale and with iteration — the tenth batch costs almost nothing, and when requirements change, you regenerate instead of re-hiring.
The deeper economic shift is in what you are buying. A labeling budget buys a static asset that depreciates as your product changes. A generation pipeline is a capability: it produces training data, eval cases, and regression tests on demand, forever, with dials you can turn. The teams that treat synthetic data as infrastructure rather than as a one-time dataset purchase are the ones that extract most of its value.
Licensing and provenance
Synthetic data inherits legal questions from its generator, and they deserve more attention than they usually get. Most commercial model providers restrict using their outputs to train competing models, so a distillation pipeline built casually on a third-party API may violate its terms of service. The provenance of the generator's own training data adds another layer of unresolved questions that courts and regulators are still working through. None of this is a reason to avoid synthetic data; it is a reason to know which model generated every example you train on, under what terms, and to keep records. Provenance tracking — generator, version, prompt, filter results, for every example — is cheap to build into a pipeline at the start and painful to reconstruct afterward, and it is exactly what you will need the day a customer, an auditor, or a lawsuit asks what your model was trained on.
Difficulty as a design variable
One of the least appreciated advantages of manufacturing your data is that difficulty becomes a dial rather than an accident. Real-world datasets have whatever difficulty distribution the world happened to produce, which is usually a heavy mass of easy cases and a thin, unrepresentative scattering of hard ones. A generator can be pointed at the hard region deliberately: longer documents, more distractors, deeper nesting, more steps between the question and the answer, more plausible wrong paths. Models learn most from examples near the edge of their competence, and a pipeline that can measure where that edge sits — by watching where the model's success rate crosses fifty percent — can keep generating just beyond it, a moving curriculum that follows the model as it improves.
The same dial matters for evaluation. An eval set whose difficulty is uncontrolled tells you little when scores move; an eval set with explicit difficulty tiers tells you whether an improvement came from mastering the middle or finally cracking the hard tail. Difficulty engineering is tedious to retrofit and nearly free to build in from the start, which is an argument for deciding early what makes an example hard in your domain and encoding that understanding into the generator's conditioning grid.
Synthetic data for retrieval systems
Retrieval-augmented systems have their own data hunger: they need query-document pairs to train and evaluate embedding models, rerankers, and the retrieval layer as a whole, and those pairs almost never exist for a private corpus. The synthetic solution inverts the problem. Instead of finding queries that match documents, walk the corpus and generate, for each chunk, the questions that chunk answers — then train and test retrieval on those generated pairs. The document is real, the pairing is correct by construction, and only the question's phrasing is synthetic, which makes this one of the safest and highest-yield applications of generation in the entire toolkit.
The subtleties are instructive. Generated questions tend to borrow the vocabulary of the passage they came from, which makes retrieval look easier than it is — real users ask with their own words, not the document's. Good pipelines force paraphrase distance: generate the question, then rewrite it as someone who has not read the document would ask it, drop the shared keywords, and keep only versions that still have a single correct source. Add unanswerable questions and near-miss distractor passages, and the same process that trained the retriever produces the stress tests that keep it honest.
Deduplication and contamination
Two hygiene problems grow quietly inside every generation pipeline. The first is internal: generators repeat themselves, and near-duplicate examples concentrate the training signal on whatever the generator finds easiest to say, amplifying the mode-collapse pressure that diversity engineering was meant to fight. Exact-match deduplication catches almost none of this, because the duplicates are paraphrases; embedding-similarity and fuzzy-hash deduplication are the working tools, and running them typically removes a startling fraction of a naive pipeline's output.
The second problem is external: contamination between what you train on and what you measure with. When the same generator, the same seeds, or the same prompt templates feed both the training set and the eval set, the eval stops being a measurement and becomes a memory test. The discipline is mechanical rather than clever — separate the seed pools, separate the generation runs, check eval examples against the training set with the same similarity tooling used for deduplication, and quarantine anything close. Every impressive synthetic-data result should be interrogated for contamination first, including your own.
A reference pipeline
Pulling the threads together, a production-grade synthetic data pipeline has a recognizable shape. It starts from seeds: real examples, real failure cases, a task taxonomy, a persona library. It generates with explicit diversity conditioning across a designed grid rather than free-running. It filters in stages — structural checks, consistency checks, model judges, and wherever possible ground-truth verifiers — expecting to discard a large fraction. It deduplicates near-neighbors, audits distributions, and includes a human reading pass. It mixes with real data deliberately, holds evals strictly out of the synthetic process, and records provenance for every surviving example. And it runs as a loop, not a batch: production failures become seeds, seeds become families of new training and test cases, and the pipeline turns every observed weakness into coverage.
None of these stages is exotic. The reliability comes from having all of them rather than the two most convenient ones, and from resisting the temptation to skip the audit because the samples you glanced at looked fine. They always look fine. That is the trap.
Synthetic data is not a dataset you buy once — it is a pipeline you operate, and its output is only as trustworthy as the verification it passed through.
Common failure modes
A few patterns account for most synthetic-data disappointments. The homogeneous corpus: a million examples, four hundred actual behaviors, a student model that is confidently narrow. The unfiltered firehose: generation without a quality gate, and a training run that faithfully learns the generator's error rate. The leaky eval: test cases generated by the same process as training data, producing beautiful metrics and a product that fails on contact with users. The style infection: a student model that inherits the teacher's verbal tics so strongly that every response sounds like the same person, because in a sense it is. The privacy laundering: sensitive data regenerated with a model and declared anonymous without any test of whether it is. And the silent drift: a pipeline that ran unattended for six months while the product, the users, and the failure modes it was designed around all changed underneath it.
Each has a remedy earlier in this post — diversity engineering, staged filtering, eval independence, style-aware auditing, empirical privacy testing, and treating the pipeline as an operated system with an owner rather than a fire-and-forget script.
When not to use it
Synthetic data is the wrong tool in a few identifiable situations. When the phenomenon you need lives outside your generator's knowledge — genuinely novel domains, brand-new products, populations the model has barely seen — generation produces confident fiction, and a small real collection effort beats a large synthetic one. When the total need is small, the pipeline's fixed costs dominate and a week of expert labeling is simpler and better. When correctness cannot be verified even approximately and errors are expensive — medical advice, legal conclusions — the filter problem is unsolved, and synthetic data belongs in testing rather than training. And when what you actually lack is understanding of your users rather than volume of examples, no generator can tell you what you have not yet observed. Synthetic data amplifies knowledge you already have a seed of; it cannot conjure knowledge you do not.
Where this is heading
Three trends are converging to make synthetic data more central rather than less. Frontier labs increasingly describe their training pipelines as data factories, with generation, verification, and curation as the core production line — the constraint on model quality is shifting from how much data exists to how much can be verified. Reinforcement learning from verifiable rewards is turning training itself into a synthetic-data process, where the model generates its own attempts and learns from the checked outcomes. And agentic systems are creating demand for a kind of data — long, tool-rich, environment-grounded trajectories — that essentially cannot be collected from the wild at scale and must be manufactured in simulation. The craft is consolidating around a simple identity: in a world where generation is nearly free, the scarce resources are verification, diversity, and grounding in reality. Teams that own those own their data supply.
Getting started without overbuilding
All of this can sound like a mandate to build a data factory before you are allowed to generate a single example, and that would be the wrong lesson. The right starting point is small and concrete: take one real failure your system exhibited last week, prompt a strong model to produce twenty variations of it, read all twenty, keep the good ones as regression tests, and notice what the bad ones have in common. That single exercise teaches most of the field's lessons in an afternoon — the generator's fluency, its sameness, the necessity of reading what it made, and the surprising usefulness of even a small, verified batch.
From there, grow the machinery only as the failures demand it, in the same spirit as any other engineering discipline. Add conditioning when you notice sameness. Add filters when defects slip through. Add deduplication when the corpus swells. Add provenance tracking before the corpus matters, because afterward is too late. The pipelines described in this post were not designed in one sitting by anyone; they accreted, batch by audited batch, in response to observed problems. What separates the teams that succeed with synthetic data from the teams that quietly abandon it is not the sophistication of their first pipeline but the habit of never trusting a batch they have not measured, and never measuring with anything the batch could have contaminated.
The bottom line
Synthetic data works — not as magic, but as manufacturing. Generation is the easy stage; the value is created in the stages around it: the diversity engineering that decides what to make, the filters and verifiers that decide what survives, the real-data anchors that keep the distribution honest, the audits that catch what the metrics miss, and the provenance that lets you stand behind the result. Done casually, synthetic data produces the illusion of scale: enormous corpora that teach models to be fluent replicas of their generator, evaluated by mirrors. Done seriously, it is the closest thing this field has to a controllable data supply — one that turns every verified capability into training signal and every observed failure into test coverage.
The teams getting the most from it share a posture rather than a toolkit. They treat generated examples as untrusted input from a biased process, and verify accordingly. They spend more on checking than on generating, and consider that ratio correct. They keep real data in the loop as the definition of the task, not a garnish. And they operate the pipeline the way they operate any production system: instrumented, audited, owned, and improved in response to observed failure. Data was always the constraint. Synthetic data does not remove it — it moves it, from what you can collect to what you can verify. That is a much better place for it to live.