Choose Your Ground Truth: A Field Guide to Synthetic Data for Evals

You need an eval set but don’t have a hundred real production failures to build it from, so you reach for synthetic data — and most first attempts quietly produce garbage. A field guide to the techniques that actually work, from real-incident seeds to personas to RAG-grounded generation, with one throughline: synthetic data needs its own eval, so choose your technique backwards from the eval you want.