Blog · Research · Oct 21, 2026 · 5 min
Language models answering as real people, scored once on survey answers nobody had read: a frontier model met 0 of 6, prompted voices overstated births threefold, and carrying the last shares forward beat a small model. Lives fitted to records came closer, a model's estimate counted as a prior meant fewer people to ask, and subgroup splits landed inside the survey's own noise in the United States and in 30 of 31 African countries.
Can a language model answer a survey on behalf of a population? The idea is attractive: ask a model to answer as each person in a sample, and read off how a country, a party or an age group feels about a question nobody has asked yet. The field has moved quickly. A 2025 Stanford brief reports agents built from two-hour interviews that matched people's General Social Survey answers 85% as accurately as people matched their own answers two weeks later, and a December 2025 study on social desirability bias in silicon samples finds model answers drifting towards what sounds acceptable.
Living Worlds asks the question in its plainest form: when a model speaks for a population, how close does it come to what that population said, on answers nobody had read?
Every test here was registered before the scored answers were read, and scored once. Most used the 2024 General Social Survey with 2024 held out; others used women's records in national surveys and Afrobarometer round 10. In the later tests each bar was set at the survey's own sampling noise: the error the true shares themselves would stay under 95 times in 100. A model that has seen a published answer can repeat it; a model scored once on a sealed year has to know it.
A frontier model speaking for a population had Gemini 3.8 Flash answer as each of 1,936 people. On gun-permit opinion split by party and age it was 16.4 points from the 2024 survey, playing each group's stereotype, and 17 to 20 points off on five more fresh questions. Lives fitted to records missed by 3.22 points with every age gap right, just over the bar of 3.0, and matched 2 of the 5 further questions. Recorded as Voice not met (0 of 6); records matched 2 of 6. That bar sat below the survey's own sampling noise, so the next tests set each bar at that noise.
A population asked a question, against the measured answer asked 2,872 people about welfare spending. Carrying the last measured shares forward came within 2.5 points across six party-and-wording groups and met the rule. A regression came within 2.8 but one party's wording gap fell outside its interval. A small language model answering as each person missed by 58.9 points, saying nearly everyone thinks too much is spent. Voice not met; carry-forward met. The 2024 overall shares were seen before registration and are reported as seen.
A voice for a question nobody has asked before asked a frontier model directly for each group's share on five fresh questions, levelled to each question's 2024 national share. Its mean error was 6.2 points, ahead of giving every group the national share (8.2), but it landed inside the noise on 1 of 5, where 3 were needed. Not met (1 of 5 inside the noise).
Voices that set the level asked voiced lives about births directly. They overstated the total about threefold, 5.24 births per woman against 1.62, though their own records cut the error by a third. Not consistent. So in Living Worlds the population's rate sets the level and the voice moves each life above or below it.
Voices that set the level, learned from lived records trained a voice on the records of lives in twelve other worlds built on the same UK 2019 tables, then asked about 300 women in a new one, once each. It gave 1.75 births per woman against 1.62 and a mean age at childbearing of 30.3 against 30.7, inside both bars. Met. In that world births depend on age alone, and the voice adds nothing beyond it.
On real women, A voice against real lives ranked 582 held-out women in the GSS panels as well as a regression (AUC 0.646) and was the best calibrated of five methods, but beating age alone was not shown. Mixed. Births, from a voice that knows more than age then predicted who would have a baby in three years far better than age alone (AUC 0.728 against 0.620) across 4,453 women, but the registered adjustment pulled its share to 0.155, below the observed 0.178. Ranking met; level missed. Births with the level set once, on a survey nobody had opened froze the voice and set the level once: AUC 0.699 against 0.622 for age alone, and a predicted share of 0.224 against the observed 0.234 (95% interval 0.222 to 0.246). Met.
Lives fitted to records, on five fresh 2024 questions answered four of five within the 3.0-point bar across six party-and-age groups, with every age gap right; carrying the last shares forward missed every question, by 4.9 to 8.3 points. Met (4 of 5). Subgroup splits, held to the survey's own noise split five fresh questions on public spending by party and age, and all five landed inside the survey's own noise, where four were needed; levelled only to the 2022 national share, all five passed too. Met (5 of 5).
Thirty-one African countries, split by place and age carried the same question to Afrobarometer round 10, whose fieldwork ran in 38 African countries in 2024 and 2025. Given each country's national answer and each person's other answers, Living Worlds split 1,556 unseen questions across city and countryside and three age groups. In 30 of 31 countries, and 89% of country-question pairs, the split landed inside the survey's own noise. Met (30 of 31).
The model still has a use. A model's estimate as a prior: fewer people to ask counted Gemini 3.8 Flash's estimates as 50 respondents and combined them with 50 real ones per party-and-age group. On five fresh questions the mean error was 3.15 points, closer than 75 real respondents alone (3.46). Met. A little real data goes further.
A model answering as each person plays stereotypes: 0 of 6, a 58.9-point miss on welfare, births overstated threefold. Lives fitted to records, with the population's own level set first, came inside the survey's own noise on subgroup splits in the United States and in 30 of 31 African countries. The model earns its place as a prior, worth part of a sample.
Real people still came closest: 100 real respondents alone came closer still (3.01), and on police use of force the model added nothing. Subgroup splits are given each question's national share; they divide a known total. The direct-share voice met 1 of 5. In the simulated-births world the voice adds nothing beyond age.
Living Worlds fits lives to records first, then lets them speak. Its site shows the market-research case in exactly these terms: with a model's estimate, 50 real respondents per group came closer than 75 alone. People decide; the register shows how far each answer can be trusted.
Sources
Blog
Living Worlds' life expectancy forecasts, frozen twenty years back and scored once on years no run had seen: met for women and for men on the latest national figures, not met against 2024 national figures or across 155 countries, with honest intervals for women and a stacked UN model that did not help.
Census-built populations, agents grounded in interviews and language models speaking for whole groups have all advanced in 2025-26. The evidence now shows clearly which of them match real people, and the dividing line is what each simulated person is built from.
A population lived month by month in the United States from 1950 matched how many children Americans have had and how many brothers and sisters they report, closer than a frontier model answering from what it has read. The father's side, first counted through partnerships, missed, and became the next test.