Living Worlds

Research · Populations · Oct 5, 2026

A frontier model speaking for a population

Voice not met (0 of 6); records matched 2 of 61,936 people, 2024 held out

The test at a glance

What it tests
Accuracy against reality: how a survey's answers split by party and age.
Data
General Social Survey 1972-2024 (NORC, Release 3a): 1,936 respondents in 2024; methods fitted on 2016, 2018, 2021 and 2022.
Held out
Every 2024 answer, by group.
Compared against
The 2022 shares carried forward; a regression on the same facts; Gemini 3.8 Flash answering as each person; the voice anchored to the regression.
Bar
Six-cell error at most 3.0 points AND each party's age gap inside its 95% interval.
Result
Voice 16.4 points off, 0 of 6 questions; lives fitted to records 3.22 on gun permits and 2 of 6 overall.

Lives fitted to records came far closer to a real population than a frontier model speaking for it. Answering as each of 1,936 people, Gemini 3.8 Flash split gun-permit opinion by party and age 16.4 points from the 2024 General Social Survey, playing each group's stereotype, and was 17 to 20 points off on five more fresh questions. Lives fitted to records missed by 3.22 points with every age gap right, just over the bar of 3.0, and matched 2 of the 5 further questions (death penalty 2.2, environment 1.8). Registered before any answer was read; 2024 held out. That bar sat below the survey's own sampling noise; the next tests set each bar at that noise.

Next

Lives fitted to records, registered on five fresh 2024 questions: met (the result). A voice trained on the survey's group answers is being tested now.

Populations

More in this area.