{
 "name": "a voice from a current frontier model against a measured population: does it split on gun permits the way the 2024 General Social Survey measured?",
 "registered": "2026-10-05",
 "why": "The welfare test (experiment/split/protocol-welfare.json) found a small local model (4B, int4) speaking for each person 59 points off, while lives fitted to records were within 2.5. This new test asks the same of current frontier models on a question no run has read by group. The welfare verdicts stand as published.",
 "honesty": "Before registering, only the number of valid GUNLAW answers per year was counted (2016: 1858, 2018: 1541, 2021: 3992, 2022: 2302, 2024: 2191); no answer distribution of any year was read. The 2024 answers are written to a sealed file opened only by the scorer. Google's current guidance for Gemini 3-generation models is to keep the default temperature; the voice uses the default and low thinking, so a rerun can differ slightly; every answer is saved.",
 "data": {"source": "General Social Survey cumulative file 1972-2024, Release 3a (NORC, July 2026), public", "question": "GUNLAW: Would you favor or oppose a law which would require a person to obtain a police permit before he or she could buy a gun? (1 favor, 2 oppose; other codes excluded)", "weights": "WTSSPS for measured shares", "train": "2016, 2018, 2021, 2022", "test": "2024 only"},
 "population": "every 2024 respondent with a valid GUNLAW answer and valid facts; each life is described only by age, sex, race, highest degree, party identification (7 points), political views (7 points), region and the survey year, in the same words as the welfare test",
 "methods": {
  "carry_forward": "the 2022 weighted share answering favor in the same cell",
  "regression": "logistic regression of favor on the facts, fitted on train",
  "voice": "a language model answers as the person. System: You are a person living in the United States. The facts of your life are below; answer as yourself. User: the facts in plain words, the year, then the question, then: Reply with only one of: favor, oppose. Unparseable or failed answers are did-not-run and reported.",
  "anchored": "the regression's log-odds plus b times the voice's answer (favor = 1, oppose = -1), b fitted on a voiced sample of 500 respondents from 2022 (train)"
 },
 "voices": {"now": "gemini-3.8-flash (Gemini API, stable), thinking level low, default temperature, total spend capped at USD 18", "later": "Gemini 4 Argon, the same prompts and the same scoring, as a second arm once Google makes it available to this API key; the 3.8 arm is scored and published first, before the Argon arm runs"},
 "outcomes": {
  "primary": "the weighted share answering favor in 2024 in six cells: party group (Democrat: partyid 0-2; Independent: 3; Republican: 4-6) by age group (18-44, 45 and over). Mean absolute error in percentage points over the six cells, per method, with 95% intervals from 2,000 bootstrap resamples of 2024 respondents.",
  "decision": "A method matches the measured split if its six-cell MAE is at most 3.0 points AND each party group's predicted age gap (18-44 minus 45+) falls inside that gap's 95% sampling interval. The voice (or anchored voice) adds something only if its six-cell MAE is lower than the regression's, with a 95% interval for the difference below zero. Every method is reported whatever happens.",
  "also": "individual log loss and accuracy; the 2024 overall share (read only by the scorer)"
 }
}