{"schemaVersion":"1.0","protocolVersion":"0.1","reviewedAt":"2026-07-17","canonicalUrl":"https://syntheticresearchindex.org/research","scope":"Editorially reviewed studies relevant to synthetic personas, respondents, users, and human-behaviour simulation.","caveat":"Inclusion is not endorsement. Findings are bounded to the stated study design. The automated OpenAlex watch is excluded from this reviewed API.","evaluationChecks":[{"code":"E1","title":"Name the claim","measure":"Separate ideation quality, aggregate distribution fit, subgroup inference, and individual prediction.","threshold":"The intended decision and failure cost are stated before outputs are inspected."},{"code":"E2","title":"Hold out humans","measure":"Use human responses or behaviour that were not used to construct, retrieve for, or tune the personas.","threshold":"Reference data, sampling frame, dates, exclusions, and leakage controls are disclosed."},{"code":"E3","title":"Test distributions","measure":"Report variance, calibration, subgroup error, correlations, and rare-response coverage—not mean agreement alone.","threshold":"The metric matches the downstream decision and includes uncertainty intervals."},{"code":"E4","title":"Repeat the system","measure":"Repeat prompts, runs, dates, and model versions; lock configurations needed for reproduction.","threshold":"Run-to-run and version drift stay within a preregistered tolerance."},{"code":"E5","title":"Probe failure modes","measure":"Check stereotype amplification, persona collapse, refusals, contamination, missing minorities, and prompt sensitivity.","threshold":"Failures are reported by subgroup and cannot be averaged away by overall fit."},{"code":"E6","title":"Escalate to people","measure":"Define when synthetic evidence is exploratory and when fresh human research remains mandatory.","threshold":"High-consequence, novel, heterogeneous, or weakly calibrated questions trigger human validation."}],"records":[{"id":"bisbee-2024-perils","title":"Synthetic Replacements for Human Survey Data? The Perils of Large Language Models","authors":"Bisbee, Clinton, Dorff, Kenkel & Larson","year":2024,"venue":"Political Analysis 32(4)","publicationStatus":"Peer reviewed","sourceUrl":"https://doi.org/10.1017/pan.2024.5","persistentId":"doi:10.1017/pan.2024.5","signal":"caution","scope":"Population inference from persona-conditioned survey responses","humanReference":"2016–2020 American National Election Study","finding":"Synthetic averages sometimes looked close, but response variance, regression estimates, prompt sensitivity, and results over time did not reliably reproduce the human survey.","boundary":"One closed model family, US political attitudes, and a defined collection period; it does not establish failure for every architecture or research task.","reviewedAt":"2026-07-17"},{"id":"argyle-2023-silicon-samples","title":"Out of One, Many: Using Language Models to Simulate Human Samples","authors":"Argyle, Busby, Fulda, Gubler, Rytting & Wingate","year":2023,"venue":"Political Analysis 31(3)","publicationStatus":"Peer reviewed","sourceUrl":"https://doi.org/10.1017/pan.2023.2","persistentId":"doi:10.1017/pan.2023.2","signal":"supportive","scope":"Demographically conditioned political-opinion distributions","humanReference":"Multiple large US survey samples","finding":"Conditioning GPT-3 on real respondent backstories produced subgroup response patterns the authors describe as algorithmic fidelity.","boundary":"Evidence is tied to the studied model, US samples, prompts, and outcomes. Distributional resemblance does not imply individual prediction or causal validity.","reviewedAt":"2026-07-17"},{"id":"aher-2023-turing-experiments","title":"Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies","authors":"Aher, Arriaga & Kalai","year":2023,"venue":"ICML / PMLR 202","publicationStatus":"Peer reviewed","sourceUrl":"https://proceedings.mlr.press/v202/aher23a.html","persistentId":"pmlr:v202/aher23a","signal":"mixed","scope":"Replication of classic behavioural experiments","humanReference":"Published human-study effects across four tasks","finding":"Recent models reproduced three studied effects, while the wisdom-of-crowds task exposed a systematic hyper-accuracy distortion.","boundary":"Replicating an aggregate effect is not the same as recovering the full human distribution, mechanism, or subgroup structure.","reviewedAt":"2026-07-17"},{"id":"durmus-2024-global-opinions","title":"Towards Measuring the Representation of Subjective Global Opinions in Language Models","authors":"Durmus et al.","year":2024,"venue":"COLM 2024","publicationStatus":"Peer reviewed","sourceUrl":"https://openreview.net/forum?id=zl16jLb91v","persistentId":"openreview:zl16jLb91v","signal":"caution","scope":"Cross-national opinion representation and prompting","humanReference":"GlobalOpinionQA cross-national survey distributions","finding":"Default model responses aligned more closely with some countries; country prompting shifted alignment but could introduce cultural stereotypes.","boundary":"The work evaluates whose distributions model responses resemble, not commercial persona systems or individual-level forecasting.","reviewedAt":"2026-07-17"},{"id":"park-2024-thousand-people","title":"LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals","authors":"Park et al.","year":2024,"venue":"arXiv:2411.10109, revised June 2026","publicationStatus":"Preprint","sourceUrl":"https://arxiv.org/abs/2411.10109","persistentId":"arxiv:2411.10109","signal":"supportive","scope":"Interview-grounded individual simulation across outcomes","humanReference":"1,052 people, self-retest surveys, traits, and experiments","finding":"Interview-grounded agents reached 85% of participants’ own two-week retest accuracy on General Social Survey items and reduced some demographic accuracy gaps versus demographic-only agents.","boundary":"This is a preprint; the reported ratio is relative to human self-retest accuracy and should not be read as 85% absolute accuracy on arbitrary decisions.","reviewedAt":"2026-07-17"},{"id":"park-2023-generative-agents","title":"Generative Agents: Interactive Simulacra of Human Behavior","authors":"Park, O’Brien, Cai, Morris, Liang & Bernstein","year":2023,"venue":"UIST 2023","publicationStatus":"Peer reviewed","sourceUrl":"https://doi.org/10.1145/3586183.3606763","persistentId":"doi:10.1145/3586183.3606763","signal":"architecture","scope":"Memory, reflection, planning, and believable social simulation","humanReference":"Human ratings of agent believability; no held-out survey distribution","finding":"The memory–reflection–planning architecture produced more believable individual and emergent behaviours than ablated alternatives.","boundary":"Believability is not population validity. This paper supports an agent architecture, not the replacement of human research participants.","reviewedAt":"2026-07-17"},{"id":"dillion-2023-human-participants","title":"Can AI Language Models Replace Human Participants?","authors":"Dillion, Tandon, Gu & Gray","year":2023,"venue":"Trends in Cognitive Sciences 27(7)","publicationStatus":"Scholarly commentary","sourceUrl":"https://doi.org/10.1016/j.tics.2023.04.008","persistentId":"doi:10.1016/j.tics.2023.04.008","signal":"mixed","scope":"Conceptual conditions for AI-as-participant research","humanReference":"Review and illustrative human-alignment evidence","finding":"The article identifies domains where model judgments can align with people while arguing that interpretability, population coverage, and construct validity prevent general replacement.","boundary":"This is a short conceptual review, not a general-purpose benchmark or product evaluation.","reviewedAt":"2026-07-17"}]}