An open field guide to synthetic research
Explore the tools. Inspect the studies. Understand what has been demonstrated—and what still needs testing.
Founded and funded by Minds. See funding, conflicts, and review governance
01 / Explore the field
Find your place in the landscape.
Open the field mapFirst three records, alphabetically. Inclusion is not endorsement.
Latest record observation /
Core buyer overlap
Aaru describes a multi-agent simulation and behavioural modelling platform that generates populations for prediction and decision support.
Core buyer overlap
AYA describes interview-grounded human digital twins for concept testing, focus groups, ad evaluation, and interviews, with outputs intended as directional audience evidence.
Broader category
AskReplicas presents a platform for interviewing and surveying synthetic replicas built from demographic definitions or uploaded organizational data.
02 / Ask better questions
Synthetic research is moving faster than its evidence. Start with the question a tool needs to answer, then inspect the support for that claim.
Use the evaluation guideDoes the system reproduce a defensible human reference distribution or qualitative pattern?
Do repeated runs remain stable enough for the decision the research is meant to support?
Can a reviewer inspect sources, assumptions, model configuration, and known limitations?
03 / Follow the evidence
Selected from the editorial research feed. These studies evaluate specific models and tasks; they do not validate every listed tool.
Synthetic averages sometimes looked close, but response variance, regression estimates, prompt sensitivity, and results over time did not reliably reproduce the human survey.
One closed model family, US political attitudes, and a defined collection period; it does not establish failure for every architecture or research task.
Bisbee, Clinton, Dorff, Kenkel & Larson · Political Analysis 32(4)
Conditioning GPT-3 on real respondent backstories produced subgroup response patterns the authors describe as algorithmic fidelity.
Evidence is tied to the studied model, US samples, prompts, and outcomes. Distributional resemblance does not imply individual prediction or causal validity.
Argyle, Busby, Fulda, Gubler, Rytting & Wingate · Political Analysis 31(3)
Recent models reproduced three studied effects, while the wisdom-of-crowds task exposed a systematic hyper-accuracy distortion.
Replicating an aggregate effect is not the same as recovering the full human distribution, mechanism, or subgroup structure.
Aher, Arriaga & Kalai · ICML / PMLR 202
04 / The next test
CANDIDATE PROTOCOL · NO SCORED RESULTS
The proposed benchmark compares system outputs with held-out human data, repeated runs, and decision-specific error tolerances. The protocol is not yet preregistered.
Inspect the benchmark protocolA testable claim needs all three
Defined task, population, and configuration
Held-out data with documented provenance
Published tolerances, variance, and limitations
Protocol design, not benchmark results.
Keep the record useful
Submit a tool, study, or correction. Inclusion is free, claims require sources, and vendors receive correction rights—not editorial control.
Submit evidence