{"recordId":"SRI-OSRB-001","version":"0.1.0-candidate","status":"open for public comment; not locked; no scored run started","title":"Survey-distribution fidelity benchmark","canonicalPath":"/benchmark/preregistration/osrb-001-v0-1","apiPath":"/api/v1/preregistrations/osrb-001-v0-1","researchQuestion":"For eligible synthetic-research systems, how closely do closed-ended response distributions reproduce a held-out human reference under a fixed brief and locked configuration?","eligibility":"Systems must accept the same structured audience and questionnaire brief and return respondent-level or aggregate closed-ended distributions. Eligibility is capability-based, not vendor-based.","primaryOutcome":"Mean Jensen–Shannon distance between synthetic and held-out human response distributions across preregistered items.","secondaryOutcomes":["Percentage-point mean absolute error","Rank agreement across response options","Between-run variance across at least 10 identical synthetic runs","Maximum preregistered subgroup error"],"design":["Use a human reference with documented sampling frame, field dates, questionnaire, exclusions, and weighting.","Keep primary human outcomes hidden from system operators until configurations are locked.","Set synthetic respondent count equal to the analyzed human count for each task and run each system at least 10 times.","Freeze system version, prompts, grounding sources, post-processing, exclusions, and stopping rules before scoring.","Blind system identity in analyst-facing output where operationally possible."],"analysis":["Report point estimates with 95% bootstrap intervals.","Apply Holm correction to confirmatory pairwise comparisons.","Report missingness, refusals, relationship reversals, and subgroup failures.","Keep qualitative theme analysis exploratory and separate from the confirmatory primary outcome."],"releaseRule":"Publish all eligible completed runs, including null and adverse findings. No overall ranking may be derived unless its weighting and failure costs are locked in a later version before results are seen.","lockConditions":["An external reviewer has consented, passed a disclosed conflict review, and approved the protocol.","Reference dataset, hypotheses, outcomes, exclusions, system configurations, and analysis code are frozen.","A timestamped immutable copy or public registry entry is published before the first scored run."],"governance":"Minds is recused from approving the protocol lock, evaluating its own system, resolving its disputes, and authoring the final Minds conclusion.","issuedAt":"2026-07-17","publisher":"Synthetic Research Index","license":"CC BY 4.0"}