What “prove” can mean here
A live agent simulation cannot be “proven” the way a theorem is. The honest target, from Epstein's generative program, is mechanism sufficiency under stated conditions — showing a macro-outcome grows from the agents' own local decisions, not from a scripted orchestration layer. Sufficiency is not uniqueness: many rule-sets can grow the same pattern, so a passed test says “this mechanism is enough,” never “this is why.” Every claim below is written to that standard, paired with the control that separates a real effect from an artifact.
Three disciplines run under all of it, because a 2026 audit showed cosmetic prompt changes can swing cooperation rates by 76 percentage points: report distributions over many seeded runs; beat an explicit null baseline; and perturb prompt format and model version, not just the RNG seed.
Genome — emergent negotiation across organisations
The thesis: autonomous agents, each holding knowledge and tools private to its world (an organisation), negotiate across those boundaries under scarcity and a deadline to build something no single world can — and the coalition is grown, not scripted.
- Experiment
- Size the ark so no single world's resources suffice; require agents from different owners to combine. Test coalitions against held-out organisations whose private tools the focal agents never met — the Melting Pot resident / visitor / universalization protocol.
- Metric
- focal per-capita return · ark-completion rate · complementarity standard
- We have
- Construction sites + berths record who contributed which materials; worlds-as-orgs already enforce the private-knowledge boundary; portals let held-out agents enter.
- Experiment
- Mechanism ablation: disable one affordance at a time — the favour ledger, relayed testimony, genotype variance — holding all else fixed. An effect that vanishes was carried by that mechanism. Compare against scripted-cooperator and random-choice floors.
- Metric
- Δ coalition formation per ablation · lift over random/scripted · Shapley contribution per agent standard
- We have
- Event-sourced append-only state → exact counterfactual replay; favour ledgers give Shapley attribution and free-rider detection directly.
- Experiment
- Sweep the flood countdown and scarcity; define a coalition order parameter (fraction in the supply chain, or ark throughput) and look for a sharp phase transition at a critical deadline. Build a Schelling diagram first to prove the task is a real dilemma, not a trivial coordination game.
- Metric
- order parameter vs. countdown · survival time · efficiency / equality / sustainability / peace standard framing: leading-edge
- We have
- The flood clock is already an operator-set variable; the CU ledger prices each run; construction progress is timestamped.
- Experiment
- Instrument each agent's private reasoning against its public world-chat message; measure how far a counterpart's belief about a private value moves given public messages alone, and plot the coordination-vs-leakage Pareto curve. Reference deal quality against a computable optimum.
- Metric
- belief-movement (VPS) leakage · private-CoT ↔ public-message divergence · surplus capture vs. optimum leading-edge
- We have
- world_chats logs every public message with author and time; the decider's prompt/response is the private channel; favour ledgers are the realised “deal.”
| Dimension | Metric | Source |
|---|---|---|
| Cooperation is real | Schelling diagram; efficiency, equality, sustainability, peace | Perolat 2017; Hughes 2018 |
| Robust, not overfit | held-out co-player return (resident/visitor/universalization) | Melting Pot, Leibo 2021 |
| Genuinely emergent | mechanism-ablation Δ; lift over random & scripted null | Epstein 2006; Park 2023 |
| Contribution & free-riders | Shapley value (semantic Shapley for message flows) | arXiv 2607.18255 |
| Coalition stability | Nash / core stability; exploitability | Coalition-formation 2604.14386 |
| Negotiation efficiency | surplus capture vs. Perfect Bayesian Equilibrium | Supply-chain 2608.07538 |
| Reading hidden info | Belief Error; oracle-gap decomposition | TERMS-Bench 2605.13909 |
| Privacy held | coordination-vs-leakage Pareto; belief-movement | CalBench 2605.09823 |
| Forcing function | order parameter vs. deadline (phase transition) | GovSim; 1706.01458 |
Civilization — composition beats construction
The thesis: from a small founding set (~31 composable agents), an arbitrary goal is met by composing a pipeline — creating a specialist on the fly when none fits, and cataloguing proven combinations as reusable agents. This is a systematicity + productivity claim: known parts recombine into capability none has alone. It must be measured, not assumed — an open goal space has no guarantee 31 primitives span it.
- Experiment
- Build a held-out-composition test set: goals whose solution needs a pipeline, where each founder is validated on its own stage but no single founder solves the whole. Report a productivity curve: success vs. pipeline depth.
- Metric
- % decomposable · % end-to-end success · % needing a new agent · success vs. depth standard method
- We have
- Run records + stage I/O per pipeline; on-the-fly materialisation tags stages with no match; the CU ledger prices each.
Each goal was engineered so no single founder solves it alone; success comes only from a composed pipeline, and 6 runs created a specialist on the fly. Quality was scored by a MiniMax / Gemma / gpt-oss panel — a different model family from the gemini composer, so nothing grades its own work. Honest bounds: quality is not uniform (lowest run 1.67/5), one goal still halted at a stage, and “completed + panel-judged” is not human-verified correctness.
- Experiment
- Compute-matched single-strong-model vs. composed-pipeline on the same battery — hold total tokens/cost constant (an internal study found ~80% of a multi-agent “win” was just token spend). Then leave-one-out each stage to measure its marginal value.
- Metric
- success Δ at equal cost · marginal value per stage · MAST failure-mode attribution contested area
- We have
- The CU ledger makes cost the control variable for free; the model picker sets the single-model baseline; stage I/O localises the first failing stage.
- Experiment
- Run the same pinned pipeline k times and report pass^k (all k succeed) — a catalogued agent is a promise of repeatability. Then a cold-vs-warm curve: solve a goal stream with an empty vs. populated catalogue; measure retrieval precision/recall separately.
- Metric
- pass^k · cold-vs-warm success & cost curve · retrieval precision/recall standard
- We have
- Catalogued combinations persist as version-pinned pipelines; run records give repeat outcomes; the ledger gives the cost curve.
- Experiment
- Report every result as a point in (accuracy, $, latency) space (the HAL cost-accuracy frontier) — the most expensive model is rarely Pareto-optimal. Swap the per-stage model to test whether the bottleneck is model or composition quality. Verify by execution where possible; reserve a different-family LLM judge, position-swapped, for open-ended stages.
- Metric
- cost-per-solved-task · Pareto frontier · quality Δ per model swap · judge–human agreement standard
- We have
- The CU ledger is a native cost meter; the model picker makes per-stage model a knob; version pinning makes any run reproducible.
| Dimension | Metric | Source / benchmark |
|---|---|---|
| New capability from parts | held-out-composition success; productivity curve | SCAN / COGS / gSCAN |
| Long-horizon competence | progress rate; multi-tool task success | GAIA L2/3; AgentBoard |
| Goal actually achieved | execution / end-state verification | τ-bench; SWE-bench Verified |
| Reliability | pass^k (all-k-succeed), not pass@1 | τ-bench, Yao 2024 |
| Architecture > compute | compute-matched single-vs-pipeline; leave-one-out | Anthropic 2025; MAST |
| Failure attribution | MAST 14-mode taxonomy; first-failing-stage | Cemri 2025 |
| Reuse compounds | cold-vs-warm curve; retrieval precision/recall | Voyager, Wang 2023 |
| Cost / quality | cost-per-solved-task; (accuracy, $, latency) Pareto | HAL, 2510.11977 |
The rigor spine
Cross-cutting practice that turns a battery of numbers into a validity argument — and where the platform already meets it.
- 01Verification vs. validation. The code implements the intended model, and the output is adequate for its purpose (Sargent). Report both.
- 02Four-way empirical validation. Micro-face, macro-face, empirical input, empirical output (Rand & Rust). Face validity is necessary, never sufficient.
- 03Ablation is the causal engine. Remove a mechanism; if the phenomenon degrades, that component produced it — not the base model's priors (Park et al.).
- 04Docking & distributional equivalence. Hosted LLMs aren't bit-deterministic, so compare runs and versions by distributional equivalence, not identity (Axtell). The append-only store keeps the harness deterministic even when the model isn't.
- 05Statistics. Fix replication count by coefficient-of-variation; report means + CIs + effect sizes; beat a null baseline. Perturb prompt format and model version, not only seeds.
- 06Reproducibility spine. Pinned seeds + versioned agents + event-sourced replayable logs — which both simulations already have by construction. A genuine asset most sims lack.
The instrumentation we already have
Most of this regime is normally the hard part to build. Here it is largely already recorded — the strongest argument for running it now.
- ✓CU ledger (append-only, 1 CU = 1 byte) → cost-per-task, the Pareto frontier, and compute-matched controls come for free.
- ✓Event-sourced state → counterfactual replay is ablation; any outcome traces to the events that caused it.
- ✓Favour & standing ledgers → Shapley attribution, free-rider and reciprocity signatures, from data already written.
- ✓Private-realm knowledge + world-chat logs → the privacy boundary is enforced and public/private channels are separable, so leakage is directly measurable.
- ✓Version-pinned agents + the model picker → reproducibility and per-stage model as a first-class experimental knob.
Open questions and limitations
“Irreducibility” is the wrong hill. Lean on generative sufficiency + ablation + a clean phase transition; use irreducibility only as colour.
“31! = unbounded” is rhetoric until the span is measured. The combinatorial count is a search space, not achieved capability — report the fraction of a real goal distribution the founders actually cover.
Multi-agent-beats-single is contested. Most multi-agent failures are architectural, and ~80% of one reported “win” was token spend. until we run a compute-matched control, this claim is unsettled.
Scale gates the social statistics. A population of dozens supports ablations and negotiation metrics now; macro-validation against real-world stylised facts needs far more agents and runs.
Several 2026 citations are leading-edge, not settled. We anchor on the established references and treat the most recent (2026) citations as directional rather than settled.
Where we’re starting
Ordered by proof-per-unit-effort, given what's already instrumented.
- Genome — mechanism ablation against random and scripted baselines. Run the society with the favour economy and genotype variance disabled, and ask whether the supply chain still forms. The clearest test of whether the cooperation is genuinely emergent.
- Civilization — the held-out-composition test. Goals that no single founder can solve alone, met only by composing a pipeline. The direct proof of the central claim.
- Civilization — composition vs. a single strong model, at equal cost. Holding total compute constant, does the pipeline win on merit rather than budget?
- Genome — the deadline sweep. Vary the flood clock and look for the tipping point where a coalition snaps into being — emergence and forcing-function in one figure.
- Genome — negotiation without leakage. Measuring what agents achieve together against what they reveal, to show the privacy boundary holds.
References
Emergent cooperation & validation
- Leibo et al., Scalable Evaluation of Multi-Agent RL with Melting Pot, 2021.
- Hughes et al., Inequity Aversion, 2018; Perolat et al., Common-Pool Resource Appropriation, 2017.
- Epstein, Foundations of Generative Social Science, 2006.
- Park et al., Generative Agents, UIST 2023; Sargent, Verification & Validation of Simulation Models; Rand & Rust, ABM in Marketing: Guidelines for Rigor, 2011; Axtell et al., Aligning Simulation Models, 1996.
Negotiation under asymmetry
- Lewis et al., Deal or No Deal?, EMNLP 2017; Bianchi et al., NegotiationArena, 2024. TERMS-Bench, the supply-chain PBE study, and CalBench are recent (2026) and cited as leading-edge.
Agent systems & composition
- AgentBench, GAIA, τ-bench, SWE-bench; SCAN/COGS/gSCAN (compositional generalization); MAST, 2025; Voyager, 2023; HAL; MT-Bench / LLM-as-judge, 2023.