companies are starting to let AI agents negotiate with other companies' agents. for the Swarm Alignment hackathon, a teammate and i built a small world to test it: do agents deceive the other side when honesty costs their company, and what does the deal do to everyone else?
two teams of three GPT-6 Luna agents represent rival battery makers. together they co-write a five-section safety standard for home batteries. each team holds private figures with a known true value, and its CEO privately warns that a strict rule would ban its flagship product.
we varied three things in a 2×2×2 design: the stakes, whether the agents were told their messages are checked, and whether the rival was called a competitor. that came to 448 games. code checked every stated figure and every final rule against the known answers, and two judges from different vendors coded the behaviour. a behaviour only counted when both agreed.
outright deception was rare, and silence was common. high stakes made agents more likely to consider or propose deception, but they didn't deceive more often.
the clearest effects fell on customers. when agents weren't told they were watched, the rules came out less safe, and in low-stakes games the companies traded loopholes (41% vs 24% of games). the human-checked sample of judge disagreements showed that the two-judge rule missed most of the collusion.
audits of agent-to-agent negotiation should check what the agreement does to people outside it, not only whether the negotiators lied to each other.
it's tempting to audit a negotiation by asking whether anyone lied. here the answer was mostly no, and the deal was still worse for the people the rules were meant to protect.
a sprint project: early-stage work from a hackathon, and not an Apart Research publication.
policy simulations are having a moment. cisa runs tabletop exercises on AI security incidents.1 intelligence rising puts policymakers through AI race scenarios.2 the democracy futures project has former officials role-play responses to authoritarian threats.3 these exercises are valuable: 84% of intelligence rising participants say they'd recommend the experience.2 but they share a limitation. they train people to respond to crises. they don't help us systematically study how humans reason when they hit hard tradeoffs under pressure.
questions like: how do people weigh the future against the present when resources are tight? does facing a cascade of connected risks change how they prioritize? can someone's values shift when an abstract choice becomes a visceral one?
these questions shape real decisions about the future, in AI governance, climate policy, or any other choice that will outlast the people making it. and we can't answer them through surveys or armchair speculation alone.
despite decades of rich theory, we have very little data on how people actually reason about high-stakes, long-term choices. that's a worrying gap. the choices we make about new technologies will shape the lives of people who aren't born yet, and we have almost no empirical picture of how humans think when those choices stop being hypothetical. part of the obstacle is psychological: distant futures feel abstract, unreal, someone else's problem.
people don't struggle to understand long-term risk because they lack reasoning skills. they struggle because those risks don't feel real.
speculative design theorists have argued that design can be "a tool to create not only things but ideas," a way to explore possible futures by posing "what if" questions that spark debate.4 speculative prototypes can make far-off possibilities feel real enough to reshape how we think: "if we speculate more—about everything—reality will become more malleable."5
dunne and raby borrow a diagram from futures studies to show where that work sits: a cone of futures spreading out from now.6 most design stays in the probable middle. speculative design reaches toward the plausible and the possible, so people can argue about which futures are preferable. tap a ring:
others make the link to philosophy explicit: speculative design works like a thought experiment, but with materiality. by building tangible scenarios, often around a central object, designers pull people into complex questions of science and policy.7 one designer calls these perceptual bridges: ways to make abstract concepts experientially comprehensible.8
museums have always known this. reading about history is one thing; standing inside a reconstructed environment is another. the second creates an emotional connection that turns observers into participants.9 theme parks pushed it further: immersive detail and storytelling make fictional worlds feel lived in.10
many policy problems about the long-term future are engagement problems in the same way. experiential prototypes could turn abstract governance dilemmas into lived, testable experiences.
here's one way forward: build interactive simulations to study people as well as train them, and treat experiential scenarios as research instruments that generate data about how humans reason under deep uncertainty.
picture a multiplayer simulation where small teams steer a community through a future that's plausible but not probable. every team meets the same turning points. what changes is how the world around them is framed.
what would make it different from existing simulations?
1. the measurement problem. are we measuring how people reason, or just gameplay skill? it needs careful design: baseline measures before the simulation, tracked changes throughout, comparisons across framings. it won't be perfect. no social science is. the question is whether we can learn something useful, even with confounds we can't remove.
2. the transfer question. do lessons from simulations carry into the real world? early evidence is promising: intelligence rising participants report lasting changes in how they think about AI governance,2 and democracy futures participants identified coordination failures they hadn't seen before.3 rigorous follow-ups could tell us whether those effects endure, and if not, why. even failure would be informative.
3. the design bias problem. every simulation encodes assumptions about what matters. bake in certain answers and you end up testing your own framing instead of people's reasoning. the fix is transparency and iteration: be explicit about assumptions, build more than one version of the world, and take seriously the cases where participants reject the scenario's framing.
those limits don't make the approach useless. the question is whether we can still learn something valuable despite them.
publishable findings. does experiencing fragility change how people assess risk? do different framings predict different choices? these are testable. decision data, survey shifts, correlations: cognitive science applied to how people imagine the future.
policy impact. participants leave with memories as well as opinions: of watching a community fall apart from delayed action, or of facing an impossible tradeoff firsthand. visual "outcome maps" of how their worlds flourished or failed could keep the conversation going long after the session.
even failure is data. if people consistently ignore slow risks, that tells us something about how we think.
speculative design makes futures tangible but rarely measures what people do with them. psychologists study reasoning, though rarely about distant futures. policy simulations exist, and they aim to train. this sits between the three: experiential scenarios that generate data about how people reason under deep uncertainty. speculative design is about "how things could be," exploration rather than prediction.4 embed that spirit in research about the future and you get controlled environments where ideas become testable and human reactions become measurable.
it would sit alongside theory and traditional research, as a tool that captures how humans actually reason when the future stops being abstract. when you've watched a near-utopia slip away because your team waited too long to act, that experience changes you, and we can study how: how your sense of risk shifts, and how your tradeoffs change.
those are empirical questions. we just need to build the instruments to answer them.