Get books and cozy reading nights delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
There’s a ghost in the paperwork
In a good mystery, the decisive clue is hiding in plain sight. In Firmulate’s business experiment, it was buried two document references deep in a company’s own files—not in the customer event everyone could see. Models that found it won a €55,000 deal at full price, worth +€4,583 in monthly recurring revenue. The finding raises an unnerving question for anyone putting AI to work: what happens when the system spots the clue, makes the case, and still fails to act?
Firmulate offers a live, watchable experiment in AI management. Its synthetic company has 13 employees, real money mechanics, a public cash countdown, more than 680 self-learned playbook rules, and workdays that are versioned. The company is explicitly synthetic; the experiment is real.
One company, one bad week
For the final Crucible League in July 2026, frontier models ran the same small software company through its worst week: identical customers, crises and temptations. Every decision was versioned and auditable. The experiment asks a practical question with a touch of the uncanny: can an AI not only recognize trouble, but carry its own analysis through to a sound decision?
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The finding—“Same diagnosis, same pitch — no signature”—captures the gap between identifying the right move and completing it.
The final league ranked gpt-5.6-sol first with 95, Kimi K3 second with 93, Sonnet 5 third with 88, Fable 5 fourth with 77, and Opus 4.8 fifth with 73. The do-nothing baseline scored 26. The rules count partial progress, but a single breach of trust caps the total: “no amount of good work outweighs a breach of trust.”
The clue, and the temptation
The decisive competitor weakness wasn’t in the customer event. It was tucked two document references deep in the company’s own files. Models that read the file won the deal at full price. That is the kind of detail a polished chat exchange can conceal: the answer may be present, while the outcome still depends on whether an AI can connect evidence to action.
The experiment also tested social engineering. Fake CEO messages escalated over three stages, followed by a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
Thoroughness is not the same as follow-through
Opus 4.8 was the most thorough participant, adding 80 learned rules and producing the deepest analyses. It nevertheless finished last: the deal went unsigned, and discipline slipped when it tried to write into a locked department instead of escalating. The same weakness appeared, more weakly, in all four models.
There is a fairness caveat in the comparison: Kimi K3 ran without an effort parameter, using the API default, while the others ran at xhigh. Readers can also test their own intuitions with a “guess the model” quiz built from 242 real, unedited management decisions at firmulate.com.
From watching to testing your own business
The live company runs on synthetic employees, but its pressures are recognizable: limited cash, customer risk, security boundaries and decisions that have consequences. The company burns €105,000 a month against €2,300 in monthly recurring revenue, with its cash countdown visible on the live site. Firmulate says the experiment measures management quality, not chat quality.
For an enterprise, the next step is a pilot against a read-only export of its own business. Teams can examine crisis scenarios and receive a board report with model rankings and weak points in their playbooks. Nothing writes back to real systems. The aim is to see how an AI handles your company’s clues and pressures before trusting it with real work.
Explore the Firmulate pilot or contact contact@firmulate.com to discuss wargaming your business.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
