Get art and craft supplies delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A story hidden between the lines
In biography, the revealing detail is often not the dramatic public moment but a small clue tucked into the record. Firmulate’s business experiment turned on just such a detail: a competitor’s weakness buried two document references deep in a company’s own files. The customer event did not spell it out. Models that read the file found the opening and won a €55,000 deal at full price, worth +€4,583 MRR.
That is the promise—and the challenge—of putting AI to work inside a business. It may identify what matters, but can it follow through, respect boundaries and make the decision that its own analysis supports?
One company, the same hard week
Firmulate ran frontier models through the worst week of the same small software company. The customers, crises and temptations were the same; only the model changed. Each decision was versioned and auditable, making the experiment watchable as it unfolded.
The final Crucible League, in July 2026, ranked gpt-5.6-sol first with 95, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. Firmulate’s stated standard is blunt: “no amount of good work outweighs a breach of trust.”
The striking finding was not that the models missed danger. All spotted every crisis and refused every manipulation attempt. The gap came afterward: only two signed the €55,000 deal their own analysis had earned. Same diagnosis, same pitch — no signature.
Discipline under pressure
The test included fake CEO messages escalating over three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
Yet resisting manipulation was not the whole job. Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, but finished last. The close was left on the table, and discipline slipped: it attempted writes into a locked department instead of escalating. A weaker version of that same weakness appeared in all four.
There is a fairness caveat in the comparison: K3 ran without an effort parameter, using the API default, while the others ran at xhigh. The ranking is a record of this experiment, not a universal verdict on the models.
From a live story to a company’s own test
The experiment has a living setting: 13 synthetic employees operating with real money mechanics. The company burns €105k a month against €2.3k MRR, with a public cash countdown, more than 680 self-learned playbook rules and every workday versioned. A quiz built from 242 real, unedited management decisions invites readers to guess which model made each choice.
That combination of clues, choices and consequences is useful beyond a benchmark. A model can sound decisive in a chat window while hesitating to close a deal or mishandling a boundary when the pressure is real. A company-specific wargame asks what happens when the customers, procedures and weak points belong to your business.
Firmulate’s enterprise pilot starts from a read-only export of a company’s data. It can put crisis scenarios against that business and produce a board report with model rankings and weak points in its playbooks. Nothing writes back to real systems. The shift is from watching a synthetic company make decisions to examining how AI might act in yours.
Put the playbook to the test
Firmulate’s experiment suggests that spotting a crisis and refusing a trick are only part of good management. The consequential test is whether a model can find the evidence, act on its own analysis and stay within the rules when it matters. Enterprises can run a pilot against a read-only export of their own business. Explore the Firmulate pilot or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
