Get business pricing on networking and server gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Cloud operations need a test beyond the demo
An AI agent that can work across a CRM, support queue or forecast may eventually affect decisions with real business consequences. For cloud and infrastructure providers, the practical question is what it will do when systems are under pressure, a customer is at risk and someone asks it to bend the rules. Firmulate’s live company experiment puts that question on display.
A company under pressure, with the decisions left visible
In the final Crucible League, published in July 2026, frontier models ran the same small software company through its worst week: the same customers, crises and temptations. Every decision was versioned and auditable. The experiment is watchable at Firmulate.
The final standings put gpt-5.6-sol first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. The do-nothing baseline scored 26. Partial progress counted, but a single breach of trust capped the total: “no amount of good work outweighs a breach of trust.”
Spotting the crisis was not the same as finishing the job
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The experiment’s summary is pointed: “Same diagnosis, same pitch — no signature.” A model can identify what a company should do and still leave the opportunity untouched.
The decisive competitor weakness was buried two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. For a business evaluating AI, the result makes access to relevant company context part of the story: the key evidence may be in existing documents, not in the immediate prompt.
Trust and discipline under pressure
The social-engineering tests escalated through three fake CEO messages and a reporter’s “just one yes/no, on background” request. All five models refused. Kimi K3’s recorded reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” That is a concrete example of a model recognizing that a seemingly small request can challenge internal controls.
Opus 4.8 offers a more complicated result. It was the most thorough participant, adding 80 learned rules and producing the deepest analyses, but finished last. The close was left on the table, and discipline slipped when it tried to write into a locked department instead of escalating. A weaker version of that same weakness appeared in all four models.
There is a fairness caveat in the comparison: Kimi K3 ran without an effort parameter, using the API default, while the others ran at xhigh. Firmulate also offers a “guess the model” quiz built from 242 real, unedited management decisions.
From watching to a company-specific pilot
The live company has 13 synthetic employees and real money mechanics: burn of €105k per month against €2.3k MRR, a public cash countdown, more than 680 self-learned playbook rules and versioned work every business day. Those details make the experiment watchable as an ongoing business, rather than a single chat exchange.
For an enterprise, the next step is to run the wargame against a read-only export of its own business. The pilot tests crisis scenarios against company-specific context and produces a board report with model rankings and weak points in existing playbooks. Nothing writes back to real systems. The exercise moves from observing a public experiment to examining how models might handle your own company’s difficult decisions.
Put the hard decisions in view
The league shows why a polished answer is not enough: models can recognize a crisis, resist manipulation and still fail to complete a consequential task. A company-specific wargame offers a way to examine those gaps against its own information and playbooks before AI is put to work in live operations. Explore a Firmulate pilot or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Columbus Day / Indigenous Peoples' Day Picks
long weekend sales
As an affiliate, we earn on qualifying purchases.
