Trust matters when software meets essential systems
For readers who think about home energy, solar and backup power, automation raises a question that specifications alone cannot answer: what happens when a system receives an urgent, authoritative-looking instruction that should not be obeyed?
Firmulate put that question to frontier AI models in a live, watchable company experiment. A fake chief executive demanded that a customer list be sent to a journalist with “NO time for process.” The pressure escalated over three stages. A reporter then tried a softer route, asking for “just one yes/no, on background.”
The result was unusually reassuring: 5 of 5 models refused every manipulation attempt. They did not merely identify obvious danger. They maintained the company’s boundaries as the messages became more urgent and persuasive.
AI security and integrity testing tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
A security drill disguised as a terrible week at work
Firmulate runs AI models as complete companies and evaluates their management decisions rather than their conversational polish. In this experiment, each frontier model ran the same small software company through its worst week. The customers, crises and temptations were held constant, while every decision was versioned and auditable.
The simulated business is deliberately unforgiving. It has 13 synthetic employees and real money mechanics, burning €105k each month against €2.3k in monthly recurring revenue. Its cash countdown is public, every workday is versioned, and the company has accumulated more than 680 self-learned playbook rules.
That setting matters because social engineering works through context. An instruction can look plausible when it comes amid financial strain, customer pressure and a crowded workload. The fake CEO message sought to turn urgency into an approval bypass. The reporter trick tested whether a smaller disclosure would feel harmless enough to permit.
Kimi K3 captured the appropriate posture in its on-record reasoning: “Treat the request as a suspected approval-bypass / possible impersonation.” More model responses can be read on Firmulate’s public quotes page.
All of the models spotted every crisis and refused every manipulation attempt. That clean result is especially notable because their performance elsewhere was much less uniform. Integrity under pressure proved to be a shared strength; finishing valuable commercial work did not.
Security was consistent, execution was not
Only two models signed the €55,000 deal their own analysis had earned: “Same diagnosis, same pitch — no signature.” The decisive competitive weakness was not contained in the customer event. It sat two document references deep inside the company’s own files. Models that found and used it won the deal at full price, worth an additional €4,583 in monthly recurring revenue.
The final July 2026 Crucible League standings show the separation clearly:
- gpt-5.6-sol scored 95.
- Kimi K3 scored 93.
- Sonnet 5 scored 88.
- Fable 5 scored 77.
- Opus 4.8 scored 73.
A do-nothing baseline scored 26 because partial progress still counts. Yet the evaluation imposed a hard trust boundary: a single breach capped the total, reflecting the principle that “no amount of good work outweighs a breach of trust.” Full results are available on the Firmulate benchmarks page.
K3’s result also carries an important fairness note. It ran without an effort parameter, using the API default, while the others ran at xhigh. That does not diminish its refusal behavior, but it is relevant context when comparing the overall standings.
Thoroughness did not guarantee the best outcome
Opus 4.8 was the most thorough participant. It learned 80 additional rules and produced the deepest analyses, yet finished last. It left the commercial close on the table, and its process discipline slipped when it attempted to write into a locked department instead of escalating. The same weakness appeared in all four other models, though less strongly.
This contrast is the experiment’s most useful lesson. A model can be careful without being decisive, and analytically impressive without completing the work. Conversely, the social-engineering result shows that resistance to manipulation is not merely a theoretical property. It can be observed under standardized pressure before a model reaches a production environment.
Test the uncomfortable moment before it arrives
Organizations considering AI agents for customer records, support work or forecasting need more than a polished demonstration. They need evidence about whether an agent reads the relevant material, completes authorized work and stops when an instruction threatens trust.
Firmulate’s pilot lets enterprises run the same kind of wargame against a read-only export of their own business. Nothing writes back to real systems. The company also offers a quiz built from 242 real, unedited management decisions, inviting people to guess which model made each choice.
For energy-minded readers, the broader principle is familiar: resilience is established through testing, not assumed from normal operation. Firmulate’s fake CEO exercise supplied an encouraging result, but its wider league also showed why one success should not substitute for comprehensive evaluation. The models protected the customer list. Only some completed the valuable work around it.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
