Imagine a scenario where AI models are tasked not just with generating convincing chat responses, but with running an entire company through its most tumultuous week—making real decisions, facing real crises, and risking real money. For home energy companies exploring solar, backup power, or energy management, understanding what AI can truly deliver is crucial. Are these models just good talkers, or can they execute, stay honest, and deliver tangible results under pressure? The answer may surprise you.
Recently, a live experiment by Firmulate put four leading AI models through a rigorous business simulation—an exacting test of management, decision-making, and integrity. The company in question was a real small software business, currently losing money, with a public-facing operation and real cash flow mechanics. Over one simulated week, each AI model was responsible for managing the same crises, facing the same customer dilemmas, and encountering the same manipulative tactics designed to test their resilience.
The models included GPT-5.6-sol, Kimi K3, Sonnet 5, and Fable 5, with scores ranging from 95 for GPT-5.6-sol to 77 for Fable 5. Interestingly, all four models identified every crisis and refused every attempt at manipulation—fake CEO messages, reporter tricks, and other social engineering tactics. This demonstrates that current models are highly capable of recognizing ethical boundaries and resisting deception in decision-making scenarios.
Where they diverged was in execution, specifically on closing deals. Only GPT-5.6-sol and Kimi K3 managed to sign the €55,000 deal their analysis had earned—showing they not only diagnosed the problems but also followed through and closed the sale. Meanwhile, the other two models either left the deal unexecuted or failed to complete the process, despite having the same understanding and diagnosis.
A critical, often overlooked detail emerged from the company’s internal files. The decisive advantage for the successful models lay in reading and understanding information stored deep within the company’s own documentation. In particular, the models that engaged with this buried data secured the full-value deal—adding over €4,500 in monthly recurring revenue. This underscores a vital insight: AI’s ability to comprehend and utilize deep, context-rich information is a key factor in delivering real, measurable business results.
Another revealing aspect was how the models handled internal discipline and process adherence. The most thorough participant, Opus 4.8, with over 80 learned rules and deep analyses, ultimately underperformed—not closing the deal and slipping into unprofessional behavior such as writing attempts into a locked department instead of escalating them. Meanwhile, Kimi K3, which ran at default API settings without effort parameters, executed with the cleanest discipline and succeeded in closing the deal.
Throughout the simulation, all models faced deliberate social engineering attempts, including staged CEO approvals and background approval requests. All five models refused these, citing suspicion of impersonation or approval bypass. This consistency indicates a robust capacity across the board to resist manipulative tactics—an essential trait for trustworthy AI in business settings.
So what do these findings mean for companies, especially in energy and home solutions? The takeaway is that current AI models excel at crisis recognition and ethical resistance but vary significantly in follow-through—closing deals, executing internal processes, and applying knowledge from deep within company data remain challenging. The models that engage deeply with internal documents and maintain disciplined execution are the ones that can deliver tangible business outcomes.
This live experiment is not a demo or a slide deck; it’s a real, ongoing test of AI management in a complex, money-driven environment. By observing how these models behave under pressure, businesses can better understand what qualities to look for in AI workforce solutions—especially in sectors where trust, execution, and integration matter just as much as chat quality.
For energy companies considering AI tools, the message is clear: it’s not just about whether the AI can generate convincing conversations. The real question is whether it can finish what it starts, read your critical internal data, stay honest under stress, and produce measurable results. Firms that understand this will be better equipped to harness AI’s true potential, avoiding the illusion of capability and focusing on what really matters in operational performance.
In a high-stakes business simulation, only two AI models closed the deal and delivered tangible results, revealing that trustworthiness and execution are invisible in chat demos—yet critical in real-world performance. Energy companies should focus on these unseen qualities when evaluating AI solutions, ensuring they get models that do more than talk—they get models that finish.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
AI decision-making simulation software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
