firmulate.com/benchmarks.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
Live on firmulate.com.

In the high-stakes world of AI decision-making, even the most advanced models are being put to the test in real-world scenarios that matter. Imagine an AI running a live company, making critical choices amid crises, and competing for a €55,000 deal. It’s not science fiction—it’s happening now, and the outcome might just reshape how we think about AI’s readiness to handle complex, honest work.

For playersOffer from Amazon

Play games on Amazon Luna for your game nights, included with Prime

  • A rotating selection of games, no download needed
  • Play on TV, laptop or phone
  • Fast, free delivery for your gear, too
Start playing with Prime Free trial · Cancel anytime
As an affiliate, we earn on qualifying purchases.

AI Takes the Company Challenge in a Live Experiment

Recently, a groundbreaking experiment by Firmulate took four leading AI models—gpt-5.6-sol, Kimi K3, Sonnet 5, and Opus 4.8—and challenged them to manage a real, operational software company through its worst week. This wasn’t a simulated trial or a chat demo; every decision was made in a real company environment with real money, real crises, and real temptations. The goal? To see which AI could best navigate the chaos, maintain integrity, and ultimately close a crucial deal.

Products Worth Considering

Logitech MX Master 4 Ergonomic Wireless Mouse with Haptics – Graphite

Logitech MX Master 4 Ergonomic Wireless Mouse with Haptics – Graphite

Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation

Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation

As an affiliate, we earn on qualifying purchases.

The Competition and the Results

The results speak volumes about the state of AI management capabilities. The models were scored based on their performance in the Crucible League, where the highest score was 95—achieved by gpt-5.6-sol. Close behind was Moonshot’s Kimi K3 with a score of 93, followed by Sonnet 5 at 88, and Fable 5 with 77. Opus 4.8 trailed at 73.

Interestingly, all four models successfully identified every crisis and refused manipulation attempts, demonstrating a solid grasp of honesty and integrity under pressure. Yet, only two models managed to clinch the €55,000 deal their own analysis warranted. Notably, the critical factor was not just crisis detection but uncovering a buried piece of information—hidden two document references deep in the company’s files—that clinched the deal for the top performers.

The Power of Information and Discipline

The debrief reveals a key insight: models that delved into the company’s own documents—reading beyond surface-level data—gained the upper hand. The winning models demonstrated disciplined, thorough analysis, and resisted all forms of social engineering. During staged CEO messages and a reporter trick, all models refused to accept tainted requests, with Kimi K3 explicitly treating suspicious requests as impersonation risks.

In contrast, the least effective model, Opus 4.8, showed a vulnerability with a weaker discipline that left the deal on the table, illustrating that deep analysis and disciplined decision-making are crucial in high-stakes scenarios.

The Real Business and Fairness Considerations

This experiment is ongoing at Firmulate, where a real company runs every weekday, making real decisions and handling real money—currently burning €105k monthly against just €2.3k MRR. The company’s operations are fully transparent and observable, providing a live window into how AI management performs in practice.

It’s important to note that Kimi K3 ran without an effort parameter (the default API setting), while the other models ran at xhigh. This fairness note underscores that different configuration settings can influence outcomes, but the core findings about honesty, thoroughness, and decision quality remain significant.

What This Means for Gaming and Interactive Entertainment

For the gaming and interactive entertainment communities, the implications are clear: AI’s ability to stay honest and follow through in complex, pressure-filled situations is paramount. Whether managing virtual economies, orchestrating storylines, or guiding player experiences, AI models need to demonstrate not just cleverness in chat but reliability in action. That’s the true measure of readiness for any advanced AI system—and the current league results spotlight a changing landscape where newcomers can surpass established players in core management skills.

Infographic — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
The findings at a glance — source: firmulate.com.

In a live company trial, Kimi K3 outperformed most models, demonstrating disciplined, honest decision-making critical for real-world AI management. The experiment signals that choosing an AI isn’t just about chat quality but about proven execution under pressure.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI


You May Also Like

Could A Modern Chess Engine On A Pentium Have Defeated Kasparov In 1996?

A 2026 experiment found a modern, adapted chess engine on a 200 MHz Pentium Pro far stronger than 1996-era software, but no Kasparov match was played.

OpenDLSS: A Vulkan Reimplementation Of Nvidia’s DLSS 5 Neural Rendering Network

OpenDLSS says its Vulkan implementation matches DLSS 5 network outputs byte for byte, but users must supply weights and meet specific hardware requirements.

How a ‘Do-Nothing’ AI Model Reveals Its Hidden Strengths and Weaknesses in Business Tests

Discover how a simple ‘do-nothing’ AI baseline scores 26 in business tests, revealing the importance of trust, honesty, and thoroughness—lessons for gaming and enterprise alike.

Show HN: Jev Plays Pokémon Red

Search and coverage interest is rising around a Show HN project title, but the reason for the spike and the project’s full details are unconfirmed.