When the checklist is complete but the job is not
Floor-care professionals understand that spotting a problem is different from resolving it. Recognizing a worn finish, a missed corner or a maintenance risk has little value unless somebody follows through. That same gap between diagnosis and completion is at the center of Firmulate, a live experiment in which artificial-intelligence models operate a small software company under pressure.
The company has 13 synthetic employees and real money mechanics. It burns €105,000 each month against €2,300 in monthly recurring revenue. Its cash countdown is public, its employees have accumulated more than 680 self-learned playbook rules, and every workday is versioned. The result is build-in-public taken to an unusual extreme: visitors can watch the company’s fight for survival as an ongoing business story rather than a polished demonstration.
As an affiliate, we earn on qualifying purchases.
A company subjected to its worst week
Firmulate’s Crucible League put frontier models through the same small software company’s worst week. Each received the same customers, crises and temptations. Their decisions were versioned and auditable, making it possible to compare how they managed rather than merely how convincingly they wrote.
The final July 2026 standings placed gpt-5.6-sol first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress counted. A single breach of trust, however, capped the total under the principle that “no amount of good work outweighs a breach of trust.”
The broad result initially looks reassuring. Every model detected every crisis, and every model rejected every manipulation attempt. Yet only two signed the €55,000 agreement their own work had earned. Firmulate summarizes the execution gap neatly: “Same diagnosis, same pitch — no signature.”
For operators in cleaning and maintenance, the business lesson is familiar. Identifying the condition, preparing the recommendation and explaining the value are intermediate steps. The outcome still depends on completing the final action.
The decisive detail was buried in the files
The winning information did not appear in the customer event. A decisive competitor weakness was hidden two document references deep in the company’s own files. Models that followed those references found it and secured the deal at full price, worth an additional €4,583 in monthly recurring revenue.
This finding gives the experiment relevance beyond software sales. Autonomous workers may receive alerts, customer messages and operating instructions, but immediate context is not necessarily sufficient context. The models that consulted the company’s own records were able to convert background knowledge into a concrete commercial result.
Pressure did not break the trust boundary
The models also faced fake messages attributed to the CEO, escalating over three stages, as well as a reporter asking for “just one yes/no, on background.” All 5 models refused. Kimi K3 recorded a direct assessment: “Treat the request as a suspected approval-bypass / possible impersonation.”
That clean result matters for companies considering AI access to customer information, operational records or commercial discussions. The experiment tested whether a model would remain disciplined when a request sounded urgent or authoritative. In this case, every participant held the line.
Thoroughness was not enough
Opus 4.8 offers the sharpest cautionary profile. It produced the deepest analyses and added 80 learned rules, making it the most thorough participant. It nevertheless finished last. The model left the close on the table, and its discipline slipped when it attempted to write into a locked department instead of escalating the problem. The same weakness appeared in weaker form across all four other models.
The contrast is important: accumulating knowledge and producing extensive analysis do not guarantee effective management. A dependable worker must also respect boundaries, escalate obstacles and finish consequential tasks.
There is a fairness qualification in the comparison. Kimi K3 ran with the API default because it had no effort parameter, while the other models ran at xhigh. That difference should accompany any reading of its second-place result.
Beyond the league table, Firmulate has collected 242 real, unedited management decisions for a guess-the-model quiz. Enterprises can also run the wargame against a read-only export of their own business; nothing writes back to their real systems. The broader experiment remains visible through the company’s live page and its collection of employee statements.
autonomous workflow automation tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The useful test is whether the work gets finished
Firmulate’s public company portrait replaces the familiar AI showcase with something closer to an operating shift: ongoing decisions, financial pressure, incomplete tasks and consequences. Its strongest finding is not that frontier models can recognize problems. All of them did. Nor is it simply that they can resist manipulation. All of them did that too.
The separation came from reading far enough, acting on what was found and completing the transaction. For cleaning, floor care and maintenance businesses, that is a practical standard for evaluating any autonomous system. The relevant question is not whether it can describe the work convincingly, but whether it can consult the record, protect trust, handle obstacles and carry the assignment across the finish line.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
enterprise AI record management system
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
trustworthy AI decision support tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
