Essay No. 46 · AI
Don't Automate the Org Chart
An autonomous firm still needs objectives, allocation, authority, work, and stewardship. Its organization should earn its shape by delivering something people want.
The easiest mistake to make when building an autonomous company is copying a human corporate directory.
Software builders look at a traditional office, see titles like Sales Director, Operations Manager, Data Analyst, and Quality Assurance Officer, and decide to recreate them in code. They write prompts telling one artificial intelligence (AI) model that it is an executive, another that it is a middle manager, and a third that it is a junior worker. They connect these bots with chat channels and expect a working business to emerge.
Consider a practical example: a small retailer wants to merge two supplier product lists into a single, store-ready catalog. If the retailer hires an outside service to do this, the final spreadsheet has to be accurate. Prices, quantities, and product IDs must line up cleanly. Even if every column passes a basic formatting check, a spreadsheet that accidentally combines a blue shirt and a red shirt into a single item is a failed delivery. Neat columns do not matter if the data inside them is wrong.
When builders design an automated service around fake corporate job titles, they quickly run into trouble. Having an AI "Operations Manager" talk to an AI "Quality Officer" offers zero guarantee that the retailer gets an accurate catalog, at a fair price, without the human business owner stepping in at midnight to fix broken files.
To understand why copying the org chart fails—and how to build an autonomous business that actually works—we have to look at why human companies succeed in the first place.
The human secret: clean charts, messy reality
On paper, every human company looks tidy. The organizational chart hangs on the wall with neat boxes and straight reporting lines. Written job descriptions list clear duties, and every employee appears to stay in a predictable lane.
In reality, human organizations are a mess—and that mess is the exact reason they work.
No successful human business runs strictly by its job descriptions. When real work hits on a Tuesday afternoon, formal titles take a back seat to getting the job done:
-
An account manager spots a glaring typo in a client contract and fixes it instead of sending it back to the legal department.
-
A customer support rep notices that a supplier's spreadsheet uses the wrong currency and walks over to a coworker's desk to sort it out on the spot.
-
A warehouse worker sees that a red shirt and a blue shirt share the same product code, realizes that makes no sense, and sets them aside before they get mixed together.
Human workers have common sense, shared context, and social intuition. They constantly step outside their official roles to patch the gap between what their job description says and what the customer actually needs. The org chart on the wall is a polite fiction; the real company survives on constant, informal problem-solving.
The autonomous dilemma: no common sense, total definition
An autonomous organization does not have human workers quietly fixing mistakes in the background.
Software has no common sense. It does not feel embarrassed when it sends a broken file to a paying customer. It cannot "read the room," and it does not notice obvious real-world problems unless it has been specifically programmed to look for them.
When an autonomous business interacts with the real world, it faces a task that is infinitely harder than running a human office: just about everything has to be defined in advance.
Think back to the retailer's product catalog. In the real world, incoming supplier files are messy. One supplier lists prices in US dollars; the other uses British pounds. One lists shirt sizes as "S, M, L," while the other uses neck measurements in inches. Some rows are missing descriptions entirely.
A human worker handles these quirks on the fly. An autonomous firm cannot. Before the software can do the work, the business must establish strict, explicit rules for every step:
-
What the customer actually bought: Defining the exact file format, deadline, and quality promise upfront.
-
What gets rejected at the door: Accepting catalogs with clear product IDs and documented units, while automatically pausing or rejecting orders that would force the system to guess missing facts.
-
How to verify the truth: Keeping locked, untouched copies of the original supplier files so the final catalog can always be checked against the raw source data.
While a human company relies on people to figure things out as they go, an autonomous company must turn every fuzzy business expectation into a hard rule.
The escalating ladder of intelligence
Because the real world is unpredictable, an autonomous firm cannot use the same tool for every task.
If you use a basic, rigid computer script to merge messy product descriptions, it will crash the moment a supplier changes a column name. On the other hand, if you use a massive, expensive large language model (LLM) to read and rewrite every single row of a 100,000-item catalog, your server bill will cost more than the customer paid for the job—and the AI will eventually hallucinate a wrong price along the way.
To survive, an autonomous company has to deploy intelligence on an escalating scale:
-
Level 1: Basic Code
Fast, exact, almost free
(Math, formatting, matching IDs)
(Only if data is messy)
-
Level 2: Simple Classifiers
Quick, predictable sorting
(Grouping product categories)
(Only if human-like judgment is required)
-
Level 3: AI Agents (LLMs)
Flexible, slow, expensive
(Resolving confusing descriptions)
In their engineering guide Building Effective Agents, researchers at Anthropic point out that builders should always prefer simple, predictable workflows over complex AI agents—introducing multi-agent intelligence only when simpler code cannot handle the job.
Here is what makes an autonomous organization so difficult to engineer: the system has to decide which level of intelligence to use before the work even starts.
There is no human office manager reading incoming emails each morning to decide which jobs are easy and which ones are hard. The autonomous system’s front door—its intake gate—has to inspect the customer's files automatically. Before spending a dime on heavy computing power, the system must evaluate the job: Can this catalog be merged with two pennies of basic code, does it need a specialized classifier, or does it require an expensive AI agent to untangle?
Setting objectives, allocating resources, and choosing the right tool must happen automatically at the very beginning of the pipeline.
Why synthetic roles collapse under real work
This brings us back to why traditional job titles fall apart when you turn them into software.
On paper, setting up an AI "Catalog Builder" and an AI "Quality Inspector" sounds responsible. It looks like a real company with checks and balances. But when difficult real-world work arrives, those titles offer fake security.
Imagine the "Catalog Builder" AI looks at the red and blue shirts, sees that they have similar product numbers, and merges them into one item. It writes a confident summary explaining why the merge made sense. Next, the "Quality Inspector" AI reads the Builder's summary. Because both agents are powered by similar language models and look at the same summary, the Inspector agrees with the Builder and stamps the file "Approved."
Giving two bots different job titles does not make them independent. It just gives you two chatbots nodding along to the same mistake.
A recent study by researcher Mert Cemri and his colleagues (Why Do Multi-Agent LLM Systems Fail?) examined why multi-agent systems break down. They found that failures cluster around three main problems: bad system design, agents misunderstanding each other as they pass messages back and forth, and verification steps that fail to catch real errors.
For an autonomous check to be real, it cannot rely on role-playing. It requires structural boundaries—the kind described in the government cybersecurity standard NIST SP 800-53 (Control AC-5, Separation of Duties). While written for computer security, its core rule applies directly to autonomous work: a boundary is only real if it restricts what an actor is allowed to do.
In practice, that means:
-
Separate Permissions: The part of the system that builds the catalog cannot have the power to rewrite the rules used to grade it.
-
Direct Access to Facts: The checker must compare the final spreadsheet directly against the original supplier files, not against the builder's explanation.
-
Power to Stop the Line: The verification step must have the hard authority to block a bad delivery, no matter what the rest of the system wants.
Where familiar roles still make sense—and how to test them
There is one strong argument for keeping a few familiar organizational roles: a company needs long-term memory that no single customer order can see.
If you treat every retailer's order as an isolated workflow that disappears the second the file is delivered, nobody is watching the bigger picture. Someone—or some system—has to notice that a specific supplier's catalogs are always broken and eating up profits. Someone has to decide whether supporting a new file format is worth the cost.
A dedicated coordinator or steward role that looks across hundreds of past orders can catch these long-term patterns much better than a narrow worker script. Similarly, Anthropic's June 2025 report on its multi-agent research system showed that using a lead agent to coordinate specialist sub-agents produced better results on complex problems than using a single agent alone—even though it used more computing power and created new coordination challenges.
We should not reject a manager or specialist role just because it looks like a human job title. Instead, we should test the organization on its results, not its appearance.
If we want to know whether a role-based setup beats a direct workflow pipeline for our catalog business, we can run both side-by-side on the exact same orders, with the same budgets and the same messy edge cases—and with zero hidden human help behind the scenes. We can measure both setups on four plain facts:
-
Accuracy: Did the blue and red shirts stay separate?
-
Total Cost: How much computer power did the job take?
-
Error Catching: Did the system stop when given bad data, or did it guess and fail?
-
Adaptation: When the same error happened five times in a row, did the system update its rules on its own?
If a specialist or coordinator role improves those numbers enough to pay for its extra computing cost, it stays. If it just passes messages back and forth without improving the final catalog, it gets cut.
How the system actually runs without humans
Saying that an autonomous company needs smart intake, escalating tools, and strict checks raises a final, practical question: if there are no humans in the loop, who writes the basic code, who checks the work, and who stops the AI from draining the bank account when it gets stuck?
If a human programmer still has to step in every time a new catalog format arrives, the business isn't autonomous—it is just a normal software service with fancy tools. To run without daily human management, an autonomous organization relies on four self-governing mechanisms:
1. AI builds the assembly line, not the product
When a messy file arrives from a brand-new supplier, the system cannot wait for a human to write custom code, and it cannot afford to have an expensive AI model process all 100,000 rows by hand.
Instead, the intake system pulls a tiny sample of 20 rows. It wakes up an advanced AI agent once. That agent's job is not to convert the catalog; its job is to write a cheap, standard computer script that converts those 20 rows. Once that script passes every verification test on the sample, the expensive AI turns off, and the newly written script processes the remaining 99,980 rows for pennies. Expensive intelligence is used to build the tool, not do the repetitive labor.
2. Blind testing and math locks
To keep a checker from lazily agreeing with a builder, the system uses blind reverse-testing.
First, basic code checks the hard math: Did we start with 5,000 total items across both supplier files and end up with 5,000 items in the final catalog? If the numbers do not match, the job fails instantly. Second, when an AI verifier checks a sample of the final catalog, it works in a "clean room." It never sees the builder's notes or excuses. It is simply given the final merged row and forced to match it backward to the locked original supplier files. If two distinct product IDs went in (the red shirt and the blue shirt) and only one came out, the reverse check fails and blocks the delivery automatically.
3. Hard budgets as circuit breakers
In a human office, a manager notices if an employee spends three days stuck on a twenty-dollar order. In an unattended AI system, two agents can get stuck in an endless loop, arguing with each other all weekend and running up a five-thousand-dollar server bill.
To prevent this without a human boss, every customer order gets a hard compute wallet the second it enters the system. If a retailer pays $20 for a catalog merge, the system locks that job into a strict spending cap of, say, $2 in computing power. Every time the system has to retry a failed test or escalate to a smarter AI model, it spends from that $2 wallet. If the balance hits zero before the file passes inspection, an automatic circuit breaker trips: the system kills the job, refuses to send bad data, and notifies the customer or owner.
4. Turning today's hard lesson into tomorrow's cheap rule
In a human company, departments build long-term memory through experience. An autonomous company builds memory by saving its proven code.
Whenever an expensive AI agent successfully writes a new rule to handle a weird supplier quirk—and that rule passes every independent verification gate—the system automatically saves that script into its permanent library. A problem that cost fifty cents of heavy AI thinking on Monday becomes a free, automatic lookup rule on Tuesday.
Let the organization earn its shape
Passing a narrow test on catalog files does not mean software is ready to run an entire portfolio of companies with zero human oversight. Cloud servers, payment processors, and outside suppliers remain real-world dependencies, and every time a business takes on a new type of work, it creates new risks that have to be tested.
Still, the path forward is clear. A single human owner can set the high-level goals, provide the budget, and define the boundaries, while an autonomous organization allocates resources, builds its own workflows, and keeps promises to paying customers. As that organization takes on bigger responsibilities, its internal rules and checks will become more important, not less.
For the retailer waiting on their product catalog, the job titles on your software diagram do not matter. What matters is a firm that can take an order from a stated need to an accurate, usable result. The organization behind that delivery should make clear how the work was done, what had the authority to spend money or change rules along the way, and which independent checks had the power to stop a bad file from going out the door.
When an organization's chart emerges from solving those real problems, we will have a real reason to believe the system can do the work its diagram describes.
Sources
- Building effective agents — anthropic.com
- NIST SP 800-53, Control AC-5 (Separation of Duties) — nvlpubs.nist.gov
- Multi-agent research system (June 2025) — anthropic.com
- Why Do Multi-Agent LLM Systems Fail? (October 2025 revision) — arxiv.org