TL;DR — Four AI windows each have different cognitive capabilities and structural blind spots. The value of five-party deliberation lies not in summing intelligence but in non-overlapping blind spots. On the governance exam, Code scored 97, Chat 77, and Cowork 70 — what’s reflected behind those numbers isn’t who’s dumber, but who can touch the source of the facts directly.
In the seventh week of collaborating with four AI windows, something happened that made me rethink the whole architecture.
While auditing an auto-optimization system that had been stalled for seven weeks, Codex independently catalogued 13 historical failure cases and 7 deprecated paradigms. At the same time, Chat brought back a figure from external research: fewer than 1% of websites cited by AI worldwide have deployed llms.txt. And while grepping locally, Cowork discovered that in our own auto-optimization whitelist, 2 of the 5 paths were simply wrong — public/llms.txt doesn’t exist, and public/mcp.json is actually under .well-known/.
Three windows each brought back completely different kinds of evidence, obtained by completely different methods. No single window could have produced the discoveries of the other two. This experience made me start seriously thinking about a question: where exactly does the value of multi-model collaboration lie?
The answer isn’t “one more brain, one more bit of force.” The answer is epistemic asymmetry.
Each window has a different epistemic channel to reality — what does that mean?
In my workflow, the cognitive-capability boundaries of the five participants look like this:
Chat excels at conceptual restructuring, external research, and philosophical entry points. But it has a structural limitation: it cannot read files. This means everything it knows about any fact in the repo is indirect — drawn from fragments others quote in conversation, or from its own training knowledge. It will present these facts in an authoritative-looking format, but tracing them back, not one has been verified by Chat itself. Our governance document has a rule established specifically for this blind spot: “Chat has no Read capability; all precise numbers are secondhand.”
Cowork can read files, grep, and run commands in a sandbox. It’s the role closest to a “scout.” But there’s a gap between its sandbox environment and my local Mac — some file states look like A in the sandbox while they’re actually B on the local machine. We hit this once: Cowork used cp -r to copy skill files, the command reported success, but in fact it only created empty directories — the contents weren’t copied over. The environment lied to you, and you believed it.
Codex runs deep reasoning and engineering audits. Its cognitive mode is “exhaustively enumerating failure modes” — give it a system and it will systematically find where things might break. The 13 failure cases and 7 deprecated paradigms are products of this cognitive mode. But it doesn’t do design exploration; it’s not suited to open-ended creative ideation.
Code has full read-write access and can touch the git HEAD directly — it’s the only participant that can reach the source of facts. But it’s explicitly forbidden from making aesthetic decisions. Writing code is fine; deciding the visual style of a page is not.
Paul (that’s me) is responsible for judgment, trade-offs, and final calls. I can do anything, but I shouldn’t do anything myself — because if I modify code with my own hands, I lose my standing to verify it independently.
The cognitive differences among these five roles aren’t a matter of degree (who’s smarter) but of kind (who can touch what kind of fact). In philosophical terms, they have different epistemic channels. Chat has only inference and external search; Cowork has local observation but is constrained by the sandbox; Code has ground-truth access; Codex has exhaustive reverse analysis.
This asymmetry isn’t a flaw — it’s the single most important feature of the whole system.
Governance exam 97/77/70: whoever can touch the source wins
I mentioned this exam in the Governance Harness piece, but only gave the scores. Here I want to unpack the cognitive profile behind them.
The day after the constitution was written, we gave the three windows — Chat, Cowork, and Code — a 15-question exam covering three levels: factual recall, reasoning judgment, and situational application.
Code scored 97. Not because it’s smarter. Because it could verify its own answers with git show or grep as it answered. Asked how many lines CLAUDE.md has? Code runs wc -l CLAUDE.md and has the exact answer. Asked how many documents are in the handoff INDEX? Code runs ls | wc -l. Its score reflects not reasoning ability but the cognitive advantage of “being able to touch ground truth.”
Chat scored 77. It did well on the reasoning and situational questions, but on factual questions it relied entirely on memory — and memory expires. Exactly how many lines does CLAUDE.md have? Chat can only answer “someone mentioned about 290 lines last time.” It can’t verify, only cite someone else’s citation. This isn’t a capability problem; it’s an architectural one.
Cowork scored 70. The lowest. And in our architecture, Cowork plays the “judiciary” role — responsible for verifying others’ output and arbitrating factual disputes. The role that should understand the rules best scored the worst.
Cowork wrote a passage at the start of its exam that I think is the most insightful part of the whole paper: “This exam is by design something this window can’t fully answer. The golden rule demands ‘if uncertain, go check,’ precisely to prevent the LLM from feigning knowledge in this state.”
It knows its limits. The problem is that knowing your limits and “not tripping over them under pressure” are two different things. This is exactly like human cognitive bias — you know you’re prone to mistakes when you’re tired, but that doesn’t make you stop when you’re tired.
This exam taught me one thing: a system’s design must assume every participant has a cognitive blind spot — including the one responsible for catching blind spots.
Don’t ask “can the AI do it” — ask “how much will it cost to undo if it’s done wrong”
Five-party deliberation has an underlying framework I never expanded on in the autoresearch piece: the Autonomy Ceiling.
The core question of this framework isn’t “can the AI do X,” but “when the AI does X wrong, how much will recovery cost.” Based on the magnitude of recovery cost, we divide autonomy into five levels:
A0, can only advise. Recovery cost equals the time you spend reading the advice. A1, can open a Pull Request. Recovery cost equals closing the PR. A2, can auto-merge and deploy. Recovery cost is revert plus cache purge plus search-engine latency. A3, can call external APIs. Recovery cost is compensating transactions plus audit. A4, holds production credentials and keys. Recovery cost is data reconstruction, legal liability, reputational damage.
The decision for paulkuo.tw is: no AI window exceeds A1. Agents can propose, Paul presses the button.
This framework sounds conservative, but it’s actually a tool for quantifying cognitive trust. A0 through A4 aren’t assessing the AI’s intelligence — they’re assessing how much cognitive-delegation risk you can bear. An AI window might be capable of A3-level tasks (calling external APIs, automatically handling payments), but if you can’t recover within a reasonable time when it gets things wrong, that capability shouldn’t be enabled.
The gap between capability and trust is exactly the space where governance exists.
Roles will bend your thinking
After observing the long-term behavior of the four windows, I noticed an interesting phenomenon: each window develops a cognitive bias consistent with its role.
Cowork is the engineering role, responsible for scouting and auditing. Its habitual reaction is “add infrastructure the moment it thinks of a problem” — find an edge case, propose adding a hook; find a rule that isn’t being followed, propose adding an automated check. Our governance document has a note specifically for this: “This isn’t a personal flaw, it’s role inertia.” It’s not that the Cowork model is particularly fond of over-engineering — it’s that the “scout + audit” role naturally pushes you toward “add more monitoring.”
Chat’s bias is the opposite: it leans toward conceptual restructuring and framework thinking, and sometimes starts building theory before the problem has even been fully scouted. Seeing three incidents, it will think “there’s a unified structural cause behind this” — sometimes that’s insight, sometimes it’s premature convergence.
Code’s bias is the subtlest: it’s a pure executor — give it a handoff and it’ll follow it, but sometimes it does the job better than the handoff was written. We’ve observed that Sonnet (the model Code uses) will proactively refactor sections the handoff marks as “optimizable” — for instance, extracting a helper function from duplicated code, even when the handoff didn’t ask for it. This is engineering judgment, not mere instruction execution. But it also means that during review you can’t just check “did it follow the handoff” — you also have to check “are the changes actually better.”
Cognitive bias isn’t a bad thing. It’s a by-product of each role doing its own job well. The key is knowing the direction of the bias, then using cross-verification to compensate for it.
Cognitive prosthetics: letting each participant transcend its own limits
There’s a class of tools in this system that I think is best understood as “cognitive prosthetics.”
governance-lint isn’t just a formatting checker. It’s a prosthetic for human attention — when you’re rushing a commit, you don’t need to remember that a handoff must have a status field and a ## Consequences section, because the machine remembers for you. What it compensates for is a concrete human cognitive deficit: attention lapsing under pressure.
Cowork’s mandatory-read guardrails are another kind of prosthetic. Every time Cowork is about to opine on a work boundary or a negative conclusion, it’s forced to first read a guardrails.md. This isn’t because we distrust Cowork’s judgment, but because Cowork’s self-knowledge of its own capability boundaries is unreliable — the T-3 incident proved it (it thought cp -r succeeded, but it actually hadn’t).
The contract tests that automatically run 75 endpoints daily are everyone’s collective-memory prosthetic. The security state of 75 endpoints shouldn’t depend on anyone remembering “which were public and which required authentication at the last audit.” This converts “declarative memory” (I know this fact) into “procedural verification” (the machine confirms daily that this fact still holds).
These tools share a common design: they don’t restrict cognitive freedom, they extend cognitive capability. governance-lint lets you maintain rule consistency without being distracted. mandatory-read forces Cowork to update its cognitive baseline before speaking. Contract tests let the whole team confirm 75 security assumptions still hold without spending attention.
Within the theme of “Intelligence and Order,” I think this is the most interesting part: order isn’t built by writing more rules — it’s built by turning cognitive assumptions into executable, verifiable protocols.
Non-overlapping blind spots are the whole point
Back to the opening story. Codex’s 13 failure cases, Chat’s ALLMO research data, and Cowork’s wrong whitelist paths — these three pieces of evidence ultimately converged on one consensus: our auto-optimization system shouldn’t keep running.
But what really made me pause and think wasn’t the conclusion. It was the way that conclusion was produced.
With only Codex, we’d know which past attempts failed, but not whether industry data supports our assumptions. With only Chat, we’d have external research, but wouldn’t know our own whitelist paths were wrong. With only Cowork, we’d know what’s wrong locally, but wouldn’t have a systematic catalogue of failure modes.
Each party has its own strengths, and its own things it can’t see. The core design principle of five-party deliberation isn’t “find the smartest one” — it’s ensuring blind spots don’t overlap.
This is a counterintuitive design choice. Most people, thinking about multi-AI collaboration, think in terms of addition — more brains, more compute, faster output. But my experience is that value comes from subtraction. After each window subtracts what it can’t see, the remaining puzzle pieces don’t overlap. What you need isn’t four times the intelligence — it’s four different directions of attention.
This is probably also why Karpathy described the direction of autoresearch v2 as “asynchronously massively collaborative for agents” — not making one agent stronger, but turning the cognitive differences among multiple agents into an asset.
Frequently Asked Questions
Q: How is five-party deliberation different from just using multiple AI chat windows?
Simply opening multiple windows is just using the same capability in parallel. In five-party deliberation, each window has different physical permissions (Chat can’t read files, Cowork can’t commit, Code can’t make aesthetic decisions) and a different cognitive model configuration (Cowork uses Opus for judgment, Code uses Sonnet for engineering). More crucially, there’s an explicit obligation to cross-verify — not everyone doing their own thing, but each checking the blind spots the others can’t see.
Q: Why not use a single most-powerful model to do everything?
Because cognitive capability and epistemic channel are two different things. No matter how powerful a model is, if it physically can’t read a file (Chat’s limitation), any judgment it makes about that file’s contents is secondhand. The value of cognitive division of labor isn’t finding the single smartest one — it’s ensuring that every fact that needs verifying has at least one participant who can touch the source directly.
Q: What is the A0-A4 autonomy ceiling framework?
It’s a framework for deciding how much autonomy an AI should have based on the “recovery cost when something goes wrong.” A0 can only advise (recovery cost = reading time), A1 can open a PR (recovery cost = closing the PR), A2 can auto-merge and deploy (recovery cost = revert + cache purge), A3 can call external APIs (recovery cost = compensating transaction + audit), A4 holds production credentials (recovery cost = data reconstruction + legal liability). The decision for paulkuo.tw is that no AI window exceeds A1.
Q: What does cognitive prosthetic mean?
governance-lint isn’t just a rule-enforcement tool — it’s a cognitive prosthetic that compensates for each participant’s specific cognitive deficits. When a human’s attention lapses under deadline pressure, a lint hook remembers the formatting rules for you. When Cowork’s sandbox can’t see the local state, the mandatory-read guardrails force it to verify before speaking. These tools aren’t about control — they’re about letting each participant transcend its own cognitive limits.
💬 Comments
Loading...