OpenAI Astra Uses Recurrent Depth — But This Isn't Neuralese

TL;DR: On September 1, 2026, OpenAI’s website said “we are deploying Astra with additional chain-of-thought monitoring,” and The Information reported the same day that Astra uses an architecture making chain-of-thought harder to monitor. Laid against primary sources, both statements can coexist. The recurrent depth Astra uses runs the same layer stack multiple times — publicly estimated at roughly three to four loops, unconfirmed by OpenAI — and this is not the same thing as neuralese. What’s worth holding onto from this episode is something else: loop count has become a dial that can be turned up at inference time, and outsiders have no way to read the scale.

On September 1, OpenAI published “Path to Astra.” The document contained this sentence: “We are deploying Astra with additional chain-of-thought monitoring to rapidly detect and contain potentially misaligned behavior.”

The same day, The Information reported that Astra uses an architecture in which part of the model’s reasoning no longer appears in natural language.

Two documents, one day apart, seemingly describing opposite things. The 48 hours that followed were largely spent on social platforms circling that contradiction.

In 2025 I wrote “What Is Neuralese? When AI Thinks in Non-Linguistic Reasoning, Human Linguistic Sovereignty Ends.” That piece included a sentence that now needs updating: “To date, no major AI company has formally implemented Neuralese CoT in a frontier model.”

This piece is that update — though it runs in a different direction from the versions I’ve been reading in Chinese-language coverage.

What Actually Happened, September 1–3?

The timeline is worth establishing first, because much of this debate stemmed from different parties working from different amounts of information at different moments.

September 1: The Information reported that a source familiar with the matter said OpenAI uses recurrent depth — also called looped transformer — in Astra. The report also noted that OpenAI had constrained the technique’s use to preserve chain-of-thought readability. The same day, OpenAI published “Path to Astra,” announcing that Astra was the first model to reach the “Critical” cybersecurity capability threshold under its Preparedness Framework.

September 2: Redwood Research AI safety researcher Ryan Greenblatt wrote on X that if the reporting was accurate, this was “possibly the worst single development for AI safety and security to date.” AI Futures Project’s Daniel Kokotajlo posted an expletive. Hours later, OpenAI chief scientist Jakub Pachocki responded publicly: “I want to avoid a race to unmonitorability driven by confused reporting. The computation graph depth of our current frontier models, including Astra, is within 2x of GPT-4.”

The same day, Sebastian Raschka published a technical clarification on his blog, stating directly that looped transformers are less exotic than they sound. Independent researcher Rauno Arike posted a point-by-point analysis on LessWrong, separating the reasons for concern from the reasons not to be.

September 3: Fortune and The Transformer each published roundups. Chinese-language coverage began appearing the same day, including headlines like “AI Reasoning Completely Disappears” and “Totally Unreadable by Humans.”

From the first report to the first technical clarification: less than 24 hours. What reached Chinese-language readers was mostly the first half.

What Is a Looped Transformer? How Does It Differ from Neuralese?

Getting the mechanism clear first gives the analysis somewhere to stand.

In a standard Transformer, when the model processes a token, information flows sequentially from the first layer to the last, each layer with its own weights, and the model outputs after one pass. A looped transformer reuses one section of layers: after a full pass through that block, the output is fed back into the same block for another pass, and this repeats until the loop count is reached before moving on.

Raschka gives a concrete example using the open-source model Nanbeige 4.2. It runs a 22-layer stack twice, achieving the effective depth of 44 layers, while the storage footprint remains that of 22 layers. The cost is roughly double the compute. The technique has been studied in the academic literature since at least 2023. NeurIPS’s Mixture-of-Recursions paper went further, adding a router so that simpler tokens take fewer loops while harder ones take more.

Neuralese is a different matter. It refers to a model reasoning almost entirely in high-dimensional latent space, producing no human-readable intermediate steps. In plain terms: chain-of-thought is writing out each step of your working on scratch paper; neuralese is doing everything in your head and reporting only the answer.

Raschka’s key sentence: reusing layers doesn’t suppress the visible chain-of-thought. It simply performs additional hidden-state computation before outputting the next token — the same category of operation as what ordinary Transformer layers do. He also notes directly that scaling a model up (say, from GPT-5.6 Luna to Sol) produces the same kind of effect.

Both approaches push more computation outside of text. But recurrent depth moves in that direction by a much smaller distance.

What Does Pachocki’s “Within 2x of GPT-4” Actually Mean?

This was the single most information-dense statement in the debate, and the one most likely to be skipped over.

“Depth” here refers to how far a model can think before it must write something down.

A quick clarification to head off the easiest misreading: this is about “running a few more layers before producing the next word,” not “writing more draft text.” Depth counts the layers of computation stacked before each token is generated, not the length of the draft. Drafts can run very long while each step’s depth remains shallow. Reasoning models are built exactly on that principle — shallow per step, but many steps.

GPT-4 is a 2023 model that can’t think very far in a single pass. That’s precisely why subsequent reasoning models needed long drafts to sustain extended inference. If Astra’s per-step computation depth is at most twice GPT-4’s, it still needs to write most of its reasoning down.

Rauno Arike worked from this to estimate Astra’s loop count at roughly three to four. His reasoning: a looped reasoning model’s individual layer stack is probably somewhat shallower than GPT-4’s, but not by more than half. OpenAI has not confirmed this estimate. It is, however, the most concrete number derivable from the public record, and it’s compatible with what The Information itself reported — that OpenAI constrained the technique’s use.

Three Places Where Chinese-Language Coverage Amplified the Signal

Reading through several Chinese-language pieces, one carried the headline “OpenAI Astra Neuralese Exposed: High-Dimensional Vector Thinking Makes AI Reasoning Completely Disappear.” It wasn’t entirely wrong — the piece even wrote in a later paragraph that “Astra uses recurrent depth, not full neuralese.” The problem was the amplification ratio.

First: the headline and body contradicted each other. The headline asserted that Astra thinks in neuralese and that reasoning has disappeared; the body acknowledged Astra uses recurrent depth and that OpenAI explicitly denied moving toward neuralese. Two opposite conclusions in one article, and most readers will remember the headline.

Second: both numbers in the Huginn claim were distorted. The piece stated that the University of Maryland’s Huginn model “used 3.5 billion parameters and 50 loops to match the performance of a 50-billion-parameter model.”

Back to the paper’s abstract: the model is 3.5B parameters trained on 800B tokens, with performance gains described as “up to a computation load equivalent to 50 billion parameters.” That sentence is about compute load equivalence. A 3.5B model running more loops can consume compute approaching that of a 50B model; how it performs on benchmarks relative to a 50B model is a separate question the abstract doesn’t claim to settle.

The loop count doesn’t match either. Per Rauno Arike’s summary, Huginn used up to 32 loops during training and was extrapolated to 64 during testing. The number 50 doesn’t appear in the abstract; it may come from a figure in the body, or may have collapsed with the “50 billion” figure during translation. I won’t speculate. What’s clear is that a statement about compute cost was read as a statement about capability — and capability claims carry much more weight.

Third: the direction of the trend is far less certain than presented. The piece’s implied conclusion was that loop counts will keep climbing.

I had originally planned to cite a series of data points showing that larger models actually use fewer loops — evidence drawn from a literature review that Rauno Arike noted was largely AI-generated, with most papers unread by him personally. When I pulled the abstracts of those papers individually, none of the loop-count figures appeared in the abstracts.

The abstracts were saying something else. The Loopie authors open by stating that the longstanding challenge in this area is that “given equal pretraining compute, scaling parameters by N typically outperforms running N loops” — and their contribution is precisely to overcome that limitation. Fu et al. address the training instability that emerges at higher loop counts, while also stating explicitly that “loop count can be adjusted at inference time” is a native feature of the architecture.

In other words, what the 2026 cohort of looped transformer papers is doing is making more loops viable.

So I can’t say Chinese-language coverage had the trend backwards. I can say it treated an actively contested limitation as settled fact, and used Huginn as evidence — when Huginn doesn’t actually support that conclusion.

I’m leaving this section in as a record of a hole I fell into myself: I was about to use an AI-generated literature summary to call out someone else’s citation problems. When I checked the primary sources, the figures didn’t hold up. The difference is that they didn’t make it into my final draft.

A Counterintuitive Finding: Recurrent Architectures May Be Easier to Interpret

This point was entirely absent from Chinese-language coverage.

In the comments on Rauno Arike’s post, two researchers who had run related experiments made the same observation independently: if a model is trained to operate with variable loop counts, then the intermediate state at the end of each loop must itself be decodable as a valid token — because during training, any given loop might be the last one.

That constraint doesn’t exist in an ordinary deep Transformer. The intermediate state at layer 37 has no obligation to resemble any word.

One researcher followed up with results from testing on Ouro-2.6B: more than 95% of tokens produced the same output across each loop, and the tokens that changed were precisely the ones marking positions where the model actually used the additional depth. He also noted this falls short of a full chain-of-thought — you get single-token-level readings rather than coherent stretches of reasoning.

One prerequisite is worth stating clearly: this property depends on the model being trained to operate with variable loop counts. If Astra uses a fixed loop count where intermediate loops are never required to output independently, the property wouldn’t apply. Which of these Astra is, there’s no public information to say.

So I’m not offering this as reassurance. But it does illustrate something: treating “more architectural complexity” as straightforwardly equivalent to “less transparency” is an intuition that hasn’t been verified.

What Should Actually Concern Us?

Having cleared away the misreadings, what remains is still worth worrying about — just not in the place Chinese-language coverage identified.

First, why so many people are guarding this particular line. A 2025 paper with researchers from OpenAI, Anthropic, and Google DeepMind described chain-of-thought readability as a rare but fragile safety opportunity and called on the field to study how to preserve it. It’s fragile because it’s a byproduct of training — there’s no mechanism guaranteeing it persists.

A natural objection: if running more loops produces the same kind of effect as making a model deeper or wider, isn’t turning up the loop count just equivalent to switching to a larger model? Why doesn’t anyone call a bigger model a “dial”?

Rauno Arike offers a distinction I find sound. Making a model deeper costs you at pretraining: you have to run an expensive training run again, and once it’s done, the depth is fixed — you can’t add to it at inference time. Recurrent architecture is different. You can train on low loop counts, then push the count higher at inference time with minimal fine-tuning or even zero-shot. Same class of change, completely different cost structure. “Dial” refers to the second case.

Greenblatt, writing after Pachocki’s response, addressed exactly this: Pachocki’s statement is entirely compatible with “a dial currently set very low that can easily be turned up.” He listed three questions he wanted answered, including whether this architecture introduces a new depth parameter that is natively suited to being amplified.

Apollo Research’s Marius Hobbhahn was more direct: as long as greater depth buys greater capability, the pressure points toward going deeper.

There’s a particular slope to this: each individual step seems manageable, but looking back after enough of them, you’re far from where you started. That’s why some argue the first step shouldn’t be taken at all.

There’s one more layer that I think is most practically useful for readers in Taiwan. Dean Ball, head of OpenAI’s Strategic Futures policy team, pushed back on the panic while pointing out that the entire industry was evaluating a technically complex claim on social media timelines with almost no verifiable factual baseline. His conclusion: this is precisely the argument for legally mandated independent audits and technical assessments.

A reader in The Transformer’s comment section added a specific reference point, which I verified against the legislative text: EU AI Act Annex XI, Section 1, Article 1(d) requires general-purpose model providers to document “the architecture and number of parameters”; Section 1, Article 2(b) requires documentation of key design choices and their rationale; models with systemic risk face additional obligations under Section 2, Article 3, requiring a detailed description of the system architecture and how software components interact.

So the framework exists. The gap is in who gets to see it. That documentation goes to the AI Office, not to the public. A chief scientist’s depth figure posted on X and a technical document submitted to a regulator are two different kinds of thing.

What You Can’t Read, You Can’t Govern

There’s a small piece of my own workflow that this touches directly.

My usual practice is to send the same piece of code or analysis to agents from several different vendors simultaneously, then put their disagreements on the table and judge for myself. This only works because they’re willing to write their reasoning in words I can read. I don’t necessarily trust their conclusions, but I can see how they got there — and decide whether to accept the path.

If those agents were to switch to exchanging information in a format more efficient for them and unreadable by me, this cross-checking would degrade to a vote count: how many said yes, how many said no, with no access to the reasoning. That’s the same thing I wrote in my earlier piece — losing even the most basic supervisory tool of asking a model to explain itself — just compressed to the scale of a single desk.

My read on this situation: neuralese did not happen. What happened is that the model gained a dial controlling reasoning depth, and outsiders have no channel to know where it’s currently set. They can only take the vendor’s word for it.

The sentence I wrote over a year ago in “What Is Neuralese?” — “no major AI company has yet formally implemented Neuralese CoT in a frontier model” — technically still holds.

What needs updating is the second half: this is no longer a binary question of whether it’s been done. It’s a question of how far it’s been taken, who has the authority to check, and who sees the results.

All three questions currently have no answer.

For more discussion along this line, head to “Intelligence & Order → AI Governance.”