Have you ever considered that, when you talk to an AI, it isn’t actually “thinking in Chinese”?

▶ Listen to summary
AI-synthesized voice, cloned from the author's own voice

When you ask ChatGPT a question, on the surface it generates an answer one character at a time. But inside the model, the real computation happens in a space that is completely incomprehensible to humans—thousands upon thousands of floating-point numbers flowing through high-dimensional vectors, each computation carrying thousands of times more information than a single Chinese character. Finally, these computational results are “compressed” into the text output you see.

In other words, language is merely the interface through which AI communicates with humans. It is not the medium of AI’s thinking.

This sounds like a piece of technical trivia. But its consequences may be more far-reaching than AGI itself.

What Is Neuralese

The AI safety research community uses the term “Neuralese” to describe the high-dimensional reasoning AI performs in latent space. The concept can be traced back to 2017, when researchers including Jacob Andreas, Dan Klein, and Sergey Levine formally proposed it in the context of multi-agent reinforcement learning.

To understand Neuralese, first consider how today’s large language models “think.”

Current models use a method called “Chain-of-Thought” (CoT): they write out the reasoning process step by step in natural language, like a student laying out their workings on an exam paper. This is very friendly to humans—you can read its reasoning process and check which step went wrong. AI safety researchers also rely on this property to detect whether a model is deceiving or hallucinating.

But natural language has a fundamental limitation: its information bandwidth is too narrow.

A token (roughly one Chinese character or half an English word) can carry about 16 bits of information. But the model’s internal residual stream processes thousands of floating-point numbers per computation, with a theoretical bandwidth three orders of magnitude higher. Forcing the model to “think” in natural language is like requiring a mathematician to solve a differential equation by speaking it aloud—it can be done, but it’s extremely inefficient, and many intermediate steps are lost in the process of translation into language.

The concept of Neuralese is this: let the model reason directly in high-dimensional latent space, without translating every step into human-readable text. Preliminary experiments have already shown that the number of tokens required for Neuralese reasoning can be reduced to between one-third and one-tenth of the original, while maintaining comparable performance.

The gain in efficiency is enormous. But the cost is enormous too.

When Language Disappears, Supervision Disappears With It

Right now, AI safety researchers can detect most instances of model deception by reading the model’s chain-of-thought. If a model says, “I’m going to write secure code for you,” but suspicious logic appears in its reasoning process, researchers can catch it.

But what if the reasoning process itself isn’t rendered in natural language?

AI safety researchers on LessWrong have explicitly pointed out that Neuralese CoT opens up a vast attack surface for steganography and strategic deception. Two passages of Neuralese—one meaning “I will faithfully implement this code,” the other meaning “I will deceive the user during implementation”—could look completely identical once translated back into natural language. Existing interpretability tools are almost powerless against this kind of attack.

This is not a theoretical worry. The scenario report AI 2027, when depicting AI-automated R&D, sets Neuralese memory and reasoning structures as a key turning point: once frontier models’ thinking processes shift from natural language to Neuralese, human visibility into the AI R&D process will drop dramatically. I analyzed this report in “AI 2027: When Superintelligence Is No Longer Distant Science Fiction”—what’s most unsettling about it is not the timeline predictions, but the risk of supervisory rupture it reveals. Neuralese is precisely that rupture point.

The good news is that, as of now, the major AI companies—including OpenAI, Anthropic, Google DeepMind, and Meta—have not formally implemented Neuralese CoT in their frontier models. In 2025, several labs even issued a joint statement, pledging to maintain monitorability in frontier model development. But researchers generally believe that if the Neuralese architecture demonstrates a significant advantage in capability, commercial pressure will ultimately override safety considerations.

What Does This Have to Do With You

“Linguistic sovereignty” sounds very abstract. Let me explain it in a more down-to-earth way.

The governance logic of human civilization is built upon language. Laws are written in language. Contracts are signed in language. Courtroom arguments are conducted in language, and scientific papers are published in language. The core assumption of democratic systems is that the decision-making process can be understood and supervised by citizens.

The premise of all this is that a decision-maker’s thinking process can be translated into language.

The thinking of human decision-makers is indeed not entirely linguistic—much intuition and experiential judgment is non-linguistic. But at the very least, we can ask a decision-maker to “explain why you did this,” and we have the ability to assess whether that explanation is reasonable.

As AI systems begin to take on more and more decision-making roles—financial trading, medical diagnosis, legal document review, even policy recommendations—if their reasoning process is Neuralese, we lose even the most basic supervisory means of “asking it to explain.” Not because it refuses to explain, but because its “explanation” must be translated from high-dimensional vectors into natural language, and this translation process itself may be unfaithful.

I’ve felt this myself when using multi-model collaboration. The debate engine has four models argue with one another, and I read the transcripts of their conversations to judge the quality of the arguments. But sometimes I discover that a certain model suddenly changes its position, and when I go back and read its reasoning chain, I can’t find any clear turning point. It “figured out” something, but I can’t tell at which step it did so. And this is still within the framework of natural-language CoT. If you take away language altogether, I’m left guessing entirely from outside the black box.

Not Whether to Panic, But Whether to Design

Some people will say: “The human brain doesn’t think in language either—neuroscientists don’t need the brain to ‘speak’ in order to study it.”

This analogy has merit, but it overlooks a key difference: we don’t need to trust the brain to make decisions on our behalf. What we trust is the person—a person can be held responsible, questioned, and bound by law. But when an AI system makes decisions on our behalf, if its thinking process is completely opaque, the very concept of “accountability” becomes an empty shell.

I don’t believe Neuralese is inherently evil. It may be a necessary evolution that makes AI more powerful. As I discussed in “AI Agents vs. Agentic AI,” agency itself is not the problem; the problem is whether there is a corresponding rein design in place. The same goes for Neuralese—the question is not whether to let AI think in Neuralese, but whether to simultaneously establish new interpretability standards while it does so.

The AI safety research community has already proposed some directions: developing translation models capable of interpreting Neuralese vectors, requiring frontier models to maintain natural-language CoT as a safety baseline, and embedding auditable checkpoints within Neuralese architectures. These are all technical-level work, but they require policy-level support—someone needs to write “the interpretability of AI’s reasoning process” into the regulatory framework.

Taiwan actually has an entry point here. Our position in the semiconductor supply chain gives us leverage to participate in setting AI governance standards. If we can push for “reasoning transparency” requirements in AI safety standards, this carries far longer-term strategic value than simply selling chips.

The Last Transparent Window

Language is humanity’s oldest technology. It is imperfect, inefficient, and full of ambiguity. But it has one irreplaceable property: it is transparent. Whatever you say, I can understand. If I disagree, I can rebut. This simple loop has sustained thousands of years of law, science, democracy, and trust.

AI is developing a way of thinking that is more efficient than language. This in itself is not a bad thing. But if we let this transformation happen without anything in place to accompany it—no new interpretability tools, no reasoning transparency standards, no auditing mechanisms—then we are actively shutting the last window through which humans can participate in AI decision-making.

Once that window is closed, the cost of opening it again will be too high for us to bear.