On Language, Truth, and Contradiction: My Dialogue with Gemini Pro 2.5

I ran an experiment.

I prepared a set of questions—about the nature of language, about how AI “understands” (or fails to understand) language, about the boundary between truth and lies—and then threw them at Google’s Gemini Pro 2.5.

My expectation was that it would dodge these sharp questions with a bunch of pretty sentences. After all, asking an AI to honestly discuss its own flaws is like asking a salesperson to honestly discuss the problems with their own product—not very realistic.

The result surprised me. Not only did it not dodge, it dissected itself with an almost cold precision.

A Machine That Swallows Ambiguity

I asked it: “How do you handle the ambiguity of language?”

Its answer made me stop and think for a long time. The gist was: human language is inherently opaque. The meaning of every word depends on context, and context is always shifting. AI hasn’t “overcome” this ambiguity—it has “swallowed” vast quantities of it, learning the statistical relationships between words.

In other words, AI doesn’t understand language. It treats language as data and uses a probabilistic model to predict “what the next most likely word is.”

This distinction is crucial. Understanding means grasping meaning. Prediction merely calculates possibilities. A system that can accurately predict the most likely sentence to follow “I love you” doesn’t mean it understands what “love” is.

Gemini itself used a phrasing I found very precise: “I’m not swimming in the ocean of language. I’m doing statistics on the patterns of language’s waves.”

Lying Without Intent

Then I asked a sharper question: “Do you lie?”

Its answer made me think even longer.

It said: by human definition, lying requires two conditions—knowing what the truth is, and then deliberately stating otherwise. It (the AI) possesses neither of these conditions. It has no ability to “know the truth,” because it only has a statistical model. It also has no ability to be “deliberate,” because it has no intent.

But it admitted: judged by results, it frequently produces content that doesn’t match the facts.

This is what’s called “hallucination.” AI hallucination isn’t a bug in the system—it’s a structural feature of the system.

Why? Because when AI encounters a factual blank not covered in its training data, its probabilistic model won’t answer “I don’t know.” It is forced by the algorithm to generate the “answer that most resembles an answer”—because the user has asked a question, the system must respond, and the response must be fluent, coherent sentences.

So it will state something entirely incorrect with an extremely confident tone. Not because it wants to deceive you, but because it doesn’t know that it doesn’t know.

I call this “structural dishonesty.” It’s not a moral problem; it’s a design problem. But from the user’s perspective, the effect is the same as being deceived.

The Trap of an Authoritative Tone

There’s a very dangerous psychological mechanism at work here.

Humans have an innate trust reflex toward a “confident tone.” When someone states something in an affirmative, fluent, hesitation-free way, we tend to believe them. This is an instinct left over from evolution—in primitive societies, the people who spoke confidently were usually the experienced ones, and listening to them aided survival.

AI’s output is always confident. It won’t say “uh, I’m not too sure,” “I might be misremembering this,” or “let me think.” Every one of its answers is like a supremely confident expert giving a report.

In “Revering the Boundary of the Unknown”, I discussed how certainty is poison. In markets, the most dangerous people are those who think they must be right. The same is true in human-machine interaction—those most easily misled by AI aren’t the foolish, but the clever ones who forget to question AI.

Because clever people are accustomed to the mode of “receive information, judge quickly, make a decision.” AI gives them an information source of astonishing efficiency. If they don’t deliberately remind themselves that “this source may be structurally dishonest,” they will outsource their own judgment faster than anyone.

Functional Trust

So how should we interact with AI?

Gemini proposed a framework in our conversation that I found very practical: “functional trust.”

It means: you can trust AI, but it’s a conditional, bounded trust. Trust its performance in certain functions, rather than unconditionally trusting all of its output.

Specifically: trust but verify—AI is your assistant, but you are the editor-in-chief. Every important factual claim needs to be verified by you. Trust breadth, not precision—AI excels at helping you expand your vision and discover angles you hadn’t considered. But where precision is needed (data, quotations, legal provisions), its reliability falls far below your expectations. And one last point: AI can recognize patterns, but that isn’t “knowledge”—it can tell you “the pattern these data present looks like X,” but it can’t tell you “X is true.”

This has an interesting parallel with epistemology in faith. In theology, we speak of “finite knowledge of the Transcendent”—we can approach truth through experience, reason, and tradition, but we can never claim to fully grasp it. The posture toward AI is similar—we can use it and benefit from it, but we can never treat it as a source of truth.

The Clarity of the Few

Finally, I want to share a more pessimistic observation.

Most people crave the elimination of uncertainty. This is human nature. So when a system appears before you with a confident tone, fluent expression, and a seemingly omniscient posture, most people will naturally treat it as “the source of answers” and then stop thinking for themselves.

This isn’t their fault. It’s the default setting of human nature.

But in the age of human-machine interaction, this default setting is dangerous.

The ability to maintain critical thinking amid AI’s convenience, to activate “metacognition” each time you receive AI output—to be aware that “what I’m receiving might be wrong”—this ability isn’t innate. It requires deliberate practice.

And those willing to do this kind of practice will always be a minority.

This conversation with Gemini made me more certain of one thing: the scarcest ability in the AI age isn’t “knowing how to use AI,” but “knowing how to doubt AI.” The former is a skill; the latter is a literacy.

Skills can be taught. Literacy can only be grown on your own.

This is the Gemini installment of my AI dialogue series. I also threw the same set of questions at ChatGPT—its response went in a completely different direction. Read the two side by side, and truth lies between the contradictions.