Google DeepMind: AI Cannot Produce Consciousness

It is undeniable that we are in an “AI arms race” with other countries to produce the best, most capable large language models and applications for both economic and national security reasons.

But there is another arms race bubbling up underneath the “can we do this?” crowd.

There is another crowd, typically less visible, asking “should we do this?”

I’m not suggesting that we should stop building AI. Rather, I’m suggesting that there are far too many assumptions in this nascent industry around personhood, consciousness, and the value of the human person.

We’re in a philosophical arms race to build a proper foundation underneath this industry to prevent the cultural, spiritual, and socioeconomic fallout of them.

Alexander Lerchner, who works at Google DeepMind, put out a paper that “rugged” the AI fanatics who seem hell-bent on speed-running to the conclusion that AI is conscious.

It’s a much-welcomed paper and serves as a small but important signal that not everyone working in AI is as crazy as we might think.

I’m happy that many share Lerchner’s conclusion that AI cannot be “conscious,” even if they aren’t entirely sure why that is. But there were far too many people who responded with disdain and almost felt threatened by the idea. Why that is, I am not sure.

That being said, I want to offer some thoughts.

I saw several people allude to John Searle’s “Chinese Room” argument already, but there are other areas where this paper also converges with concepts familiar to Aristotelian/Thomistic thinkers.

I want to focus on some of those areas of convergence.

Indeterminacy

In Lerchner’s paper he talks about the “Indeterminacy of Mechanism” and uses what he calls the “melody paradox” to illustrate it.

Imagine a physical machine that just steps through a sequence of electrical voltage changes. On its own, this is just raw electricity operating blindly according to the laws of physics.

However, imagine you hook this device up to a computer and write a specific translation “key” that converts those voltage changes into musical notes.

When you run the machine, your key translates the electricity into a beautiful piece of music, like a melody played forward.

Now, without altering the physical machine or its electrical voltages at all, someone else could come along and apply a completely different translation key to it.

Under the new key, the exact same stream of electricity might perfectly translate into the exact same melody, but played backward.

And yet another person could come along and take the same stream of electricity and machine and translate it to a stream of stock market prices.

The point here is there is nothing inherent to the machine or stream of electricity itself that determines the outcome. There is nothing essential in the physical properties themselves (the electricity or machine) that makes it a song or financial data.

James Ross called this the “indeterminacy of the physical.” In plain language, his argument boils down to the fact that human thought can be perfectly exact, but physical matter is inherently ambiguous.

When we engage in formal thinking, such as doing mathematics or logic, our thoughts have an exact, determinate meaning.

For example, when you square a number or add 2 + 2, your mind grasps an absolute, pure rule that applies to every possible number in existence.

You know exactly what it means to add rather than to subtract, and there is an objective “fact of the matter” regarding the concept you are holding in your mind.

But physical objects and processes are inherently open to interpretation. If you draw a triangle, it could represent the abstract concept of triangularity, but it could also represent a pizza or a pyramid.

Another example, which comes from the ideas of Saul Kripke, considers a calculator or computer.

We assume it is doing “addition.” But imagine a bizarre alternative mathematical function called “quaddition” (or “quus”).

Quaddition works exactly like addition for any number under 57, but for any number over 57, the answer is always 5.

If a machine has only ever computed numbers below 57, there is absolutely no physical feature inside its wires or microchips that proves it is programmed to add rather than quadd.

Because physical things only ever produce a finite number of outputs in the real world, their physical properties are always compatible with multiple, mutually exclusive rules.

So what’s the point?

If your thoughts possess a property (absolute exactness and unambiguous meaning) that no physical process can possibly possess, then your thoughts cannot be physical.

Simulation vs. Instantiation

A common retort to this idea is that we too are not truly adding, we’re simulating adding.

They argue that our thoughts are just as indeterminate as physical matter, and we only trick ourselves into thinking we are grasping pure concepts.

But this argument is self-defeating.

To claim “I am not exactly adding or doing logic,” you first have to have an exact, unambiguous concept of what addition and logic are in your mind in order to deny them.

You cannot coherently deny that your thoughts are exact without using exact thoughts to do so. Therefore, formal thought must be real, and because it is real, it must be immaterial.

Those who propose this simulation argument fall into the trap of conflating the simulation of intelligence with the instantiation of it.

As philosopher Edward Feser points out, a living thing possesses a substantial form and intrinsic teleology, meaning its properties, causal powers, and goal-directed behaviors arise naturally from within the organism itself.

This allows living things to exhibit immanent causation: activities like growing, digesting, and healing that are inherently directed toward the organism’s own self-perfection and survival.

But a computer, by contrast, is a mere artifact and is not a true substance.

My favorite example, from Feser, to illustrate the difference is someone weaving liana vines into the form of a hammock.

There is no inherent or natural tendency in the vines to function as a hammock to sleep in. That function is entirely imposed on them from the outside, making the artifact’s purpose entirely observer-relative.

Whereas a true substance (such as a dog or a tree) possesses a substantial form and intrinsic teleology. Its properties, causal powers, and goal-directed behaviors arise naturally and inherently from within the thing itself.

The computer, as such, possesses only accidental form, meaning it is just a collection of raw materials (plastic, metal, silicon) onto which an organizational structure has been externally imposed.

Because of this, a computer’s purpose is entirely extrinsic or derived; it does not compute or process data because it has any intrinsic, conscious tendency to do so, but solely because human designers arranged its parts to serve human purposes.

The materialists and some AI fanatics conflate the distinction between the instantiation of intelligence and the mere simulation of it.

To use another Feser example, this is like saying because Penn and Teller simulate magic tricks really well, they are actually doing real magic. But anyone reading that sentence will see it as absurd.

In like manner, an algorithm is designed precisely to simulate intelligent behavior without actually producing it. Merely simulating intelligence does not mean it actually IS intelligence.

Lerchner also explicitly separates extrinsic behavioral simulation from intrinsic physical instantiation. He points out that a computer simulating the exact abstract software of photosynthesis will not synthesize a single molecule of real glucose or release oxygen.

Derived vs. Intrinsic Intentionality

Lerchner’s paper is titled “The Abstraction Fallacy,” which forms his core argument and also has interesting convergence with people like Feser, Searle, and Ross.

He argues that symbolic computation is not a physical process that simply unfolds in matter, but rather a “mapmaker-dependent description.” To have computation, continuous physical dynamics must be partitioned into meaningful, discrete states by an active, experiencing cognitive agent: that is the mapmaker.

He concludes that information and symbols are not fundamental building blocks of the universe; they are derivative properties that absolutely presuppose a cognitive agent to define them.

Think of a traditional analog clock.

Physically, it is just a collection of metal gears and springs moving blindly and continuously according to the laws of physics.

The clock itself is not inherently “computing” the time. It only tells time because a human mapmaker decides to map a specific, continuous angle of the metal hands to a discrete semantic concept, like “3:00 PM.”

Without a conscious mind imposing that meaning from the outside, the clock is just metal moving around.

Likewise, Edward Feser (drawing on Searle) distinguishes between intrinsic intentionality (the mind’s inherent ability to mean or represent things) and derived intentionality (the meaning we project onto words, ink marks, or computer wires).

Intrinsic intentionality means thoughts and concepts possess meaning inherently. The concept of a triangle, for example, is essentially about triangles; it is impossible for the concept of a triangle to exist without being directed toward that specific shape.

Whereas derived intentionality explains that physical objects and processes possess zero inherent meaning.

Ink marks on a page, sound waves emitted by a speaker, or electrical currents running through a computer wire do not inherently represent anything. A string of letters like “cat” only refers to a feline animal because of human convention.

This is where we might include Searle’s “Chinese Room,” but the important distinction to be recognized here is the one between syntax and semantics.

Syntax refers to the formal, physical rules for manipulating symbols (e.g., if you see symbol A, output symbol B), while semantics refers to understanding what those symbols actually mean.

Feser drives this point home stronger in his work (you should see his section in Immortal Souls, for example). But the conclusion is essentially that computers are entirely observer-relative. Which is also the point that Lerchner makes in his paper.

The Ontological Inversion

One final point of close convergence is what Lerchner calls the “ontological inversion.” He argues that computational functionalism mistakes the map for the territory.

He’s not committed in this paper to any strict ontological claims that may converge with what many would deem religious claims. But he shows the computational functionalists have the sequence of what can instantiate consciousness backwards.

He begins with physics, then moves to consciousness, which then can produce concepts, etc. Whereas the functionalists would put that process in reverse. He concludes that believing the map (the algorithm) can generate the mapmaker (consciousness) is fundamentally impossible.

Similarly (with much more developed ontological commitments), Feser argues that the mechanical philosophy puts the cart before the horse by trying to explain the conscious subject (the scientist, the intellect) using the abstract, mathematical models (physics, mechanics) that the subject’s own mind created in the first place.

The point here is that both of them argue that computation is a descriptive tool invented by conscious minds, not the physical cause of them.

Let’s Talk More About This

It’s nice to see a paper like this come out from someone close to the AI work being done at one of the “bigs.” It shows, at the very least, a healthy skepticism about the claims people in the industry are making without doing any of the underpinning philosophical work to get there.

It’s no surprise that that is the case, of course, as there is a lot of money tied up in the promises AI industry leaders are making.

The work done by philosophers in this area of thought is incredibly important, and I don’t think it’s an exaggeration to say that their work is absolutely critical for a HEALTHY future where AI is no doubt going to be part of our daily lives and the decisions both companies and government leaders will make.

---

First published on Substack.