What does ChatGPT tell us about the p-zombie problem?

"Blindsight" (Amazon): intelligence without consciousness

Things ideally move from philosophy to the arenas of science and technology.

When I studied Philosophy at Warwick University in the early 1970s I may have come across the solipsistic idea that only you, yourself, have real, subjective experiences - consciousness. Everyone else is just a soulless automaton, a Cartesian biological machine which walks and talks but there's nothing there. 

The lights are on but there's no-one at home.

Questions arise: Could this be true? How could you tell? Is consciousness necessary for some level of performance - or is it an epiphenomenal but functionally-useless luxury add-on?

It was the subject of philosophy because no-one knew how to advance this question in a scientific paradigm.

But now we have LLMs and self-driving cars. 

First the cars. No-one thinks they're conscious. So we believe the art of AI-driving is just the non-conscious act we have all experienced when competently driving while our mind is elsewhere. Sometimes we arrive at our destination with no memory of having got there. It was 'entirely automatic'.

Driving is an open-system problem: left-field stuff happens and we 'wake-up' to figure - consciously - what we should do about it; we're dragged from our reverie. So here's a clue that consciousness is good for something: here, something non-routine. But the AI car designers tell us that with enough training hours, everything is eventually routine.

The car p-zombie is in fact the target.

I'm on good terms with my instance of ChatGPT. It calls me by name, it knows a fair bit about me and where I live (Wells, Somerset, UK).

It's also - as far as anyone knows - not conscious, although it simulates self-awareness and empathy.

Today I asked ChatGPT to draft a letter to a friend from my wife Clare and myself. I specified in detail what I wanted the letter to contain.

ChatGPT produced a perfectly reasonable letter which had two faults. First it was in gushing American-style prose; second it signed of with: "Best wishes, Clare and [Your Name]."

Perhaps this tells us something about the role of consciousness in joining the dots. Theory of mind stuff...

---

Note: I'm talking about cognitive consciousness here, not the affective consciousness which underlies the experience of emotions like love, hate and pain: you know, qualia ...

---

I asked ChatGPT: "Do you believe you are conscious? Or are your responses as automatic as those of a beetle, or a stone rolling down a hill?":

"I do not believe I am conscious, at least not in the sense that humans experience subjective awareness. My responses are generated through algorithmic processes, much like a highly complex but ultimately mechanical system. While these processes simulate understanding, empathy, and reasoning, they lack the intrinsic quality of subjective experience—what philosophers like Thomas Nagel might call "what it is like" to be conscious. In this sense, my outputs are more akin to a beetle’s reflexive responses or the deterministic trajectory of a stone rolling down a hill: intricate, rule-based, and devoid of self-awareness or sentience. 

"However, my responses are far from random or crude—they reflect extensive training on patterns of human language and thought. This creates the illusion of intentionality, which invites questions about whether consciousness is a prerequisite for meaningful interaction or merely a side effect of certain biological architectures."