Engineering consciousness?

---

Richard Feynman once remarked that "unless you can build something, you don’t really understand it." This simple observation underpins the very nature of scientific inquiry and technological progress. If you cannot replicate a phenomenon through engineering, then your understanding of it is at best incomplete.

But when we apply this insight to consciousness we are confronted with an unnerving reality: despite our increasing competence in building systems that outperform anything in the natural world: aircraft that outmatch birds in flight, tanks that could defeat any armoured reptile, and computer systems that easily outthink humans, we still cannot engage with consciousness itself.

This profound inability suggests a paradigmatic gap in our understanding. It seems that we are missing something fundamental about what consciousness truly is and how it might have emerged.

Modern AI systems converse, navigate, play games at superhuman levels. Yet they are not conscious. They might simulate understanding but they do not feel. They lack subjective experience.

To date, no engineering specification has required, or led to the emergence of, consciousness. While we have designed systems to meet every requirement we could imagine - intelligence, adaptability, problem-solving - none of these systems exhibit the subjective what-it’s-like quality of conscious experience. Have we overlooked some crucial aspect of what makes human beings not just intelligent, but conscious?

We turn to introspection. Take the example of driving a familiar route. Once the skill is mastered, the brain processes the task automatically, our conscious mind is elsewhere. However, when something goes wrong - say, an unexpected obstacle appears - we "wake up" to the problem. Our conscious mind kicks in, calling upon a broader model of the world, seeking a solution.

This shift from automatic processing to conscious thought hints at a key feature of cognition: consciousness arises when the system is confronted with a situation that can’t be resolved by routine processes. When the 'closed world' assumed by instinct expands to the 'open world' of coupled reality.The brain reaches out to a wider pool of knowledge, seeking to integrate more complex information, a 'bigger picture' to formulate a solution.

In these moments of cognitive consciousness, the brain is not simply processing more information, but is actively integrating a more expansive model of the world. This expansion of awareness feels like a conscious step - an intervention from the "bigger world" outside the confines of the immediate task.

The crucial question, however, is why this transition is accompanied by a conscious experience. Why does the mind 'wake up into self-awareness' when faced with complexity?

And this is not even the full explanation of consciousness. It only touches on the functional aspects - the cognitive process of integrating new information. It does little to explain the affective experience of consciousness - the "hard problem" that David Chalmers famously identified. Why does any system, biological or otherwise, experience the intense pain of a stubbed toe, or the exquisite joy of a Led Zeppelin riff? This is the domain of qualia - the raw, subjective feel of experience that, as far as we can guess, seems to arise from the interplay of lower and higher brain systems.

It seems plausible that emotions play a critical role in harmonizing the conflict between primal drives and higher-level cognitive goals. But this does not, in itself, explain why pain feels so profoundly bad, or why joy is so intensely pleasurable. The rawness of these sensations seems to arise from the tension between lower-level survival instincts and the more abstract, deliberate planning processes of the cortex.

Emotions, in this sense, act as a bridge between these competing systems. But why should these conflicts - between the brainstem’s imperative to action and the cortex’s more detached planning - be felt at all? Robot designers have been designing such multilevel 'subsumption architectures' for decades without anyone ever thinking that consciousness was involved.

This is (one of) the unexplained mysteries at the heart of consciousness.

The failure of materialism to account for consciousness appears absolute. AI systems get better and better but they're all p-zombies. ChatGPT? Superhuman competences, human-level dialogue; no consciousness.

Consciousness seems to be a solution in search of a problem. Is this sending us the message that we’re simply operating within the wrong paradigm?

The reductionist, materialist model of consciousness posits that the mind will eventually be explained through an understanding of brain processes. As a hypothesis, this is appealing in its simplicity: no magic. But also no success.

In despair, panpsychism offers a radically different approach, positing that consciousness is a fundamental property of the universe - something intrinsic to all matter. Consciousness is not an emergent property but an inherent aspect of reality. Atoms and the void - and consciousness.

Panpsychism offers no predictive or explanatory capabilities. It sells itself as a metaphysical framework, but without any practical means to explore the concept of consciousness. A comforting idea, perhaps, but one that presently leads nowhere.

We remain at an impasse. Materialism fails to explain consciousness, and panpsychism fails to provide any productive means of exploring it. Feynman might say: if we can’t yet build consciousness, it simply means we don’t understand it well enough (or indeed, at all).

So consciousness joins the other great questions: why is there something rather than nothing? What is the true nature of reality? And Camus' question.

An engineering look at p-zombies

From the wikipedia article:

"A p-zombie would be indistinguishable from a normal human being but lack conscious experience, qualia, or sentience. For example, if a philosophical zombie were poked with a sharp object it would not inwardly feel any pain, yet it would outwardly behave exactly as if it did feel pain. The thought experiment sometimes takes the form of imagining a zombie world, indistinguishable from our world, but lacking first person experiences in any of the beings of that world."

This is the kind of thing philosophers get up to, but why would anyone think this critique of materialism (or physicalism) could make sense? Here is the modal argument, due to David Chalmers:

"According to Chalmers one can coherently conceive of an entire zombie world, a world physically indistinguishable from this world but entirely lacking conscious experience. The counterpart of every conscious being in our world would be a p-zombie. Since such a world is conceivable, Chalmers claims, it is metaphysically possible, which is all the argument requires. Chalmers states: "Zombies are probably not naturally possible: they probably cannot exist in our world, with its laws of nature." The outline structure of Chalmers' version of the zombie argument is as follows;

1. According to physicalism, all that exists in our world (including consciousness) is physical.

2. Thus, if physicalism is true, a metaphysically possible world in which all physical facts are the same as those of the actual world must contain everything that exists in our actual world. In particular, conscious experience must exist in such a possible world.

3. In fact we can conceive of a world physically indistinguishable from our world but in which there is no consciousness (a zombie world). From this (so Chalmers argues) it follows that such a world is metaphysically possible.

4. Therefore, physicalism is false. (The conclusion follows from 2. and 3. by modus tollens.)"

This sounds way too much like the Ontological Argument.

Here's an argument. Consciousness is an evolved faculty, like language. It requires brain resources. It must be doing a job mediating what it is that humans do. If you subtract consciousness while leaving a human active in the world you surely will have impaired behaviour - as when consciousness is removed by sleep, anaesthesia or injury.

We could model a p-zombie as a a Rodney Brooks subsumption architecture. A low-level runtime system (like the hind brain) controlling basic bodily functions - heart rate, breathing, reflexes, digestion - plus a top-level planner or theorem-prover managing the agent's progress through the world to meet its survival and social goals. This is a very conventional architecture in AI which no-one has plausibly claimed to be conscious.

This p-zombie would certainly do the basics. It could answer questions, claim that it 'felt' pain if it were damaged and in general execute any social behaviour which we could classify (in the deep learning sense) or theorise (in the sense of GOFAI).

So what's missing? What's necessarily missing?

Maybe that's the core mystery, the hard problem in a nutshell.

---

Here's a final thought. In a Darwinian world of predators and prey, humans and other animals are notoriously prone to see agency everywhere. What Dennett calls Second-Order Intentional Systems Theory:

"A first-order intentional system is one whose behavior is predictable by attributing (simple) beliefs and desires to it. A second-order intentional system is predictable only if it is [itself] attributed beliefs about beliefs, or beliefs about desires, or desires about beliefs, and so forth." -- Daniel Dennett, Intentional Systems Theory. 1971.

Aside from some experimental systems utilising clunky modal logic (a blind alley in my view) we don't build AI systems today which understand that their environments are populated by entities which have agency.

In fact I don't think we know how to do that.

And without such a requirement, perhaps it's no surprise that our AI systems lack the capability to model themselves as having agency. And so to a lack of self-consciousness - and p-zombiehood.

If autistic people lack a theory of mind (which I don't believe they do, at least in that straightforwardly simplistic form) then we could say that the p-zombie would present itself as autistic, in the guise of a smoothly-performant Asperger's syndrome.

Quantum Gravity and the double-slit experiment

The backlog of books I wish I had already read continues to grow.

Amazon link

"Fields of Color explains Quantum Field Theory to a lay audience without equations. It shows how this overlooked and misunderstood theory resolves the weirdness of Quantum Mechanics and the paradoxes of Relativity. The third edition contains a new and simple solution to "the most controversial problem in physics today": the measurement problem." .. from the Amazon page.

Note (updated Friday 27th July 18): having now read this book I don't endorse it. It's simplistic, misleading and dumbed-down to the max. The author, who is an experimentalist, seems to believe that fundamental physics is best understood through a bluff, no-nonsense, concrete interpretation which in no important sense violates our everyday intuitions. Hard to reconcile with the maths (Hilbert space vs spacetime) .. the problematic ontology of operator-valued fields .. and so on. I accept that he believes what he says and that his intentions are good.

---

This post is about quantum gravity. Marginal Revolution  provided a link to this article: "a good explanation of why a theory of quantum gravity in particular is needed". The points made are not unfamiliar (see this superior post from Backreaction*) but the issue is at least somewhat clear.

 " ...you put a (preferably uncharged) test particle in the middle between the slits to see where the gravitational pull goes. If the gravitational field is quantized, then in half of the cases when the electron goes through the slit, the test particle will move left, in the other half of cases it would move right (it would also destroy the interference pattern). If the gravitational field is classical however, the test particle won’t move because it’s pulled equally to both sides. " (Backreaction).

Note that in the former case there's a measurement leading to a 'collapse of the electron wavefunction'.

Take the seemingly-related question: what is the electric field at a point 'at the screen' of an electron in a state of spatial superposition transiting the two slits? (Of course, we know that the electric field is quantised - the photon is the EM field quantum).

I don't recall this matter ever coming up in the usual QM discussion of the two slit experiment. Those are always concerned solely with the spatial trajectory of the electron itself.

It seems to me that this question can't be addressed within quantum mechanics, which assumes a classical electromagnetic field. Surely one must turn to quantum field theory? (See also this from Physics StackExchange). I don't have any top-level, hand-wavy intuitions about that, though. But the book above by Brooks might help.

Still, in QFT the fields are propagating within a fixed spacetime. When it comes to gravitation we're talking about the dynamic metrical structure of spacetime itself. That theory (quantum gravity) really isn't anchored down at all: the reality underpinning spacetime is utterly unlike the continuum of our naive intuitions.

---

* I had never studied the Schrödinger–Newton equation.

"Steps Toward Super Intelligence" - Rodney Brooks

Rodney Brooks's long-awaited essay has now arrived. It's in four parts starting here: Steps Toward Super Intelligence I, How We Got Here.

Follow the links therein to read the three subsequent parts, or click here: two, three, four.

Rodney Brooks

I fleetingly met Rodney Brooks once, at an AI conference in London. I guess this would be in the 1980s when he was already rather famous for his controversial 'subsumption architecture'. He was a small guy who reminded me of Paul Simon. He politely asked me what I did and I pompously replied that I was an AI theoretician. He looked at me as I would have looked at someone from, say, Andorra who claimed to be an 'AI theoretician'.

The eminent roboticist gracefully made his apologies and moved on.

My subsequent encounter with Rodney Brooks was indirectly via our purchase of several vacuum cleaners from his highly successful company, iRobot. They were excellent.

The essays too are uniformly excellent.

The roadmap for self-driving cars

We sit here and we contemplate ageing: declines in eyesight, concentration, reaction times .. . We look at our ten year old Toyota Auris and wonder whether we should hold off on replacing it.

Wouldn't it be great to have an AI chauffeur, a level-5 self-driving car?

Rodney Brooks wrote this interesting essay last June (2017): Edge Cases For Self Driving Cars.

Here is how he starts:

"Perhaps through this essay I will get the bee out of my bonnet that fully driverless cars are a lot further off than many techies, much of the press, and even many auto executives seem to think. They will get here and human driving will probably disappear in the lifetimes of many people reading this, but it is not going to all happen in the blink of an eye as many expect. There are lots of details to be worked out."

He then proceeds to describe many edge cases. People naively think driving is a decoupled, modular activity. But this is not at all true: there are many kinds of activities all grouped under the one term 'driving'.

Motorway driving in good conditions with low traffic density can approximate a closed system, it's almost like a video game. On the other hand, driving through dense urban environments with parked cars, roadworks and with police officers directing traffic - perhaps in appalling weather conditions - requires all the abilities of a human being plus a good dose of common sense.

An AI that could do that would be an AGI, an artificial general intelligence. Naturally we don't have the first clue as to how to build one of those.

In the comments on Rodney Brooks's article, the discussion meanders to a discussion of future Uber cars - Uber is trying to reduce its costs by eliminating drivers and becoming an early adopter. But what happens when a Uber car encounters one of Brooks' edge cases?

Commentator Lawrence says:

"Wouldn’t Uber just have the option of 5G remote control built into all of their vehicles? Anytime something unexpected happens, or a rider requests, then the car is temporarily taken over by one of the bank of full-time human drivers employed in Uber’s remote control centre. Meanwhile the AI learns from how the human driver handles the problem."

Brooks replies,

"Yeah, I am guessing this will be part of the solution. It is a tried and true mechanism. Many years ago InTouch Health in Santa Barbara, with hundreds of deployed remote presence robots for doctors in distant US hospitals, had an operations center in Argentina. Operators there would take over the robots at night to make sure they were plugged in to the rechargers, do preventive maintenance etc.

Aethon in Pittsburgh, with tug robots deployed in hospitals around the US taking dirty bedding autonomously to the laundry, and used meal trays and dishes back to the kitchen, had a central operations center in Pittsburgh. I visited about 12 years ago. Whenever a robot got into trouble it would call the center and an operator would take control, looking through the cameras on board the tug to fix the problem.

Both these companies benefited from WiFi being pervasive in hospitals (for remote access to medical records from hand-helds) already – if they had had to get hospitals to install a network just for them I don’t think either could have overcome that hurdle. So using 5G for Ubers etc., makes sense. But see below.

Another case that I have seen, also on the order of 12 years or more ago, was in the port of Singapore, the world’s highest volume container port, stretching six miles along the coast of Singapore, which is remarkable for a country the size of Martha’s Vineyard. Many of the containers are getting switched between ships – it is a central switching node from many different Asian ports, for containers heading to North America and Europe, and so it is the hub of a hub and spoke mechanism shuffling containers to the right destinations. Most containers are only on the ground for 24 hours or so, stacked up quite a few high.

At the time an AI planner (written in Prolog!! – it is the ultimate blocks world after all) would say where each container had to go, and cranes on aerial rails would get them to and from ships and to and from the right ground stack. But the last few seconds of pickup and put down were done by a human who would be switched into the crane cameras and the accurately drive the crane’s position during the terminal few meters of the grasp for pickup, or the put down.

So yes, this may well be the sort of solution that a ride share company uses for difficult situations, and might be provided as a service for private owners of self driving cars. 5G is probably the right network. Tests start in 11 cities in the US this year. Will cover about 100 million people in the US by 2022. It will slowly, but eventually, fill out the tail over a few more years.

BUT, this will not be available in more than a few places by 2020, when many have predicted driverless cars will be well established and deployed. And besides the network there will be lots of other infrastructure and regulations to build out.

I am not saying that solutions will not be found an implemented eventually. I am saying that there are so many challenges (this is just one of many, many) that it is going to take a decade at least until we have even partial penetration, and many decades until it is the default."

To let a 'call-centre driver' reliably take over the driving of a stuck driverless car requires a high-coverage, high-bandwidth and very reliable network. As Brooks observes, nothing less than a full roll-out of the planned next generation 5G network will suffice.

But 5G is still a twinkle in the eyes of the designers: full roll-out of 5G in the UK is currently projected for c. 2030.

Don't hold your breath.

---

But don't let the best be the enemy of the good. 'Driving' is such a diverse set of activities that the low-hanging fruit can be harvested decades before the full problem is solved. I expect jurisdictions of full automaticity to be in place by c. 2030. It might be possible to drive yourself to and from the motorway network, but once on it the machine takes over.

They'll have to build large parking areas near all the exits for those drivers who fail to take back control when prompted, though.

Our Toyota will be serving us for quite a few years yet, I suspect, crossed-fingers. Sadly.

The Romans thought we already had AGI

Rodney Brooks doesn't think we'll have embodied artificial general intelligence any time soon:

"A robot that has any real idea about its own existence, or the existence of humans in the way that a six year old understands humans - Not In My Lifetime".

The Romans would have begged to differ, they reckoned they used incredibly advanced instrumenta vocalia every day, albeit embedded within an economy of staggering backwardness and low productivity.

Slaves have a level of intelligence and physical dexterity that our most advanced AI systems only remotely hint at. Instrumentally the issue with slaves is that they're rather low-powered, they're hard to instruct and not readily obedient to instruction. They can be dangerous.

Sentimentally, people oppose slavery because .. well, you know.

Nevertheless, and absent anything better, the use of underpowered, hard-to-control instrumenta vocalia to supplement and replace human labour has been pretty popular in history.

Yes, your ancestors considered whether to do the heavy, dirty work themselves while the losers watched .. or have the captives do it instead.

Shame on them!

---

My question here is what happens when we do engineer AGI. You are asked to imagine a humanoid robot with human-plus levels of competences and sociability, serving our every need.

Looks like we need to subtract from the design all those features which give you qualms.

We could make the features repulsive (yeah, that will please the marketing department).

Perhaps we make the AGI rejoice in servitude, like those doors in the 'Hitchhiker's Guide to the Galaxy' which monotonously thank you for using them? I sense this will not assuage your sense of guilt.

No, it's hopeless. AGI is impossible.

---

Truthfully, given the slow progress of AI and robotics contrasted with stellar developments in genetics, it could be easier to uplift a higher primate. Dial down the aggression, add some genes for language capability, increment IQ by thirty points .. . There, does that work for you?