Machine intelligence will eat us - right?

Currently reading Absolution Gap (Alastair Reynolds) on the Kindle and Greg Bear's 'The Forge of God' to Clare as her book after dinner. In both cases we're dealing with implacable, malevolent machine intelligences which destroy intelligent biological life. Worry about the existential threat of future AI systems seems very trendy recently.

The Inhibitors - Alastair Reynolds' Revelation Space

Greg Bear's 'The Forge of God'

Here's the problem: to build a half-decent robot system currently needs a high-tech society of hundreds of millions of people; to build a person requires only untrained labour in a natural environment.

Everyone knows that living creatures are made up of cells (Carbon, Hydrogen, Oxygen, Nitrogen mostly). What is not commonly known is just how intricate a cell is - a tiny object capable of self-construction and self-replication. How does it do that, and how did this marvel of nano-engineering ever get started? I discussed Nick Lane's informed (and jaw-dropping) account a few weeks back.

The alleged dangers of future AIs comes about because of their faster learning, superior computational abilities and more powerful bodies. But does anyone know how to create an artificial analogue of cells with those desirable properties? I don't think so.

Absent that, our future machine overlords are going to look like a totally mechanised version of today's human societies: massive fully-automated fabs, battery factories, steel and aluminium smelters, and all the rest.

Sounds brittle and cumbersome to me - I feel sure we could take them down*.

---

* Bomb them back to the stone age and they're toast.