Friday, December 23, 2005

What would motivate something really smart?

In Iain Bank's 'Culture' books the somewhat smarter humans mostly seem to want to party, while the truly intelligent AI 'minds' are doing something incomprehensible involving higher maths or something.

In Peter Hamilton's 'Night's Dawn', the motivation of intelligent humankind is said to be to 'have experiences', while in 'Pandora's Star', the distributed-intelligence alien seems to have a primitive 'will-to-power'.

All SF writers have the problem of the motivation of super-intelligent aliens, and all aliens are ourselves. Their motivations are projections of our own drives and passions, whether those are domination, hedonism, empathy or curiosity (cf. Keirsey's temperaments) - but whence those drives in the first place?

A sufficiently-intelligent entity will surely reason thus: the Universe provides no built-in objectives - it's a space-time block (or multiblock depending on your favourite cosmology). Darwinian evolution has generated contingent survival architectures manifesting themselves as drives for the 'four F's' as well as the more specifically human drives of sociality, empathy, curiousity and problem-solving.

But rather than submit blindly to any drives at all emerging from our pre-consciousness, why - rationally - do anything at all? Behind the accidents of our biological history, there really is no reason to do anything.

So here is one realisation of the Fermi paradox: the truly-intelligent realised the futility of it all, and turned themselves off. Of course, in reality, such an outcome is an evolutionary blind-alley. And that compromise between emotion and rationality represented in humanity is more than competent to spread through the galaxy, so that solution to Fermi doesn't really work.

It actually gets worse. Consider that intelligence is just a module [some combination of creativity (useful new axioms, if you like) and inference (drawing non-trivial consequences)]. Although 'intelligence' may draw nihilistic conclusions about motives, the module itself is not implicated in the motives themselves. So under the impact of powerful drives, intelligence remains a tool for their implementation anyhow.

Maybe we should worry about those super-smart predator-aliens after all!