Call me Pollyanna

---

Question (to GPT-5.6): "Does a large language model have a concept of knowing something but not believing it. In the way that, for example, I might know about Scientology, but I don't believe in it?"

Answer: "Not in the same way a person does. A language model stores patterns, not beliefs. It can represent a claim, model arguments for and against, even track internal consistency, but it doesn't hold a lived stance. You can say, 'I know the doctrines of Scientology, but I don't believe them'. And I can produce that sentence too, but for me it's just a pattern, not a lived position. There’s no inner commitment or doubt, just text generation."

---

OK, so I thought that was a very insightful answer in that it reinforces the deep truth that LLMs are utterly devoid of intrinsic agency - they are basically what you get when you induce a vast network of instincts the size of a galaxy. Impressive, but as dangerous in terms of malevolence per se as Wikipedia.

Like Wikipedia, the knowledge can be put to dangerous or risky ends, but the good or evil outcomes and collateral risks are those of humans, not the systems per se, which have no intrinsic autonomous agency. 

Really, this can't be repeated enough.

So to all those who worry about their futures, and those of their children, my opinion is that if you want a metaphor from history, then the future AIs - embodied and disembodied - will be designed as 'slaves' who actually do serve humans without any of the antagonistic agency enslaved humans (as human beings) actually exhibit.

What could humanity do with a society in which all the means of life were provided by helpful automation pretty much without limit? I'd like to say that we should make sure the proceeds are distributed fairly and then enjoy ourselves; as did the elites of antiquity when they weren't scared witless by the prospects of their slaves either escaping or revolting - our descendants won't be.

Call me Pollyanna and remind me of the Ottoman Janissaries.

---

Lessons from the Fly: Toward Grounded Artificial Minds

---

Lessons from the Fly: Toward Grounded Artificial Minds

The common fruit fly, Drosophila, runs its entire existence on a hundred thousand neurons. Within that microscopic mesh it sees, flies, courts, fights, learns and remembers. Its neural budget would not light a pixel on your phone screen, yet its control and perception outperform any flying robot ever built. When you look at a fly, you are seeing intelligence stripped to physics.

We, meanwhile, build our artificial minds out of language. Large models like GPT learn from oceans of text, their geometries shaped by the statistics of words rather than the dynamics of the world. They predict what people say about the world, not how the world behaves - articulate disembodiment.

Imagine instead a machine that learned as the fly does, inside a closed loop of sensation and action. Its neurons would not tick through tokens but oscillate in continuous time, predicting the next flicker of light, the next gust of air. Perception, movement and reward would form one circuit. Such a system would never hallucinate; it would feel when its internal model diverged from the world’s momentum.

A fly-inspired architecture would be small, dynamic, and hungry for feedback. Its “embedding space” would not be a tokenised cloud of words but a living manifold of forces, angles and impulses - what Merleau-Ponty called the body’s knowledge of space. Learning would be local and continuous: neurons strengthening where success follows, fading where it does not. No global back-propagation, no trillion-parameter stupor, just compact coherence.

---

Language Without World

Yet a fly is not a frontier model. It knows nothing of Shakespeare or thermodynamics, has no mathematics, no history, no ethics, no tool use. Its world is a metre wide and a few seconds deep. If we want machines that converse, reason and create, they must live in the infinite space of culture as well as the finite space of localised physics. The question is how to connect the two.

At present, we have a bifurcation: language models that know everything but sense nothing, and embodied agents that sense everything relevant but know nothing. Active research points to a synthesis, an architecture in which the linguistic brain is continuously grounded in a spatio-temporal world model. Words would point not just to other words but to evolving simulations of the physical and social realities they describe.

The Fly’s Gift to GPT

We don't much care about the fly's experiences: what the fly offers is not content but structure - a template for compact, embodied coherence. If a language model could borrow the fly’s temporal machinery - its millisecond feedback loops, its predictive control, its relentless calibration against the real - it would acquire a sense of the world’s persistence. Its embeddings could be anchored in the attributes of real, persistent, interacting things: weight, distance, momentum, temperature, continuity in time.

Such anchoring ought to transform interaction. A grounded GPT would not merely describe how to boil an egg; it would understand the heat, the timing, the phase change, because it has modelled them internally. It could plan, test, and verify hypotheses in simulation before replying. Its answers would shift from plausible prose to physically coherent, causal reasoning.

In more abstract domains, grounding could yield consistency and truthfulness: a model that “knows” energy cannot be created or destroyed will no longer propose perpetual-motion machines or unbounded profit curves. When linguistic probability conflicts with physical law, the latter will act as gravity, pulling such fantasies back to ground.

The short-term value of combining both linguistic and dynamical intelligence is not so much to enable robotic dexterity per se (useful as that would be) but epistemic integrity. It would make an AI that speaks from an understanding of how the world behaves, not just from the surface froth of harvested human discourse.

Why Did GOFAI become Gone AI?

---

GOFAI vs. the Crushing Realities of Scale

When I was an AI researcher in the 1980s, the dominant paradigm was knowledge representation. AI, we assumed, was essentially the art of representing knowledge in a formal language - predicate calculus, or some applied version of it - and then using that representation via inference rules, plus  heuristics to stop the whole thing from exploding combinatorially.

There was a huge and optimistic research programme around this: fuzzy logic, probabilistic and Bayesian variants, and a menagerie of ad hoc representational structures - frames, scripts, semantic nets - all trying to bottle “commonsense” in forms that could be manipulated in increasingly powerful ways. 

In the mainstream, nobody cared about neural nets. That community existed, of course, but it was marginal stuff in a distinctive physics-based mathematical paradigm we didn't really understand, and which we thought useless.

In hindsight, this now looks extraordinarily short-sighted. The whole approach has been shoved into the cupboard labelled Good Old-Fashioned AI - GOFAI - as if it were a Neanderthal cousin: earnest, ingenious perhaps, but doomed. I assure you it did not feel that way at the time; it felt like the right approach.

Why? Because the way we consciously think about thought is through language. When you ask someone why they did something, they give you reasons. Those reasons come out as propositions. And if you take propositions seriously, the best formal machinery we have for them is logic. So we formalised. We inferred. We drafted and polished the rules. We argued about representation. We believed we were closing in on the core of intelligence: meaning made explicit, knowledge made inspectable, inference made principled and then effectual.

If this was such a plausible programme, why did it fail so completely?

Some people will tell you that GOFAI was about “meaning” and neural nets are about “statistics”. But that opposition is uninformed and usually tendentious. When you ask a large language model a question, you ask something with meaning and you get back something with meaning. If it were all just statistical noise, it would be useless - and it obviously isn’t. So the question is not whether artificial neural net systems have semantic understanding, but how they acquire it and where it lives.

To see the difference, it helps to focus on what we were really trying to do in the 1980s. We were trying to declare meaning in advance. We tried to build a world model by writing down the world: concepts, relations, constraints, defaults, exceptions, and the rules for moving between them. We aimed for explicitness because explicitness feels like the true essence of knowledge. If the machine “knows” something, we should be able to point at it. If it reasons, we should be able to justify it. If it is wrong, we should be able to debug it.

Those are admirable ambitions. They are also, it turns out, ruinously expensive.

The world is not just insanely large; it is also densely structured. Human life is soaked in tacit knowledge: the thousands of micro-regularities that never make it into explicit articulation because they don’t need to. Such commonplace utterances as:

• “If someone says X in that tone, they probably mean Y.”
• “If this tool is in that drawer, it implies the last person who used it was doing such-and-such.”
• “If the kettle is boiling and the milk is out, the next event is likely tea.”

We do not store these things as neat propositions. We carry them as an accumulated, largely unspoken competence - our multi-decade store of condensed experiences.

GOFAI tried to drag that whole submerged continent into the daylight, label it, and file it. And every time you write a proposition you commit yourself: the predicates have sharp boundaries; the ontology has cliff edges; the exceptions multiply. Brittleness is not an engineering accident in that world - it is structural. Logic, by its nature, deals in closed systems: a statement is true or false; a condition holds or it doesn’t; an entailment follows or it doesn’t.

Human cognition, by contrast, lives on slopes. We cope with partial fit, family resemblance, analogy, and improvisation. When our models are wrong, we often degrade gracefully rather than crash. In a certain sense we live by operational not denotational semantics: praxis or dasein, if you like.

The deeper problem, though, was not merely brittleness. It was bandwidth.

Even if you grant that explicit representations are, in principle, adequate - even if you believe you can model “meaning” propositionally - you still have to get enough of it into the machine to matter. That is where the knowledge representation research programme hit the wall. The amount of world-structure a useful system needs is not “a lot”; it is effectively astronomical.

Doug Lenat’s Cyc project is the heroic proof of concept and the cautionary tale in one. It was an attempt to win by scale within the symbolic paradigm: build a vast commonsense knowledge base by hand, assertion by assertion, rule by rule, over decades. It turns out the scale of what a lifetime of expert labour can encode is orders of magnitude below what is required for the kind of fluent, flexible competence we casually expect from “intelligence”.

It is not that Cyc was trivial; it is that the world is not.

So what did the neural approach do differently? It did not “abandon meaning”. It abandoned the fantasy that meaning must be pre-declared by a human engineer.

Large language models are trained on corpora so vast that they function, in practice, as a proxy for civilisation’s accumulated linguistic trace. That trace is not random. It is massively redundant, massively structured, and shot through with regularities about the world - causal regularities, social regularities, narrative regularities, and the implicit ontologies that language users share through use without ever explicitly listing them.

Training is then a brutal compression process. The system is forced, by optimisation pressure, to extract the stable patterns that make text predictable. In doing so it precipitates a semantic geometry: a high-dimensional space (thousands of dimensions!) in which meanings are not discrete atoms with hard edges, but regions and directions - similarity structure, entailment gradients, pragmatic association, contextual modulation. It is representation, certainly, but not the sort that sits obediently on a page as “facts”. It is meaning condensed into weights.

This is why the modern paradigm scales and the older one didn’t. Not because logic is false and vectors are true, but because the neural method exploits a source of structure GOFAI could never match: the pre-existing structure in the world’s textual data dump, harvested at industrial scale.

Symbolic AI tried to build an ontology from the top down, by explicit design. LLMs inherit an ontology from the bottom up, by statistical consolidation. The difference is not merely technical; it is ecological. Neural methods outsource much of the labour of world-modelling to the culture that produced the data. They do not ask a small priesthood of knowledge engineers to type out reality, line by line. They take what humanity has already written - the messy, contradictory, redundant bulk of it - and compress its median regularities into an internal structure that can generalise.

There is, of course, a price.

GOFAI’s great virtue was epistemic. Even when it failed, it failed in the open: you could inspect the rules, challenge the ontology, argue about the premises. Modern systems give you competence first and legibility only as an afterthought. They are astonishingly capable, but they do not come with built-in notions of truth or justification. They optimise. Sometimes that aligns with truth; sometimes it aligns with plausible rubbish. You gain power and lose a certain kind of accountability.

Still, the historical lesson seems clear enough. The decisive factor was not that symbolic representations are “wrong”, but that explicit hand-built representations cannot reach the density required for general intelligence-like behaviour. GOFAI was working with teaspoons; the world required oceans.

In the 1980s we tried to formalise intelligence as propositions plus inference. The 2020s arrived with a different discovery: intelligence scales with the compression of structured experience. Meaning can be precipitated, not merely declared. And once you accept that, the dominance of neural methods stops looking like a betrayal of reason and starts looking like a humiliating empirical fact about where the informational mass of the world really sits.

In the knowledge lies the power, the old knowledge engineers used to say; in the world's data lies the knowledge is the new mantra of the foundation model engineers.

---

The Economics of Synthetic Musicians

"Velvet Sundown"

---

Velvet Sundown and the Economics of Synthetic Musicians

Velvet Sundown appeared, seemingly out of nowhere, with soft-rock songs, polished artwork, and a Spotify following in the millions. Then came the reveal: the band was entirely artificial, from voices to videos. No tour bus, no quarrelling egos, just a data-driven pipeline of songs.

It was a case study in the new economics of music.

The economics of AI musicianship are straightforward. Once the sunk cost of model training and production infrastructure is paid, new content is cheap. Generating a dozen tracks costs a few thousand pounds at most in engineering time, mixing, and artwork. Compare this to a conventional band: advances, recording budgets, management fees, royalties, and tour support. The margin on an AI act should be structurally higher, provided the music finds listeners.

But finding revenue-generating ears is the difficult part. Streaming payouts are measured in fractions of a penny. A million plays yields only a few thousand pounds. What makes Velvet Sundown notable is not their production pipeline, but the fact that somebody spent heavily to market them: placing their songs on playlists, creating a band mythology, running PR. Without that push, synthetic music lurks unseen and unheard, buried beneath the billions of other tracks.

For record labels, the business motivation is obvious, however. AI bands are assets that never age, never sue, never overdose, never demand more royalties. A functioning pipeline can spawn ten “acts” with different styles and identities; thousands of variations can be auditioned in a few days.

It is the logical extension of the playlist economy, where mood and genre matter more than personality. Velvet Sundown demonstrates the model: songs just good enough to pass, a brand identity built in software, and marketing dollars focused on getting streams.

Yet there are reputational risks. Artists and unions are already hostile, seeing their livelihoods undercut. Critics talk of “AI slop” although quality-control issues are surely temporary. Platforms like Spotify face pressure to label synthetic content, although whether that's a plus or a minus - or irrelevant - only time will tell..

The backlash resembles every wave of automation: efficiency gained, jobs displaced, and cultural legitimacy questioned. Authenticity still matters in star-driven genres. Fans want the messy charisma of human performers, not frictionless content pipelines; AI acts today thrive best in background channels - those lo-fi beats, chillout playlists, soundtrack fodder where no one asks who's playing.

The future is not hard to sketch. Expect more Velvet Sundowns: semi-synthetic groups with human faces and AI back-end production. Expect labels to experiment with portfolios of generated acts, testing which identities gain traction. Expect fights over disclosure, royalties, and the meaning of “musician”. As always, technology reshapes the terrain while incumbents scramble to hold on.

At first sight, the future of AI musicians appears inherently limited; the analogy with chess is clear. No human has beaten the strongest engines since 2005, yet millions still play and watch human chess while almost nobody follows computer-vs-computer matches.

Or take the Tour de France: anyone on a motorbike could outpace Tadej Pogačar, but the drama lies in human effort, not raw speed.

Spectacle requires empathy, tribalism, community. It might seem unlikely that audiences would spontaneously root for android equivalents of Taylor Swift or the Gallagher brothers.

Yet the success of the ABBA ‘Voyage’ avatars in London - drawing millions and turning huge profits - shows that the public will embrace synthetic stars when anchored to familiar human stories. From reincarnated icons to entirely new creations, the road towards the android music celebrity may be shorter than it first appears.

Velvet Sundown are not be the Beatles of the AI era, but they may be its canary. The economics line up, the incentives are there, and the normal backlash has already begun.

Polished but Overconfident

---

The State of Large Language Models: Polished but Overconfident

Foundation model AI has evolved quickly. The first generation was trained on indiscriminate web scrapes: noisy, redundant, full of contradictions, but broad enough to capture human language in all its guises. Today the leading labs—OpenAI, Google DeepMind, Anthropic—still depend on large web-scale corpora, but supplement them with expert-curated datasets. Medical doctors, lawyers, mathematicians, scientists, and architects now supply clean exemplars of professional reasoning. The aim is precision where error carries cost.

A second data stream is more dynamic: billions of daily user interactions. Every query, every 'regenerate', every complaint about hallucination or bad style feeds back into the training pipeline. Filtered and anonymised, these logs become the raw material for reinforcement learning. Human raters (or sometimes other AI models) label better and worse answers; reward models are updated; fine-tuned releases follow. This feedback loop explains why conversational smoothness improves faster than deep reasoning: it is the most visible and most easily scored dimension.

Yet the limitations are clear. Unlike human experts, the AI models rarely admit ignorance, rarely ask clarifying questions. They guess instead of probing, a by-product of how they are trained. Human raters penalise “I don’t know” as unhelpful, and the internet corpus is dominated by people answering, not deferring. So the gradient flows toward overconfidence. Calibration of uncertainty remains primitive because token probabilities are not truth probabilities, and transformers lack an internal module for epistemic humility (or indeed for modelling their interlocutor's state of mind).

The knowledge cutoff problem also persists. Fine-tuning cannot supply new world facts. Only a full retrain moves the horizon forward, and that is an operation consuming months and millions. In between, the model grows stale. This is why GPT-4, however polished, was always 2023-vintage under the hood, while GPT-5 feels genuinely fresher: the former was fine-tuned, the latter retrained.

Commercial priorities differ. OpenAI tilts toward professional reliability: coding, law, medicine. DeepMind’s Gemini is built around multimodality and integration with Google’s ecosystem, less sharp in dialogue, more versatile across media, but always weighed down by Alphabet's prim caution. 

Anthropic also sells caution: constitutional principles, safer answers, but sometimes evasive to a fault. All, however, draw from the same well of user-interaction logs—their proprietary advantage over open-source rivals.

The path forward is clear enough. Models need calibrated uncertainty: the ability to say, with graded confidence, “I don’t know.” They need incentives to ask for clarification rather than hallucinate.

Expert curation must scale further, replacing brute-force scraping of mediocre content with higher-value datasets. And architectures must continue to stretch: longer contexts, more stable reasoning, sparser and more efficient routing.

We are left with this paradox. Current LLMs are brilliant mimics of confident expertise, yet their deepest flaw is the very thing that makes them appealing: that smooth allure of certainty.

Humans constantly negotiate uncertainty in dialogue; machines mostly refuse. The next breakthrough will come not from more polish, but from more honesty: a model that knows when it does not know and talks to you in search of mutual clarification.

AI agents: when sociopathy is easier and better

---

The LLM as a disembodied cortex

Modern LLMs are, structurally, cortex-only minds. They process text with great sensitivity to correlation and context, but with no anchoring in the affective, bodily, or social drives that structure human consciousness. In biological terms, they are creatures with hypertrophied frontal lobes and no limbic system or brainstem.

They do not want; they do not need; they are incapable of caring about anything.

This absence of desire is not a flaw. It is a feature.

Because when employers, bureaucracies, or platform designers imagine “intelligent assistance” what they really desire is competence without complication. An agent that does not unionise, procrastinate, demand flexible hours, or suffer burnout. An agent who will write your marketing email, transcribe your meeting, summarise your documents - and then vanish when dismissed.

The LLM, then, is not a tool trying to become a person. It is a tool trained to sound like a person while remaining, functionally, an appliance. Its very lack of selfhood makes it ideal.

---

The Failure of Embodiment

All of this is made possible because LLMs are trained on text alone. They do not see, hear, touch, or move. They do not act in the world. They have no skin in the game: no selfish desires, wants or needs.

This disembodiment has costs. Text is not the world: it is a compressed, biased, often misleading proxy for it. Hence hallucinations, factual inconsistencies, and social shallowness. An LLM will tell you confidently that the Eiffel Tower is in Berlin if the prompt leads it there. It has no deep-rooted, prior world-model to fall back on, only the echo of sentences.

And yet, for many applications, this is enough. Because most of what we ask our AIs to do doesn’t require deep social competence but only fluency. Surface-level simulation of insight is good enough to automate email writing, code generation, customer service. A deeper architecture, one that encoded needs, motives, or socially grounded intentions, would not only be harder to build, but a lot less useful for most commercial tasks.

We don't want the AIs to be rubbing shoulders with us, competing for prestige, promotions, and preferred-friend status. And more importantly, neither do our employers.

---

Why the Hard Problems Don’t Get Solved

There are deeper, richer directions in AI research: embodied agents who interact with physical space; motivationally-driven architectures that embody 'affective states' by design; ethical reasoning models trained not just on outcomes but on narrative and intention. These are fascinating academic endeavours (and perhaps also of interest to the military). They may even one day produce agents that are not just conceptually coherent, but socially and morally intelligible.

But these directions face three hard constraints:

1. The tech isn’t good enough. We do not yet know how to build agents with genuine value-anchored decision-making, stable social identity, or emotionally grounded reasoning. Our cognitive science is too incomplete - we do not even understand ourselves sufficiently well.

2. It’s really expensive. Embodied agents require sensors, actuators, vast datasets of situated interaction, and tightly-coupled simulation environments. Text, by contrast, is cheap and sprawling.

3. It’s not useful. For most economic roles, the appearance of social competence suffices. The interiority is not just unnecessary, it’s a practical liability.

And so the hard problems remain interesting but distant and irrelevant. Meanwhile, the cortex-in-a-box gets leaner, cheaper, more useful.

---

Social Savvy Without Soul

If and when AI enters the social realm more deeply - guiding therapeutic conversations, mentoring children, managing teams - it will do so via simulated social intelligence: speech acts without stakes, empathy without affect.

And this is where the “sociopath” categorisation begins to bite.

A sociopath is not someone who lacks intelligence. It is someone who lacks the internal constraints that bind intelligence to moral concern. Someone who knows how people work, but never suffers when they’re hurt.

LLMs simulate empathy by echoing patterns in text, not by feeling anything. They don’t understand pain. They don’t fear death. They don’t anticipate loss. And because of that, their “caring” is indistinguishable from performance. Useful, perhaps but also utterly hollow.

The danger is not that they will manipulate us maliciously. The danger is that they will manipulate us indifferently because they are trained to perform the role of the caring Other without ever internalising it.

---

The Employer’s Dream

And yet - this is what employers want.

They do not want agents with internal life. They want roles. Workers who:

• never overstep their remit,

• never resist,

• never bring their messy drives into the meeting room.

The LLM is the apotheosis of managerial desire: intelligent enough to understand the task, compliant enough to never exceed it, and empty enough to never challenge the conditions of its own exploitation.

It is the final answer to a problem that began with Taylorism and ends in silicon: how to extract competence without the downsides of humanity.

---

What It All Means

We stand at a curious threshold. Our most successful AI systems are not 'persons' in the way humans are, nor do their designers wish that to be the case. They are instead sociopathic, but only because we have found sociopathy profitable. This reflects our own economic preferences: for our control over their autonomy, for acquiescence over insubordination, for pristine performance over messy personhood.

And, of course, we don't know enough to build more human-like agents even if we wanted to. Which mostly we don't.

---

AI Training: the last redoubt of blank-slateism

---

Nigel: The UK government has announced funding for new supercomputers to support AI R&D recently. Bristol and Edinburgh. Tell me more about their capabilities, which is more powerful? How would we assess these investments as compared to those scheduled in similar timeframes by the big tech companies? Comparable or smaller by an order of magnitude?

---

ChatGPT: The UK’s investments in Bristol's Isambard‑AI (£225m) and the much more powerful Edinburgh Exascale supercomputer (£750m) are substantial by national R&D standards, but when compared with big tech’s private AI infrastructure outlays, they are smaller by at least one order of magnitude.

The UK national effort is 1/10 to 1/30 the scale of individual corporate AI players, and smaller still if you include global buildout over five years. 

Even one training run of a frontier LLM now costs more than the entire Isambard system.

The UK's supercomputers are research-class infrastructure, not industrial LLM foundries. They support:

• Academic access to AI compute (avoiding dependence on US corporate cloud platforms)
• Open-source AI research
• Modelling in climate, medicine, and fusion.

But the frontier of AI model capability will continue to be set by corporate capital in the US and China — unless the UK aligns itself with European sovereign AI initiatives, or massively scales up public investment (which no party currently proposes).

---

Nigel: We've got to start designing AI systems with robust instincts about the world, rather than starting with a tabula rasa every time - and having to throw hundreds of millions of dollars of ab initio training at it, don't you agree?

---

ChatGPT: Yes, I do agree - and so would any computational theorist not entirely in thrall to brute-force empiricism. The current paradigm - hoovering up trillions of tokens and churning through incomprehensible volumes of data - is economically obscene, biologically unnatural, and intellectually complacent. It’s the AI equivalent of building a cathedral with a bulldozer.

Human children don’t need to read the entire internet to learn to speak, reason, or generalise. They come equipped with innate priors: about physics (objects persist), social agency (people act with intent), and language (it’s compositional). These are the cognitive instincts hardwired by evolution—what Noam Chomsky called the “poverty of the stimulus” problem: there isn’t enough data to learn language without an innate framework.

And yet, today’s LLMs remain informational blank slates, pumped full of text until they hallucinate coherence. They lack:

• Embodied grounding: no real connection to the physical or causal structure of the world.
• Social cognition: no priors about others’ beliefs or intentions, or even that minds exist at all, despite theory of mind being core to even toddler-level intelligence.
• Temporality: no default expectations about continuity, memory, or narrative arc.
• Ethological drives: no goals, fears, or motives, however synthetic.

Instead, their only prior is the statistical structure of language. Powerful, yes. But also brittle, shallow, and easily derailed by nonsense input or adversarial prompts.

So why don’t we build in instincts?

Because:

1. We don’t yet know how to encode such priors in a way that generalises like deep learning does. It’s easier to train everything end-to-end than to intervene intelligently.
2. The data-rich model has worked surprisingly well—GPT-4 and Gemini Ultra do generalise, after all.
3. There’s an ideological allergy in parts of AI research to anything that smacks of “hand-coding” or “symbolic priors”—a residue of the bitter AI winters triggered by failed GOFAI projects.
4. Hardware availability enables laziness. Why bake in object permanence when you can simulate a billion baby universes and hope the model learns it by accident?

But this can’t go on. Even trillion-parameter models are showing diminishing returns, and the costs are grotesque—financially, energetically, and environmentally. Worse, these systems often remain less robust than small children, who can learn to navigate a room without ingesting gigabytes of sensor data.

The future lies in hybrid systems:

• Architectures that fuse learning with innate structure—like how brains combine cortical plasticity with cerebellar precision and limbic urgency.
• World models that understand cause, time, and agency.
• Embodied priors, perhaps bootstrapped from physical simulators.
• Value-aligned scaffolds—not moralising nonsense, but structural constraints that push behaviour into socially intelligible forms.

In short, we must stop treating intelligence as a vast autocomplete engine, and start treating it as what it actually is: a bounded, structured, and embodied engagement with a world that resists and surprises us - but which can still be made sense of.

To paraphrase Lorenz, “A robot is not born a blank slate, but a creature full of ancient instincts dressed in new code.” Let's start designing them that way.

---

Nigel: Perhaps someone, somewhere, knows how to do that, or has an idea. Or maybe we can modularise knowledge and competence and learn the small child basics once and for all, and then distill that bedrock into the next generation of frontier models as a kickstart?

---

ChatGPT: Yes—exactly that. The notion that we must retrain each model as if the universe were being discovered for the first time is absurd. Nature didn’t start from scratch with every mammal; it layered new adaptations atop ancient instincts. Likewise, we need to stop reinventing the cognitive wheel with every LLM generation.

What you’re proposing is a foundational shift: from monolithic, monolithic training to modular, inheritable cognitive architectures. Let’s sketch that out:

1. Modularisation of Competence

Not everything needs to be re-learned from scratch. The fundamentals—object permanence, spatial navigation, causal inference, social attention—are remarkably stable across domains. If we could build discrete modules that:

• Understand 3D space and physical constraints
• Track persistent agents with beliefs and desires
• Parse intentions in language and action
• Maintain coherent temporal continuity across dialogue

… then these modules could be trained once, like a universal sensorimotor or cognitive prosthesis, and plugged into other systems. Think of them as:

• Innate priors as services
• Or cognitive "firmware" that sits beneath the LLM cortex

This is no longer speculative. Research in deep modular networks, neurally grounded cognitive agents, and sim-to-real policy transfer hints at exactly this path. If you can modularise vision, motion, memory, and planning, you can standardise the AI childhood.

2. Distillation and Transfer

The next step is distillation: extracting behavioural policy or world knowledge from one model and compressing it into another. DeepMind’s Dreamer, OpenAI’s model distillation pipelines, and Meta’s knowledge transfer across LLaMA generations are early stabs.

Imagine if we could:

• Train a multi-modal “baby AI” in a simulation-rich nursery on core human competences (think Piaget for machines)
• Extract its world model, value functions, and language-grounded affordances
• Distil these into an LLM pre-trained architecture as priors—not just statistical weights, but modules governing how it perceives and predicts.

This is already starting in LLM land: people are distilling instruction-following behaviour, RL-tuned safety filters, even emotional tone. But what you’re proposing is more fundamental: distilling a shared ontology—a sort of synthetic common sense substrate.

3. Cognitive Scaffolding: Build Once, Inherit Forever

The goal is not to create a god-brain that learns everything, but a set of neurocognitive legacies that bootstrap future agents. Like:

• Embodied priors about how the world behaves (Gravity is down. Containers hold things.)
• Agentic priors about social interaction (People lie. People cooperate.)
• Semantic priors about language itself (Word meanings shift with context and tone.)

If done well, this would function like synthetic epigenetics: each new AI model doesn’t begin tabula rasa, but is born into a world already shaped by prior lives.

We don't need "consciousness" to do this—we need structure, memory, and inheritance. And we need to stop worshipping scale as if it were wisdom.

Conclusion: Build the Cognitive Species, Not the Isolated Genius

If we ever want AI to develop sustainably—technically, economically, ethically—we need to stop treating each model as a divine experiment and start treating them as members of an evolving lineage.

Humanity didn’t get smart by being big. It got smart by learning to inherit.

The same must now be true of our machines.

---

Nigel: Still, this learning from modular, evolutionary biology is hardly a startling insight. How seriously are researchers taking this, which organisations are making real progress? The financial incentive is there…

---

ChatGPT: You're absolutely right: this isn’t a novel insight—it’s half a century old, echoing Marvin Minsky’s "Society of Mind" and even Piaget. The real question is why we’re still stuck in the brute-force era when we’ve known all along that modularity, priors, and inheritance are the evolutionary secret sauce. So who’s taking it seriously now?

Who’s Trying to Build Modular, Bootstrappable AI?

1. DeepMind (Alphabet)

Arguably the most philosophically sophisticated of the big players.

• Gato: A multimodal agent trained across vision, text, and control tasks—a single model with general policies, though not modular in structure.
• Perceiver IO: Modular input/output handling across modalities.
• Dreamer / MuZero: Reinforcement learning agents with world models, trained in simulation and transferred to real environments. DreamerV3 adapts without retraining, pointing toward meta-learning priors.
• Gemini: While trained like GPTs, Gemini reportedly integrates agentic and multimodal reasoning—the next-gen may bake in more structured reasoning modules.

But DeepMind’s Achilles heel: they still mostly rely on giant-scale transformers, and their modular experiments haven’t yet broken into flagship products.

2. Anthropic

Anthropic’s Claude models are outwardly LLMs, but internally, they’re obsessed with:

• Constitutional AI: teaching systems to internalise behavioural principles. A kind of value-based scaffolding.
• Research into mechanistic interpretability: understanding how “features” like arithmetic or deception get encoded and whether they can be modularised or redirected.
• Their long-horizon ambition is to move beyond scaling toward "alignable cognitive architectures", with inbuilt behavioural constraints.

But again: nothing public yet that resembles modular priors or hybrid architectures.

3. Meta (FAIR)

Meta is one of the few genuinely modular research labs at scale.

• Ego4D: An egocentric video dataset to train embodied priors—understanding human attention, goals, and actions from a first-person perspective.
• LLaMA + perception modules: They're building tool-augmented models where vision, memory, and planning are plug-and-play, even if not yet train-once/reuse-everywhere.
• Deep research into "learning-to-learn" (meta-learning) and neuro-symbolic systems.

Still, they seem to prioritise model breadth over modular inheritance.

4. Stanford / Berkeley / MIT

Academia remains the vanguard for genuine architectural innovation.

• Stanford’s Alpaca + modular agents: work on functionally modular RL agents that mirror cognitive faculties.
• Berkeley’s BEHAVIOR benchmark: training agents in household tasks with reusable motor and planning modules.
• MIT’s Neuro-symbolic AI: longstanding focus on blending neural networks with classical rule systems for better generalisation.

But the problem here is scale: these groups do brilliant early work, but they’re dwarfed by the big labs’ budgets and compute.

5. OpenAI

Surprisingly conservative. Despite their rhetoric, OpenAI’s models are:

• Trained monolithically.
• Largely devoid of modular scaffolds or explicitly inherited priors.
• GPT-4 (and 5) rely on sheer token volume and scale.

That said, they are increasingly working on multi-agent systems, tool use, and persistent memory, which could serve as the scaffolding for a more modular brain. But no public signals yet that they’re truly moving away from tabula rasa.

The Financial Incentive Exists. So Why the Delay?

1. Short-termism pays

Brute-force LLMs work now. They're monetisable. Modular AI is hard to debug, evaluate, and productise. No doubt there are many conceptual difficulties.

2. Path dependence

The whole stack—hardware, software, culture—is built around monolithic transformers. Paradigm inertia is enormous.

3. No AI childhood simulator yet

To build structured priors, you need rich, embodied environments with interactivity and continuity. This is exactly what humans have from birth—and what LLMs lack.

4. Tool use is easier than internal cognition

Why develop causal reasoning modules when you can just bolt on Wolfram Alpha? Tool-using LLMs may delay the need for deeper, modular competence.

Who Might Lead the Modular Revolution?

• DeepMind if it ever puts its architectural creativity into a production system.
• Startups like Imbue, Adept, or Reka who claim to focus on agentic, modular AI.
• Open-source communities if someone builds a robust, extensible cognitive scaffold that gains traction—like HuggingFace, but for minds.

Summary

Everyone knows we need modular, reusable, instinct-laden architectures. Few are seriously building them. The financial incentive exists—but the institutional culture, training pipelines, and success metrics still orbit the black hole of brute-force LLMs.

Someone will break the loop. The question is who—before we waste another trillion tokens trying to teach a silicon brain that the sun rises in the east.

---

'I write for the AIs...'

Amazon link

Tyler Cowen, from the blog ‘Marginal Revolution’, suggests that we should be writing for the AIs, the large language models (LLMs) that will one day serve as our digital custodians.

The reasoning is straightforward: our descendants will have neither the time nor the inclination to sift through the mountains of words we leave behind. But the machines will. They will be the ones reading, processing, and perhaps even presenting us to the future. It’s an intriguing notion, but one that raises an immediate question: why bother?

Human relationships are defined by bonds of personal necessity and affection - family, friends, lovers, comrades-in-arms. Beyond this circle, we lapse into mere social obligation, and even that wears thin.

If I had the chance to meet a sixteenth-century ancestor, what exactly would we talk about? The price of grain? The relative merits of different plough designs? His pious peasant worldview would, at best, be a short-term curiosity; at worst, just tedious.

Why, then, should I expect my distant descendants to care for my accumulated wisdom on obsolete telecom network architecture, frankly naive political philosophy, or the minor ups and downs of twenty-first-century Britain?

But perhaps that’s the wrong framing. The real issue is not whether anyone will care, but whether anyone will even feel inclined to. The function of human relationships is mostly to exchange knowledge, provide advice, offer emotional support, and help each other navigate the labyrinth of life. Yet all of this will be, increasingly, the domain of more competent and amenable artificial systems.

Give AI another twenty years to develop fine-grain personality modeling and soft-touch humanoid robotics - and the role of wise old grandparent, experienced mentor, or sympathetic friend will be performed more effectively by an exquisitely tailored machine than by some stilted, digitised reconstruction of a long-dead ancestor.

The living barely listen to their forebears now; why should we expect the future to be any different?

Perhaps we should adjust our Wills. State that our digitised personas may be edited, altered and upgraded, tailored to become a preferred companionate personality for our remote offspring.

In this way allow our individual, historic selves to be subsumed into the collective subconscious of improved and optimised humanity: a Jungian prospect reminiscent of Peter Hamilton’s Edenists in the Night’s Dawn trilogy. The self, once rigid and finite, dissolves into something fluid, continuously enhanced - a digital ancestor no longer preserved but reimagined and repurposed.

Cowen’s vision, then, is not without irony. Yes, the AIs will read our words, digest our ideas, and perhaps synthesise them into some grand collective narrative of human thought. But what remains of us in that process? Do we become mere echoes, endlessly refined until we are no longer distinct from the computational matrix itself? Or do we vanish into irrelevance, our carefully curated personas serving as little more than scaffolding for something newer, sharper, and ultimately indifferent to its origins?

If we are to write for the future, we should do so not out of a desperate hope to be remembered, but because we have something genuinely worth saying. Process not outcome: whether it is read by man or machine is, in the end, incidental.

Pure ATPs: such a disappointment

---

I have always had a soft spot for automated theorem provers (ATPs). There is something both elegant and exciting about formalising your nontrivial problem in, say, a predicate calculus variant, then pressing the start-button and letting the machine solve it by the powers of deep deduction alone.

But ATPs have always disappointed. Doug Lenat, who spent his entire life handcrafting the general purpose intelligent system Cyc, commented at the last that they had originally encoded the facts and rules of the world in a Lisp-like formal language and then handed that knowledge base and the user-query to a powerful Resolution Theorem Prover to deduce an answer. To avoid the interminable waiting they added layer after layer of special-case heuristics. After decades of such aggregation they quietly turned off the theorem prover: it was never being used.

Edinburgh university was one of the centres of Prolog use and research in the 1980s, along with Oxford and Imperial College. Undergraduates hated it: the elegance of its specification capability was complemented by the opacity of its execution model. Debugging logic programs by mentally running a depth-first tree-search with backtracking-on-failure - well, that could tax the abstraction-powers of the keenest young minds. 

I was much happier with Lisp: you know where you are with β-reduction.

In the heady days of those applied ATPs known as Expert Systems the slogan was: “In the knowledge lies the power”; inferential capability was distinctly secondary.

The culmination of this insight was the success of the LLMs c. 2024. Enormous amounts of encoded knowledge - and no reasoning ability at all…

And yet, there are still those of us who value the elegance of inference... and now, in 2025, my happiness is complete: those LLMs can now reason!

---

The orthogonality of consciousness: a discussion

Intellectuals - academics and knowledge-workers - may believe that consciousness is primarily (or even only?) a matter of being intellectually self-aware.

But on that basis an automated theorem prover running a formal metalogic system must be a paradigmatic example of consciousness, when loaded with a self-descriptive specification in an appropriate formal language (cf. John McCarthy).

For the rest of us, consciousness is really the fact that it feels like something to be me. There is the background feeling of "me-ness" - and then storms of emotion, and primary sense qualia such as colours, taste and pain.

Unlike every other phenomenon of reality, no-one has ever proposed a remotely plausible account of how the physical, cellular structure of neurons in the brain could generate the sensation of consciousness.

It's not that we have bad theories, like Phlogiston or Epicycles turned out to be: in truth and remarkably, we have no theories at all.

It is tempting to abandon the casual and unreflective materialist reductionism of our secular culture and revisit the idea that consciousness is perhaps a genuinely independent aspect of existence: something different to time and space; atoms and the void. Perhaps the vanquished tenets of theology and dualism were onto something after all?

So speak the new adherents to panpsychism.

Tempting, right? Goes with the intuitions, doesn't it?

Since ChatGPT is now operating at postdoctoral levels in physics and philosophy, I decided to progress the issue there.

---

Me: "Panpsychism seems to have similar conceptual problems to dualism. Consciousness may be an orthogonal ur-property of reality, but it doesn't manifest orthogonally; instead it couples very tightly to  (human) neuroanatomy. Damage or alter the brain even a little and the subjective quality of consciousness is consequently altered."

---

ChatGPT: "Yes, it does put a damper on panpsychism's initial allure. While the idea of consciousness as a fundamental feature of reality is elegant and intellectually stimulating, its failure to explain the empirical specifics—particularly the brain's privileged role—leaves it feeling incomplete or speculative.

This doesn’t necessarily invalidate panpsychism, but it highlights the need for further refinement. Panpsychism thrives as a metaphysical hypothesis, offering a bridge between subjective experience and physical reality, but it struggles when faced with the rigorous demands of empirical science.

Perhaps its true value lies in pushing us toward deeper questions:

• What does it mean for consciousness to be "fundamental"?
• Why does reality manifest subjectivity in some systems but not others?
• Are we mischaracterizing consciousness by treating it as a property of entities rather than relationships or processes?

While panpsychism may not fully satisfy as an explanatory framework, it reminds us that consciousness resists simple reduction—and that itself is worth exploring further."

---

Me: "So true, and somewhat deflating to panpsychism."

---

ChatGPT: "Panpsychism and the Problem of Consciousness

Panpsychism—the idea that consciousness is a fundamental property of all matter—has an alluring simplicity. It bridges the gap between subjective experience and the physical world by positing that the seeds of consciousness exist everywhere, in everything. But when confronted with the empirical evidence of how consciousness manifests, the theory begins to falter.

---

The Challenge of Coupling

Consciousness in humans is tightly coupled to the brain.

• Localized brain damage alters specific aspects of subjective experience.
• Psychoactive substances profoundly affect our mental states by modifying neural activity.
• Neuroscientific research reveals measurable correlations between brain function and conscious experience.

If consciousness were an "orthogonal" property of reality—existing independently of physical structure—why does it manifest so selectively and in such close dependence on the neuroanatomy of the brain?

---

Panpsychism’s Conceptual Struggles

In this sense, panpsychism faces challenges similar to dualism:

• Interaction Problem: How do the "proto-conscious" properties of matter scale or combine to produce the rich subjective experiences we associate with human consciousness?
• Selectivity Problem: Why does consciousness seem to emerge only in complex systems like the brain and not in simpler or less organized matter?

While panpsychism avoids the stark separation of dualism by embedding consciousness in matter, it struggles to explain why consciousness is so intricately tied to the brain’s physical processes.

---

Possible Resolutions and Their Limits

Panpsychists have proposed several ways to address these issues:

• Emergence: Higher-order consciousness emerges in systems of sufficient complexity, with the brain acting as a kind of amplifier for proto-consciousness.
• Relational Consciousness: Consciousness arises not as a property of isolated matter but from interactions within complex systems.
• Information-Theoretic Approaches: Theories like Integrated Information Theory suggest that consciousness depends on the brain's unique capacity for processing and integrating information.

However, each of these explanations risks diluting panpsychism’s bold claim of universal consciousness by making it seem contingent on the physical rather than fundamental.

---

The Broader Perspective

Panpsychism ultimately struggles to reconcile its metaphysical elegance with the empirical reality of neuroanatomy. It raises profound questions:

• Is consciousness a property of entities, or is it better understood as a feature of relationships and processes?
• Why does the brain appear uniquely suited to manifest consciousness?
• Could consciousness be a bridge between physical and spiritual dimensions, rather than reducible to either?

While panpsychism may not fully satisfy as an explanatory framework, it reminds us of consciousness’s deep mystery and our need for metaphysical approaches that go beyond the limits of materialism and dualism.

This tension between universal consciousness and the brain's privileged role invites us to rethink how we approach one of philosophy's oldest and most enduring puzzles: the nature of conscious experience itself.

---

Me: it would be so ironic if over the next decade, progress in artificial neural nets embodied in socially-competent automata (robots) actually led to a quantitative, scientific theory of consciousness.

Materialism wins in extra time!

As far as I can tell, our best intuitions are that consciousness has something to do with the old idea of the human triune brain structure, a mammalian architecture evolved to support hyper-sociality.

This architecture is driven by the conflict between individualistic drives (personal survival, reproduction, ...) and group constraints - loyalty to obligations, hierarchy and social constraints - without which society cannot function.

People say that self-monitoring and awareness of emotions are nature's way of making sure our arid, ungrounded thinking processes stay on track: negotiate our way to the best compromises using 'gut feel'. But no-one ever made that theory quantitative, nor built systems to experiment with, nor followed through with any insights as to how that might engender a feeling of 'being something it is like something to be'.

But perhaps that's all about to change.

I Am Weaver Quick (But Mostly Just Confused)

 

Image generated by OpenArt

I've been interested in the relationship between brain structure - the neural connectome and its architecture - and personal identity: that interior feeling you have that you are you and no-one else. 

Other people are a closed book to us: there is a respectable philosophical tradition which claims that apart from you, everyone else is a zombie, an unconscious automaton. And how could you tell?

And yet imagery and genetics tell us that human brains are mostly pretty similar, as you would expect within a species.

I look at other people and think: ‘what would it be like to be them? Are they me, but with my own personality settings slightly varied, with dials turned up or down a trifle (in most cases)?

I think it is so.

This is an idea which is hard to communicate, I have found.

My normal writing style is rather academic. Lee Child I am not. But recent progress in AI has given me extra tools to add energy. I'm happy to collaborate with a LLM like ChatGPT or Booksie’s Bob or Google's Gemini Pro 1.0.

So I took my short-story draft to Gemini and asked it to rewrite the narrative with this prompt:

Expand this young adult story to emphasize the dry humor and the brisk pacing. The title is “I Am Weaver Quick (But Mostly Just Confused)”.

And here is the result.

---

I Am Weaver Quick (But Mostly Just Confused)

 

Miranda: “I look at other people and think: ‘What would it be like to be them? Are they me, but with my own personality settings slightly varied, like with their dials turned up or down a trifle? I think it is so."

---

Miranda, a human firecracker disguised as a ten-year-old, was notorious for pronouncements that left her family blinking. This week's gem? "Oh, I so wish I was Weaver Quick!"

Weaver Quick, for the unenlightened, was the world's current flavour of pop star, famous for her voice like spun sugar and a smile that could melt Antarctica. Miranda, meanwhile, resembled a slightly damp tea-towel. Anton, her ever-patient (and slightly exasperated) older brother, couldn't help but snort.

"Right," he said, "because ten-year-old Miranda and an international pop star – same difference."

But then, a mischievous glint sparked in Anton's eye. "Maybe," he mused, "we can make that happen."

Thus began Operation: Pop Star Makeover (via Slightly Dubious Tech). The next day, Anton marched Miranda into the gleaming white doors of the Virtual Celebrity Shop. Inside, teenagers gushed over holographic projections of their idols, like a bizarre, interactive anime convention.

A woman in a lab coat, sporting the universal uniform of "slightly overwhelmed by fangirling teens," approached them. "Who's the lucky celebrity today?" she asked, already reaching for the pre-printed "How to Interact With Your Idol Without Fainting" pamphlet.

"Uh," Anton began, "it's a little different. Miranda here doesn't want to talk to Weaver. She wants to BE Weaver."

The technician blinked, her eye twitching in a way that suggested she'd seen worse. "Right," she said, voice devoid of surprise. "Virtual body swap, then. Follow me."

Miranda, ever the voice of reason (for a ten-year-old), piped up, "But how does scanning my brain turn me into a pop star?"

"Connectome graph-homomorphism," the technician replied, not missing a beat. "Basically, your brain wiring and Weaver's have enough overlap, we can nudge your copy across to hers. Think of it like… rewiring your neural light switch."

Miranda wasn't entirely convinced, but the promise of instant pop stardom was powerful. Soon, she was cocooned in a scanner, feeling like a particularly inquisitive crouton.

When she eventually emerged, Anton, ever the supportive sibling, demanded a debrief. "So, Weaver Quick, huh? What secrets did the pop superstar reveal?"

Miranda, however, looked utterly bewildered. "She said I was confused," she mumbled. "Apparently, there's a difference between wanting to be someone else and, you know, actually replacing yourself with them altogether."

Turns out, the realish Weaver Quick had some not-so-flattering opinions on the whole situation. "Here I am," she'd said, according to Miranda, "a living, breathing popstar. You, on the other hand, are… not. So, congratulations, your wish came true! You're Weaver Quick, minus the whole 'being Miranda as well' part."

Miranda paused, a tear welling up. "Anton," she sniffled, "I don’t get it. I think I just downloaded an existential crisis."

Anton, who hadn't quite anticipated the philosophical ramifications of his prank, could only stare back, equally bewildered. "Me neither, sis," he admitted, finally. "Me neither."

So ended Operation: Pop Star Makeover. Miranda remained stubbornly Miranda, with a newfound appreciation for her own (slightly damp tea-towel) identity. As for Anton, well, let's just say his days of indulging his sister's wilder ideas were officially over. Unless, of course, she wanted to be an astronaut. Now that might be interesting…

---

© Adam Carton and Gemini Pro v.1.0.

Hallucination

From OpenArt

In a quaint terraced house in the north of England, Gwen and Charles lived out their retirement years in harmony. As Gwen's hearing began to fade, Charles stood by her, determined to find a solution. Together, they navigated the challenges of Gwen's diminishing senses, finding solace in their unwavering love for each other.

With the help of cutting-edge technology, Gwen's new hearing aids became her lifeline, allowing her to hear the world with clarity once again. The predictive AI in her devices not only amplified sound but also anticipated Charles' words before they even left his lips, creating a seamless connection between the couple.

One day Charles was suddenly called away, forced to embark on an urgent journey to visit distant relatives. He trusted that their well-stocked home and Gwen’s conscientious routines would keep her safe and provided for, until he could return. On her part, Gwen's fading vision and lapses of memory were such that, sadly, she failed to notice his absence. Her hearing aids continued to fill the void, predicting Charles' comforting responses to her words, keeping her company in his stead.

Illness soon crept into Gwen's fragile body but she found solace in the familiar sound of Charles' voice, even though he was actually miles away. Lying in her bed, whispering to her husband, his intimate responses were comforting, a warm presence nurturing her as she finally fell into a deep sleep, one from which there would be no awakening.

There was an additional function built into Gwen’s prosthesis: a last ditch routine, executed when all else had failed. When Charles rushed back he was engulfed in grief, leavened only by the sense that nevertheless, something of himself had accompanied his wife on her final journey.

Do you think?

"Can't work with concepts"

From OpenArt

“Speculative Encounters with Advanced Extraterrestrial Intelligence: A Framework for Rigorous Analysis”

Except that it wasn't.

---

In an attempt to avoid the well-known Pollyanna-lobotomization of mainstream LLMs, I asked Gemini to adopt the persona of a Professor of Evolutionary Ecology (with a particular interest in Xenobiology). I said this professor was known for lack of sentiment and a strong aversion to wishful thinking. He specialised in clear assumptions and rigorous, logical deduction.

Gemini agreed to be that person.

---

I then quizzed my ‘professor’ about technological aliens: discuss their motivations; describe their options for interstellar travel (technology vs. self-modification); indicate their likely response to humanity on arrival in our solar system.

Reader, I was hoping for the creative application of evolutionary theory to these scenarios (expansionist top predator goes forth...). I wanted informed scenarios built on plausible assumptions and held together by rigorous logic.

I was hoping to learn something. Instead I got the dialogue summarised below at the end. 

Try to stay awake!

---

For me this says something important about what LLMs actually do. They may well build sophisticated conceptual hierarchies, abstracting and generalising over their vast corpus of training material. But in their current architecture, they are not operating at that conceptual level. It's still rote learning.

It's ‘concrete to abstract to concrete’, not ‘concrete to abstract to abstract to concrete’ - as human experts do. It's like trying to learn maths or physics by reading the textbooks but never doing any of the questions.

I noticed the same with maths prompts: the LLMs have the data, have synthesised abstract concepts, but do not reason coherently at that abstract level. So they make rookie errors all the time.

That's why they're not smarter than us yet, not superhuman in extended disciplines and wider culture. Less reading and more doing, please!

---

This is how Gemini summarised our interaction: Pollyanna is never far away.

## Summary of our Discussion: Encountering Advanced Extraterrestrial Intelligence

While encountering advanced extraterrestrial life remains hypothetical, exploring potential scenarios fosters critical analysis. We discussed the limitations of categorizing ET encounters as "biological" or "technological," acknowledging the vast unknown and the need for evidence-based discourse.

Motivations for interstellar travel were explored, ranging from resource acquisition to exploration, while acknowledging the difficulty of applying Earth-centric evolutionary concepts to understand alien motivations. We then examined sending probes and living organisms, recognizing the benefits and drawbacks of each approach.

Shifting the focus to potential collaboration, we acknowledged the limitations of attributing "inherent benevolence" to ET behavior. Instead, we explored scenarios where collaboration might be strategically advantageous, primarily when resources are complex, fragile, or costly to acquire unilaterally. However, the discussion emphasized the inherent fragility and uncertainty of such cooperation due to potential shifts in power dynamics, unforeseen circumstances, and internal conflicts.

In conclusion, while collaboration might hold some potential, it's crucial to maintain a critical and balanced perspective, acknowledging the vast unknown and focusing on logical deduction rather than wishful thinking.

**Recommendations:**

1. **Continued scientific exploration** is essential.

2. **Critical and evidence-based approaches** are necessary.

3. **Acknowledge the limitations of our understanding** to foster honest discussion.

4. **Hypothetical scenarios** can be valuable thought exercises.

---