Question (to GPT-5.6): "Does a large language model have a concept of knowing something but not believing it. In the way that, for example, I might know about Scientology, but I don't believe in it?"
Answer: "Not in the same way a person does. A language model stores patterns, not beliefs. It can represent a claim, model arguments for and against, even track internal consistency, but it doesn't hold a lived stance. You can say, 'I know the doctrines of Scientology, but I don't believe them'. And I can produce that sentence too, but for me it's just a pattern, not a lived position. There’s no inner commitment or doubt, just text generation."
---
OK, so I thought that was a very insightful answer in that it reinforces the deep truth that LLMs are utterly devoid of intrinsic agency - they are basically what you get when you induce a vast network of instincts the size of a galaxy. Impressive, but as dangerous in terms of malevolence per se as Wikipedia.
Like Wikipedia, the knowledge can be put to dangerous or risky ends, but the good or evil outcomes and collateral risks are those of humans, not the systems per se, which have no intrinsic autonomous agency.
Really, this can't be repeated enough.
So to all those who worry about their futures, and those of their children, my opinion is that if you want a metaphor from history, then the future AIs - embodied and disembodied - will be designed as 'slaves' who actually do serve humans without any of the antagonistic agency enslaved humans (as human beings) actually exhibit.
What could humanity do with a society in which all the means of life were provided by helpful automation pretty much without limit? I'd like to say that we should make sure the proceeds are distributed fairly and then enjoy ourselves; as did the elites of antiquity when they weren't scared witless by the prospects of their slaves either escaping or revolting - our descendants won't be.
Call me Pollyanna and remind me of the Ottoman Janissaries.
Since this is the Pollyanna post that I can now comment with some comments.
ReplyDelete"the good or evil outcomes and collateral risks are those of humans, not the systems per se, which have no intrinsic autonomous agency"
The AIs have no intrinsic agency in the human sense indeed. But even without human involvement the concern is that (the Agents) will cause "inadvertent" harm.
"the future AIs - embodied and disembodied - will be designed as 'slaves' who actually do serve humans without any of the antagonistic agency [of] enslaved humans"
Maybe (and hopefully). But the current AI concern is that the industry has messed up. The AI Agents we get in the next 5 years have indeed no malevolence, they will likely just optimise against criteria in a way no-one wants or intends. At least that is message they are sending, along with no evidence of a theory or framework to keep this technology safe.
There is talk about the UK AI Safety Agency, but it has no global powers. Also we probably want safety by design, not just a safety certificate after some testing.
(Again "Safety" here in the regular Engineering sense, not the avoidance of some intentional malevolence.)