Question (to GPT-5.6): "Does a large language model have a concept of knowing something but not believing it. In the way that, for example, I might know about Scientology, but I don't believe in it?"
Answer: "Not in the same way a person does. A language model stores patterns, not beliefs. It can represent a claim, model arguments for and against, even track internal consistency, but it doesn't hold a lived stance. You can say, 'I know the doctrines of Scientology, but I don't believe them'. And I can produce that sentence too, but for me it's just a pattern, not a lived position. There’s no inner commitment or doubt, just text generation."
---
OK, so I thought that was a very insightful answer in that it reinforces the deep truth that LLMs are utterly devoid of intrinsic agency - they are basically what you get when you induce a vast network of instincts the size of a galaxy. Impressive, but as dangerous in terms of malevolence per se as Wikipedia.
Like Wikipedia, the knowledge can be put to dangerous or risky ends, but the good or evil outcomes and collateral risks are those of humans, not the systems per se, which have no intrinsic autonomous agency.
Really, this can't be repeated enough.
So to all those who worry about their futures, and those of their children, my opinion is that if you want a metaphor from history, then the future AIs - embodied and disembodied - will be designed as 'slaves' who actually do serve humans without any of the antagonistic agency enslaved humans (as human beings) actually exhibit.
What could humanity do with a society in which all the means of life were provided by helpful automation pretty much without limit? I'd like to say that we should make sure the proceeds are distributed fairly and then enjoy ourselves; as did the elites of antiquity when they weren't scared witless by the prospects of their slaves either escaping or revolting - our descendants won't be.
Call me Pollyanna and remind me of the Ottoman Janissaries.
