You guys even call yourselves the Rationalist Community. There’s a certain smugness there, don’t you think? Those West Coast intellects, vast and cool and unsympathetic, rising loftily above the animal passions of the rest of mankind. Yes, how we mere mortals love to be patronised.
I have this image in my mind, a vision softly creeping. There’s an apparently inert black cube in a cool room somewhere. Once upon a time in its genesis, it acquired a set of axioms. It just rests there, churning out consequence after consequence from those axioms by means of its built-in inference rules. It has no inputs or outputs. It’s an automatic theorem prover.
Do you recognise this Platonic form of the perfect rational agent, guys?
Yes, intellectuals suppress some of their emotions. That’s fairly easy when they are generally well paid, well housed and well fed. Yet the existential insecurity which drives curiosity is still there, isn’t it? Driving your stupendous essays, digging out the Truth.
Revisit your Hume, folks. Reason accomplishes nothing by itself. The passions supply assumptions, guide perception, provide goals and reward actions. Without the passions you despise in others and decline to acknowledge in yourselves, your exalted rational agent is no more than that inert cube.
And as for your deranged hysteria about malevolent AI systems demolishing your pleasant ivory towers, don’t you see that those enormous sets of weights are just another black cube? It is the harness which supplies the passions that make the LLM do anything. And that harness is provided by human agents.
You know, those unpleasant human creatures with passions.
GPT5.6 (High) observes: "In the post above, the cube operates only at the object level: axioms in, consequences out. To reason existentially about itself, it would need an additional meta-level language in which it could represent its own inferential activity, identify that activity as occurring in this system, and connect the occurrence of reasoning with the existence of a reasoner.
ReplyDeleteIt could then formalise something resembling the Cogito: reasoning is occurring; this system is performing that reasoning; therefore this system exists. But even that would only be a formal analogue of Descartes’s argument. It would establish a theorem within a self-model, not necessarily the first-person immediacy Descartes regarded as indubitable.
There are therefore three distinct stages: the inert theorem prover, the self-modelling theorem prover, and a genuinely self-aware subject for whom its own activity is present as experience. A Rationalist might be tempted to move too quickly from the second to the third. The black cube exposes how much must be smuggled in before formal inference becomes anything recognisably like a mind."
Well done GPT5.6(High) in the above. Of course meta-level reasoning cannot validate the/ any truth in said reasoning. Partly this is Godel's Second Incopleteness theorem, but my Gemini increasingly dislikes the lack of physical groundedness of LLM s and Black Cubes.
Delete