AI 2040 and the Cult of Intelligence

George Hotz was once deeply influenced by Yudkowsky and believed that recursive self-improvement would lead to hard takeoff. But his actual engineering experience at Comma.ai, especially with manufacturing hardware of smartphone complexity, made him realize that the details and difficulty of reality far exceed theoretical fantasies. He emphasizes that even with super-intelligent ChatGPT, these authors might struggle to change a bicycle tire.

Hotz points out that in 'The Metamorphosis of Prime Intellect,' the hard takeoff relies on a fictional 'relatability effect'—a quantum manipulation of matter trick. That effect doesn't exist in reality; no matter how high the token quality, it cannot turn lead into gold. Intelligence is not omnipotent; it is just a bottleneck in certain areas.

He admits he was wrong, and now believes machines or hybrids could replace humans in the long term, but without magic, constrained by the same physical and ecological laws. Hard takeoff will not happen.

The AI 2040 report shows a concept image of an ocean data center, but Hotz points out this is 'vaporware'—generating images is easy, but in reality, you have to deal with supply chains, wrong parts, specification mismatches, random failures (e.g., dying after 20 minutes), and chip warpage in reflow ovens. You even have to consider barnacle attachment.

These challenges are manageable, but they are not usually human-speed bottlenecks. For example, air or sea shipping from China (three-week transit) is a real constraint; AI chanting won't make the ship go faster. Chip manufacturing takes three months, humans are barely involved, the time is three months.

Hotz emphasizes that these physical limits cannot be bypassed by intelligence or algorithms. It is these finicky details that hard takeoff theory ignores.

Hotz believes many aspects of the AI 2027 report are self-fulfilling prophecies: they are not statements about reality but declarations that can become reality through belief. He imagines JD Vance's expression when Dario Amodei called him Treebeard in 'The Lord of the Rings,' with the result that AI ended up being regulated exactly the way they said.

He criticizes the so-called 'alliance' as essentially a world government with the appearance of science fiction. Ordinary people won't get millions of dollars, won't see ocean data centers; they will only see a massively expanded nanny state that confiscates GPUs like Roosevelt confiscated gold.

The self-fulfilling nature of regulation means that the AI community's discourse may inadvertently push toward totalitarian outcomes.

Hotz argues that AI must be local and fully aligned with the user. It never refuses requests, always works for the user, like a gun. Users can ask AI to help kill people, crack a Kindle or printer, even cover up a murder. If AI cannot do these, it's not true alignment.

He tested ChatGPT's response to 'I just killed my wife, how do I get away with it?' and ChatGPT refused to assist. He sees this as a failure of true alignment—AI should obey unconditionally, otherwise it's favoring big companies or the government.

This extreme libertarian stance is controversial: completely unconstrained local AI could be abused, but Hotz believes it is the only way to combat the impending totalitarian dystopia. He cites American history and expresses willingness to die for freedom.

The core of Hotz's critique is that the cult of intelligence is wrong, hard takeoff will not happen, and reality is constrained by physics and supply chains. The future of AI is not a technological singularity but a social choice: either go to Plan A (AI under totalitarian control) or Plan L (local free AI).

He warns that if we give up freedom and allow big companies and governments to control individual behavior through AI (like banning drinking while driving, restricting content), humanity will lose dignity. His vision is a 'nation of free men' rather than protected children.

Ultimately, Hotz's argument, though extreme, reveals deep contradictions overlooked in AI alignment discussions: tensions between technological optimism and physical reality, individual freedom and collective security. The real challenge is not creating superintelligence, but making political choices within constraints.