Gosh AI is jolly isn’t it? Every day I have this deep rooted (and well shared) feeling of anxiety that it’s going to put a lot of people out of work and cause societal breakdown within five years.
This feeling emerged somewhere in 2023, after ChatGPT had launched and just about every other generative AI tool came out or got suddenly better. AI is now making regular headlines for scything through the job market. Graduates and managers alike are feeling the pinch.
But what if we take AI to its furthest extreme? The one we’re vaguely familiar with from the Terminator or The Matrix franchises? You know the story where machines are attempting to destroy us?
This scenario, I’ve pleasantly found, is utterly implausible. We’re not going to have killer humanoid robots or giant squid like monsters flying after us any time soon.
No, there’s something far more terrifying.
The ‘Intelligence Explosion’
There’s reasonable consensus that there will be a major technological event in the next 15 years. This was first defined as an ‘Intelligence Explosion’ by Irving Good in the 1960s, and has since been labelled ‘the Singularity’ by computer scientists like Ray Kurzweil. It’s also covered extensively by Nick Bostrom in his book Superintelligence.
The sudden explosion of knowledge occurs via ‘recursive self improvement’ - that is the ability for AI to programme itself and drive an astonishing rate of scientific progress in a super speed feedback loop. Philosopher Sam Harris explains this event and its many conundrums in a 2018 Ted Talk.
Philosopher Sam Harris in one of Ted’s most sobering talks.
Imagine 20,000 years of technological progress occurring in a week. Such pace of change is utterly incomprehensible to humans. Meanwhile, how could we possibly keep such an intelligence under control and aligned to our own goals? This is what’s commonly known as the control problem.
Let’s get a bit of perspective. We are by far the most intelligent species on Earth. Our shaping of the landscape to our own goals affects many other species, and in some cases destroys them. If we take building a road from A to B, there might be an ant hill in the way. We don’t hate ants, but the construction team is not going to think twice about flattening and tarmacking it. Bye bye ant hill. (To paraphrase Elon Musk).
A superintelligent AI could have a similar level of power and goal setting. It views humans at a similar level of cognitive ability as we view ants. It is uncaring about exterminating ants to reach its goals. If it sees us as antithetical to its goals then it could even make us extinct. AI doesn’t exactly have to be evil to do this, just as humans aren’t when building a road.
The Paperclip Maximiser
Nick Bostrom famously marks out the control problem via a thought experiment called the Paperclip Maximiser. In this, a superintelligent AI agent has the mundane goal of maximising paperclip production.
It is increasingly brilliant at its task, but eventually it realises humans might switch it off. It also realises humans are made of atoms, which could be better off rearranged into paperclips. So it makes humans extinct. Unchallenged, it turns all matter in the universe into paperclips.
We don’t generally equate superintelligence to paperclip production. But the point is that AI without alignment, even with mundane goals, may just treat the Earth like an ant hill. When it finds out there’s a whole universe out there and how to colonise it, what are we in the grand scheme of things? Just reasonably clever apes circling an average star. (To paraphrase Stephen Hawking).
Thankfully if AI does ever want to convert all matter into paperclips, then we’re unlikely to have a drawn out physical war as visualised in science fiction.
Just think of the colonial wars of early modernity. In 1519 relatively small band of Spaniards arrived in Mexico with a few horses, cannons and steel armour. They fought against (and allied themselves to) tens of thousands of indigenous warriors who didn’t even have metal weapons. They had clubs and blades made from volcanic glass. It didn’t take much to smash them to bits and conquer the Aztecs within two years. This was a military technology gulf of a few thousand years. Make that technology gulf 20,000 years, and we can see where this is going.
Molecular nanotechnology is the end game. If you have robots that are able to rearrange atoms, then there is almost infinite power in building infrastructure. There’s also unfettered potential for covert and instantly deadly weapons systems. We might want to write into the super agent’s prompt that it is to have no control over nanorobots, but it may have already decided on its treacherous turn.
Without alignment, it would be over very quickly indeed - which is perhaps reassuring in the scheme of things. We invent something incredible and we’re out like a light. It’s not the kind of stuff that Hollywood stories are made of, so I made a YouTube video all about it.
