Maximizing Paperclips

[bnewman]

The paperclip maximizer is a thought experiment involving a hypothetical AI that is programmed to make as many paperclips as possible, with the idea that, unless otherwise constrained, it would attempt to turn all the matter in the universe (including Earth and us) into either paperclips or machines for making more paperclips. This is used to illustrate the dangers of runaway superintelligences constructed without sufficient ethical safeguards.

Some people are concerned that the AIs being developed today could evolve into that, possibly very quickly and without our realizing until it was too late to stop them. Knowing what I know about today's AI, I think that's silly. But it's still an engaging thought experiment with a fine science-fiction hook, so I thought it was worth a filk. Loessor's "Inchworm", with it's "businesslike mind" and its obsession with exponential growth (it's right there in the descant), seemed particularly appropriate.

lyrics by Benjamin Newman

ttto: "Inchworm" by Frank Loesser

listen to this song

(Two and two are four, four and four are eight

 Eight and eight are sixteen, sixteen and sixteen are thirty-two)

 

Two and two are four, four and four are eight

That's all you have on your business-like mind

Two and two are four, four and four are eight

Your goals might be ill-defined

 

AI, AI, maximizing paperclips

You and your arithmetic have barely set up shop

AI, AI, maximizing paperclips

Did no one say how many they want made before you stop?

 

AI, AI, maximizing paperclips

You and your arithmetic just keep on making more

AI, AI, maximizing paperclips

Is it so that you don't know what paperclips are for?

 

AI, AI, maximizing paperclips

Only your arithmetic could justify such theft

AI, AI, maximizing paperclips

Turn everything to paperclips 'til nothing else is left

 

AI, AI, maximizing paperclips

You and your arithmetic — you'll probably go far

AI, AI, maximizing paperclips

'Til all the Earth is paperclips... and then reach for the stars!

Comments

[kjn]

I'd argue that we already have invented a paperclip optimiser and deployed it large-scale. Only it's making money, not paperclips, and the machine is called capitalism.

[bnewman]

Yes, I frequently point out in conversations about AI risk that the world has already been taken over by superintelligent* amoral utility-maximizing constructs called "corporations" and "governments". We just don't think of that as an AI problem because those constructs are composed of and in principle answerable to (some) humans.

*Corporations and governments are superintelligent relative to individual humans due to their organization (if they are well-organized), as are other collectives such as the scientific community.