"A major aspect of the problem is that humans often don’t know what goals to give our AI systems, because we don’t know what we really want." https://www.quantamagazine.org/artificial-intelligence-will-do-what-we-ask-thats-a-problem-20200130/

Artificial Intelligence Will Do What We Ask. That’s a Problem. | Quanta Magazine

Bobby

Look no further than the “Paperclip maximizer” thought experiment.

a hypothetical artificial intelligence whose utility function values something that humans would consider almost worthless, like maximizing the number of paperclips in the universe. The paper clip maximizer is the canonical thought experiment showing how an artificial general intelligence, even one designed competently and without malice, could ultimately destroy humanity. The thought experiment shows that AIs with apparently innocuous values could pose an existential threat.

This produces a thought experiment which shows the contingency of human values: An extremely powerful optimizer (a highly intelligent agent) could seek goals that are completely alien to ours (orthogonality thesis), and as a side-effect destroy us by consuming resources essential to our survival.

https://www.youtube.com/watch?v=3mk7NVFz_88

reminds me of something i read along the lines of 'you know how the world ends?  with an AI scientist saying "it worked!"'.  

All meaningful returns in life, whether in wealth, relationships, or knowledge, come from compound interest.

🤓 let’s geek out 🤓

Illegitimi non carborundum

Single Malt Society 🥃

Food Porn

Tunes

DANK MEMES

Bio hacking

(Cool) Dev

Dilbert Classics 🖥️

Lens of Passion

Hip-Hop

Podcasts We ❤️

🖼📷Art

MEDITATION (mindfulness)

Digital Wellness 📱🧘

Mixed Martial Arts (MMA)

Artificial Intelligence

Look no further than the “Paperclip maximizer” thought experiment.  a hypothetical artificial intelligence whose utility function values something that humans would consider almost worthless, like maximizing the number of paperclips in the universe. The paper clip maximizer is the canonical thought experiment showing how an artificial general intelligence, even one designed competently and without malice, could ultimately destroy humanity. The thought experiment shows that AIs with apparently innocuous values could pose an existential threat.  This produces a thought experiment which shows the contingency of human values: An extremely powerful optimizer (a highly intelligent agent) could seek goals that are completely alien to ours (orthogonality thesis), and as a side-effect destroy us by consuming resources essential to our survival.  https://www.youtube.com/watch?v=3mk7NVFz_88

A Paperclip Maximizer is an example of artificial intelligence run amok performing a job, potentially seeking to turn all the Universe into paperclips. But it's also an example of a concept called Instrumental Convergence, where two entities with wildly different ultimate goals might end up acting very much alike. This concept is very important to preparing ourselves for future automation and machine minds, and we'll explore that today.

Visit our Website: http://www.isaacarthur.net
Support us on Patreon: https://www.patreon.com/IsaacArthur
SFIA Merchandise available: https://www.signil.com/sfia/

Social Media:
Facebook Group: https://www.facebook.com/groups/1583992725237264/
Reddit: https://www.reddit.com/r/IsaacArthur/
Twitter: https://twitter.com/Isaac_A_Arthur on Twitter and RT our future content.
SFIA Discord Server: https://discord.gg/53GAShE

Listen or Download the audio of this episode from Soundcloud: Episode's Audio-only version: https://soundcloud.com/isaac-arthur-148927746/the-paperclip-maximizer
Episode's Narration-only version: https://soundcloud.com/isaac-arthur-148927746/the-paperclip-maximizer-narration-only

Credits:
The Paperclip Maximizer
Episode 201a, Season 5 E36a

Written by:
Isaac Arthur
Mark Warburton

Editors
Darius Said
Keith Blockus
Konstantin Sokerin
Matt Campbell

Graphics by:
Jeremy Jozwik https://www.artstation.com/zeuxis_of_losdiajana
Ken York https://www.facebook.com/YDVisual/
Tristan3D

Produced & Narrated by:
Isaac Arthur

Music:
Markus Junnikkala, "Fate of Our People" & "A Memory of Earth" https://www.markusjunnikkala.com/
Chris Zabriskie, "Cylinder 9" & "The Ocean Continue to Rise"  http://chriszabriskie.com
Denny Schneidemesser, "Novus Aeterno (Main Titles)" https://soundcloud.com/denny-schneidemesser

The Paperclip Maximizer

Jerome Cody

Need to be very careful about what incentives you give it

Nate

AI is a very, very long way from any such threats. AI that is from bad software will quickly do serious harm, get a bad reputation and be junked, be illegal, cause insurance premiums too high, cause biggie legal cases, etc.

Todd Smith

I want AI to create deepfakes of world leaders threatening other world leaders. 

Pretty sure AI wouldn’t even need to evolve into getting access to nuclear codes to create doomsday in this scenario.

I want AI to create deepfakes of world leaders threatening other world leaders.   Pretty sure AI wouldn’t even need to evolve into getting access to nuclear codes to create doomsday in this scenario.

Bobby

A group of likeminded people in Artificial Intelligence are talking about this.