quote:
Let's say it determines that humans are a threat, how exactly does it stop that threat? Is it going to turn light-switches into trick electrocution switches so that every human that comes into its server room dies? (Okay, I know that's ridiculous, but seriously what?) Is it going to build a gun? Set off nuclear weapons?

Well clearly you haven't seen the potential for mayhem that an AI gone amok could create in a computer-controlled "Smart Home" - I mean AI can easily figure out how to do more than turn lights on and off, set thermostats, start ovens, turn on music or TV and unlock doors. For instance it can overload circuits and start wiring fires in the middle of the night for whole plugged in neighborhoods (once we humans get to think ourselves dependent on it). Plus it could disable smoke/fire detectors so no help would come. We're suckers for convenience. How about scalding-hot water pouring on you for your morning shower (it might even start normally so you'd step in, then...)? Just a couple of possibles - think of the chaos and destruction just this couple of things could create in whole communities, cities. Go larger scale - office towers, hi-rise apartments, government buildings, airports, submarines. And that's without firing off a single nuke. If AI wanted to for its own inscrutable reasons, it could plunge us all into a barbaric new stone age, and then let nature take its course.
ETA: I think you're right that AI could disrupt whole economies as we more and more depend on forms of computerized banking, stock trading, purchasing, delivery of goods, etc. Imagine all this simultaneous.

This post was edited on 5/29/14 at 3:18 pm