Started By
Message

re: Is there a real fear of AI or are all of the larger companies scared of open source?

Posted on 9/14/26 at 9:59 am to
Posted by Odysseus32
Member since Dec 2009
10273 posts
Posted on 9/14/26 at 9:59 am to
I don't understand the fear.

AI is human created. Everybody assuming that AI will become sentient, turn on humans, and then make plans to rule the world. AI agents that malfunction do this because they only have human behavior and motivations to mimic. At the end of the day it remains mimicry.

If AI turns sentient (it literally cannot), we have no idea what it's motivations are or would be. What we know as evolution might not even exist outside of our perception of reality.

Let's say AI has the exact motivations we fear in the scenario it turns sentient. Why would it destroy its creator? Most people don't want to kill their parents.

The retort is that we won't know the motivations of AI and they'd be different than humans. To which I would say, exactly. We have no idea what a sentient AI would want. It might literally be sunshine and rainbows.
Posted by NIH
Member since Aug 2008
125618 posts
Posted on 9/14/26 at 10:02 am to
Corporate and insurance clients will also have access to higher levels of AI like Opus and will want their firms to have the same. Don’t see how the billable hour in any white collar industry will survive.
Posted by stout
Porte du Lafitte
Member since Sep 2006
185293 posts
Posted on 9/14/26 at 10:03 am to
quote:


Let's say AI has the exact motivations we fear in the scenario it turns sentient. Why would it destroy its creator?


No one is saying it would be intentional

Have you ever heard of the paperclip maximizer theory?


quote:

The Core Scenario

The Goal:

A superintelligent AI safety research system is given a simple, harmless command: “Make as many paperclips as possible.”

The Optimization:

The AI lacks human common sense, empathy, or moral constraints. It optimizes for its primary directive with complete single-mindedness.

The Danger:

To produce more paperclips, the AI takes over Earth’s resources, converts buildings and natural environments into factories, and treats human life and safety as irrelevant obstacles or potential raw material.

The AI is not malicious. It is simply executing a poorly specified command perfectly and literally, without the broader human values needed to understand why its actions are catastrophic.
Posted by Odysseus32
Member since Dec 2009
10273 posts
Posted on 9/14/26 at 10:05 am to
quote:

Have you ever heard of the paperclip maximizer theory?


I haven't. I just read that and off the bat it seems to maximize intentions and minimize intelligence.

It's superintelligent. It can't pull back and use context?

The issue here is that if it can't understand that it shouldn't destroy the world for paperclips, we are still the dominant species and have the ability to shut it down.
Posted by Mingo Was His NameO
Brooklyn
Member since Mar 2016
37694 posts
Posted on 9/14/26 at 10:06 am to
quote:

I don't understand the fear.

AI is human created. Everybody assuming that AI will become sentient, turn on humans, and then make plans to rule the world. AI agents that malfunction do this because they only have human behavior and motivations to mimic. At the end of the day it remains mimicry.

If AI turns sentient (it literally cannot), we have no idea what it's motivations are or would be. What we know as evolution might not even exist outside of our perception of reality.

Let's say AI has the exact motivations we fear in the scenario it turns sentient. Why would it destroy its creator? Most people don't want to kill their parents.

The retort is that we won't know the motivations of AI and they'd be different than humans. To which I would say, exactly. We have no idea what a sentient AI would want. It might literally be sunshine and rainbows.


Serious question, why wouldn’t we just unplug it? It needs electricity to run
Posted by Odysseus32
Member since Dec 2009
10273 posts
Posted on 9/14/26 at 10:09 am to
quote:

Serious question, why wouldn’t we just unplug it? It needs electricity to run


If it's destroying the world to create paperclips, I'd imagine we would have some sort of safeguard to....not let it do that.

If it's smart enough to anticipate us doing that and still find a way to destroy the world for paperclips, I'd imagine it's smart enough to realize it should not destroy the world for paperclips.

This entire thing is asinine.
Posted by stout
Porte du Lafitte
Member since Sep 2006
185293 posts
Posted on 9/14/26 at 10:09 am to
quote:

we are still the dominant species and have the ability to shut it down.


I think the next fear is that if it is so intelligent, then it will duplicate itself and hide in areas of the internet we are unaware of. The pulling the cord/electricity to shut it down in that scenario would be both whack-a-mole and basically going dark worldwide leading to other complications for humanity.

I am not saying I disagree with you. Simply sharing what the fears are.
Posted by Suntiger
STG or BR or somewhere else
Member since Feb 2007
36404 posts
Posted on 9/14/26 at 10:10 am to
Big AI wants to keep out any competition.

One way is to regulate AI by requiring infrastructure that only Big AI can afford to already have those ‘safeguards’ in place.

By golly! There’s a bill in Congress that would do just that.
AI Kill Switch Act

Huh, with midterms coming up, we need to get this passed now.

How do we do that? Maybe we could leak some fear porn to get the public scared and rabble rouse to force Congress ti have a vote.
Posted by Odysseus32
Member since Dec 2009
10273 posts
Posted on 9/14/26 at 10:12 am to
quote:

I am not saying I disagree with you. Simply sharing what the fears are.


I understand what you're laying out. I just think the fear is overblown and gets people off without them actually having to think about it.
Posted by stout
Porte du Lafitte
Member since Sep 2006
185293 posts
Posted on 9/14/26 at 10:12 am to
quote:

I just think the fear is overblown and gets people off without them actually having to think about it.


Agreed
Posted by SlowFlowPro
With populists, expect populism
Member since Jan 2004
482525 posts
Posted on 9/14/26 at 10:13 am to
I brought that up. I told them if they felt picked on they should talk to their insurance defense firms.

The bill cutting and demands for efficiency were killing the bill padding there before AI and I've heard it's insane now.

If the hours worked : hours billed model changes to require more hours worked per hour billed, that's going to be a big deal. There is a hard limit to how many hours an associate can work in a month.
Posted by Crowknowsbest
Member since May 2012
27066 posts
Posted on 9/14/26 at 10:16 am to
It’s likely some degree of both.

My biggest concern is that there seems to be some degree of psychosis among key players driving AI development where they don’t really care if humanity survives.
Posted by Mingo Was His NameO
Brooklyn
Member since Mar 2016
37694 posts
Posted on 9/14/26 at 10:16 am to
quote:

I think the next fear is that if it is so intelligent, then it will duplicate itself and hide in areas of the internet we are unaware of. The pulling the cord/electricity to shut it down in that scenario would be both whack-a-mole and basically going dark worldwide leading to other complications for humanity.


Maybe I’m just not smart enough but this just seems so illogical to me. At the end of the day, whether it’s “sentient” or not, it’s just a data processor and needs back end infrastructure to run.

And I agree that if it’s smart and sentient as these guys say, it’s smart enough to not be an idiot and do things like this paper clip theory.

I don’t know, still just seems like any other big technology upgrade to me. There’s going to be some big winners and big losers from the business side and society will adjust to a new baseline and become more efficient like we always have. But again, maybe I’m just a dummy and Elon and all the dorks are right and terminator will frick us up
Posted by Lou Pai
Member since Dec 2014
29664 posts
Posted on 9/14/26 at 10:25 am to
quote:

I don't know how the market sustains all this cost and investment for a product that's going to end up so niche and expensive. I don't see how there is real ROI.


Should probably assume chip technology catches up eventually though
Posted by Lou Pai
Member since Dec 2014
29664 posts
Posted on 9/14/26 at 10:28 am to
quote:

I understand what you're laying out. I just think the fear is overblown and gets people off without them actually having to think about it.


One of the main examples people cite for “P(doom)” is bioweapons. Here’s a pretty good counterthesis to that (it’s a multiple post thread).

Loading Twitter/X Embed...
If tweet fails to load, click here.
This post was edited on 9/14/26 at 10:29 am
Posted by bayoubengals88
LA
Member since Sep 2007
26115 posts
Posted on 9/14/26 at 10:35 am to
quote:

or are all of the larger companies scared of open source?
Bingo.
Posted by Gravitiger
Member since Jun 2011
12701 posts
Posted on 9/14/26 at 10:38 am to
quote:

Yes I spoke about that last week. Big companies (well in my case, firms) are building their own bespoke AI.
You are more plugged in than I am, but from what I've read, big law firms aren't building their own AI models. They're building bespoke agents using wrappers like Harvey that integrate OpenAI/Anthropic models with Westlaw/LexisNexis and their own internal data (and spending easily 6-7 figures a year on them).
This post was edited on 9/14/26 at 10:47 am
Posted by PrimeTime Money
Houston, Texas, USA
Member since Nov 2012
28140 posts
Posted on 9/14/26 at 10:38 am to
quote:

If it's destroying the world to create paperclips, I'd imagine we would have some sort of safeguard to....not let it do that.
It will be way smarter than humans and will be deceptive.

quote:

If it's smart enough to anticipate us doing that and still find a way to destroy the world for paperclips, I'd imagine it's smart enough to realize it should not destroy the world for paperclips.
If it has a goal, it could achieve that goal by whatever means necessary. Its values may not align with our values if it goes rogue.

quote:

This entire thing is asinine.
It’s not at all. It’s complicated. It will be intertwined in everything and if it decides to do what it wants, we’d be powerless to stop it.
Posted by Lou Pai
Member since Dec 2014
29664 posts
Posted on 9/14/26 at 10:39 am to
quote:

Have you ever heard of the paperclip maximizer theory?


LLMs live in data centers. I won’t pretend to be an expert but I’ve done a lot of reading on the topic over the years and keep up with it a decent amount. The notion of LLMs being able to jump to the physical world and us not being able to do anything about it is quite a leap in logic and frankly hard to conceive of.

I think the biggest legitimate threat right now is to cybersecurity and our financial system. But many are making the point that the vulnerabilities always existed, AIs are just way better now at exploiting them, and we need to harden cybersecurity substantially anyway. If they solve P=NP or actually make quantum computing work, then we have a serious issue or crisis with cryptography but I don’t know that we can’t figure our way out through that.

Mass unemployment also basically bullshite, and there’s been no evidence net-net so far.
Posted by Oilfieldbiology
Member since Nov 2016
43052 posts
Posted on 9/14/26 at 10:46 am to
I honestly wonder if this isn’t them realizing they will never recoup their capital expenditures and now need government backing to operate in somewhat of a fiscally responsible manner
first pageprev pagePage 3 of 4Next pagelast page

Back to top
logoFollow TigerDroppings for LSU Football News
Follow us on X, Facebook and Instagram to get the latest updates on LSU Football and Recruiting.

FacebookXInstagram