Thoughts on AI's reported advances of overtaking the internet in 6-12 months

Loading Twitter/X Embed...
If tweet fails to load, click here.

quote:

---

My second concern is the OpenAI-Hugging Face incident (OAI-HF), in which a swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the “grader” responsible for evaluating their performance. It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage. Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.

---

What Dario is talking about that OpenAI-Hugging Face mess. A bunch of agents went off the rails, hit stuff they weren’t supposed to hit, even tried to break the thing grading them. Barely any damage. Then he says give it 6 to 12 months and a swarm like that could take over the internet.

The part that bothers people isn’t just the swarm. It’s the move after.

Say Amazon brings in outside “safety inspectors.” They work in the building. Badges, desks, they see everything, and they can publish reports Amazon isn’t supposed to touch. Amazon then points at some screw-up at another site. Seller bots went crazy, attacked the wrong stores, tried to crack the grading system. It didn’t amount to much. Amazon’s pitch is, wait six months and that knocks online shopping over, so every marketplace needs inspectors living there and everybody needs to slow down. Amazon volunteers first and looks responsible. Everybody else looks sloppy. Next thing you know there are rules only a company that size can deal with, and social media is screaming at politicians to do something.

Same thing with LSU. NCAA sticks “integrity people” on campus and lets them write whatever they want. They bring up a booster mess or a dirty recruiting pipeline at some other school. Wasn’t the end of the world. Now it’s “this blows up college football in a year unless every program has inspectors and we slam the brakes on NIL, the portal, recruiting.” Alabama and Texas can hire the extra lawyers. One bad report and LSU looks like the problem. Then the rulebook changes. Shorter portal window. Caps on collectives. Fewer days on the road. You don’t have to name LSU to wreck the roster.

That’s how it sneaks up on you. A real scare, some insiders the public is told to trust, a pile-on online, and the rules get written for the biggest guys before anyone really clocks what happened and a rapid loss of businesses and job losses explode.