Skype cofounder Jaan Talinn is on a mission to save us from feral AI

quote:

---

“It began three and a half billion years ago in a pool of muck, when a molecule made a copy of itself and so became the ultimate ancestor of all earthly life. It began four million years ago, when brain volumes began climbing rapidly in the hominid line. Fifty thousand years ago with the rise of Homo sapiens. Ten thousand years ago with the invention of civilization. Five hundred years ago with the invention of the printing press. Fifty years ago with the invention of the computer. In less than thirty years, it will end.”

Jaan Tallinn stumbled across these words in 2007, in an online essay called “Staring into the Singularity.” The “it” is human civilization. Humanity would cease to exist, predicted the essay’s author, with the emergence of superintelligence, or AI that surpasses the human intellect in a broad array of areas.

Tallinn, an Estonia-born computer programmer, has a background in physics and a propensity to approach life like one big programming problem. In 2003, he had co-founded Skype, developing the backend for the app. He cashed in his shares after eBay bought it two years later, and now he was casting about for something to do. “Staring into the Singularity” mashed up computer code, quantum physics, and ­Calvin and Hobbes quotes. He was hooked.

Tallinn soon discovered that the essay’s author, self-taught theorist Eliezer Yudkowsky, had written more than 1,000 articles and blog posts, many of them devoted to superintelligence. Tallinn wrote a program to scrape Yudkowsky’s writings from the internet, order them chronologically, and format them for his iPhone. Then he spent the ­better part of a year reading them.

---

quote:

---

After finishing the last of the essays, Tallinn shot off an email to Yudkowsky—all lowercase, as is his style. “i’m jaan, one of the founding engineers of skype,” he wrote. Eventually he got to the point: “i do agree that…preparing for the event of general AI surpassing human intelligence is one of the top tasks for humanity.” He wanted to help. When he flew to the Bay Area for other meetings soon after, he met Yudkowsky at a Panera Bread in Millbrae, California, near where he lives. Their get-together stretched to four hours. “He actually, genuinely understood the underlying concepts and the details,” Yudkowsky recalls. “This is very rare.” Afterward, Tallinn wrote a check for $5,000 to the Singularity Institute for Artificial Intelligence, the nonprofit where Yudkowsky was a research fellow. (The organization changed its name to Machine Intelligence Research Institute, or MIRI, in 2013.) Tallinn has since given it more than $600,000.

The encounter with Yudkowsky brought Tallinn purpose, sending him on a mission to save us from our own creations. As he connected on the issue with other theorists and computer scientists, he embarked on a life of travel, giving talks around the world on the threat posed by superintelligence. Mostly, though, he began funding research into methods that might give humanity a way out: so-called friendly AI. That doesn’t mean a machine or agent is particularly skilled at chatting about the weather, or that it remembers the names of your kids—though super-intelligent AI might be able to do both of those things. It doesn’t mean it is motivated by altruism or love. A common fallacy is assuming that AI has human urges and values. “Friendly” means something much more fundamental: that the machines of ­tomorrow will not wipe us out in their quest to attain their goals.

---