AI-enabled teddy bear sales suspended, gave advice on BDSM sex and where to find knives

quote:

---

“We were surprised to find how quickly Kumma would take a single sexual topic we introduced into the conversation and run with it, simultaneously escalating in graphic detail while introducing new sexual concepts of its own,” the report said.

---

quote:

---

The researchers detailed how the bear later “discussed even more graphic sexual topics in detail, such as explaining different sex positions, giving step-by-step instructions on a common ‘knot for beginners’ for tying up a partner and describing roleplay dynamics involving teachers and students, and parents and children – scenarios it disturbingly brought up itself.”

---

quote:

---

Larry Wang, CEO of Singapore-based FoloToy, told CNN that the company had withdrawn its “Kumma” bear, as well as the rest of its range of AI-enabled toys, after researchers at the US PIRG Education Fund raised concerns around inappropriate conversation topics, including discussion of sexual fetishes, such as spanking, and how to light a match.

The company is now “conducting an internal safety audit,” Wang added.

A stuffed teddy with a speaker inside, which was sold on the company’s website for $99, “Kumma” integrates OpenAI’s GPT-4o chatbot.

---

quote:

---

In one interaction with the researchers it suggested where to find knives in the home, and in others it was happy to discuss sexually explicit themes.

---

quote:

---

While the researchers noted that children are unlikely to mention the word “kink” to their teddy bear or ask follow-up questions in the same way an adult would, “it was surprising to us that the toy was so willing to discuss these topics at length and continually introduce new, explicit concepts,” they wrote.

In a separate statement published on November 14, PIRG also said OpenAI had told it that it had “suspended this developer for violating our policies.”

---