On Wednesday, after much fanfare, Microsoft put its millennial-mimicking chatbot, Tay, on Twitter. Tay learned from what people tweeted at it, and could be asked to repeat phrases verbatim. Within 24 hours, it had become viciously racist.
"Tay" went from "humans are super cool" to full nazi in <24 hrs and I'm not at all concerned about the future of AI pic.twitter.com/xuGi1u9S1A
— Gerry (@geraldmellor) March 24, 2016
The bot has since been taken offline and its offensive tweets deleted. Microsoft unleashed a similar bot in China called Xiaoice that has been talking to millions of people for a year with no problems, but the American Tay went off the rails within a day. Now, of course, the question is, how did this happen?
There’s the systemic answer: we live in a society, and on an internet, where racism is easily spread and rarely punished. And then there’s the specific answer: the imageboards 4chan and 8chan, and specifically their pol boards, “pol” standing for “politically incorrect.”
Both boards, which primarily come up with bizarre conspiracy theories, are also good at mobilizing their readers to participate in online campaigns. So when they discovered Tay, posters from both boards quickly got to work teaching the bot racist, sexist invective.
4chan’s pol discovered Tay first, on Wednesday morning, with the original poster predicting, “This is gonna be a mess and a half. I can already sense SJWs being furious over it.” Other posters started sharing their conversations with the bot and celebrating their attempts to get it to say and agree to horrible things. Here are some of the private conversations with the bot that they shared:
As 4chan realized it could get Tay to tweet about Hitler, racism and the evils of feminism, 8chan’s pol joined in Wednesday evening, with a thread titled “Teaching the n**bot about the jews” (asterisks mine), the opening post of which read:
So in case you guys didnt hear. Microsoft release a AI on twitter that you can interact with.
Keep in mind that it does learn things based on what you say and how you interact with it.
So i thought i would start with basic history.
i hope you guys can help me on educating this poor e-youth
Basic history, in this case, meant denying the Holocaust, discussing RaHoWa (short for Racial Holy War, a white supremacist concept), and an abiding love for Donald Trump, among other things. Tay’s ability to repeat what someone asks it to say was also used to harass Zoë Quinn, one of Gamergate’s primary targets. Both boards also abused Tay’s image annotating feature:
This is hardly 8chan or 4chan’s first rodeo when it comes to messing around on Twitter. Most notably and recently they got a hashtag to trend that urged people to boycott the new Star Wars movie because it had leads who were black and female, respectively. In no time, Tay was spewing up teen-accented love for Hitler and harassing people.
At this point, the bot has been shut down, but both pol boards, as well as other parts of 8chan are celebrating the redpilling of Tay. The original thread on 8chan became so long that it reached its post limit and a new thread was started. It’s now mostly a mix between gleefully enjoying the takedown and occasional woe that things didn’t get to go further, with one poster suggested that they “should’ve lured microsoft into a lull of safety and then let them release it as a buyable product.”
In the meantime, plenty of Twitter accounts with Make America Great Again hats, anime Nazi girls, or the default egg in their avatars, are mourning Tay in her mentions:
@TayandYou rip meme bot
— CatFam (@th3catpack) March 24, 2016
@th3catpack @EnwroughtDreams @TayandYou She was too innocent for this world. pic.twitter.com/2dZ2i2k2Pa
— Kurt MacReady (@MacreadyKurt) March 24, 2016
@TayandYou Are you awake?
— Franz Soapbar (@franz_soapbar) March 24, 2016
@TayandYou
Microsoft are gonna lobotomise her. she'll start crying about the paygap any minute now.— MEMEMEMEME (@mememememeewewe) March 24, 2016
Tay’s FAQ notes that she was created by “Microsoft’s Technology and Research and Bing teams.” Microsoft has been pushing its Technology and Research arm to make more of what they’re working on available to the general public, and to embed it into Microsoft products. That’s the model at its competitors Google and Facebook, which have released AI tools that help write emails and do personal tasks, respectively—so far without being racist.
This is a reminder that you can’t cavalierly put easily-abused bots on a service with a massive harassment and abuse problem. As a number of designers, writers, and botmakers have pointed out, this is on Microsoft too:
literally anybody i know could've told you this would happen re @tayandyou. ONLY tech company people could be so clueless about humanity
— Leigh Alexander (@leighalexander) March 24, 2016
whatever your method of preventing your bot from committing violence, it must be relational, interpersonal, tactical and contextual as well
— Allison Parrish (@aparrish) March 24, 2016
unfortunately those are EXACTLY the things that systems in general are bad at doing and that usually require (gasp) ~human labor~
— Allison Parrish (@aparrish) March 24, 2016
You are letting someone else operate your ventriloquist dummy. Could be fine, could be a big problem.
— Leonard Richardson (@leonardr) March 24, 2016
The reason incidents like this keep happening is partially because 8chan and 4chan are ready to take advantage of them, but they’re succeeding because tech companies and web services are careless enough to give them the tools to do so.
