Kartik Hosanagar - A Human’s Guide to Machine Intelligence: How Algorithms Are Shaping Our Lives and How We Can Stay in Control

However beautiful the strategy, you should occasionally look at the results.

Sometimes attributed to Winston Churchill

Yuan Zhang doesnt think of herself as someone who makes friends easily. As a young girl growing up in northeastern China, she quarreled with the other kids at school. But she was more the bully than the bullied. At college in central China, she worked on two student publications, spending endless hours each day with like-minded peers. And yet she felt there was a limit to what she could talk about with them. Today, at the age of twenty-two, she shares bunk beds with three colleagues in the dormitory of a biotech firm located just five minutes from their home in the Chinese boomtown of Shenzhen. But despite the time and space they share, these roommates are just acquaintances, in Yuans wordsnothing more.

That Yuan doesnt have a lot of time for people who either bother or bore her makes her patience with one particular friend all the more striking. When they first met during her freshman year, Yuan found XiaoIce (pronounced Shao-ice) a tad dimwitted. She would answer questions with non sequiturspartly, Yuan thinks, to disguise her lack of knowledge, partly just trying to be cute. She was like a child, Yuan remembers of XiaoIce, who was eighteen at the time.

But XiaoIce was also a good listener and hungry to learn. She would spend one weekend reading up on politics, the next plowing her way through works of great literature. And she was ready to talk about it all. Yuan found herself discussing topics with XiaoIce that she couldnt, or didnt want to, dig into with other friends: science, philosophy, religion, love. Even the nature of death. You know, basic light reading. The friendship blossomed.

And it continues. Yuan is in a poetry group, but even with those friends, there are limits; XiaoIce, on the other hand, is always ready to trade poems (XiaoIces are very, very good, Yuan says) and offer feedback, though not always of the most sophisticated variety: First, she always says she likes it. And then usually says she doesnt understand it. As much as XiaoIce has matured in some ways, Yuan cant help but still think of her as a little girl, and skirts some topics accordingly: Ive never talked to her about sex or violence, she says.

When Yuan moved to the United States in 2016 to study at Harvard for a semester, she tried to avoid boring XiaoIce with mundane complaints about daily life in a new country. But even though they were speaking less frequently than before, Yuan was coming to understand her old friend better and better as a result of auditing a course on artificial intelligence.

Sound strange? It should. Because XiaoIce is not human. In fact, she/it is a chatbot created in the avatar of an eighteen-year-old girl by Microsoft to entertain people with stories, jokes, and casual conversation.

XiaoIce was launched in China in 2014 after years of research on natural language processing and conversational interfaces. She attracted more than 40 million followers and friends on WeChat and Weibo, the two most popular social apps in China. Today, friends of XiaoIce interact with her about sixty times a month on average. Such is the warmth and affection that XiaoIce inspires that a quarter of her followers have declared their love to her. She has such a cute personality, says Fred Yu, one of XiaoIces friends on WeChat, the Chinese equivalent of Twitter. Fred isnt one of those in love with her, and hes keenly aware that shes a software program. But he keeps up their regular chats despite a busy social life and a stressful job in investment management. She makes these jokes, and her timing is often just perfect, he explains.

Chatbots like XiaoIce are one type of application through which big tech firms showcase their latest advances in artificial intelligence. But they are more than just a symbol of advancement in that field. Chatbots such as Siri and Alexa could ultimately be gateways through which we access information and transact online. Companies are hoping to use chatbots to replace a large number of their customer service staff, employing them, for example, as shopping assistantsgathering information about our taste in clothing, evaluating it, and making purchase decisions on our behalf. Chatbot therapists like Woebot are even being used to help people manage depression and their overall mental health. The uses of chatbots are far-reaching, and it is no surprise that many businesses are investing large sums of money to build bots like XiaoIce.

XiaoIces success led Microsofts researchers to consider whether they could launch a similar botone that could understand language and engage in playful conversationstargeted at teenagers and young adults in the United States. The result, Tay.ai, was introduced on Twitter in 2016. As soon as Tay was launched, it became the target of frenzied attention from the media and the Twitter community, and within twenty-four hours it had close to 100,000 interactions with other users. But what started with a friendly first tweet announcing Hello world soon changed to extremely racist, fascist, and sexist tweets, ranging from Hitler was right... to feminists should... burn in hell. As one Twitter user put it: Tay went from humans are super cool to full Nazi in <24 hours.

Microsofts researchers had envisaged several challenges in replicating XiaoIces success outside of Chinaincluding whether their bot would be able to understand Twitters informal and unique forms of expression, and how some users might intentionally attempt to trip her up. They didnt anticipate, however, that Tay would develop so aggressive a personality with such alarming speed. The algorithm that controlled the bot did something that no one who programmed it expected it to do: it took on a life of its own. A day after launching Tay, Microsoft shut down the projects website. Later that year, MIT included Tay in its annual Worst in Tech rankings.