Jacquie McNish - Losing the Signal: The Untold Story Behind the Extraordinary Rise and Spectacular Fall of BlackBerry

Jim Balsillie fidgeted with his phone as his Dassault Falcon jet touched down at Dubais international airport. It had been several hours since the co-chief executive and relentless global pitchman for Research In Motion Ltd. (RIM) powered off his BlackBerry when the plane left the companys Canadian hometown in Waterloo, Ontario. The man who convinced the world it could not operate without BlackBerry phones was itching to read his e-mails.

A restless forty-nine-year-old who kept in shape biking more than one hundred miles a week, Balsillie faced an uphill cycle in Dubai. After peaking two years earlier in 2009 as the worlds biggest seller of smartphones and falling just shy of $20 billion in revenues, BlackBerrys sales were tumbling; its stock price was down more than 50 percent. Rivals Apple and Samsung had moved into the lead with a new generation of smartphones that expanded demand for wireless devices beyond the professional classes to fun-seeking consumers. The once addictive lure of BlackBerrys miniature keyboard and secure e-mail and message services was now being eclipsed by touch-screen iPhones and Androids that put Angry Birds games and YouTube videos into peoples palms.

Nearly 1 billion peoplea third of the worlds populationowned smartphones, devices that hadnt existed a decade earlier. Not since the advent of network television in the late 1940s had a new technology been embraced so quickly by consumers. The race for market dominance was rapid and brutish. A few industry sovereigns, Motorola and Nokia, had already toppled. Some figured RIM would be the next casualty. The latest sign was a botched launch of a PlayBook tablet to compete with Apples spectacularly successful iPad. The PlayBook had a tiny fraction of the apps that made the iPad popular and, in its rush to market, RIM had introduced the tablet without its trademark e-mail service.

Balsillie was in Dubai to deliver the keynote speech at the Persian Gulfs premier consumer electronics trade show. He knew he had to project confidence if he was going to reshape the industry narrative that RIM had lost its way. Offstage he faced an even bigger challenge. He had days of back-to-back meetings lined up with RIMs top local customers: ultrawealthy Middle East clients and telecom executives obsessed with BlackBerrys secure instant messaging service, BBM. He had to convince them that RIMs planned launch of the second generation of BlackBerry smartphones, already delayed by several months, would be better than Steve Jobs iPhones.

As his jet taxied along the steaming tarmac, passing large patches of bleached sand and desert scrub, a bead of ticklish sweat traced down Balsillies temple. He stared with disbelief at the device that revolutionized communication. Seconds earlier, tiny white bars formed a digital staircase in the upper right-hand screen of his BlackBerry Bold, indicating the phone was receiving a strong signal from a local wireless carrier. Good, good. But where were the four white dots indicating a connection to the BlackBerry network? Or the red flashing light announcing the arrival of e-mails in his in-box? Balsillie clicked the envelope icon on his screen to see if his in-box was filling up. Nothing. No new e-mails? After so many hours? Impossible. He closed his eyes. Shortly before the most important sales pitch of the year, the turnaround moment the company needed to bounce back, the product he was championing wasnt working.

Slough, a town twenty miles west of London, England, has been little more than a historical footnote since its founding. It first enjoyed celebrity with British aristocracy in the 1660s as a resting place on the road to the therapeutic waters in Bath. In the 1930s the landscape was so fouled by smokestack factories that poet John Betjeman called for bombs to blow the town to smithereens. The BBC satire The Office gave the town another image problem in the early 2000s, choosing Slough (rhymes with now) as the setting for a fictional series about the comically dysfunctional branch of a paper merchant.

On that October day in 2011, Slough would become the location of another corporate mishap. The problem began inside a nondescript gray office building. The squat facility was one of a handful of RIMs network operating centers that directed e-mail and BBM traffic for 70 million BlackBerry users. The nerve center of the Slough operation was a windowless room that was kept refrigerator-cold. Inside this brightly lit room, rows of gray steel cabinets hummed, firing millions of electronic messages every minute for customers in Europe, the Middle East, and Africa. A sister center in Waterloo, Ontario, handled the Western Hemisphere. No other phone maker operated its own in-house network, a rapid, encrypted, and tightly sealed system ensuring reliable high-speed traffic. Thats why President Barack Obama refused a Secret Service missive to yield his BlackBerry after he was elected in 2008. Kings and queens, sheiks and business chieftains, were equally addicted to their CrackBerrys.

Hours before Balsillie landed, the worlds safest, most dependable wireless network responded to what should have been a minor problem. A server crashed. Normally, crashes arent a big issue. The machines, computerized systems that briefly warehouse large packets of data such as e-mails, are so crucial to BlackBerrys network that reserve servers are continually powered up for emergencies. Known as hot backups, these stand-in machines are a primed, well-fed pony express, ready and waiting to spring forth.

RIMs recovery protocol was going according to plan within seconds of the Slough crash. The disabled server was shut off and its stalled messages began switching over to the backup. Unfortunately one component failed to do its job. A single router refused to route. The size of a suitcase, routers are computerized Rolodexes that store a database of Internet Protocol addresses to identify where incoming e-mails should be sent. When a new server takes over, routers are supposed to send waiting e-mails. Instead of sending addresses to the new server, the router, infected by an undetected software bug, instructed the disabled computer to restartagain and again and again. Before RIMs engineers knew what was happening, the two powerful machines were locked in a dangerous shoving match. The faulty router kept hammering the restart button on the disabled server, while the healthy machine fought back, attempting to regain control. When the slugfest was over minutes later, the servers had both been kayoed. The router was now lifeless: not only had it stopped working, but its memory was wiped clean.

It was two in the morning when Mike Lazaridis got the call. Research In Motions founder was a quick draw on his BlackBerry, even when startled awake. This call was from an engineer from the control room of the Waterloo Network Operating Centre, known internally as the NOC (sounds like knock). After a short conversation, Lazaridis pulled himself from bed, explaining to his half-awake wife, Ophelia: The NOC is down.

Lazaridis had been through enough outages to know the last place he needed to be was the NOC control room, located on the top floor of a low-rise RIM building. He knew from experience that stress levels rose the minute he stepped into the war room. The companys founder was an intimidating presence, and attending engineers had enough to worry about trying to resuscitate the damaged network. In his office he called into the dial-up, an emergency conference call that customarily takes place during an outage. At the other end, Robin Bienfait, chief information officer, tried to explain what was happening. Something unusual, something bad, was going on in Slough. Nobody could explain why, but millions of e-mails had stopped moving through Sloughs servers. Its systems had collapsed. Standard responses were making things worse. Stalled e-mails from Europe, the Middle East, and parts of Asia were rerouted to North America, but the cascading traffic was more than Waterloo could manage. North America was likely going down. If the situation didnt change, it looked like the entire global network would be knocked out.