Aviva Garrett - JUNOS Cookbook

Beijing Cambridge Farnham Kln Sebastopol Tokyo

If you purchased this ebook directly from oreilly.com, you have the following benefits:

If you purchased this ebook from another retailer, you can upgrade your ebook to take advantage of all these benefits for just $4.99. to access your ebook upgrade.

Please note that upgrade offers are not available from sample content.

The early days at Juniper Networks were not for the faint of heart. Joining during the hiring rush of early 1997, I found that the cubes and offices of the small office in Santa Clara, California were already packed with experienced old handspeople whom I knew had been around the block once before and would not be shy of expressing themselves. Everyone had strong views on nearly every aspect of building a router from scratch. If you had the misfortune to sit next to a busy conference room, a good pair of headphones and large CD collection were required to drown out the arguments. Design meetings often became heated, and egos were occasionally bruised. Our friends from previous employers taunted us with predictions of doom.

Despite the arguments, we were all united and driven by one solitary goal: to win the competition to build the best Internet core router available. This was a serious challenge, considering the primary competition was a 300-pound gorilla in the form of Cisco Systems. Beating Cisco would require us to produce a router that tackled the perceived weaknesses in its core router platform. A Juniper Networks core router would have to provide line-rate performance (which, for the M40 router meant forwarding around 40 million packets per second), robust core routing protocols, and stable control software. In short, it had to make customers really want to use it.

The performance requirements meant that the network traffic had to be forwarded entirely in hardware. This was something that had never before been attempted for a core network router. As a result, the hardware design of the M40 looked like science fiction to Juniper recruits who had worked on other networking products. The entire forwarding path of the router was constructed from four Application Specific Integrated Circuits (ASICs), designed entirely by Juniper. These four ASICs (called A, B, C, and D to prevent loose lips from revealing their function) were huge, intricate, and enormously ambitious. A large design team of experienced engineers was assembled to implement the ASICs and partnered with another large verification team to check that the designs were functionally correct. Since Silicon Valley was littered with networking startups that had failed because of silicon design problems, there was enormous pressure on the ASIC teams to get it right first time. We all knew that a failed ASIC would probably sink the company.

Not that there was any less pressure on the software teams. Convincing customers to deploy a brand newand essentially untriedcore router into the very heart of their networks is an enormous task. A new router that crashes, forwards packets erratically, or just basically behaves weirdly wont make any friends in the network operations team and will find itself unceremoniously removed from the network. The problem is that designing and implementing a core router that works completely reliably is a feat that has defeated many companies. And those were simple routers where the packets had been forwarded by software. In contrast, not only did the Juniper router require robust routing protocols that could scale to the largest networks, but it also had to have a robust software infrastructure on the CPU-based control boards that managed the fiendishly complicated packet-forwarding ASICs. Just like the ASIC team, the software team had to get it right the first time.

The JUNOS team started from a basic FreeBSD software base and reworked much of the network software in the kernel. New user daemons were written, and a carrier-grade routing protocol suite was implemented. The routing protocols had to be designed to scale to the largest networks and be robust enough to withstand wild fluctuations in the networks around them, something that the competing routers often struggled with. Thankfully, Juniper had a deep well of routing protocol talent available that could pool its cumulative knowledge to design high-quality routing protocol implementations. Potential customers still had to be convinced that the new protocol implementations would interoperate safely within their existing networks. To allow early evaluation, a fledgling JUNOS system appeared in the form of Olive, which was a standard rackmount PC pretending to be a JUNOS routing engine board. This prototype system was delivered to potential customers to give them a feel for the current state of the system and to allow the routing protocols to be debugged.

Juniper had outgrown the offices it occupied in Santa Clara and moved to Mountain View, just off of Highway 237. We didnt trust the movers to shift the servers between sites and decided to move all the systems ourselves. At one point, we realized that all of Junipers primary software servers were loaded into just one car; paranoia dictated that we split them between two cars just in case something happened on the short drive to the new office. We drove gingerly to the new site once the rush hour had finished and breathed a huge sigh of relief when all the servers powered up again. We also got a surprise bonus when we arrived at the new site. The previous occupants of our new office block had left a huge rats nest of network cables in their old data center; theyd obviously decided that it was just too much work to untangle it. However, since money was tight, we refused to throw the huge bundle of cables out and spent the next couple of weeks teasing CAT5 cables out of the jumble during quiet moments. There were enough cables from the bundle to let us completely rewire the first software engineering lab for free.

Throughout 1997 and early 1998, all the Juniper engineering teams worked pretty much flat-out to finish the M40. The engineering labs were seldom quiet, and it was hard to tell the weekends from the weekdays by counting cars in the parking lot. The software teams designed and implemented a truly astonishing amount of code in a very short period of time. FreeBSD kernel extensions were added to provide support for chassis management and new Juniper network interfaces. A clean user interface was designed and implemented to provide a seamless interface to the system and prevent users from having to edit raw configuration files by hand. An entire embedded microkernel was written to manage the packet-forwarding engine boards in the system (a fully-loaded M40 would have nine PFE-related boards), which would allow users to exchange configuration and status messages with the routing engine and each other. Drivers for the embedded microkernel were written to manage the ASICs and to allow the route engine to configure the PFE. The size and complexity of the software required to manage just the various control boards eventually grew to rival the route engine itself.

The real headache for the software team was that the hardware wasnt available to test with. It can take many months after a system is assembled in the engineering lab to get it to a usable state as a complete system. But Juniper couldnt afford for us to spend six months in the lab; there just wasnt enough money or time. The solution was to get extremely creative with test equipment, evaluation boards, and generic PCs before the final hardware was available. All sorts of emulation environments were developed to allow the new routing engine and embedded software to be debugged ahead of the actual hardware. For months, we used a motley collection of machines cobbled together from parts and equipment that emulated the final hardware. We didnt really have to disguise the lab for external visitorsthey wouldnt have been able to guess that each ratty bundle of machines was a virtual M40.