Foreword
Like many others, I am a firm believer in using tools to expand our understanding of how systems really work. In fact, I began my career as a performance tools developer. My boss at that time had many simple sayings; among them was one of my favorites, Our team is only as good as our people and our people are only as good as our tools. As a manager, Ive made it a priority to ensure we dedicate many of our top engineers to tools development and I have always encouraged using tools to teach and grow engineers.
Teaching books such as Tariks are meant to help improve the productivity, understanding, and confidence of others. Being an individual who has had both a lifelong passion for tools and learning, as well as someone who has spent more than a decade and a half working in Windows, it is an honor and a pleasure for me to write the foreword to Tariks insightful book.
For decades, the Windows team has worked tirelessly to improve the core capabilities of the platform and to make it more suitable for increasingly diverse hardware configurations and software stacks. This hard work has paid off; Windows today is the preeminent platform for developers, consumers, and businesses across the globe. With more than 1 billion PCs and customers, Windows is both the market-leading server and client computing platform. Variants of Windows run on small form-factor mobile devices, within embedded intelligent systems, on uniform and non-uniform (NUMA) memory architectures, on the XBOX gaming console, and on single CPU systems as well as those with hundreds of processors. Windows runs in clustered configurations with failover capabilities and sits atop hypervisors and Virtual Machines. And of course, Windows is in the cloud.
Enormous platform success and diversity can be accompanied by an equally enormous and diverse set of technical challenges. Thousands of engineers at Microsoft, and tens to hundreds of thousands outside of Microsoft, are involved in building, debugging, and troubleshooting a multitude of diverse configurations and solutions. To understand issues in a vastly diverse problem space, foundational tools, techniques, and concepts are essential.
To that end, Tarik has done an excellent job in detailing how a set of key foundational Windows tools work. In detailing these tools, Tarik succeeds in expanding the readers awareness of key operating system concepts, architectures, strengths, and limitations. With the concepts understood and the tools mastered, readers can expect to be able to tackle all types of performance and debugging challenges. I believe Tariks book will become a staple for a broad set of individuals, from novices to experts.
Michael Fortin, Ph.D.
Distinguished Engineer, Windows Fundamentals
Microsoft Corporation
Introduction
One exciting aspect of software programming is that there are usually many ways to accomplish the same goal. Unfortunately, this also presents software engineers with unique challenges when trying to make the best design or implementation choice for each situation. Experience plays a major role, and the learning process is often progressive as one learns to avoid past mistakes. Sadly, though, experience is often a variable concept. I have met several software engineers who, after spending a very long time working on an area, still lacked a basic understanding of how it really worked beyond the repetitive day-to-day tasks they grew accustomed to. I have also met others who have perfected their craft in a field after only a few years of working experience.
This book introduces a few techniques for methodically approaching software development problems primarily using the two Swiss Army knives of expert Microsoft Windows developersnamely, the Windows debuggers (WinDbg) and the Windows Performance Toolkit (Xperf). By focusing on why features and components work the way they do in the system rather than simply on how they work or what they do, this book tries to accelerate the process of learning by experience and to minimize the number of mistakes made when approaching new problems. An important part of the process is learning to compare and contrast with known solutions to existing ones.
Software engineering is still inherently a practical science. (Some might even argue its an art rather than a science.) While there is certainly no substitute for real experience, the topic can definitely be approached with the same methodical persistence that works so well for other scientific disciplines. In fact, this approach works even better in software engineering because all behaviors can be explained rationally. After all, it is all just codewhether its your own code or code written by others that you end up consuming in your softwareand code can always be traced and understood.
Although this book deals with several architectural pillars of the Windows operating system as part of its debugging and tracing experiments, my main goal in writing it is less about covering those details and more about encouraging and developing this critical mindset. I hope to demonstrate how this approach can be used for solving a few interesting problems as part of this book, and that you continue to systematically apply debugging and tracing as you expand your learning beyond the topics directly covered here.
This book is not really about teaching native or managed code programming, either, although youll find several good coding examples in the companion source code. Because it takes an inside-out look at how to explore the system using debugging and tracing tools, this book will probably appeal more to software engineers with the desire to understand system internals rather than those with a need to quickly learn how to make use of a specific technology. However, I believe the approach and mindset it aspires to inculcate are applicable regardless of the technology or level of expertise. In fact, contrary to what many think, the higher the level of technology involved, the harder it becomes to grasp what goes on behind the scenes and the more expertise is needed in order to investigate failures when things inevitably go awry and require debugging skills to save the day. In pure C, for example, a call to malloc is just that: a function call. In C++, a call to the new keyword is emitted by the compiler as a call to the new operator function to allocate memory for the object, followed by code to construct the object (again, emitted by the compiler to possibly initialize a virtual pointer and invoke the constructors of the base classes, construct member data objects, and finally invoke the user-provided constructor code for the target leaf class). In C# (.NET), things get even more involved, because a one-line call to the new keyword might involve compiling new code at runtime, performing security checks by the .NET execution engine, loading the modules where the target type is defined, tracking the object reference for later garbage collection, and so on.
Who Should Read This Book
This book is aimed at software engineers who desire to take their game to the next level, so to speak, and perfect their mastery of Windows as a development platform through the use of debugging and tracing tools.
Assumptions
Readers should have basic familiarity with the C/C++ and C# programming languages. A basic knowledge of the Win32 and .NET platforms is helpful, but not strictly required, because this book makes every effort to introduce basic concepts before expanding into more advanced topics.
Organization of This Book