Duncan Epping - VMware vSAN 7.0 U3 Deep Dive

Cormac Hogan is a Chief Technologist in the Office of the CTO in the Cloud Infrastructure Business Group at VMware, focusing predominantly on Kubernetes platforms running on vSphere. Cormac has previously held roles in VMwares Technical Marketing, Integration Engineering and Support organizations. Cormac is the owner of CormacHogan.com, a blog site dedicated to storage, virtualization and container orchestration. He can be followed on Twitter @CormacJHogan.

Duncan Epping is a Chief Technologist working for VMware in the Office of CTO of the Cloud Infrastructure Business Group. Duncan is responsible for ensuring VMwares future innovations align with essential customer needs, translating customer problems to opportunities and functioning as the global lead evangelist for Cloud Infrastructure. Duncan is the owner of VMware Virtualization blog Yellow-Bricks.com and has various books on the topic of VMware including the vSphere Clustering Deep Dive series. He can be followed on Twitter @DuncanYB.

When talking about virtualization and the underlying infrastructure that it runs on, one component that always comes up in conversation is storage. The reason for this is simple: In many environments, storage is a pain point. Although the storage landscape has changed with the introduction of flash technologies that mitigate many of the traditional storage issues, the pain of managing storage platforms has unfortunately not disappeared.

Storage challenges range from the operational effort, or complexity, to performance problems, or even availability constraints. The majority of these problems stem from the same fundamental problem: legacy architecture. The reason is that most storage platform architectures were developed long before virtualization existed, and virtualization changed the way these shared storage platforms are used.

In a way, you could say that virtualization forced the storage industry to look for new ways of building storage systems. Instead of having a single server connected to a single storage device (also known as a logical unit or LUN for short), virtualization typically entails having one (or many) physical server(s) running many virtual machines connecting to one or multiple storage devices. This did not only increase the load on these storage systems, but it also changed the workload patterns, increased the total capacity required, and added much complexity.

As you can imagine, for most storage administrators, this required a major shift in thinking. What should the size of my LUN be? What are my performance requirements, and how many spindles will be necessary? What kind of data services are required on these LUNs, and where will virtual machines be stored? Not only did it require a major shift in thinking, but it also required working in tandem with other IT teams. In the past server admins and network and storage admins could all live in their own isolated worlds, they now needed to communicate and work together to ensure availability of the platform they were building. Whereas in the past a mistake, such as a misconfiguration or under-provisioning, would only impact a single server, it could now impact many virtual machines.

There was a fundamental shift in how we collectively thought about how to operate and architect IT infrastructures when virtualization was introduced. Now another collective shift is happening all over again. This time it is due to the introduction of software-defined networking and software-defined storage. But lets not let history repeat itself, and lets avoid the mistakes we all made when virtualization first arrived. Lets all have frank and open discussions with our fellow datacenter administrators as we all aim to revolutionize datacenter architecture and operations!

This book is targeted at IT professionals who are involved in the care and feeding of a VMware vSphere environment. Ideally, you have been working with VMware vSphere for some time and perhaps you have attended an authorized course in vSphere, such as the Install, Configure, and Manage class. This book is not a starters guide, it is a deep dive, there should be enough in the book for administrators and architects of all levels.

We do want to point out, that for readability reasons we had to create screenshots of most CLI commands, this may result in different font sizes unfortunately as a result of how ebooks are structured. If we find a way around this then we will upload a new version over time.

It has been eight years since we first released vSAN in 2014. At the time, hyperconverged infrastructure (HCI) was a brand new concept surrounded by some amount of uncertainty and even doubt. The storage industry had always been dominated by purpose-built hardware and specialized appliances, but trends in flash as a new storage medium plus continued growth in computing and networking performance were opening the opportunity for new architectures and new business models to disrupt the industry.

Meanwhile, cloud was just taking off. Developers were eager for a better experience with more self-service and automation. As IT organizations looked to meet that demand and build their own cloud operations, software-defined became the next big idea, promising to replace traditional purpose-built appliances with software-based services. The goal was a cloud operating model in the data center, with lower costs thanks to scale out, x86 economics and reduced operational overhead. This in turn provided an opening for HCI, to be that architecture that delivered on the promise of software-defined and that delivered better economics, greater simplicity, and superior scalability.

Since that time, the category has seen explosive growth. vSAN has become the most widely deployed HCI software in the market with more than 30,000 customers. vSAN has become the building block for many customers private clouds with deployments ranging from a couple of servers to thousands of nodes spread across multiple sites and hosting petabytes of data. The workloads on vSAN themselves have matured to include business critical, high-performance databases and applications. In short, HCI has become mainstream. Along the way, customer requirements and expectations have expanded. HCI has evolved from hyperconverged infrastructure to hybrid cloud infrastructure with requirements for full-stack networking and management, multi-cloud deployments and support for modern application architectures.