Blog
21 April, 2024
No matter how much has happened in the world, I love that it is still possible to be surprised. One event that consistently surprises – and delights – is VMware Explore 2022. This conference brought together a myriad of creative and IT-focused people under one roof, showcased the latest enhancements and features of VMware, encouraged the interest of specialists, and connected like-minded professionals from across Europe.
Although a number of VMware innovations were presented during the conference, today I will focus on one that left a huge impression: vSAN 8. vSAN is an enterprise-class storage virtualization software that provides the easiest path to HCI and hybrid cloud.
This year US VMware Explore will go down in the history books – it is the day the new vSAN 8 was released to the world and is now available to all users. The improvements made in this upgrade are incredibly beneficial, and I believe it will be hard to overstate their impact on society.
To better delve into the innovations presented at this conference, take a step back and understand how technology, industry, and customer needs have changed. Let us first look at the state of the industry, i.e., how it all looked ten years ago when vSAN was first introduced.
Spinning disks have been the preferred media type for shared storage in hyper-converged architectures this year. Although NAND Flash devices existed, they could not compete due to limited capacity and performance due to old BUS protocols. Hyperconverged infrastructures used networks that typically used only one to ten gigabit Ethernet in the underlying access architecture. As a result, they were used for traditional VM workloads with low resources and were mostly limited to on-premises data centers.
But today’s hardware looks completely different. The capacity of NAND Flash devices has increased significantly, and the devices themselves have become much more affordable. Especially with the introduction of the new QLC Flash. These devices are often used on the high-performance NVMe bus protocol to bring new capabilities.
25, 40, or even 100-gigabit connection technology is increasingly used in modern networks. In addition, new high-performance protocols like RDMA over Converged Ethernet or others were introduced.
Workloads in these environments typically consist of complex, multi-tiered applications powered by resource-intensive VMs and containers. In addition, these environments often use data services such as encryption and space-efficiency features to meet ever-increasing levels of compliance and value. Finally, today’s complex workloads can be found running beyond the core on-premises data center but also at the edge and in the cloud through services like VMware Cloud on AWS, Azure VMware Solutions, and others.
The evolution of the data center has changed how we think about technology and how it affects customers. This is why the world has developed various solutions that help customers manage their data and ensure that they are getting the most out of their investments. As the industry continues to evolve, VMware is trying to answer the question of how to provide customers with the best possible experience while also introducing new features and capabilities.
The new version of vSAN is called vSAN 8. In addition, it introduces the Express Storage Architecture (ESA), an optional alternative storage architecture, to the existing vSAN Original Storage Architecture (OSA). When used with certified hardware vSAN-ReadyNodes, the vSAN Express Storage Architecture (ESA) unlocks maximum performance, scalability, resiliency, and modern data services with virtually no impact on performance. The ESA opens today’s hardware capabilities for today’s and tomorrow’s workloads.
The main difference between the vSAN Express and the Classic Storage Architecture is that the former is a two-tier architecture designed to accommodate various storage devices. The later version (ESA) is a single-tier architecture ideal for high-performance NVMe-based TLC Flash drives. With the release of vSAN 8, you can choose the architecture that best fits your current hardware while also allowing you to deploy the new high-performance clusters.
The newly accommodated ESA clusters can co-exist with other clusters running older versions of vSAN or those upgraded to vSAN 8 with the Original Storage Architecture.
vCenter Server can manage clusters of different versions, so you can upgrade your environment at your own pace. However, VMware encourages all of its customers to upgrade to vSAN 8, even if the Original Storage Architecture remains, this upgrade will provide vSAN enhancements to the Original Storage Architecture.
Original Storage Architecture (OSA) is still very important, can be used next to ESA, and will continue to be supported for many years.
vSAN ESA provides an efficient process and store data in vSAN. This architecture also provides a new snapshot engine that meets high-performance levels. In addition, the vSAN Express Storage Architecture simplifies administration.
vSAN ESA introduces two components:
• LFS – The new proprietary log-structured file system. This new layer is an innovative way to obtain incoming rights quickly but with minimal overhead. It helps prepare the data and any subsequent changes to the data for efficient storage on flash devices while minimizing compute and network resources.
• Optimized Data Structure. It allows vSAN to store data and associated metadata extremely quickly and efficiently, which becomes especially important as the density of flash devices increases.
The combination of these two components gives vSAN the ability to efficiently store data using a single tier of storage devices.
Let’s talk about Log Structure File System (LFS), and how it works. The vSAN LFS developed to allow the system to write many small I/O’s quickly, combine them into large blocks and effectively write them. Incoming IO is compressed, combined, and packed with related metadata and written to a persistent, durable log, which resides on the object data structure called Performance Leg. This allows vSAN immediately acknowledge writes to guest VMs to keep latency as low as possible. As data accumulate in that log, the previous log entries will be written to the object data structure known as Capacity Leg. As a result, IOs will be aligned for full stripe write using raid-6 erasure coding. At this point, this is the most efficient way to write data. Meanwhile, the metadata is sent to the metadata log for additional time, where it will be possible to access it very efficiently. However, eventually, when the metadata log is completely filled, the metadata will be stored on the Capacity Legs of the object in an efficient B-Tree structure. Thanks to the new vSAN LFS feature, we can get raid-6 resiliency and space efficiency at raid-1 performance.
With the rise of NVMe devices, achieving the highest write performance required several significant improvements at the lowest level of the vSAN stack. As a result, VMware developed the new Log Structured Object Manager (LSOM). This feature is built on a new high-performance IO engine that can handle extremely high write workloads, reducing the overhead resources required for metadata. While these improvements were critical in providing high efficiency to the upper layers of vSAN in sending data to devices without constraints, it should be recognized that performance and efficiency are only some of the benefits of LSOM. This new approach also completely removes the idea of disk groups used in the Original Storage Architecture (OSA) which eliminates a Failure Domain and allows storage devices to be serviced, added, or removed easily and efficiently.
The vSAN 8 ESA will distribute data in a slightly different way compared to the previous architecture. The new object format used by the ESA stores data in two legs: the Performance Leg and the Capacity Leg. These legs reside in the same object and interact with the vSAN LFS. In addition, it uses the same distributed object manager for storing data and metadata. This is done to ensure performance and capacity from the same tier.
In many ways, the vSAN 8 Express Storage Architecture is more than just individual improvements added to the vSAN layer. For example, VMware has moved some data services to the very top of the vSAN stack to reduce the resources required for specific tasks. From now on, the data is compressed and encrypted at the highest level, checksums are reused in an innovative way to reduce duplicate efforts, and finally, vSAN ESA uses a proprietary Log-Structured File System (LFS) by coalescing many small IOs and writes them as large IOs. These capabilities make the vSAN Express Storage Architecture unique.
Express Storage Architecture allows you to create a high-performance storage pool using a single tier of storage devices. All devices chosen for use with vSAN will have the capacity and performance of a vSAN cluster.
By using the Express Storage Architecture, VMware aimed to further simplify the maintenance and administration of the solution. There was a need to ensure that users could have an optimal level of resistance that was also space efficient while achieving maximum performance. Various architectural changes helped accomplish this. From now on, administrators can use raid-6 to increase resiliency and predictable levels of space efficiency without sacrificing performance.
When customers choose to implement a raid-5 policy, the ESA looks at the cluster’s host count to determine which of two raid-5 placement options should be used. For instance, a 4+1 data placement scheme distributes data with parity across 5 hosts and will be used in a cluster with 6 or more. A 2+1 data placement scheme distributes data with parity across 3 hosts and will be used in a cluster with 3 to 5 hosts. And if the number of hosts is reduced to below 6 for a period of time, the ESA will then automatically adjust to the 2+1 raid-5 placement scheme to maintain the cluster at least one spare host in case of subsequent failures. It’s important to remember that vSAN with ESA provides the same level of performance with erasure coding as a raid-1 mirroring policy, which by default provides the ability to store data at a more space-efficient level and a higher level of resiliency that adapts to the size of the cluster.
The new compression capabilities improved the data compression ratio as high as 8:1 for every 4k block written. This is an up to 4 times improvement from the Original Storage Architecture. The new compression feature in vSAN is moved to the upper layers of the platform. This feature can reduce the amount of CPU effort required to compress the data and also helps reduce the network traffic because replica traffic is always compressed. In addition, using storage policies, the new vSAN 8 Express Storage Architecture allows you to enable and disable application-level compression on a per-VM or per-VMDK basis.
While the Original Storage Architecture has data encryption functionality to encrypt the data in the vSAN cluster, the new ESA has an entirely new data encryption mechanism. Encryption remains a cluster service, but like the compression feature, encryption occurs in the upper layers of vSAN. This encryption method only happens once, and because it is high in the stack, all vSAN traffic passing through the hosts will also be encrypted. In this way, encryption requires significantly less overhead.
The vSAN ESA’s ability to process storage traffic at near-device-level rates makes it ideal for running high-performance applications. However, this high processing rate can lead to network bottlenecks when running complex workloads. So, with vSAN 8, the new adaptive traffic shaping capability was introduced. This helps ensure that vSAN appropriately prioritizes VM IO over resync activity if network contention occurs. This can provide more consistent performance for these demanding workloads that may otherwise saturate your network connection.
With the help of the Native Snapshot Engine, customers can create point-in-time data states with minimal performance degradation of the VM. The new native snapshot capabilities of vSAN8 ESA are integrated directly into vSphere.
In previous versions of vSAN, customers enrolled in the Customer Experience Improvement Program (CEIP) used vSAN health checks to proactively identify emerging issues with the platform. VMware engineers noticed that not all clients enrolled in CEIP. From now on, vSAN’s Proactive Insights capability allows customers who have a vCenter server connected to the Internet to get the latest and greatest health checks even though they are not enrolled in CEIP.
This new technology delivers performance without sacrificing efficiency and improving vSAN while maintaining the foundation of how the system works, opens new horizons for customers, and helps improve every level of resiliency, simplicity, and performance. I was thrilled to be a part of VMware Explore 2022 and to be among the first to be introduced to this significant leap forward for VMware.
Written by: Sirijus Dailidonis, Architect, Modern Infrastructure