Somebody asked me if VMFS3 multi-extents datastores are supported by Storage DRS. Although they are supported and fully operational in Storage DRS, one must ask if this construct of large datastores should be used in a datastore cluster.
Resource aggregation and flexibility
Storage DRS Datastore clusters offer flexibility in adding and removing datastores dynamically and allow the administrator to focus on macro management by reducing the number of entities to be managed.
By using datastore cluster, micro management of single datastores is something from the past, such as the tedious task of virtual machine placement. The administrator no longer needs to find a datastore that provide adequate space, while still ensuring that placement of the virtual machine will not result in an I/O bottleneck. Let alone monitoring the current workload next to the ever-expanding workload; application lifecycles are changing drastically and virtual machine server sprawl is still one of the top concerns of the modern administrator. Keeping track and managing such an environment is very challenging. By allowing Storage DRS to manage (initial) placement of virtual machines, the administrator only needs to monitor overall available space and IO performance of the datastore cluster itself.
If the cluster requires more space of more IO performance the administrator can dynamically add more datastores to the datastore cluster and allow Storage DRS to find an optimal distribution of the current workload. The option “Run Storage now” in the datastore cluster view allows the administrator to trigger a Storage DRS invocation immediately.
Using Storage DRS and particularly space load-balancing can reduce the need of multi-extents as well. By allowing Storage DRS to monitor space utilization, the free space used as a safety buffer can be greatly reduced. Each ESXi host reports the virtual machine space utilization and the datastore utilization; Storage DRS will trigger an invocation if the configured space utilization is violated. A common practice is to assign a big chunk of space as safety buffer to avoid out of space situation of a datastore, which might lead to downtime of the active virtual machines. I’ve seen organization using requirements of 30% free space on datastores. By reducing slack space, a higher consolidation ratio can be achieved (if IO performance allows this), or a reduction in LUN sizes. Reducing LUN sizes can be used to provision additional datastores to the datastore cluster. More datastores benefits Storage DRS by offering more load balancing options, more datastores increase the number of queues, which benefits IO management at ESXi level and at SIOC at cluster level. Essentially this configuration is the complete opposite of VMFS extends. However if larger size datastores are necessary, vSphere 5 offers VMFS5.
VMFS5
VMFS5 allows datastores up to 64 terabyte of contiguous space. ESXi 5.0 allows a VMDK size up to 2 terabyte of space, providing sufficient space for most virtual machines configurations. If the virtual machine requires more than 32 virtual machine disks of 2 terabyte it’s recommended to disable the default affinity rule (keep all disks together) and allow Storage DRS to distribute the virtual machine disk files across all datastores inside the datastore cluster. This granularity allows Storage DRS to find a suitable datastore for each virtual disk that aligns with the performance requirements of that specific virtual disk.
vSphere 4.1 HA and DRS book for only $19.95
We lowered the price of the vSphere 4.1 HA and DRS technical Deepdive book permanently. As of this week you can obtain one of the coolest books in the virtualization section at Amazon for only $19.95. 30 5-star reviews couldn’t be wrong. Here is just a random selection of two of those 5-star reviews:
B. Riley: The term “deepdive” is regularly abused in the technology world these days. There’s nothing more disheartening than walking into a one hour session at a conference entitled deepdive, and finding out that it’s neither deep, nor a dive. It ends up being more like sitting in a couple inches of warm water in a plastic kiddie pool.
When these guys say deepdive, they mean it. This book is packed with helpful information from the first, to the last page. Somehow, they even manage to read minds. They know what you’re thinking as a VMware administrator, and they’ll tell you the why, and the best practice.
Lots of books have good overviews of HA and DRS, but none goes as deep as this. It’s very well-written, and highly recommended for anyone who is running, or thinking about running an HA/DRS environment.
This book is, as Jeremy Clarkson would say, “absolutely brilliant”!
Chris Dearden: Ever had a series of discombobulated thoughts and ideas that have suddenly clicked into place & the plans come into focus? That’s exactly what happened when I read Frank & Duncan’s book. Even though I have a fair few years experience with Enterprise virtualisation , my knowledge of what’s deeply under the covers of the availability options of vSphere was made up of blog posts I’d read , anecdotes from colleagues and a few slides from trainers. It was enough to get me by, but there was always that nagging feeling that I wasn’t fully in control of what was happening.
After reading the book ( in a morning – for a tech book it’s one that you can work though in a short amount of time and still get value from ) I had a real epiphany / light bulb moment / matrix moment / and all of those concepts and ideas suddenly had a deeper meaning and the big picture was visible. For anyone who thinks they know about HA / DRS : read this and *really* know about it.
Get your copy now at: vSphere 4.1 HA and DRS technical deepdive Amazon page
Impact of load balancing on datastore cluster configuration
This article is a part of the series on architecture and design on datastore clusters. This article zooms in on why it’s recommended to use similar type disks in a datastore cluster.
In-tier balancing solution
SDRS can be considered as an “in-tier” balancing solution, suggesting that a datastore cluster should be populated with datastores that provide similar performance, continuity, capacity or service level. Although it’s not a technical requirement to have similar configured datastores, using heterogeneous configurations in a datastore cluster can lead to unexpected results. Understanding the SDRS’ main goal and the load balancing process can assist you in architecting your datastore cluster.
SDRS load balancing goal
The main focus of SDRS is to correct imbalance from both a space utilization and latency perspective on the datastore level. SDRS determines the imbalance level (space or latency) of the datastore cluster and migrates one or multiple virtual machine disk to solve the imbalance.
In order to select an appropriate migration candidate (virtual machine) SDRS relies on device and workload modeling to understand the impact of a workload on the latency of the datastore, SDRS uses virtual machine statistics and datastore utilization to understand the impact of virtual machine placement on the space utilization of a datastore.
Modeling
Let’s take a closer look at modeling. SDRS captures device performance to create a performance model; by using the SIOC injector and a reference workload it understands and learns the performance of each device. This way SDRS gets a clear picture of the datastores inside the datastore cluster. Workload modeling is used by SDRS to understand and learn the virtual machine workloads inside the datastore cluster. The workload modeling process creates a workload metric of each virtual disk and analyzes the impact of the data points on latency.
SDRS combines and correlates the outcome of device and workload modeling and space utilization into a unified recommendation. This means that when SDRS decides to migrate a specific VMDK, it considers the workload metric of the virtual disk and analyzes the impact of that specific workload on the latency of the destination datastore. If both IO metric and space utilization functions are enabled on the datastore cluster, SDRS combines the outcome of device modeling, workload modeling and space utilization and weights them regarding to violated threshold. Interesting enough, even when you disable IO load balancing, SDRS attempts to take overall IO statistics into account when finding a suitable datastore.
Impact of load balancing construct on datastore cluster configuration
Although SDRS analyzes devices and each virtual machines’ workload it’s is key to understand that SDRS’ main priority is to correct the threshold violation of datastore. Although it tries to find the best suitable datastore for a specific workload, modeling is still used as a metric to understand and achieve the goal of getting the best overall performance out of the datastore cluster.
In other words, modeling is used for balancing the load on the datastores and not to respect specific wishes of a virtual machine disk. In one way you can argue that SDRS load balancing has somewhat of a socialistic nature. Benefit for the society (datastores inside a datastore cluster) outweighs the individual need (single virtual machine performance). Let’s look at an example to better understand this concept.
Example scenario
VM1 is running on a datastore1. SDRS determined that the normalized load* is 5ms latency. VM2 and VM3 are running on a datastore2. SDRS considers datastore2 to have a normalized load of 20ms latency, violating the default threshold of 15ms.
Normalized load: SDRS aggregates the device modeling and workload modeling into a metric called normalized load.
SDRS moves VM3 to datastore1; at this point the overall latency of the datastore2 is reduced 13ms. However due to moving VM3 to datastore1, the latency is increased from 5ms to 12ms. At this point the increase in latency will impact the workload of VM1, however the “society” benefits from the move because after the move no datastore is violating the latency threshold of the SDRS cluster anymore.
In this scenario the overall IOPS will be higher, which aligns with the goal of SDRS utilizing overall capacity and performance.
Note: As this subject is complex enough, I used a very simple example. In this scenario the latency “moved” with the VM. In real life this is not necessarily the fact, when a virtual machine is moved the latency will go up with the same amount at which the latency went down on the source.
Load Balancing in a Heterogeneous configuration
What if the datastore cluster contains a mix of datastores that are backed by different types of disks? For a moment, let’s focus on the performance impact of a heterogeneous configuration.
As mentioned before, device and workload modeling helps SDRS to find the most suitable datastore for a specific workload, however when combining different types of disk, for example, SSD, FC and SATA, it is not uncommon to see the fastest datastore fill up first.
If one of the smaller SSD’s run out of space, SDRS is required to solve the space utilization threshold violation and will migrate a workload from a faster datastore to a slower datastore, prioritizing space utilization over IO utilization. Although future invocations of the SDRS algorithm might solve the problem by moving VMDK’s around to find a more optimal balance, no priority or guarantees can be assigned to a specific virtual disk avoiding potential decrease in performance of a specific VMDK.
Now at this point most of you wonder if VASA and storage profiles can be used in such a configuration to associate specific profiles to virtual machines and make these VMs compliant to specific datastores. SDRS does not incorporate storage profiles compliancy in the load balancing algorithms and unfortunately not every storage vendor offers VASA providers of their arrays. Some excellent articles about VASA and Profile driven storage can be found at Yellow Bricks.com and blogs.vmware.com/vSphere/storage
VASA: http://blogs.vmware.com/vsphere/2011/08/vsphere-50-storage-features-part-10-vasa-vsphere-storage-apis-storage-awareness.html
Profile driven storage: http://www.yellow-bricks.com/2011/07/13/vsphere-5-0-profile-driven-storage-what-is-it-good-for/
To guarantee specific performance to virtual machines it is recommended to uses similar type disks to back the datastores of a datastore cluster. This configuration offers a stable and predictable service level to the virtual infrastructure. If multiple types of disks are available, it is recommended to split and create multiple datastore clusters each containing groups of identical types of disks.
Previous articles in the SDRS short series Architecture and design of Datastore clusters:
Part1: Architecture and design of datastore clusters
Part2: Partially connected datastore clusters
Cyber Monday deal!
We are long time fascinated by the whole Black Friday and Cyber Monday craze in the USA. Unfortunately we do not celebrate Thanksgiving in the Netherlands and none of the shops are participating in something similar as Black Friday.
This year we thought it was a great idea to participate in some form and what better than to offer our vSphere 5 Clustering Technical Deepdive e-book for a price you cannot resist. We just changed the price of the vSphere 5 Clustering Technical Deepdive to $ 4.99 and 3.99 for our European friends. Yes that is correct…. Less than 5 dollars for over 350 pages of deepdive material.
What better way than recover from the madness of Black Friday and just sit back and relax reading this amazing piece of work? This is most definitely the deal of the year for all virtualization fanatics! Keep in mind that this is a limited offer, Tuesday the 29th the price will be back to “normal” again.
US – ebook – $ 4.99
UK – ebook – £ 3.99
DE – ebook – € 3.99
FR – ebook – € 3.99
Pick it up, tell your friends / colleagues / family about it… Here are some snippets from Amazon reviews, but with 15 extremely positive reviews, all of them 5 out of 5, you know you can’t go wrong:
“If you’re serious about VMware virtualization this book is a must have. Regardless of you responsibilities with a virtual infrastructure administrative, or from a architecture design stand point this book is for you. The level of knowledge and depth which Frank and Duncan cover in this book about the new clustering changes in vSphere 5 is priceless. The design tips and illustrations through the book are truly invaluable. There is no other book that gets into the core of all the different vSphere 5 cluster technologies like this one, ”
“Whether you are longing to know about the transition from AAM to FDM, best practices for DRS and DPM, or are just curious to know what those acronyms are this is a great book! The technical detail, practical advice, and comparative analysis throughout make this book one of the most thorough yet concise technical books available.”
“The book is clearly written, a special emphasis has been made on making it understandable even for professionals like me who use vSphere daily yet do not manage huge production environments. The book goes to great lengths to explain all possible scenarios and I found answers to all my questions. Not only sections cover HOW the technology works, but the authors go as far as explaining the way the algorithms are working, which will satisfy the curiosity of everyone.”
“The complete explanations provide the reader all of the information needed to make informed decisions about their environment with excellent diagrams to provide strong visual reinforcements.”
Please remember that we are offering the book for the price listed above, depending on your location Amazon might charge an additional cost!
New job role
The last two years I enjoyed working as an architect within the PSO organization of VMware, designing and reviewing the most interesting virtual infrastructures in Europe. However today I signed my new contract, accepting a position within the Technical Marketing team.
Starting December I will focus on resource management and disaster avoidance technologies. My new role allows me to collaborate with the Product managers and the R&D organization on products such as DRS, Storage DRS, vMotion, Storage vMotion and FT. My main tasks will be developing best practices, white-papers, documentation and technical presentations, educating field organizations and of course the customers.
Although I enjoyed working within the PSO organization, I can’t wait to get started. Thanks to all the people who made my move possible and offering me such an opportunity!