APPLY USER-DEFINED STORAGE CAPABILITIES TO MULTIPLE DATASTORE AT ONCE
To get a datastore cluster to surface a (user-defined) storage capability, all datastores inside the datastore cluster must be configured with the same storage capability. When creating Storage Capabilities, the UI does not contain a view where to associate a storage capability with multiple datastores. However that does not mean the web client does not provide you with the ability to do so. Just use the multi-select function of the webclient. Go to Storage, select the datastore cluster, select Related Objects and go to Datastores view. To select all datastores, click the first datastore, hold shift and select the last datastore. Right click and select assign storage capabilities. Select the appropriate Storage capability and click on OK. The Datastore Cluster summary tab now shows the user-defined Storage Capability. Get notification of these blogs postings and more DRS and Storage DRS information by following me on Twitter: @frankdenneman
AVOIDING VMDK LEVEL OVER-COMMITMENT WHILE USING THIN DISKS AND STORAGE DRS
The behavior of thin provisioned disk VMDKs in a datastore cluster is quite interesting. Storage DRS supports the use of thin provisioned disks and is aware of both the configured size and the actual data usage of the virtual disk. When determining placement of a virtual machine, Storage DRS verifies the disk usage of the files stored on the datastore. To avoid getting caught out by instant data growth of the existing thin disk VMDKs, Storage DRS adds a buffer space to each thin disk. This buffer zone is determined by the advanced setting “PercentIdleMBinSpaceDemand". This setting controls how conservative Storage DRS is with determining the available space on the datastore for load balancing and initial placement operations of virtual machines. IdleMB The main element of the advanced option “PercentIdleMBinSpaceDemand” is the amount of IdleMB a thin-provisioned VMDK disk file contains. When a thin disk is configured, the user determines the maximum size of the disk. This configured size is referred to as “Provisioned Space”. When a thin disk is in use, it contains an x amount of data. The size of the actual data inside the thin disk is referred to as “allocated space”. The space between the allocated space and the provisioned space is called identified as the IdleMB. Let’s use this in an example. VM1 has a single VMDK on Datastore1. The total configured size of the VMDK is 6GB. VM1 written 2GB to the VMDK, this means the amount of IdleMB is 4GB. PercentIdleMBinSpaceDemand The PercentIdleMBinSpaceDemand setting defines percentage of IdleMB that is added to the allocated space of a VMDK during free space calculation of the datastore. The default value is set to 25%. When using the previous example, the PercentIdleMBinSpaceDemand is applied to the 4GB unallocated space, 25% of 4GB = 1 GB. Entitled Space Use Storage DRS will add the result of the PercentIdleMBinSpaceDemand calculation to the consumed space to determine the “entitled space use”. In this example the entitled space use is: 2GB + 1GB = 3GB of entitled space use. Calculation during placement The size of Datastore1 is 10GB. VM1 entitled space use is 3GB, this means that Storage DRS determines that Datastore1 has 7GB of available free space. Changing the PercentIdleMBinSpaceDemand default setting Any value from 0% to 100% is valid. This setting is applied on datastore cluster level. There can be multiple reasons to change the default percentage. By using 0%, Storage DRS will only use the allocated space, allowing high consolidation. This is might be useful in environments with static or extremely slow data increase. There are multiple use cases for setting the percentage to 100%, effectively disabling over-commitment on VMDK level. Setting the value to 100% forces Storage DRS to use the full size of the VMDK in its space usage calculations. Many customers are comfortable managing over-commitment of capacity only at storage array layer. This change allows the customer to use thin disks on thin provisioned datastores. Use case 1: NFS datastores A use case is for example using NFS datastores. Default behavior of vSphere is to create thin disks when the virtual machine is placed on a NFS datastore. This forces the customer to accept a risk of over-commitment on VMDK level. By setting it to 100%, Storage DRS will use the provisioned space during free space calculations instead of the allocated space. Use case 2: Safeguard to protect against unintentional use of thin disks This setting can also be used as safeguard for unintentional use of thin disks. Many customers have multiple teams for managing the virtual infrastructure, one team for managing the architecture, while another team is responsible for provisioning the virtual machines. The architecture team does not want over-commitment on VMDK level, but is dependent on the provisioning team to follow guidelines and only use thick disks. By setting “PercentIdleMBinSpaceDemand” to 100%, the architecture team is ensured that Storage DRS calculates datastore free space based on provisioned space, simulating “only-thick disks” behavior. Use-case 3: Reducing Storage vMotion overhead while avoiding over-commitment By setting the percentage to 100%, no over-commitment will be allowed on the datastore, however the efficiency advantage of using thin disks remains. Storage DRS uses the allocated space to calculate the risk and the cost of a migration recommendation when a datastore avoids its I/O or space utilization threshold. This allows Storage DRS to select the VMDK that generates the lowest amount of overhead. vSphere only needs to move the used data blocks instead of all the zeroed out blocks, reducing CPU cycles. Overhead on the storage network is reduced, as only used blocks need to traverse the storage network. Get notification of these blogs postings and more DRS and Storage DRS information by following me on Twitter: @frankdenneman
STORAGE DRS DEMO AVAILABLE ON VMWARE TV
If you haven’t seen Storage DRS in action, check out the Storage DRS demo I’ve created for VMwareTV. Get notification of these blogs postings and more DRS and Storage DRS information by following me on Twitter: @frankdenneman
HOW TO CREATE VM TO HOST AFFINITY RULES USING THE WEBCLIENT
This article shows you how to create a VM to Host affinity rule using the new webclient. 1. Select host and clusters in the home screen. 2. Select the appropriate cluster. 3. Select the tab Manage and click on Settings. 4. Click on the » to expand the Cluster setting menu. 5. Select DRS Groups. 6. Click on Add to create a DRS Group. The dropdown box provides the ability to create a VM DRS group and a Host DRS group. The behavior of this window is a little tricky. When you create a group, you need to click on OK to actually create the group. If you create a VM DRS group first and then select the Host DRS group in the dropdown box before you click OK, the VM DRS group configuration is discarded. 7. Create the VM DRS Group and provide the VM group a meaningful name. 8. Click on “Add” to select the virtual machines. 9. Click on OK to add the virtual machines to the group. 10. Review the configuration and click on OK to create the VM DRS Group. 11. Click on “Add” again to create the Host DRS Group. 12. Select Host DRS Group in the dropdown box and provide a name for the Host DRS Group. 13. Click on “Add” to select the hosts that participate in this group. 14. Click on OK to add the hosts to the group. 15. Review the configuration and click on OK to create the Host DRS Group. 16. The DRS Groups view displays the different DRS groups in a single view. The groups are created, now it’s time to create the rules. 17. Select DRS Rules in the Cluster settings menu. 18. Click on “Add” to create the rule. 19. Provide a name for this rule and check if the rule is enabled (default enabled) 20. Select the “Virtual Machines to Hosts” rule in the Type dropbox. 21. Select the appropriate VM Group and the corresponding Host Group. 22. Select the type affinity rule. For more information about the difference between should and must rule, read the article: “Should or Must VM-Host affinity rules?”. In this example I’m selecting the should rule. 23. Click on Ok to create the rule. 24. Review your configuration in DRS rules screen. Get notification of these blogs postings and more DRS and Storage DRS information by following me on Twitter: @frankdenneman
TECHNICAL PAPER: “VMWARE VCLOUD DIRECTOR RESOURCE ALLOCATION MODELS” AVAILABLE FOR DOWNLOAD
Today the technical paper “VMware vCloud Director Resource Allocation Models” has been made available for download on VMware.com. This whitepaper covers the allocation models used by vCloud Director 1.5 and how they interact with the vSphere layer. This paper helps you correlate the vCloud allocation model settings with the vSphere resource allocation settings. For example what happens on the vSphere layer when I set a guarantee on an Org VDC configured with the Allocation Pool Model. It provides insight on the distribution of resources on both the vCloud layer and vSphere layer and illustrates the impact of various allocation model settings on vSphere admission control. The paper contains a full chapter about allocation model in practice and demonstrates the effect of using various combinations of allocation models within a single provider vDC. Please note that this paper is based on vCloud Director 1.5 http://www.vmware.com/resources/techresources/10325
VM STORAGE PROFILES AND STORAGE DRS - PART 2 - DISTRIBUTED VMS
Mentioned in part-1 of the Storage DRS and VM Storage Profiles series, Storage DRS expects “storage characteristics –alike” datastores inside a single datastore cluster. But what if you have multiple tiers of storage and you want to span the virtual machine across them? Storage profiles can assist in deploying the VMs across multiple datastore clusters safely and inline with your SLAs. Storage DRS Datastore architecture When you have multiple tiers of storage, its recommended to create multiple datastore clusters and each datastore cluster contains disks from a single tier. Let’s assume you have three different kinds of disks in your array: SSD, FC 15K and SATA. Datastores backed by disk out a single pool are aggregated into a single datastore, resulting in three datastore clusters. Having multiple datastore clusters can increase the complexity of the provisioning process, using VM storage profiles ensures you that virtual machines or disk files are placed in the correct datastore cluster. Assign storage capabilities to datastores All datastores within a single datastore cluster are associated with the same storage capability.
VM STORAGE PROFILES AND STORAGE DRS - PART 1
In my previous article about how to configure storage profiles using the web client I stated that different storage profiles could be assigned to a single virtual machine. Storage profiles can be used together with Storage DRS. Let’s take a closer look on how to use storage profiles with Storage DRS. Architectural view VM storage profiles need to be connected to a storage capability to function. The storage capability itself needs to be associated to one or more datastores. A virtual machine in its whole can be associated with a storage profile, or you can use a more granular configuration and associate different storage profiles to the VM working directory and / or VMDK files. Datastore cluster storage capabilities You might have noticed that there isn’t a datastore cluster element depicted in the diagram. The storage capabilities of a datastore cluster are extracted from the associated storage capabilities of each datastore member. If all datastores are configured with the same storage capability, the datastore cluster surfaces this storage capability and becomes compliant with the connected VM storage profiles. For example, “Datastore cluster – Tier 1 VMs and VMDKs” contains 4 datastores. NFS-F-01, NFS-F-02, NFS-F-03 are associated with the storage capability “SSD low latency disk (Tier-1 VMs and VMDKs)” while datastore NFS-F-04 is associated with storage capability “FC 15K – High Speed disk (Tier 2 VMs and VMDKs)”. When reviewing the Storage Capabilities of the datastore cluster, no Storage Capability is displayed: The VM Storage Profile “Tier 1 VMs and VMDK” is connected to the Storage Capability “SSD low latency disk (Tier-1 VMs and VMDKs)”. When selecting storage during the deployment of a virtual machine, the datastore cluster is considered incompatible with the selected VM Storage Profile. Incompatible, but there are three datastores available with the correct Storage capabilities? Although this is true, Storage DRS does not incorporate storage profiles compliancy in its balancing algorithms. Storage DRS is designed with the assumption that all disks backing the datastores are “storage characteristics-alike”. Manually selecting a datastore in the datastore cluster is only possible if the option “Disable Storage DRS for this virtual machine” is selected. Placing the VM on the specific datastore and then enabling Storage DRS later on that VM is futile. Storage DRS will load balance the VM if necessary, but it doesn’t take the VM storage profile compatibility into account when load balancing. So if you have, please remove this “workaround” in your operation manuals :) After removing the datastore with the dissimilar storage capability (NFS-F-04), the Datastore cluster surfaces “SSD – Low Latency disk (Tier-1 VMs and VMDKs)” and becomes compatible with virtual machines associated with the Tier-1 VM storage Profile. Part 2 will cover distributing virtual machine across multiple datastores using Storage Profiles. Get notification of these blogs postings and more DRS and Storage DRS information by following me on Twitter: @frankdenneman
VMWORLD EUROPE SESSIONS
At VMworld Europe I will participate in Meet the Expert sessions, Group discussions and three breakout sessions. Here’s an overview of my public schedule. I hope to see you all in my sessions or participate in the group discussions. Or schedule a time slot in Meet the Expert if you have a question about Storage DRS or DRS that you want to ask me. Tuesday 9 October: 11:00 – 12:00: Group Discussion 28 Resource Management 12:30 - 13:30: INF-VSP1168 Architecting a Cloud Infrastructure 14:00 – 15:00: INF-VSP1683 Resource Pool Best Practices Wednesday 10 October: 11:00 – 12:00: Meet the Experts 05 12:30 – 13:30: Group Discussion 28 Resource Management 15:30 – 16:30: INF-STO1545 Architecting and designing (SDRS) Datastore Clusters Thursday 11 October: 10:30 – 11:30: Meet the Experts 11
HOW TO ATTACH VM STORAGE PROFILES TO A VIRTUAL MACHINE USING THE WEB CLIENT
Virtual Machine Storage Profiles are used to identify the storage capabilities necessary in order to properly run the application within the virtual machine. VM Storage Profiles need to enabled first on the hosts and/or Cluster before you are able to assign them to a virtual machine. 1. Select VM Storage Profiles in the Home screen 2. Select the Icon in the middle to enable VM Storage profiles 3. Select the cluster or host that you want to enable Check if the host has the appropriate license 4. Create Storage Profiles A VM storage profile is attached to a Storage capability. In turn a Storage Capability profile is attached to a datastore. For more information about storage capabilities please read the article: vSphere 5.0 Storage Features Part 11 – Profile Driven Storage by Cormac Hogan. 5. Go back to home 6. Select VM and Templates 7. Select Datacenter in the left pane 8. Select the menu option “Related Objects” in the right pane 9. Select the menu option “Virtual Machines” 10. Right click on a virtual machine 11. Select All vCenter Actions 12. Select Storage Profiles 13. Select Manage Storage Profiles 14. Apply the VM storage Profiles to the working directory and disk Please note that you can assign different storage profiles to the virtual machine working directory and each single VMDK. The working directory is where the .vmx, .nvram, .log resides. This is listed in the UI as the Home VM location. Each VMDK can be assigned with a different storage profile to align it with your SLA’s.
HOW TO CREATE A DATASTORE CLUSTER USING THE NEW WEB CLIENT.
vSphere 5.1 main user interface is provided by the web client, during beta testing I spend some time to get accustomed to the new user interface. In order to save you some time, I created this write-up on how to create a datastore cluster using the web client. I assume you already installed the new vCenter 5.1. If not, check out’s Duncan post on how to install the new vCenter Server Appliance. Before showing the eight easy steps that need to be taken when creating a datastore cluster, I want to list some constraints and the recommendations for creating datastore clusters. Constraints: • VMFS and NFS cannot be part of the same datastore cluster. • Similar disk types should be used inside a datastore cluster. • Maximum of 64 datastores per datastore cluster. • Maximum of 256 datastore clusters per vCenter Server. • Maximum of 9000 VMDKs per datastore cluster Recommendations: • Group disks with similar characteristics (RAID-1 with RAID-1, Replicated with Replicated, etc.) • Leverage information provided by vSphere Storage APIs - Storage Awareness The Steps 1. Go to the Home screen and select Storage 2. Select the Datastore Clusters icon in Related Objects view. 3. Name and Location The first steps are to enable Storage DRS, specify the datastore cluster name and check if the “Turn on Storage DRS” option is enabled. When “Turn on Storage DRS” is activated, the following functions are enabled: • Initial placement for virtual disks based on space and I/O workload • Space load balancing among datastores within a datastore cluster • IO load balancing among datastores within a datastore cluster The “Turn on Storage DRS” check box enables or disables all of these components at once. If necessary, I/O balancing functions can be disabled independently.If Storage DRS is not enabled, a datastore cluster will be created which lists the datastores underneath, but Storage DRS won’t recommend any placement action for provisioning or migration operations on the datastore cluster. When you want to disable Storage DRS on an active datastore cluster, please note that all the Storage DRS settings, e.g. automation level, aggressiveness controls, thresholds, rules and Storage DRS schedules are saved so they may be restored to the same state at the moment Storage DRS was disabled. 4. Storage DRS Automation Storage DRS offers two automation levels: No Automation (Manual Mode) Manual mode is the default mode of operation. When the datastore cluster is operating in manual mode, placement and migration recommendations are presented to the user, but are not executed until they are manually approved. Fully Automated Fully automated allows Storage DRS to apply space and I/O load-balance migration recommendations automatically. No user intervention is required. However, initial placement recommendations still require user approval. Storage DRS allows virtual machines to have individual automation level settings that override datastore cluster-level automation level settings. Similar to when DRS was introduced, I recommend to start using manual mode first and review the generated recommendations. If you are comfortable with the decision matrix of Storage DRS you can switch to fully automated. Please note that you can switch between modes on the fly and without incurring downtime. 5. Storage DRS Runtime Settings Keep the defaults for now. Future articles expand upon the Storage DRS thresholds and advanced options. 6. Select Clusters and Hosts The “Select Hosts and Clusters” view allows the user to select one or more (DRS) clusters to work with. Only clusters within the same vCenter datacenter can be selected, as the vCenter datacenter is the boundary for Storage DRS to operate in. 7. Select Datastores By default, only datastores connected to all hosts in the selected (DRS) cluster(s) are shown. The Show datastore dropdown menu provides the options to show partially connected datastores. The article partially connected datastore cluster gives you insight of the impact of this design decision. 8. Ready to Complete The “Ready to Complete” screen provides an overview of all the settings configured by the user. Review the configuration of your new datastore cluster and click on finish.