DOWNLOAD THE VSPHERE 4.1 AND 5.0 CLUSTERING DEEPDIVE FOR FREE
Do you want a free Kindle copy of the vSphere 4.1 HA and DRS Deepdive or the vSphere 5.0 Clustering Deepdive? Today and tommorow, Thursday June the 6th, you can download the Kindle (US Kindle Store) copy of both these books for free! So make sure you pick it up either today or tomorrow, it might be the only time this year it is on promo.
WANT TO HAVE MY FORMER JOB?
My old technical marketing team of VMware is looking for someone to cover resource management. If you have a passion for resource management of virtual infrastructures and like to help VMware’s field personnel, partners and customers understand the technology then this job might be something for you. A large part of my role was bridging between engineering, product management, product marketing and the field / customers. Provide information to the R&D side of VMware how the products are used and what features customers are requesting. You create collateral in every way or form to help the customer and field personnel understand and adopt the features. I always enjoyed working with the different teams at VMware. The cloud resource management and vMotion team are an awesome group to work with. Be prepared to deep dive with these guys Marianas trench style. Having a customer facing background helps you provide the team valuable information to align the features to the customer wishes. In this role you assist product marketing and product management in achieving their tactical and strategic plans. Besides working with the responsible engineering and product marketing teams you collaborate with your technical marketing colleagues. You have the ability to interact with guys such as Ken Werneberg, Cormac Hogan, Mike Foley, William Lam, Alan Renouf or Rawlinson Rivera on a daily basis. If you have thorough understanding of the vMotion features, DRS, Storage DRS, SIOC and DPM and love to help customers adopt these features, then apply now! http://jobs.vmware.com/job/Palo-Alto-Sr_-Technical-Marketing-Manager-Resource-Management-Job-CA-94301/2593496/ Please note Be aware that this is my former role and that I no longer work for VMware. Therefor I cannot answer any further inquiries. Please contact the VMware career team.
NEW STORAGE DRS WHITEPAPER AVAILABLE AT VMWARE.COM
Last Friday my last and latest whitepaper about Storage DRS was published on VMware.com. Go to http://www.vmware.com/resources/techresources/10363 and download the whitepaper: “Understanding vSphere 5.1 Storage DRS”. Download and read this whitepaper if you want to learn more about the five key elements of Storage DRS. Here’s a little snippet from the whitepaper: Step 1. Determine Whether Datastores Are Violating the Space-Utilization Threshold If the space utilization of a datastore exceeds 80 percent, the datastore violates the threshold and the vSphere Storage DRS load-balancing algorithm is invoked. vSphere Storage DRS attempts to avoid an out-of-space situation and therefore runs a load-balancing operation as soon as the datastore exceeds its space-utilization threshold. This operation can be outside of the normal load-balancing interval of every 8 hours. The space-utilization threshold is a soft limit, enabling vSphere Storage DRS to place virtual machines in the datastore cluster even if all datastores exceed the space-utilization threshold. vSphere Storage DRS attempts to generate prerequisite migrations before virtual machine placement. If this fails, the virtual machine is placed on the datastore that provides the best overall cluster balance. This performance applies to space-utilization load-balancing operations as well, even if all datastores violate the space-utilization threshold. vSphere Storage DRS tries to keep space utilization near the threshold across all datastores. Download: http://www.vmware.com/files/pdf/vmw-vsphr-5-1-stor-drs-uslet-101-web.pdf
CLOUDPHYSICS WEBINAR: EXPERT TIPS FOR MANAGING DATASTORE SPACE IN A VSPHERE ENVIRONMENT
Tonight I will join Erik Haus and Krishna Raj Raja of CloudPhysics to talk about Datastore space management in a virtual infrastructure. During the webinar Krishna will show you how the new CloudPhysics “Datastore Space” card and “Snapshots Gone Wild” card help you to identify and resolve space problems. Join us! The event will start at 6:00 pm Amsterdam Time on May 9, 2013. (9:00 AM PDT) Go to the CloudPhysics site to register!
EMBARKING ON A NEW ADVENTURE - JOINING PERNIXDATA AS TECH EVANGELIST
Sometimes something comes along that makes you feel you need to get involved with. Something that makes you want to leave the comfortable position you have now and take up the challenge of starting all over again. Help turn that something into something big. Well that something is in my case PernixData and its Flash Virtualization Platform. Joining PernixData means I’m leaving the great company of VMware and an awful lot of great colleague behind. Some of them I consider to be good friends. During my years at VMware I learned a lot and words cannot describe how awesome those years were. Designing the vCloud environment for the European launching partner, consulting a lot of the Fortune 500 firms, participating in VCDX panels around the world and co-authoring three books are some of the highlights during my time at VMware but I’m sure I’m forgetting a lot of other great moments. Being a part of the technical marketing team was amazing! Besides working alongside the best bloggers in the world I had the privilege to work with the engineers on a daily basis. Having a job that allows you to think, talk and write about technology you absolutely love is great and difficult to let go. But opportunities do come along and as I mentioned in the beginning some of these opportunities spark the desire to become a part of that story. When I attended a technical preview of the Flash Virtualization Platform at PernixData I got excited. I think just as excited as when I saw my first vMotion. Meeting the founders and the team made me realize that this company and product was more than just a single product, this platform is a game changer in the world of virtual infrastructure and datacenter design. Which drove me to the decision to accept a position with PernixData as Technology Evangelist. As the Technology Evangelist I’m responsible for helping the virtualization community understand PernixData’s Flash Virtualization Platform (FVP). And as the first international employee I also will be focusing on expanding the European organization. I will be starting at PernixData soon, can’t wait to start
VSPHERE 5.1 UPDATE 1 RELEASE FIXES STORAGE VMOTION RENAME "BUG"
vSphere 5.1 update 1 is released today which contains several updates and bug fixes for both ESXi and vCenter Server 5.1. This release contains the return of the much requested functionality of renaming VM files by using Storage vMotion. Renaming a virtual machine within vCenter did not automatically rename the files, but in previous versions Storage vMotion renamed the files and folder to match the virtual machine name. A nice trick to keep the file structure aligned with the vCenter inventory. However engineers considered it a bug and “fixed” the problem. Duncan and I pushed hard for this fix, but the strong voice of the community lead (thanks for all who submitted a feature request) helped the engineers and product managers understand that this bug was actually considered to be a very useful feature. The engineers introduced the “bugfix” in 5.0 update 2 end of last year and now the fix is included in this update for vSphere 5.1 Here’s the details of the bugfix:
AWESOME READ: STORAGE PERFORMANCE AND TESTING BEST PRACTICES
The last couple of days I’ve been reading up on EMC VPLEX technology as I’m testing VPLEX metro with SIOC and Storage DRS. Yesterday I discovered a technical paper called “EMC VPLEX: Elements Of Performance And Testing Best Practices Defined” and I think this paper should be read by anyone who is interested in testing storage or even wanting to understand the difference between workloads. Even if you do not plan to use EMC VPLEX the paper delivers some great insights about IOPS versus MB/s. What to expect when testing for transactional-based workloads and throughput-based workload? Here’s a little snippet: “Let’s begin our discussion of VPLEX performance by considering performance in general terms. What is good performance anyway? Performance can be considered to be a measure of the amount of work that is being accomplished in a specific time period. Storage resource performance is frequently quoted in terms of IOPS (IO per second) and/or throughput (MB/s). While IOPS and throughput are both measures of performance, they are not synonymous and are actually inversely related – meaning if you want high IOPS, you typically get low MB/s. This is driven in large part by the size of the IO buffers used by each storage product and the time it takes to load and unload each of them. This produces a relationship between IOPS and throughput as shown in Figure 1 below.” Although it’s primarily focused on VPLEX, the paper helps you understand the different layers of a storage solution and how each layer affects performance. Another useful section is the overview of good benchmark software which describes the basic operation of each listed benchmark program. The paper is very well written and I bet even a joy to read for both the beginner as well as the the most hardened storage geek. Download the paper here.
MIGRATING VMS BETWEEN DRS CLUSTERS IN AN ELASTIC VDC
In the article “Migrating datastore clusters by changing storage profiles in a vCloud“ I closed with the remark that vCD is not providing an option to migrate virtual machines between compute clusters that are part of an elastic vDC. Fortunately my statement was not correct. Tomas Fojta pointed out that vCD does provide this functionality. Unfortunately this feature is not exposed in the vCloud organization portal but in the system portal of the vCloud infrastructure itself. In other words, to be able to use this functionality you need to have system administrator privileges. In the previous article, I created the scenario where you want to move virtual machines between two sites. Site 1 contains compute cluster “vCloud-Cluster1” and datastore cluster “ DSC-Site-1”. Site 2 contains “vCloud-Cluster2” and datastore cluster “DSC-Site-2” . By changing the VM storage profile from Site-1 to Site-2, we have vCD instruct vSphere to storage vMotion the virtual machine disk files from one datastore cluster to another. Now at this point we need to migrate the compute state of the virtual machine. Migrate virtual machine between clusters Please note that vCD refers to clusters as resource pools. To migrate the virtual machine between clusters, log into the vCloud director and select the system tab. Go to the vSphere resources and select Resource Pools menu option. The UI displays the clusters that are a part of the Provide vDC. Select the cluster a.k.a. resource pool in which the virtual machine resides. Select the virtual machine to migrate, right click the virtual machine to have vCD display the submenu and select the option “Migrate to…” The user interface allows you to choose how you want to select the destination resource pool for the virtual machine: Either automatic and let vCD select the resource pool for you, or select the appropriate resource pool manually. When selecting automatic vCD selects the cluster with the most unreserved resources available. If the virtual machine happens to be in the cluster with the most unreserved resources available vCD might not move the virtual machine. In this case we want to place the virtual machine in site 2 so that means we need to select the appropriate cluster. We select vCloud-Cluster2 and click on OK to start the migration process. vCD instructs vSphere to migrate the virtual machine between clusters with the use of vMotion. In order to use vMotion, both clusters need to have access to the datastore on which the virtual machine files reside. vCD does not use “enhanced’ vMotion where it can live migrate between host without being connected shared storage. Hopefully we see this enhancement in the future. When we log into vSphere we can verify if the life migration of the virtual machine was completed. Select the destination cluster, in this case that would be vCloud-Cluster2, go to menu option Monitor, select tasks and click on the entry “Migrate virtual machine” In the lower part of the screen, you get more detailed information of the Migrate-virtua-machine entry. As you can seem the virtual machine W2K8_RS_SP1 is migrated between servers 10.27.51.155 and 10.27.51.152. As we do not change anything to the storage configuration, the virtual machine files remains untouched and stay on the same datastore. To determine if vCD has updated the current location of the virtual machine, log into vCD again, go to the menu option “Resource Pools” and select the cluster chosen as destination as the previously org cluster.
3 COMMON QUESTIONS ABOUT DRS PREFERENTIAL VM-HOST AFFINITY RULES
On a regular basis I receive questions about the behavior of DRS when dealing with preferential VM to Host affinity rules. The rules configured with the rule set “should run on / should not run on” are considered preferential. Meaning that DRS prefers to satisfy the requirements of the rules, but is somewhat flexible to run a VM outside the designated hosts. It is this flexibility that raises questions; lets see how “loosely” DRS can operate within the terms of conditions of a preferential rule: Question 1: If the cluster is imbalanced does DRS migrate the virtual machines out of the DRS host group? DRS only considers migrating the virtual machines to hosts external to the DRS host group if each host inside the group is 100% utilized. And if the hosts are 100% utilized, then DRS will consider virtual machines that are not part of a VM-Host affinity rule first. DRS will always avoid violating an affinity rule Question 2: When a virtual machine is powered on, will DRS start the virtual machine on a host external to the DRS host group? By default DRS will start the virtual machine on hosts listed in the associated Host DRS group. If all hosts are 100% utilized – or – if they do not meet the virtual machine hardware requirements such as datastore or network connectivity, then DRS will start the virtual machine on a host external to the Host DRS group. Question 3: If a virtual machine is running on a host external to the associated host DRS group, shall DRS try to migrate the virtual machine to a host listed in the DRS host group? The first action DRS triggers during an invocation is to determine if an affinity rules is violated. If a virtual machine is running on a host external to the associated Host DRS group then DRS will try to correct this violation. This move will have the highest priority ensuring that this move is carried out during this invocation.
MIGRATING DATASTORE CLUSTERS BY CHANGING STORAGE PROFILES IN A VCLOUD
vCloud director 5.1 supports the use of both storage profiles and Storage DRS. One of the coolest features and unfortunately relatively unknown is the ability to live migrate virtual machines between datastore clusters by changing the storage profile in the vCloud director portal. In my lab I’ve set up a provider vDC that contains two compute clusters. Each compute cluster connects to two datastore clusters. Datastore Cluster “vCloud-SDC-Gold” is compatible with the VM storage profile “vCloud-Gold-Storage”, while Datastore Cluster “vCloud-SDC-Silver” is compatible with the VM storage profile “vCloud-Silver-Storage”. When creating a vApp the default storage profile of the organization vDC is applied to the vApp and all its virtual machines. In this case, the VM storage profile Gold is applied to all the virtual machines in the vApp. You can determine which VM Storage Profile is associated with the virtual machine by selecting the properties of the virtual machine in the “My Cloud” tab. Please note that vCloud Director does not show the VM Storage Profile at the vApp level! By selecting the drop-down box, all storage profiles that are associated with the organization vCD are displayed. By selecting the Storage Profile “vCloud-Silver-Storage” vCloud Director determines that the virtual machine is stored on a datastore that is not compatible with the associated storage profile. In other words the current configuration is violating the storage level policy. To correct this violation, vCloud director instructs vSphere to migrate the virtual machine via Storage vMotion to a datastore that is compatible with the VM storage Profile. In this case the datastore cluster “vCloud-DSC-Silver” is selected as the destination. Storage DRS determines the most suitable datastore by using its initial placement algorithm and selects the datastore that has the most amount of free space and the lowest I/O load. To demonstrate the feature, I selected the virtual machine “W2K8_R2-SP1”. The VM storage profile “vCloud-Gold-Storage” is applied and Storage DRS determined that the datastore “nfs-f-vcloud03” of the datastore cluster “vCloud-DSC-Gold” was the most suitable location. By changing the Storage Profile to “vCloud-Silver-Storage” vCloud director instructed vSphere to migrate it to the datastore cluster that is compatible with the newly associated VM storage profile. When logging into the vCenter server managing the ESXi hosts the following task is running: After the task is complete, vCenter shows that the virtual machine is now stored on datastore “nfs-f-vcloud06” in the datastore cluster “vCloud-DSC-Silver”. The power of abstraction The abstraction layer of vCloud Director makes this possible. When changing the storage profile directly on the vSphere layer, nothing happens. vSphere will not migrate the virtual machine to the appropriate datastore cluster that is compatible with the selected VM storage profile. Useful for stretched clusters? The reason why I was looking into this feature in my lab is due to an conversation with my esteemed colleagues Lee Dilworth and Aidan Dalgleish. We were looking to an alternative scenario for a stretched cluster. By leveraging the elastic vDC feature of vCloud director, a seperate DRS cluster is created in each site. Due to the automatic initial placement engine on the compute level, we needed to find a construct that can provide us a more deterministic method of virtual machine placement. We immediately thought of the VM profile storage feature. Create two datastore clusters, one per site and associate a profile storage based on site name to the respective datastore clusters. When creating the vApp, just select the site-related Storage Profile to place the virtual machine in a specific site. Due to the compatibility check, vCloud Director determines that in order to be compliant with the storage profile it places the virtual machine on the compute cluster in the same site. For example, if you want to place a virtual machine in site 1, select the VM storage Profile “site 1”. vCloud director determines that the virtual machine needs to be stored in datastore cluster “DSC-Site-1”. The compute cluster Site-1 is the only compute cluster connected to the datastore cluster, therefor both the compute and storage configuration of the virtual machine is stored in Site 1. This configuration works perfect if you want to simplify initial placement if you have multiple sites/locations and you always want to keep the virtual machine in the same site. However this solution might not be optimal for a Stretched cluster configuration where failover to another site is necessary. Connectivity to all datastores necessary As this feature uses storage vMotion instead of cross-host/datastore vMotion, means that the cluster needs to be connected to both datastore clusters. When selecting the different storage profile, the storage state is migrated to another datastore cluster. However it doesn’t move the compute state of the virtual machine. This means that storage is moved to site B, while the compute state is still in Site A. vCloud director does not provide an option to migrate the virtual machine to a different compute cluster within the provider vDC. You can either solve it by logging into the vCenter server that manages the ESXi hosts and manually vMotion the virtual machines to cluster in Site B, or power-off the virtual machine in vCloud Director, then change the storage profile and power-on the virtual machine. Both “solutions” are not very enterprise-level scenario’s therefor I think this is not yet suitable as a stretched cluster configuration