One of the cool “under the hood” improvements vSphere 5 offers is the sliding scale function of the Mem.MinFreePct.
Before diving into the sliding scale function, let’s take a look at the Mem.MinFreePct function itself. MinFreePct determines the amount of memory the VMkernel should keep free. This threshold is subdivided in various memory thresholds, i.e. High, Soft, Hard and Low and is introduced to prevent performance and correctness issues.
The threshold for the low state is required for correctness. In other words, it protects the VMkernel layer from PSOD’s resulting from memory starvation. The soft and hard thresholds are about virtual machine performance and memory starvation prevention. The VMkernel will trigger more drastic memory reclamation techniques when it approaches the Low state. If the amount of free memory is just a bit less than the Min.FreePct threshold, the VMkernel applies ballooning to reclaim memory. The ballooning memory reclamation technique introduces the least amount of performance impact on the virtual machine by working together with the Guest operating system inside the virtual machine, however there is some latency involved with ballooning. Memory compressing helps to avoid hitting the low state without impacting virtual machine performance, but if memory demand is higher than the VMkernels’ ability to reclaim, drastic measures are taken to avoid memory exhaustion and that is swapping. However swapping will introduce VM performance degradations and for this reason this reclamation technique is used when desperate moments require drastic measurements. For more information about reclamation techniques I recommend reading the “disable ballooning” article.
vSphere 4.1 allowed the user to change the default MinFreePct value of 6% to a different value and introduced a dynamic threshold of the Soft, Hard and Low state to set appropriate thresholds and prevent virtual machine performance issues while protecting VMkernel correctness. By default vSphere 4.1 thresholds was set to the following values:
| Free memory state | Threshold | Reclamation mechanism |
| High | 6% | None |
| Soft | 64% of MinFreePct | Balloon, compress |
| Hard | 32% of MinFreePct | Balloon, compress, swap |
| Low | 16% of MinFreePct | Swap |
Using a default MinFreePct value of 6% can be inefficient in times where 256GB or 512GB systems are becoming more and more mainstream. A 6% threshold on a 512GB will result in 30GB idling most of the time. However not all customers use large systems and prefer to scale out than to scale up. In this scenario, a 6% MinFreePCT might be suitable. To have best of both worlds, ESXi 5 uses a sliding scale for determining its MinFreePct threshold.
| Free memory state threshold | Range |
| 6% | 0-4GB |
| 4% | 4-12GB |
| 2% | 12-28GB |
| 1% | Remaining memory |
Let’s use an example to explore the savings of the sliding scale technique. On a server configured with 96GB RAM, the MinFreePct threshold will be set at 1597.6MB, opposed to 5898.24MB if 6% was used for the complete range 96GB.
| Free memory state | Threshold | Range | Result |
| High | 6% | 0-4GB | 245.96MB |
| 4% | 4-12GB | 327.84MB | |
| 2% | 12-28GB | 696.32MB | |
| 1% | Remaining memory | 696.32MB | |
| Total High Threshold | 1597.60MB |
Due to the sliding scale, the MinFreePct threshold will be set at 1597.96MB, resulting in the following Soft, Hard and low threshold:
| Free memory state | Threshold | Reclamation mechanism | Threshold in MB |
| Soft | 64% of MinFreePct | Balloon | 10244.26 |
| Hard | 32% of MinFreePct | Balloon, compress | 511.23 |
| Low | 16% of MinFreePct | Balloon, compress, swap | 255.62 |
Although this optimization isn’t as sexy as Storage DRS or one of the other new features introduced by vSphere5 it is a feature of vSphere 5 that helps you drive your environments to higher consolidation ratios.
Follow @frankdenneman on twitter
Subscribe to RSS Get updates sent by RSS
vSphere 5 Clustering technical deepdive By Duncan Epping and Frank Denneman
vSphere 4.1 HA and DRS technical deepdive By Duncan Epping and Frank Denneman
July 27th, 2011 at 01:27
Is the First Table wrong in the book?
July 27th, 2011 at 01:31
Very cool article Frank. Also, this kb recommend to set 2% on ESXi 4.1 with more than 64GB : http://kb.vmware.com/kb/1033687
July 27th, 2011 at 07:24
Hi Ron,
Thanks for spotting this error, no the book is correct, the table in the article isn’t.
I corrected the mistake in the table.
July 27th, 2011 at 15:33
Frank, when does the breaking down of large page tables to enable TPS take place on Nehalem systems? If this is at high I would think it would be advantageous to have a buffer between High and Soft.
August 18th, 2011 at 13:49
Hi,
has there been any change to TPS regarding when (physical RAM utilization) large pages are broken down to small pages?
thanks
October 13th, 2011 at 22:57
Frank, thanks for posting this useful piece of information. However, I noticed that the last 2 tables were not accurate (Thanks to YP from Kingston). Please refer to my blog @ http://voiceforvirtual.com/2011/10/13/minfreepct-mem-reclamation-esx-5-0/ for a clarification.
May 17th, 2012 at 10:45