frankdenneman.nl
frankdenneman Frank Denneman is the Chief Technologist for AI at VMware by Broadcom. He is an author of the vSphere host and clustering deep dive series, as well as a podcast host for the Unexplored Territory podcast. You can follow him on Twitter @frankdenneman

Virtual Machine memory overhead

1 min read

Every virtual machine running on an ESX host consumes some memory overhead additional to the current usage of its configured memory. This extra space is needed by ESX for the internal VMkernel datastructures like virtual machine frame buffer and mapping table for memory translation (mapping physical virtual machine memory to machine memory). Two kinds of virtual machine overhead exists:
Static overhead
Static overhead is the minimum overhead that is required for the virtual machine startup. DRS and the VMkernel uses this metric for admission control and VMotion calculations. The destination host must be able to back the virtual machine reservation and the static overhead otherwise the VMotion will fail.
Dynamic overhead
Once the virtual machine has started up, the virtual machine monitor (VMM) can request additional memory space. The VMM will request the space, but the VMkernel is not required to supply it. If the VMM does not obtain the extra memory space, the virtual machine will continue to function but this can lead to performance degradation. The VMkernel treats virtual machine overhead reservation the same as VM-level memory reservation and it will not reclaim this memory once it used.
Overhead memory used in admission control
As mentioned before, DRS and the VMkernel will not allow the virtual machine to be powered up if reservations cannot be guaranteed, this means that the effective memory reservation for a virtual machine is the user configured memory reservation (VM-level reservation) plus the overhead reservation.
Resource pool memory reservations
This means that during the design phase of a resource pool, the memory overhead of a virtual machine must be included in the calculation of the memory reservation specified on the resource pool. The behavior of dynamic overhead must also be taken into account.
Table 3.2 of the vSphere resource management guide list the overhead memory on virtual machines. VMware vSphere Online Library – Table 3.2 overhead memory
Please be aware of the fact that memory overheads are growing with each new release of ESX, so keep this in mind when upgrading to a new version. Verify the documentation of the virtual machine memory overhead and check the specified memory reservation on the resource pool.

frankdenneman Frank Denneman is the Chief Technologist for AI at VMware by Broadcom. He is an author of the vSphere host and clustering deep dive series, as well as a podcast host for the Unexplored Territory podcast. You can follow him on Twitter @frankdenneman
«
»

Memory Deep Dive Summary

This is last part of the memory deep dive. As the total series count 7667 words, I thought it would be a good idea...
frankdenneman
Mar 2, 2015 4 min read

Memory Deep Dive: NUMA and Data Locality

This is part 6 of the memory deep dive. This is a series of articles that I wrote to share what I learned while...
frankdenneman
Feb 27, 2015 7 min read

Memory Deep Dive: DDR4 Memory

This is part 5 of the memory deep dive. This is a series of articles that I wrote to share what I learned while...
frankdenneman
Feb 25, 2015 6 min read

7 Replies to “Virtual Machine memory overhead”

  2. Thanks for the info.
    What dynamic overhead memory is used for?
    I remember that in ESX 3 memory overhead was different for 32 and 64 bit VMs. Has this been changed in vSphere?

  3. Nitro, Michael dynamic overhead totally depends on the workload and can change while the virtual machine is running.
    Shadow page tables play an important role in sizing the dynamic overhead as it can shrink and grow according to the virtual machine guest os workload. When the VMM wants to add more shadow page tables, it will request an increase in the amount of overhead memory.
    Michael, yes the memory overhead is different, but whats even more suprising is that VMware stopped mentioning the virtual machine guest OS. Im not sure if it doesn’t have impact anymore on the size of the overhead or that just listed the worst-case scenario

  4. Pingback: Technology Short Takes #1 - blog.scottlowe.org - The weblog of an IT pro specializing in virtualization, storage, and servers
  5. Pingback: Vertial memory | CftcStx

  6. Been a year, but still the info is usefull. Anyway the link to the site doesn’t even come close to make proper sizing. Backing it with own example:
    RP with 1 VM: 1 vCPU and 1024MB running Win7 32bit
    Using the table I’d need at least 123,73MB of reservation on RP (if it’s the static you mention). I’m able to poweron VM when I have at least 181MB reservation though. Minute or two after VM poweron the resource allocation tab shows overhead consumption at 104MB and overhead reservation at 161MB. So there’s no way to match any of the numbers.
    The differences rise when we switch from Win7 32bit to RHEL6 32bit – I’m not able to poweron the VM at 181MB, but after I do (expandable, don’t want to search for single number) the resource tab shows 42MB of consumption and 101MB overhead reservation.
    Haven’t check the VMware site yet to find any other table, but this table is of no use to do proper minimal sizing. Any info on that?

  7. Pingback: What is static overhead memory?

Comments are closed.