jump to navigation

Memory overcommitment and techniques to allow this June 13, 2012

Posted by vbry21 in VMware blogs.
Tags: ,
add a comment

Memory Reclamation Techniques in ESXi

I’ve mentioned the four main memory reclamation techniques mentioned in the Resource Management Guide, but there’s also Host cache as well. The purpose of these techniques is to allow over commitment of  the ESXi hosts physical memory

Transparent Page Sharing

Transparent page sharing is a mechanism in which an ESX/ESXi host tries to conserve physical memory so that it will not have to resort to other memory management techniques. Transparent page sharing is enabled by default.

The VMkernel dynamically scans memory to look for duplicate pages. The VMkernel detects when different virtual machines have memory pages with identical content and arranges for those pages to be shared. That is, a single physical page is mapped into each virtual machine’s address space. If a virtual machine tries to modify a page that is shared, the VMkernel creates a new, private copy for that virtual machine and then maps that page into the address space of that virtual machine only. The other virtual machines continue to share the original copy.

When a virtual machine has been suspended and gets resumed, it does not immediately participate in the memory-sharing system. Its pages become shared over time. So if you intend to suspend and resume large batches of virtual machines, plan to have enough memory.

Balloon Driver

When a virtual machine must yield memory, the best thing is to let the guest operating system in the virtual machine select which pages of memory to give up. The virtual machine knows which pages have been least recently used and which pages can easily be refreshed from a backing store on disk.

The term “balloon driver” is an informal way of referring to the vmmemctl device driver, which is used to perform memory de-allocation or reallocation.

The balloon driver is installed when you install VMware Tools within the VM. The balloon driver’s function is to demand memory from the guest operating system and later to relinquish it, under the control of the VMkernel.

The guest operating system in the virtual machine is not aware of the communication taking place between the balloon driver and the VMkernel. The guest operating system is aware that the balloon driver is installed but is not aware of its purpose.

When a system is not under memory pressure, no virtual machine’s balloon is inflated. But when memory becomes scarce, the VMkernel chooses a virtual machine and inflates its balloon. That is, it tells the balloon driver in the virtual machine to demand memory from the guest operating system. The guest operating system complies by yielding memory, according to its own algorithms. The relinquished pages can be assigned by the VMkernel to other virtual machines.

Memory Compression

Memory compression can enable you to effectively use existing hardware and increase consolidation ratios by enabling greater virtual machine density per host. Memory compression is transparent to the guest operating system.

The process of compressing memory pages is much faster when compared to virtual machine paging operations because a normal page operation involves a disk I/O. The graphic illustrates how memory compression makes more host memory available in an attempt to prevent paging virtual machine memory out disk. For example, with memory compression, each memory page being considered for a swap operation is compressed and stored in a per-VM compression cache. In the example, memory pages A and B are compressed and stored as half-pages in the compression cache. Although both pages are removed from virtual machine guest memory, the actual reclaimed memory size is one page.


When a virtual machine is powered on, the system allocates a VMkernel swap file for it. The VMkernel swap file serves as a backing store for the virtual machine’s RAM contents. If a virtual machine cannot get enough memory through ballooning, the VMkernel forcibly reclaims memory from other virtual machines. The VMkernel copies the contents of the pages of these virtual machines to their corresponding swap files before giving the pages to the virtual machine that needs memory.

The size of the VMkernel swap file is determined by the difference between the virtual machine’s configured memory (or its memory limit) and its reservation.

Whenever VMkernel swap is being actively used, performance is not optimal. A good practice is to configure your hosts so that all normal running memory needs of the virtual machines (as determined by monitoring under load) can be accommodated using physical memory.

When the virtual machine is powered off, the VMkernel swap file of the virtual machine is deleted. When the virtual machine is powered back on, the VMkernel swap file for the virtual machine is re- created.