T-Mem

Transcendent Memory and Linux

Dan Magenheimer, Chris Mason, Dave McCracken, Kurt Hackel Oracle Corp.

first.last@oracle.com

Abstract

Managing a fixed amount of memory (RAM) optimally is a long-solved problem in the Linux kernel. Managing RAM optimally in a virtual environment, however, is still a challenging problem because: (1) each guest kernel focuses solely on optimizing its entire fixed RAM allocation oblivious to the needs of other guests, and (2) very little information is exposed to the virtual machine manager (VMM) to enable it to decide if one guest needs RAM more than another guest. Mechanisms such as ballooning and hot-plug memory (Schopp, OLS’2006) allow RAM to be taken from a selfish guest and given to a needy guest, but these have significant known issues and, in any case, don’t solve the hard problem: Which guests are selfish and which are needy? IBM’s Collaborative Memory Management (Schwidefsky, OLS’2006) attempts to collect information from each guest and provide it to the VMM, but was deemed far too complex and attempts to upstream it have been mostly stymied. Transcendent Memory (tmem for short) is a new approach to optimize RAM utilization in a virtual environment. Underutilized RAM from each guest, plus RAM unassigned to any guest (fallow memory), is collected into a central pool. Indirect access to that RAM is then provided by the VMM through a carefully crafted, page-copy-based interface. Linux kernel changes are required but are relatively small and not only provide valuable information to the VMM, but also furnish additional "magic" memory to the kernel, provide performance benefits in the form of reduced I/O, and mitigate some of the issues that arise from ballooning/hotplug.

system at any given time, a large percentage of RAM is sitting unused or idle. But this RAM is not really empty; Linux — and any modern operating system — uses otherwise idle RAM as a page cache, to store pages from disk that might be used at some point in the...