Linux swap space mini-HOWTO

versions:  2002-Oct-25,
2002-Nov-01,04,07,08,10,11,12,17
2003-Jun-13
Randy Dunlap <rddunlap AT osdl.org>
======================================================================

Copyright (c) 2002-2003 Randy Dunlap.

This document may be distributed only subject to the terms and
conditions set forth in the LDPL (Linux Documentation Project License)
at "http://www.tldp.org/COPYRIGHT.html".


1.  Introduction

Linux uses swap space as "extra" memory for pages of application
memory that are not being actively used by the application but have
been modified (written to).  The swap space size plus RAM size is the
total amount of virtual memory for the system.  When most of the system's
real memory is in use, and there is a need for more, some data will be
moved into swap to free real RAM memory for use by applications or for
kernel use, such as for driver buffers, files, or network packets.
This is called swapping out.  When the data that is in swap space needs
to be used it is swapped back in from swap space.  The rate at which data
is swapped to and from one or more swap spaces can be monitored with the
'vmstat' command's swap-in (si) and swap-out (so) columns.

All of this is triggered by a high demand for memory.  That can include
running many/large programs, but even file or network I/O require
allocating memory and can trigger the kernel to scavenge for more
memory, which can trigger swapouts.

Linux kernel code and data are not swappable and are never moved to
swap.  User code never needs to be written to swap space because it
already exists on disk and can be read in from there if it is required
again.  User data can be written to swap space and read back in when
needed.

A possible alternative to swap space that is allocated statically
is "swapd", a "dynamic swapping manager for Linux."  It is available
from URL::http://ftp.linux.hr/pub/swapd/ .  One person reports using
swapd exclusively on several Linux systems, without static swap space.
He reports that it works reliably and consumes almost no resources.
swapd is tuned by editing the "swapd.conf" file.  Example file:

memlimit 131072
pause 10000
swapsize 65536
timeout 8640

2.  How much swap space to use

That's tough to say.  If your system (and applications) are using
lots of memory (more than the amount of physical memory in your
system), then you needs lots of swap space.

Of course you may decide that your system's environment shouldn't
support (use) swap space, either because of performance issues, or
you don't have any [fast] storage devices available for swapping, or
because you have large RAM and will never use it all up, so you
should never see a need for swapping.

Typical total swap space size is equal to your system's RAM size.
Early versions of Linux 2.4 (through 2.4.9) liked swap space size
of 2 * system RAM size, but that's no longer the case.

One negative about providing too much swap space is that errant
programs may run for an extended time and use up RAM and swap space.
This probably causes system slowness due to excessive swapping, and
could lead to the wrong process being killed by the OOM killer when
the kernel runs out of memory (see the OOM Killer section below).
This failure can be prevented by not defining huge amounts of swap
space unless you know that it is required.  In other words, swap
space can be an area for system slowness, so allocate it wisely.

Swap space usage can be monitored with the 'free' command or by
reading the /proc/swaps and /proc/meminfo files.

The install programs of most Linux distributions do a reasonable job
of allocating swap space for small to medium systems.

3.  Swap space limits

[Historical:]
Linux 2.4.10 and later, and Linux 2.5 support any combination of swap
files or swap devices to a maximum number of 32 of them.
Prior to Linux 2.4.10, the limit was any combination of 8 swap files
or swap devices.  On x86 architecture systems, each of these swap
areas has a limit of 2 GiB.

The current swap space limits are:

(1) both Linux 2.4.x and 2.5.x kernels support swap spaces of up to
    64 GiB in size

(2) Linux 2.4.x supports 32 swap spaces and 2.5.x supports 32 swap
    spaces

(3) mkswap(8) needs fixes for creating swap spaces larger than 2 GiB
    merged back to the util-linux package from one or more distro
    packages; this work is pending.  Some distro versions of mkswap(8)
    support more than 2 GiB swap spaces.

4.  OOM (Out-of-Memory) killer

If the Linux VM can't find memory to allocate when it's needed, it
puts in-use user data pages on the swap-out queue, to be swapped out.
If the VM can't allocate memory and can't swap out in-use memory,
the Out-of-memory killer may begin killing current userspace processes.
The rationale for OOM killing is described in the Linux-MM docs [2].

5.  Setting up swap space

Managing swap with Linux is easy and flexible.  Linux's swap spaces can
be turned on and off without a reboot and even while they are in use.

Swap space can be allocated in its own partition (maybe safer) or in
files in a live filesystem.  Logical Volumes can be used as swap
devices.

There are two (2) versions (formats) of swap space supported by
Linux 2.1 (development kernel) and later (through 2.4).
Linux 2.0 requires "version 0" swap space.  Linux 2.1.117 and
later support both version 0 and version 1 swap space.  Linux 2.5
only supports version 1 swap space.  'mkswap' can format swap space
in either format.  See 'man mkswap' for details.

To use a file in a filesystem for swap space, choose the swap space
size that you want for this swap area (up to 2 GiB on x86) and then
enter (as described in 'man mkswap'):
    dd bs=1024 count=1M if=/dev/zero of=/path/to/swapfile.n
This writes a 1 GiB file (1 MiB * 1024) to "/path/to/swapfile.n".
You can do this multiple times to use more swap files on large-memory
systems.  Swap files cannot be sparse; they must be fully allocated
before using them.

Of course, you can also allocate swap space in its own partition(s).
This usually requires a bit more planning of disk space allocation.

Swap space in partitions or in files is described in the /etc/fstab
file ('man 5 fstab').  Swap spaces can be given different priorities
('man 2 swapon') that determine how the swap spaces will be used.
Higher priority swap spaces will be exhausted first and swap spaces
with equal priorities will be striped.  The priority can be specified
at swap space activation with the 'swapon' command or in the /etc/fstab
file.  For example:

# swap space in device partitions
/dev/hda2        none    swap    pri=5,defaults 0 0
/dev/hde2        none    swap    pri=5,defaults 0 0
# swap space in swap files
/work/swapfile.1 none    swap    pri=5,defaults 0 0
/work/swapfile.2 none    swap    pri=5,defaults 0 0

After swap space is allocated, it must be formatted (initialized)
with swap header information.  This only needs to be done one time.
mkswap /dev/hda2
mkswap /dev/hdb1
mkswap /work/swapfile.1
mkswap /work/swapfile.2

Swap space is enabled during the system init procedure (scripts) by
running "swapon" ('man swapon').  Swap space that is in filesystems
is also enabled at this time if the filesystem volumes are already
mounted.

If you add some swap space after system init, you can use the 'swapon'
command to begin using it.  For example:
swapon /dev/hdf1
swapon /work/swapfile.3

Using multiple swap partitions or swap files on different disks
has an effect of "striping" swap requests across those disks,
similar to RAID-0 striping.

It's generally a good idea to have multiple swap areas (of the same
priority) if you have multiple devices on independent channels.  This
lets the kernel swap in parallel.  It can also be a good idea to keep
swap on less-used devices/channels, so heavy non-swap I/O is not
hindered by swap I/O.

In Linux 2.4 and earlier, swap files are less robust than swap
devices (partitions) because of the need to read metadata from the
filesystem for swap files, making them prone to OOM deadlocks when
allocating pages and buffer_heads to use in swapping.
This has changed in Linux 2.5.  Using a swap file has no disadvantage
compared to swapping to a device (partition).  The kernel doesn't
need to allocate any memory to get a swapcache page to disk.

This is interesting because swap files are much easier to administer
(add, remove, resize) and easier to stripe.  It is now feasible
(in the Linux 2.5.40 timeframe) to eliminate swap devices (partitions)
completely and not be penalized in performance.

If you have two disks then it is very sensible to create a swap file
on each one and perform an equal-priority stripe between them.  This
will give the best raw swap I/O bandwidth, but it could cause additional
seeking between swap and regular file data.

Dedicating an entire disk to swap will obviously reduce the seeking
problem.

But really, if your application is dependent on swap performance, you
need more RAM.  Swap should be viewed as a lightweight background
optimization to make unused pages available for other work, rather
than as a cure for an underprovisioned machine.

6.  Disabling swap space

Swap space is usually turned off (disabled) by system shutdown by
using the 'swapoff' command ('man swapoff').
swapoff /work/swapfile.1
swapoff /work/swapfile.2
swapoff /dev/hdb1
swapoff /dev/hda2
or by using 'swapoff -a', which disables all swap partitions that are
listed in the /etc/fstab file (this does not apply to swap files, only
to swap partitions).

7.  Other uses of swap space

Swap space is used by the "tmpfs" filesystem as overflow storage
whenever cache memory usage needs to be reduced.
tmpfs also supports limit checking (maximum space usage).
See its documentation in the kernel source tree in the file
"linux/Documentation/filesystems/tmpfs.txt".

tmpfs is also used to support "shmfs".
shmfs is a filesystem used to support POSIX shared memory.
glibc 2.2 and later expect tmpfs to be mounted at /dev/shm for POSIX
shared memory open and unlink.

With both tmpfs and shmfs you probably want to limit how big they
can grow.  Use a mount option of size=xxxM to do so.  For example,
in /etc/fstab:

tmpfs           /tmp            tmpfs   size=750M,mode=2777     0 0
shmfs           /dev/shm        tmpfs           defaults        0 0

Swap devices (partitions) can also be used for "software suspend"
(swsusp) in Linux 2.5 and as a destination for saving a kernel crash
dump in versions of Linux which support these features.

Using a swap file(s) also allows you to share swap space with other
OSes as described in the Linux Swap Space Mini-HOWTO [3] and on the
linux-kernel mailing list [4].

CREDITS

Thanks to everyone who commented on initial and updated versions of this
mini HOWTO, including but not limited to:  Andries Brouwer, Bernd Eckenfels,
Mark Hahn, Troels Walsted Hansen, Bill Irwin, Dave Jones, Pavel Machek,
Andrew Morton, Vince Mulhollon, Wladimir Mutel, Ha Shao, Shane Shrybman,
and Ed Tomlinson.  Some of their comments are used or quoted here.

TRANSLATIONS

The original English version of this mini-HOWTO is available at
"http://www.xenotime.net/linux/swap-mini-howto.txt".

A Hungarian translation by Gabor Micsko is available at
"http://www.hup.hu/modules.php?name=News&file=article&sid=1976".

A Polish translation by Jacek Politowski is available at
"http://jacek.rallypl.eu.org/tlumaczenia/swap-mini-howto.txt".

REFERENCES

1.  Joe Knapka, "Outline of the Linux Memory Management System".
    URL::http://home.earthlink.net/~jknapka/linux-mm/vmoutline.html
2.  Rik van Riel:  "Linux-MM docs: the OOM killer"
    URL::http://linux-mm.org/docs/oom-killer.shtml
3.  Swap-Space, Linux Swap Space Mini-HOWTO.  Updated: July 2002.
    How to share your Linux swap partition with Windows. 
    URL::http://www.tldp.org/HOWTO/mini/Swap-Space.html
4.  URL::http://marc.theaimsgroup.com/?l=linux-kernel&m=103623636324081&w=2