Free command output Parsing

From: http://www.cnblogs.com/iroultron/archive/2011/12/25/2301362.html

 

I believe many of my colleagues are not very familiar with the buffers cached in the free command. Today I will make a detailed analysis of the results of the free command machine.

The following is the free situation of the Debian system running on the virtual machine. The VM has only 128 MB of memory and is allocated with a swap zone of MB.

Total used free shared buffers cached

Memory: 124788113432 11356015308 75568

-/+ Buffers/cache:22556 102232

Swap: 245752 20 245732

The first line (which can be understood as memory allocation from the Operating System Perspective)

Total	Total physical memory size.
Used	Allocated size. Note that for the operating system, any memory used is used.
Free	Unallocated physical memory size.
Shared	The shared memory size is mainly used for inter-process communication. For example, Oracle uses the shared memory of GB.
Buffers	It is mainly used for data buffering of Block devices, such as recording metadata (directory, permission, and other information) of the file system ).
Cached	It is mainly used for file content buffering.

# Here I want to add some knowledge about file systems ^_^

# The file system can be roughly divided into two parts:

# (1) Metadata: metadata, including the directory structure, file name, size, modification time, and permissions.

# (2) filedata: The content actually stored in the file.

# Because the speed of various components in the computer varies greatly, the CPU cache is usually the fastest, followed by the memory, and the slowest is the hard disk and other external devices.

# The slower the device, the cheaper it is, therefore, the hard disk is used to store a large amount of data. (If you don't need to pay for the memory one day, you don't have to be suffering from buffer or cache. <(￣)> ).

# Therefore, the file system is placed on the hard disk (inevitable tragedy ~~). However, the hard disk speed is too slow. To improve Io performance, the concept of "cache" was born.

# "Cache" refers to dividing a block in the memory as a buffer area between the hard disk and the process. Write the data here and then do other things, the read process needs to be searched here before finding the hard disk. This greatly improves the Read and Write efficiency and shortens the IO wait time, otherwise, when you run commands such as man iptables on Linux, you will wait for n times...

# There is also a concept of "physical memory" and "virtual memory". This concept is hard to explain. Let's go to some Google posts and see it. There are too many things involved ....

So here, buffer is actually used to store the metadata of the file system (so that every time we ls will soon become faster ), the cache caches the content of recently read/written files (the first time the file is read, it will be very slow, and it will be very soon in the future, because there is a cache ), I understood this free command and understood it for more than half.

Continue...

The second line (which can be understood as memory allocation from the application perspective)

-/+ Buffers/cached:22556 102232

Here,-/+ actually refers to-Buffers/cached and + buffers/cached.

-Buffers/cached	= Used (first line)-Buffers-cached	It is actually the size of the "physical memory" currently "actually used" by the program.
+ Buffers/cached	= Buffers + cached	The two values can be understood as the memory size used by the "temporarily lent" system as the "buffer.

You can simply add and subtract the following:

From the perspective of the application, the system can actually allocate so much memory to it:11356 (free) + 15308 (buffers) + 75568 (cached) = 102232 (free)

The total physical memory size of the system113432 = 22556 (-Buffers/cached) + 102232 (+ buffers/cached)

Therefore, the first line of data is actually from the operating system perspective.

For the operating system, the allocated memory is occupied by some data, but the specific data is used for nothing.

In fact, as mentioned above, this part of physical memory is used as a "cache.

But is the application unusable?

Of course not !!!!

Otherwise, I don't need to write it anymore. Everyone understands it...

As mentioned above, the buffers and cached memory is "Lent" to the operating system. The reason is "borrowed ", the main reason is the difference between "Virtual Memory" and "physical memory.

For the application, I applied for a kb memory and the operating system agreed. I gave the program an address and told him that the desired memory exists here. Then the program took the address and used it.

But in fact, most of the time the program will not immediately use the memory, but use it at, so before it is used, the system can take this part of physical memory to do other things, which has improved the performance of the entire system, for example, the "cache" here ".

Of course, this understanding of "Virtual Memory" is veryOne-sidedThe actual situation is much more complex than this. For example, this involves the program runtime space, code compilation, linking, loading, and so on. It must be combined to explain why virtual memory is needed, in fact, "Virtual Memory" is an abstract concept, which greatly simplifies the design complexity of the operating system and applications.

Although I want to talk too much about it, I can understand the concept of "Virtual Memory" here. Pai_^

Here, the operating system is equivalent to "borrowing" some memory to improve the system's Io performance.

For "Virtual Memory" and "physical memory", you can use the PS aux command for a brief look:

User PID % CPU % memVsz RSSTty stat Start Time Command

Root 1471 0.0 0.32152 428? S/usr/bin/daemon/etc/init. d/MPT-statusd check_mpt

Root 1474 0.0 0.41748 556? S/bin/sh/etc/init. d/MPT-statusd check_mpt

The vsz column shows the size of the "virtual memory" allocated by the corresponding program, while RSS is the memory actually used by the program. vsz is actually larger than RSS, it indicates that the program does not use the amount of memory he has applied for. If it is allocated, it will definitely be idle... Your machine cannot view webpages while running QQ as it is now... Opening chrome will be finished ..

Careful colleagues found that the above-mentioned "borrow" process is very similar to the banking business. The cash in hand is stored in the bank, and the bank gives us a card, there is a card number on the card. The bank says, "You can get the money at any time with this card and password ". Then we can safely store the money in the bank. when we need the money, we will take the card to get the money. At this time, the bank will also take our money to invest and build bridges.

The only difference may be that the Bank has interest ~~~~~

Go back to the topic.

Well, since we say that the memory is "borrowed", of course some programs will take the address to get the memory (just like taking the money ).

At this time, the operating system will find the free memory from the free in the first line for the program.

But you must have discovered that most of the memory is occupied by buffers and cached. What should you do if you have a large memory demand?

The answer is very simple. Take Back the "loan" that has been released ~~~~

Then release the memory from buffers and cached, and write the data in the released space back to the hard disk to prevent data loss.

What if buffers and cached are gone? The "bank" only needs to borrow money from other places ~~~

This "borrow" refers to borrowing from the hard disk, that is, allocating swap space, and switching some memory pages to swap space.

Of course, the actual processing system will not wait until buffers and cached are fully recycled to use the swap space, which is much easier than you think, because sometimes the performance cost of narrowing down the cache is higher than that of using swap, the Linux system is still very smart ~~~

Come back and continue.

When will the system start to recycle the memory of buffers and cached?

Here we will look at the data:

Root @ debian01 :~ #CAT/proc/meminfo

Memtotal: 124788 KB

Memfree: 11596 KB

Buffers': 15332 KB

Cached: 75568 KB

Swapcached: 20 KB

Active: 54884 KB

Inactive: 45564 KB

Active (Anon): 2124 KB

Inactive (Anon): 7560 KB

Active (File): 52760 KB

Inactive (File): 38004 KB

Unevictable: 0 KB

Mlocked: 0 KB

Hightotal: 0 KB

Highfree: 0 KB

Lowtotal: 124788 KB

Lowfree: 11596 KB

Swaptotal: 245752 KB

Swapfree: 245732 KB

Dirty: 0 KB

Writeback: 0 KB

Anonpages: 9544 KB

Mapped: 5884 KB

Shmem: 136 KB

Slab: 8784 KB

Sreclaimable: 4940 KB

Sunreclaim: 3844 KB

Kernelstacks: 528 KB

Pagetables: 456 KB

Nfs_unstable: 0 KB

Bounce: 0 KB

Writebacktmp: 0 KB

Commitlimit: 308144 KB

Committed_as: 43272 KB

Vmalloctotal: 897016 KB

Vmallocused: 6344 KB

Vmallocchunk: 879908 KB

Hardware0000upted: 0 KB

Hugepages_total: 0

Hugepages_free: 0

Hugepages_rsvd: 0

Hugepages_surp: 0

Hugepagesize: 4096 KB

Directmap4 KB: 16384 KB

Directmap4m: 114688 KB

Meminfo has a lot of data, so we will not elaborate on it here. We only pay attention to one data.

Lowfree: 11596 KB

In fact, when the free part of memory is smaller than this value, the system will consider whether to recycle the physical memory of the "cache" part. Of courseThis is not absolute, Which can be used as a reference threshold.

OK, the last part.

After talking about this, is there any way to force release ??

Of course !!! If the bank owes you money, you don't want it ???

The solution is actually very simple. Just execute the following command on the console:

Echo 3>/proc/sys/Vm/drop_caches

"3" indicates that all the parts that can be released in buffers and cached are released.

Root @ debian01 :~ # Echo 3>/proc/sys/Vm/drop_caches

Root @ debian01 :~ # Free

Total used free shared buffers cached

Mem: 124788 26764980240148 6144

-/+ Buffers/cache: 20472 104316

Swap: 245752 20 245732

After manual release, you can test it. If you run the commands similar to the following several times, you will obviously find that it has gone up again...

Ls-La/dev/

Find/-name *. Sh

Man iptables

Root @ debian01 :~ # Free

Total used free shared buffers cached

Mem: 124788 8736837420030692 25512

-/+ Buffers/cache: 31164 93624

Swap: 245752 20 245732

OK. Here we will talk about free-related things...

It has been written for a long time ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~