Unit one Memory Caches
Memory Cache
Learning Goals
:
A. Using memory to improve service times for slow-running subsystems
.
11.1
strategies for using memory
Memory usage Policiesa. reduces the memory consumer load A. slab cache allocates buffer B. reduce or delay the service time of slow-running subsystems A. filesystem Metadata file system Source data: Buffer cache (Slab cache) B. disk Io:page cachec. interprocess Communications interprocess communication: Shared Memory d. network io:buffer Cache, ARP cache, connection trackingc. Adjust memory to pay attention to:a. how to recycle the page to avoid the pressure of memory b. io slow how to do? 1) For small write adjust memory effect is not very obvious 2) for large data write use reordering is more efficient. The main factor affecting disk IO is page cache.
11.2
A Closer look at demand paging
page SchedulingA. Assign a page box to process a only if the process is storing data. A resident page is a B that resides in main memory (RAM). Non-resident pages are placed on disk or not in use. B Allow memory to be used for excessive use of system memory a. Useful in scientific computing applications. B. Address references that facilitate the execution of programs and memory C. Memory (RAM+SWAP) is not used when the process stores data.
11.3
Tuning Page Allocation
page AssignmentA. Parameter setting vm.min_free_kbytes memory minimum reserved space B. Using applications will allocate and use large chunks of memory irregularly. C. Conclusion: A. Reduced service time for page scheduling B. Memory minimum reserved space cannot be used by other programs. C. Memory minimum reserve space is too large, it will cause pressure on the zone_normal.
11.4
Tuning Overcommit
Adjustment
OvercommitA. Parameter setting vm.overcommit_memory:a. 0=henuristic overcommit Default, the program rejects and alarms when run as memory is not allocated. B. 1=always Overcommit the number of applications to be allocated, no matter how much memory is enough. C. 2=commit all swap plus a percentage of RAM (could be >100) Vm.overcommit_ratio The default value is Swp+ram*50%b. Review the Committed_as role in/proc/meminfo: Evaluate how much memory is required to avoid memory overflow when the system is under load. C. Conclusion: A. Allow the kernel to satisfy requests for large virtual address space B. Warning: A process crashes when memory overflows or memory is used too much.
11.5
Slab Cache
Cache AllocationA. Slab is used to store table objects in the kernel. A. Frequent application of objects allocated to the page reduce the burden (slab cache) b. For example: FileSystem metadata (Dentry and inode caches) B. Monitoring: Cat/proc/slabinfoslabtopvmstat–m The following command to see if the kernel supports Slab:grep config_slab/boot/config-*
11.6 ARP Cache ARP
Cache TableA. The ARP table is used to map the hardware address with the protocol address (MAC----IP) A. The ARP table is cached in slab (grep arp/proc/slabinfo) b. The garbage filter periodically deletes the old or outdated table entries in the list. B If the ARP cache is insufficient, it causes: a. Intermittent delay of communication between hosts. B. The ARP table is not stable. C. The ARP cache is too general to cause stress to zone_normal. D. View the ARP table entry: IP Neighbor list cat/proc/net/arp E. emptying the ARP table IP neighbor flush Dev eth0
11.7 Tuning ARP cache ARP
Cache TuningA. Soft upper limit soft limit NET.IPV4.NEIGH.DEFAULT.GC_THRESH2 default value 512 (BAR): ARP table entry number of entries reaches this value after 5 seconds to clear B. Hard upper limit Net.ipv4.neigh.default.gc_thresh3 The default value of 1024C. Table Item garbage cleanup interval (seconds) Net.ipv4.neigh.default.gc_interval Default 30 sec Note: Clears the ARP table every 30 seconds after the limit value 128 is reached
11.8
Page Cache
Page CachingA. Most page activities are generated by I/O operations. A. read file: File read from disk to memory B. These data pages are read in the page cache. B. The page cache always checks the IO request condition A. Directory reads B. Regular file read and write C. Block device files Read and write D. Accessing the memory-mapped file E. Accessing the Swap page. C. Pages in the page cache contain data related to file data.
11.9
Tuning Page Cache
Page Cache TuningA. View the address size/proc/meminfob that is assigned to the page cache. Adjust the length/size of the page cache allocated in memory Vm.lowmem_reserve_ratio the size assigned to the page cache Vm.vfs_cache_pressure virtual memory Reclaim director and INO De buffers the tendency of the larger the more easily recycled. C. Adjust the arrival/completion rate Vm.page-cluster Default value 3,2^3 units: pages, which must be 2^n, how many pages to swap with memory at a time. Vm.zone_reclaim_mode Page Recycling
11.10
Anonymous pages
Anonymous PageA. The anonymous page holds the consumable data b for other processes. The stored data is not related to the file, but includes: a. Program data-array, heap address, and so on B. Anonymous memory range C. Dirty Page D. Shared memory range for interprocess communication C. View usage grep anon/proc/meminfo CAT/PRO/PID/STATMD. Anonymous pages can exchange data with swap. Anonymous pages = rss–shared
11.11
SysV IPC
inter-process communicationA. The following situation consumes memory a more. Semaphores Semaphore: Orchestrate the preemption of shared resources between processes B. Message queues Messages queue: Exchange information between Processes C. Share Memory Share: The data to be accessed between processes is in the same place in memory sb. View SysV shared memory IPCS See what is currently in use Ipcs–l view the limit of shared memory C. Use the POSIX shared memory file system to do one of the fastest storage dd If=/dev/zero 0f=/dev/shm/test bs=1m count=50 shared Memory File system path:/DEV/SHM can read and write data but data is lost after reboot.
11.12
Tuning SysV IPC
Inter-process communication performance tuninga. set semaphore Quantity (flags) Kernel.sem The maximum number of semaphores per semaphore set, default is 250, maximum number of semaphores allowed for the entire system , the default is 32000, each IPC call affects the maximum number of semaphores, the default is 32, the maximum number of semaphore sets, and the default is 128. Note: When running high concurrent applications (e.g. databases), increase the maximum number of semaphores in the system to reduce the scramble for semaphores between processes b. The number of messages and size (non-pageable) kernel.msgmni=16, the maximum number of message queues, the default 16kernel.msgmnb=16384, the maximum number of bytes for a single message queue, the default 16384,kernel.msgmax=8192, The maximum message length (the entire message block size). Default 8192, note: For small amounts of data exchange C. Shared memory size and quantity kernel.shmni=4096, maximum number of system-wide shared memory segments, default 4096, Kernel.shmmax maximum size (in bytes) per shared memory segment: kernel.shmall= 2097152, the number of shared memory pages (the total amount of memory that the system can share at one time: page, shmmax/page_size), default 2097152 Note: Applicable yu Hai data exchange (e.g. Oracle database)
11.13
viewing memory with free
View Memory ReleaseA. Use the FREE-LTM command to see the overall situation of memory usage B. Calculate/buffers/cache units (MiB) used=used–bufers–cachedfree= Free + buffers + cached Grep–i memory/var/log/dmesggrep e820/v Ar/log/dmesg
11.14
Other commands to view memory usage
To view memory-enabled situations other commandsA. View the system memory Cat/proc/meminfocat/proc/zoneinfoa. Total Physical Memory B. Memory cache size C. Used with unused B. Pages Tables page Table CAT/PROC/VMSTATC. Summary Summary vmstat–sd. IO Devices IO device cat/proc/iomem
RHCA442 Learning notes-unit11 Memory Cache
