Swapniess and overcommit and other hadoop optimization, vm. overcommitmemory

Swapniess and overcommit and other hadoop optimization, vm. overcommitmemory
The size of the generated file, in KB.

Ulimit-f 1
Dd if =/dev/zero of1_1g.txt bs = 1023 count = 1 OK
Dd if =/dev/zero of1_1g.txt bs = 1024 count = 1 OK
Dd if =/dev/zero of1_1g.txt bs = 1025 count = 1 Not Allowed
Ulimit-f ulimit

Limits the cpu usage time

Ulimit-t 1

Echo 'LS-aRl/'> 1.sh & bash 1.sh the program is automatically killed after it is run for a while.

Explanation of firefoxbug overcommit

Http://www.firefoxbug.com/index.php/archives/2800/
Http://blog.csdn.net/houjixin/article/details/46412557
Http://linuxstorm.blog.51cto.com/5218690/1609334

cat /proc/sys/vm/overcommit_memoryecho 0 > /proc/sys/vm/overcommit_memorycat /proc/sys/vm/overcommit_memory

Http://blog.csdn.net/metaxen/article/details/7188689
Http://www.51testing.com/html/87/43487-204904.html
Http://www.vpsee.com/2013/10/how-to-configure-the-linux-oom-killer/

Dynamic Link Library

Http://m.blog.csdn.net/blog/zhangmiaoping23/45675119 # write very good

Swapniess update

Http://www.slideshare.net/technmsg/improving-hadoop-performancevialinux? Next_slideshow = 1
Http://www.pythian.com/blog/what-exactly-is-swappiness/
Http://www.linuxvox.com/2009/10/what-is-the-linux-kernel-parameter-vm-swappiness/
Different http://lwn.net/Articles/100978/ kernels have different interpretations of swapniess

# I personally think there is no big difference between repair and modification. because swap is not widely used in clusters, or very few swap is used. cat/proc/sys/vm/swappinessecho 10>/proc/sys/vm/swappinessecho "vm. swapniess = 10 ">/etc. sysctl. conf

THP

cat /sys/kernel/mm/redhat_transparent_hugepage/defragcat /sys/kernel/mm/redhat_transparent_hugepage/enabledecho never > /sys/kernel/mm/redhat_transparent_hugepage/defragecho never > /sys/kernel/mm/redhat_transparent_hugepage/enabled

Nic upgrade

After a mega frame is opened, it can be increased by 10% ~ 20%

Page cache

Echo 1>/proc/sys/vm/drop_caches
Dd if =/dev/zero of = 512M-file count = 512000 bs = 1 k
Time cat 512M-file &>/dev/null
Echo 1>/proc/sys/vm/drop_caches
Time cat 512M-file &>/dev/null

Copyright Disclaimer: This article is an original article by the blogger and cannot be reproduced without the permission of the blogger.