Swapniess and overcommit and other hadoop optimization, vm. overcommitmemory

Source: Internet
Author: User

Swapniess and overcommit and other hadoop optimization, vm. overcommitmemory
The size of the generated file, in KB.

Ulimit-f 1
Dd if =/dev/zero of1_1g.txt bs = 1023 count = 1 OK
Dd if =/dev/zero of1_1g.txt bs = 1024 count = 1 OK
Dd if =/dev/zero of1_1g.txt bs = 1025 count = 1 Not Allowed
Ulimit-f ulimit

Limits the cpu usage time

Ulimit-t 1

Echo 'LS-aRl/'> 1.sh & bash 1.sh the program is automatically killed after it is run for a while.

Explanation of firefoxbug overcommit

Http://www.firefoxbug.com/index.php/archives/2800/
Http://blog.csdn.net/houjixin/article/details/46412557
Http://linuxstorm.blog.51cto.com/5218690/1609334

cat /proc/sys/vm/overcommit_memoryecho 0 > /proc/sys/vm/overcommit_memorycat /proc/sys/vm/overcommit_memory

Http://blog.csdn.net/metaxen/article/details/7188689
Http://www.51testing.com/html/87/43487-204904.html
Http://www.vpsee.com/2013/10/how-to-configure-the-linux-oom-killer/

Dynamic Link Library

Http://m.blog.csdn.net/blog/zhangmiaoping23/45675119 # write very good

Swapniess update

Http://www.slideshare.net/technmsg/improving-hadoop-performancevialinux? Next_slideshow = 1
Http://www.pythian.com/blog/what-exactly-is-swappiness/
Http://www.linuxvox.com/2009/10/what-is-the-linux-kernel-parameter-vm-swappiness/
Different http://lwn.net/Articles/100978/ kernels have different interpretations of swapniess

# I personally think there is no big difference between repair and modification. because swap is not widely used in clusters, or very few swap is used. cat/proc/sys/vm/swappinessecho 10>/proc/sys/vm/swappinessecho "vm. swapniess = 10 ">/etc. sysctl. conf
THP
cat /sys/kernel/mm/redhat_transparent_hugepage/defragcat /sys/kernel/mm/redhat_transparent_hugepage/enabledecho never > /sys/kernel/mm/redhat_transparent_hugepage/defragecho never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
Nic upgrade

After a mega frame is opened, it can be increased by 10% ~ 20%

Page cache

Echo 1>/proc/sys/vm/drop_caches
Dd if =/dev/zero of = 512M-file count = 512000 bs = 1 k
Time cat 512M-file &>/dev/null
Echo 1>/proc/sys/vm/drop_caches
Time cat 512M-file &>/dev/null

Copyright Disclaimer: This article is an original article by the blogger and cannot be reproduced without the permission of the blogger.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.