Linux Command-based ulimit

System performance has always been a topic of interest. How can we achieve the most effective performance tuning through the simplest settings, and how to ensure program operation with limited resources, ulimit is a simple method that we often use when dealing with these problems.
Ulimit is a built-in function of linux. It has a set of parameters used to set limits on the usage of shell processes generated by it and their sub-processes. This article will explain in detail the ulimit function, usage, and impact in later chapters,
In addition, we will give a detailed example to illustrate its impact on resource usage restrictions.

Ulimit Functions
Assume that 10 users log on to a Linux host at the same time, and the 10 users open 500 documents at the same time with unlimited system resources, assuming that the size of each document is 10 MB, the system's memory resources will be greatly challenged.
The actual application environment is much more complex than this assumption. For example, in an embedded development environment, all aspects of resources are very scarce. For the number of file descriptors enabled, allocate the stack size, CPU time, virtual memory size, and so on,
All have very strict requirements. Reasonable resource limits and allocation are not only necessary to ensure system availability, but also closely related to the performance of software running on the system. At this time, ulimit can play a major role, it is a simple and effective way to implement resource restrictions.
Ulimit is used to restrict the resources occupied by shell startup processes. It supports the following types of restrictions: the size of the created Kernel File, the size of the process data block, the size of the file created by the Shell Process, the size of the Memory Lock, the size of the resident memory set,
The number of file descriptors opened, the maximum size of the stack allocated, the CPU time, the maximum number of threads for a single user, and the maximum virtual memory that the Shell process can use. At the same time, it supports restrictions on hard and soft resources.
As a temporary limit, ulimit can be used to log on to shell sessions by using its command. The limit ends when the session ends and does not affect other shell sessions. For long-term fixed restrictions,
The ulimit command can be added to a file read by the logon shell to act on a specific shell user.

In linux, core files are not generated by default. You must open them with ulimit-c unlimited.
Ulimit uses some Parameter options to manage different types of system resources.
Ulimit Command Format: ulimit [options] [limit]

Table 1. ulimit parameter description

Option [options] Meaning example
-H: Set the hard resource limit. Once set, it cannot be added. Ulimit-Hs 64; hard resources are restricted. The thread stack size is 64 KB.
-S sets the soft resource limit. It can be added after setting, but cannot exceed the hard resource setting. Ulimit-Sn 32; restrict soft resources and 32 file descriptors.
-A displays all current limit information. Ulimit-a; displays all current limit information.
-C: maximum core file size, in blocks. Ulimit-c unlimited; the size of the generated core file is not limited.
-The maximum data segment size of the d process, in Kbytes. Ulimit-d unlimited; do not limit the Data Segment Size of the process.
-F process can create the maximum number of files, in blocks. Ulimit-f 2048; restrict the maximum file size that can be created by a process to 2048 blocks.
-L maximum lockable memory size, in Kbytes. Ulimit-l 32; the maximum lockable memory size is 32 Kbytes.
-Maximum memory size of m, in Kbytes. Ulimit-m unlimited; maximum memory is not limited.
-N indicates the maximum number of file descriptors that can be opened. Ulimit-n 128; a maximum of 128 file descriptors can be used.
-The size of the P-channel buffer, in Kbytes. Ulimit-p 512; limit the MPs queue buffer size to 512 Kbytes.
-S-thread stack size, in Kbytes. Ulimit-s 512; limits the thread stack size to 512 Kbytes.
-T indicates the maximum CPU usage time, in seconds. Ulimit-t unlimited; the maximum CPU usage time is not limited.
-U maximum number of processes available to users. Ulimit-u 64; limit that users can use up to 64 processes.
-V process maximum available virtual memory, in Kbytes. Ulimit-v 200000; Limit the maximum available virtual memory to 200000 Kbytes.

You can use ulimit in the following ways:

In the user's startup script
If you are using bash, you can add ulimit-u 64 to the. bashrc file in the user's directory to limit that you can use up to 64 processes. In addition, you can add ulimt to the startup script equivalent to. bashrc.

In the application startup script
If you want to restrict the myapp of an application, you can write a simple script startmyapp.
Ulimit-s 512
Myapp
In the future, you only need to use the script startmyapp to start the application, and then you can limit the thread stack size of the application myapp to 512 KB.

Enter
User @ tc511-ui: ~> Ulimit-p 256
The buffer limit for the MPs queue is 256 K.

Valid range of user processes
As a kind of work that limits the use of resources, ulimit has its own scope. So, is the object restricted by a single user, a single process, or the whole system? In fact, ulimit limits the Current shell Process and its derived sub-processes.
For example, if a user runs two shell terminal processes at the same time and only executes ulimit-s 100 in one environment, the size of the files created in the shell process is limited accordingly, at the same time, another shell terminal, including its running subroutines, will not be affected:

Is there any way to limit the resources of a specific user? The answer is yes, by modifying the system's/etc/security/limits configuration file. This file not only limits the resource usage of the specified user,
You can also restrict the use of resources in a specified group. Each row of the file is a qualified description in the following format:
<Domain> <type> <item> <value>
Domain indicates the user or group name. You can also use * as a wildcard. Type can have two values: soft and hard. Item indicates the resource to be limited. There may be many candidate values, such as stack, cpu, and nofile,
The maximum stack size, cpu usage, and number of opened files. By adding a corresponding line of description, You can have corresponding restrictions. For example:
* Hard noflle 100
The configuration statement for this row limits the maximum number of files that can be created by any user to 100.

Now we can limit the resources of processes and users separately. It seems enough, but it is not. Many applications need to impose a general limit on the resource usage of the entire system. In this case, we need to modify the configuration file under/proc.
The/proc directory contains many parameters in the current state of the system, such as/proc/sys/kernel/pid_max,/proc/sys/net/ipv4/ip_local_port_range, etc, you can roughly guess the limited resource type from the file name.
This directory contains a large number of files, which are not described here. If you are interested, you can open the relevant documents to view the instructions.

Example of ulimit system resource management
Ulimit allows you to restrict system resources in shell processes. This chapter lists some examples of using ulimit to restrict user processes, describes these restrictions and their impacts, and describes how ulimit limits system resources, to adjust the system performance.

Use ulimit to limit shell memory usage
This section shows you how to use the-d,-m, and-v options to limit the memory used by shell.
Before limit
[Root @ localhost lianxi] # echo "testwwww"> test
[Root @ localhost lianxi] # ll test
-Rw-r --. 1 root 9 May 10 10:21 test
After restriction
[Root @ localhost lianxi] # ulimit-d 1000-m 1000-v 1000
[Root @ localhost lianxi] # ll test
Segmentation fault
The above code is run on centos6.5-64 bits, which may be inconsistent with the error messages of other versions.

With the above ulimit settings, we have set the maximum memory size that can be used by the current shell to less than kb.
As shown in the preceding result, the ls operation fails. According to the error message provided by the system, we can see that the ls error is caused by memory allocation failure when the libc library is called. Let's take a look at the size of the libc library file:
According to the information above, the size of the libc library file is 1.5 MB. The upper limit of memory usage set by ulimit is 1.5 kb, which is less than MB. This fully proves the function of ulimit to limit shell memory usage.

Use ulimit to limit the size of files created by shell
Next, we will show you how to use the-f option to limit the file size that can be created by shell.
Any size of files can be created before the limit
[Root @ localhost lianxi] # cp win2k3_r2_ent_sp1_cd2.iso test1
[Root @ localhost lianxi] # ll
Total 273980
-Rw-r --. 1 root 9 May 10 10:21 test
-Rw-r --. 1 root 140273664 May 10 test1
-Rw-r --. 1 root 140273664 Jul 31 2006 win2k3_r2_ent_sp1_cd2.iso
After restriction
[Root @ localhost lianxi] # ulimit-f 100000
[Root @ localhost lianxi] # cp win2k3_r2_ent_sp1_cd2.iso test3
File size limit exceeded

Use ulimit to limit the number of sockets that a program can create
Consider a realistic demand. For a server program in the C/S model, it creates multiple socket ports for multiple client program requests to respond. If a large number of clients send requests to the server at the same time,
In this case, the server will need to create a large number of socket connections. However, in a system, it is often necessary to limit the maximum number of sockets that a single server program can use for other server programs.
How can we achieve this? The answer is that we can use ulimit! Careful readers may find that the previous section does not seem to limit the ulimit option used by socket. Yes,
Ulimit does not directly limit the number of sockets. However, we have the-n option, which is used to limit the maximum value of the file descriptor that a process can open. In Linux, all resources are files,
A common file is a file, a disk printer is a file, and a socket is also a file. Creating a new socket connection in Linux is actually creating a new file descriptor.

776? 00:00:00 rsyslogd
[Root @ localhost fd] # cd/proc/776/fd
[Root @ localhost fd] # ll
Total 0
Lrwx ------. 1 root 64 May 10 0-> socket: [10173]
L-wx ------. 1 root 64 May 10 :53 1->/var/log/secure
L-wx ------. 1 root 64 May 10 :53 2->/var/log/maillog
Lr-x ------. 1 root 64 May 10 :53 3->/proc/kmsg
L-wx ------. 1 root 64 May 10 :53 4->/var/log/messages
L-wx ------. 1 root 64 May 10 :53 5->/var/log/cron

840? 00:00:00 sshd
[Root @ localhost fd] # cd/proc/840/fd
[Root @ localhost fd] # ll
Total 0
Lrwx ------. 1 root 64 May 10 0->/dev/null
Lrwx ------. 1 root 64 May 10 :53 1->/dev/null
Lrwx ------. 1 root 64 May 10 :53 2->/dev/null
Lrwx ------. 1 root 64 May 10 3-> socket: [10348]
Lrwx ------. 1 root 64 May 10 4-> socket: [10352]

Therefore, we can use ulimit-n to limit the maximum number of file descriptors that a program can open, so as to limit the number of socket creation.

Use ulimit to limit the size of the shell multi-threaded program stack (increase the number of available threads)
In the last example, we will introduce how to use-s (unit: KB) to limit the thread stack size, thus reducing the memory usage of the entire multi-thread program, increase the number of available threads. This example is taken from a real case.
The problem we encountered was that the system had the following restrictions on our multi-threaded program:
# Ulimit-v 200000
As described earlier in this article, this means that our program can only use less than 200 MB of virtual memory. Because our program is a multi-threaded program, the program will create a new thread as needed during runtime,
This will inevitably increase the total memory demand. At the beginning, we set a stack size limit of 1024 (1232 is used in this example ):
# Ulimit-s 1232
When our program starts, we can use pmap to view its memory usage. We can see multiple data segments that occupy kb. These are the stacks used by the threads created by the Program:
# Pmap-x pid
Every time a new thread is created, we need to allocate a new memory space of 1232KB, and our total virtual memory limit is 200 MB. So if we need to create more threads, one way to improve is to reduce the fixed stack size of each thread,
This can be achieved through ulimit-s:
# Ulimit-s 512
We set the stack size to kb. Then we can use pmap to check whether our settings work:
# Pmap-x pid
From the above information, we can see that we have successfully changed the thread stack size to kb, so that when the total memory usage limit remains unchanged, we can use the methods described in this section to increase the number of threads that can be created to improve the multi-thread performance of the program.

To sum up, the ulimit command in linux provides a convenient way for resource restrictions and system performance optimization. From the user's shell STARTUP script, the application startup script, and directly on the console,
You can use this command to restrict the use of system resources, including the size of the created Kernel File, the size of the process data block, the size of the file created by the Shell Process, the size of the Memory Lock, the size of the resident memory set, the number of opened file descriptors,
The maximum size of the stack allocated, the CPU time, the maximum number of threads for a single user, the maximum virtual memory that the Shell process can use, and so on. The example in this article intuitively illustrates the use of ulimit and its effects,
Obviously, ulimit is very practical for our applications and development on the Linux platform.

840? 00:00:00 sshd
[Root @ localhost fd] # pmap-x 840

[Root @ localhost ~] # Ulimit-a: Display All set values