Linux ulimit command

Original article link

In Linux, core files are not generated by default. You must open them with ulimit-C unlimited.

Overview

System performance has always been a topic of interest. How to achieve the most effective performance tuning through the simplest settings and how to ensure the performance with limited resourcesProgramUlimit is a simple method that we often use when dealing with these problems. Ulimit is an internal key function in Linux. It has a set of parameters used to set limits on the usage of shell processes generated by it and their sub-processes. This article will explain in detail the ulimit function, usage, and impact in later sections, and explain in detail the impact of ulimit on resource usage restrictions with examples.

Ulimit functions and usage

Ulimit Functions

Assume that 10 users log on to a Linux host at the same time, and the 10 users open 500 documents at the same time with unlimited system resources, assuming that the size of each document is 10 MB, the system's memory resources will be greatly challenged.

The actual application environment is much more complex than this assumption. For example, in an embedded development environment, all aspects of resources are very scarce. For the number of file descriptors enabled, the size and size of stack allocation, CPU time, virtual memory size, and so on all have very strict requirements. Reasonable resource limits and allocation are not only necessary to ensure system availability, but also closely related to the performance of software running on the system. At this time, ulimit can play a major role, it is a simple and effective way to implement resource restrictions.

Ulimit is used to restrict the resources occupied by Shell startup processes. It supports the following types of restrictions: the size of the created Kernel File, the size of the process data block, the size of the file created by the Shell Process, the size of the Memory Lock, the size of the resident memory set, the number of opened file descriptors, And the allocation stack. the maximum size, CPU time, the maximum number of threads for a single user, and the maximum virtual memory that a shell process can use. At the same time, it supports restrictions on hard and soft resources.

As a temporary limit, ulimit can be used to log on to shell sessions by using its command. The limit ends when the session ends and does not affect other shell sessions. For long-term fixed restrictions, the ulimit command statement can be added to a file read by the login shell, acting on a specific shell user.

Figure 1. Use of ulimit 

The following sections describe how to use ulimit to limit resources.

How to Use ulimit

Ulimit uses some Parameter options to manage different types of system resources. In this section, we will explain how to use these parameters.

Ulimit Command Format: ulimit [Options] [limit]

For more information about options and simple examples, see the following table.

Table 1. ulimit parameter description

Option [Options]	Description	Example
-H	Set hard resource limits. Once set, they cannot be added.	Ulimit-HS 64; hard resources are restricted. The thread stack size is 64 KB.
-S	Set soft resource limit. After setting, you can add but cannot exceed the hard resource setting.	Ulimit-Sn 32; restrict soft resources and 32 file descriptors.
-	Displays all current limit information.	Ulimit-A; displays all current limit information.
-C	The maximum core file size, in blocks.	Ulimit-C unlimited; the size of the generated core file is not limited.
-D	The maximum data segment size of a process, in Kbytes.	Ulimit-D Unlimited; do not limit the Data Segment Size of the process.
-F	Maximum number of files that can be created by a process, in blocks.	Ulimit-F 2048; restrict the maximum file size that can be created by a process to 2048 blocks.
-L	Maximum lockable memory size, in Kbytes.	Ulimit-l 32; the maximum lockable memory size is 32 Kbytes.
-M	The maximum memory size, in Kbytes.	Ulimit-M unlimited; maximum memory is not limited.
-N	Maximum number of file descriptors that can be opened.	Ulimit-N 128; a maximum of 128 file descriptors can be used.
-P	The size of the MPs queue buffer, in Kbytes.	Ulimit-P 512; limit the MPs queue buffer size to 512 Kbytes.
-S	Thread stack size, in Kbytes.	Ulimit-s 512; limits the thread stack size to 512 Kbytes.
-T	The maximum CPU usage time, in seconds.	Ulimit-T unlimited; the maximum CPU usage time is not limited.
-U	Maximum number of processes available to users.	Ulimit-u 64; limit that users can use up to 64 processes.
-V	Maximum available virtual memory of a process, in Kbytes.	Ulimit-V 200000; Limit the maximum available virtual memory to 200000 Kbytes.

 

You can use ulimit in the following ways:

• In the user's startup script

  If you are using Bash, you can add ulimit-u 64 to the. bashrc file in the user's directory to limit that you can use up to 64 processes. In addition, you can add ulimt to the startup script equivalent to. bashrc.

• In the application startup script

  If you want to restrict the MyApp of an application, you can write a simple script startmyapp.

  Ulimit-s 512 MyApp

  In the future, you only need to use the script startmyapp to start the application, and then you can limit the thread stack size of the application MyApp to 512 KB.

• Enter
  

  User @ tc511-ui: ~> Ulimit-P 256

  The buffer limit for the MPs queue is 256 K.

Valid range of user processes

As a kind of work that limits the use of resources, ulimit has its own scope. So, is the object restricted by a single user, a single process, or the whole system? In fact, ulimit limits the Current Shell Process and its derived sub-processes. For example, if a user runs two shell terminal processes at the same time and only executes ulimit-S 100 in one environment, the size of the files created in the shell process is limited accordingly, at the same time, another shell terminal, including its running subroutines, will not be affected:

Shell Process 1

Ulimit-S 100 Cat testfile> newfile File size limit exceeded

Shell Process 2

Cat testfile> newfile Ls-s newfile 323669 newfile

 

Is there any way to limit the resources of a specific user? The answer is yes, by modifying the system's/etc/security/limits configuration file. This file not only limits the resource usage of the specified user, but also limits the resource usage of the specified group. Each row of the file is a qualified description in the following format:

<Domain> <type> <item> <value>

 

Domain indicates the user or group name. You can also use * as a wildcard. Type can have two values: soft and hard. Item indicates the resource to be limited. There may be many candidate values, such as stack, CPU, and nofile, which respectively indicate the maximum stack size, CPU usage, and number of opened files. By adding a corresponding line of description, You can have corresponding restrictions. For example:

* Hard noflle 100

 

The configuration statement for this row limits the maximum number of files that can be created by any user to 100.

Now we can limit the resources of processes and users separately. It seems enough, but it is not. Many applications need to impose a general limit on the resource usage of the entire system. In this case, we need to modify the configuration file under/proc. The/proc directory contains many parameters in the current state of the system, such as/proc/sys/kernel/pid_max,/proc/sys/NET/IPv4/ip_local_port_range, etc, you can roughly guess the limited resource type from the file name. This directory contains a large number of files, which are not described here. If you are interested, you can open the relevant documents to view the instructions.

Example of ulimit system resource management

Ulimit allows you to restrict system resources in Shell processes. This chapter lists some examples of using ulimit to restrict user processes, describes these restrictions and their impacts, and describes how ulimit limits system resources, this allows you to adjust system performance.

Use ulimit to limit shell memory usage

This section shows you how to use the-D,-M, and-V options to limit the memory used by shell.

First, let's take a look at the call of the LS command without setting the ulimit limit:

Figure 2. LS command usage when ulimit is not set 

You can see that the LS command is running normally at this time. Set ulimit as follows:

>Ulimit-D 1000-M 1000-V 1000

 

Next, let's take a look at the meanings of the three options described in the previous chapter:

-D: set the maximum value of the data segment. Unit: kb.

-M: set the maximum value of resident memory that can be used. Unit: kb.

-V: sets the maximum value of virtual memory. Unit: kb.

With the above ulimit settings, we have set the maximum memory size that can be used by the current shell to less than kb. Next, let's take a look at the results of running the LS command:

Haohe @ sles10-hehao :~ /Code/ulimit> ls test-l /Bin/LS: Error while loading shared libraries: libc. so.6: failed to map segment From shared object: cannot allocate memory

 

As shown in the preceding result, the LS operation fails. According to the error message provided by the system, we can see that the LS error is caused by memory allocation failure when the libc library is called. Let's take a look at the size of the libc library file:

Figure 3. view the libc file size 

According to the information above, the size of the libc library file is 1.5 MB. The upper limit of memory usage set by ulimit is 1.5 kb, which is less than MB. This fully proves the function of ulimit to limit shell memory usage.

Use ulimit to limit the size of files created by Shell

Next, we will show you how to use the-F option to limit the file size that can be created by shell.

First, let's take a look at the situation where ulimit-F is not set:

Figure 4. view files 

The testfile size of an existing file is 323669 bytes. Now we use the cat command to create a copy of testfile:

Figure 5. Create a copy when ulimit is not set 

From the above output, we can see that we have successfully created a copy newfile of testfile.

Next we will set ulimt-F 100:

> Ulimit-F 100

-F indicates that it is used to set the maximum value of a file that can be created by shell. The Unit is blocks.

Now let's execute the same Copy command to see what the result will be:

Figure 6. Create a copy when setting ulimit 

Failed to copy the testfile. The file size exceeds the limit when an error message is displayed. The default size of the next block in Linux is 512 bytes. Therefore, the above ulimit means to limit the maximum size of files that can be created by Shell to 512x100 = 51200 bytes and smaller than 323669 bytes. Therefore, the file creation fails and meets our expectations. This example shows how to use ulimit to control the maximum file that can be created by shell.

Use ulimit to limit the number of sockets that a program can create

Consider a realistic demand. For a server program in the C/S model, it creates multiple socket ports for multiple client program requests to respond. If a large number of clients send requests to the server at the same time, the server needs to create a large number of socket connections. However, in a system, it is often necessary to limit the maximum number of sockets that a single server program can use for other server programs. How can we achieve this? The answer is that we can use ulimit! Careful readers may find that the previous section does not seem to limit the ulimit option used by socket. Yes, ulimit does not have any option to directly limit the number of sockets. However, we have the-n option, which is used to limit the maximum value of the file descriptor that a process can open. In Linux, all resources are files. Common files are files, disk printers are files, and socket files are also files. Creating a new socket connection in Linux is actually creating a new file descriptor. As shown in (view the file descriptor information currently opened by a process ):

Figure 7. view the process opening file descriptor 

Therefore, we can use ulimit-N to limit the maximum number of file descriptors that a program can open, so as to limit the number of socket creation.

Use ulimit to limit the size of the shell multi-threaded program stack (increase the number of available threads)

In the last example, we will introduce how to use-s (unit: KB) to limit the thread stack size, thus reducing the memory usage of the entire multi-thread program, increase the number of available threads. This example is taken from a real case. The problem we encountered was that the system had the following restrictions on our multi-threaded program:

Ulimit-V 200000

As described earlier in this article, this means that our program can only use less than 200 MB of virtual memory. Because our program is a multi-threaded program, the program will create new threads as needed during runtime, which will inevitably increase the total memory demand. At the beginning, we set a stack size limit of 1024 (1232 is used in this example ):

# Ulimit-s 1232

 

When our program starts, we can use pmap to view its memory usage. We can see multiple data segments that occupy kb. These are the stacks used by the threads created by the Program:

Figure 8. Stack used by the program thread 

Every time a new thread is created, we need to allocate a new memory space of 1232kb, and our total virtual memory limit is 200 MB. So if we need to create more threads, one way to improve is to reduce the fixed stack size of each thread, which can be achieved through ulimit-S:

# Ulimit-s 512

 

We set the stack size to kb. Then we can use pmap to check whether our settings work:

Figure 9. Set the stack size after ulimit 

From the above information, we can see that we have successfully changed the thread stack size to kb, so that when the total memory usage limit remains unchanged, we can use the methods described in this section to increase the number of threads that can be created to improve the multi-thread performance of the program.

Summary

To sum up, the ulimit command in Linux provides a convenient way for resource restrictions and system performance optimization. You can use this command to restrict the use of system resources from the user's shell STARTUP script, application startup script, and directly on the console, including the size of the created Kernel File, the size of the process data block, the size of the file created by the Shell Process, the size of the Memory Lock, the size of the resident memory set, the number of opened file descriptors, And the allocation the maximum stack size, CPU time, the maximum number of threads for a single user, and the maximum virtual memory that a shell process can use, and so on. The example in this article intuitively illustrates the use of ulimit and its effects. It is obvious that ulimit is very practical for our applications and development work on the Linux platform.