AIX system space monitoring

Generally, Unix administrators have a set of common tools, techniques, and systems for assisting process management. This article provides a variety of key utilities, command line chains, and scripts used to simplify each process. Some of these tools come from the operating system, and most of the skills come from long-term experience and requirements for reducing the workload of system administrators. This seriesArticleFocuses on maximizing the use of tools available in a variety of UNIX environments, including methods to simplify Task Management in heterogeneous environments.

Getting Started: DF

Monitoring disk space is an important part of UNIX Administrator's work. This article describes some tools required to complete management, including DF, Du, find, and even quota usage. Let's take a look at the powerful functions of DF.

For many administrators, using the DF tool may be a bit like intermittent neural movement, because it provides a snapshot view of all the files that are used and available in the file system through a command. The default output of DF may contain different information based on your UNIX environment.

Most modern DF variants show disk space, usage, and availability, and usually include mount points (sometimes devices ). For example, BSD-based Mac OS X displays information as shown in Listing 1.

Listing 1. Default disk space information of Mac OS X
Filesystem 512-blocks used avail capacity mounted on
/Dev/disk0s2 268435456 119741344 148182112 45%/
Devfs 195 195 0 100%/dev
Fdesc 2 2 0 100%/dev
<Volfs> 1024 1024 0 100%/. Vol
/Dev/disk0s3 43474520 7251448 36223072 17%/volumes/untitled

Note that the displayed size is 512 bytes, not kilobytes. By default, almost all traditional UNIX operating systems report sizes in 512 bytes, rather than bytes. To obtain information in the unit of 1 K, you can use the-K command line option, as shown in Listing 2.

Listing 2. Use the DF command with the-K Option
$ DF-K
Filesystem 1k-blocks used avail capacity mounted on
/Dev/disk0s2 134217728 59870704 74091024 45%/
Devfs 97 97 0 100%/dev
Fdesc 1 1 0 100%/dev
<Volfs> 512 512 0 100%/. Vol
/Dev/disk0s3 21737260 3625724 18111536 17%/volumes/untitled

Some variants may also support optional block sizes, such as megabytes (using-m) and gigabytes (using-g), as shown in listing 3.

Listing 3. Use the DF command with the-G Option
$ DF-G
Filesystem 1g-blocks used avail capacity mounted on
/Dev/disk0s2 128 57 70 45%/
Devfs 0 0 0 100%/dev
Fdesc 0 0 100%/dev
& Lt; volfs & gt; 0 0 0 100%/. Vol
/Dev/disk0s3 20 3 17 17%/volumes/untitled

Obviously, while increasing the size of the displayed block, the details in the information are also reduced, but the optional size is a fast disk monitoring method, especially for ultra-large disk capacity. For other DF tools (especially Solaris), empty blocks and file availability information are reported by default. Most file systems have the maximum number of files that can be stored (this value is so large that it usually does not reach this limit). Therefore, the entire system may be filled with files, there is still available disk capacity, but no more files can be stored.

You can see the default output example of Solaris in Listing 4.

Listing 4. Solaris default output
$ DF
/(/Dev/DSK/c0t0d0s0): 14877208 blocks 914042 files
/Devices (/devices): 0 blocks 0 files
/System/Contract (ctfs): 0 blocks 2147483618 files
/Proc (Proc): 0 blocks 16109 files
/Etc/mnttab: 0 blocks 0 files
/Etc/svc/volatile (SWAP): 5737216 blocks 147177 files
/System/object (objfs): 0 blocks 2147483532 files
/Usr (/dev/DSK/c0t0d0s3): 9076010 blocks 863695 files
/Dev/FD (FD): 0 blocks 0 files
/Var (/dev/DSK/c0t0d0s4): 8110796 blocks 483714 files
/Tmp (SWAP): 5737216 blocks 147177 files
/Var/run (SWAP): 5737216 blocks 147177 files
/Export/Home (/dev/DSK/c0t0d0s7): 69362510 blocks 4272812 files

You can use the-K command line option to switch the display content to a format similar to the previous example, where the data is summarized, obtain a more readable format (see listing 5 ).

Listing 5. Use the-K Option
$ DF-K
Filesystem Kbytes used avail capacity mounted on
/Dev/DSK/c0t0d0s0 7644629 206026 7362157/
/Devices 0 0 0%/devices
Ctfs 0 0 0 0%/system/Contract
Proc 0 0 0%/proc
Mnttab 0 0 0%/etc/mnttab
Swap 2868600 1016 2867584 1%/etc/svc/volatile
Objfs 0 0 0 0%/system/Object
/Dev/DSK/c0t0d0s3 8261237 3723232 4455393/usr
FD 0 0 0 0%/dev/FD
/Dev/DSK/c0t0d0s4 4130238 74849 4014087/var
Swap 2867584 0 2867584 0%/tmp
Swap 2867624 40 2867584 1%/var/run
/Dev/DSK/c0t0d0s7 35611388 930133 34325142 3%/export/home

All DF variants accept a directory or path, and then display the disk space information of the file system containing the path. For example, the following shows how to obtain the space information of the root file system:

$ DF-k/
Filesystem Kbytes used avail capacity mounted on
/Dev/DSK/c0t0d0s0 7644629 206026 7362157/
 

Alternatively, DF is used to display the space information of a user's home directory:

$ DF-K ~ MC
Filesystem Kbytes used avail capacity mounted on
/Dev/DSK/c0t0d0s7 35611388 930133 34325142 3%/export/home
 

DF output

The DF tool is actually short for disk free, and this is exactly what the tool displays. If you view the sample output again, you can extract some important elements from it:

Filesystem Kbytes used avail capacity mounted on
/Dev/DSK/c0t0d0s7 35611388 930133 34325142 3%/export/home
 

The first column usually has the title corresponding to the block size data, which shows the total disk size. The used column shows the number of used blocks on the file system or device. The avail column shows the number of unused (available) blocks on the file system.

The capacity column provides the percentage of disk space used in the total size. This information can provide intuitive instructions on the available space.

By default, 90% of the size of the file system created by UNIX can be used to store user data. The remaining 10 percent is retained by the system, so that the root user can perform emergency maintenance. If you try to add or create a file that exceeds this limit, an error is returned (exceeding the file system space ).

With this urgent maintenance space, you can use it to restore a full disk. For example, as an administrator, you can use this space to create a quick backup or compress existing data without using a secondary File System for this purpose. You can use the tunefs tool or adjust the reserved free space when creating a file system. This is crucial for large-capacity disks, because a buffer of 10 per cent may be equal to the potential space of many gigabytes. A good practice is to reserve at least 1% of the storage space to provide a minimum buffer space before it is fully exhausted. Before you study this issue in depth, especially if you have used a large amount of disk space, you need to determine who used all the space.

Use du

The du command does not display the free space of the disk, but displays the disk usage information. The du tool is used to determine the disk usage of files and directories. To use this tool, change it to a directory and run it (see Listing 6 ).

list 6. du command
$ CD/var
$ du
16. /lost + found
4. /SADM/install/admin
22. /SADM/install/logs
28448. /SADM/install
4. /SADM/PKG/sunwocfd/install
4. /SADM/PKG/sunwocfd/save/pspool/sunwocfd/install
16. /SADM/PKG/sunwocfd/save/pspool/sunwocfd
18. /SADM/PKG/sunwocfd/save/pspool
20. /SADM/PKG/sunwocfd/save
28. /SADM/PKG/sunwocfd
4. /SADM/PKG/sunwcsu/Inst
...

The output shown in Listing 6 has been adjusted as appropriate. By default, du displays the usage of each file and directory in the current or specified directory. The obtained value is the file size. The unit is the default block size in the system, which is the same as the unit used by DF. It may or may not be 1 K, but you can use the-K command line option to force the display in 1 K blocks.

You may need to abstract the information based on the top-level files or directories you view. Use the-s option to open the abstract view. The following is a summary version of Solaris installation displayed in the/var directory:

$ Du-sk
70818.
 

Note that it displays the summary of the current directory. To obtain summary information for all files and directories, use the * wildcard (see listing 7 ).

Listing 7. Use the * wildcard to obtain summary information
$ Du-SK *
382 Adm
950 Apache
683 apache2
6837 appserver
1 Audit
162 Cache
3 cc-Cr
2 crash
4 Cron
31 DMI
22 dt
6 FM
2 imq
1 inet
3 krb5
4 LD
1 LDAP
937 lib
6 log
8 lost + found
2 lp
2 Mail
1 MySQL
1 News
3 NFS
38 NIS
2 NTP
10034 OPT
1 preserve
96 run
49687 SADM
15 SAF
3 samba
2 sma_snmp
131 SNMP
39 Spool
4 statmon
663 SVC
14 TMP
10 uucp
24 yp

Be careful when using this tool, especially when using it in the user directory, because the displayed information does not contain hidden files. In other words, files And Directories With the spof prefix. You may want to use the followingCodeRow to obtain all summary information in the user directory:

$ Du-SK *. [a-zA-Z0-9] *
 

The last very useful option of the du command is-D, which can prevent the du command from exceeding the file system boundary. For example, to determine the disk usage of the root file system rather than any other file system, you can use the-D option:

$ Du-DSK/
 

Some systems do not have this option, but have the-x command line option. This option only contains the File Usage of devices or file systems with the same path as you specified.

Find the disk usage of a specific user

To find the disk space used by a user, you must use the find command together with the du command to only report the disk usage of a specific user.

$ Find.-user Mc-type F-exec Du-K {}\;
 

-The "user" option allows you to specify "find" to only report files belonging to specific users. -The type option forces "find" to return only the path of a specific type of project (in this example, a file). This prevents "du" from containing directories because these directories may belong to a user, but it contains many user files. Then, execute the du command for each path to report disk usage.

To obtain the summary information, that is, the total space used by a specific user, you can use awk to aggregate the information and print the final value:

$ Find.-user Mc-type F-exec Du-K {}\; | awk'{S =
S + $1} end {print"Total used:", S}'
Total used: 123721
 

For groups, you can use the-group option for find based on the same principle:

$ Find.-Group mcslp-type F-exec Du-K {}\; | awk'{S = S + $1} end {print
 "Total used:", S}'
Total used: 542485
 

However, if the disk quota is enabled, there is a simpler method.

Quota

The quota system automatically monitors the disk usage of each user in the file system. The quota environment not only allows you to monitor disk usage, but also allows you to set corresponding limits, when the usage limit is exceeded, a warning is provided or you are directly prohibited from using disk space that exceeds the allocated quota. The lower limit (issuing a warning) is called a soft limit, and the upper limit (prohibiting the creation of files beyond this limit) is called a hard limit. Some systems may also allow you to control the number of files owned by each user.

The exact method for starting a quota on your computer depends on the operating system you are using. By default, most Unix systems support quotas. In Linux, you may need to build a new kernel that includes quota support. However, most operating systems usually use a file called quotas for each file system, which contains space usage restrictions for each user.

To enable the quota, you must first create the file and ensure that only the root user can edit the quota file:

$ Touch/export/home/quotas
$ Chmod 600/export/home/quotas
 

Then, use the quotaon command to enable the quota function:

$ Quotaon/export/home
 

Finally, you must edit the appropriate quota for each user. You can use the edquota command and specify the corresponding user to complete this task:

$ Edquota MC
 

This will open the Default Editor (or VI, if you have not set an optional Editor), which contains a simple table for setting the quota value. As you can see below, the quota has been enabled for the user's home directory file system, and the soft limit is 200,000 kb, and the hard limit is 400,000 kb. The file limit is set to zero, which indicates that no limit is set. You can set and create any number of files.

FS/export/home blocks (soft = 200000, hard = 400000) inodes (soft = 0,
Hard = 0)
 

If more file systems have quotas, You need to configure more lines for these file systems.

If you need to configure a quota for multiple users, You can first set a quota for one user, and then use the-p command line option for edquota. This will use the specified user as the basis to create a new user. For example, if you want to set the quota for the user names of the MC to the drop-down list, drop-down, and SH, run the following command:

$ Edquota-P Mc slp tw sh
 

File limit warning using quota

When a user creates a file that exceeds its soft limit, they will receive the following warning:

Quota_ufs: Warning: Over disk limit (PID 1738, uid 101, inum 94, FS/export/Home)
 

Please note that you can use edquota-t to change the duration of the issue within seven days.

If you try to create a file that exceeds the hard limit, the system will terminate the write process and truncate the file according to the corresponding restrictions:

$ Mkfile 210000 K overlimit
Quota_ufs: over hard disk limit (PID 1843, uid 101, inum 130, FS
/Export/Home)
Overlimit: initialized 191873024 of 215040000 Bytes: Disc quota exceeded
 

By running the quota command, any user can check their own quota limits and disk usage:

$ Quota
Over disk quota on/export/home, remove 199993 K within 7.0 days
 

Quota Management

The quota command allows the system administrator to check the quota of any user, you should use the-V command line option to provide a complete report on the file system, usage, and limits (see listing 8 ).

Listing 8. Use the-V Option
$ Quota-V MC
Disk quotas for MC (UID 101 ):
Filesystem usage quota limit timeleft files quota limit
Timeleft
/Export/home 399993 200000 400000 days 6.9 0 0
 

To obtain a report describing the disk and quota usage of all users in a file system, you can use the repquota command and specify the file system to be reported (see listing 9 ).

List 9. run the repquote command
$ repquota-V/export/home
/dev/DSK/c0t0d0s7 (/export/Home ):
block limits file limits
User used soft hard timeleft
MC +-399993 200000 400000 days 6.9 0 0

To ensure that the quota information is up-to-date, you should use the quotacheck command. This verifies the file storage data by using the quota information used to report the quota information. You should use cron to automatically run this task, which is appropriate once a day (because this process is time-consuming ).

Quota skills and traps

The quota system provides the best combination of monitoring and automatic management, but you should note that the quota file system is enabled and the restrictions you set, because they may hinder your work, instead of controlling their disk usage.

For example,ProgramMembers may need more space to build an application. By providing an unrestricted compilation area (by setting an optional temporary directory), you can avoid this problem while maintaining their home directory quota, or, you can set a high hard limit (or even the maximum capacity of the file system) while setting the appropriate soft limit.

The result should be a warning when the soft limit is reached, without actually implementing the hard limit. Users should still be able to build and create temporary files, but because these files are deleted during the build process, users should not be prohibited from creating the files they need.

Quotas can also be used to complete monitoring tasks and remind you about the space usage of special user accounts. In the past, I used quotas to monitor web user accounts, such as the so-called nobody accounts and other accounts, to ensure that they cannot write files to file systems that do not have access permissions. To complete this task, you can set the Hard Limits of the file system and users you want to protect to 1 kb.

Automatic Monitoring

It is feasible to manually monitor disk space usage, but you do not want to continuously run DF (or even du) to determine the used or available disk space. You can automate this process and automatically send emails to the Administrator (or administrator group) when the available space reaches a certain level. The script in listing 10 is used to monitor disk space. You can set the warning (warninglimit) and emergency (lowlimit) limits, as well as the list of file systems to be checked.

Listing 10. Monitoring disk space
#! /Bin/sh

Warninglimit = 500000
Lowlimit = 250000

Filesystems ="/Export/data/export/home/"

For FS in $ filesystems
Do
Size = 'df-K $ FS | grep $ FS | awk ' {Print $4 ;} ' '
If [$ size-Le $ lowlimit]
Then
Mailx-S " Urgent: Low disk space for $ FS ($ size) "
Break
Fi
If [$ size-Le $ warninglimit]
Then
Mailx-S " Warning: Low disk space for $ FS ($ size) "
Fi
Done

The key line is to extract the size of free disk space in each file system:

Size = 'df-K $ FS | grep $ FS | awk'{Print $4 ;}'`
 

This script uses DF to extract only the rows for grep, and then uses awk to extract the 4th columns of data. This column is the size of free space.

Then, you can check the free space based on warninglimit/lowlimit and generate appropriate errors. To avoid both lower limit and warning errors, the script first checks the lower limit and uses break to jump to the next file system in the loop before attempting to test the warning.

Conclusion

Monitoring disk space is an important task for administrators. Exhausting disk space may seriously affect users' work. In severe cases, data may be lost or the system may crash, because the system cannot obtain the disk space required for running.

Using DF, you can determine the free space and space usage of the entire file system, but this only describes part of the information. To determine where the disk space is used, you need to use the du tool to study different directories. By using find, you can even use du to find the space used by a specified user. Quota is a better choice for more automated user-level disk usage and control.

 

Transfer http://unix-cd.com/unixcd12/article_5875.html