NFS mounting and umout Problems

In the fast cache project, commands such as STAT/opendir/DF-K are suspended when the NFS mount directory is created and the NFS network is disconnected. the problem arises. How can I check the accessibility of the NFS mount directory in the program and the program is not suspended?

Solution:

1. The Mount mode is soft. When a major timeout is reached, stat can be returned to the caller. however, in hard mode, when a major timeout arrives, only one error is reported on the console and no result is returned. test: After a soft method is created several minutes, the timeout time is not accurate, and the subsequent time is far greater than the actually calculated time value.

Major timeout calculation formula: If retry = 1, timeout = timeo + 2 * timeo. if retry = 2, timeout = timeo + 2 * timeo + 4 * timeo. the following is the mount command:

Mount-T nfs-o rw, TCP, soft, rsize = 8192, wsize = 8192, timeo = 100, Intr, retry = 1 135.251.208.34:/vol/vol1/nfs

2. Use the signal method. since the hard mode can be interrupted by signals after the intr option is set. you can check the time of the STAT command in another thread. Once the time exceeds the threshold, the SIGINT signal is sent.

However, the test shows that in a multi-threaded environment, the SIGINT is sent to the specified thread using pthread_kill (TID, SIGINT) to attempt to interrupt stat. however, the sleep operation can be significantly interrupted. it seems that stat and sleep are not the same at the kernel level. however, in a single-threaded environment, it is feasible to use kill (PID, SIGINT). I have never understood why.

3. Open up a monotonous thread to perform the stat operation, that is, it returns the check result to the working thread for use, and the working thread does not perform the stat operation. this method cannot completely prevent hanging, because the subsequent directory access may be suspended, that is, an NFS exception occurs between two check cycles, the fast cache switchover happens during these two check cycles.

=====

Each time a customer sends a request to the NFS server, it expects the operation to be completed within a given interval (specified by the timeo option. If no confirmation is received within this time period, a so-called time-out (minor timeout) occurs. The operation is retried and the time-out interval doubles. A master timeout occurs when the maximum timeout value reaches 60 seconds or the number of retrans is reached ).

For hard mode: A major timeout will cause the client program to print a message on the console and start again, and it will continue.

For the soft method, a major timeout generates an I/O error message for the calling process and returns immediately.

The following is an example: Mount-T nfs-o rw, TCP, soft, rsize = 8192, wsize = 8192, retry = 1, timeo = 10, Intr 135.251.208.34: /vol/vol1/nfs

(Calculation of the timeout value returned immediately after the above Timeout: timeo/10 + 2 * timeo/10. Because retry = 1, it indicates only one retry, and its timeout value is 3 seconds, timeo is measured in 0.1 seconds)

During the test, you can use the following method to disable the IP address to prevent access from IPC. iptables-A input-s 135.251.208.34-J reject.
Run the iptables-F command to enable

====================================

NFS server can be considered as a file server. It allows your PC to mount the files shared by the remote NFS server to its own system through the network, in the client's view, remote files using NFS are like local files.
The NFS protocol has been available in multiple versions since its birth, such as NFS V2 (rfc1094) and NFS V3 (rfc1813) (the latest version is V4 (rfc3010 ).
Ii. Main differences between NFS Protocol versions
Major differences between V3 and V2:
1. File Size
V2 supports up to 32 bit file sizes (4 GB), while NFS V3 supports 64 bit file sizes.
2. File Transfer size
V3 does not have a limited transmission size. V2 can only be set to 8 KB at most. You can use-rsize and-wsize to set it.
3. complete information is returned
V3 has added and improved the return of many errors and successful information, which can bring great benefits to server settings and management.
4. Added support for TCP transmission protocol
V2 only provides support for the UDP protocol, which has great limitations in some demanding network environments. V3 has added support for the TCP protocol.
* 5. asynchronous writing
6. Improved server Mount Performance
7. Better I/O writes performance.
9. Enhanced network operation efficiency, making network operation more effective.
10. Better disaster recovery functions.

Asynchronous write feature (New in V3:
Whether NFS V3 can be written asynchronously is an optional feature. The NFS V3 client sends an asynchronous write request to the server. before replying to the client, the server does not have to write data to the memory (stable ). The server can determine when to write data or aggregate and process multiple write requests, and then write data. The client can maintain a copy of the data, in case the server cannot completely write the data. When the client wants to release this copy, it will use this operation process to the server to ensure that each operation step is complete. Asynchronous writing allows the server to determine the best data synchronization policy. Make sure that the data can be submitted synchronously as much as possible. Compared with V2, this mechanism can better achieve data buffering and more parallel (balanced ). NFS
The V2 server cannot write data to the storage before writing data to the storage.

V4 compared with V3:
1: improved access and execution efficiency on the Internet
2: enhanced security features in protocols
3: enhanced cross-platform features
Iii. Specific operations and settings of the client and server
Before talking about the operation of NFS server, let's look at some things related to NFS server:
RPC (Remote Procedure Call)
NFS itself does not provide information transmission protocols and functions, but NFS allows us to share data over the network, because NFS uses some other transmission protocols. These Transport Protocol warriors use this RPC function. NFS itself is a program that uses RPC. NFS is also an RPC server. Therefore, the RPC service must be started wherever NFS is used, whether it is an NFS server or an NFS client. In this way, the server and client can implement the corresponding program port through rpc. We can understand the relationship between RPC and NFS in this way: NFS is a file system, while RPC is responsible for information transmission.

Daemons to be started for NFS
PC. nfsd: Mainly used to detect complex logon permissions.
Rpc. mountd: Responsible for the NFS file system. When the client logs on to the server through rpc. nfsd, it manages the files accessed by clinet.
NFS server requires two packages on the RedHat Linux platform: nfs-utils and Portmap.
Nfs-utils: provides two NFS daemons suites: rpc. nfsd and rpc. mountd.
Portmap: NFS can be regarded as an RPC server program. to start an RPC server program, you must do the corresponding work of the port, and such a task is completed by Portmap. In general, Portmap is used for port mapping.

I. server-side settings (taking Linux as an example)
Server-side settings are all set in the/etc/exports file. The format is as follows:
Directory host name 1 or IP1 (parameter 1, parameter 2) host name 2 or ip2 (parameter 3, parameter 4)
The above format indicates that the same directory is shared to two different hosts, but the permissions and parameters provided to the two hosts are different. Therefore, the permissions obtained by the two hosts are set separately.
You can set the following parameters:
RW: read/write permission;
RO: Read-Only permission;
No_root_squash: if the user logging on to the NFS host is a root user, the user has the root permission. this parameter is insecure and is not recommended.
Root_squash: Are you logged on to the NFS master? C. What is the purpose of sharing? If the user is root? R, then ???? User's? Why? Why? Why? Why? UID? GID all? ? Into nobody, then ?? Identity;
All_squash: No matter what users log on to the NFS host, they will be reset to nobody.
Anonuid: sets all users logging on to the NFS host to the specified user ID, which must exist in/etc/passwd.
Anongid: Same as anonuid,? Group ID!
Sync: data is synchronized to the storage.
Async: The data is temporarily stored in the memory and not directly written to the hard disk.
Insecure allows unauthorized access from this machine.

For example, you can edit/etc/exports as follows:
/Tmp * (RW, no_root_squash)
/Home/Public 192.168.0. * (RW) * (RO)
/Home/test 192.168.0.100 (RW)
/Home/Linux * .the9.com (RW, all_squash, anonuid = 40, anongid = 40)
After setting, run the following command to start NFS:
/Etc/rc. d/init. d/Portmap start (Portmap is started by default in RedHat)
/Etc/rc. d/init. d/nfs start

Exportfs command:
If we modify/etc/exports after starting NFS, do we have to restart NFS? At this time, we can use the exportfs command to make the change take effect immediately. The command format is as follows:
Exportfs [-aruv]
-A: All content in Mount or unmount/etc/exports
-R: Re-mount the directory shared in/etc/exports
-U: umount directory
-V: in export? R wait, and output the detailed information to the screen.
Example:
[Root @ test root] # exportfs-RV <= All are re-export once!
Exporting 192.168.0.100:/home/test
Exporting 192.168.0. *:/home/Public
Exporting * .the9.com:/home/Linux
Exporting *:/home/Public
Exporting *:/tmp
Reexporting 192.168.0.100:/home/test to Kernel

Exportfs-au <= All are uninstalled.

Operations for customer segments:
1. The showmout command is helpful for NFS operations and troubleshooting. Let's take a look at the usage of showmount.
Showmout
-A: this parameter is generally used on the NFS server and is used to display the Cline machine that has mounted the local NFS directory.
-E: displays the export directory on the specified NFS server.
For example:
Showmount-e 192.168.0.30
Export list for localhost:
/Tmp *
/Home/Linux * .linux.org
/Home/Public (everyone)
/Home/test 192.168.0.100
2. Mount the NFS directory:
Mount-t nfs hostname (orip):/directory/Mount/Point
Example:
Linux: Mount-t nfs 192.168.0.1:/tmp/mnt/nfs
Solaris: Mount-f nfs 192.168.0.1:/tmp/mnt/nfs
BSD: Mount 192.168.0.1:/tmp/mnt/nfs

3. Other optional parameters of mount NFS:
Hard mount and soft mount:
Hard: the NFS client will constantly try to connect to the server (in the background, there will be no prompts, and some prompts will still be given in some versions of Linux) until mount.
Soft: it will try to connect to the server on the frontend, which is the default connection method. When an error message is received, the Mount attempt is terminated and related information is provided.
Example: Mount-F nfs-O hard 192.168.0.10:/nfs
Whether it is hard or soft depends on the information you access. For example, if you want to run X program through NFS, you will never expect unexpected situations (for example, the network speed suddenly slows down and the NIC plug is plugged in) the system outputs a large number of error messages. If you use the hard method, the system will wait until it can establish a new connection with the NFS server for transmission. In addition, if it is non-critical data, you can also use the soft method, such as FTP data, so that the session process will not be suspended when the remote machine is temporarily unable to connect or shut down.

Rsize and wsize:
File Transfer size setting: V3 does not limit the transmission size. V2 can only be set to 8 KB at most. You can use-rsize and-wsize to set the file transfer size. The setting of these two parameters has a great impact on the NFS execution efficiency.
BG: If the Mount operation fails, the system transfers the Mount operation to the background and continues to try the Mount operation until the Mount operation is successful. (BG is usually used when setting the/etc/fstab file to avoid affecting the startup speed due to the possible Mount failure)
FG: the opposite of BG. It is the default parameter.
Nfsvers = N: sets the NFS version to be used. The default value is 2. The setting of this option depends on whether the server supports NFS ver 3.
Mountport: Specifies the Mount port.
Port: Set the port based on the export output from the server. For example, if the server uses port 5555 to output NFS, the client needs to use this parameter for the same setting.
Timeo = N: set the time-out period. When data transmission encounters a problem, the system tries to re-transmit the data based on this parameter. The default value is 7/10 (0.7 seconds ). If the network connection is not very stable, we recommend that you increase the value and use the hard Mount method. We recommend that you also add the intr parameter so that you can terminate any pending file access.
Intr allows a notification to interrupt an NFS call. It is useful when the server does not respond and must be abandoned.
UDP: Use UDP as the NFS transmission protocol (NFS V2 only supports UDP)
TCP: use TCP as the NFS transmission protocol
Namlen = N: specifies the maximum file name allowed by the remote server. The default value is 255.
Acregmin = N: sets the minimum cache time before file update. The default value is 3.
Acregmax = N: sets the maximum cache time before file update. The default value is 60.
Acdirmin = N: sets the minimum cache time before directory update. The default value is 30.
Acdirmax = N: sets the maximum cache time before directory update. The default value is 60.
Actimeo = N: Set acregmin, acregmax, acdirmin, and acdirmax to the same value, which is disabled by default.
Retry = N: set the time required for reconnecting when network transmission fails. The default value is 10000 minutes.
Noac: Disable the cache mechanism.
Use multiple parameters at the same time: Mount-T nfs-O timeo = 3, UDP, hard 192.168.0.30:/tmp/nfs
Note that the options of the NFS client and server are not necessarily the same, and sometimes there are conflicts. For example, if the server exports data in read-only mode, but the client mounts the data in writable mode, an error occurs when writing the data. Generally, the configuration of the server prevails when the server and client are in conflict.

4. How to Set/etc/fstab
The format of/etc/fstab is as follows:
Fs_spec fs_file fs_type fs_options fs_dump fs_pass
Fs_spec: this field defines the device or remote file system where the file system to be loaded is located. For NFS, this parameter is generally set to 192.168.0.1:/NFS.
Fs_file: Local mount point
Fs_type: For NFS, you only need to set this field to NFS.
Fs_options: Mount parameter. For available parameters, refer to the preceding Mount parameter.
Fs_dump-this option is used by the "dump" command to check how often a file system should be dumped. If no dump is required, set this field to 0.
Fs_pass-this field is used by The fsck command to determine the sequence of the file system to be scanned at startup. The value of the "/" pair of the root file system should be 1, other file systems should be 2. If the file system does not need to scan at startup, set this field to 0.

5. Introduction to NFS-related commands
Nfsstat:
Checking the running status of NFS is helpful for adjusting the running status of NFS.
Rpcinfo:
Displays RPC execution information, which can be used to detect RPC running conditions.

Iv. NFS Optimization
Optimization steps:
1. Measure the execution efficiency of the current network, server, and each client.
2. analyze collected data and draw charts. Find out special situations, such as high disk and CPU usage, high disk usage time
3. Adjust the server
4. Repeat steps 1 to 3 until you reach the desired performance.

There are many problems related to NFS performance. You can usually consider the following options:

The wsize and rsize parameters are used to optimize the NFS execution efficiency.
Wsize and rsize have a great impact on NFS performance.
Wsize and rsize set the size of the data blocks between the server and the client. The reasonable settings of these two parameters are related to many aspects, in addition to software, there are also hardware factors that will affect the setting of these two parameters (such as Linux kernel, Nic, and switch ).
The following command can test the NFS execution efficiency. The read and write performance can be tested separately to find the appropriate parameters. You can write scripts to test the reading and writing of a large amount of scattered data. During each test, it is best to execute mount and unmount again.
TIME dd If =/dev/Zero of =/mnt/home/testfile BS = 16 k count = 16384
The wsize used for testing. rsize is preferably a multiple of 1024. For NFS V2, 8192 is the maximum value of rsize and wsize. If NFS V3 is used, the maximum value you can try is 32768.
If the set value is relatively large, it is best to enter the directory on the mount on the client and perform some common operations (LS, Vi, etc.) to see if there is any error message. When LS is used, files cannot be fully listed or error messages may occur. different operating systems have different optimal values, therefore, different operating systems must be tested.

Set the maximum number of NFSD copies.
The number of NFSD copies in Linux is in/etc/rc. d/init. d/NFS is set in the startup file. The default value is 8 nfsd. The setting of this parameter is generally based on the number of possible clients, similar to wsize and rsize, we also need to test the latest value.

UDP and TCP
You can set it manually or automatically.
Mount-T nfs-O sync, TCP, noatime, rsize = 1024, wsize = 1024 export_machine:/exported_dir/Dir
UDP provides fast transmission speed and non-connection transmission convenience. However, UDP is not stable over TCP during transmission, when the network is unstable or hackers intrude into the network, it is easy to greatly reduce the performance of NFS or even paralyze the network. Therefore, you must select the transmission protocol for networks in different situations. NFS over TCP is relatively stable, and NFS over UDP is faster. When there are few machines with good network conditions, using UDP protocol can bring better performance. When there are many machines and the network conditions are complex, we recommend using TCP protocol (V2 only supports UDP protocol ). It is better to use UDP protocol in the LAN because the LAN has stable network guarantee and UDP can provide better performance. We recommend that you use TCP protocol in the wide area network, TCP enables NFS to maintain optimal transmission stability in complex network environments. Can refer to this article: http://www.hp.com.tw/ssn/unix/0212/unix021204.asp

Version Selection
V3 is the default option (Red Hat 8 uses V2 by default, and Solaris 8 and later uses V3 by default). You can select it through vers = mount option.
In Linux, The nfsvers = n option is used for selection.

V. NFS troubleshooting
1. nfsd is not started.
Make sure that the NFS output list exists. Otherwise, NFSD will not start. The exportfs command can be used to check the file. If the exportfs command does not return results or the returned results are incorrect, check the/etc/exports file.
2. the mountd process is not started.
The mountd process is a Remote Procedure Call (RPC). Its function is to respond to the application for client-side installation (Mount) of the file system. The mountd process finds the/etc/xtab file to find out which file systems can be used by remote clients. In addition, through the mountd process, you can know which file systems have been assembled by remote file systems and the list of remote clients. You can run the rpcinfo command to check whether mountd is properly started. Normally, the output list contains rows like this:
100005 1 udp 1039 mountd
100005 1 tcp 1113 mountd
100005 2 UDP 1039 mountd
100005 2 TCP 1113 mountd
100005 3 UDP 1039 mountd
100005 3 TCP 1113 mountd
If not, check whether the Portmap component is installed.
Rpm-Qa | grep Portmap
3. FS type NFS no supported by Kernel
Kernel does not support the NFS file system. Re-compile the kernel to solve the problem.
4. Can't contact Portmapper: RPC: Remote System Error-connection refused
This error message is displayed because the Portmap on the sever side is not started.
5. Mount clntudp_create: RPC: program not registered
NFS is not started. You can run the showmout-e host command to check whether the NFS server is started normally.
6. Mount: localhost:/home/test failed, reason given by server: Permission denied
This prompt may appear when the client wants to mount the NFS server. It means that the local machine has no permission to mount the directory on the NFS server. The solution is to modify the NFS server.
7. blocked by the firewall
Many people have neglected this reason. in a network environment with strict requirements, we usually close all ports on Linux and open them only when any port is needed. NFS uses port 111 by default, so we must first check whether this port is enabled, and also check the TCP_WRAPPERS settings.

Vi. NFS security
NFS security is mainly manifested in the following four aspects:

1. Newbie's access control mechanism for NFS is difficult to implement, and the accuracy of control objectives is difficult to achieve
2. NFS does not have a real user authentication mechanism, but only has a process verification mechanism for rpc/mount requests.
3. Earlier NFS versions allow unauthorized users to obtain valid file handles.
4. In remote rpc calls, a SUID program has the superuser permission.

How to enhance NFS security:
1. Reasonably set the shared directory in/etc/exports. It is best to use anonuid and anongid so that the client mounted to the NFS server has only the minimum permission. It is best not to use root_squash.
2. Use iptable firewall to restrict the range of machines that can connect to the NFS server
Iptables-A input-I eth0-P TCP-s 192.168.0.0/24 -- dport 111-J accept
Iptables-A input-I eth0-P UDP-s 192.168.0.0/24 -- dport 111-J accept
Iptables-A input-I eth0-P TCP-s 140.0.0.0/8 -- dport 111-J accept
Iptables-A input-I eth0-P UDP-s 140.0.0.0/8 -- dport 111-J accept
3. To prevent possible DoS attacks, you need to set the nfsd copy quantity properly.
4. Modify/etc/hosts. Allow and/etc/hosts. Deny to restrict the client.
/Etc/hosts. Allow
Portmap: 192.168.0.0/255.255.255.0: Allow
Portmap: 140.116.44.125: Allow

/Etc/hosts. Deny
Portmap: All: deny
5. Change the default NFS port
NFS uses port 111 by default, but you can also use the port parameter to change the port so that security can be enhanced to a certain extent.
6. use Kerberos V5 as the login verification system