File sharing between multiple servers in Linux

Because the project has a picture upload and import the exported module, so when the project through the cluster mode of deployment to consider the file sharing problem.

File sharing is either through a unified file system to manage, or to do file sharing between the system, the former extensibility is better, can add the server anytime and anywhere, the latter is simple, through the configuration of the server share directory can, but not good expansion. The page function of our project is relatively simple, and is used by the company's internal personnel, PV is very small, so use the latter. The configuration is documented here.

The steps are as follows:

1. In general, the NFS package is installed by default, and before you configure Server for NFS, check to see if the package is installed. If not installed, Nfs-utils Portmap the two packages on the second CD/redhat/pms
#rpm –q nfs-utils Portmap

1.1 Starting the NFS service
Service NFS Start

2. Server-side settings are set in/etc/exports this file, set the format as follows
#vi/etc/exports

Directory host name to be shared 1 or IP1 (parameter 1, parameter 2) hostname 2 or IP2 (parameter 3, parameter 4)
For example:
/home/test * (Rw,no_root_squash)

3. Load the parameters modified by the 2nd step
# EXPORTFS-RV

4.NFS Client Configuration
#mount-T NFS hostname (ORIP):/home/test/ebs/test

5. Automatically mount when system starts
To write NFS shared information to an/etc/fstab file to automatically mount an NFS shared directory
Add the following line to the Fstab file
172.16.2.91:/home/test/ebs/test NFS Defaults 0 0
End

Additionally, configure the boot-from-boot NFS

Add the following information in the/etc/profile file
Etc/init.d/nfs start

*************************************************************************************************************** *********************************************************
Exports file Configuration instance

Configure the SJD folder to be shared with all hosts

Server for NFS start and stop
Query service Portmap status start service portmap start stop service Portmap stop

Start NFS Service serivce NFS Start (because NFS service requires PORTMAP service support, so portmap to start first)

6. Showmount command displays information for NFS server
Showmount 192.168.0.1 (IP for NFS server)

To view the shared output on the server

Showmount–d "NFS Server host Address" is used to display shared directories that have been mounted by NFS clients in Server for NFS

The SHOWMOUNT–A option is used to display a list of shared directories that have been mounted on the NFS server and the address of the NFS client that mounts the shared directory

7. Exportfs command
When the system administrator modifies the/etc/exports file, it does not automatically take effect on Server for NFS. The Exportfs management tool is available in the Server for NFS package to manage the contents of the exports file settings.
Re-export shared directory EXPORTFS–RV

Stop outputting all directories EXPORTFS–AUV

Output (enabled) all Directories Exportfs AV

8. NFS Client Configuration
Configuring the Use of NFS clients in Linux
1. Mount the shared directory on the NFS Server Mount–t NFS 192.168.0.1:/sjd/mnt (view again mounted), already mounted, you can see the file inside

2. Automatically mount when system starts
To write NFS shared information to an/etc/fstab file to automatically mount an NFS shared directory
Add the following line to the Fstab file
192.168.0.1:/sjd/mnt NFS Defaults 0 0

Resources:

Take a look at some of the NFS server-related things before you go about NFS server operations:
RPC (Remote Procedure Call)
NFS itself does not provide the protocol and functionality for transmitting information, but NFS allows us to share information over the network, because NFS uses some other transport protocols. And these transport protocol Warriors use this RPC function. It can be said that NFS itself is a program that uses RPC. Or, NFS is also an RPC server. So whenever you use NFS, you start the RPC service, either NFS Server or NFS CLIENT. This allows the server and client to implement the program PORT correspondence via RPC. You can understand the relationship between RPC and NFS: NFS is a file system, and RPC is responsible for the transfer of information.
NFS needs to start the daemons
PC.NFSD: The main complex login permission detection and so on.
RPC.MOUNTD: The file system responsible for NFS, when the client side through the RPC.NFSD landing server, the Clinet access to the server files for a series of management
NFS server requires a total of two packages under the Redhat Linux platform: nfs-utils and Portmap
Nfs-utils: Offers two NFS daemons kits for RPC.NFSD and RPC.MOUNTD
Portmap:nfs can actually be seen as an RPC server program, and to start an RPC server program, do the work of port, and this task is done by Portmap. Popular saying Portmap is used to do the mapping of the port.
One: Server-side settings (Linux for example)
Server-side settings are set in the/etc/exports file, set the format as follows:
Directory host name to be shared 1 or IP1 (parameter 1, parameter 2) hostname 2 or IP2 (parameter 3, parameter 4)
The above format indicates that the same directory is shared with two different hosts, but the permissions and parameters provided to the two hosts are different, so the permissions are set for two hosts respectively.
The main parameters that can be set are the following:
RW: Permission to read and write;
RO: read-only permission;
No_root_squash: If the user who is logged on to the NFS host is the root user, he will have root privileges and this parameter is not secure and is not recommended for use.
Root_squash: When the user who is logged in to the NFS host using the shared directory is root, then the user's permissions will be compressed into an anonymous user, usually his UID and GID will become nobody that identity;
All_squash: No matter what user is logged on to the NFS host, it will be reset to nobody.
Anonuid: The user who will log on to the NFS host is set to the specified user ID, which must exist in/etc/passwd.
Anongid: With Anonuid, but become group ID is!
Sync: Data is written to the memory synchronously.
Async: The data is temporarily stored in memory and is not written directly to the hard disk.
Insecure allows unauthorized access from this machine.
For example, you can edit the/etc/exports to:
/tmp * (Rw,no_root_squash)
/home/public 192.168.0.* (rw) * (RO)
/home/test 192.168.0.100 (rw)
/home/linux *.the9.com (rw,all_squash,anonuid=40,anongid=40)
Once set, you can start NFS with the following command:
/etc/rc.d/init.d/portmap Start (Portmap is started by default in Redhat)
/etc/rc.d/init.d/nfs start
EXPORTFS command:
If we change/etc/exports after we start NFS, do we have to restart NFS? At this point we can use the EXPORTFS command to make the change take effect immediately, the command format is as follows:
Exportfs [-aruv]
-A: All content in Mount or unmount/etc/exports
-R: Re-mount/etc/exports in the shared directory
-u:umount Directory
-V: In export, the detailed information is output to the screen.
Specific examples:
[Root @test root]# EXPORTFS-RV
2. Mount NFS Directory Method:
MOUNT-T NFS Hostname (ORIP):/directory/mount/point
Specific examples:
Linux:mount-t NFS 192.168.0.1:/tmp/mnt/nfs
Solaris:mount-f NFS 192.168.0.1:/tmp/mnt/nfs
Bsd:mount 192.168.0.1:/tmp/mnt/nfs
3. Additional optional parameters for Mount NFS:
Hard mount and Soft mount:
The HARD:NFS client will constantly try to connect to the server (in the background, no hint will be given, and some versions of Linux will still give some hints) until mount.
SOFT: The connection to the server is attempted in the foreground, and is the default connection method. When an error message is received, the mount attempt is terminated and the relevant information is given.
Example: Mount-f nfs-o hard 192.168.0.10:/nfs/nfs
The question of whether to use hard or soft depends largely on what information you visit. For example, if you want to run X program via NFS, you will never want to make the system output a lot of error messages because of some unexpected situations (such as the slow speed of the network, Plug and unplug the NIC), and if you use hard mode at this time, the system will wait. Until the connection can be re-established with NFS server to transfer information. In addition, if it is non-critical data, you can also use the soft method, such as FTP data, so that the remote machine temporarily connected or closed when you do not suspend your session process.
Rsize and Wsize:
File transfer Size setting: V3 does not limit the transmission size, the V2 can only be set to 8k, it is possible to use-rsize and-wsize to set. The setting of these two parameters has a great impact on the performance of NFS.
BG: If mount is not successful when Mount is executed, the system moves the mount operation to the background and continues to mount until the mount is successful. (BG is usually used when setting the/etc/fstab file to avoid possible mount not affecting the boot speed)
FG: Just the opposite of BG, is the default parameter
Nfsvers=n: Set the NFS version to be used, by default 2, and this option depends on whether the server Side supports NFS VER 3
Mountport: Setting the port of Mount
Port: Based on server-side export ports, for example, if the server uses 5555 port to output NFS, then the client needs to use this parameter for the same setting
Timeo=n: Sets the time-out period and, when data transmission encounters a problem, attempts to retransmit according to this parameter. The default value is 7/10 wonderful (0.7 seconds). If the network connection is not very stable, then increase this value, and recommend the use of hard mount mode, and preferably with the Intr parameter, so you can terminate any pending file access.
INTR allows notifications to interrupt an NFS call. Useful when the server is not responding and needs to be discarded.
UDP: Transport protocol using UDP as NFS (NFS V2 only supports UDP)
TCP: Transport protocol using TCP as NFS
Namlen=n: Sets the longest file name allowed by the remote server. The default for this value is 255.
Acregmin=n: Set minimum cache time before file update, default is 3
Acregmax=n: Set maximum cache time before file update, default is 60
Acdirmin=n: Set minimum cache time before directory update, default is 30
Acdirmax=n: Set max cache time before directory update, default is 60
Actimeo=n: Set Acregmin, Acregmax, Acdirmin, Acdirmax to the same value, default is not enabled.
Retry=n: Sets the number of times to try to reconnect when the network transmission fails. The default value is 10000 minutes
NOAC: Close the cache mechanism.
Ways to use multiple parameters simultaneously: mount-t nfs-o timeo=3,udp,hard 192.168.0.30:/tmp/nfs
Note that the options for NFS clients and servers are not necessarily identical and sometimes conflict. For example, the server is exported in a read-only manner, and the client is mount in a writable manner, although it can succeed on mount, but an error occurs when trying to write. When the server and client configuration conflict, the server configuration will prevail.
4. Setting method of/etc/fstab
The/etc/fstab format is as follows:
Fs_spec fs_file fs_type fs_options fs_dump fs_pass
Fs_spec: This field defines the device or remote file system on which the file system you want to load is located, which is typically set to this parameter for NFS: 192.168.0.1:/nfs
Fs_file: Local mount point
Fs_type: For NFS, this field is only set to NFS.
Fs_options: Mounting parameters can be used to refer to the Mount parameter above.
Fs_dump-This option is used by the "Dump" command to check how fast a file system should dump and set the field to 0 if no dump is required
Fs_pass-This field is used by the fsck command to determine the order of the file systems that need to be scanned at startup, the root filesystem "/" should have a value of 1 for the field, and the other filesystem should be 2. If the file system does not need to be scanned at startup, the field is set to 0.
5. Introduction to some of the commands related to NFS
Nfsstat:
Viewing the running Status of NFS is a great help in adjusting the operation of NFS
Rpcinfo
View RPC execution information, tools that you can use to detect RPC health.
Iv. NFS Tuning
Tuning steps:
1. Measure the execution efficiency of the current network, server and each client.
2. Analyze the collected data and draw the chart. Find special cases such as high disk and CPU usage, high disk use time
3, adjust the server
4. Repeat the first and third steps until you reach the desired performance
There are a number of issues related to NFS performance, and you can usually consider these options:
Wsize,rsize parameters to optimize the execution performance of NFS
Wsize and Rsize have a great impact on the performance of NFS.
Wsize and Rsize set the size of the data block between the server and the client, and the reasonable setting of these two parameters is related to many aspects, not only the software but also the hardware factors will affect the setting of these two parameters (such as Linux KERNEL, network card, switch, etc.).
The following command can test the performance of NFS, and read and write performance can be tested separately to find the appropriate parameters. You can write scripts to test the read and write of large amounts of data that you want to test. It is best to perform the mount and unmount repeatedly at each test.
Time DD If=/dev/zero of=/mnt/home/testfile bs=16k count=16384
The wsize,rsize for testing is preferably a multiple of 1024, 8192 is the maximum number of rsize and wsize for NFS V2, and if you are using NFS V3 the maximum value you can try is 32768.
If you set a larger value, it should be best to go to the directory on the client, do some general operations (LS,VI, etc.), and see if there are any error messages. There may be typical problems with LS when the file is not fully listed or error message, different operating systems have different best values, so for different operating systems to be tested.
Set the best copy number for NFSD.
The copy number of NFSD in Linux is set in/etc/rc.d/init.d/nfs this startup file, the default is 8 NFSD, the setting of this parameter is generally based on the number of possible clients to be set, and Wsize, Rsize is also going to pass the test to find the nearest value.
UDP and TCP
You can set it up manually, or you can make a selection automatically.
Mount-t Nfs-o sync,tcp,noatime,rsize=1024,wsize=1024 Export_machine:/exported_dir/dir
UDP has a fast transmission, non-connected transmission of convenient features, but UDP in the transmission of no TCP to stabilize, when the network is unstable or hacking when it is easy to make NFS performance greatly reduced or even paralyzed the network. Therefore, for different situations of the network to have a choice of transmission protocol. NFS over TCP is more stable and NFS over UDP is faster. The use of UDP protocol can bring better performance when the machine has less network condition, and it is recommended to use the TCP protocol (V2 only supports UDP protocol) when the machine is more complicated and the network condition is complex. The use of UDP protocol in LAN is good, because LAN has a relatively stable network guarantee, using UDP can bring better performance, the TCP protocol is recommended in WAN, TCP protocol can let NFS maintain the best transmission stability in complex network environment. Refer to this article: http://www.hp.com.tw/ssn/unix/0212/unix021204.asp
Selection of versions
V3 as the default choice (RED HAT 8 defaults to using V2,solaris above by default using V3) and can be selected by vers= mount option.
Linux is selected through the nfsvers=n of the Mount option.
Five, NFS fault resolution
1, NFSD did not start up
The first step is to confirm that the NFS output list is present, otherwise NFSD will not start. The EXPORTFS command can be used to check if the EXPORTFS command has no results returned or is not returned correctly, you need to check the/etc/exports file.
2, MOUNTD process does not start
The MOUNTD process is a remote procedure call (RPC) that responds to requests from the client to install the (Mount) file system. The MOUNTD process finds out which file systems can be used by remote clients by locating the/etc/xtab file. In addition, through the MOUNTD process, users can know what file systems are currently being assembled by remote file systems and learn the list of remote clients. See if Mountd is up and running can be viewed using command rpcinfo, which normally should look like this in the list of outputs:
100005 1 UDP 1039 Mountd
100005 1 TCP 1113 Mountd
100005 2 UDP 1039 Mountd
100005 2 TCP 1113 Mountd
100005 3 UDP 1039 Mountd
100005 3 TCP 1113 Mountd
If you don't get up, you can check if the Portmap component is installed.
Rpm-qa|grep Portmap
3. FS type NFS no supported by kernel
Kernel does not support NFS file system, recompile kernel can be resolved.
4. Can ' t contact Portmapper:RPC:Remote system Error-connection refused
This error message occurs because the Portmap on the sever side is not started.
5. Mount Clntudp_create:RPC:Program not registered
NFS does not start up, you can use the SHOWMOUT-E Host command to check if NFS server is up and running properly.
6, Mount:localhost:/home/test failed, reason given by Server:permission denied
This hint is a hint that may occur when the client wants to mount NFS server, meaning that the native does not have permission to mount the directory on NFS server. The workaround is, of course, to modify the NFS server.
7, blocked by the firewall
For this reason, many people neglect that in the strict network environment, we usually shut down all the ports on Linux, when we need to use which port to open. For NFS, the default is to use port 111, so we'll first check if this port is open, and the Tcp_wrappers setting.
VI. NFS Security
The non-security of NFS is mainly reflected in the following 4 aspects:
1, novice access control mechanism for NFS is difficult to do handy, the accuracy of control objectives is difficult to achieve
2. NFS does not have a real user authentication mechanism, but only the process validation mechanism for Rpc/mount requests
3. Earlier NFS allows unauthorized users to obtain valid file handles
4. In RPC remote Call, a SUID program has superuser privileges
Ways to enhance the security of NFS:
1, reasonable set/etc/exports in the shared directory, it is best to use Anonuid,anongid to make Mount to NFS server client only have the minimum permissions, it is best not to use Root_squash.
2. Use the iptable firewall to limit the range of machines that can be connected to NFS server
Iptables-a input-i eth0-p tcp-s 192.168.0.0/24--dport 111-j ACCEPT
Iptables-a input-i eth0-p udp-s 192.168.0.0/24--dport 111-j ACCEPT
Iptables-a input-i eth0-p tcp-s 140.0.0.0/8--dport 111-j ACCEPT
Iptables-a input-i eth0-p udp-s 140.0.0.0/8--dport 111-j ACCEPT
3, in order to prevent possible Dos attacks, it is necessary to set the NFSD copy number reasonably.
4, modify/etc/hosts.allow and/etc/hosts.deny to limit the purpose of the client
/etc/hosts.allow
Portmap:192.168.0.0/255.255.255.0:allow
Portmap:140.116.44.125:allow
/etc/hosts.deny
Portmap:ALL:deny
5. Change the default NFS port
NFS uses 111 ports by default, but you can also use the port parameter to change this so that you can increase security to some extent.
6. Using Kerberos V5 as the login verification system

File sharing between multiple servers in Linux