Nginx Fastdfs Distributed Memory Module test method _nginx

Look back to see Fastdfs update very quickly, also see fastdfs-nginx-module_v1.01.tar.gz nginx module, so today on a test machine test-test a few days to see stable instability, in consideration of replacing waste resources lustre!

Environment:

storage1:192.168.6.100
storage2:192.168.6.101
tracker:192.168.6.102

1. On each machine, download and install Fastdfs

Nginx $> wget http://fastdfs.googlecode.com/files/FastDFS_v2.04.tar.gz
# extract
nginx $> tar zxvf fastdfs_ v2.04.tar.gz
nginx $> cd Fastdfs
# because I'm adding nginx modules to Fastdfs, so I don't need fastdfs to support HTTP so I don't need to remove #WITH_HTTPD = 1 comments before, Direct compilation of
Nginx $>./make.sh
nginx $>/make.sh. Install

2. Modify the tracker and storage configuration files
# Modification of the tracker

Tracker $> vim/etc/fdfs/tracker.conf

Disabled=false #配置是否生效 bind_addr=192.168.6.102 #绑定IP port=22122 #服务端口 connect_timeout=30 #连接超时时间 network_timeout=60 #
The network timeout for tracker server, in seconds. Base_path=/home/yangzi #目录地址, which creates data (storing server information), logs, log files max_connections=256 #系统提供服务最大连接数 work_threads=4 #线程数, Usually set CPU number store_lookup=2 upload Group (volume) mode 0: Polling mode 1: Specify Group 2: Balance Load (select the maximum remaining space of the Group (volume) upload) If you specify the upload to a fixed group at the application level, this parameter is bypassed store_group= Group1 when the previous parameter is set to 1 o'clock (store_lookup=1, that is, the group name is specified), you must set this parameter to a group name that exists in the system. If you choose another way to upload, this parameter will not work. Store_server=0 choose which storage server to upload (after a file is uploaded, this storage server is equivalent to the storage server source for this file, Will push this file to the same group of storage server to achieve sync effect) # 0: Polling Way # 1: Sorting by IP address Select First server (minimum IP address) # 2: Sorted by Priority (upload priority is set by storage server, parameter is named up load_priority) store_path=0 Select which directory in the storage server to upload.
Storage server can have multiple base path (which can be understood as multiple disks) that holds files. # 0: Take turns, multiple directories in sequence to store the file # 2: Select the largest remaining space directory storage file (note: The remaining disk space is dynamic, so the stored directory or disk may also change) download_server=0 choose which storage server as the download service # 0: Polling mode, you can download the current file any storage server # 1: Which is the source storage server on which one (previously said this storage server source is generated) is the previous uploadTo which storage server is that? reserved_storage_space = 4GB Storage server reserved space to ensure system or other application demand space (indicates that if the same group of servers have the same size as the hard disk, the smallest is the same. That is, as long as one of the servers in the same group meets this standard, the standard takes effect because they are backed up) log_level=info #选择日志级别 run_by_group= #操作系统运行FastDFS的用户组 run_by_user= # The operating system runs FASTDFS user allow_hosts=* #可以连接到此 Tracker server IP range (affects all types of connections, including clients, storage server) Sync_log_buff_interval
= 10 # Sync or refresh log information to the hard disk at intervals of seconds # Note: Tracker server log is not always write hard disk, but write memory first.
Check_active_interval = 120 # Detects storage server's surviving time in seconds. # Storage Server periodically sends a heartbeat to tracker server, if tracker server has not received a heartbeat in storage server within a check_active_interval, There will be a view that the storage server is offline. Therefore, the value of this parameter must be greater than the heartbeat interval configured by the storage server.
Typically configured to be twice times or 3 times times the storage server heartbeat time interval. Thread_stack_size = size of 64KB # line stacks. The FASTDFS server side is threaded.
To correct this, the tracker server thread stack should not be less than 64KB, not 512KB. # The larger the line stacks, the more system resources a thread occupies.
If you want to start more threads (v1.x the corresponding parameter is max_connections, V2.0 is work_threads), you can reduce the value of this parameter appropriately. Storage_ip_changed_auto_adjust = true # This parameter controls whether the cluster automatically adjusts when the storage server IP address changes.
Note: Automatic tuning is only completed when the storage server process restarts. Storage_sync_file_max_dElay = 86400 # V2.0 introduced parameters. Maximum latency for synchronizing files between storage servers, by default of 1 days. Adjust storage_sync_file_max_time = V2.0 introduced parameters according to the actual situation.
The maximum time the storage server needs to synchronize a file, by default, is 300s, or 5 minutes. Http.disabled=true # HTTP service does not take effect of course, when compiling, I have removed with_httpd macros, HTTP.SERVER_PORT=80 # HTTP Service Port # The following parameters can only be used to open the HTTP service Http.che ck_alive_interval=30 http.check_alive_type=tcp http.check_alive_uri=/status.html Http.need_find_content_type=true

Modification of the #两台 storage.conf

Storage $> vim/etc/fdfs/storage.conf

Disabled=false #配置是否生效 group_name=group1 #storage所在组 (volume) bind_addr=192.168.6.100 # binding IP, another too storage IP for 192.168.6.101 Client_bind=true #bind_addr通常是针对server的.
This parameter is valid when BIND_ADDR is specified. port=23000 # is the storage service Port connect_timeout=30 # Connection timeout, for socket socket function Connect NETWORK_TIMEOUT=60 # Storage Server network timeout, unit
is seconds.
HEART_BEAT_INTERVAL=30 # heartbeat interval, in seconds stat_report_interval=60 # Storage Server reports to tracker server the time interval, in seconds, of the disk's remaining space.  Base_path=/home/eric # Base_path directory address, root directory must have subdirectories will automatically generate # will generate data (datastore), logs log file max_connections=256 # Maximum number of connections Buff_size =
256KB # Sets the buffer size for the queue node.
Work_threads=4 # Number of worker threads disk_rw_separated = true # disk IO read/write is separate, default is separate. Disk_reader_threads = 1 # Number of read threads for a single storage path, the default is 1 disk_writer_threads = 1 # The number of write threads for a single storage path, and the default value of 1 sync_wait_msec=200 # when synchronizing files, if from
Binlog did not read the file to sync, sleep n milliseconds after re-read, 0 for no sleep, and immediately try to read again.
Sync_interval=0 # After synchronizing the previous file, synchronize the next file interval in milliseconds, 0 for no sleep, and direct synchronization of the next file. sync_start_time=00:00 sync_end_time=23:59 # allows the system to synchronize the time period (the default is all day).
Generally used to avoid the peak synchronization of some problems and settings, I believe SA will understand. WRITE_MARK_FILE_FREQ=500 # put Storage's MarkThe time interval at which files are periodically synchronized to disk, in seconds Store_path_count=1 # The storage server supports multiple paths (such as disks) when files are stored.
This configures the number of base paths in which files are stored, usually with only one directory. Store_path0=/home/eric # Configure Store_path paths one at a A and the index number is based on 0.
Note that there is 0,1,2 behind the configuration method ..., you need to configure 0 to Store_path-1.
# If you don't configure Base_path0, it's the same path as Base_path. SUBDIR_COUNT_PER_PATH=32 # Fastdfs When storing files, a level two directory is used. This configures the number of directories to hold files tracker_server=192.168.6.188:22122 # Tracker_server list to write port Oh log_level=info # log Level run_by_group= # run Stor
Age User group Run_by_user= # Run Storage User allow_hosts=* # allows you to connect IP list file_distribute_path_mode=0 # files in the data directory to disperse the storage strategy.
# 0: Take turns storing # 1: Random storage file_distribute_rotate_count=100 # This parameter is valid when the above parameter File_distribute_path_mode is configured to 0 (alternate storage mode). #当一个目录下的文件存放的文件数达到本参数值时, subsequent uploaded files are stored in the next directory Fsync_after_written_bytes=0 # When writing a large file, every n byte is written, and the system function Fsync is forced to synchronize the content to the hard disk. 0 indicates the time interval, in seconds sync_binlog_buff_interval=60, that fsync sync_log_buff_interval=10 # Sync or refresh log information to the hard disk is never invoked
Sync binglog (Update action Log) to hard disk time interval, in seconds sync_stat_file_interval=300 # The time interval, in seconds, to sync storage stat files to disk. The size of the THREAD_STACK_SIZE=512KB # line stacks.
The FASTDFS server side is threaded.
# The larger the line stacks, the more system resources a thread occupies. UpLOAD_PRIORITY=10 This storage server as the source, the upload file priority, can be negative. The smaller the value, the higher the priority. This corresponds to the store_server= 2 o'clock configuration in tracker.conf if_alias_prefix= check_file_duplicate=0 # to detect if the uploaded file already exists.　If it already exists, there is no file content, and a symbolic link is created to conserve disk space. Combined with the use of FASTDFH. 1 is detection, 0 is not detected, we do not use FASTDFH of course 0 Key_namespace=fastdfs # When the last parameter is set to 1 or Yes (True/on is also possible), in Fastdht namespace Keep_alive=0 # and FAs TDHT Servers connection (whether it is a persistent connection) # The following HTTP configuration is not much to say Http.disabled=true http.domain_name= http.server_port=80 http.trunk_siz E=256KB Http.need_find_content_type=true

3. Establish the root directory of tracker and storage

# tracker
Tracker $> mkdir-p/home/eric
# storage
Storage $> mkdir-p-/home/yangzi

4. Download on a storage, for example, I download Nginx and fastdfs-nginx-module modules on the 192.168.6.100.

Storage $> wget http://www.nginx.org/download/nginx-0.8.53.tar.gz
storage $> SVN export http:// Fastdfs-nginx-module.googlecode.com/svn/trunk/fastdfs-nginx-module-read-only

5. Compile and install Nginx with Fastdfs-nginx-module module

Storage $> tar zxvf nginx-0.8.53.tar.gz
storage $> CD nginx-0.8.53 storage. $>/configure
/local/nginx--add-module=/root/fastdfs-nginx-module-read-only/src
Storage $> make
storage $> make Install
# copy mod_fastdfs.conf to/etc/fdfs/
storage $> cp/root/fastdfs-nginx-module-read-only/src/mod_ fastdfs.conf/etc/fdfs/

6. Modify Nginx configuration file increase

Storage $> vim/usr/local/nginx/conf/nginx.conf

# Add a little bit
location/m00 {
Alias/home/eric/data;
Ngx_fastdfs_module;
}

7. Make a soft connection to the storage storage directory

Storage $> Ln-s/home/yangzi/data/home/yangzi/data/m00

8. Start two storage and tracker Nginx

# Start Tracker
tracker $>/usr/local/bin/fdfs_trackerd/etc/fdfs/tracker.conf
# start storage storage $>
/ usr/local/bin/fdfs_storaged/etc/fdfs/storage.conf
# start Storage2
storage2 $>/usr/local/bin/fdfs_ storaged/etc/fdfs/storage.conf
#在 Storage boot nginx
storage $>/usr/local/nginx/sbin/nginx

9, upload the file test.
# Modify Client configuration file

Storage $> vim/etc/fdfs/client.conf
connect_timeout=30
network_timeout=60
Base_path=/home/yangzi
tracker_server=192.168.6.102:22122
log_level=info
#下面参数无所谓了反正没有用到http service
Http.tracker_ Server_port=80

Storage $> vim a.html
test fastdfs!

Storage $>/usr/local/bin/fdfs_test/etc/fdfs/client.conf Upload a.html This are FASTDFS client test program v2.04 Copy Right (C) 2008, Happy fish/yuqing Fastdfs May is copied only under the terms of the GNU general public License V3, whic
H May is found in the FASTDFS source kit.

Please visit the Fastdfs home Page http://www.csource.org/for more detail. Base_path=/home/yangzi, connect_timeout=30, network_timeout=60, Tracker_server_count=1, anti_steal_token=0, anti_ Steal_secret_key length=0 tracker_query_storage_store_list_without_group:server 1. Group_name=group1, ip_addr=192.168.6.100, port=23000 group_name=group1, ip_addr=192.168.6.100, port=23000 Storage_ Upload_by_filename group_name=group1, remote_filename=m00/00/00/wkggvez3y9maaaaaaaaadigvbpc73.html source IP address:192.168.6.100 file timestamp=2010-12-02 17:16:03 file size=14 file crc32=674197143 file url:http://192.168.6.10 0/group1/m00/00/00/wkggvez3y9maaaaaaaaadigvbpc73.html Storage_upload_slave_by_filename GrouP_name=group1, remote_filename=m00/00/00/wkggvez3y9maaaaaaaaadigvbpc73_big.html source IP address:192.168.6.100 File timestamp=2010-12-02 17:16:03 file size=14 file crc32=674197143 file url:http://192.168.6.100/group1/m00/00/00/ Wkggvez3y9maaaaaaaaadigvbpc73_big.html

Open IE Browser access:

This article is from the "linuxer" blog, please be sure to keep this source http:// deidara.blog.51cto.com/400447/440175