"Reprint" "Java Second technology picture upload" based on Nginx and Fastdfs, complete the upload and display of pictures

Based on Nginx and Fastdfs, complete the uploading and display of product images.

One, traditional image storage and display way

A problem exists:

1) When uploading a large amount of concurrent images, you need to load balance the Web application, but there is a picture sharing problem

2) The Web application server has limited storage space, which is a Web application server, not a storage server.

3) The Web application server's own IO read and write performance is not high, the picture upload download, the speed is slow

4) When the Web application server accesses the picture, because the picture content is large, the concurrency is big, it will occupy the bandwidth of the Web application server, so the other functions of the Web application server will be affected greatly.

Solution:

1) Store the image in a picture server with high IO read and write performance.

2) When downloading a picture, use the HTTP server to read the picture directly from the image server.

Second, cluster mode

(1) Nginx Quick Start

Nginx is a high-performance HTTP and reverse proxy server, also a IMAP/POP3/SMTP server. Mainly applied to:http Server , reverse proxy server \ Load balancer server , virtual host .

installation of Nginx (see also "e-commerce common Software Installation article")

Nginx usage (implemented by modifying the nginx.conf configuration file)

1) Virtual Host configuration

Each computer in the network is divided into multiple virtual hosts, and each virtual host can provide the WWW service separately.

a) port-based configuration (IP address consistent, port inconsistent )

#基于80端口的虚拟主机 server { Listen; server_name 192.168.242.128; Location / { root HTML; Index index.html index.htm; } }

#基于81端口的虚拟主机 server { Listen Bayi; server_name 192.168.242.128; Location / { Root html81; Index index.html index.htm; } }

b) based on the domain name configuration

① Modify the Hosts file to implement the mapping configuration for IP addresses and domain names:

Hosts file location: C:\Windows\System32\drivers\etc\hosts

We recommend that you modify the local DNS configuration using the switchhosts tool.

② Configuration nginx.conf

#基于www. aaa.com domain name of the virtual host Server {         listen 80 ; server_name  www.aaa.com; location  /{ Root htmlaaa; Index index.html index.htm; } }

#基于www. bbb.com domain name of the virtual host Server { LISTEN&N Bsp;80 ; server_name www.bbb.com; location / { root htmlbbb; Index index.html index.htm; } }

2) Reverse proxy configuration

The usual proxy server, which is used only to proxy connection requests to the Internet by the internal network, while the reverse proxy (Reverse

Proxy) means to accept a connection request on the Internet by using an agent server.

Simple understanding: Tomcat as the center, with internal and external network as the premise, Tomcat initiative is a forward proxy, and Tomcat

A passive accept request is a reverse proxy.

3) Load Balancing configuration

Simple understanding: The reverse proxy load balancing technique is to load balance by dynamically forwarding a connection request from the Internet to multiple servers on the internal network in a reverse proxy manner.

① Load Balancing method : Hard Load (F5 server), soft Load (Nginx)

② Load Balancing Strategy : polling ,ip_hash , etc.

③ Specific configuration

Tomcat+nginx Demo:

In a reverse proxy case, only the 8080-port service responds in the background when accessed through the domain name www.tomcat1.com.

When the load balancer is configured, the backend will add a 8282 server response service in addition to the 8080 port response when accessed through the domain name www.tomcat1.com.

Modify the Nginx configuration file and modify it as follows:

Upstream tomcat1{ #Weight weight, default is 1, the higher the weight, the greater the chance of being allocated server 192.168.242.128:8080 weight= 2; server 192.168.242.128:8282; } #配置一个虚拟主机 Server { Listen 80; server_name www.tomcat1.com; Location / { Proxy_pass HTTP://TOMCAT1; } }

(2) Fastdfs

Fastdfs is an open source Distributed file system written in C language . Fastdfs is tailor-made for the Internet, taking into account such mechanisms as redundant backup , load Balancing , and linear scaling , and focusing on high availability , high performance indicators, Using Fastdfs it is easy to build a high-performance file server cluster to provide file upload , download and other services.

1) Fastdfs Architecture (Tracker server + Storage server)

The role of the ① Tracker server is to load balance and schedule the storage server , request Tracker Server directly when the file is uploaded, and then Tracker Server can find storage server to provide file upload service according to some policies. So you can call tracker a tracking server or a dispatch server .

② Storage Server is a file store , and the files uploaded by the client are eventually stored on the Storage server, Storage Instead of implementing its own file system, server uses the operating system's file system to manage files. You can call storage a storage server .

Specific architectures such as:

2) Tracker Cluster

The Tracker server in a FASTDFS cluster can have more than one, Tracker server is equal to each other while providing services, Tracker server does not have a single point of failure. The client requests tracker Server to poll, and another tracker if the requested tracker cannot provide the service.

3) Storage Cluster

The storage cluster uses a packet storage method . A storage cluster consists of one or more groups consisting of one or more storage servers, and the storage server within the group is an equal relationship between the different groups of storage servers that do not communicate with each other, within the same group storage The servers are connected to each other for file synchronization, ensuring that the files on each storage in the group are exactly the same . The storage capacity of A group is the smallest storage server capacity in the group, the total capacity of the cluster storage is the sum of the storage capacity of all the groups in the cluster , it is obvious that the hardware and software configuration of the storage server in the group is best consistent.

The advantage of using the packet storage method is that it is flexible and controllable. such as uploading a file, can be directly specified by the client to upload the group can also be selected by the tracker scheduling. When a packet storage server has a large access pressure, the storage server can be increased in that group to expand the service capacity ( vertical scaling ). When the system capacity is insufficient, the group can be increased to expand the storage capacity ( horizontal expansion ).

4) Storage Status Collection

Storage server connects all the tracker servers in the cluster and periodically reports its status to them, including statistics such as disk space remaining, file sync status, file upload download count, and so on.

5) File Upload process

After the client uploads the file, the storage server returns the file ID to the client, which is used for subsequent access to the file's index information. File index information includes: Group name, virtual disk path, data level two directory, file name.

N Group name : The name of the storage group where the file was uploaded, and the storage server returns after the file upload is successful, which requires the client to save itself.

n Virtual Disk path : The virtual path of the storage configuration, corresponding to the disk options store_path*. If Store_path0 is configured, it is M00, if STORE_PATH1 is configured M01, and so on.

n data Level two directory : A level Two directory created by the storage server under each virtual disk path for storing data files.

n filename : Different from file upload. is generated by the storage server based on specific information, including information such as the source storage server IP address, file creation timestamp, file size, random number, and file name extension.

6) File download process

Tracker quickly defines a file based on the requested file path, which is the file ID.

For example, request the file below:

1. By group name Tracker can quickly locate the storage server group that the client needs to access, and select the appropriate storage server to provide client access to the group1.

2. The storage server can quickly locate the directory where the file resides, based on the file storage virtual disk path and the data file level two directory, and locate the files that the client needs to access based on the file name.

"Reprint" "Java Second technology picture upload" based on Nginx and Fastdfs, complete the upload and display of pictures