Worthy of our deep research and learning: from scratch to build an indestructible web system mainstream architecture

This article tags: Web System Mainstream architecture to build a Web system architecture cache server database schema a talk on the internet of tech start-ups

Topic Description:

1. Current status of website system architecture

2, web system mainstream architecture analysis

3, the Internet technical team early formation experience sharing

This article mainly combined with my previous in the Haier e-commerce platform and now some of the actual structure of the company's experience, combined with the actual situation and personal understanding, to share with you some common technical architecture and application techniques for building web systems.

First of all, to discuss with you a question, is the current traditional it enterprise or traditional enterprise IT systems current system architecture is how?

I have experienced the NEC software, Haier Group, Qingdao Airlines, and so on to some extent belong to the traditional enterprises. I am also in the last three years in the Haier business e-commerce platform, only to more contact with the Internet thinking and technical framework.

When I was discussing with Qingdao Airlines for more than a year and other it peers in Qingdao, I found a phenomenon: the technical architecture of many traditional enterprises is more and more similar to the technical architecture of Internet enterprises, or the traditional enterprises are more and more inclined to the mainstream technology architecture and server deployment of the Internet.

Although the traditional enterprise may not have the Internet Enterprise's large traffic, the data volume, high concurrency (the Internet enterprise really high traffic is so few), but the two in the technical framework of many aspects, the direction is consistent, personal feeling this is a better phenomenon. The traditional enterprises can learn from some excellent technology architecture and deployment ways of Internet enterprises, so as to better protect their business system and improve the efficiency of system utilization. A mature open source technology architecture can also save a lot of it costs for your business.

Qingdao Airlines has its own official website, occasionally engaged in a ticket, promotion or Shuyun Spring Festival, and sometimes by some network malicious attacks. At this point, although the flow will be steep, but in the current technical framework can be completely resisted.

This article will analyze the current mainstream of some Web technology architecture , may be more suitable for small and medium-sized internet companies or some of the large and medium-sized traditional enterprises, the technical quality of the key to see how the actual situation, I hope you can have a harvest, can be in their respective areas of the framework of the system, can be helpful.

Schema master diagram

Next go to the point, first look at this general picture:

Network access Layer

1 firewall

All access requests (except for intra-enterprise access) are subject to enterprise-level firewall devices. Whether it is the company's own computer room or the managed IDC room, the most outer layer is the firewall to check all access requests. For some malicious Trojan implants and so on, the firewall will block most of the. As a Web schema, the outermost layer must choose the firewall, and the firewall of the minimum structure of the general choice of two different models.

2 Safety Equipment

The firewall will then be connected to the IPs (Intrusion Prevention System), the WAF (Web Application Protection system). This area is mainly for network security, system security to do detection and protection, you can use commercial equipment (recommended), insufficient funds of enterprises can also use open source equipment, here recommend an open source product Ossim, interested students can understand.

3 Load Balancing

After the network security protection, next is our hard-load equipment (this layer is optional), the general hard load balancing equipment mainly has f5,a10, relatively expensive, enterprises can choose according to the situation.

Hard loads are typically followed by a soft load (of course, both soft and hard loads can be left to one or both). The soft-load layer also typically deploys a reverse proxy server, which acts as a reverse proxy, and also acts as a security guard.

Typically on a network plan, the layer is located in the DMZ area, and the servers below it are located in the intranet. This block separates the external requests and the direct interaction between the intranet and improves security. In general, the technical choice of this layer has NGINX,APACHE,HAPROXY,LVS and so on. Most should be a nginx, can do load balancing, can also do reverse proxy, and relatively high concurrency efficiency is better.

About the differences, there are many online, interested students can be a lot of comparison. It is stated that the LVS is working on the Network 4 layer only for distribution, no traffic generated, the other three are working on the 7 layer above, if not suitable for hard-load equipment, it is recommended to use LVS as a load device for traffic forwarding, and then the Nginx or Haproxy. Apache in some traditional enterprises exist or use more, also more stable.

Front-end architecture

Generally after load balancing is mounted on a variety of application servers . Deploying an application server typically deploys a single server, such as static resources (JS,CSS, pictures, files), to reduce the bandwidth and IO of the application server and improve access efficiency. Deploy these static resources to static resource servers, file servers, picture servers, and so on. Generally, if we have a CDN, these static resources will be placed on the CDN to increase the speed of network loading. The technical architecture of common file server and picture server is Fastdfs,mogilefs,graphicsmagick and so on.

However, SMEs recommend direct purchase of cloud services. One is to reduce operation and maintenance costs, the second is to improve the speed of access, general cloud services are paired with a CDN. Build your own file or image server operation and maintenance costs are relatively high, the technical requirements are also more in-depth. Everyone here needs to think carefully when it comes to architecture.

Application Server

1 Web Application Server

Application servers are generally tomcat,iis,resin and so on. Generally there is an application depending on the situation there will be more than one server (at least 2), the application to decouple, the application between the dependencies as far as possible to use interface interaction (as far as possible to avoid database dblink, etc.). When you do the application system decoupling can refer to the current more popular service, micro-services and other technical architecture such as dubbox, but need to have a certain understanding of development. Although our team has experienced and is doing dubbox service, but I do not participate in many, so also hope to learn more from you.

2 Message Queuing server

There are several benefits to increasing the Message Queuing server:

1. Because the Message Queuing server is much faster than the database, it can quickly process and return data;

2. The Message Queuing server has better extensibility;

3. In the case of high concurrency, delaying writing to the database can effectively reduce the pressure on the database.

Message Queuing is often used in high-concurrency applications (such as snapping), high-speed data interactions between different system modules. Commonly used Message Queuing technology has ACTIVEMQ,RABBITMQ, and so on, these technologies themselves have a good cluster or the main preparation mechanism, and have a monitoring page, very convenient and rapid expansion and use. When monitoring is in use, the script (curl gets the value of the Monitoring page and the HTTP Staus of the Monitoring page) or other way to monitor, realizes the automatic fault alarm.

3 Cache Server

Data cache server, often deployed with Memcached, Redis, etc., currently should be the majority of Redis bar. In addition, the session Problem of application server cluster is often used in Redis. Redis's own Sentinel mode, cluster cluster (3.0 + support) avoids single points of failure, facilitates horizontal and vertical scaling, caches hotspot data to improve access efficiency, and is often used in high concurrency environments.

It is important to note that not all Web architectures require Message Queuing or database caching, depending on the system's concurrency and volume of traffic. The best is what is right for you.

Database tier

1 Database Connection Pool

Between the application and the database to avoid the application of direct connection to the database, the use of database connection pool way. The advantages of database connection pooling technology include resource reuse, faster corresponding speed, unified connection management, avoiding connection leakage and other benefits. Commonly used have c3p0,dbcp,druid and so on, here strongly recommend Druid.

2 Database Schema

The database connection pool is followed by the database. The database kind is also more, commonly has oracle,mysql and so on. Of course, a system uses a set of databases to avoid using the same database for multiple sets of applications.

Due to the importance of the database, database scenarios need to be taken into account. Including the implementation of high availability of databases, load balancing, and some e-commerce platform also need to realize the read-write separation, the horizontal vertical split of the database, to achieve complex database applications.

Oracle's Common architecture is RAC,DG (Dataguard), and Oracle is cost-efficient, so many SMBs choose MySQL. MySQL also has different branches and technical solutions, such as the official version of MARIADB,PERCONADB. Commonly used high-availability architecture has replication, Cluster, different branches have support, here I recommend you use more than MariaDB10.0 version, the efficiency is relatively high.

MySQL middleware is also more, used to support load balancing, read and write separation, sub-database sub-table and so on. such as ONEPROXY,MYCAT, etc. are very good MySQL database middleware, it is recommended that you have time to study more, the structure of a stable and reliable database cluster.

Database backup and recovery This is not a separate statement, as described in the following disaster preparedness scenarios.

3 Storage devices

General Enterprises will have professional storage equipment. The raid selection, primary and standby architecture of the storage device need to be architected and discussed with the storage vendor in advance. As one of the most critical devices, be sure to avoid a single point of failure, otherwise it will lead to an entire IT system outage.

The above is an overview of the common Web system architecture, as well as some common technical solutions to the description, there are shortcomings, please give us a lot of advice, learn from each other.

Disaster preparedness Plan

Next, let's talk about backup related issues. Regardless of the traditional enterprise or the Internet, backup must be a key and important task. Without backup, it means that the system does not have the most basic protection.

Common disaster preparedness plan is the same city hot preparation, the way of disaster preparedness, that is, the three centers of the two ways. The network delay in the same city can generally be relatively small, so in the use of real-time hot standby method is feasible. The application server, database and so on through the real-time synchronization, data transmission to the other room in the same city, to achieve cross-room hot standby.

A way to use delayed backups offsite. The backup set of the local computer room is transmitted over the network to a remote machine room to realize the remote disaster preparation. One of the remote disaster preparedness is a data delay, the general day.

No matter the application server, or the way of data backup, there are many ways, because the time limit will not be shared with you. It is important to note that the test scheme of the backup set must be combined with the disaster preparedness plan and strictly timed according to the test plan to ensure the accuracy of the backup set.

Other aspects

As a complete set of IT technology architecture solutions, there are many aspects to consider, such as monitoring solutions. Our common monitoring program has Lepus monitoring database, Zabbix, script three ways. By mail, Ali is more than SMS and other ways to send alarms, log server for collecting and analyzing logs, such as elk. There's a general enterprise that has a data platform for analyzing its own data.

In addition, the general enterprises in order to save costs will consider virtualization, the server and other hardware resources to virtualize, improve utilization to save the cost of enterprise, and thus lay the foundation for the private cloud construction of the enterprise. I would like to have the opportunity to communicate with you the virtualization + Private cloud technology solutions, which we are now embarking on, very practical reference.

About the initial technical team of Internet startups

These are the technical aspects of the architecture, the next time to share briefly on the technical team during the start-up of some of the lessons and ideas, welcome to shoot bricks.

The main points are:

1. Core business as the center, the initial technical team to continuously meet business needs. Avoid blindly expanding the size of the team and adopting an overly strong technical architecture to match the needs and scale of business development.

2. The proposal is primarily outsourced, but the core business and technology architecture must be dominated and not shaken by the people in their own teams. To have the ability to take over quickly after outsourcing projects.

3. Team to small and fine, flat management, less management, multi-technical posts, the team should have a common goal and development vision.

The traditional enterprise transformation of the Internet should pay particular attention to, even if deep pockets, but also careful. Personally, experienced IT team just a year or so of time from 13 to 200 supposedly people, business planning has not broken the situation, the end of a large number of layoffs, desperately struggling in a state.

Friendship reminds you, when you notice that team members work is not saturated, but the same position also has the recruitment plan, must consider whether the recruitment or the attrition.

Looking at a group of down-and-down startups, the lessons of failure are well worth our deep research and learning.

is written at the end: for Freedom look outside the world, and it this line, not to go to Google data, finally, Amway an accelerator agent. a red apricot accelerator , to Google data is the absolute first choice, fast connection, easy to use. I bought the 99￥ for a year, after registering with this link (http://my.yizhihongxing.com/aff.php?aff=2509 ) promo code WH80, lifetime 85 percent , split down, only 7 yuan per month, special benefits.

This article tags: Web System Mainstream architecture to build a Web system architecture cache server database schema A talk on the internet of tech start -ups

Turn from SUN's BLOG-focus on Internet knowledge, share the spirit of the Internet!

Original Address: " worthy of our deep research and learning: starting from scratch to build indestructible web system mainstream architecture "

Related reading: Aaron swartz– The internet genius of the life course: every moment asked himself, now the world what is the most important thing I can participate in doing? "
Related reading: " site environment Apache + PHP + mysql xampp, how to implement a server on the configuration of multiple sites?" "

Related reading: What is the engineer culture? Why are the engineers alive? Why do I need an engineer culture as an IT or internet company? "

Related reading: useful for programmers: 2017 latest in Google's Hosts file download and summary of the various hosts encountered the problem of the solution and configuration of the detailed

related reading: the Win10 perpetual activation tutorial and how can I see if the Windows system is permanently activated? 》

Related blog:SUN ' S blog -Focus on Internet knowledge and share the spirit of Internet! Go and see:www.whosmall.com

Original address: http://whosmall.com/?post=233

Worthy of our deep research and learning: from scratch to build an indestructible web system mainstream architecture