My view on the architecture design of platform website

Intermediary transaction SEO diagnosis Taobao guest Cloud host technology Hall

From an architectural architect's perspective, architecture is a set of guidelines for building systems. With this set of guidelines, we can divide a complex system into a set of simpler subsystems that should remain independent and consistent with the entire system. And each subsystem can continue to subdivide, thus forming a complex enterprise-class architecture.

A selection of technical solutions and physical structures

How to choose the technical solution and the physical structure, for many people just contact the platform Web site development, this may be a headache. The source of these problems is simply the ability to improve the efficiency of development, so that the platform has high performance and load. There are a few questions that I've encountered that are common:

A) Development of languages and databases

When it comes to developing languages and databases, many people begin to compare languages, the most common of which are: "ASP.net and Java which is good", "interpretative language and compiled language which is good" and so on. The most important thing I feel is that you and your team are best at developing languages and databases, which is the old saying: "工欲善其事, Prerequisite!", while the development of the language and the database help. If you choose a language that is not very familiar, perhaps the language and database are better than the language you have in basic performance, but the learning curve is definitely long in the development process. And when you encounter problems because of unfamiliar reasons, waste more time to find solutions, and find the method is not necessarily the best, perhaps not as well as you own familiar language to solve the fast.

Perhaps a friend would say: "I am familiar with these kinds of development languages and databases," It depends on how familiar you are with these development languages and databases, the more in-depth understanding of the characteristics of various development languages and databases, the more help to improve development efficiency. and the current mainstream development language and database provides performance tuning, only in-depth understanding of the development of language and database characteristics and principles, then performance tuning is very easy.

Personal awareness of the importance of these two points, development efficiency and performance.

B Maturity framework or self realization

Many of the current users of the mainstream development language have provided the framework for their own implementation, such as the "S-s-h" portfolio in Java, Python's Djangoo, and so on. My personal experience is to use the open source mature framework as much as possible, because the platform developed early using a mature open source framework to improve development efficiency and quality assurance. I have taken over a platform of the revision, the framework is the front developers write their own, some of the design ideas are not very mature, resulting in the platform in the load increased performance is poor, rectification up very troublesome, can only 1.1 points of separation, time-consuming and experience.

Some friends may ask what is the mature framework, personal summary of several points:

1 can provide the use guide, for example Cookbook, uses guide and so on. With these offers, getting started is easy to use, easy to maintain, and helps you understand its characteristics and principles.

2 have official support, such as the official discussion community, mailing list, etc., and have a bug collection processing mechanism. There is a saying that the tree under the good shade, with official support, when the use of the process encountered problems, directly through the search for the use of previous experience and problems to solve problems, encountered a bug, submitted to, can also find a solution to the law.

3 official in constant update release stable version. This is important, if the official in time to help you solve the current known or unknown bugs, so for the user, there is no worries, if the official stop the update, then I suggest to change the home earlier, because if the framework is good, then it will be better and more and more official will continue to update it. And the stability is always the first place, you can do without affecting the production environment for seamless upgrade updates.

4 users around a lot, often can see the relevant discussion or summary. At present, many mature frameworks are published by foreign developers, if the user E is not good is also a nuisance, then if there are many similar users and a lot of discussion, then for users is a kind of gospel, together to explore and learn.

In addition, the best is open source framework, platform initial access is small, so the performance requirements are not high, the use of mature framework will not have any problems. As the volume of traffic increases dramatically, so does the performance requirements, and the hidden problems in some frameworks arise. At this time, if the framework of open source, users can understand its source code, insight into its implementation mechanism, according to their own actual situation to tune. If not then users can only change direction to solve problems, all roads to Rome.

c) Web server/db Server/cache Server related

The Web server/db Server/cache Server is important in architecture design, and I personally feel that this piece must be used with forward-looking, easy to configure, able to monitor and maintain products, summed up several points:

1 rich and in-depth configuration options. If you can provide rich and in-depth configuration options, you can easily operate on security and performance tuning without disrupting the actual production environment.

2 is based on the high concurrency model. For example, the popular nginx based on Epoll in recent years can effectively reduce connection processing time and increase concurrent number.

3 Support load balancing and request distribution. When the platform has increased access, the single server is certainly difficult to support, this time need to increase the server to share the pressure, this time server load balancing and request distribution is very important.

4 efficient caching mechanism. An efficient caching mechanism helps the platform to increase load capacity and reduce the read and processing time of duplicate resources. such as squid,varnish for small file caches, memcached for database caching, and so on.

5 real-time State monitoring mechanism. Real-time monitoring status reports can help the platform maintenance staff to quickly understand the performance of the platform, according to the situation to adjust.

If it is open source, it would be better to have a deeper understanding of its source code and configure and customize it according to its actual needs.

d) Operating system

Select the appropriate operating system, the most important personal feeling is stable and safe, easy to manage and maintain, easy to monitor. Stable and secure operating system in general, the official will continue to release patches and new versions to solve bugs and vulnerabilities. and the official or third party will continue to provide new management and maintenance monitoring tools, and enable management maintenance personnel to write scripts to maintain management. and the appropriate operating system allows developers to take full advantage of its characteristics and maximize the performance of the platform.

f) Physical Structure

The physical structure here refers to how the server is built. Some friends may have limited resources only one server, some friends have enough resources to have more than 10 servers or more, I personally feel that this is not a problem. Early in the platform, I think most of the traffic is not high, the Web server/db server/cache server on a server is no problem. But it's best to be able to predict what size the platform will grow, and when it comes to architectural design, decide what to do with the physical architecture, and prepare for future schema upgrades. Here, think of the former Baidu architect Thunder said a word, when your membership number reached 5 times times or 10 times times, the architecture will be upgraded.

Two platform research and development

Prior to a good technical plan, entered into the real research and development process, personal sense platform for the development of the site is different from the traditional IT project research and development, because formerly is the customer/demand analyst/art to negotiate between, and now the platform Web site research and development will be more contact with a role called products, The product determines what the final platform Web site is, what the function is, what the process and use case of each function is like, that is, prototype design. And after the developer is implemented, the testers are tested. For the prototype design, please see my other article "Project Requirements prototype design".

In the process, the product will often ask the research and development staff: "So-and-so function is so, you quickly to achieve and solve." This function is not correct, to change. That function is problematic, and the developer may be busy with other functions, so it's easy to create conflicts. Here I recommend the use of agile development methods, set up a short release cycle for iterative development, product issues to be unified in a cycle of resolution, to the next cycle together, to the next cycle of the next cycle of functional improvement and bug fixes. And the use of Jira this mature project management system Management, for the previous changes to leave history, experience.

So in the normal development process, especially team development, I personally feel the need to pay attention to several points:

1 appropriate development tools. or the words "工欲善其事, prerequisite!", the use of appropriate development tools and plug-ins, can improve development efficiency, save development costs. The team uses a unified development tool to reduce the chance of error, prevent version conflicts, and so on.

2 How to control code quality. Because the team's level has high and low, so team development, need to establish a fixed development norms, such as: "Naming norms", "Code package Reference specification." When someone solves a function, it is best to draw a flowchart and a description of the design intent to make it work, to ensure the quality of the code and to reduce the chance of error.

3 new frameworks need to be introduced. Sometimes, a member will feel that the new features of a certain framework are very useful or very suitable for the problem at hand, so to introduce this new framework, my advice, on the basis of a good understanding, is not to introduce a bunch of features that are not in use because of an attribute that would make the project code redundant.

4 knowledge summarization and training. When a member encounters a problem, and after solving or learning something new, we might as well come up with a discussion, perhaps to help improve the performance of the platform, to provide better design ideas.

Three-Frame optimization

"Premature optimization is the root of all evils", so on the architecture optimization, I put it on the completion of the research and development after the launch. There is no hundred percent usable architecture, it depends on your actual business processes and operating conditions to optimize. When you run for a period of time, collect certain data, find out the weakness of the performance of targeted adjustment and optimization, when the platform's load strength to a certain extent, you have to immediately start to do the schema upgrade.

Some friends will ask, sometimes the site is inexplicably slow, but do not know how to do, or by experience to change this change that option, a little better but not completely. My experience is to start with the data, circle from the periphery, and find the source. Start collecting logs from the periphery, such as access_log access log or Sql_log database operation log, find the most access to the 10 logs and the longest execution of 10 logs, and then according to the log to find out exactly what caused the operation, and then a section of the solution. If it doesn't work out, then consider refactoring. The other problem solution is similar to this, do not repeat. From my own experience, it is often because of the deterioration of several functional points, causing the overall performance deterioration.

So in research and development, the realization of the function point should be considered, the front-end part, page, picture and so on the size and effective caching, the back-end of local data and global data cache Efficient use of database layer SQL statements as far as possible to avoid cross table query, database index utilization.

Four other related

Storage

As the amount of access to the platform's web site grows, so does the data, so it's important to have a good way of storing the data early on.

Now more common is the hash URL, according to the hash of the file name to choose to store different directories, such as 20091014131213_abc.xxx then stored to 2009/10/14/a/20091014131213_ Abc.xxx in such a directory, convenient later according to the directory to divide the server.

Search

Data search has become a problem as the number of visits to the platform Web site grows. There must be a friend who would say that the direct database fuzzy query has any problems, you think when your data table has millions of data you use like can not use index, that is the whole watch scan, how much time to take, a person query is not a problem, the hundreds of, then your platform will not stop the dishes. Fortunately now there is a mature program lucene, as long as it is provided by the interface to implement, you can use.