What is the technical architecture of Amazon?

And this: http://www.nedworks.org /~ Mark/presentations/San/wikimedia%20?ture=

Architecture of Amazon
Amazon evolved from a small online bookstore to the largest online bookstore in the world today. They opened up new ways for customers to evaluate, review, and recommend products.
Platform

• Inux
• Orace
• C ++
• Per
• Mason
• Java
• JBoss
• Servets

Status

• More than 55 million active customer accounts
• Over 1 million active retail partners worldwide
• The number of services required to build a page is between 100 and 150.

Architecture

• What do we mean by scalability? For a service, when we increase the system resources allocated to it, its performance growth can be proportional to the invested resources, so we can say that the service is scalable. Generally, performance improvement means that more service units can be provided, or larger work units can be provided, such as increasing data sets.
• Amazon's architecture has undergone tremendous changes. From the first two-tier architecture, we have switched to a distributed, decentralized service platform that provides many different applications.
• At first, only one application was used to interact with the backend, which was completed by C ++.
• The architecture will evolve over time. Over the years, Amazon has focused its primary focus on backend databases, trying to accommodate more product data, more customer data, and more order data, and allow it to support multiple international sites. By 2001, front-end applications were obviously unable to make any additional efforts. Databases are divided into many small parts. A service interface is created for each part, and this interface is the only way to access data.
• Databases gradually evolve into shared resources, making it difficult to add capacity on the basis of all services. The evolution of frontend and backend processing is greatly limited because they are shared by too many different teams and processes.
• Their architecture is loosely coupled and built around services. Service-Oriented Architecture provides them with isolation features, allowing them to quickly and independently develop many software components.
• Gradually, Amazon has hundreds of services and several application servers that aggregate information from the service. The application that generates the http://Amazon.com site page is located above an application server. Applications that provide Web service interfaces, customer service applications, and seller interfaces are similar.
• Many third-party technologies are difficult to apply to websites like Amazon, especially communication infrastructure technologies. They work well within a certain range, but they will not work if the scope is expanded. Therefore, Amazon had to develop its own basic technologies.
• It is not technically "hanging ". Amazon uses JBoss/Java in some places, but only uses servets, and does not fully use the technologies involved in J2EE.
• C ++ programs are used to process requests. Programs developed by PER/Mason are used to generate content on the page.
• Amazon does not like middleware because it looks more like a framework than a tool. If a middleware is used, it will be troubled by the software model adopted by the middleware. You can only choose their software. It is almost impossible to use different software. You are trapped! Message-oriented middleware, data persistence layer middleware, Ajax, and so on. They are too complicated. If middleware can be provided as smaller components, more like a tool rather than a framework, it may be more attractive to us.
• Soap-Related Web solutions seem to want to solve all the problems of distributed systems again.
• Amazon provides soap and rest Web Services. About 30% of users use web services such as soap. They seem to be Java and. Net users and use WSD to generate remote object interfaces. Around 70% of users use rest. They seem to be PHP and per users.
• Whether using soap or rest, developers can get the object interface to access Amazon. What developers want is to complete the work without worrying about what is transmitted over the network cable.
• Amazon wants to build an open community around their services. They chose web services because of its simplicity. In fact, it is a service-oriented architecture. In short, you can only access the required data through interfaces. These interfaces are described by WSD, but they use their own encapsulation and transmission mechanisms.
• The architecture development teams are all small teams and all are organized around different services.
• The Amazon Service is an independent feature delivery unit. This is how Amazon organizes his internal team.
• If you have a new business suggestion or want to solve a problem, you can organize a team. Due to the communication cost, each team is limited to 8 ~ 10 people. They are called two pizza teams. Because two pizzas can make the team members full.
• Teams are all small-scale. They are authorized to solve a problem or enhance a service in any way they like.
• For example, they created such a team and its function is to find unique words and phrases in a book. This team created an independent service interface for that feature and has the right to do anything they think they need to do.
• Deployment
• They created a special infrastructure to manage dependencies and deploy services.
• The goal is to allow all the right services to be deployed on one host. All application code, monitoring mechanism, and license mechanism should be in a "host.
• Everyone has a system to solve these problems.
• The output of the deployment process is a virtual machine. You can run them using EC2.
• To verify the effectiveness of the new service, it is worthwhile to view the service from the customer's perspective.
• From the customer's perspective, the Service focuses on the value you want to deliver to users.
• Forces developers to focus on the value delivered to customers, rather than thinking about how to build a technology before thinking about how to use it.
• Start with the brief features you will see and check whether the services you build are valuable from the perspective of the customer.
• End the design process with minimal design. If you want to build a large distributed system, simplicity is the key.
• State management is a core issue for large-scale Scalable systems.
• Internally, they can provide unlimited storage space.
• Not all operations are stateful. The checkout procedure is stateful.
• By analyzing the sessionid of the recently clicked page, this service can provide users with recommended product suggestions.
• They track and store all the data, so maintaining the status is not a problem. There are some separation states that need to be maintained for a session. The provided services will keep the information, so you only need to use these services.
• Eric brewers 'Cap Theory -- or three attributes of the system
• Three attributes of the system: consistency, availability, and network partition tolerance.
• Any system that shares data must have at least two of these three attributes.
• Network partition tolerance: divides nodes into small groups. You can see other groups but not all other nodes.
• Consistency: Write a value and read it again. The returned value should be the same as the written value. This is not the case in a partition system.
• Availability: it is not always readable or writable. The system may tell you that data cannot be written because data consistency needs to be maintained.
• To achieve scalability, you must partition the system. Therefore, you must select between high consistency or high availability for specific systems. You must find the appropriate overlap between availability and consistency.
• Select a specific implementation method based on service needs.
• For the checkout process, you always want to put more items into the customer's shopping cart, because this can generate income. In this case, you need to select high availability. The error is hidden from the customer and will be analyzed later.
• When a customer submits an order, we need to focus more on maintaining high consistency. Because several different services-credit card service, distribution service, and report function-access those data at the same time.

Lessons learned

• To build a truly scalable system, you must change your mind or mindset. Chaotic methods may work well in probability sense. In a traditional system, we are presenting a perfect world, with nothing going wrong or stopping. Then we construct complex algorithms in this perfect world. In fact, things always go wrong. This is what you must accept. For example, try to think about how to quickly restart the server and restore data. With suitable distribution data and services, you may have taken another step toward 100% Failover-free. Create a self-healing (SEF-heaing) and Self-Organizing (SEF-organizing) system architecture.
• Create an architecture without sharing. For development and deployment, the infrastructure also shares resources, just like resources shared at the logic and data layers, and you may encounter problems. It may cause a lock mechanism and a blocking mechanism, and cause a deadlock. A service-oriented architecture allows parallel and separate development processes. This allows the development of functional features to be "scalable" and matches the growth of the system.
• Open the System and Its APIs at the same time, so that you can create an ecosystem around your application.
• The only way to manage a huge distributed system is to make everything as simple as possible. The easy way to keep things is to ensure that there are no hidden requirements and hidden dependencies during design. Use as few technologies as possible to solve your problem. Artificial creation of complex system hierarchies that are not needed does not benefit the company.
• Organizations around services can provide agility. You can do things in parallel, because the output result is a service. This shortens the time for products and services to be put on the market. You need to create an infrastructure to ensure that the service can be quickly built.
• There must be a bunch of hype messages before any real implementation.
• Use Service Eve Agreement (SA) internally to manage services.
• Anyone can quickly add web services for their products. Implement some of your products in the form of services and start to use these services.
• Because of performance, reliability, and cost control, you may need to build your own infrastructure architecture. Building these infrastructures on your own does not have to be caused by the mistakes of a company that has closed Amazon. Self-built systems may not be easy to use. However, compared with third-party systems, you can quickly patch, debug, and deploy your own infrastructure.
• Use the speculative methods like "method" and "purpose" to distinguish between good and bad. I have participated in several previous speeches by Amazon employees and found that this is a unique and interesting feature that Amazon and other companies have made. The underlying principle is that by assigning the right to choice to real customers, we can see which approach is most appropriate and discover the real needs of customers based on these tests. Avinash Kaushik calls this option to avoid the impact of Hippo (the highest paid sans e ithe room, the highest paying person in the house. The goal can be achieved through a/B testing and web anaytics and other technical means. If you have any questions about what to do, first develop some features so that people can use these features. Then, you can see which alternative method can bring you the desired results.
• Create a frugal environment. For example, Amazon uses the door as a table.
• Understand what you need. Amazon had a bad early experience with a recommendation system that did not reach the expected goal: "This is not the Book recommendation system we need. Amazon needs a Book recommendation system that can work well based on a small amount of scattered data, such as customers' scores and purchase records. And the response speed is fast enough. This system also needs to adapt to a large amount of data and a large number of book categories. It also helps readers find some book demands that they really need but cannot discover ."
• User-friendly projects-people follow this project because they are interested in it-can inspire more value and creativity. Do not underestimate the power of interest.
• Let everyone participate in the process of creating a product. When Christmas big procurement comes, pack books in the warehouse. This is the team spirit.
• Create a site that can be used for testing. After the test is passed, the site can be launched to the public.
• For read-only data used by Web servers, a robust, clustered, redundant, and distributed file system is perfect.
• If the update fails, you need to roll back to the previous normal state. Develop a tool if necessary.
• Turn to a more in-depth service-based architecture
• During the interview, you need to pay attention to the following three key points: enthusiasm, creativity, and proficiency. At Amazon, the biggest feature of success is enthusiasm for work.
• To hire such employees, they have amazing debugging technology and system knowledge, and most importantly, they can cope with very difficult problems under high pressure.
• Innovation can only come from the bottom layer. The most knowledgeable person is the most likely to solve the problem. Any organization dependent on innovation must be able to accommodate a certain degree of chaos. Loyalty and obedience are not your tools.
• The spirit of innovation must be everywhere.
• Anyone should have the opportunity to try, learn, and practice. Changes in positions, obedience, and traditional habits should not have much right. An effective assessment method is required for the vigorous development of innovation.
• Embrace innovation. In front of employees of the Company, Jeff Bezos will personally issue the 'just do it' award, an old pair of Nike shoes, to those innovative employees.
• Do not pay compensation based on performance. Good benefits and high salaries should be given, but most people should be able to enjoy them. In other ways, we can express our appreciation for some outstanding employees. 'Distribution by labor 'sounds good, but it is impossible to be fair in a large company. Some non-monetary rewards, such as an old pair of Nike shoes, are actually good. It is also a way to express gratitude, indicating that someone cares about them.
• Rapid expansion. Large rivals like Barnes & nobe keep behind you. Amazon was once not the largest bookstore on the Internet, not the second or even the third place. However, their vision planning and implementation methods finally make them laugh at the end.
• In a data center, only 30% of the time spent by employees solving infrastructure problems is related to profit creation. Other 70% of the time is spent on "heavy" hardware procurement, software management, load balancing, maintenance, and other things to cope with the increased capacity challenge.
• Direct access to data is prohibited. This means that you can make your service scalable and have better reliability without affecting your customers. This is similar to googe's ability to improve all applications by improving the independence and distribution of server stacks.
• Create a unified service access mechanism. This makes the service easy to integrate and completes Request Routing decentralization, distributed request tracing, and other advanced infrastructure technologies.
• Allows any developer in the world to access http://Amazon.com for free through web service interfaces. This is also an important factor in success. Because of its many innovations, Amazon's own team alone cannot be imagined or implemented.
• Developers know which tools are the best to use and what tools are most suitable for their current work.
• Do not impose too many restrictions on engineers. Provide incentives for the completion of certain functions, such as integration with the monitoring system or integration with other infrastructure tools. However, for other functions, ensure the independence of the team.
• Similar to artists, developers can make their jobs the best if they have enough freedom, but they also need good tools. Provide as many support tools as possible, and provide environment support for service development, so that the environment will not become a hindrance to development.
• Who builds and runs. This links developers with the daily operations of the software they develop. It also brings them daily connections with customers. This type of customer feedback loop is crucial for improving service quality.
• Every two years, developers should stay with the customer service department for a period of time. There, they can hear real customer service calls, answer customer service emails, and have a deep understanding of the impact of what they develop as technicians.
• Let everyone listen to "voice from customers". The content is a real story about the user experience produced by customers who use their websites. This allows the management and engineers to realize that we are developing these technologies for real people. Through the customer service statistics, we can find out what is wrong or what is the real pain point of the customer.
• Like googe, Amazon's infrastructure is their core competitiveness. Through some relatively simple underlying services, they can build very complex applications. They can independently increase the capacity of each service, maintain the availability of non-parallel systems, and quickly release new services without having to modify a large number of system configurations.

 

From: http://www.zhihu.com/question/20105139