The evolution of large-scale Internet architecture

respect for the original works, Ming: This post for the excellent post integration The 10 years of Taobao technology 1. Taobao technology these 10 years 1.1. Taobao Status Quo

High concurrency has become a huge challenge for internet companies. For example, the 2015 "Double 11" global Carnival officially came to an end, the real cat turnover also reached a record 91.217 billion yuan. Participating countries and regions to reach 232, double 11 Alipay peak processing number of transactions per second is 85,900, online number of peak reached 45 million.

Taobao's core technology (domestic and international top, which is still 2011 data)

Owns the country's largest distributed Hadoop cluster (ladder, 2000 nodes, 24000 cores CPU,48000GB memory, 40PB storage capacity)

Nationwide distribution of 80+CDN nodes, can automatically find the nearest node to provide services, support traffic over 800Gbps

Baidu's search engine is not inferior to the search for billions of of goods, the world's largest electric business platform

Top-load balancing systems, top-tier distributed systems, top-tier Internet ideas, functional diversity, and extremely stable operation

Rich ecological industry and advanced data mining technology

...... Lots and lots of 1.2. Taobao technology Evolution, excerpted from the "Taobao Technology" ten years

Horse in April 7, 2003 secretly called Alibaba's 10 employees, came to Hangzhou a secret roughcast room, asked them in one months or so time to make a c2c website. The result, of course, is to buy directly, a website based on lamp architecture, formerly known as Phpauction, an auction site developed by the old us. Of course, you have to make changes to use.

2003 end, Taobao registered users 230,000, PV 310,000/day, half a year turnover 33.71 million

It is obvious that MySQL cannot afford such a large amount of traffic, the database bottleneck appears. Thankfully, Ali's DBA team is strong enough to use Oracle instead of MySQL. Oracle has already had a powerful concurrency access design--connection pooling, which is much less expensive to connect from a connection pool than a separate connection. but PHP did not provide the official support language connection pool characteristics, so Dolong predecessors with Google (not Baidu) to search an open source of SQL Relay, so the database software bottlenecks temporarily resolved.

With the attendant hardware performance bottlenecks, Ali buys EMC's SAN storage devices, plus Oracle's high-performance RAC, and hardware capacity is temporarily out of the question.

Because the problem with SQL relay is really too serious, 2004 so Taobao finally made the decision of the Trans-era- use Java rewrite the Web site.

Taobao asked the Sun's senior engineer to help do the Java architecture. So how do they modify the programming language without changing the use of the site--modular replacement, today wrote a module, opened a new domain name, will connect to the module, while other modules unchanged, wait until the completion of all modules, the original domain name to give up. Sun insists on using EJB as the control layer, plus using ibatis as a persistence layer , a scalable and efficient Java EE application is born.

After sending away the Sun's Daniel, Ali's data storage again encountered the bottleneck, so reluctantly bought an IBM minicomputer, also has IoE (IBM + Oracle + EMC) such legend

At the end of 2004, Taobao registered users 4 million, PV 40 million/day, the total network turnover of 1 billion.

Spring was born in 2005, and the spring framework was an integral part of Web applications, and in Taobao, Spring reached the point that Rod Johnson designed it to replace EJB.

2005 end, Taobao registered users 13.9 million, PV 89.31 million/day, the number of 16.63 million products.

In view of the future development, such a facility structure will only be able to meet the present requirements. As a result,CDN technology came in handy, the first use of commercial ChinaCache, and later use of Dr Zhangwensong to build a low-power CDN network , Taobao performance is getting better.

At the end of 2006, Taobao registered users 30 million, PV 150 million/day, the number of goods 50 million, the total net turnover of 16.9 billion yuan.

Taobao used NetApp's commercial storage system before 2007, but it was still not enough to cope with the rapid growth trend. In the same year, Google unveiled GFs's design ideas , and with that in mind, Taobao has also developed its own filesystem--tfs each user has 1GB of picture storage space on TFS , These benefit from the TFS cluster's file storage system and a large number of image servers. Taobao uses real-time generation of shrinkage graphs, global load balancing, and level one and level two caching to ensure that the image is optimized for access and efficient access.

Taobao's server software uses Tengine, an optimized nginx module.

Taobao has isolated UIC (User information Center) for all modules to call. Dolong predecessors wrote the tdbm for it again, completely based on the memory of the data cache (refer to the memcached). Then, Taobao will merge Tbstore and TDBM, write out the Tair, a key-value based distributed caching Data system. then upgrade your ISearch system.

At the end of 2007, Taobao registered users 50 million, PV 250 million/day, the number of goods 100 million, the total net turnover of 43.3 billion yuan.

...

Dubbo is the core framework of the SOA service governance solution within Alibaba, providing 3,000,000,000+ access to 2000+ services every day, and is widely used in the Alibaba Group's member sites. Dubbo has been used by many non-Ali companies since 2011 when it was open source. (Wsdl,uddi and SOAP (Http+xml) are the basic components of the SOA Foundation. WSDL is used to describe services; UDDI is used to register and find services, and soap, as a transport layer, is used to send messages between consumers and service providers. SOAP is the default mechanism for Web services, and other technologies enable other types of bindings to be serviced. A consumer can find a service in the UDDI Registry (registry), obtain a WSDL description of the service , and invoke the service via SOAP.

2. Technology Development Process Summary 2.1. single node architecture

2.2. Cluster Architecture

2.3. cluster + distributed architecture