ebay: Using MongoDB to create business-critical multi-Datacenter applications

EBay: Using MongoDB to create business-critical multi-Datacenter applications

As the world's top ten retail brand, ebay has more than 170 million active users, and has 1 billion shopping lists spanning 190 markets around the world, so that on a scale, ebay is not allowed to have any downtime. That's why companies rely on MongoDB to provide enterprise-class platform standards and user-facing applications.

At this year's MongoDB World Conference Conference, ebay's chief NoSQL Dba,feng Qu the practical design patterns that he and his team have developed to support an enterprise-wide MongoDB deployment with a suite of architecture-resilient applications.

Mr. Qu first explored the changes in the concept of "usability" in recent years. In the past, it was normal for the Web site to periodically stop maintenance, but in recent years, with today's globalization of services, users and businesses are no longer accepting such a stop-service operation. In addition, most organizations deploy their services on commercial hardware platforms rather than previous Sun Solaris/sparc. Commercial hardware is relatively inexpensive, but the frequency of problems is relatively high. These factors have a substantial impact on the software team's perception of usability. This is also the original intention of ebay to create the "Flex design Pattern", which is to create the best database system to maximize average failure time (MTTF) while minimizing average recovery time (MTTR).

Developers can choose from five enterprise-accepted database standards when creating applications. In addition to MongoDB, you can choose a relational database such as Oracle and MySQL and two other NoSQL database systems. Mr. Qu's team will give some advice on their choice to ensure that the database system you choose can support data read mode, user pressure, and so on.

Ebay currently operates over 3,000 non-relational database instances, supporting a wide range of applications and petabytes of data transfer between applications. In the past, Oracle was a recording system, the non-relational database only maintained some temporary data, now the non-relational database scenario has matured, not only have consistent, specific to the backup and timely recovery, MongoDB in some of ebay's scenes can also be used as a recording system.

While ebay's non-relational databases provide built-in resiliency, they can also choose different design tradeoffs to influence the performance of the application. The DBA chooses primarily to consider the following aspects: availability, consistency, durability, recoverability, scalability, and performance. For example, a NoSQL database that uses a point-to-point, non-master design, must start the process of data repair and rebalancing after a node failure, and these processes are costly. A rebalancing process can seriously affect the throughput and latency of the system, and the client waits for recovery to cause a blockage in the connection, causing the application to fail. To avoid this, ebay has added an application-level shard to the top of the non-primary database, which was originally designed for Oracle. In this way, the DBA can decompose a large cluster into a series of subsets, placing the rebuilt load on an individual node, affecting only the individual query operations. It is in order to respond to different types of database behavior that ebay has built a resilient design pattern.

Mr. Qu showed us the MongoDB elastic design pattern.

In this design mode, the 7 nodes of the MongoDB replica set are distributed on ebay in three different datacenters in the United States. This mode allows the database cluster to remain available when the primary node encounters a failure. You can assign a priority to MongoDB replica set members, which determines which nodes can be referred to as the new master node after the primary node encounters a failure. For example, you can set the node in DC1 to be elected as the primary node after the main node failure. Only the nodes in the DC1 have failed, the members of DC2 can be elected as the main node, of course, the new node election is based on the node from the original node is synchronized to the latest data is the primary node. An extension of this pattern is to use MongoDB's "most write attention" to guarantee write persistence across data centers.

MongoDB's standard design pattern is the basis of ebay's "high-density, highly available read mode", which is used to support ebay's product catalog modules. In order to cope with the pressure of the Product catalog module, it is necessary to expand the membership of the MongoDB replica set to 50, providing a distributed data architecture for read extensibility and resiliency.

ebay has developed "high-performance read-write mode" to support high-density write pressure, which distributes the MongoDB cluster's shard servers across different data centers in the United States.

Similarly, developers can set up specific read-write concerns for this pattern, adjust system durability and resiliency to meet the needs of different applications.

Mr. Qu points out that with the enhanced functionality of MongoDB, MongoDB can be used to meet more application needs:

The Zone Sharding feature introduced in the MongoDB3.4 allows ebay to support applications that require high levels of distributed and persistent write availability.

• Overridable actions to be released in MongoDB3.6 to reduce application-side exception handling code

If you want to learn more about ebay's design patterns, you can refer to Feng Qu's share at the MongoDB conference.

Ebay: Creating a business-critical multi-Datacenter application using MongoDB