Java high Concurrency High performance distributed framework design of the Micro Service architecture from scratch

MicroServices architecture Mode (Microservice Architect pattern). Nearly two years of crazy growth in services and advances in cloud computing technology have focused on microservices architecture

A microservices architecture is an architectural pattern that advocates dividing a single application into a small set of services that are coordinated and co-ordinated to provide the ultimate value to the user. Each service runs in its own separate process, with a lightweight communication mechanism between the service and the service (typically HTTP-based restful APIs). Each service is built around a specific business and can be independently deployed to a production environment, a class production environment, and so on. In addition, we should try to avoid a unified, centralized service management mechanism, for a specific service, should be based on the business context, choose the appropriate language, tools to build it.

Micro-Service Architecture Benefits

Firstly, the connotation and advantages of micro-service (microservices) are introduced briefly, and the essence of micro-service architecture is to solve bigger and more practical problems with some clearer functions and more concise services. The MicroServices architecture splits the service, uses a relatively separate service to manage all aspects, communicates with one another using a unified interface, and the architecture becomes complex with obvious advantages:

Complexity can be controlled: in the application of decomposition at the same time, avoiding the original complexity of the endless accumulation. Each micro-service focuses on a single function and clearly describes the service boundary through a well-defined interface. With its small size and low complexity, each microservices can be fully controlled by a small-scale development team, maintaining high maintainability and development efficiencies.

What is micro-service architecture

Micro-Service Architecture Benefits

Standalone deployment: Because MicroServices have a separate running process, each microservices can also be deployed independently. There is no need to compile and deploy the entire application when a microservices change. Applications consisting of microservices are equivalent to a series of parallel publishing processes that make publishing more efficient, while reducing the risk to the production environment and ultimately reducing the application delivery cycle.

Technology selection Flexibility: Under the micro-service architecture, technology selection is to be centralized. Each team can freely choose the most suitable technology stack according to the needs of its own service and the status quo of industry development. Because each microservices is relatively simple, the risk of upgrading the technology stack is low, and even a complete refactoring of a microservices can be feasible.

Fault tolerance: When a build fails, in the traditional architecture of a single process, the failure is likely to spread within the process, resulting in a global application is not available. In a microservices architecture, failures are isolated in a single service. If the design is good, other services can achieve application-level fault tolerance through the mechanism of retry and smooth degradation.

Extensions: Monolithic architecture applications can also be scaled out to replicate entire applications to different nodes. When the different components of the application differ in the scale requirements, the MicroServices architecture demonstrates its flexibility because each service can be scaled independently according to the actual requirements.

Internet high concurrency related nouns

Pageviews (Page views)

Unique browse volume (unique pageviews)

Number of independent visitors (unique visitors)

Number of repeat visitors (repeat visitors)

Number of page views per visitor (Page view per user)

High concurrency

Before I mistook the high concurrency solution is a thread or queue can be resolved, because high concurrency is a lot of users in the access, resulting in incorrect system data, loss of data phenomenon, so think of the queue to solve, in fact, the way the queue solution can also be handled, such as we are in the auction of goods, Forward comments microblogging or seconds to kill goods, etc., the same time the traffic is particularly large, the queue in this special role, all requests placed in the queue, in milliseconds units, orderly, so that no data loss system data is incorrect.

After checking the data, there are two kinds of solutions for high concurrency, one is to use the cache, the other is to use to generate static pages, and to optimize our writing code from the most basic places to reduce unnecessary waste of resources: (

1. Do not frequent new objects, use singleton mode for classes that only need to have one instance in the entire application. For string connection operations, use StringBuffer or StringBuilder. Classes for utility types are accessed through static methods.

Avoid using the wrong way, such as exception can control the introduction of methods, but exception to retain stacktrace consumption performance, unless it is necessary not to use the instanceof to make conditional judgments, try to use the ratio of criteria to determine the way. Using classes that are efficient in Java, For example, ArrayList is better than vector performance. )

High concurrency-issues that need to be addressed

One: App Cache

Two: HTTP caching

Three: Multilevel caching

Four: Pooling

V: Asynchronous concurrency

Six: Capacity expansion

Seven: Queues

High concurrency-app cache

Heap Cache

Use Java heap memory to store cached objects. The advantage of using heap caching is that there is no serialization/deserialization, which is the fastest cache. The disadvantage is also obvious that when the amount of data cached is large, the GC (garbage collection) Pause time is longer and the storage capacity is limited by the heap size. Caching objects are typically stored through soft reference/weak references, which means that when heap memory is low, this portion of memory can be forcibly reclaimed to free up heap memory space. The heap cache is typically used to store hot data. Yes

Guava cache: Caches and Concurrentmap are very similar, but they are not exactly the same. The fundamental difference is that CONCURRENTMAP will hold all the added objects until it is shown to be removed. caching, in order to limit the use of its memory, is usually configured to automatically remove objects. In some cases it is also useful to not automatically remove objects, such as Loadingcache, which automatically loads cached objects.

Ehcache 3.x: is a widely used open source Java distributed cache. Primarily for general purpose caches, Java EE and lightweight containers. It features memory and disk storage, cache loaders, cache extensions, cache exception handlers, a gzip cache servlet filter, and support for rest and soap APIs.

Mapdb:mapdb is an embedded, pure Java database that provides concurrent HashMap, TreeMap, and Queue that can store data based on out-of-heap or disk

High concurrency-app cache

Out-of-stack caching

That is, the cached data is stored in out-of-heap memory, which reduces the GC pause time (the heap objects are moved out of the heap, the GC scans and the moving objects are less), but the data needs to be serialized/deserialized and therefore much slower than the heap cache. With Ehcache 3.x, mapdb implementation

Disk Cache

That is, the cached data is stored on the track, and the data exists while the JVM restarts, and the heap cache/out-of-heap cache data is lost and needs to be reloaded. With Ehcache 3.x, mapdb implementation

Distributed cache

In-process caches and disk caches, there are two issues in the case of multiple JVM instances:

1, single-machine capacity problem;

2. Data consistency issues (what about inconsistent cache data for multiple JVM instances?) , this problem does not have to tangle, since the data allows caching, it means to allow a certain period of inconsistency, so you can set the cache data expiration time to update the data regularly;

3, Cache misses, need to back to the source to db/service Request changeable problem: each instance in the case of cache misses will be back to the source to the DB load data, so the multi-instance DB overall access to a lot of the solution is to use such as a consistent hash shard algorithm. Therefore, these scenarios can be considered to be resolved using distributed caching.

You can use Ehcache–clustered (with Terracotta server) to implement distributed caching between Java processes. The best approach is to implement distributed caching using Redis.

High concurrency-HTTP caching

Browser caching is when we use the browser to access some Web pages or HTTP services, based on the cache settings returned by the server response header to cache the response content to the browser, the next time you can directly use the cached content or only need to go to the server to verify whether the content expires. This benefit can reduce the amount of data that travels back and forth between the browser and the server, saving bandwidth and improving performance.

Workaround: Content does not need to be dynamic (compute, render, etc.) faster, the closer the content is to the faster the user. Like Apache traffic server, squid, varnish, nginx and other technologies can be used to cache content. And the CDN is used to speed up user access:

That is, the user first access to the CDN node across the country (using such as ATS, squid implementation), if the CDN killed, will be back to the central Nginx cluster, if the cluster does not have a hit cache (the cluster's cache is not required, depending on the actual hit situation, etc.), and finally back to the backend application cluster.

High concurrency-multilevel cache (distributed cache)

High concurrency-pooling

In the application development process, we often use pooling technology, such as Object pool, connection pool, thread pool, etc., through pooling to reduce some of the consumption, to improve performance.

Object pooling reduces the overhead of creating objects and garbage collection by reusing objects. However, the pooling cannot be too large and the scanning time of the GC is affected by the General Assembly.

Connection pooling, such as database connection pooling, Redis connection pooling, HTTP connection pooling, improves performance by reusing TCP connections to reduce the time to create and release connections.

The thread pool is similar and improves performance by reusing threads. That is, the purpose of pooling is to improve performance through multiplexing techniques.

High Concurrency-expansion

1, read and write separation: When the database traffic is not very large, we can properly increase the server, the database master-slave replication method will read and write separation

2, Vertical partitioning: When the write operation once increased, then the master-slave database will spend more time on the data synchronization, this time the server is also overwhelmed; then there is the vertical partitioning of data, the vertical partitioning of data is to write more frequent data tables, such as user table _user, or Order table _orders, then we can separate this two table, put on a different server, if the two tables and other tables exist in the table query, then only the original SQL statement to split, first query a table, in the query another, although this will consume more performance, but compared to the kind of large data synchronization, The burden or the reduction of a lot;

3, the horizontal partition: But often things are not satisfactory, may take a vertical partition can support a period of time, because the site is too hot, the amount of traffic is 100w a day, jumped to 1000w, this time can take data separation, we can be based on the user ID different distribution, such as take% 2, form, Of course, this form of future expansion has a great limit, when I was increased from 10 to 20, all the data have to be re-partitioned, then it will be a very large amount of computation; several common algorithms: hashing algorithm: The way of using user_id%; Range: can be divided according to the range of user_id character values, such as 1-1000 for one area, 1001-2000 for another, etc. mapping relationship: is the corresponding partition of the user_id exist in the database to save, when the user operation, first to query the partition, and then to operate.

You can click to join the group: 650385180 "Java Architecture" there is the Java senior Daniel Live instruction Knowledge Point is the high-end route (if you want to change jobs but the technology is not enough or the work of the bottleneck I have a free live course in Java Talk about the high-end knowledge of the base of the bad don't into yo as long as you have 1-5 years of development experience can add group to find me to the classroom link Note: Is free of development experience do not enter OH)

Java high Concurrency High performance distributed framework design of the Micro Service architecture from scratch