Large Web site system and Java Middleware Practice reading notes

Reprint: http://blog.csdn.net/ioscodelover/article/details/45047869

1. Distributed systems are relatively centralized, which means that multiple computers collaborate with each other through message communication to provide services externally, and can solve the scalability and single point problems of mainframe;

2. Network I/O is bio/nio, and Aio,aio refers to the thread after the message is not processed or wait for processing after the end of the response, but after the message delivery continues after the processing, only the callback to the callee, the message processing is completed automatically by the callee completion callback, that is, asynchronous io, JAVA7 Support aio;

3. The distributed system has several difficulties: the lack of a global clock (can be taken to the individual cluster to do the time serial number), the independence of the fault (to consider other modules reliable and unreliable), single point of failure processing (can split the split, can change the cluster of the cluster, not split to the most granular backup for automatic Recovery) , Business challenges (two-phase submission agreement, eventual consistency, etc.);

4. Single-machine Application Evolution process:

1) after the single-machine load alarm, the database and the application server to share;

2) Application Server load alarm after the application server to do the cluster, involving load balancing devices, involving session issues can be stored in the session cookie or session copy or unified session service management center;

3) database pressure is larger, read and write separation, involving data replication problems and application of data source selection problem, read the library can also be cached, the data real-time requirements are not high can also use search engines as read;

4) In addition to the cache can be used to cache data, can also be used to cache the page, caching all the real-time requirements of the content is not high;

5) introduce the distributed storage (file/kv) system to alleviate the database access;

6) the database still has a bottleneck after reading and writing, and the database is split vertically by using special library.

7) the database vertical split and then encountered the database Single-machine bottleneck, consider the horizontal split, at this time the introduction of data access middleware;

8) After the database problem is solved, The new challenge adopts the application splitting service mode, and the service framework middleware of the message middleware is introduced at this time;

There are three main types of 5.Java middleware: service Framework middleware, message middleware and data access Middleware.

The difference between 6.votatile and synchronized: volatile means that each read and write is directly read and written to main memory, will not be copied into the thread store, that is, all the threads operate the same part, which does not mean that only one thread can operate simultaneously; sychronized is a lock. Maintain consistency of data;

The classes in the 7.Java Atomics package are classes that support atomic operations, and their performance is significantly improved because Java uses hardware features to support it;

8.CountDownLatch is waiting for a thread to call latch.await, waiting for latch to be reduced to 0 to go down, while the release thread uses Latch.countdown to release 1 per call;

9.CyclicBarrier means that all threads calling barrier.await wait until the other thread executes to this statement, and barrier can also preset a thread to run after the condition is met;

10.Exchanger refers to the two threads are executed to this statement when the exchange of a signal, and then each continue to execute;

11.Future refers to the invocation of the thread that gets the result is asynchronous, the code can be executed first, wait until it is really going to use the return value if it still does not return to stop, Futuretask is its implementation class.

12. Concurrent Containers If there is a suitable scenario as far as possible using Java to provide, do not use their own lock-based implementation;

13. Dynamic proxy, refers to the class and interface, Interface handler such as the name passed in to the Proxy.newproxyinstance method to dynamically create the proxy, can be triggered before and after the method trigger or even trigger the corresponding processing, which in the service call framework, such as the client or the server, some can use beans can let the user configure the interface, thereby generating the agent, by the agent to place Calls from outside, and is replaced before and after the Call.

14. The service framework is broadly implemented as Follows:

1) The Caller: Configure the remote service through the bean configuration, the service provider address is usually given through a configuration center, the dynamic agent in the interface when the call through the service framework method to obtain a remote corresponding service provider address, pick-up request parameters, etc. to serialize the socket directly (using bio/ nio, etc.), The response results are returned to the corresponding object back to the upper level of the caller, the processing of timeouts and other problems can be achieved through the future, but also to achieve a combination of multiple calls between the optimization, that is, the initiation of multiple similar asynchronous calls, the use of their results will not be stuck;

2) Service Party: Configure the remote service provider through Bean configuration, Register to the corresponding configuration center after startup, start several threads (pool, to handle the flow Control) to listen to a request of a port, when the connection arrives call the concrete implementation class to do some back to the serialization tool, After the call, some serialization operations are returned to the remote caller, and the jar package conflict between the application itself and the framework is resolved by classloader;

3) service registry: mainly used to manage the registration to the caller and the service provider, while it can both route management, information viewing and other functions, routing can be refined to classes or methods, but also by grouping to the same room provider identified; service upgrade is generally adopted version number of the way, that is, the service upgrade, new and old coexistence, Old caller full upgrade after the old service;

15. Service Framework in combat optimization, that is, service management needs of content: service information management, Service Quality Management assessment, service capacity assessment, service-dependent display, service distribution display, Service statistics, Service reports, service metadata, etc. service management also has the service limit flow, Upper and lower line, downgrade, route, Service Authorization management and so on;

16. Service Framework and ESB similarities and differences are service-oriented, the service framework mainly consider the homogeneous system does not consider heterogeneous, the ESB will consider the implementation of different vendors;

17. Difficulties in database horizontal/vertical splitting:

1) the transaction mechanism of single machine is broken, and the control of distributed transaction should be considered;

2) Some single library operations need to operate in multiple libraries, the table connection needs to be implemented by application;

3) FOREIGN KEY constraints require a program to guarantee rather than a database;

4) dependent on the self-increment sequence of the library to be changed;

18. Distributed transaction has two-phase commit protocol, but it is too complex and has performance problem for large web site, the core principle of which is a few service majority, it is not to introduce the distributed transaction, if must introduce or not pursue strong consistency and only require final consistency. That is, the unfinished work is done without rollback by retrying;

19. Large web site consistency theory: CAP, that is, consistency, response/availability, part of the problem can still work (partition-tolerance), these properties are mutually exclusive, so more is to abandon the full consistency, only the pursuit of eventual consistency;

20. Cross-library Query and to sort is the most difficult problem to deal with, equal to the data of all the sub-library to be detected and then the operation, should try to avoid this situation, especially paging to the back of the data volume more in, if the amount of data is huge, you can consider using a search engine;

21. The design of data access middleware is generally under the Jdbc/orm framework to deal with routing, SQL analysis, etc., generally there will be a bunch of data sources to deal with, including data partitioning rules (modulo is not easy to expand, consider a consistent hash, when someone exits the takeover, A physical node can be virtualized with multiple virtual nodes to alleviate the problem of overly concentrated tasks, and the data layer needs to be transformed as Follows: SQL parsing, rule handling, SQL rewriting (the table names in each library may be different), data source selection, SQL execution, and result sets returning the merge processing steps.

22. Data access middleware can be introduced in the application as a jar package, or you can consider deploying an application alone, where business applications only deal with the application, and the rules are parsed and executed by the Application.

23. Read and write sharing data synchronization, mainly by Otter to complete, based on the database log parsing, the international station can be achieved Second-level delay, the domestic can do ms, Ali internal now in the cell, more and more live programs, also need these synchronization technology.

24. Smooth data migration is actually the first full-volume migration, and then the change of the old library in the process is executed in the new library, the recursive process will be shorter and smaller, when the need to migrate a little increment, pause the write operation of this part of the data, and then quickly complete processing, switch routes to the new Library.

25. Message Middleware can decouple the dependencies between applications, and generally use the industry to solve the problem of real-time requirements such as asynchronous calls, such as texting or synchronizing data, logging These.

26. The consistency of message delivery and business processing needs to be guaranteed for a long time, with the exception of some that would allow the user to repeat at will, so a transaction would have to be introduced in a similar way, but the cost of the distributed transaction was too high, so a relatively eclectic scenario would be:

1) send message to message middleware;

2) Message Middleware Inbound messages;

3) The message middleware returns the result;

4) Business operations;

5) send the result of the business operation to the message middleware;

6) Change the status of messages in the store;

Only 5th 6th of this scenario can cause inconsistencies, but in this case the message can be reversed by asking the source to send the message through the unhandled message state of the message Middleware.

27. message middleware generally has topic and queue two, there are different application scenarios, and sometimes need to be cascaded way to deal with.

28. The reliability of the message sending and sending process is ensured by local storage + retry, and the message that failed is retained and re-sent after the message center is resumed, while for the storage side the database can be used to store the message, and the Two-machine memory can be used to speed up the Speed. Make the storage run only in memory and back up between the two machines, and once one hangs off, the other drops the disk and takes over the Message.

29. The message processing side generally needs to ensure the power of the message processing operation to prevent the message delivery error has duplicate message generation;

Large Web site system and Java Middleware Practice reading notes