High Throughput System Design Optimization suggestions

High Throughput System Design Optimization suggestions
High Throughput System

For example, we need to arrange a plan for a project. Each module can run multiple tasks in parallel by multiple people, or work in serial by one or more people, but there is always a key path, this path is the project duration. The response time of a system call is the same as that of a project plan. The critical path is the system impact time. The key path consists of CPU operations, IO, external system response, and so on.

For a system user, starting from clicking a button, link, or issuing a command, the system displays the result as desired by the user to terminate, the time spent in the whole process is the user's intuitive impression on the software performance, that is, the response time we call. When the response time is short, the user experience is good. Of course, the response time for the user experience includes personal subjective factors and objective response time. When designing software, we need to consider how to better combine these two parts to achieve the best user experience. For example, when a user queries a large amount of data, we can present the extracted data to the user and continue to perform data retrieval during the user's viewing process, at this time, the user does not know what we are doing in the background, and the user is concerned with the response time of user operations.

The throughput of a system is usually determined by QPS (TPS) and concurrency. Each system has a relative limit value. Under the Access pressure of application scenarios, as long as one item reaches the maximum value of the system, the system's throughput will not go up. If the pressure continues to increase, the system's throughput will decrease, because the system is overloaded, context switching, memory, and other consumption lead to a reduction in system performance, which determines the system response time.

Buffer)

A buffer zone is a specific memory area. The purpose of a buffer zone is to improve system performance by mitigating performance differences between upper and lower layers of applications. In daily life, a typical application of buffering is a funnel. Buffering can coordinate the performance difference between the upper-layer components and the lower-layer components. When the upper-layer components have better performance than the lower-layer components, it can effectively reduce the wait time for the upper-layer components to the lower-layer components. Based on this structure, the upper-layer application component can return the operation without waiting for the lower-layer component to accept all the data. This accelerates the processing of the Upper-layer components and improves the overall system performance.

Buffer using BufferedWriter

BufferedWriter is a buffer usage. Generally, the buffer should not be too small. A buffer that is too small cannot play a real buffering effect, and the buffer should not be too large. A large buffer will waste the system memory and increase the GC burden. Try to add a buffer in the I/O component to improve performance. The sample code of a buffer is shown in Listing 1.

Listing 1. sample code before adding a buffer

import java.awt.Color;import java.awt.Graphics;import java.awt.Graphics2D;import java.awt.Image;import javax.swing.JApplet;public class NoBufferMovingCircle extends JApplet implements Runnable{ Image screenImage = null; Thread thread; int x = 5; int move = 1; public void init(){ screenImage = createImage(230,160); } public void start(){ if(thread == null){ thread = new Thread(this); thread.start(); } }@Overridepublic void run() {// TODO Auto-generated method stubtry{System.out.println(x);while(true){x+=move;System.out.println(x);if((x>105)||(x<5)){move*=-1;}repaint();Thread.sleep(10);}}catch(Exception e){}}public void drawCircle(Graphics gc){Graphics2D g = (Graphics2D) gc;g.setColor(Color.GREEN);g.fillRect(0, 0, 200, 100);g.setColor(Color.red);g.fillOval(x, 5, 90, 90);}public void paint(Graphics g){g.setColor(Color.white);g.fillRect(0, 0, 200, 100);drawCircle(g);}}

The program can complete the Left and Right translation of the red ball, but the effect is poor, because each interface refresh involves re-drawing the image, which is time-consuming, so the screen jitter and white light effect are obvious. You can add a buffer for a better display effect. The code is shown in Listing 2.

Listing 2. Sample Code after adding a buffer

Import java. awt. color; import java. awt. graphics; public class BufferMovingCircle extends NoBufferMovingCircle {Graphics doubleBuffer = null; // buffer public void init () {super. init (); doubleBuffer = screenImage. getGraphics ();} public void paint (Graphics g) {// use the buffer to optimize the original painting method doubleBuffer. setColor (Color. white); // first draw doubleBuffer in the memory. fillRect (0, 0,200,100); drawCircle (doubleBuffer); g. drawImage (screenImage, 0, 0, this );}}

Use Buffer for I/O operations

In addition to NIO, there are two basic methods to perform I/O operations using Java:

• Use InputStream and OutputStream-based methods;
• Use Writer and Reader.

No matter which method is used for file I/O, if the buffer can be reasonably used, it can effectively improve the I/O performance.

The following shows the buffer components that can be used with InputStream, OutputStream, Writer, and Reader.

OutputStream-FileOutputStream-BufferedOutputStream

InputStream-FileInputStream-BufferedInputStream

Writer-FileWriter-BufferedWriter

Reader-FileReader-BufferedReader

Packaging file I/O with the buffer component can effectively improve file I/O performance.

Listing 3. Sample Code

Import java. io. bufferedInputStream; import java. io. bufferedOutputStream; import java. io. dataInputStream; import java. io. dataOutputStream; import java. io. fileInputStream; import java. io. fileNotFoundException; import java. io. fileOutputStream; import java. io. IOException; public class StreamVSBuffer {public static void streamMethod () throws IOException {try {long start = System. currentTimeMillis (); // replace it with your own file DataOutputStream dos = new DataOutputStream (new FileOutputStream ("C: // StreamVSBuffertest.txt"); for (int I = 0; I <10000; I ++) {dos. writeBytes (String. valueOf (I) + "/r/n"); // cyclically writes data for 10 thousand times} dos. close (); DataInputStream dis = new DataInputStream (new FileInputStream ("C: // StreamVSBuffertest.txt"); while (dis. readLine ()! = Null) {} dis. close (); System. out. println (System. currentTimeMillis ()-start);} catch (FileNotFoundException e) {// TODO Auto-generated catch blocke. printStackTrace () ;}} public static void bufferMethod () throws IOException {try {long start = System. currentTimeMillis (); // replace it with your own file DataOutputStream dos = new DataOutputStream (new BufferedOutputStream (new FileOutputStream ("C: // StreamVSBuffertest.txt "); For (int I = 0; I <10000; I ++) {dos. writeBytes (String. valueOf (I) + "/r/n"); // cyclically writes data for 10 thousand times} dos. close (); DataInputStream dis = new DataInputStream (new BufferedInputStream (new FileInputStream ("C: // StreamVSBuffertest.txt"); while (dis. readLine ()! = Null) {} dis. close (); System. out. println (System. currentTimeMillis ()-start);} catch (FileNotFoundException e) {// TODO Auto-generated catch block e. printStackTrace () ;}} public static void main (String [] args) {try {StreamVSBuffer. streamMethod (); StreamVSBuffer. bufferMethod ();} catch (IOException e) {// TODO Auto-generated catch blocke. printStackTrace ();}}}

The running result is shown in Listing 4.

Listing 4. Running output

88931

Obviously, the performance of the Code that uses the buffer is much faster than that without the buffer. The code shown in listing 5 performs a similar test on FileWriter and FileReader.

Listing 5. FileWriter and FileReader code

Import java. io. bufferedInputStream; import java. io. bufferedOutputStream; import java. io. bufferedReader; import java. io. bufferedWriter; import java. io. fileNotFoundException; import java. io. fileReader; import java. io. fileWriter; import java. io. IOException; public class WriterVSBuffer {public static void streamMethod () throws IOException {try {long start = System. currentTimeMillis (); FileWriter fw = new File Writer ("C: // StreamVSBuffertest.txt"); // replace it with your own file for (int I = 0; I <10000; I ++) {fw. write (String. valueOf (I) + "/r/n"); // 10 thousand data writes in a loop} fw. close (); FileReader fr = new FileReader ("C: // StreamVSBuffertest.txt"); while (fr. ready ()! = False) {} fr. close (); System. out. println (System. currentTimeMillis ()-start);} catch (FileNotFoundException e) {// TODO Auto-generated catch blocke. printStackTrace () ;}} public static void bufferMethod () throws IOException {try {long start = System. currentTimeMillis (); BufferedWriter fw = new BufferedWriter (new FileWriter ("C: // StreamVSBuffertest.txt"); // replace it with your own file for (int I = 0; I <10000; I ++) {fw. writ E (String. valueOf (I) + "/r/n"); // 10 thousand data writes in a loop} fw. close (); BufferedReader fr = new BufferedReader (new FileReader ("C: // StreamVSBuffertest.txt"); while (fr. ready ()! = False) {} fr. close (); System. out. println (System. currentTimeMillis ()-start);} catch (FileNotFoundException e) {// TODO Auto-generated catch block e. printStackTrace () ;}} public static void main (String [] args) {try {StreamVSBuffer. streamMethod (); StreamVSBuffer. bufferMethod ();} catch (IOException e) {// TODO Auto-generated catch blocke. printStackTrace ();}}}

The running output is shown in Listing 6.

Listing 6. Running output

129531

As shown in the preceding example, you can effectively improve the system's file read/write performance and reduce the response time by using the buffer properly for reading or writing files.

Cache

Cache is also a memory space opened up to improve system performance. Cache is mainly used to save data processing results and provide the next access. In many cases, data processing or data acquisition may be very time-consuming. When the amount of requests to this data is large, frequent data processing will consume CPU resources. The purpose of caching is to temporarily store these hard-won data processing results. When other threads or clients need to query the same data resources, the processing process for the data can be omitted, directly obtain the processing result from the cache and immediately return it to the request component to improve the system response time.

Currently, there are many Java-based Cache frameworks, such as Ehcache, OSCache, and JBossCache. The EHCache cache comes from Hibernate and is its default data cache solution. The OSCache cache is designed with OpenSymphony and can be used to cache any object or even some JSP pages or HTTP requests; JBossCache is a cache framework developed by JBoss for data sharing between JBoss clusters.

Taking EHCache as an example, the main features of EhCache include:

1. Fast;
2. Simple;
3. Multiple cache policies;
4. There are two levels of cached data: memory and disk, so there is no need to worry about capacity;
5. Cached data is written to the disk when the VM is restarted;
6. Distributed cache can be implemented through RMI and pluggable APIS;
7. Listener interface with cache and cache manager;
8. Supports multiple cache manager instances and multiple cache regions of one instance;
9. Provides Hibernate cache implementation.

Because EhCache is a cache system in the process, once the application is deployed in the Cluster Environment, each node maintains its own cache data. When a node updates the cache data, these updated data cannot be shared among other nodes, which not only reduces the node running efficiency, but also causes data not to be synchronized. For example, if A website is deployed on nodes A and B as A cluster, when the cache of node A is updated and the cache of Node B is not updated, the user may be browsing the page, it will be the updated data, but it will be the data that has not yet been updated. Even though we can also use the Session Sticky technology to lock users on a node, however, Session Sticky is obviously not suitable for some systems with strong interactivity or non-Web methods. Therefore, you need to use the EhCache cluster solution. Listing 7 shows the EHCache sample code.

Listing 7. EHCache sample code

Import net. sf. ehcache. cache; import net. sf. ehcache. cacheManager; import net. sf. ehcache. element;/*** Step 1: generate a CacheManager object * Step 2: generate a Cache object * Step 3: Add a key to the Cache object, element * @ author mahaibo */public class EHCacheDemo {public static void main (String [] args) {// specify ehcache. location of xml String fileName = "E: // 1008 // workspace // ehcachetest // ehcache. xml "; CacheManager manager = new CacheManager (fileName); // retrieve all cacheName String names [] = manager. getCacheNames (); for (int I = 0; I <names. length; I ++) {System. out. println (names [I]);} // generates a Cache object based on the cacheName. // The first method is Cache cache = manager. getCache (names [0]); // method 2. defaultCache must exist in ehcache, and "test" can be changed to any value. // Cache cache = new Cache ("test ", 1, true, false, 5, 2); // manager. addCache (cache); // Add Element to the Cache object. Element elements include key and value pairs to form cache. put (new Element ("key1", "values1"); Element element = cache. get ("key1"); System. out. println (element. getValue (); Object obj = element. getObjectValue (); System. out. println (String) obj); manager. shutdown ();}}

Object reuse

Object reuse pool is a common system optimization technology. Its core idea is that if a class is frequently requested for use, you do not have to generate an instance every time. You can save some instances of this class in a "pool, to be used, you can directly obtain it from the pool. This "pool" is called an object pool. In terms of implementation details, it may be an array, a linked list, or any collection class. Object pools are widely used, such as thread pools and database connection pools. The thread pool stores reusable thread objects. When a task is submitted to a thread, the system does not need to create a new thread, but obtains an available thread from the pool to execute this task. After the task ends, the thread does not need to be closed, but it is returned to the pool so that it can be used again next time. Because the creation and destruction of threads are time-consuming, the thread pool can improve the performance in a system with frequent thread scheduling. The database connection pool is also a special object pool used to maintain the set of database connections. When the system needs to access the database, it does not need to establish a new database connection, but can be obtained directly from the pool. After the database operation is complete, it does not close the database connection, instead, the connection is returned to the connection pool. Because the creation and destruction of database connections are heavyweight operations, it is also significant to improve the system performance to avoid frequent operations. Currently, the database connection pool components that are widely used include C3P0 and Proxool.

Taking C3P0 as an example, it is an open source JDBC connection pool, which implements data source and JNDI binding and supports the JDBC3 specifications and standard extensions of JDBC2. Currently, open-source projects using it include Hibernate and Spring. If the configuration is in the JNDI mode, as shown in listing 8.

Listing 8. Tomcat data source configuration

<Resource name="jdbc/dbsource"  type="com.mchange.v2.c3p0.ComboPooledDataSource"  maxPoolSize="50" minPoolSize="5" acquireIncrement="2" initialPoolSize="10" maxIdleTime="60" factory="org.apache.naming.factory.BeanFactory"  user="xxxx" password="xxxx"  driverClass="Oracle.jdbc.driver.OracleDriver"  jdbcUrl="jdbc:oracle:thin:@192.168.x.x:1521:orcl"  idleConnectionTestPeriod="10" />

Parameter description:

1. IdleConnectionTestPerio: After the database is restarted or the process is killed for some reason, C3P0 does not automatically reinitialize the database connection pool. When a new request needs to access the database, at this time, an error will be reported (because the connection is invalid), and the database connection pool will be refreshed at the same time. The connection that has expired will be discarded. When the second request arrives, it will return to normal. C3P0 currently does not provide the parameter for obtaining the number of retries after a connection has failed. Only the parameter for obtaining the number of retries after a new connection fails.
2. AcquireRetryAttempts: this parameter is used to set a frequency parameter for the system to automatically check whether the connection in the connection pool is normal. The unit of time is second.
3. AcquireIncremen: Number of connections simultaneously obtained by c3p0 when connections in the connection pool are exhausted. That is to say, if the number of connections used has reached maxPoolSize, c3p0 will establish a new connection immediately.
4. MaxIdleTim: In addition, C3P0 does not close the unused connection pool by default, but recycles the connection pool to the available connection pool. As a result, the number of connections increases, so you need to set maxIdleTime (the default value is 0, in seconds. maxIdleTime indicates the maximum time that a connection in the idle state can survive.

If spring is used and the project does not use JNDI and does not want to configure Hibernate, you can directly configure C3P0 to dataSource, as shown in listing 9.

Listing 9. Spring Configuration

<Bean id = "dataSource" destroy-method = "close"> <property name = "driverClass"> <value> oracle. jdbc. driver. oracleDriver </value> </property> <property name = "jdbcUrl"> <value> jdbc: oracle: thin: @ localhost: 1521: test </value> </property> <property name = "user"> <value> Kay </value> </property> <property name = "password"> <value> root </value> </property> <! -- The minimum number of connections retained in the connection pool. --> <Property name = "minPoolSize" value = "10"/> <! -- The maximum number of connections retained in the connection pool. Default: 15 --> <property name = "maxPoolSize" value = "100"/> <! -- Maximum idle time. connections are discarded if they are not used within 1800 seconds. If it is 0, it will never be discarded. Default: 0 --> <property name = "maxIdleTime" value = "1800"/> <! -- The number of connections that c3p0 obtains at the same time when connections in the connection pool are exhausted. Default: 3 --> <property name = "acquireIncrement" value = "3"/> <property name = "maxStatements" value = "1000"/> <property name = "initialPoolSize" value = "10"/> <! -- Check all idle connections in the connection pool every 60 seconds. Default: 0 --> <property name = "idleConnectionTestPeriod" value = "60"/> <! -- Defines the number of repeated attempts after a new connection fails to be obtained from the database. Default: 30 --> <property name = "acquireRetryAttempts" value = "30"/> <property name = "breakAfterAcquireFailure" value = "true"/> <property name = "testconnectioncheckout" value = ""false"/> </bean>

There are many similar practices, so you can search for them on the Internet.

Computing Mode Conversion

Computing mode conversion is famous for the time-to-space mode. It is usually used for embedded devices, or when the memory and hard disk space are insufficient. By sacrificing the CPU, you can obtain the work that requires more memory or hard disk space.

A very simple time-space-changing algorithm implements the value exchange between a and B variables. The most common way to exchange two variables is to use an intermediate variable, and introducing additional variables means to use more space. The following method can be used to avoid intermediate variables and achieve variable exchange, at the cost of introducing more CPU operations.

Listing 10. Sample Code

a=a+b;b=a-b;a=a-b;

Another useful example is the support for unsigned integers. In Java, unsigned integers are not supported. This means that the use of Short instead is required for unsigned bytes, which also means a waste of space. The following code demonstrates the use of bitwise operations to simulate unsigned Byte. Although more CPU operations are required in the process of Value Setting and value setting, the demand for memory space can be greatly reduced.

Listing 11. unsigned integer operation

Public class UnsignedByte {public short getValue (byte I) {// convert byte to an unsigned number short li = (short) (I & 0xff); return li ;} public byte toUnsignedByte (short I) {return (byte) (I & 0xff); // convert short to unsigned byte} public static void main (String [] args) {UnsignedByte ins = new UnsignedByte (); short [] shorts = new short [256]; // declare a short array for (int I = 0; I <shorts. length; I ++) {// The array cannot exceed the shorts [I] = (short) I;} byte [] bytes = new byte [256]; // Replace the short array for (int I = 0; I <bytes. length; I ++) {bytes [I] = ins. toUnsignedByte (shorts [I]); // store the data in the short array to the byte array} for (int I = 0; I <bytes. length; I ++) {System. out. println (ins. getValue (bytes [I]) + ""); // retrieves the unsigned byte from the byte array }}}

The running output is shown in List 12. The length is limited and only 10 is displayed.

Listing 12. Running output

0 1 2 3 4 5 6 7 8 9 10

If the CPU capability is weak, you can sacrifice space to improve the computing capability. The instance code is shown in listing 13.

Listing 13. Improving computing capability

Import java. util. arrays; import java. util. hashMap; import java. util. map; public class SpaceSort {public static int arrayLen = 1000000; public static void main (String [] args) {int [] a = new int [arrayLen]; int [] old = new int [arrayLen]; Map <Integer, Object> map = new HashMap <Integer, Object> (); int count = 0; while (count <. length) {// initialize the array int value = (int) (Math. random () * arrayLen * 10) + 1; if (map. get (value) = null) {map. put (value, value); a [count] = value; count ++ ;}} System. arraycopy (a, 0, old, 0,. length); // copy all data from array a to the old array long start = System. currentTimeMillis (); Arrays. sort (a); System. out. println ("Arrays. sort spend: "+ (System. currentTimeMillis ()-start) + "ms"); System. arraycopy (old, 0, a, 0, old. length); // restore the original data start = System. currentTimeMillis (); spaceTotime (a); System. out. println ("spaceTotime spend:" + (System. currentTimeMillis ()-start) + "ms");} public static void spaceTotime (int [] array) {int I = 0; int max = array [0]; int l = array. length; for (I = 1; I <l; I ++) {if (array [I]> max) {max = array [I] ;}} int [] temp = new int [max + 1]; for (I = 0; I <l; I ++) {temp [array [I] = array [I];} int j = 0; int max1 = max + 1; for (I = 0; I <max1; I ++) {if (temp [I]> 0) {array [j ++] = temp [I] ;}}}

The spaceToTime () function is used to sort arrays. It does not cost space and uses index subscript of arrays to indicate the data size. Therefore, mutual comparison between numbers is avoided, this is a typical idea of changing the space for time.

Conclusion

There are many ways to deal with and process high-throughput systems. The author will introduce them in series to cover all fields. This article describes the optimization and suggestions for the buffer, cache operation, Object reuse pool, and computing mode conversion. The optimization suggestions and solutions are verified starting with the actual code demonstration. The author has always believed that there are no optimization solutions that are effective, and the reader needs to make selection and practices based on the actual situation.

This article permanently updates the link address: