Comparison and testing of computing performance between rundry reports and set computing reports

1. Purpose

Test the performance of the rundry report and set computing Report on the same hardware and Web Container, and compare the performance differences in grouping, sorting, filtering, connection, and ranking in the report, and concurrency. During the test, the rundry report uses the built-in computing engine of the report tool, and the Set Computing report uses its built-in set computing engine.

2. Environment Description

Test Model: Dellinspiron 3420

CPU: Intel Core [email protected] * 4

Ram: 4G

HDD: West digital WDC (500g5400 rpm)

Operating System: Win7 (x64) SP1

JDK1.6

Database:Oracle11gr2

Tomcat:6.0.36 _ x64

TomcatjvmMemory:Xms512m-Xmx2048m

Rundry report version: 4.5.6

Report version5.0

3. Data Description

The test uses three tables T1, T2, and T3. The following table shows the information of the three tables, where the data volume is the number of records.

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/44/19/wKiom1PfNvLhdrimAABeYAl9fIo158.jpg "style =" float: none; "Title =" jisuan_report_1.jpg "alt =" wkiom1pfnvlhdrimaabeyal9fio158.jpg "/>

Table T1 and table T2 have the same structure as follows:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/44/19/wKioL1PfOAySwYQoAACBVEIm-LY989.jpg "style =" float: none; "Title =" jisuan_report_2.jpg "alt =" wKioL1PfOAySwYQoAACBVEIm-LY989.jpg "/>

T3 table structure:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/44/19/wKioL1PfOAyRN3SIAABXzKEk7Co067.jpg "style =" float: none; "Title =" jisuan_report_3.jpg "alt =" wkiol1pfoayrn3siaabxzkek7co067.jpg "/>

4. use case description 4.1 groups

Use table T2 to group data by date2 and date respectively in the report, and then group and summarize data by SQL:

Ds1: Select * From T2.

 

Report Format:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/44/19/wKiom1PfNvOhHpHNAABCHTLOZ00595.jpg "style =" float: none; "Title =" jisuan_report_4.jpg "alt =" wkiom1pfnvohhphnaabchtloz00595.jpg "/>

4.1.1 statement implementation

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/44/19/wKiom1PfNvSi8qoMAABnBwJnCBE925.jpg "style =" float: none; "Title =" jisuan_report_5.jpg "alt =" wkiom1pfnvsi8qomaabnbwjncbe925.jpg "/>

4.1.2 set computing report implementation

Set calculator script

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/44/19/wKioL1PfOA6T3bNvAADRBD4nstI953.jpg "style =" float: none; "Title =" jisuan_report_6.jpg "alt =" wkiol1pfoa6t3bnvaadrbd4nsti953.jpg "/>

Report Template

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/44/19/wKiom1PfNvXifjbbAAB3Po7Ppuw035.jpg "style =" float: none; "Title =" jisuan_report_7.jpg "alt =" wkiom1pfnvxifjbbaab3po7ppuw035.jpg "/>

4.2 sorting

UseT1Table, take 0.5 million records, sort the report by date field, the report shows 5 fields, take the number of SQL:

Ds1: Select * from T1 where rownum <500000.

 

Report Format:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/44/19/wKioL1PfOA-SFyPrAABmC_G3uw4369.jpg "style =" float: none; "Title =" jisuan_report_8.jpg "alt =" wKioL1PfOA-SFyPrAABmC_G3uw4369.jpg "/>

4.2.1 statement implementation

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/44/19/wKioL1PfOA_Dk0uhAABlnzLtDU8346.jpg "style =" float: none; "Title =" jisuan_report_9.jpg "alt =" wkiol1pfoa_dk0uhaablnzltdu8346.jpg "/>

4.2.2 set computing report implementation

Set the compute script:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/44/19/wKiom1PfNvbCo4HYAACq24BdbNs265.jpg "style =" float: none; "Title =" jisuan_report_10.jpg "alt =" wkiom1pfnvbco4hyaacq24bdbns265.jpg "/>

Report Template:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/44/1A/wKioL1PfORDRg8kvAABW3qax5UY155.jpg "style =" float: none; "Title =" jisuan_report_11.jpg "alt =" wkiol1pfordrg8kvaabw3qax5uy155.jpg "/>

4.3 Filter

UseT1Table, which is filtered by the ID field in the report. After filtering, the data volume is 82. Fetch SQL:

Ds1: Select * From T1.

 

Report Format:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/44/1A/wKioL1PfORHySl_VAABdiZWqg3o390.jpg "style =" float: none; "Title =" jisuan_report_12.jpg "alt =" wkiol1pforhysl_vaabdizwqg3o390.jpg "/>

4.3.1 statement implementation

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/44/19/wKiom1PfN_jBY7FLAABfXP5qXAw692.jpg "style =" float: none; "Title =" jisuan_report_13.jpg "alt =" wkiom1pfn_jby7flaabfxp5qxaw692.jpg "/>

4.3.2 set computing report implementation

Set calculator script

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/44/19/wKiom1PfN_jBeAsPAACXUYC0xJk675.jpg "style =" float: none; "Title =" jisuan_report_14.jpg "alt =" wkiom1pfn_jbeaspaacxuyc0xjk675.jpg "/>

Report Template:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/44/1A/wKioL1PfORLBRHo1AABg3XSlcTE798.jpg "style =" float: none; "Title =" jisuan_report_15.jpg "alt =" wkiol1pforlbrho1aabg3425cte798.jpg "/>

4.4 connection

Use table T2 and table T3 to connect to table T3 Based on the ID field T2 in the report:

Ds1: Select * From T2 where userid <267427523

DS2: Select * from T3 where userid <10485202

Ds1 records: 7171; DS2 records: 12730

Report Format:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/44/19/wKiom1PfN_nQcT5TAAA-iu6PYRE106.jpg "style =" float: none; "Title =" jisuan_report_16.jpg "alt =" wKiom1PfN_nQcT5TAAA-iu6PYRE106.jpg "/>

4.4.1 statement implementation

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/44/1A/wKioL1PfORLR931RAABZNu-8a1g854.jpg "style =" float: none; "Title =" jisuan_report_17.jpg "alt =" wKioL1PfORLR931RAABZNu-8a1g854.jpg "/>

4.4.2 set computing report implementation

Set the compute script:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/44/1A/wKioL1PfORPSTyn8AAESKiZHbsM133.jpg "style =" float: none; "Title =" jisuan_report_18.jpg "alt =" wkiol1pforpstyn8aaeskizhbsm133.jpg "/>

Report Template:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/44/19/wKiom1PfN_rSgu7CAABAwIs_b34408.jpg "style =" float: none; "Title =" jisuan_report_19.jpg "alt =" wkiom1pfn_rsgu7caabawis_b34408.jpg "/>

4.5 ranking

Use the T3 table to rank the table in the report based on the sumtime. The SQL statement is as follows:

Ds1: Select * from T3 whereuserid <8883948

The number of records in the dataset ds1 is 10430.

 

Report Format:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/44/1A/wKioL1PfORTjXimEAABPdpzoWco011.jpg "style =" float: none; "Title =" jisuan_report_0000jpg "alt =" wkiol1pfortjximeaabpdpzowco011.jpg "/>

4.5.1 statement implementation

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/44/1A/wKiom1PfOZfS0sYTAACAhwwwEDg384.jpg "style =" float: none; "Title =" jisuan_report_21.jpg "alt =" wkiom1pfozfs0sytaacahwwwedg384.jpg "/>

4.5.2 set computing report implementation

Set the compute script:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/44/1B/wKioL1PfOrKRHy0wAAE6EO28oX8124.jpg "style =" float: none; "Title =" jisuan_report_22.jpg "alt =" wkiol1pforkrhy0waae6eo28ox8124.jpg "/>

Report Template:

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/44/1A/wKiom1PfOZmTVBqBAABxj0WOPFE802.jpg "style =" float: none; "Title =" jisuan_report_23.jpg "alt =" wkiom1pfozmtvbqbaabxj0wopfe802.jpg "/>

4.6 concurrent groups

Use the 5.1 grouping case to perform multi-concurrency grouping tests. Here Four concurrency tests are used.

5. Test Method

Deploy the rundry report application and the centralized computing report application using two identical Tomcat (such as JVM). The report data sources are the same. The rundry report uses the SQL dataset to retrieve the data directly; A set computing report uses a set of datasets, and the number is completed in DFX. Compare the performance of reports such as grouping and sorting completed in the report. Restart Tomcat before each test case to ensure that no other reports occupy memory and other resources.

In addition, due to the configuration of the test machine, different data tables are used in different use cases. Therefore, different use cases are not vertically comparable. The main difference is horizontal, that is, the performance difference between the rundry report and the centralized report.

6. Test Results

* The following table shows the data unit: seconds.

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/44/1B/wKioL1PfOrOTfb6ZAAGKo0YN90U058.jpg "style =" float: none; "Title =" jisuan_report_24.jpg "alt =" wkiol1pforotfb6zaagko0yn90u058.jpg "/>

[Description]

Here, the time record is counted from the SQL query to the report calculation. The two time points are directly provided in the logs of the dry report, and the time interval is reduced; in a set computing report, you need to add the debug information in the script to record the time when the number is obtained, and then compare it with the time when the report is calculated in the report log.

7. interpretation and analysis

A set computing report stores computing in the Set Computing script. The set computing tool uses a more efficient hash algorithm, which greatly improves the computing of Data associations such as grouping, hard traversal computation, such as filtering and sorting, is not significantly improved at the algorithm level. A small amount of improvement is manifested mainly because the computation in the Set calculator does not carry the display attribute.

All the operations in this test are single-threaded. In fact, the cube can also support multi-threaded parallel computing to fully utilize the CPU's multiple cores, further improving performance, even in sorting and filtering operations, it can be improved several times. Most reporting tools, including rundry reports, currently do not support multi-thread parallel computing and cannot use multiple CPU cores to improve performance.