Actual Analysis of Oracle large data volume Query

I just made a data extraction of 50 million data records. The current table is still performing the insert operation, with hundreds of data entries per minute. Therefore, the query efficiency after the extra-score area is improved by more than 10 times.

I just made a data extraction of 50 million data records. The current table is still performing the insert operation, with hundreds of data entries per minute. Therefore, the query efficiency after the extra-score area is improved by more than 10 times.

Oracle Database:

I just made a data extraction of 50 million data records. The current table is still performing the insert operation, with hundreds of data entries per minute.

This table is partitioned by time and by month. There are no indexes. Currently, there are 14 fields, with an average of 30 bytes for each field. Partitions in the current table are one partition per month from 201101 to 201512.

Test server: xeno 5650,32-core cpu, win2003 operating system, 16 GB physical memory; test tool plsql

1. Initial query:

String. Format (@ "select * from
(Select r. id, r. carcode, r. longtitude, r. latitude, r. velocity, r. gpstime, r. isonline from t_gps_record r where id in (
Select min (id) from t_gps_record r where carcode = '{0 }'
Group by to_char (gpstime, 'yyyy-MM-dd HH24: Mi '))
And carcode = '{0 }'
And gpstime> (select nvl (select max (gpstime) from t_gps_carposition where carcode = '{0}'), (select min (gpstime) from t_gps_record where carcode = '{0}') from dual)
Order by gpstime asc
) Where rownum <= 200 ", row [" carcode "]. ToString ());

The query starts with 200 pieces of data. The query takes 2 minutes and 16 seconds;

Later, we checked 20 records for 2 minutes and 14 seconds. They basically had nothing to do with the number of records.

2. Later I wrote the minimum time as fixed:

String. Format (@ "select * from
(Select r. id, r. carcode, r. longtitude, r. latitude, r. velocity, r. gpstime, r. isonline from t_gps_record r where id in (
Select min (id) from t_gps_record r where carcode = '{0 }'
Group by to_char (gpstime, 'yyyy-MM-dd HH24: Mi '))
And carcode = '{0 }'
And gpstime> to_date ('2017-11-1 00:00:00 ', 'yyyy-mm-dd HH24: mi: ss ')
Order by gpstime asc
) Where rownum <= 200 ", row [" carcode "]. ToString ());

The query time is 1 minute 34 seconds.

3. query without extra points

Select r. id, r. carcode, r. longtitude, r. latitude, r. velocity, r. gpstime, r. isonline from t_gps_record r where id in (
Select min (id) from t_gps_record r
Group by carcode, to_char (gpstime, 'yyyy-MM-dd HH24: Mi '))
And gpstime> = to_date ('2017-11-1 9:00:00 ', 'yyyy-mm-dd HH24: mi: ss ') and gpstime <= to_date ('2017-11-1 9:59:59 ', 'yyyy-mm-dd HH24: mi: ss ')
Order by gpstime asc

Query time: 3 minutes 29 seconds, 1426 entries in total

4. Add partition Query

Select r. id, r. carcode, r. longtitude, r. latitude, r. velocity, r. gpstime, r. isonline from t_gps_record r where id in (
Select min (id) from t_gps_record partition (GPSHISTORY201111) r
Group by carcode, to_char (gpstime, 'yyyy-MM-dd HH24: Mi '))
And gpstime> = to_date ('2017-11-1 9:00:00 ', 'yyyy-mm-dd HH24: mi: ss ') and gpstime <= to_date ('2017-11-1 9:59:59 ', 'yyyy-mm-dd HH24: mi: ss ')
Order by gpstime asc

Query after adding a partition: 17 s, 1426 entries in total

Therefore, the query efficiency after the extra-score area is improved by more than 10 times, so it is very important to create a partition table with large data volumes.

Related reading:

Oracle Parallel Query

Oracle user information query operation statement

Performance problems of querying the maximum and minimum values of a column in a single Oracle table

The recycle bin causes slow usage of Oracle query table space