[Summary] problems that need to be paid attention to during large-scale data testing and data preparation ([protect existing data] [large-scale data impact normal testing] [do not worry about data deletion ])

Sometimes we need to perform a large-scale data test and insert a large amount of data into the database.

There are three points to consider:

[Protect existing data]

This has two purposes:

1. We only want to test the inserted data.

2. After the test, we need to delete the data we inserted.

How to do

(It can be used as a condition for Data Extraction in a program and as a condition for differentiation)

Example

For example, the condition for data extraction is select * from user where updatetime = one hour before the current system time

The current time is 20120517

In this way, we can adjust the updatetime of the existing data in the system to a very distant day in the future, such as 20120626

In this way, data storage will not be affected when the program is executed.

(Preparations: For the inserted data, we use the stored procedure to define a relatively early time, such as 20120501)

In this way, we can achieve the above two goals!

Benefits:

Protecting the stored data is not affected.

------------------------------------------

[Large-scale data impact test]

This is when I go home from work today.

When I was in the car, I suddenly thought of it!

This is precisely because of this. This is the most important thing !!

I used to think that after a large amount of data is made, we can no longer perform common tests.

Because a select condition suddenly extracts millions of data records.

But this is not the case, because we ignore a problem,

That is, we can change the extraction conditions so that this large amount of data is not extracted.

Or the extraction condition updatetime mentioned above, the current time is 20120517

The storage data is 20120501

We can adjust the data to 20120505.

(Although it takes a long time to change large-scale data, it is more time-consuming than deleting data)

In this way, large-scale data will no longer affect our normal tests.

(If the project permits debugdate, we can also modify the debugdate,

We will not allow millions of pieces of data to be extracted. This function is available for our project and you can set the debugging time,

For example, if you set the debugging time to 20120202, when the program is running,

The current system time is 20120202)

Benefits;

So that my test can proceed smoothly, without the influence of others and scale data.

---------------------------------------

[Do not worry about deleting data]

It is a very troublesome task to do data every time.

For example, this time, it takes about six hours to make 3 million pieces of data (both the postgre and Oracle.

It takes about one and a half hours to delete the data.

After one-time tests are completed, performance tests are also performed if the performance is poor.

Do not rush to delete the data, even before the next performance test, there will be a common functional test.

We can also save the data through the above method.

Making large-scale data does not affect our normal tests!

Benefits:

In this way, we don't have to spend a lot of time preparing to delete data every time.

Prepare once!