04-12 problems caused by stability test-DB partition number

This should be the first time I have talked about stress testing and stability testing. The reason is that I started to pay attention to such testing because it can indeed discover that it is very hidden, it is also difficult to find problems in functional testing.

In our product, logs are written into ms SQL SERVER. By default, each partition is 5 GB, and a new partition is automatically created if it exceeds 5 GB. If the traffic is hit at full capacity, 2-3 partitions can be written on average every day. I usually run stress testing and stability testing for a maximum of seven days, and the number of partition is about 10. Once in a while, I kept an environment and continued to hit the traffic for one month.

The problem occurs:

When the number of partition exceeds 40, all subsequent new partition errors will occur, and a new partition will be created continuously within one hour, with a hard disk full of TB. Terrible? Yes.

DEV checked the code and found root cause. This problem is exposed only when the partition reaches about 40. In fact, the user is normal, and will certainly encounter it after a period of time.

 

This is just an example. During stress testing and stability testing, I also found many problems, such as memory leakage, handle leakage, and service restart.

In the future, I will be responsible for this part. accumulation is very important.