Thoughts on the disaster caused by an SQL script

This afternoon, my colleague updated a script on the PRD database server. Unexpectedly, a very common operation resulted in disastrous consequences, since our system recently changed the way of interacting with SAP, we used to access sap through another intermediate interface, recently migrated and changed interfaces.Program. However, shortly after my colleague's script was updated, the user found that some products in the order had been inexplicably changed, and the 500 boxes of sesame oil were selected on the order, after the order was generated, it was found that 500 boxes of olive oil were changed. Although we did not find a large number of error data, there was only one user, an error occurred while placing an order. The risk scope is relatively small. We finally breathed a sigh of relief and immediately fixed the issue and checked again and again, but a problem was also exposed. We did not strictly perform tests and review such global changes, so this emergency occurred. Fortunately, the final loss is not great.

However, this event also exposes some weak links in our development, such as testing. However, such global changes related to the system are not strictly implemented.CodeReview: there was no simulated environment test during execution in the PRD environment. After the developer completes, a confirmation was conducted, especially the execution of such database scripts, simply perform some tests and then execute them directly. Sometimes you can't see the problem simply, and if you haven't fully considered it during the test, the test will also come out, because this time does not cause all data errors, there is a problem only in special circumstances. Therefore, there are still some vulnerabilities in process management. Concerning the global system and key changes, after developers complete the development, they should first complete the unit test. Relevant colleagues must carefully verify the code and review the code, and then test the code again, after the test passes the OK command, it is handed over to the test group for a comprehensive test. For such global changes, even a small script requires a test case and a comprehensive test. Before executing the PRD environment, be sure to test again in a simulation environment closest to the PRD environment, and confirm OK before execution in the PRD environment. The ancient cloud: a thousand miles away. Absolute Truth. Remember it next time.