How can we better process user-uploaded JSON data?

When users upload JSON data to our server, we need to parse the data and then upload it to the cloud. Now we have two opinions: the first is to verify the JSON format on the server, including the data that needs to be uploaded to the cloud after resolution. Because the user uploads JSON data to our server, we need to parse the data and then upload it to the cloud. We have two comments:

The first is to verify the JSON format on the server, including the data to be uploaded to the cloud after resolution. Because the layer of JSON Nesting is deep, we need to determine whether it is null for each layer of parsing. If it is null, the system will directly handle the error.

The second is to assume that the data uploaded by the user is valid. As long as an exception is caught in the outermost layer, the thread will not be suspended, and no judgment or verification is required for parsing and processing in the thread. The two have their own advantages and disadvantages. What should I do in this case?

Reply content:

When users upload JSON data to our server, we need to parse the data and then upload it to the cloud. We have two comments:

The first is to verify the JSON format on the server, including the data to be uploaded to the cloud after resolution. Because the layer of JSON Nesting is deep, we need to determine whether it is null for each layer of parsing. If it is null, the system will directly handle the error.

The second is to assume that the data uploaded by the user is valid. As long as an exception is caught in the outermost layer, the thread will not be suspended, and no judgment or verification is required for parsing and processing in the thread. The two have their own advantages and disadvantages. What should I do in this case?

Incorrect data must be out of the first door.

First, it cannot be assumed thatProcessing failed (throwing an exception) = data error. There must be no connection between these two points. There will always be data that is logically required for business, but if left blank, it will not cause fatal errors of the program.

Of course, it can also be said that the data validation is carried out gradually with the process of transaction processing. But this will cause two problems:

1. The delay of time does not immediately result in feedback on whether the data is legal or not. Users always try the API in unexpected ways, And they want immediate feedback when making mistakes, instead of waiting for the server to fail for half a day.
2. If half of the transactions are executed, we need to discard them to waste resources on the server. It is definitely better not to start the transaction for illegal data.

Please verify the data as soon as the user data is entered. For RESTful APIs, you can consider returning the corresponding HTTP Error status code and the JSON format error message body to help users get correct feedback.

Digress.

API interfaces are a little worse. For the webpage form interfaceNode. jsIt will show a huge advantage: For Node. js programs, the front-end and back-end data validation code isDirect ReuseOf!

For the general PHP backend, you may have to find a solution. I suggest you define a unified JSON format validation table, define which data fields require which constraints (required, email, number, etc.), and check the same validation table in the front and back sections, write common code for verification.

What is more taboo is the verification code for the same purpose. Manually use the native JS and PHP to write two copies-maintain the synchronization of the two copies of the same purpose code, and you will know that it will be dead!

What are your definitions of empty data: if the correct data allows null data, empty judgment and error processing are required, capture exceptions on the outer layer.

1. Determine whether the key data is empty before uploading
2. upload to the server for full verification
3. handle exceptions when displaying or using them

1. Is exceptions related to thread disconnection inevitable? The thread is suspended due to Data Exception check. It can only be said that the exception handling mechanism of the program is not completed properly. Many people like to capture the exception and directly throw the stack information, instead of handling the exception. Every time I see this kind of processing, I will be crazy. If it is thrown out, you have to analyze the reason for the throttling, or even make a decision. For example, if. I rely on it. For else, even if you do not record the log or perform other processing, you must print it at least to let people know about debugging. The problem occurs here. Instead of following up step by step with IDE debug. (Digress, too long .)... I believe that if lz is empty, it will directly handle the error. The so-called processing is to directly kill the thread... ...

2. As for the second point, I don't know whether lz is based on the customer's kindness or the customer's own. I assume that "the data uploaded by the user is legal". For programmers, this assumption is the most fatal, whether from design or coding, because you cannot represent all the operations of the customer at all. In fact, this proves the above view, lz programs are not robust enough to handle exceptions. During writing, the normal input state is considered, but the Abnormal Input state is not considered. Therefore, when designing or coding, it is best not to assume what the customer is like. If the assumption is true, please come up with data that can support the assumption. As mentioned above, assume that the uploaded data is valid. If you persuade the boss or the customer that your assumption is true (please ensure that each input is valid ?) Your assumption is as follows: During the upload, the program has strictly verified the data, and the data basically conforms to the format (Sorry, it will always happen ), the data obtained by subsequent programs is legal. How to prove that the subsequent program obtains data is legal, and the verification program is taken out, this is proof. Of course, the verification process is not rigorous enough, or some are not included in the verification process, which is another aspect. After all, some people will ask: how do you prove the accuracy of your verification program and whether it has been verified ..

3. Finally, in terms of security, data must at least be verified. Verified data can at least filter out known potential risks.

4. For lz problems, unless the data uploaded by the user is not important, data verification is required.