I wrote a program specifically designed to capture the data provided by an api. However, in order to prevent the old data status from being changed every time new data is inserted. (Insert cannot be used all the time, because it is possible that there are four pieces of data in this id, three pieces of data in the next time, and five pieces of data in the next time... I wrote a program specifically designed to capture the data provided by an api. However, in order to prevent the old data status from being changed every time new data is inserted. (Insert cannot be used all the time, because it is possible that there are four data records for this id, three data records for the next time, and five data records for the next time), so I used several arrays to record the data, for comparison, a database can be written only when data is different.
For example:
Do {Foreach (event id) {Foreach (first kind of data) // However, there are one to three differences in this kind of data {Foreach (first kind of data in small data) {// The following table lists the numbers of each vertex If (! $ M1 ['id'] ['comparison of the first data'] ['comparison key of the first data']) {$ m1 ['id'] ['comparison of the first data'] ['comparison key of the first data'] = implode ($ tmp1 ); // $ tmp: sqlInsert ($ tmp );} else {If ($ m1 ['id'] ['differences of the first data'] ['small data key values in the first data'])! = Implode ($ tmp )) {$ m1 ['id'] ['comparison of the first data'] ['comparison key of the first data'] = implode ($ tmp ); sqlInsert ($ tmp) ;}}} Foreach (second data type) // However, there are one or three differences in this kind of data {Foreach (small data in the second type) {// the following values are different If (! $ M2 ['id'] ['difference of the second data'] ['small data key value in the second data']) {$ m2 ['id'] ['differences of the second data'] ['small data key value in the second data'] = implode ($ tmp1 ); // $ tmp is the sqlInsert ($ tmp1 );} else {If ($ m2 ['id'] ['differences of the second data'] ['small data key values in the second data'])! = Implode ($ tmp1 )) {$ m2 ['id'] ['differences of the second data'] ['small data key value in the second data'] = implode ($ tmp1 ); sqlInsert ($ tmp1) ;}}} Foreach (level 3) // There are no small numbers in this group. {// the following values are different If (! $ M3 ['id'] ['third type of data'] ['empty']) {$ m3 ['id'] ['third type of data'] ['empty'] = implode ($ tmp2 ); // $ tmp: sqlInsert ($ tmp2 );} else {If ($ m3 ['id'] ['third type of data'] ['empty'])! = Implode ($ tmp2) {$ m3 ['id'] ['third type of data'] ['empty'] = implode ($ tmp2 ); sqlInsert ($ tmp2) ;}}// the following information about the upload metadata of the upload event's local id, etc. If (! $ M4 ['id'] ['event'] ['empty']) {$ m4 ['id'] ['event'] ['empty'] = implode ($ tmp5 ); // $ tmp: sqlInsert ($ tmp2 );} else {If ($ m4 ['id'] ['event'] ['empty'])! = Implode ($ tmp5) {$ m4 ['id'] ['event'] ['empty'] = implode ($ tmp5 ); sqlInsert ($ tmp5) ;}}// the following program is a response line. If the event's local id does not provide any information, unset ($ m1 ['id'] ~ $ M4 ['id']) sleep (3);} while (condition );
The program is like this, but about one day of execution, the host's cpu will carry to 100%, and such a program, I have about four to five threads to run .....
Each program uses four arrays 3D arrays for "record" operations. The IDS may contain 0-15
The first and second categories of data have 1-10 groups of small data.
Is this a correct way of thinking? Or is there any more efficient execution method?
The func in p.s. sqlinsert mainly refers to "changing the status of the previous data and then inserting a new one 」. The data update time ranges from 2 seconds to 10 minutes. After investigation, it is not an SQL problem, but php's own load is too heavy.
Reply content:
I wrote a program specifically designed to capture the data provided by an api. However, in order to prevent the old data status from being changed every time new data is inserted. (Insert cannot be used all the time, because it is possible that there are four data records for this id, three data records for the next time, and five data records for the next time), so I used several arrays to record the data, for comparison, a database can be written only when data is different.
For example:
Do {Foreach (event id) {Foreach (first kind of data) // However, there are one to three differences in this kind of data {Foreach (first kind of data in small data) {// The following table lists the numbers of each vertex If (! $ M1 ['id'] ['comparison of the first data'] ['comparison key of the first data']) {$ m1 ['id'] ['comparison of the first data'] ['comparison key of the first data'] = implode ($ tmp1 ); // $ tmp: sqlInsert ($ tmp );} else {If ($ m1 ['id'] ['differences of the first data'] ['small data key values in the first data'])! = Implode ($ tmp )) {$ m1 ['id'] ['comparison of the first data'] ['comparison key of the first data'] = implode ($ tmp ); sqlInsert ($ tmp) ;}}} Foreach (second data type) // However, there are one or three differences in this kind of data {Foreach (small data in the second type) {// the following values are different If (! $ M2 ['id'] ['difference of the second data'] ['small data key value in the second data']) {$ m2 ['id'] ['differences of the second data'] ['small data key value in the second data'] = implode ($ tmp1 ); // $ tmp is the sqlInsert ($ tmp1 );} else {If ($ m2 ['id'] ['differences of the second data'] ['small data key values in the second data'])! = Implode ($ tmp1 )) {$ m2 ['id'] ['differences of the second data'] ['small data key value in the second data'] = implode ($ tmp1 ); sqlInsert ($ tmp1) ;}}} Foreach (level 3) // There are no small numbers in this group. {// the following values are different If (! $ M3 ['id'] ['third type of data'] ['empty']) {$ m3 ['id'] ['third type of data'] ['empty'] = implode ($ tmp2 ); // $ tmp: sqlInsert ($ tmp2 );} else {If ($ m3 ['id'] ['third type of data'] ['empty'])! = Implode ($ tmp2) {$ m3 ['id'] ['third type of data'] ['empty'] = implode ($ tmp2 ); sqlInsert ($ tmp2) ;}}// the following information about the upload metadata of the upload event's local id, etc. If (! $ M4 ['id'] ['event'] ['empty']) {$ m4 ['id'] ['event'] ['empty'] = implode ($ tmp5 ); // $ tmp: sqlInsert ($ tmp2 );} else {If ($ m4 ['id'] ['event'] ['empty'])! = Implode ($ tmp5) {$ m4 ['id'] ['event'] ['empty'] = implode ($ tmp5 ); sqlInsert ($ tmp5) ;}}// the following program is a response line. If the event's local id does not provide any information, unset ($ m1 ['id'] ~ $ M4 ['id']) sleep (3);} while (condition );
The program is like this, but about one day of execution, the host's cpu will carry to 100%, and such a program, I have about four to five threads to run .....
Each program uses four arrays 3D arrays for "record" operations. The IDS may contain 0-15
The first and second categories of data have 1-10 groups of small data.
Is this a correct way of thinking? Or is there any more efficient execution method?
The func in p.s. sqlinsert mainly refers to "changing the status of the previous data and then inserting a new one 」. The data update time ranges from 2 seconds to 10 minutes. After investigation, it is not an SQL problem, but php's own load is too heavy.