MongoDB Execution Mongoexport exceptions and analysis (for numeric types of queries)

Today in the use of mongoexport export to meet certain conditions of the data, encountered an error, is now recorded, and for this error on the MongoDB digital type to do further study.

Background and error information

Today you receive a business requirement to export data from the MONGODB database order collection that meets the following criteria:

Db.qqwj_order.find ({"Source": Numberint ("+"), "Batch": "Order Reminder in Payment: 2018/9/5", "msgcontent":/Not yet completed online payment/})

Through the MongoDB client tool "Nosqlbooster for MongoDB" query check, the statement execution is normal, showing the corresponding number of records is 15265.

To export the data using the Mongoexport command, execute the following command:

/data/mongodb/mongobin344/bin/mongoexport-h 172.x.x.xxx--port Port--db Database-U account-P ' password '--authenticationdatabase authentication database --type=csv-c qqwj_order-f msgcontent,rec_createtime- q ' {"Source": Numberint ("+"), "Batch": "Order Reminder in Payment: 2018/9/5", "Msgcontent":/Not yet complete online payment/} ' -o/data/mongodb_back/sms.csv

But execution error:

XXX is not valid JSON:json:cannot Unmarshal string into Go value of type JSON. Numberint

The error is as follows:

Error inference and testing

Because the error message in the Numberint keyword, at this time to see our query conditions just also have this keyword, so speculation is not the problem.

As a result, the Numberint ("21") in the Export command is replaced directly with 21 and executed again.

The execution commands are:

/data/mongodb/mongobin344/bin/mongoexport-h 172.x.x.xxx--port Port--db Database-U account-P ' password '--authenticationdatabase authentication database --type=csv-c qqwj_order-f msgcontent,rec_createtime-q ' {"Source": +, "Batch": "Order Reminder in Payment: 2018/9/5", "msgcontent":/ Online payment not yet completed/} '-o/data/mongodb_back/sms.csv

Execution results are

The results show that the data was successfully exported after the modification.

Error analysis and Principle research

Why the query is viewed, the data is "source": Numberint ("21"), but the execution of the export command in the shell is written as "source": Numberint ("21") will be an error. And be sure to convert to "Source": 21

The source field queried by the query is displayed:

Obviously is "Source": Numberint ("21"), why copy to Shell, execute error???

Look back and find the principle. We know that currently MongoDB supports 4 of data types.

• Double
• 32-bit integer
• 64-bit integer
• Decimal (New in version 3.4.)

The MongoDB client can execute the query, but the export cannot be performed in the shell, so will it be related to these two tools? Will it be related to the inserted numberint (number) or Numberint (' number ')?

The following assumptions are tested for validation.

Inserting test data by Nosqlbooster for MongoDB mode

Inserting test data in shell mode

To view the inserted data type through $type

1 "Execution Db.numbers.find ({n:{$type: 1}})//type double; query data of type double

The results of the above query show that, whether through the client or shell, when a number does not indicate a data type, the inserted numeric data is double by default.

2 "execute command db.numbers.find ({n:{$type: +}})//type is 32-bit integer; query data of type 32-bit integer

The query table name above, regardless of whether through the client or shell, the specified Numberint (5) or Numberint (' 5 ') background is converted to a unified 32-bit integer type storage.

3 "execution command Db.numbers.find ({n:{$type:)})//type for 64-bit integer query type is 64-bit integer data

The query table name above, regardless of whether through the client or shell, the specified Numberlong (5) or Numberlong (' 5 ') background is converted to a unified 64-bit integer type storage.

The above test shows that when we store digital data, it is automatically dumped (regardless of the client tool, Shell or "Nosqlbooster for MongoDB", whether Numberlong (5) or Numberlong (' 5 '); numberint (5) or Numberint (' 5 ').

A little confused, huh? So, why is the query an error?

Look back. Error tip: XXX is not valid JSON:json:cannot unmarshal string into Go value of type JSON. Numberint.

It means that the shell thinks we've passed a character type of data to JSON. Numberint.

Then I will convert the Numberint ("21") in the Export command to Numberint (21)

The execution commands are:

/data/mongodb/mongobin344/bin/mongoexport-h 172.x.x.xxx--port Port--db Database-U account-P ' password '--authenticationdatabase authentication database --type=csv-c qqwj_order-f msgcontent,rec_createtime-q ' {"Source": numberint (+), "Batch": "Order Reminder in Payment: 2018/9/5", " Msgcontent ":/Not yet complete online payment/} '-o/data/mongodb_back/sms.csv

Execution was also successful.

Conclusion

Said a lot under the conclusion:

The export command that failed to execute is:

/data/mongodb/mongobin344/bin/mongoexport-h 172.x.x.xxx--port Port--db Database-U account-P ' password '--authenticationdatabase authentication database --type=csv-c qqwj_order-f msgcontent,rec_createtime-q ' {"Source":numberint ("+"), "Batch": "Order Reminder in Payment: 2018/ 9/5 "," msgcontent ":/Not yet complete online payment/} '-o/data/mongodb_back/sms.csv

The successful export commands are:

/data/mongodb/mongobin344/bin/mongoexport-h 172.x.x.xxx--port Port--db Database-U account-P ' password '--authenticationdatabase authentication database --type=csv-c qqwj_order-f msgcontent,rec_createtime-q ' {"Source":+, "Batch": "Order Reminder in Payment: 2018/9/5", " Msgcontent ":/Not yet complete online payment/} '-o/data/mongodb_back/sms.csv

And

/data/mongodb/mongobin344/bin/mongoexport-h 172.x.x.xxx--port Port--db Database-U account-P ' password '--authenticationdatabase authentication database --type=csv-c qqwj_order-f msgcontent,rec_createtime-q ' {"Source": numberint, "Batch": "Order Reminder in Payment: 2018/9/ 5 "," msgcontent ":/Not yet complete online payment/} '-o/data/mongodb_back/sms.csv

Three export commands are marked in red in different places.

P. S 1: Later the author delved into why the same query, the query results, some show "n": 5; some show "n": Numberint ("5"). Hehe "" "version of the different.

Display of older versions (partial)

Display of the new version (e.g. nosqlbooster4mongo-4.7.1)

P. S 2: When storing digital data, exactly what type of data is stored, in fact, the driver of the language. For example, in the Ruby and Python languages, when serializing integers, the driver automatically determines whether the encoding is 32-bit integer or 64-bit Integer;shell needs to be specified for display.

This article copyright belongs to the author, without the consent of the author, thank you!!!