Shapefile cannot afford to hurt

Recently, we have processed road links nationwide and want to create a complete Path Based on Link names, levels, and attributes.

To ensure efficiency, the shapefile format is used as the intermediate format and gdal is directly used to manipulate the shapefile format.

Data export and link belong to the same judgment, linestring flip, linestring link, data export, and other modules have been compiled.

Finally, the question of how to manipulate the national LINK (about 10 million pieces of data) came:

Just a few hundred million records, the disk space is occupied by 10 Gb!

Shit!

I forgot the length limit of the DBF string, and the DBF string is not longer. This disk space is too large to be occupied ~~

The error message "gdal" indicates that the field length is 255, Which is truncated. The number of fields and the length of field names are also limited.

These are all hard-hitting shapefile.

Convert to directly manipulate MySQL to see how efficient it is.