Mysql large tables are split into csv files for export, and mysql is split into csv files for export.
Recently, the company has tens of millions of rows of large tables that need to be split into different csv files according to the city id field.
I wrote an automated shell script.
Under/home/hdh
Linux-xud0:/home/hdh # llTotal 16-Rwxrwxrwx 1 root 902 Dec 28 07:47
Converts a able to a CSV file, and a DataTable to a csv file.
DataTable is used in. net projects to cache data. DataTable indicates a table with data in memory. CSV files were first used in a simple database. Because of their simple format and strong openness, they were first marked by their own gallery by the scanner. A csv
Mysql exports csv files separated by commas (,) and mysql exports csv files by commas (,).
In actual work, CleverCode often needs to export some reports or log data. If you directly create a page, if the number of times is not large, the requirements are also different. Therefore, exporting csv files directly is more intuitive.1 Export
This article mainly introduces the example of reading a csv file from python. This example briefly describes how to operate a csv file in python. For more information, see the following code:
Import csvFor line in open ("test.csv "):Name, age, birthday = line. split (",")Name = name. strip ('\ t \ r \ n ');Age = age. strip ('\ t \ r \ n ');Birthday = birthday. strip ('\ t \ r \ n ');Print (name + '\ t' +
Asp.net exports Excel/Csv text format data, csv text format
I just started to work on Excel-related projects, so I should record all the problems regardless of the size.
By chance, when adding data, all the data is converted into numbers, and the result output is automatically converted into scientific notation. This is a powerful excel function that can automatically recognize numbers and strings, it's too
#-*-coding:utf-8-*-ImportMathImportReImportCSVImportreprdefean_checksum (eancode):"""returns the checksum of a EAN string of length, returns-1 if the string has the length of the wrong""" ifLen (eancode)! = 13: return-1Oddsum=0 Evensum=0 Total=0 Eanvalue=Eancode Reversevalue= Eanvalue[::-1] Finalean=reversevalue[1:] forIinchRange (len (Finalean)):ifI% 2 = =0:oddsum+=Int (finalean[i])Else: Evensum+=Int (finalean[i]) Total= (Oddsum * 3) +evensum Check= Int (10-math.ceil (total% 10.0)
Java reads and exports csv files of multiple types of data, and java reads and exports csv files
1. Main Function
Public class csvtest {Public static void main (String [] args ){
List
Boolean isSuccess = CSVUtils. exportCsv (new File ("D:/1.csv"), dataList );System. out. println (isSuccess );
}
}
2. Car
Public class Car {String id;Double utc;Double x;Double y;Fl
Commons-csv is a toolkit provided by the Apache Open source organization for reading and writing CSV files, simpler, straightforward code:
Package com.gujin.csv;
Import Java.io.FileReader;
Import Java.io.PrintWriter;
Import Org.apache.commons.csv.CSVFormat;
Import Org.apache.commons.csv.CSVParser;
Import Org.apache.commons.csv.CSVPrinter;
Import Org.apache.commons.csv.CSVRecord;
Import Org.junit.Test; p
The first paragraph is read file, download.
The second paragraph is a string download.
Copy the Code code as follows:
$fileName = "Prefs.csv";Header (' Content-type:application/octet-stream ');Header (' content-disposition:attachment; Filename= '. $fileName);Header (' content-transfer-encoding:binary ');Header (' Content-length: '. FileSize ($fileName));ReadFile ($fileName);
Copy the Code code as follows:
$fileName = "Pref_". Date ("Ymdhis"). ". csv
Python csv module, pythoncsv
1 import csv # Comma Separated Value 2 3 # class Dialect 4 # delimiter = None # separator 5 # doublequote = None # The element is a quote, double the quote. The default value is true. If it is set to false, you need to set the delimiter. Otherwise, an error is returned. 6 # escapechar = None # Escape Character 7 # lineterminator = None # line terminator 8 # quotechar = None # re
Csvhelper is a. NET library that reads and writes a CSV file. Csvhelper can be downloaded through the Package manager of Visual Studio. Automatic mapping definition: In the absence of a mapping file, the default is auto-mapping, and the automatic mapping is sequentially mapped to the properties of the class.
GitHub Address
Read
Read all the records
var csv = new Csvreader (TextReader); var records =
13.1.CSV—CSVFile Reading and Writing the so-called CSV (Comma separated Values) format is the most common Import and export format for spreadsheets and databases. there is no ' CSV standard ', so the format was operationally defined by the many applications which read and write it. the lack of a standard means, subtle differences often exist in the data produc
PHP imports and exports CSV files to the mysql database. Because of work requirements, we need to export mysql data into csv files, and then we need to use php to import csv files to the mysql database, next I will introduce to you that we need to export data from mysql into a csv file because of work requirements. the
Label:For millions of data volumes of CSV file, the file size may reach hundreds of m, if simply read, it is likely to have a time-out or a card-dead phenomenon.Batch processing is necessary in order to successfully import data from a CSV file into a database.The following function reads a few rows of data specified in a CSV file:/** * Csv_get_lines read a few ro
1 project development, many times to import the external CSV file into the database or export the data as a CSV file, then how to implement it? In this paper, we will use PHP and combine MySQL to realize the import and export of CSV format data. 3 We first prepare the MySQL data table, assuming that the project has a record of student information table student, a
For millions of data CSV files, the file size may reach hundreds of M, and if a simple read is likely to be timed out or stuck to death.
Batch processing is necessary to successfully import data from a CSV file into a database.
The following function reads a few rows of data specified in a CSV file:
The code is as follows
Copy Code
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.