Python Data visualization Programming combat-Import data

1. Importing data from a CSV file

Principle: The With statement opens the file and binds to object F. There is no need to worry about closing the data file after the resource is finished, and the context manager of the with will help. The Csv.reader () method then returns the reader object, which iterates through all the rows of the file being read.

1 #!/usr/bin/env python2 3 ImportCSV4 5filename ='Ch02-data.csv'6 7data = []8 Try:9 With open (filename) as F:TenReader =Csv.reader (f) Onec =0 A          forRowinchReader: -             ifc = =0: -Header =Row the             Else: - data.append (Row) -c + = 1 - exceptCSV. Error as E: +     Print "Error reading CSV file at line%s:%s"%(Reader.line_num, E) -Sys.exit (-1) +  A ifHeader: at     PrintHeader -     Print '==================' -  -  forDataRowinchData: -     PrintDataRow

Experimental results:

2. import file data from Excel

The Excel file can be converted to a CSV file and then imported through the above method, but if you want to automate the processing of a large number of files (as part of a continuous data processing process), then manually converting each Excel file into a CSV file is not a viable option.

Principle: Use the XLRD module to open the workbook of the file, then read the contents of the cell based on the number of rows (nrows) and the number of columns (ncols), and return a Xlrd.book instance by calling the Open_workbook () method.

1 Importxlrd2  fromXlrd.xldateImportxldateambiguous3 4File ='ch02-xlsxdata.xlsx'5 6WB = Xlrd.open_workbook (filename=file)7 8WS = Wb.sheet_by_name ('Sheet1')9 TenDataSet = [] One  A  forRinchRange (ws.nrows): -Col = [] -      forCinchRange (ws.ncols): the Col.append (Ws.cell (R, c). Value) -         ifWs.cell_type (r, c) = =xlrd. Xl_cell_date: -             Try: -                 PrintWs.cell_type (R, c) +                  fromDatetimeImportdatetime -Date_value =Xlrd.xldate_as_tuple (Ws.cell (R, c). Value, Wb.datemode) +                 PrintDateTime (*date_value) A             exceptxldateambiguous as E: at                 Printe - Dataset.append (COL) -  -  fromPprintImportPprint -  -Pprint (DataSet)

Experimental results:

3. Importing data from a fixed-width data file

Time log files and time series-based files are the most common data sources in data visualization. Sometimes, you can read them in a CSV dialect with tab-delimited data, but sometimes they are not delimited by any special characters. In fact, the fields of these files are fixed-width, and we can match and extract the data by format.

For example (the data in this example is generated using code):

Processing method:

1. Specify the data file to read. 2. Define how data is read. 3. Read the file row by line and parse each row into separate data fields according to the format. 4. Each row is printed in the form of an individual data field.

1 Importstruct2 Importstring3 4mask='9s14s5s'5Parse =struct. Struct (Mask). Unpack_from6 Print 'formatstring {!r}, record size: {}'. Format (7 Mask, struct.calcsize (mask))8 9DataFile ='Ch02-fixed-width-1m.data'Ten  OneWith open (datafile,'R') as F: A      forLineinchF: -Fields =Parse (line) -         Print 'Fields :', [Field.strip () forFieldinchFields

Experimental results:

4. Importing data from a JSON data source

The procedure is as follows: 1. Specify the GitHub URL to read the JSON format data. 2. Use the requests module to access the specified URL and read the content. 3. Read the content and convert it to a JSON-formatted object. 4. Iterate through the JSON object and, for each of these items, read the URL value for each code base.

Principle: First, use the requests module to obtain remote resources. The Requests module provides a simple API to define HTTP verbs, and we only need to emit a get () method call. We are only interested in the Response.json () method, which reads the contents of Response.content, parses it into JSON and loads it into a JSON object.

The code is as follows:

1 Import Requests 2  from Import Pprint 3 ' https://api.github.com/users/justglowing ' 4 r = requests.get (URL)5 json_obj = R.json ()6 pprint (json_obj)

Results:

Conclusion: Last month in helping others do graduation design, with the flask, and then this month also use Java EE to write a mall website, busy dying, has not updated blog, today Sunday Library to see the Python data visualization, wander more than half, alas, or more blog it, unfortunately, Good to do the boutique series ....

Python Data visualization Programming combat-Import data