Learn python Big Data processing module pandas

For beginners

First lesson structuring data

This section basic understanding of some of the pandas data structure and the basic use of modules, a preliminary understanding of the pandas provide some of the functions, learning basic use.

Create data

A list of tuples consisting of a tuple is constructed from Python's zip as the input data of the Dataframe Rec.

in [3]: Import Pandas as Pdin [4]: Import Randomin [5]: num = random.sample (xrange (10000, 10 00000), 5) in [6]: numout[6]: [244937, 132008, 278446, 613409, 799201]in [8]: names = ' Hello the cruel World en '. Split () in [9]: namesout[9]: [' Hello ', ' the ', ' cruel ', ' world ', ' en ']in [ten]: rec = ZIP (names, num) in []: data = PD.   DataFrame (REC, columns = [u "name", U "Performance"]) in [+]: dataout[16]: Name performance 0 Hello 2449371 the 1320082 cruel 2784463 world 6134094 en 799201  

The first parameter of the Dataframe method function is the data source, the second parameter columns is the header of the output data table, or the field name of the table.

Export Data csv

On the Windows platform coding problem, we can do a simple processing, is ipython-notebook support UTF8.

import sysreload(sys)sys.setdefaultencoding("utf8")

You can then export the data.

In [31]: dataOut[31]:     姓名  业绩0    hello   2449371    the     1320082    cruel   2784463    world   6134094    en      799201 #在ipython-note里后加问号可查帮助，q退出帮助In [32]: data.to_csv?In [33]: data.to_csv("c:\\out.csv", index = True, header = [u"雇员", u"销售业绩"])

The data is exported to the Out.csv file, the index parameter refers to whether there is a primary index, the header if not specified is the data in the columns as the header, if specified is the following list of strings as the table header, but note that the number of strings after the header and data columns The number of fields is the same.

can go to C disk with notepad++ open out.csv look.

Simple data analysis

In [43]: dataOut[43]:     姓名  业绩0    hello   2449371    the     1320082    cruel   2784463    world   6134094    en      799201#排序并取前三名In [46]: Sorted = data.sort([u"业绩"], ascending=False)         Sorted.head(3)Out[46]:     姓名  业绩4    en      7992013    world   6134092    cruel   278446

Graphics output

In [71]: import matplotlib.pyplot as plt         #使ipython-notebook支持matplotlib绘图         %matplotlib inlineIn [74]: df = data         #绘图         df[u"业绩"].plot()         MaxValue = df[u"业绩"].max()         MaxName = df[u"姓名"][df[u"业绩"] == df[u"业绩"].max()].values         Text = str(MaxValue) + " - " + MaxName         #给图添加文本标注         plt.annotate(Text, xy=(1, MaxValue), xytext=(8, 0), xycoords=(‘axes fraction‘, ‘data‘), textcoords=‘offset points‘)

If you comment out the plt.annotate line, the result is as follows:

Learn python Big Data processing module pandas