python pandas dataframe tutorial

Most of the students who Do data analysis start with excel, and Excel is the most highly rated tool in the Microsoft Office Series.But when the amount of data is very large, Excel is powerless, python Third-party package pandas greatly extend the functionality of excel, the entry takes a little time, but really is the necessary artifact of big data!1. Read data from a filePandas supports the reading of mult

([arr, arr], Axis=1) # Connect two arr, in the direction of the row---------------Pandas-----------------------Ser = series () Ser = series ([...], index=[...]) #一维数组, dictionaries can be converted directly to Seriesser.values ser.index Ser.reindex ([...], fill_value=0) #数组的值, index of array, redefine index ser.isnull () pd.isn Ull (Ser) pd.notnull (Ser) #检测缺失数据ser. name= ser.index.name= #ser本身的名字, ser index name Ser.drop (' x ') #丢弃索引x对应的值ser +ser

NaNDtype:object----[' A ', ' B ', ' C ']0 A1 12 NaN3 NaNDtype:object0 b1 22 NaN3 NaNDtype:object0 10 A B,c1 1 2,32 Nan Nan3 Nan Nan0 10 A, b C1 32 Nan Nan3 Nan NanDataframe0 a-b-c1 1-2-c2 [,-,-,]Name:key2, Dtype:object0 [A, b, c]1 [1, 2, C]2 NaNName:key2, Dtype:object#String Indexs = PD. Series (['A','b','C','Bbhello','123', Np.nan,'HJ']) DF= PD. DataFrame ({'Key1': List ('abcdef'), 'Key2':['Hee','FV','W','Hjja','123', Np.nan]})Prin

:00:00 ')2. Time interval# #TimedeltasPD. Timestamp ('9/3/2016')-pd. Timestamp ('9/1/2016')# timedelta (' 2 days 00:00:00 ')PD. Timestamp ('9/2/2016 8:10AM') + PD. Timedelta ('12D 3H')# Timestamp (' 2016-09-14 11:10:00 ') 3. Time in the DataframeDates = Pd.date_range ('10-01-2016', Periods=9, freq='2w-sun') Dates"""Datetimeindex ([' 2016-10-02 ', ' 2016-10-16 ', ' 2016-10-30 ', ' 2016-11-13 ', ' 2016-11-27 ', ' 2016-12-11 ', ' 2016 -12-25 ', ' 2017-01-08 ', ' 2017-01-22 '], dtype= ' datetime64

Function Prototypes:Https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.fillna.html#pandas.DataFrame.fillnaPad/ffill: Fills the missing value with the previous non-missing valueBackfill/bfill: Fills the missing value with the next non-missing valueNone: Specify a value to replace the missing value 123456789101112131415161718192021st22232425262728293031323334353637383940414243444546474849505152535455565758596061 62 63

data conversion refers to filtering, cleaning, and other conversion operations on the data. Remove Duplicate data Repeating rows often appear in the Dataframe, Dataframe provides a duplicated () method to detect whether rows are duplicated, and another drop_duplicates () method to discard duplicate rows:Duplicated () and Drop_duplicates () methods defaultJudging all Columns, if you do not want to, the co

Using XLRD to read ExcelFilter 0 columns with a value greater than 99% and removeImport XlrdWorkbook=xlrd.open_workbook (R "123.xlsx")Table = Workbook.sheet_by_name (' Sheet1 ')Nrows=table.nrowsNcols=table.ncolsDel_col=[]For j in Range (Ncols):sum = 0For Ai in table.col_values (j):if ai = = 0.0:Sum+=1if float (sum)/nrows>=0.99:Del_col.append (j)print Del_col Using Pandas to read ExcelFilter 0 columns with a value greater than

, 164.000000f, 159.000000f, 157.000000f, 145.000000f, 135.000000f, 120.000000f, 104.000000f, 88.000000f, 77.000000f, Surface Chart Scripts # -*- coding: utf-8 -*-from matplotlib import pyplot as pltfrom mpl_toolkits.mplot3d import Axes3Dfrom pandas import DataFramedef draw(x, y, z):‘‘‘采用matplolib绘制曲面图:param x: x轴坐标数组:param y: y轴坐标数组:param z: z轴坐标数组:return:‘‘‘X = xY = yZ = zfig = plt.figure()ax = fig.add_subplot(111, projection=‘3d

Hierarchical Indexes Hierarchical indexing means you can have multiple indexes on an array, for example: a bit like a merged cell in Excel, right?Select a subset of the data based on the index to select a subset of the data from the other layer:Select data in the same way as the index in the layer:Multi-index series conversion to Dataframe hierarchical indexes play an important role in data reshaping and grouping, for example, the hierarchical index d

From OPENPYXL import load_workbook import pandas as PDdata = Pd.read_excel (' test1.xlsx ', sheetname=0) # col_data = List (data.ix[:, 5]) # Gets the fifth column that starts outside the header Row_data = List (data.ix [5,:]) # Gets the fifth row of data except the header starting with writer = PD. Excelwriter (' test2.xlsx ', engine= ' OPENPYXL ') book = Load_workbook (' test2.xlsx ') writer.book = Book result = PD.

the unique value of A, the number of occurrences (a, b) of the unique value of statistics = (1,3) c appears 1 times (A, B) = (2,4) appears 3 times - the Print(Pd.crosstab (df['A'],df['B'],normalize=true))#display in a frequency-based manner - Print('--------') - Print(Pd.crosstab (df['A'],df['B'],values=df['C'],aggfunc=np.sum))#values: A value array based on a factor aggregation - #Aggfunc: If the values array is not passed, the frequency table is computed, and if the array is passed, the calc

Python uses pandas and xlrd to read excel files, feature filtering columns, and pandasxlrd Use xlrd to read excelFilter and delete columns with 0 values over 99%.Import xlrdWorkbook = xlrd. open_workbook (R "123.xlsx ")Table = workbook. sheet_by_name ('Sheet1 ')Nrows = table. nrowsNcols = table. ncolsDel_col = []For j in range (ncols ):Sum = 0For ai in table. col_values (j ):If ai = 0.0:Sum + = 1If

Use Python for data analysis _ Pandas _ basic _ 2, _ pandas_2Reindex method of Series reindex In [15]: obj = Series([3,2,5,7,6,9,0,1,4,8],index=['a','b','c','d','e','f','g', ...: 'h','i','j'])In [16]: obj1 = obj.reindex(['a','b','c','d','e','f','g','h','i','j','k'])In [17]: obj1Out[17]:a 3.0b 2.0c 5.0d 7.0e 6.0f 9.0g 0.0h 1.0i 4.0j 8.0k NaNdtype: float64 If the current va

For example we have the dataframe like this: SPY AAPL IBM GOOG GLD2017-01-03 222.073914 114.311760 160.947433 786.140015 110.4700012017-01-04 223.395081 114.183815 162.940125 786.900024 110.8600012017-01-05 223.217606 114.764473 162.401047 794.020020 112.5800022017-01-06 224.016220 116.043915 163.200043 806.150024 111.7500002017-01-09 223.276779 117.106812 161.390244 806.650024 1

Excel has a computational function skew () for skewness, but it is unclear how to traverse with Excel, which has a large amount of data.Try using Python for resolution.The first time to learn python, did not expect to overcome the installation of various packages of sadness, incredibly successful implementation.python3.3:#this is a test case#-*-coding:gbk-*-print ("Hello