Python pandas 擷取Excel重複記錄

標籤：ica   img   das   highlight   .com   ted   python   響應   column   

pip install pandaspip install xlrd

大量記錄的時候，用EXCEL排序處理比較費勁，EXCEL程式動不動就無響應了，用pands完美解決。

# We will use data structures and data analysis tools provided in Pandas libraryimport pandas as pd# Import retail sales data from an Excel Workbook into a data frame# path = ‘/Documents/analysis/python/examples/2015sales.xlsx‘path = ‘F:/python/an.xlsx‘xlsx = pd.ExcelFile(path)df = pd.read_excel(xlsx, ‘Sheet1‘)# Let‘s add a new boolean column to our dataframe that will identify a duplicated order line item (False=Not a duplicate; True=Duplicate)df[‘is_duplicated‘] = df.duplicated([‘ip‘])# We can sum on a boolean column to get a count of duplicate order line items# df[‘is_duplicated‘].sum()# Get the records of duplicated, If you need non-dup just use False insteaddf_dup = df.loc[df[‘is_duplicated‘] == True]# Finally let‘s save our cleaned up data to a csv filedf_dup.to_csv(‘dup.csv‘, encoding=‘utf-8‘)

 ref:https://33sticks.com/python-for-business-identifying-duplicate-data/

 

Python pandas 擷取Excel重複記錄