Python Big Data Processing summary

1.shop_min=shop.drop ([' category_id ', ' longitude ', ' latitude ', ' Price '],axis=1)
Delete multiple columns in pandas

2.mall=shop_min.drop_duplicates (subset= ' mall_id ')
Pandas a column to a heavy

3.python write file with blank line
# Python 2
With open ('/pythonwork/thefile_subset11.csv ', ' WB ') as outfile:
writer = Csv.writer (outfile)

# Python 3
With open ('/pythonwork/thefile_subset11.csv ', ' W ', newline= ') as outfile:
writer = Csv.writer (outfile)

4.python File Open Files mode
R opens the file in read-only mode. The pointer to the file will be placed at the beginning of the file. This is the default mode.
RB opens a file in binary format for read-only. The file pointer will be placed at the beginning of the file. This is the default mode.
r+ open a file for read-write. The file pointer will be placed at the beginning of the file.
rb+ opens a file in binary format for read-write. The file pointer will be placed at the beginning of the file.
W opens a file for writing only. Overwrite the file if it already exists. If the file does not exist, create a new file.
WB opens a file in binary format for writing only. Overwrite the file if it already exists. If the file does not exist, create a new file.
w+ open a file for read-write. Overwrite the file if it already exists. If the file does not exist, create a new file.
wb+ opens a file in binary format for read-write. Overwrite the file if it already exists. If the file does not exist, create a new file.
A opens a file for appending. If the file already exists, the file pointer will be placed at the end of the file. In other words, the new content will be written to the existing content. If the file does not exist, create a new file to write to.
AB opens a file in binary format for appending. If the file already exists, the file pointer will be placed at the end of the file. In other words, the new content will be written to the existing content. If the file does not exist, create a new file to write to.
A + opens a file for read and write. If the file already exists, the file pointer will be placed at the end of the file. The file opens with an append mode. If the file does not exist, create a new file to read and write.
ab+ opens a file in binary format for appending. If the file already exists, the file pointer will be placed at the end of the file. If the file does not exist, create a new file to read and write.

Python Big Data Processing summary