Using Python for data analysis (7)-pandas (Series and DataFrame), pandasdataframe 1. What is pandas? Pandas is a Python data analysis package based on NumPy for data analysis. It provides a large number of advanced data structures and data processing methods.
Pandas series DataFrame row and column data filtering, pandasdataframe
I. Cognition of DataFrame DataFrame is essentially a row (index) column index + multiple columns of data.
To simplify our understanding, let's change our thinking...
In reality, to simplify the descripti
convert to a format that can be found using XPath
= Doc.xpath ('//table ')
find all the tables in the document and return a list
Let's look at the source code of the Web page and find the form that needs to be retrieved
The first behavior title of the table, the following behavior data, we define a function to get them separately:
def _unpack (Row, kind= ' TD '):
ELTs = Row.xpath ('.//%s '%kind)
# Get data based on label type return
This question mainly writes the method of sorting series and dataframe according to index or value
Code:
#coding =utf-8
Import pandas as PD
import numpy as NP
#以下实现排序功能.
SERIES=PD. Series ([3,4,1,6],index=[' B ', ' A ', ' d ', ' C '])
FRAME=PD.
stringlengthstring (' Test ')")
They convert each other.
PANDAS_DF = Spark_df.topandas ()
SPARK_DF = Sqlcontext.createdataframe (PANDAS_DF)
function application
Df.apply (f) Apply function f to each column of DF
Df.foreach (f) or Df.rdd.foreach (f) Apply the function f to each column of DFDf.foreachpartition (f) or df.rdd.foreachPartition (f) applies each block of DF to the application function f
Ma
Pandas
Spark
Working style
Single machine tool, no parallel mechanism parallelismdoes not support Hadoop and handles large volumes of data with bottlenecks
Distributed parallel computing framework, built-in parallel mechanism parallelism, all data and operations are automatically distributed on each cluster node. Process distributed data in a way that handles in-memory data.Supports Hadoop and can handle large amounts of data
How do I delete the list hollow character?Easiest way: New_list = [x for x in Li if x! = ']This section mainly learns the basic operations of pandas based on the previous two data structures.设有DataFrame结果的数据a如下所示: a b cone 4 1 1two 6 2 0three 6 1 6
First, view the data (the method of viewing the object is also applicable for series)1. V
Previously written pandas DataFrame Applymap () functionand pandas Array (pandas Series)-(5) Apply method Custom functionThe applymap () function of the pandas DataFrame and the apply (
': 12, ' C2 ': 120}]DF = PD. DataFrame (INP) for index, row in df.iterrows (): print row[ ' C1 '], Row[ ' C2 ' #10 100 #11 110 #12 + Each iteration of Df.iterrows () is a tuple type that contains the index and the data for each row.
Using the Iterrows method, the resulting row is a series,dataframe dtypes will not be retained.
The returned
How do I delete the list hollow character?
Easiest way: New_list = [x for x in Li if x! = ']
Today is number No. 5.1.
This section mainly learns the basic operations of pandas based on the previous two data structures.
Data A with dataframe results is shown below: a b cone 4 1 1two 6 2 0three 6 1 6
First, view the data (the method of viewing the object is also applicable fo
Previous Pandas DataFrame the Apply () function (1) says How to convert DataFrame by using the Apply function to get a new DataFrame.This article describes another use of the dataframe apply () function to get a new pandas Series:
Forgive me for not having finished writing this article is a record of my own learning process, perfect pandas learning knowledge, the lack of existing online information and the use of Python data analysis This book part of the knowledge of the outdated,I had to write this article with a record of the situation. Most if the follow-up work is determined to have time to complete the study of Pandas Library,
This article mainly introduced the Python pandas in the Dataframe type data operation function method, has certain reference value, now shares to everybody, has the need friend to refer to
The Python data analysis tool pandas Dataframe and series as the primary data structu
This article mainly introduces pandas in python. the DataFrame method for excluding specific rows provides detailed sample code. I believe it has some reference value for everyone's understanding and learning. let's take a look at it. This article mainly introduces pandas in python. the DataFrame method for excluding s
This article mainly introduces you to the pandas in Python. Dataframe to exclude specific lines of the method, the text gives a detailed example code, I believe that everyone's understanding and learning has a certain reference value, the need for friends to see together below.
Objective
When you use Python for data analysis, one of the most frequently used structures is the
']], columns=['p1', 'p2 ...: ', 'p3'])In [4]: dfOut[4]: p1 p2 p30 GD GX FJ1 SD SX BJ2 HN HB AH3 HEN HEN HLJ4 SH TJ CQ
If you only want two rows whose p1 is GD and HN, you can do this:
In [8]: df[df.p1.isin(['GD', 'HN'])]Out[8]: p1 p2 p30 GD GX FJ2 HN HB AH
However, if we want data except the two rows, we need to bypass the point.
The principle is to first extract p1 and convert it to a list, then remove unnecessary rows (values) from the list, and the
Pandas. DataFrame
pandas. class
DataFrame
(data=none, index=none, columns=none, dtype=none, copy=false) [Source]
Two-dimensional size-mutable, potentially heterogeneous tabular data structure with labeled axes (rows and columns). Arithmetic operations align on both row and
lines for GD and HN, you can do this:
In [8]: Df[df.p1.isin ([' GD ', ' HN '])]out[8]: p1 p2 p30 GD GX FJ2 HN HB AH
But if we want data beyond these two lines, we need to get around the point.
The principle is to first remove the P1 and convert it to a list, then remove the unwanted rows (values) from the list and then use them in the Dataframeisin()
In [9]: Ex_list = List (DF.P1) in [ten]: Ex_list.remove (' GD ') in [all]: Ex_list.remove (' HN ') in
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.