Python machine learning-predictive analytics Core algorithm: Understanding data

See original book 2.1-2.2 section

The new dataset is like a wrapped gift, filled with promise and hope!

But until you open it, it remains mysterious!

I. Structure and terminology of the underlying problem, characteristics of the machine learning data set

Typically, rows represent instances, columns represent attribute characteristics

property, the data used in the instance for prediction
Other Name: Predictive factor feature independent variable input

tags, data that needs to be predicted
Other Name: Result target Dependent variable response

2.1.1 The different types of attributes and labels determines the selection of the model
Numeric variable category variable/factor variable

Penalty regression algorithm can only deal with numerical variables: SVM kernel method k nearest Neighbor
Conversions: value variables, category variables

When the label is numeric, it's called a regression problem.
When a label is a category, it is called a classification problem.
Conversion: Regression question 170 210 classification problem >200?

Classification problems may also be simpler than regression problems

2.1.2 Considerations for New datasets
Things to check for:
Number of rows, columns
Number of category variables, range of values for categories
The missing value
Statistical characteristics of attributes and labels

Handling Missing values:
1. There is a large amount of data, directly discard missing values
2. Data is more expensive, difficult to obtain, fill missing value
Lost value interpolation: The simplest way to replace missing values with the average value of all this item per line

2.2 Classification problems

Example: Using sonar to find an unexploded mine

(1) Determine the size of the data set

Impact of data size:
1. The time required for training can be roughly judged

A method of punishing linear regression integration
1min a few minutes
10000 * 10000 3-4 hour 12-24 hour

2. If the number of columns in a dataset is much larger than the number of rows, the method of punishing linear regression is likely to get the best predictions

(2) Determining the characteristics of each attribute
Which columns are numeric, and which are category-type

(3) Get stats for attributes
Descriptive statistical information of numeric type
Number distribution of category-specific categories

Python machine learning-predictive analytics Core algorithm: Understanding data