Learning notes TF042: TF. Learn, distributed Estimator, deep learning Estimator, tf042estimator

Learning notes TF042: TF. Learn, distributed Estimator, deep learning Estimator, tf042estimator

TF. Learn, an important module of TensorFlow, various types of deep learning and popular machine learning algorithms. TensorFlow official Scikit Flow project migration, launched by Google employee Illia Polosukhin and Tang Yuan. Scikit-learn code style helps data science practitioners better and more quickly adapt to accept TensorFlow code. Contains many TensorFlow code and design patterns, allowing you to build machine learning models faster and implement applications. Avoid code duplication and focus on building more precise models. And other contrib modules.

Distributed Estimator. Estimator, a variety of machine learning and deep learning classes. You can directly use these high-level classes, or quickly create subclasses based on actual application requirements. In the graph_actions module, Estimator is implemented and concentrated in the complex distributed logic of the Training and Evaluation Models. It does not require the detailed implementation and logic of complex Supervisor and Coordinator distributed training.

Estimator accepts the custom model. The function answer name (input parameter field> return field): (1) (features, targets)-> (predictions, loss, train_op ). (2) (features, targets, mode)-> (predictions, loss, train_op ). (3) (features, targets, mode, params)-> (predictions, loss, train_op ).

The custom model accepts two parameters: features and targets. Features, data features. Target or category ID of each row of targets data features. Tf. one_hot: One-hot Encoding ). Layers. stack overlays multiple layers of layers. fully_connected fully connects to the deep neural network. Each layer has 10, 20, and 10 hidden nodes respectively. Different layers are converted and trained to obtain new data features. Models. logistic_regression_zero_init adds a layer. The logical regression model with 0 initial parameter values is used to obtain the predicted values and loss values. Contrib. layers. optimize_loss function optimization loss value. Select different optimization functions and learning rates as needed. Optimize_loss Training Operator (Training Operator). Each Training iteration optimizes model parameters and determines the model development direction. Returns the predicted value, predicted probability, or one of them.

Iris data classification. Scikit-learn datasets introduces data and cross_validation data is divided into training and evaluation. Put learn. Estimator, Scikit-learn style fit, and predict functions in my_model. Quickly define your own model functions and directly use the various functions of Estimator to directly perform distributed model training without worrying about implementation details.

Mode defines functions. Common modes include training, evaluation, and prediction, which can be found in ModeKeys. Add conditional statements to implement complex logic. Params adjustment parameters. The fit function can provide more parameters.

Create machine learning Estimator. BaseEstimator is the most abstract and basic implementation of the TensorFlow Model Training and Evaluation Class. Fit () model training, partial_fit () online training, evaluate () evaluation model, and predict () use model to predict new data. Graph_actions complex logic model training and prediction. SuperVisor, Coordinator, QueueRunner, distributed training and prediction. The learn. DataFeeder and learn. DataFrame classes automatically identify, process, and iterate data of different types. Estimators. tensor_signature checks the compatibility of data (Sparse Tensor), Making Data Reading more convenient and stable. BaseEstimator initializes learn. monitors and model storage. Learn. monitors monitoring model training.

BaseEstimator, _ get_train_ops (), _ get_eval_ops (), and _ get_predict_ops () sub-classes are implemented.

Estimator, _ get_train_ops () accepts features and targets parameters. The user-defined model function returns Operation and Tensor Tuple loss, and optimizes model parameters in each training iteration. The unsupervised learning Model Estimator ignores targets.

_ Get_eval_ops (), BaseEstimator sub-class custom metrics evaluate each model training iteration. Contrib. metrics. The custom metrics function returns a Tensor object. The Python word yellow indicates evaluating Operation, which is used in each iteration.

The custom model predicts new data and calculates the loss value. ModeKeys EVAL indicates that the function is only used for evaluation. In the contrib. metrics module, streaming_mean calculates the average stream of the loss. The average value is calculated before the loss value of this iteration is added and then the average value is calculated.

_ Get_predict_ops () to implement custom prediction. Further process the prediction results. Simple Prediction Result of predicted probability conversion, and probability smoothing (Smooting ). The function returns the Tensor object Python dictionary to represent the predicted Operation. Estimator predict () function, Estimator distributed function. Unsupervised model, similar to the Sckkit-learn transform () function.

LogisticRegressor and Estimator provide the vast majority of implementations. LogisticRegressor only needs to provide its own metrics (AUC, accuracy, precision, and recall to handle binary classification issues ), you can quickly customize the binary classification Estimator in the basic write subclass of LogiticRegressor without worrying about other logic implementations.

TF. Learn random forest model TensorForestEstimator many details are implemented in contrib. tensor_forest. Only use and expose higher order components to TensorForestEstimator. Hyperparameters are uploaded to the contrib. tensor_forest.ForestHParams constructor. The constructor params. fill () is used to construct a random forest TensorFlow graph, tensor_forest.RandomForestGraphs.

The implementation is complex and requires high efficiency. The details use C ++ to implement a separate Kernel. _ Get_predict_ops () function. In tensor_forest, C ++ implements the data_ops.ParseDataTensorOrDict () function to detect and convert and read data to support data types. The RandomForestGraphs inference_graph function predicts Operation.

The _ get_train_ops () and _ get_eval_ops () functions call the RandomForestGraphs. Transform () and RandomForestGraphs. onference_graph () functions. The transform and transform functions detect and convert features and targets to compatible data.

Adjust RunConfig runtime parameters. RunConfig, TF. Learn class, to adjust the runtime parameters. Num_cores select the number of cores used, num_ps_replicas adjust the number of parameter servers, and gpu_memory_fraction control the percentage of GPU storage used.

The RunConfig master parameter specifies the address of the master server of the training model, and the task sets the task ID. Each task ID controls a training model parameter server replica.

Initialize a RunConfig object and upload it to Estimator. The default value of the RunConfig parameter is to run a simple model locally. Only one task ID is used. The 80% GPU storage parameter is passed to Estimator. The runtime parameters are automatically used, so you do not have to worry about the details of ConfigProto and GPUOptions. Quickly change parameters for Distributed Model Training and parameter server usage.

Experiment is easy to use. It is used to create model experiments. It provides all the information required for modeling, such as Estimator, training data, evaluation data, evaluation indicators, supervisor, and evaluation frequency. You can choose to run it locally, and use RunConfig with the distributed test. LearnRunner to facilitate experiments.

Tf. app. flags defines the number of parameters that can be input from the command line, Data, model, output file path, training, and evaluation steps. Schedule test type. Local_run () Local test. Run_std_server () Standard Server test. Master_grpc_url mainly serves GRPC TensorFlow servers. Num_parameter_servers parameter number of servers.

Establish the Experiment object function, set up RunConfig in FLAGS, Estimator in machine learning model, and set up a classifier (DNNLinearCombinedClassifier ). Input_train_fn, input_test_fn, data source, training, evaluation data.

The create_experiment_fn () function is passed into LearnRunner for different types of tests, local and server tests. Test results are stored in different paths.

Deep Learning Estimator. TF. Learn contains an easy-to-use deep neural network Estimator. Classification problem DNNClassifier. _ Input_fn () is used to create data, and the layers module establishes feature columns.

Feature columns, number of hidden neural units at each layer, and classification data are transmitted to DNNClassifier to quickly establish a deep neural network model.

Model Training and Evaluation of fit () and evaluate () methods.

Each row of data has a weight. DNNClassfier: Specifies a column as the weight column and is automatically assigned to the training process. Four rows of data with different weights in each row. Add the weight column and Feature Column to features.

DNNClassifier indicates the weight column name w and the feature column name x (x conversion feature column ).

Input the custom metrics equation _ my_metric_op () and operate predictions and targets for metrics calculation. Only binary classification is considered. The second column of tf. slice () Cut predictions is used as the predicted value.

Tf. slice () input matrix input, cut Start Element begin, cut Tensor shape size, size [I] represents the I dimension want to cut the matrix shape.

You can perform any predictions or targets operation to calculate metrics. evaluate () is used to input the metrics function, and TF. Learn is used to evaluate the Model Based on metrics.

Evaluate () can provide multiple metrics, _ my_metric_op custom, tr. contrib comes.

Optimizer provides custom functions to define its own optimization function, including the Exponential decline learning rate.

Tf. contrib. framework. get_or_create_global_step. Tf. train. exponential_decay () degrades the learning rate index to avoid gradient explosion.

The breadth and depth model is DNNLinearCombinedClassifier. Google is widely used in a variety of machine learning applications. Deep Neural Networks and logistic regression are combined. Different features are combined in two different ways to better reflect the application significance and more effective recommendation results. Similar to the Kaggle competition Ensemble.

For more parameters, select different feature columns from DNNClassifier and LinearClassifier.

Gender, education, relationship, and workclass are converted to FeatureColumn. It can be divided into wide_columns and deep_columns. Wide_columns is used in LinearClassifier, and deep_columns is used in DNNClassifier. Input DNNLinearCombinedClassifier to establish the breadth and depth model. Linear and deep neural network features.

References:
TensorFlow practice

Welcome to paid consultation (150 RMB per hour), My: qingxingfengzi