R language using caret package to compare ROC curve _r language

Source: Internet
Author: User
Tags svm
Description

We've explored several algorithms before, each with its pros and cons, so when we judge a specific problem, we have to evaluate the different predictive models. To simplify this process, we use the caret package to generate and compare different models and performance. Operation

Load the corresponding package and set the training control algorithm to 10 percent cross-validation with a repeat number of 3:

Library (ROCR) library (
e1071) library (
"PROC") library (
caret)
library ("PROC") Control
= Traincontrol (method = "REPAETEDCV", number
                       = ten,
                       repeats =3,
                       classprobs = TRUE,
                       summaryfunction = Twoclasssummary)

Using GLM to train a classifier on the training data set

Glm.model = Train (churn ~.,
                  data= trainset, method
                  = "GLM",
                  metric = "ROC",
                  Trcontrol = control)

Training a classifier on training data sets using SVM

Svm.model = Train (churn ~.,
                  data= trainset, method
                  = "Svmradial",
                  metric = "ROC",
                  Trcontrol = contro L

Use the Rpart function to view the operation of Rpart on a training dataset

Rpart.model = Train (churn ~.,
                    data = trainset, method
                    = "Svmradial",
                    metric = "ROC",
                    Trcontrol = control)

Use different types of data that are already well trained to predict:

Glm.probs = Predict (Glm.model,testset[,!names (Testset)%in% C ("churn")],type = "prob")
Svm.probs = Predict ( Svm.model,testset[,!names (Testset)%in% C ("churn")],type = "prob")
Rpart.probs = Predict (rpart.model,testset[,! Names (Testset)%in% C ("churn")],type = "prob")

Generate the ROC curves for each model, drawing them in a diagram:

Glm. ROC = ROC (response = Testset[,c ("churn")],
              predictor = Glm.probs$yes,
              levels = levels (Testset[,c ("churn")])
plot (GLM. Roc,type = "S", col = "Red")

SVM. ROC = ROC (response = Testset[,c ("churn")],
              predictor = Svm.probs$yes,
              levels = levels (Testset[,c ("churn")])
plot (SVM. Roc,add = True,col = "Green")

Rpart. ROC = ROC (response = Testset[,c ("churn")],
              predictor = Rpart.probs$yes,
              levels = levels (Testset[,c ("churn")])
plot (Rpart. Roc,add = True,col = "Blue")


ROC curve Description of three classifiers

The ROC curves of different classification models are compared in the same graph, and the control parameters of the training process are 10 percent cross-validation that repeat three times, the evaluation parameters of model performance are twoclasssummary, then the classification model is established using Glm,svm,rpart and three different methods.
It can be seen from the graph that SVM's prediction results (not tuned) for training sets are the best in three classification algorithms.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.