R: Naive Bayes

Install package:

> install.packages("e1071")

Import e1071:

> library(e1071)

Find a dataset:

> data(iris)> iris    Sepal.Length Sepal.Width Petal.Length Petal.Width    Species1            5.1         3.5          1.4         0.2     setosa2            4.9         3.0          1.4         0.2     setosa3            4.7         3.2          1.3         0.2     setosa4            4.6         3.1          1.5         0.2     setosa5            5.0         3.6          1.4         0.2     setosa6            5.4         3.9          1.7         0.4     setosa7            4.6         3.4          1.4         0.3     setosa8            5.0         3.4          1.5         0.2     setosa9            4.4         2.9          1.4         0.2     setosa10           4.9         3.1          1.5         0.1     setosa11           5.4         3.7          1.5         0.2     setosa12           4.8         3.4          1.6         0.2     setosa13           4.8         3.0          1.4         0.1     setosa14           4.3         3.0          1.1         0.1     setosa15           5.8         4.0          1.2         0.2     setosa16           5.7         4.4          1.5         0.4     setosa17           5.4         3.9          1.3         0.4     setosa18           5.1         3.5          1.4         0.3     setosa19           5.7         3.8          1.7         0.3     setosa20           5.1         3.8          1.5         0.3     setosa21           5.4         3.4          1.7         0.2     setosa22           5.1         3.7          1.5         0.4     setosa23           4.6         3.6          1.0         0.2     setosa24           5.1         3.3          1.7         0.5     setosa25           4.8         3.4          1.9         0.2     setosa26           5.0         3.0          1.6         0.2     setosa27           5.0         3.4          1.6         0.4     setosa28           5.2         3.5          1.5         0.2     setosa29           5.2         3.4          1.4         0.2     setosa30           4.7         3.2          1.6         0.2     setosa31           4.8         3.1          1.6         0.2     setosa32           5.4         3.4          1.5         0.4     setosa33           5.2         4.1          1.5         0.1     setosa34           5.5         4.2          1.4         0.2     setosa35           4.9         3.1          1.5         0.2     setosa36           5.0         3.2          1.2         0.2     setosa37           5.5         3.5          1.3         0.2     setosa38           4.9         3.6          1.4         0.1     setosa39           4.4         3.0          1.3         0.2     setosa40           5.1         3.4          1.5         0.2     setosa41           5.0         3.5          1.3         0.3     setosa42           4.5         2.3          1.3         0.3     setosa43           4.4         3.2          1.3         0.2     setosa44           5.0         3.5          1.6         0.6     setosa45           5.1         3.8          1.9         0.4     setosa46           4.8         3.0          1.4         0.3     setosa47           5.1         3.8          1.6         0.2     setosa48           4.6         3.2          1.4         0.2     setosa49           5.3         3.7          1.5         0.2     setosa50           5.0         3.3          1.4         0.2     setosa51           7.0         3.2          4.7         1.4 versicolor52           6.4         3.2          4.5         1.5 versicolor53           6.9         3.1          4.9         1.5 versicolor54           5.5         2.3          4.0         1.3 versicolor55           6.5         2.8          4.6         1.5 versicolor56           5.7         2.8          4.5         1.3 versicolor57           6.3         3.3          4.7         1.6 versicolor58           4.9         2.4          3.3         1.0 versicolor59           6.6         2.9          4.6         1.3 versicolor60           5.2         2.7          3.9         1.4 versicolor61           5.0         2.0          3.5         1.0 versicolor62           5.9         3.0          4.2         1.5 versicolor63           6.0         2.2          4.0         1.0 versicolor64           6.1         2.9          4.7         1.4 versicolor65           5.6         2.9          3.6         1.3 versicolor66           6.7         3.1          4.4         1.4 versicolor67           5.6         3.0          4.5         1.5 versicolor68           5.8         2.7          4.1         1.0 versicolor69           6.2         2.2          4.5         1.5 versicolor70           5.6         2.5          3.9         1.1 versicolor71           5.9         3.2          4.8         1.8 versicolor72           6.1         2.8          4.0         1.3 versicolor73           6.3         2.5          4.9         1.5 versicolor74           6.1         2.8          4.7         1.2 versicolor75           6.4         2.9          4.3         1.3 versicolor76           6.6         3.0          4.4         1.4 versicolor77           6.8         2.8          4.8         1.4 versicolor78           6.7         3.0          5.0         1.7 versicolor79           6.0         2.9          4.5         1.5 versicolor80           5.7         2.6          3.5         1.0 versicolor81           5.5         2.4          3.8         1.1 versicolor82           5.5         2.4          3.7         1.0 versicolor83           5.8         2.7          3.9         1.2 versicolor84           6.0         2.7          5.1         1.6 versicolor85           5.4         3.0          4.5         1.5 versicolor86           6.0         3.4          4.5         1.6 versicolor87           6.7         3.1          4.7         1.5 versicolor88           6.3         2.3          4.4         1.3 versicolor89           5.6         3.0          4.1         1.3 versicolor90           5.5         2.5          4.0         1.3 versicolor91           5.5         2.6          4.4         1.2 versicolor92           6.1         3.0          4.6         1.4 versicolor93           5.8         2.6          4.0         1.2 versicolor94           5.0         2.3          3.3         1.0 versicolor95           5.6         2.7          4.2         1.3 versicolor96           5.7         3.0          4.2         1.2 versicolor97           5.7         2.9          4.2         1.3 versicolor98           6.2         2.9          4.3         1.3 versicolor99           5.1         2.5          3.0         1.1 versicolor100          5.7         2.8          4.1         1.3 versicolor101          6.3         3.3          6.0         2.5  virginica102          5.8         2.7          5.1         1.9  virginica103          7.1         3.0          5.9         2.1  virginica104          6.3         2.9          5.6         1.8  virginica105          6.5         3.0          5.8         2.2  virginica106          7.6         3.0          6.6         2.1  virginica107          4.9         2.5          4.5         1.7  virginica108          7.3         2.9          6.3         1.8  virginica109          6.7         2.5          5.8         1.8  virginica110          7.2         3.6          6.1         2.5  virginica111          6.5         3.2          5.1         2.0  virginica112          6.4         2.7          5.3         1.9  virginica113          6.8         3.0          5.5         2.1  virginica114          5.7         2.5          5.0         2.0  virginica115          5.8         2.8          5.1         2.4  virginica116          6.4         3.2          5.3         2.3  virginica117          6.5         3.0          5.5         1.8  virginica118          7.7         3.8          6.7         2.2  virginica119          7.7         2.6          6.9         2.3  virginica120          6.0         2.2          5.0         1.5  virginica121          6.9         3.2          5.7         2.3  virginica122          5.6         2.8          4.9         2.0  virginica123          7.7         2.8          6.7         2.0  virginica124          6.3         2.7          4.9         1.8  virginica125          6.7         3.3          5.7         2.1  virginica126          7.2         3.2          6.0         1.8  virginica127          6.2         2.8          4.8         1.8  virginica128          6.1         3.0          4.9         1.8  virginica129          6.4         2.8          5.6         2.1  virginica130          7.2         3.0          5.8         1.6  virginica131          7.4         2.8          6.1         1.9  virginica132          7.9         3.8          6.4         2.0  virginica133          6.4         2.8          5.6         2.2  virginica134          6.3         2.8          5.1         1.5  virginica135          6.1         2.6          5.6         1.4  virginica136          7.7         3.0          6.1         2.3  virginica137          6.3         3.4          5.6         2.4  virginica138          6.4         3.1          5.5         1.8  virginica139          6.0         3.0          4.8         1.8  virginica140          6.9         3.1          5.4         2.1  virginica141          6.7         3.1          5.6         2.4  virginica142          6.9         3.1          5.1         2.3  virginica143          5.8         2.7          5.1         1.9  virginica144          6.8         3.2          5.9         2.3  virginica145          6.7         3.3          5.7         2.5  virginica146          6.7         3.0          5.2         2.3  virginica147          6.3         2.5          5.0         1.9  virginica148          6.5         3.0          5.2         2.0  virginica149          6.2         3.4          5.4         2.3  virginica150          5.9         3.0          5.1         1.8  virginica

Sepal refers to the flower volume and petal refers to the flower volume ". Obviously, the first four columns are the characteristics of the flower volume and the petals, and the fifth column represents the corresponding classification. We can use this dataset for Bayesian training.

Let's take a look at the result of the summary Dataset:

> summary(iris)  Sepal.Length    Sepal.Width     Petal.Length    Petal.Width          Species   Min.   :4.300   Min.   :2.000   Min.   :1.000   Min.   :0.100   setosa    :50   1st Qu.:5.100   1st Qu.:2.800   1st Qu.:1.600   1st Qu.:0.300   versicolor:50   Median :5.800   Median :3.000   Median :4.350   Median :1.300   virginica :50   Mean   :5.843   Mean   :3.057   Mean   :3.758   Mean   :1.199                   3rd Qu.:6.400   3rd Qu.:3.300   3rd Qu.:5.100   3rd Qu.:1.800                   Max.   :7.900   Max.   :4.400   Max.   :6.900   Max.   :2.500

Train and view the training results:

> classifier<-naiveBayes(iris[,1:4], iris[,5]) > classifierNaive Bayes Classifier for Discrete PredictorsCall:naiveBayes.default(x = iris[, 1:4], y = iris[, 5])A-priori probabilities:iris[, 5]    setosa versicolor  virginica  0.3333333  0.3333333  0.3333333 Conditional probabilities:            Sepal.Lengthiris[, 5]     [,1]      [,2]  setosa     5.006 0.3524897  versicolor 5.936 0.5161711  virginica  6.588 0.6358796            Sepal.Widthiris[, 5]     [,1]      [,2]  setosa     3.428 0.3790644  versicolor 2.770 0.3137983  virginica  2.974 0.3224966            Petal.Lengthiris[, 5]     [,1]      [,2]  setosa     1.462 0.1736640  versicolor 4.260 0.4699110  virginica  5.552 0.5518947            Petal.Widthiris[, 5]     [,1]      [,2]  setosa     0.246 0.1053856  versicolor 1.326 0.1977527  virginica  2.026 0.2746501> classifier$aprioriiris[, 5]    setosa versicolor  virginica         50         50         50 > classifier$tables$Sepal.Length            Sepal.Lengthiris[, 5]     [,1]      [,2]  setosa     5.006 0.3524897  versicolor 5.936 0.5161711  virginica  6.588 0.6358796$Sepal.Width            Sepal.Widthiris[, 5]     [,1]      [,2]  setosa     3.428 0.3790644  versicolor 2.770 0.3137983  virginica  2.974 0.3224966$Petal.Length            Petal.Lengthiris[, 5]     [,1]      [,2]  setosa     1.462 0.1736640  versicolor 4.260 0.4699110  virginica  5.552 0.5518947$Petal.Width            Petal.Widthiris[, 5]     [,1]      [,2]  setosa     0.246 0.1053856  versicolor 1.326 0.1977527  virginica  2.026 0.2746501

In classifier:

A-priori probabilities:iris[, 5]    setosa versicolor  virginica  0.3333333  0.3333333  0.3333333

A good understanding is the prior probability of a category.
And:

$Petal.Width            Petal.Widthiris[, 5]     [,1]      [,2]  setosa     0.246 0.1053856  versicolor 1.326 0.1977527  virginica  2.026 0.2746501

It is the conditional probability of the feature petal. Width. In this Bayesian implementation, the feature is numeric data (and there is also a fractional part). Here we assume that the probability density conforms to the Gaussian distribution. For example, for the feature petal. width, the probability of being setosa complies with the Gaussian distribution where the mean is 0.246 and the standard variance is 0.1053856.

Prediction:
Predict the first data in the iris Dataset:

> predict(classifier, iris[1, -5])[1] setosaLevels: setosa versicolor virginica

Construct a new data and make predictions:

> new_data = data.frame(Sepal.Length=7, Sepal.Width=3, Petal.Length=6, Petal.Width=2)> predict(classifier, new_data)[1] virginicaLevels: setosa versicolor virginica

Next, let's take a look at how this database handles nominal features: the data is as follows:

> model = c("H", "H", "H", "H", "T", "T", "T", "T")> place = c("B", "B", "N", "N", "B", "B", "N", "N")> repairs = c("Y", "N", "Y", "N", "Y", "N", "Y", "N")> dataset  model place repairs1     H     B       Y2     H     B       N3     H     N       Y4     H     N       N5     T     B       Y6     T     B       N7     T     N       Y8     T     N       N

Bayes:

> classifier<-naiveBayes(dataset[,1:2], dataset[,3]) > classifierNaive Bayes Classifier for Discrete PredictorsCall:naiveBayes.default(x = dataset[, 1:2], y = dataset[, 3])A-priori probabilities:dataset[, 3]  N   Y 0.5 0.5 Conditional probabilities:            modeldataset[, 3]   H   T           N 0.5 0.5           Y 0.5 0.5            placedataset[, 3]   B   N           N 0.5 0.5           Y 0.5 0.5

Let's make a prediction:

> new_data = data.frame(model="H", place="B")> predict(classifier, new_data)[1] NLevels: N Y

Perfect!

Refer:

Http://www-users.cs.york.ac.uk /~ JC/teaching/Arin/r_practical/
Http://pythonhosted.org//NaiveBayes/

R: Naive Bayes