Pig's machine learning Note (13) Bayesian network

Bayesian Networks

Cherry Blossom Pig

Summary

This article is for the July algorithm (julyedu.com) Lunar machine learning 13th time online note. Bayesian Network, also known as the Reliability network, is the extension of Bayes method, and is one of the most effective theoretical models in the field of uncertain knowledge expression and inference. Bayesian networks are suitable for the expression and analysis of uncertainties and probabilistic events, and are applied to decisions that conditionally depend on a variety of control factors, and can be inferred from incomplete, imprecise, or uncertain knowledge or information. This lesson begins with the naive Bayesian model, and describes in detail the significance of Bayesian networks, the construction scheme and other derivative algorithms.

Introduction

Bayesian Network is one of the most classical algorithms in machine learning, it can estimate uncertain knowledge according to the known conditions, and the application scope is very wide. Bayesian network constructs a forward-and-loop graph based on Bayesian equations, and we can construct a graph of Bayesian networks to predict future information clearly based on existing information.

This course starts with the naïve Bayesian network, the construction of Bayesian networks, independent conditions in Bayesian networks, and we are familiar with Markov models, Markov blankets and so on. The theoretical basis is not strong enough to understand the content.

Pre-knowledge

Maximum entropy model, probability statistics

One, naive Bayes

1. Naive Bayes hypothesis ( differences from Bayesian networks )

The probability of a feature appearing, independent of other characteristics (characteristic independence), is actually: for a given classification, the characteristics are independent

Each feature is equally important (characteristic equalization)

Example: Text categorization questions:

Sample:10000 Email , each message marked as spam or not spam

Classification target: Given the 10001 message, determine if it is spam or not Junk mail

method: naive Bayesian

Category C: Junk e-mail C1, non-spam C2

Glossary, two methods of establishing:

1, the use of ready-made word dictionary,2, all the messages appear in the statistics of the words, get the dictionary.

Remember the number of words N

map each message m to a vector of dimension n xN

If the word wi appears in the mail m , then xi=1, otherwise,xi=0. That is, the vectorization of the message:m--> (x1,x2 ... XN) o

Bayesian formula: P (c|x) =p (x|c) *p (c)/p (x)

P (c1|x) =p (X|C1) *p (C1)/p (x)

P (c2|x) =p (X|C2) *p (C2)/p (x)

Notice here that x is a vector

(c|x) =p (x|c) *p (c)/p (x)

P (x|c) =p (x1,x2 ... xn|c) =p (x1|c) *p (x2|c) ... P (xn|c)

Independent hypothesis of characteristic condition

P (x) =p (x1,x2 ... XN) =p (x1) *p (x2) ... P (XN)

Feature Independent hypothesis

To bring into the formula: P (c|x) =p (x|c) *p (c)/p (x)

o The meaning of the items to the right of the equation:

NP (XI|CJ): In CJ ( This topic,CJ is either spam 1, or not spam 2) under the premise that the first I a word XI probability of occurrence

NP (xi): The probability that Word XI appears inall samples

NP (CJ): In all samples, the probability of CJ appearing in the message category

Laplace smoothing (prevents the case of 0 )

P (X1|C1) refers to the probability that the word x1 appear in the spam C1 category . (x1 is a word in the message to be examined)

Defining symbols:

N1: The number of times the word x1 appears inall spam messages . If x1 has not appeared, then n1=0.

NN: The total number of words that have occurred in all documents belonging to the C1 class.

O Get the formula:

o Laplace Smoothing:

n wheren is the number of all words. Correction of the denominator is to guarantee the probability and the 1

Similarly, treat P (x1) with the same smoothing scheme

Second, Bayesian network

A Bayesian network is formed based on whether the random variables involved in a research system are plotted independently in a direction graph according to the conditions.

Bayesian network, also known as the directed acyclic graph model (DAG), is a probabilistic graph model, which investigates a set of random variables {x1,x2) according to the topological structure of probabilistic graphs . Xn} and its n Group conditional probability distribution (CPD) properties.

In general, the nodes in a Bayesian network's forward-free graph represent random variables, which can be observable variables, or hidden variables, unknown parameters, and so on. The arrows that connect two nodes represent that the two random variables are causal ( or non-conditional ). If the two nodes are joined together by a single arrow, one of the nodes is " parents" and the other is "Fruit (children)", the two nodes will produce a conditional probability value.

Each node, given its direct precursor, is independent of its non-successor.

The formal definition of Bayesian networks:

BN (G,Θ)

G: A Direction-free graph

node of G: Random variable

The side of G: a dependent dependency between nodes

nθ: The conditional probabilityof the parameter set N node X for all conditional probability distributions:P (X|parent (X))

Second, Markov model

Bayesian networks simplify the formation of a chain model,ai+1 only with Ai , and A1,... , Ai-1 irrelevant

Hidden Markov Model:

Third, the Bayesian network to determine the conditions of independence:

Tail-to-tail

Head-to-tail

Head-to-head

(C blocking )

Four, the construction of Bayesian network

The local test results of the d-separation of each variable are computed sequentially , and the Bayesian networks are synthesized for each node.

Algorithm process:

A reasonable order for selecting variables:x1,x2,... Xn

For I=1 to n

o Add Xi nodes to the network

In x1,x2,... Parents of Xi in Xi-1 , making:

o This method of construction clearly guarantees a global semantic requirement: (

Problem, if a mixed (discrete + continuous) network is encountered, and the signal function is discretized

Pig's machine learning Note (13) Bayesian network