[FML study Note 2] Pac Learning Model

For a concept class C, if an algorithm a and a polynomial poly (...,...) exist (.,.,.,.), for any distribution of ε> 0, Delta> 0, and x, D and any target concept C, when sample size m> = poly (1/ε, 1/delta, n, size (c), inequality:

The concept class C is Pac-learnable.

(1). N: the dimension of X.

(2). Size (c ):

O (n): an upper bound on the cost of the computational representation of any element x? X. For an array of X whose dimension is N, the time complexity is O (n ).

Size (n): the maximal cost of the computation representation of C? C. The same as O (n) can be seen as the size of C.

If the sample complexity of algorithm A is about 1/delta, 1/ε, N, size (c) is polynomial, then C is efficiently Pac-learnable. When such a existence occurs, the algorithm A is called concept class C's PAC learning algorithm.

Note the following for Pac-learnability:

1. Pac is true for any distribution of x D.

2. Although the distribution is not limited, both training sample and testing sample must generate D with the same distribution.

3. What PAC solves is a concept class C's learning problem and is not targeted at a particular concept C (usually target C is unknown but C is known ).

Example:

, X = R ^ 2, concept class C is a matrix of all edges on R ^ 2 parallel to the coordinate axis, the goal is to obtain a concept C so that the point inside the matrix is 1 (blue point), and the point outside the matrix is 2 (red point ). A very simple algorithm is designed for this problem. The algorithm returns the smallest matrix containing all vertices labeled as 1.

Assuming that the matrix returned by the algorithm is R', we create four rectangles (R1, R2, R3, and R4) along the four edges of R and make their probability mass equal to ε/4, if we want to make R (R'), that is, R's gerneralization error greater than ε, then R' cannot overlap with the four matrices, otherwise R-R '= R (R ') must be less than ε. That is, R' must at least have no intersection with a matrix Ri.

So there are:

(1): Because RS and RI have no intersection, it is equivalent to no blue point falling into Ri, and Ri is in target concept R, so no red point falls into Ri, in combination, that is, if no vertex falls into the RI, the probability that each vertex falls into the Ri is ε/4, then for each Ri, the probability that M points do not fall into it is (1-ε/4) ^ m, with four Ri.

(2): Expand and scale Taylor of exp ^ (-x.

As a result:

That is, when the sample size M satisfies the above inequality, it can ensure that the concept space is Pac learnable. Here, 1/delta is ln complexity, and 1/ε is linear complexity. In general, it is efficiently Pac-learnable.

[FML study Note 2] Pac Learning Model