VC Dimension-Measure the complexity of models and samples

(1) Define VC Dimension:

The upper limit of the number of dichotomies is the growth function, the upper limit of the growth function is the boundary function:

so VC Bound can be changed to write:

below we define VC Dimension:

for an alternate set of functionsH,VC Dimensionis what it canShatterthe maximum number of dataN. VC Dimension = minimum break point-1. So inVC Boundin which(2N) ^ (k-1)can be replaced by(2N) ^ (VC Dimension). VC Dimensionand Learning AlgorithmsA, the input distributionP, the target functionFis irrelevant.

(2) the VC Dimension of PLA   

the 1D PLA has a maximum of Shatter2 points, so VC Dimension = 2;

2d pla Span style= "font-family: Arial" > up to shatter3 Span style= "font-family: Arial" > dot, so VC Dimension = 3                                                           

guess DD of PLA,VC Dimension will be equal to d+1? just prove dvc≥d+1 and dvc≤d+1

• prove VC dimension≥d+1 , just prove H can be Shatter some d+1 an input.

constructs a set of d+1 inputs:

Figure 1

The first column of gray 1 is to increase the operation of 1 dimensions for each input , this is a square of the d+1 Dimension, the diagonal is all 1, So the matrix is reversible. for any kind of output, we can always find an alternate function that makes

Figure 2

That this group of input all dichotomies is exhausted, so VC dimension≥d+1 proof

• prove Vcdimension≤d+1, the only card H cannot shatter any d+2 input

constructs a set of 4 inputs in 2D:

Figure 3

so x4 = x3 + x2-x1

VC Dimension-Measure the complexity of models and samples