My Naive Machine Learning notes

Notes:

This page records my naive machine learning notes.

• is learning feasible?
  • Hoeffding Inequaility:link
    • hoeffding inequality Formular ' s left side are about something bad happending. You don ' t want the thing to happen, so the can use a upper bound to constraint it. The right side of the hoeffding inequality is the restriction. As can see, it's either/both the larger the sample number (N) you had, or the smaller tolerance (Epslon) you set tha T can make the upperbound smaller. 
    • on the other hand, if your hyphothesis set size M which is large (say I nfinity), the upper bound of the hoeffding inequality needs to multiply this M (according to some math) and then the upper Bo und becomes infinity. So we need to the abstract quantity from the infinity to make it finite.  
    • the feasibility of learning is split into and both questions:
    1. Can we make sure this e_out (g) is close enought to e_in (g), where e_in (g) is the hypothesis G ' s in-sample error, e_out (g) is hypothesis g ' s out-sample error. --Hoeffding inequality answers this.
    2. Can we make e_in (g) small enough? --depends on the complexity of H--the number (M) of the hypothesis in the hypothesis set H, and the complexity of T He target function f--learning a non-linear target function is more prone to make e_in (g) bigger.
  • A model corresponds to a hypothesis set (h), a h contains a set of hypothesis (h), you choose one h and it are not called G , which (you believe) was approximate to the target function f.
    • How to pick G depends on the algorithm, hypothesis set and data to use, take perceptron for example, a G is picked util All the points is classified. There is multiple hypothesis that classify points correct and so how does you pick up the G?

My Naive machine learning notes