ecml:covariate-correlated Lasso for Feature selection (Cclasso)

Today read a ECML 14 article (such as the title), record.

Original link: http://link.springer.com/chapter/10.1007/978-3-662-44848-9_38

This article presents an lasso algorithm that explicitly considers the correlation between x and Y.

The method is simple to use μj= (1-|rho (AJ, y) |) 2 as a penalty factor of the regression coefficient βj .

Such as:

Therefore, each regression coefficient of the penalty is different, and the greater the y correlation variable, the penalty coefficient μj is smaller, the corresponding βj will not be 0.

The main contribution of this article is to give an efficient iterative algorithm, and prove its convergence

(Note that this is a convex problem, so if convergence is bound to the global optimal).

The initialization of the algorithm is the solution of the corresponding ridge regression.

The iteration is two steps, such as

Convergence of the algorithm: it is proved that the objective function is non-increasing (non-increasing), that is, L (α (t+1)) ≤l (α (t)).

The first two lemma were proved.

The first lemma defines an auxiliary function

It is also proved that G (α (t+1)) ≤g (α (t)).

A second lemma proves L (α (t+1))-L (α (T)) ≤g (α (t+1))-G (α (t)).

Combined with two lemma, it is concluded that L (α (t+1))-L (α (t)) ≤0.

Finally, the experiment was on two genetic data (Colon Cancer and leukemia datasets).

ecml:covariate-correlated Lasso for Feature selection (Cclasso)