無監督學習特徵–稀疏編碼、深度學習、ICA部分代表文獻——-之一

轉載http://blog.csdn.net/zhoutongchi/article/details/8191991      

  學習映射函數及在行為識別/映像分類中應用的文獻（模型與非模型之間存在關聯，演算法相互採用，沒有明確的區分，含仿生學文獻）

% 研究重點放到ICA模型及深度學習兼顧稀疏編碼

1）稀疏編碼（稀疏編碼、自動編碼、遞迴編碼）：

[1] B. Olshausen and D. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 1996.

[2] H. Lee, A. Battle, R. Raina, and A. Y. Ng. Efficient sparse coding algorithms. In NIPS, 2007.

[3] B. A. Olshausen. Sparse coding of time-varying natural images. In ICA, 2000.

[4] Dean, T., Corrado, G., Washington, R.: Recursive sparse spatiotemporal coding.In: Proc. IEEE Int. Workshop on Mult. Inf. Proc. and Retr. (2009).

[5] J. Yang, K. Yu, Y. Gong, and T. Huang. Linear spatial pyramid matching using sparse coding for image

classification. In CVPR, 2009.

[6] S. Wang, L. Zhang, Y. Liang and
Q. Pan.Semi-Coupled Dictionary Learning with Applications to Image Super-Resolution and Photo-Sketch Image Synthesis. in CVPR 2012.

[7] Yan Zhu, Xu Zhao,Yun Fu,Yuncai Liu. Sparse Coding on Local Spatial-temporal Volumes for Human action Recognition.ACCV2010,Part II,LNCS 6493.(上海交大，採用3DHOG特徵描述，3DSift稀疏編碼未注意)。

2）ICA（ISA）模型：

[1] A. Hyvarinen, J. Hurri, and P. Hoyer. Natural Image Statistics. Springer, 2009.

[2]Alireza Fathi and Greg Mori. Action Recognition by Learning Mid-level Motion Features. IEEE,2008,978-1-4244-2243.

[3] A. Hyvarinen and P. Hoyer. Emergence of phase- and shift-invariant features by decomposition of natural images into independent feature subspaces.
Neu. Comp., 2000.

[4] A. Coates, H. Lee, and A. Y. Ng. An analysis of single-layer networks in unsupervised feature learning.

In AISTATS 14, 2011.（該篇採用採用的特徵，基於BOW方法不需要檢測。形成BOW時採用映像塊相似聚類，跟據離BOW距離將映像塊特徵非線性判決，之後將正副映像以一種稀疏形式表示）

[5] Q. V. Le, W. Zou, S. Y. Yeung, and A. Y. Ng. Learning hierarchical spatio-temporal features for action

recognition with independent subspace analysis. In CVPR, 2011.

[6] Q. V. Le, J. Ngiam, Z. Chen, D. Chia, P. W. Koh, and A. Y. Ng. Tiled convolutional neural networks. In

NIPS, 2010.

[7] M. S. Lewicki and T. J. Sejnowski. Learning overcomplete representations. Neural Computation, 2000.

[8] L. Ma and L. Zhang. Overcomplete topographic independent component analysis. Elsevier, 2008.

[9] A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report, U. Toronto, 2009.

%非分類識別文獻，引入copula估計子空間，新特徵組合

[10]Nicolas Brunel, Wojciech Pieczynski,Stephane Derrode.Copulas in vectorial hidden markov chains for multicomponent images segmentation.ICASSP’05,Philadelphia,USA,March
19-23,2005.（非識別分類文獻，但是涉及到一種演算法，對估計子空間很有用，可以引入ICA模型。）

[11] Xiaomei Qu. Feature Extraction by Combining Independent Subspaces Analysis and Copula Techniques. International Conference on system Science
and Engineering,2012.

[12] Pietro Berkes, Frank Wood and Jonathan Pillow. Characterizing neural dependencies with copula models. In NIPS, 2008.

[13] Y-Lan Boureau, Jean Ponce, Yann LeCun. A theoretical Analysis of Feature Pooling in Visual Recognition. In Proceedings of the 27’th International
Conference on machine Learning, Haifa, Israel,2010.（介紹多樣池及概念，可以形成稀疏表示及產生魯棒性特徵）

3）深度學習（與ICA、RBM關聯性強，屬於多層學習）：

[1] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle.Greedy layerwise training of deep networks. In NIPS, 2006.

[2] Alessio Plebe. A model of the response of visual area V2 to combinations of orientations. Network: Computation in Neural Systems, September 2012;
23(3): 105–122.(涉及到類比人類大腦皮層感知（v1、v2、v3、v4、v5），此類文獻多，主要以猴子貓動物實驗)

[3] G. Hinton, S. Osindero, and Y. Teh. A fast learning algorithms for deep belief nets. Neu. Comp., 2006

[4] H. Lee, R. Grosse, R. Ranganath, and A. Ng. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations.
In ICML, 2009.

[5]Yann Lecun, Koray Kavukcuoglu, and Clement Farabet. Convolutional Networks and Applications in Vision. In Proc. International Symposium on Circuits
and Systems (ISCAS'10), 2010.

[6] Pierre Sermanet, Soumith Chintala and Yann LeCun. Convolutional Neural Networks Applied to House Numbers Digit Classification. Computer Vision
and Pattern Recognition,2012.

[7] Quoc V. Le. Marc’Aurelio Ranzato. Rajat Monga. Matthieu Devin. Kai Chen. Greg S. Corrado. Jeff Dean.
Andrew Y. Ng. Building High-level Features Using Large Scale Unsupervised Learning. the 29’th International Conference on Machine Learning, Edinburgh, Scotland, UK, 2012.

[8] A. Hyvarinen and P. Hoyer. Topographic independent component analysis as a model of v1 organization and receptive fields. Neu. Comp., 2001

[9]Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle. Greedy layerwise training of deep networks. In

NIPS, 2007.

[10] Q. V. Le, J. Ngiam, A. Coates, A. Lahiri, B. Prochnow, and A. Y. Ng. On optimization methods for deep learning. In ICML, 2011.

[11] H. Lee, C. Ekanadham, and A. Y. Ng. Sparse deep belief net model for visual area V2. In NIPS, 2008.

[12] G. E. Hinton, S. Osindero, and Y. W. Teh. A fast learning algorithm for deep belief nets. Neural Computation, 2006.

[13] Jarrett, K., Kavukcuoglu, K., Ranzato, M., LeCun, Y.: What is the best multistage architecture for object recognition? In: ICCV. (2009) 2146-2153.

[14] Lee, H., Ekanadham, C., Ng., A.: Sparse deep belief net model for visual area V2.In: NIPS. (2008) 873-880.

[15] Bo Chen .Deep Learning of Invariant Spatio-Temporal Feature from Video.[D].2010.

[16] Jiquan Ngiam, Zhenghao Chen, Pang Wei Koh,Andrew Y.Ng.Learning Deep Energy Models.in Proceedings of the 28’th international Conference on Machine
Learning,Bellevue,WA,USA,2011.

%以下（CRBM、SF）這些模型參考，不做重點研究，可借鑒演算法。

4）CRBM(文獻多，有博士論文)：

[1] G. Hinton. A practical guide to training restricted boltzmann machines. Technical report, U. of Toronto,

2010．

[2] G. Taylor, R. Fergus, Y. Lecun, and C. Bregler. Convolutional learning of spatio-temporal features. In ECCV, 2010.

[3] Norouzi, M., Ranjbar, M., Mori, G.: Stacks of convolutional restricted Boltzmann machines for shift-invariant feature learning. In: CVPR. (2009).

[4] Memisevic, R., Hinton, G.: Learning to represent spatial transformations with factored higher-order Boltzmann machines. Neural Comput 2010.

 

5）Slow
Feature（慢特徵學習分析（德國），代表文獻）

這種新方法以鄰幀映像為基礎研究，是一種新思路。

[1] P. Berkes and L. Wiskott. Slow feature analysis yields arich repertoire of complex cell properties. Journal of Vision,2005