Weibo collection Resource Statistics (machine learning chapter) (i)

Have seen a lot of good information on Weibo, but have never had time to look, can only be forwarded or collection, the last semester, must first put these inventory a little clear.

——————————————————————————————————————————

1.0 convolutional Neural Network cxxnet

@ Chen Tian Strange and @antinucleon complete convolutional Neural Network Code cxxnet (core 2k code) and Gpu/cpu Matrix Library Mshadow (3k code +1k comment)

Comments:

@antinucleon: Days Kiwi big, with the great God benefited greatly. The Mshadow is designed to perfectly unify CPU and GPU programming. Relies on an average of 10 rows in a layer of mshadow,cxxnet. Training imagenet on my 780 GTX can reach 211pic/sec speed.

@ li Mu m: suggested typedef tensor<gpu,2> Gpumatrix and the like, intuitive point. The constructor should be able to use the Shape2 bar, it is more convenient to drop the numbers directly?

@ Chen Tian Strange : reply @ li mu m: Because all the dimension tensor is the same template implementation, the direct loss of numbers really do not know how to engage. Maybe Cxx11 's Intializer list can

@ li mu m: reply @ Chen Tian Strange: O page link c++11 or boost is OK. But if you use c++11, it means you have a request for the GCC version.

@ Chen Tian Strange : reply @ li mu m: In fact, Tensor<gpu,x> also has the advantage, when writing code can take a template directly xpu and then input parameters is tensor<xpu,x> and then write out the code CPU, GPU is available.

@yuzzzzzzzzz: What is the high efficiency of CNN and Caffe based on this implementation?

@ Chen Tian Strange : reply @yuzzzzzzzzz: There should be no difference in efficiency, because the final compiled code should be similar. The difference is to use Mshadow to write formulas rather than to write Cuda kernel directly.

convolutional Neural Network Code cxxnet address

GPU/CPU Matrix Library Mshadow Address

——————————————————————————————————————————

2.0 cuda-convnet2.0

Google Alex Krizhevsky wrote an article on the parallel convolutional neural network in the GPU cluster and published the source code

Paper:one weird trick for parallelizing convolutional neural Networks address

Code address (PS: Google is said to close the code.google.com)

Before looking at Alex's cuda-convnet1.0 code, it is very painful, rough look at 2.0, the code is really neat.

——————————————————————————————————————————

3.0

Convolution neural Network summary code and thesis

Article address

——————————————————————————————————————————

4.0

My Deep Learning reading list

Mainly along the Bengio Pami review article found out. including several summary articles, nearly 100 papers, the presentation of the hills. All can be found on Google.
BTW: As I am interested in visual, especially detection and recognition, so the application of DL is mainly related to vision. In other areas such as speech or NLP, little or No. Individuals are very bullish on CNN and sparse Autoencoder, and this list also reflects my preference for reference only.

Comments :

@ Summer Powder _ Baidu: Personal suggestions after reading some materials, after mastering the Deep learning essence (such as can say clearly its advantages and disadvantages, in the theoretical system coordinate position), a small amount of time to update progress, more time should be spent on thinking, such as how to improve and apply. The first large-scale sparse feature of deep learning algorithm Danova is the result of continuous thinking, which is more than the efficiency of artificial features. Welcome to [email protected]

@ Bang DD: reply @ Image Vision Research: Run their code directly. Think of the DL article so much, that is, from Imagenet that began to become State-of-art, in the previous DL method, performance is not

@ Image Vision Research: reply @ Bang DD: Yes, now it's time to look at the article and study the Code, the harvest is much larger than the light to see the article

Address

——————————————————————————————————————————

5.0

The algorithm of LDA engineering practice -2 Spareslda

The Gibbs sampling algorithm in the standard LDA algorithm actually sampled is too slow, which is why the researchers based on the LDA model of the sparsity of the Sparselda algorithm, in the speed can be dozens of times times faster than the standard sampling method, so in industrial applications of course should be implemented The Sparse algorithm.

Address

——————————————————————————————————————————

6.0

@ Zhang Dong _ machine learning

This article is one of the best machine learning articles I have ever read: summed up the "12 lessons learned in machine learning practice" These lessons can not be learned in any textbooks, only in practice and specific applications to experience, very precious!

@ Dragon Star Biaoju Domingos but machine learning field of Danale, many of his work is more practical, such as Markov logic,meta-cost,sum-product Networks and so on. In addition, Tsinghua University @ Zhiyuan Liu Thu researcher translation of the Chinese version is also in place.

Pedro Domingos. A Few useful things to Know on machine learning.

Paper Address: Http://vdisk.weibo.com/s/hxqSZfjTE0X

Translated version: Machine learning those things translator: Zhiyuan Liu

Address: http://www.360doc.com/content/13/1020/18/7673502_322833764.shtml

——————————————————————————————————————————

7.0

"Using Python and OpenCV to detect the barcode on the picture" regular product packaging will have barcodes, have you ever thought of writing a program/algorithm to detect the identification of graphics code?

Address

——————————————————————————————————————————

8.0

@ Liang Bin Penny:
Over-fitting part of the reason and training data are few, sampling is not good, part of the model variable number of related, there is part of the model itself, such as linear model is not easy to fit, non-linear is easy. In fact, can be used over the fitting, plugging is inferior to sparse, more than fit model is like a number of biased people, put them ensemble good on the line, this is the ruler of the country, factory director of the road AH.

@ Nanda Zhou Zhihua:

Ensemble is usually supposed to be a weak learner when doing theoretical analysis. As for the role of overfitting, 20 years ago there was research, see Solich and Krogh 95 famous article learning with Ensembles:how overfitting can be useful, they are not you said the American army, is the European army

@ Dragon Star Biaoju:

Suggested read address

@ Silicon Valley headhunting Tomzhang: The sample is divided into 10 parts, randomly with 9 training, leaving 1 checks, do 10 trials, solve small sample problems

@ Liang Bin Penny: reply @ Silicon Valley headhunting Tomzhang: Yes, you said this is a common method 10-fold

@SMTNinja: reply @ Silicon Valley headhunting Tomzhang: You're talking about 10-fold cross validation. This can only solve the problem of inaccurate judgment on the model, which is caused by different methods of training data separation. This and overfitting are not a problem.

@ for marrying mating SCI: weak weak asked the next Liangbo, integrated learning by a number of weak learning model, that is not to refer to the strong learning model, so will conflict?

@ Liang Bin Penny: reply @ to marry mating sci: you do not adhere to the U.S. military things, weak learning can be ensemble, too fit model why can not ensemble. As long as the quantity is enough, it is equivalent to voting.

——————————————————————————————————————————

9.0

@ Cao Fat to lose weight
We recommend two machine learning courses, log Lab Annual painstaking work, Zhang Zhihua teacher in the IEEE Class and ACM Class machine learning course full record

Statistics Machine Learning Address

Introduction to Machine learning address

@ Chen Tian Strange: Yes, it seems more biased to statistics and Bayesian

——————————————————————————————————————————

10

@ William Wang

How to cheat a deep learning model? Wyoming University Three scientists have written an article that tells you how you can deliberately enter images that are not recognizable to State-of-the-art's imagenet deep learning model, but get a 99% confidence level output of the image category.

Paper Address

Weibo collection Resource Statistics (machine learning chapter) (i)