A simple blog post about Weka

Weka is a Java-written open source pattern recognition and data mining software, has been more than 10 years of history. Weka's official website is in http://www.cs.waikato.ac.nz/ml/weka/.

There are four problems with pattern recognition and data mining,

First: What is the problem?

Second: What is the data

Third: How to learn

Four: is the learning result reliable?

The first problem comes from demand. Analysis of requirements is difficult: rigorous logic, in-depth understanding of the industry macro and detail, familiar with the technical field and academic progress, there are several successful projects of practical experience, these four factors are indispensable, so usually by a team of different areas of the elite cooperation.

Weka does not solve the demand problem.

The second problem is data. Each sample corresponds to a Weka instance, and a dataset consisting of multiple samples corresponds to the Weka instances, which is the storage. For datasets, you need to select a variety of samples for training and testing, there are many options. For example, select only a subset of the samples for training and testing, handle the sample with missing attributes, select only some of the properties for training and testing, and rearrange the sample order to change the training and test results. If there is a supervised or unsupervised way to select the sample and its attributes.

A third problem is learning. If the first question can be precisely defined, then the answer to the third question must be clear. Weka provides a large number of algorithms, classifications, regressions, clustering, association rules, and so on. For beginners, the choice algorithm is a big problem, each algorithm has its advantages, but none of the algorithms is better than other algorithms on most indicators. The trick here is to do a lot of experiments and analyze the results, and do more naturally know what's good.

The fourth issue is to verify that the learner is reliable. The usual way is cross-validation, five times-fold crossover or 10 times-fold crossover. Then with the network style parameter. The general problem can be solved.

The big Data,weka recommendation is to use the command line to manipulate data and training, implement the algorithm yourself with groovy or jython if possible, or use an algorithm that can be incrementally learned. What it means to say is that Weka is not ready for big data, so it's best to use it to solve a single problem.

A simple blog post about Weka