The "end-to-end" character Verification code identification of TensorFlow based on Python language

1 Abstract

The birth of the Verification Code (CAPTCHA) itself is to automatically distinguish the natural person and the robot a set of public methods, but in recent years, the development of artificial intelligence technology, the traditional character verification has become a fake. Therefore, on the one hand, while studying and learning this code, on the other hand, we should also be wary of their Internet system Web security issues.

Keywords: AI, Python, character verification code, CAPTCHA, recognition, tensorflow,cnn, deep learning

2 Introduction

Fully automatic distinguish between computer and human public Turing Test (English: Completely automated publicly Turing test to tell Computers and humans Apart, CAPTCHA), commonly known as CAPTCHA, is a public automatic program that differentiates users from computers or people [1].

Thanks to the development of the artificial intelligence technology based on Convolutional Neural Network (CNN), the end-to-end non-segmentation method is developed directly based on the mainstream deep learning framework, and the recognition rate of over 95% can be achieved without too much trick.

Traditional machine learning method, for multi-character verification code is the piecemeal method: first divided into the smallest units, and then identified separately, and then unified. Convolutional neural network method, directly using end-to-end non-segmentation method: Input The entire picture, output the entire picture of the mark results, with a stronger versatility.

The specific differences are as follows:

The end-to-end recognition method is obviously more advantageous, because the current character-type verification code in order to prevent recognition, the multi-character has been completely fused together, the use of traditional technology is very difficult to achieve segmentation. This article focuses on the end-to-end approach.

3 citation claims

The code in this article is referenced from this article:

http://blog.topspeedsnail.com/archives/10858

Big Panda--"WTF Daily Blog"

The main problem of this project is to identify the character type verification of a certain pattern in the end-to-end.

Input content:

Model Prediction Results:

4 The work of this article

• Explains the puzzle questions about sigmoid selection as mentioned in the original author code comments and applies it to the code
• The original author's code for the module engineering, as a whole project, convenient for the study of the students to apply the model directly

In the original author code:

Def train_crack_captcha_cnn ():    output = CRACK_CAPTCHA_CNN ()    # loss    #loss = Tf.reduce_mean (tf.nn.softmax_ Cross_entropy_with_logits (output, Y))    loss = Tf.reduce_mean (Tf.nn.sigmoid_cross_entropy_with_logits (logits= Output, labels=y)        # What's the difference between the Softmax and sigmoid of the last layer to classify?    # Optimizer in order to speed up the training learning_rate should start big, then slowly decay    optimizer = Tf.train.AdamOptimizer (learning_rate=0.001). Minimize (loss) ...    

The author raises the question in the comments of the code:

There are questions about how Softmax and sigmoid are used.

Then in the article below the reader comment area also mentioned this issue, here for an overall explanation.

The dimensions of the output of CNN in the original text are Max_captcha*char_set_len , in fact these dimensions are not all completely independent distribution, but the use of sigmoid loss is still possible, equivalent to first using sigmoid for normalization , and then return the values of each dimension to the target value, and the smaller the loss, the closer the corresponding values of the two vectors are. in fact, sigmoid can be regarded as a multi-classification problem, in this example can also play a better convergence effect .

Of course, on the question of classification, looking at all machine learning frameworks, it is recommended to use Softmax for the final normalization, which is actually equivalent to a Matthew effect : The value of the larger probability of the classification becomes larger, the less likely the component value becomes smaller. There is a premise, however, that a set of data that participates in the Softmax operation must be associated , so if you want to use softmax_cross_entropy_with_logits , simply modify the network. The dimensions of the output are made into two dimensions [Max_captcha, Char_set_len], and then Softmax loss are used.

Output = CRACK_CAPTCHA_CNN () #36 x4predict = tf.reshape (output, [-1, Max_captcha, Char_set_len])  # 36 rows, 4 columns label = Tf.reshape (Y, [-1, Max_captcha, Char_set_len])

Finally, the results of the experiment using GPU training are compared:

• Sigmoid way. Training 6,000 step can achieve a 95% accuracy rate.
• Softmax Way. Train 8,000 step, achieve 90% accuracy, train 80,000 step, reach 94.7% (run for half a day)

Use Tensorboard to monitor the accuracy:

Sigmoid-6 thousand Step:

Softmax-8 thousand Step:

Softmax-8 Step:

Overall, in this example, it seems that the convergence rate of sigmoid faster, of course, this may be the external factors in this project is conducive to sigmoid bar, as for the specific reasons, such as follow-up and further study and explain it, of course, it may not be explained at all, because for CNN, the current mainstream opinion is:, Anyway the effect is good, but do not know why, science fiction is near fantasy of a technology.

GitHub Source Address:

Https://github.com/zhengwh/captcha-tensorflow

Project File Description:

• cfg.py Configuration information file
• cnn_sys.py CNN Network structure
• data_iter.py data sets that can be iterated
• gen_captcha.py Verification code generator, directly using the program to generate tagged data
• predict.py load the trained model and then make predictions for the imported images.
• train.py Training the Model
• Utils.py Some methods of public use

5 Summary

This article mainly only writes the original author not to mention the content, wants to understand the original text, may go directly to the original author page.

6 Reference

[1]	Wiki-captcha Https://en.wikipedia.org/wiki/CAPTCHA

The "end-to-end" character Verification code identification of TensorFlow based on Python language