CIFAR-10 image recognition

0. Learning Objectives

1. TensorFlow Data Reading principle
2. Deep Learning data Enhancement principles

I. Introduction to the CIFAR-10 data set

It is a small data set for ordinary object recognition and contains 10 categories of RGB color XXX Films (aircraft, cars, birds, cats, deer, dogs, frogs, horses, boats, trucks). The image size is 32 pixels * *, there are 50000 training pictures and 1000*** test picturesin the data set . Some of the code is from the official TensorFlow, and the following table lists the required official codes.

file	Use
cifar10.py	Establishing cifar-1o Predictive model
cifar10_input.py	Read the CIFAR-10 training picture in TensorFlow
cifar10_input_test.py	Cifar10_input test Case Files
cifar10_train.py	Training models with a single GPU or CPU
cifar10_train_multi_gpu.py	Training models with multiple GPUs
cifar10_eval.py	Test the performance of a model on a test set

Second, download CIFAR-10 data

Create the cifar10_download.py in the project root directory and enter the following code to create the program to download the data:

# 引入当前目录中已经编写好的cifar10模块import cifar10# 引入tensorflowimport tensorflow as tf# 定义全局变量存储器，可用于命令行参数的处理# tf.app.flags.FLAGS 是tensorflow 内部的一个全局变量存储器FLAGS = tf.app.flags.FLAGS# 在cifar10 模块中预先定义了cifar-10的数据存储路径，修改数据存储路径FLAGS.data_dir = ‘cifar10_data/‘# 如果数据不存在，则下载cifar10.maybe_download_and_extract()

After executing this code, the CIFAR-10 data assembly is downloaded to the directory Cifar10_data directory. The default storage path book tmp/cifar10_data, defined in the code file cifar10.py, is located around 53 lines.
After modifying the data store path, download the data cifar10.maybe_download_and_extract() , and if the data exists in the Data folder during the download, skip downloading the data and download the data instead. Successful download will prompt successfully downloaded cifar-10-binary.tar.gz 170052171 bytes.
After the download is complete, 8 files will appear in theCifar10_data/cifar-10-batches-bin , with the following table of names and purposes:

file name	Use
Batches.meta.txt	Store English names for each category
Data_batch_1.bin 、......、 Data_batch_5.bin	CIFAR-10 Five training sets, each training set in binary format stored 10000 32*32 color XXX image and map corresponding to the label, not a sample consists of 3,073 bytes, the first byte is not labeled, the remaining bytes are not image data
Test_batch.bin	Storage of 1000 images for testing and corresponding labels
Readme.html	Data Set Introduction file

Third, the mechanism of TensorFlow reading data

1. Normal way
   Read the data on the hard disk into memory and then provide it to the CPU or GPU processing
2. Memory Queue mode
   The normal way to read data occurs when the GPU or CPU has been idle for a period of time, resulting in inefficient operation. With memory queues, data reads and computations are placed in two threads, the read thread simply reads the file into the memory queue, and the compute thread only reads the data needed from the memory queue, which solves the GPU or CPU idle problem.
3. File name queue + memory queue
   The TensorFlow uses the file name queue + memory queue , which manages the epoch (note 1) Well and avoids the idle problem of the computational unit. For example, assuming that there are three data files to perform the epoch, the three data files are placed in the file name queue each time, and the end of the queue is labeled after the last placed data file. The memory queue reads the data file from the top of the file name queue in turn, and automatically throws an exception after the end tag is read, and the program can end after catching the exception. If you are performing the n-th epoch, put each data file in the file name queue n times.

   Note 1:
   For datasets, running the epoch is a complete calculation of all the data in the dataset, and so on, and so on. N times the epoch is the complete calculation of all the data in the data set n times

Iv. creating file name queues and memory queues

1. creates a file name queue
   with TensorFlow tf.train.string_input_producer () (note 2) function. To pass a list of file names to the function, the system will convert the non-file name queue. The Tf.train.string_input_producer () function has two important parameters, num_epochs and Shuffle , Num_epochs represents the number of epochs, shuffle indicates whether the order of files in the file name queue is scrambled, and if true means that the file name queue is not entered in the order added by the list of filenames. If it is flase, the file name queue is entered in the order added by the list of filenames.

2. Create memory queue
   does not manually create a memory queue in TensorFlow, just use the reader object to read data from the file name queue.

   Note 2:
   When the file name queue is created with Tf.train.string_input_producer (), the file name is not added to the queue, and if the calculation starts at this point, the entire system is blocked.
   After the file name queue is created, the tf.train.start_queue_runners method should be called to start the fill of the file name queue for the entire program to function correctly.

3. code

import tensorflow as tf# 新建sessionwith tf.Session() as sess:    # 要读取的三张图片    filename = [‘img/1.jpg‘, ‘img/2.jpg‘, ‘img/3.jpg‘]    # 创建文件名队列    filename_queue = tf.train.string_input_producer(filename, num_epochs=5, shuffle=False)    reader = tf.WholeFileReader()    key, value = reader.read(filename_queue)    # 初始化变量(epoch)    tf.local_variables_initializer().run()    threads = tf.train.start_queue_runners(sess=sess)    i = 0    while True:        i += 1        # 获取图片保存数据        image_data = sess.run(value)        with open(‘read/test_%d.jpg‘ % i, ‘wb‘) as f:            f.write(image_data)

V. Data enhancement

For image data, the method of data enhancement is to increase the number of training set samples with translation, scaling, color and so on, so as to achieve better results (note 3), using data enhancement can greatly improve the generalization ability of the model, and can prevent overfitting.
The commonly used image data enhancement method is the following table

Method	Description
Translation	Pan the image within a certain scale range
Rotating	Rotate an image within a certain angle range
Flip	Flip a picture horizontally or upside down
Cutting	Cut a piece on the original
Scaling	To enlarge or shrink an image within a certain scale
Color transform	Make some transformations on the RGB color space of an image
Noise disturbance	Add some artificially generated noise to the image

Note 3:
The use of data enhancement is based on the premise that these data enhancement methods do not alter the original label of the image. For example, the image of the number 6, after flipping up and down, becomes the image of the number 9.

Six, CIFAR-10 recognition model

The code that builds the model is in the cifar10.py file amount inference function, where the code is not detailed here, and the reader can read the comments in the code.
Here we train the model with the following commands:

python cifar10_train.py --train_dir cifar10_train/ --data_dir cifar10_data/

In this command, --data_dir cifar10_data/ represents the location where the data is saved, and --train_dir cifar10_train/ represents the location where the model parameters and training log information are saved

Vii. Review of Training progress

In training we often need to know the change of the loss and the training of each layer, this time we will use the Tensorboardprovided by TensorFlow. Open a new command line and enter the following command:

tensorboard --logdir cifar10_train/

where --logdir cifar10_train/ represents the location where the model training log is saved, after running the command you will see something like the following on the command line

Enter the displayed address on the browser to access the Tensorboard. Briefly explain a few common tags:

label	Description
Total_loss_1	Loss change curve, change curve will change according to time
Learning_rate	Learning Rate Change curve
Global_step	Wonderful training steps, if the training speed changes more, or more slowly, it indicates that the program may be wrong

Viii. accuracy of the test model

Enter the following command in the Command line window:

python cifar10_eval.py --data_dir cifar10_data/ --eval_dir cifar10_eval/ --checkpoint_dir cifar10_train/

--data_dir cifar10_data/ represents the location where the CIFAR-10 dataset is stored.
--heckpoint_dir cifar1o_train/ indicates that the program model is saved under the Cifar10_train/folder.
--eval_dir cifar10_eval/ Specifies a folder where test information is saved
Enter the following command to see the accuracy of the age-old training steps on Tensorboard:

tensorboard --logdir cifar10_eval/ --port 6007

Enter in the browser: http://127.0.0.1:6007, expand the Precision @ 1 tab to see how the accuracy varies with the number of training steps.

Nine, code download

Git address: https://gitee.com/bugback/ai_learning.git
Baidu Network disk: Https://pan.baidu.com/s/17HdfI2R9gsOMKi4pgundSA

CIFAR-10 image recognition