TensorFlow build your own picture data set Tfrecords (fine)

With so long TensorFlow, example ran n more, the basic flow is clear. But to do a small example of their own, various problems have come. What to do with your own picture data set. To tell you the truth, tensorflow really annoying, tube-side document tutorial on old mnist and cifar_10 this kind of good data set said, for us these beginners, completely do not know how to enter the picture. Today to share my tensorflow production data set of learning process. The
process is: Make DataSet-read DataSet-join queue
Paste the complete code first:

#encoding =utf-8 import OS import tensorflow as TF from PIL import Image CWD = OS.GETCWD () classes = {' Test ', ' test1 ', ' tes T2 '} #制作二进制数据 def Create_record (): writer = Tf.python_io.
        Tfrecordwriter ("Train.tfrecords") for index, name in enumerate (classes): Class_path = CWD + "/" + name+ "/" For Img_name in Os.listdir (class_path): Img_path = Class_path + img_name img = Image.open (img_p ATH) img = img.resize ((+)) Img_raw = Img.tobytes () #将图片转化为原生bytes print index,img _raw example = Tf.train.Example (Features=tf.train.features (feature={"Labe L ": Tf.train.Feature (Int64_list=tf.train.int64list (Value=[index)), ' Img_raw ': Tf.train.Feature (Bytes_ List=tf.train.byteslist (Value=[img_raw]))) Writer.write (example. Serializetostring ()) writer.close () data = Create_record () #读取二进制数据 def read_and_decode (filename): # CreateFile queue, unlimited number of reads Filename_queue = Tf.train.string_input_producer ([filename]) # Create a reader from file queue rea Der = tf. Tfrecordreader () # Reader reads a serialized sample from the file queue _, Serialized_example = Reader.read (filename_queue) # Get feature fr Om Serialized Example # Parse the symbolized sample features = Tf.parse_single_example (Serialized_example, features= {' label ': TF. Fixedlenfeature ([], Tf.int64), ' Img_raw ': TF. Fixedlenfeature ([], tf.string)}) label = features[' label '] img = features[' img_raw '] img = tf.de Code_raw (IMG, tf.uint8) img = Tf.reshape (IMG, [3, 1]) img = Tf.cast (IMG, tf.float32) * (./255)-0.5 L Abel = tf.cast (label, TF.INT32) return IMG, label if __name__ = = ' __main__ ': if 0:data = Create_record ("
        Train.tfrecords ") else:img, label = Read_and_decode (" train.tfrecords ") print" tengxing ", Img,label
        #使用shuffle_batch可以随机打乱输入 Next_batch down next to each other.# shuffle_batch can achieve [Img,label] synchronization, that is, the characteristics and label synchronization, otherwise the features and labels may be entered do not match # For example, only in this way, in order to make the IMG and label one by one corresponding to each extract an image and the corresponding lab The value returned by El # Shuffle_batch is the result of Randomshufflequeue.dequeue_many () # Shuffle_batch builds a randomshufflequeue and keeps the single
                                                    [Img,label], into the queue img_batch, Label_batch = Tf.train.shuffle_batch ([img, label], Batch_size=4, capacity=2000, min_after_dequeue=100 0) # Initialize all op init = Tf.initialize_all_variables () with TF. Session () as Sess:sess.run (init) # boot Queue threads = tf.train.start_queue_runners (sess=s ESS) for I in range (5): Print Img_batch.shape,label_batch val, l = Sess.run ([i
 Mg_batch, Label_batch]) # L = to_categorical (l, N) print (Val.shape, L)

Make data sets

#制作二进制数据
def create_record ():
    CWD = OS.GETCWD ()
    classes = {' 1 ', ' 2 ', ' 3 '}
    writer = Tf.python_io. Tfrecordwriter ("Train.tfrecords")
    for index, name in enumerate (classes):
        Class_path = CWD + "/" + name+ "/"
        For Img_name in Os.listdir (class_path):
            Img_path = class_path + img_name
            img = Image.open (img_path)
            img = IM G.resize ((Img_raw))
            = img.tobytes () #将图片转化为原生bytes
            #print Index,img_raw
            example = Tf.train.Example (
                features=tf.train.features (
                    feature={
                        "label": Tf.train.Feature (int64_list= Tf.train.Int64List (Value=[index]),
                        ' Img_raw ': Tf.train.Feature (Bytes_list=tf.train.byteslist (value=[img_ Raw]))
                    )
            Writer.write (example. Serializetostring ())
    Writer.close ()

The

Tfrecords file contains the Tf.train.Example Protocol memory block (protocol buffer) (the Protocol memory block contains the field Features). We can write a piece of code to get your data, fill the data into the example protocol memory block (protocol buffer), serialize the protocol memory block into a string, and pass the Tf.python_io. Tfrecordwriter writes to the Tfrecords file. read Data set

#读取二进制数据
def read_and_decode (filename):
    # Create file queue, unlimited number of reads
    Filename_queue = Tf.train.string_input_ Producer ([filename])
    # Create a reader from file queue
    reader = tf. Tfrecordreader ()
    # Reader reads a serialized sample from the file queue
    _, Serialized_example = Reader.read (filename_queue)
    # get Feature from serialized example
    # Parse symbolized sample
    features = Tf.parse_single_example (
        serialized_example,
        features={
            ' label ': TF. Fixedlenfeature ([], Tf.int64),
            ' Img_raw ': TF. Fixedlenfeature ([], tf.string)
        }
    )
    label = features[' label ']
    img = features[' img_raw ']
    img = Tf.decode_raw (img, tf.uint8)
    img = Tf.reshape (IMG, [3, 255])
    img = Tf.cast (IMG, tf.float32) * (1./)- 0.5
    label = tf.cast (Label, TF.INT32)
    return IMG, label

A example contains a dictionary containing feature (not s) in Features,features. Finally, the feature contains a floatlist, or bytelist, or int64list join the queue

With TF. Session () as Sess:
            sess.run (init)
            # boot queue
            threads = tf.train.start_queue_runners (sess=sess) for
            i in Range (5):
                print Img_batch.shape,label_batch
                val, l = Sess.run ([Img_batch, Label_batch])
                # l = to_ Categorical (l, N)
                print (Val.shape, L)

So you can get to and tensorflow the official binary data set,
Note: Start the queue that code do not forget, or the card will be used to remember the use of Val and L, otherwise the type error is reported: Typeerror:the value of a feed cannot be a TF. Tensor object. Acceptable feed values include Python scalars, strings, lists, or NumPy ndarrays. When calculating cross entropy: Cross_entropy=tf.nn.sparse_softmax_cross_entropy_with_logits (logits,labels) calculates cross entropy
The index of the maximum number of Tf.nn.in_top_k (logits,labels,1) selected Logits and the label comparison cross_entropy =-tf.reduce_sum (Y_*tf.log (Y_conv)) at the time of final evaluation Calculate the cross-entropy, so the label must be turned into a one-hot vector 2017-11-12

Add: All code is hosted on GitHub at the request of a broad range of small partners, address: github.com/tengxing
Reference article:
http://blog.csdn.net/u012759136/article/ details/52232266
http://www.shellsec.com/news/33788.html
http://blog.csdn.net/tengxing007/article/ details/54428262