TensorFlow from beginner to Proficient (eight): TensorFlow tf.nn.conv2d Tracing

Readers may recall the Tf.nn module in this series (ii) and (vi), the most concerned of which is the conv2d function.

First, the blog (ii) MNIST routine convolutional.py key source list:

  DEF model (data, Train=false): "" "the    model definition.    " " # 2D convolution, with ' same ' padding (i.e. the output feature map has    # the same size as the input). Note that {strides} is a 4D array whose    # shape matches the data layout: [image index, y, x, depth].    CONV = tf.nn.conv2d (data,                        conv1_weights,                        strides=[1, 1, 1, 1],                        padding= ' same ')    # Bias and rectified Linear non-linearity.    Relu = Tf.nn.relu (Tf.nn.bias_add (conv, conv1_biases))

See the implementation of the first convolutional layer using the tf.nn.conv2d (Input_tensor, Weight_tensor, Strides_param, Padding_method) function. Trace to tensorflow/tensorflow/python/ops/gen_nn_ops.py This file, listing the code:

def conv2d (input, filter, strides, padding, Use_cudnn_on_gpu=none, Data_format=none, Name=none): R "" "Computes a  2-d convolution given 4-d ' input ' and ' filter ' tensors. Given an input tensor of shape ' [batch, In_height, In_width, In_channels] ' and a filter/kernel tensor of shape ' [filte R_height, Filter_width, In_channels, Out_channels] ', this OP performs the following:1. flattens the filter to a 2-d mat  Rix with Shape ' [filter_height * filter_width * in_channels, Output_channels] '.     2. Extracts image patches from the input tensor to form a *virtual* tensor of shape ' [batch, Out_height, Out_width,  Filter_height * filter_width * in_channels] '.  3. For each patch, right-multiplies the filter matrix and the image patch vector. In detail, with the default NHWC format, Output[b, I, j, K] = Sum_{di, DJ, q} input[b, strides[1] * i + di, STRIDES[2] * j + DJ, Q] * Filter[di, DJ, Q, K] must have ' strides[0] = strides[3] = 1`.  For the most common case of the same horizontal and vertices strides, ' strides = [1, Stride, stride, 1] '. Args:input:a ' Tensor '.    Must be one of the following types: ' float32 ', ' float64 '. Filter:a ' Tensor '.    Must has the same type as ' input '.      Strides:a List of ' ints '.  1-d of length 4. The stride of the sliding window for each dimension of ' input '.    Must is in the same order as the dimension specified with format.      Padding:a ' string ' from: ' Same ', ' VALID '.    The type of padding algorithm to use. Use_cudnn_on_gpu:an Optional ' bool '.    Defaults to ' True '. Data_format:an Optional ' string ' From: ' "NHWC", "NCHW" '.      Defaults to ' NHWC '. Specify the data format of the input and output data. With the default format "NHWC", the data was stored in the order of: [Batch, In_height, In_width, In_channels      ].    Alternatively, the format could be "NCHW", the data storage order of: [Batch, In_channels, In_height, In_width]. NameA name for the operation (optional). Returns:a ' Tensor '.  Has the same type as ' input '. "" "  Return _op_def_lib.apply_op (" conv2d ", Input=input, filter=filter,            & nbsp                 Strides=strides, padding=padding,        &NBSP ;                     use_cudnn_on_gpu=use_cudnn_on_gpu,    &NBS P                         Data_format=data_format, Name=name)

The contents of the file are generated automatically at compile time. The generator source code is located in Tensorflow/tensorflow/python/framework/python_op_gen.h and python_op_gen.cc.

_op_def_lib is built in this way:

Def _initopdeflibrary ():  op_list = op_def_pb2. Oplist ()  Text_format. Merge (_initopdeflibrary.op_list_ascii, op_list)  op_def_registry.register_op_list (op_list)  op_def_lib = op _def_library. Opdeflibrary ()  op_def_lib.add_op_list (op_list)  return op_def_lib_initopdeflibrary.op_list_ascii = "" "%s" " "_op_def_lib = _initopdeflibrary ()

See _op_def_lib is actually a op_def_pb2. Oplist object that implements a list of all operations supported by the record TensorFlow.

(not to be continued)

TensorFlow from beginner to Proficient (eight): TensorFlow tf.nn.conv2d Tracing