A Beginner Introduction to TensorFlow (PART-1) __tensorflow

TensorFlow is one of the widely used libraries for implementing Machine learning and other algorithms involving large numb Er of mathematical operations. TensorFlow is developed by Google and it's one of the most popular Machine Learning libraries on GitHub. Google uses TensorFlow for implementing Machine learning in almost all applications. For example, if your are using Google Photos or Google voice search then you are using tensorflow models indirectly, they w Ork on large clusters of the Google hardware and are powerful in perceptual tasks.

The main aim is provide a beginner friendly introduction to TensorFlow, I assume so you already know A bit of Python. The core component of TensorFlow is the computational graph and tensors which traverse-among all the nodes through. Let ' s have a brief introduction to each one of them. tensors:

Mathematically a Tensor is a n-dimensional vector, means a Tensor can being used to represent n-dimensional. The figure above is complex to understand. We ' ll look at it ' s simplified version

The above figure shows some simplified tensors with minimum. As the dimensions keep on increasing, Data representation becomes and more complex. For example if we take a Tensor of the "form" (3x3) then I can simply call it a matrix of 3 rows and columns. If I select another Tensor of form (1000x3x3), I can call it as a vector or set of 1000 3x3 matrices. Here we call (1000x3x3) as the "shape" or Dimension of the resulting Tensor. Tensors can either be a constant or a variable. Computational Graphs (flow):

Now we understood what Tensor really means, and it's time to understand flow. This is refers to a computational graph or simply a graph, the graph can never being cyclic, each node in the graph Represe NTS An operation like addition, subtraction etc. And each operation results in the formation of new Tensor.

The figure above shows a simple computational graph. The computational graph has the following properties:

The expression for above graph : e = (a+b) x (b+1) Leaf vertices or start vertices are always tenso Rs. Means, a operation can never occur at the beginning of the graph and thus we can infer so each operation in the graph s Hould accept a Tensor and produce a new Tensor. In the same way, a tensor cannot is present as a non-leaf node, meaning the They should is always supplied as an input to the Operation/node. A computational graph always represents a complex operation in a hierarchial order. The above expression can be organized in a hierarchial way, by representing A+b as C and b+1 as D. Therefore we can write E as:

E = (c) x (d) where C = a+b and D = b+1. Traversing the graph in reverse order results in the formation of sub expressions which are combined to form the final exp Ression. When we are traverse in forward direction, the vertex we encounter always a becomes for the next dependency; Example C cannot be obtained without A and B, in the same way e cannot is obtained without for C and D. solving In the nodes of same level are independent of each. This is the ' important ' of computational graph, when we construct a graph in the way shown in the figure, it Is natural this, nodes in the same level for example C and D are independent of each, means there be no need to kno W c before evaluating D. Therefore they can be executed in parallel. Parallelism in computational graphs:

Last property mentioned above are of course one of the most important property, it clearly says this nodes at the same L are independent, means there is no need to sitting idle until C gets evaluated, you can parallely compute D while C s Till being evaluated. TensorFlow greatly exploits this property. Distributed Execution:

TensorFlow allows users to make use of the parallel computing devices to perform operations. The nodes or operations of a computational are automatically scheduled for parallel. This is happens internally, for example in the above graph, Operation C can is scheduled on CPU and Operation D can is SC Heduled on GPU. The figure below shows two prospectives distributed of execution:

The one, is a single system distributed execution where a single TensorFlow session (would be explained later) create s a single worker and the worker are responsible for scheduling tasks on various devices, in the second case, there are mul Tiple workers, they can is on same machine or on different machines, and each worker runs in its own context, in the above fi Gure, worker process 1 runs on a separate Machine and schedules operations on all available devices. Computational subgraphs:

Subgraphs are the part of main graph and are themselves computational graphs by nature. For example, in the above graph, we can obtain many subgraphs, one of them is shown below

The graph above is a section of main graph, from property 2 we can say that a subgraph always represents a sub expression, as C is the subexpression of E. Subgraphs also satisfy. Subgraphs in the same level are also independent of each and can is executed in parallel. Therefore it ' s possible to schedule entire subgraph in a single device.

The above figure explains the parallel execution of subgraphs. Here there are 2 Matrix multiplication operations, since both of them are in the same level, they are independent of each This is holds good with the last property. The nodes are scheduled on different devices Gpu_0 and gpu_1, which is because to the property of independence. exchanging data between workers:

Now we know this tensorflow distributes all it ' s operations on different devices governed by workers. It is more common that, data in the form of tensors are exchanged between workers, for example in the graph of E = (c) * (d) Once c is calculated it are desirable to pass it on further to process E, therefore Tensor the from node to node in flows D direction. This movement was done as shown in the figure:

Here tensors form Device A has been passed in to device B. This is induces some performance delays in a distributed system. The delays depends on, important property, is the size of a Tensor. The device B is in ideal mode until it receives input form device A. Need for compression:

So, it's obvious that's in computational graphs and tensors flow between nodes. It's important to reduce the delays caused through the flow before it reaches the node where it can be processed. One such idea of reducing the ' size is ' by using Lossy compression.

The data type of tensors has a major play, let ' s understand why, it's obvious that we go for higher degrees of Precision in Machine Learning operations, for example if we have float32 as the data type of a Tensor, them each value is R Epresented using a bit floating point number, so each value occupies a size of bits, the same applies for the bit ALS O. Assume a Tensor of shape (1000,440,440,3), the number of values that can is contained within the Tensor'll be 1000*44 0*440*3 if data type is-bit then it ' s times of this big number, it occupies a significant spaces in the memory and th US impose delays for the flow. Compression techniques can be used to reduce the size. Lossy compression :

Lossy compression deals with compressing the size of data and does not care about it's value, means it ' s value may become Corrupt or inaccurate during compression. But still if we have a bit floating point number like 1.01010e-12, there are a less importance that can being given to LEAs T significant digits. Changing or removing those values won't cause a much more difference in our calculation. So TensorFlow automatically converts the bit floating point numbers to a bit representation by ignoring all digits Are negligible, this reduces is the size by almost half, if it's a bit number, it's compression to bit would cause the Reduction in size by almost 75%. Thus spaces occupied by tensors can is minimized.

Once Tensor reaches the nodes, the bit representation can is bought back to it's original form just by appending 0s. Thus A or the bit representation is bought then after it reaches the node for processing.

This is ends Part-1 of the tensorflow introduction, programming and constructing simple subgraphs'll be explained in the NE XT part.

Thanks☺

Original address: https://medium.com/towards-data-science/a-beginner-introduction-to-tensorflow-part-1-6d139e038278