concept TensorFlow in category machine learning

This is an excerpt from Manning's book Machine Learning with TensorFlow, Second Edition MEAP V08. Login to get full access to this book.

A bit of history

A former scalable distributed training and learning system called DistBelief is the primary influence on TensorFlow’s current implementation. Ever written a messy piece of code and wished you could start all over again? That’s the dynamic between DistBelief and TensorFlow. TensorFlow is not the first system Google open-sourced based on an internal project. Google’s famous Map-Reduce system and Google File System (GFS) are the basis of modern Apache data processing, web-crawling and big-data systems including Hadoop, Nutch and Spark. Additionally, Google’s bigTable system is where the Apache Hbase project came from.

The library is implemented in C++ and has a convenient Python API, as well as a lesser appreciated C++ API. Because of the simpler dependencies, TensorFlow can be quickly deployed to various architectures.

That’s too much code just to calculate the inner product of two vectors (also known as the dot product). Imagine how much code would be required for something more complicated, such as solving linear equations or computing the distance between two vectors if you still lacked the computing library that TensorFlow and its associated friends like Numerical Python (NumPy) are.

When installing the TensorFlow library, you also install a well-known and robust Python library called NumPy, which facilitates mathematical manipulation in Python. Using Python without its libraries (NumPy and TensorFlow) is like using a camera without auto mode: sure, you gain more flexibility, but you can easily make careless mistakes (for the record, we have nothing against photographers who micromanage aperture, shutter, and ISO, the so-called “manual” knobs when preparing your camera to take an image). It’s easy to make mistakes in machine learning, so let’s keep our camera on autofocus and use TensorFlow to help automate tedious software development.

The syntax for tensors is even more nested vectors. For example, as shown in figure 2.2, a 2 × 3 × 2 tensor is [[[1,2], [3,4], [5,6]], [[7,8], [9,10], [11,12]]], which can be thought of as two matrices, each of size 3 × 2. Consequently, we say this tensor has a rank of 3. In general, the rank of a tensor is the number of indices required to specify an element. Machine-learning algorithms in TensorFlow act on tensors, so it’s important to understand how to use them.

Figure 2.2 This tensor can be thought of as multiple matrices stacked on top of each other. To specify an element, you must indicate the row and column, as well as which matrix is being accessed. Therefore, the rank of this tensor is 3.

It’s easy to get lost in the many ways to represent a tensor. Intuitively, three lines of code in listing 2.3 are trying to represent the same 2 × 2 matrix. This matrix represents two feature vectors of two dimensions each. It could, for example, represent two people’s ratings of two movies. Each person, indexed by the row of the matrix, assigns a number to describe their review of the movie, indexed by the column. Run the code to see how to generate a matrix in TensorFlow.

A session is an environment of a software system that describes how the lines of code should run. In TensorFlow, a session sets up how the hardware devices (such as CPU and GPU) talk to each other. That way, you can design your machine-learning algorithm without worrying about micromanaging the hardware it runs on. You can later configure the session to change its behavior without changing a line of the machine-learning code.

To execute an operation and retrieve its calculated value, TensorFlow requires a session. Only a registered session may fill the values of a Tensor object. To do so, you must create a session class by using tf.Session() and tell it to run an operator, as shown in the following listing. The result will be a value you can later use for further computations.

Listing 2.6 Using a session

import tensorflow as tf
 
x = tf.constant([[1., 2.]])      #A
neg_op = tf.negative(x)          #B
 
with tf.Session() as sess:        #C
    result = sess.run(negMatrix)    #D
print(result)                   #E

copy

Congratulations! You’ve just written your first full TensorFlow code. Although all it does is negate a matrix to produce [[-1, -2]], the core overhead and framework are just the same as everything else in TensorFlow. A session not only configures where your code will be computed on your machine, but also crafts how the computation will be laid out in order to parallelize computation.

This is an excerpt from Manning's book Machine Learning with TensorFlow. Login to get full access to this book.

TensorFlow will now be accessible from a Jupyter Notebook via a local IP address. The IP can be found by using the docker-machine ip command, as shown in figure A.3.

Figure A.3. Docker’s IP address can be found using the docker-machine ip command or can be found in the intro text under the ASCII whale.

Open a browser and navigate to http://<YOUR_IP_ADDRESS>:8888 to start using TensorFlow. In our case, the URL was http://192.168.99.100:8888. Figure A.4 shows the Jupyter Notebook accessed through a browser.

Figure A.4. You can interact with TensorFlow through a Python interface called Jupyter.

The syntax for tensors is even more nested vectors. For example, as shown in figure 2.2, a 2 × 3 × 2 tensor is [[[1,2], [3,4], [5,6]], [[7,8], [9,10], [11,12]]], which can be thought of as two matrices, each of size 3 × 2. Consequently, we say this tensor has a rank of 3. In general, the rank of a tensor is the number of indices required to specify an element. Machine-learning algorithms in TensorFlow act on tensors, so it’s important to understand how to use them.

Figure 2.2. This tensor can be thought of as multiple matrices stacked on top of each other. To specify an element, you must indicate the row and column, as well as which matrix is being accessed. Therefore, the rank of this tensor is 3.

It’s easy to get lost in the many ways to represent a tensor. Intuitively, three lines of code in listing 2.3 are trying to represent the same 2 × 2 matrix. This matrix represents two feature vectors of two dimensions each. It could, for example, represent two people’s ratings of two movies. Each person, indexed by the row of the matrix, assigns a number to describe their review of the movie, indexed by the column. Run the code to see how to generate a matrix in TensorFlow.

Listing 2.3. Different ways to represent tensors

import tensorflow as tf import numpy as np 1 m1 = [[1.0, 2.0], [3.0, 4.0]] 2 m2 = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float32) 2 m3 = tf.constant([[1.0, 2.0], [3.0, 4.0]]) 2 print(type(m1)) 3 print(type(m2)) 3 print(type(m3)) 3 t1 = tf.convert_to_tensor(m1, dtype=tf.float32) 4 t2 = tf.convert_to_tensor(m2, dtype=tf.float32) 4 t3 = tf.convert_to_tensor(m3, dtype=tf.float32) 4 print(type(t1)) 5 print(type(t2)) 5 print(type(t3)) 5

copy

1 You’ll use NumPy matrices in TensorFlow.

Because TensorFlow is primarily a Python library, you should make full use of Python’s interpreter. Jupyter is a mature environment for exercising the interactive nature of the language. It’s a web application that displays computation elegantly so that you can share annotated interactive algorithms with others to teach a technique or demonstrate code.

2.1. Ensuring that TensorFlow works

First, you should ensure that everything is working correctly. Check the oil level in your car, repair the blown fuse in your basement, and ensure that your credit balance is zero. Just kidding; we’re talking about TensorFlow.