concept TensorFlow in category python

This is an excerpt from Manning's book Deep Learning with Python, Second Edition MEAP V04. Login to get full access to this book.

After reading this chapter, you’ll have an intuitive understanding of the mathematical theory behind deep learning, and you’ll be ready to start diving into Keras and TensorFlow, in chapter 3.

TensorFlow is a Python-based, free, open-source machine learning platform, developed primarily by Google. Much like NumPy, the primary purpose of TensorFlow is to enable engineers and researchers to manipulate mathematical expressions over numerical tensors. But TensorFlow goes far beyond the scope of NumPy in the following ways:

Keras is a deep-learning API for Python, built on top of TensorFlow, that provides a convenient way to define and train any kind of deep-learning model. Keras was initially developed for research, with the aim of enabling fast deep learning experimentation.

Through TensorFlow, Keras can run on top different types of hardware (see figure 1) — GPU, TPU, or plain CPU — and can be seamlessly scaled to thousands of machines.

Figure 3.1. Keras and TensorFlow: TensorFlow is a low-level tensor computing platform, Keras is a high-level deep learning API

Try to do the same thing in TensorFlow: you will get an error, "EagerTensor object does not support item assignment".

Listing 3.4. TensorFlow tensors are not assignable

x = tf.ones(shape=(2, 2))
x[0, 0] = 0.  #1

copy

To train a model, we’ll need to update its state, which is a set of tensors. If tensors aren’t assignable, how do we do it, then? That’s where variables come in. tf.Variable is the class meant to manage modifiable state in TensorFlow. You’ve already briefly seen it in action in the training loop implementation at the end of chapter 2.

This is an excerpt from Manning's book Deep Learning with Python. Login to get full access to this book.

There are two conventions for shapes of images tensors: the channels-last convention (used by TensorFlow) and the channels-first convention (used by Theano). The TensorFlow machine-learning framework, from Google, places the color-depth axis at the end: (samples, height, width, color_depth). Meanwhile, Theano places the color depth axis right after the batch axis: (samples, color_depth, height, width). With the Theano convention, the previous examples would become (128, 1, 256, 256) and (128, 3, 256, 256). The Keras framework provides support for both formats.

TensorFlow, CNTK, and Theano are some of the primary platforms for deep learning today. Theano (http://deeplearning.net/software/theano) is developed by the MILA lab at Université de Montréal, TensorFlow (www.tensorflow.org) is developed by Google, and CNTK (https://github.com/Microsoft/CNTK) is developed by Microsoft. Any piece of code that you write with Keras can be run with any of these backends without having to change anything in the code: you can seamlessly switch between the two during development, which often proves useful—for instance, if one of these backends proves to be faster for a specific task. We recommend using the TensorFlow backend as the default for most of your deep-learning needs, because it’s the most widely adopted, scalable, and production ready.

Via TensorFlow (or Theano, or CNTK), Keras is able to run seamlessly on both CPUs and GPUs. When running on CPU, TensorFlow is itself wrapping a low-level library for tensor operations called Eigen (http://eigen.tuxfamily.org). On GPU, Tensor-Flow wraps a library of well-optimized deep-learning operations called the NVIDIA CUDA Deep Neural Network library (cuDNN).

Install cuDNN:

1. Register for a free NVIDIA developer account (unfortunately, this is necessary in order to gain access to the cuDNN download), and download cuDNN at https://developer.NVIDIA.com/cudnn (select the version of cuDNN compatible with TensorFlow). Like CUDA, NVIDIA provides packages for different Linux flavors—we’ll use the version for Ubuntu 16.04. Note that if you’re working with an EC2 install, you won’t be able to download the cuDNN archive directly to your instance; instead, download it to your local machine and then upload it to your EC2 instance (via scp).
2. Install cuDNN:

   $ sudo dpkg -i dpkg -i libcudnn6*.deb

   copy