concept PyTorch in category deep learning

This is an excerpt from Manning's book Math and Architectures of Deep Learning.
In this book, we will try to familiarize the the reader with numpy, PyTorch and similar programming paradigms alongside the relevant mathematics. Knowledge of python basics will be assumed. The reader is strongly encouraged to try out all code snippets in this book – after installing appropriate packages like numpy, PyTorch etc.
Our initial implementation will simply mimic this formula. For more complicated models ϕ (with millions of parameters and non-linearities) we cannot obtain closed form gradients like this. The next example, based on PyTorch, relies on PyTorch’s autograd (automatic gradient computation) which does not require the closed form gradient. //
Figure 3.12: Auto grad Analysis
![]()
Listing 3.2: Linear modeling with PyTorch
![]()

This is an excerpt from Manning's book Deep Learning with Structured Data.
There are a variety of deep learning frameworks available today. The two most popular are TensorFlow, which dominates in industrial applications of deep learning and PyTorch, which has a strong following in the research community.
Before exploring Keras in more depth, it’s worth quickly discussing the other major library that is currently used for deep learning: PyTorch. PyTorch was developed by Facebook and made available as open source in 2017. This article (https://builtin.com/data-science/pytorch-vs-tensorflow) gives a succinct comparison of the two libraries. The community using TensorFlow is currently larger than that using PyTorch, although PyTorch is growing quickly. PyTorch has a stronger presence in the academic / research world (including being the basis of the coding aspects of the fast.ai course described in chapter 9) while TensorFlow is predominant in industry. For the purposes of this book (which was started before the initial TensorFlow 2 beta where Keras was integrated as a front-end for TensorFlow), Keras provided the minimum barriers to entry because of its simplicity and abstraction of low-level details.

This is an excerpt from Manning's book Deep Learning with PyTorch.
PyTorch is a library for Python programs that facilitates building deep learning projects. It emphasizes flexibility and allows deep learning models to be expressed in idiomatic Python. This approachability and ease of use found early adopters in the research community, and in the years since its first release, it has grown into one of the most prominent deep learning tools across a broad range of applications.
As Python does for programming, PyTorch provides an excellent introduction to deep learning. At the same time, PyTorch has been proven to be fully qualified for use in professional contexts for real-world, high-profile work. We believe that PyTorch’s clear syntax, streamlined API, and easy debugging make it an excellent choice for introducing deep learning. We highly recommend studying PyTorch for your first deep learning library. Whether it ought to be the last deep learning library you learn is a decision we leave up to you.
At its core, the deep learning machine in figure 1.1 is a rather complex mathematical function mapping inputs to an output. To facilitate expressing this function, PyTorch provides a core data structure, the tensor, which is a multidimensional array that shares many similarities with NumPy arrays. Around that foundation, PyTorch comes with features to perform accelerated mathematical operations on dedicated hardware, which makes it convenient to design neural network architectures and train them on individual machines or parallel computing resources.
This book is intended as a starting point for software engineers, data scientists, and motivated students fluent in Python to become comfortable using PyTorch to build deep learning projects. We want this book to be as accessible and useful as possible, and we expect that you will be able to take the concepts in this book and apply them to other domains. To that end, we use a hands-on approach and encourage you to keep your computer at the ready, so you can play with the examples and take them a step further. By the time we are through with the book, we expect you to be able to take a data source and build out a deep learning project with it, supported by the excellent official documentation.
Although we stress the practical aspects of building deep learning systems with PyTorch, we believe that providing an accessible introduction to a foundational deep learning tool is more than just a way to facilitate the acquisition of new technical skills. It is a step toward equipping a new generation of scientists, engineers, and practitioners from a wide range of disciplines with working knowledge that will be the backbone of many software projects during the decades to come.
# In[51]: points_64 = torch.rand(5, dtype=torch.double) #1 points_short = points_64.to(torch.short) points_64 * points_short # works from PyTorch 1.3 onwards # Out[51]: tensor([0., 0., 0., 0., 0.], dtype=torch.float64)
In part 1 of this book, we learned a lot about models; and part 2 left us with a detailed path for creating good models for a particular problem. Now that we have these great models, we need to take them where they can be useful. Maintaining infrastructure for executing inference of deep learning models at scale can be impactful from an architectural as well as cost standpoint. While PyTorch started off as a framework focused on research, beginning with the 1.0 release, a set of production-oriented features were added that today make PyTorch an ideal end-to-end platform from research to large-scale production.

This is an excerpt from Manning's book Deep Reinforcement Learning in Action.
This appendix offers a rapid review of deep learning, the relevant mathematics we use in this book, and how to implement deep learning models in PyTorch. We’ll cover these topics by demonstrating how to implement a deep learning model in PyTorch to classify images of handwritten digits from the famous MNIST dataset.
First, though, we will take a moment to introduce PyTorch, the deep learning framework we will be using throughout this book to build neural networks.
There are many deep learning frameworks available today, with TensorFlow, MXNet, and PyTorch probably being the most popular. We chose to use PyTorch for this book because of its simplicity. It allows you to write native-looking Python code and still get all the goodies of a good framework like automatic differentiation and built-in optimization. We’ll give you a quick introduction to PyTorch here, but we’ll explain more as we go along. If you need to brush up on basic deep learning, see the appendix where we have a fairly detailed review of deep learning and more thorough coverage of PyTorch.
If you’re comfortable with the numpy multidimensional array, you can replace almost everything you do with numpy with PyTorch. For example, here we instantiate a 2 × 3 matrix in numpy:
>>> import numpy >>> numpy.array([[1, 2, 3], [4, 5, 6]]) array([[1, 2, 3], [4, 5, 6]])And here is how you instantiate the same matrix with PyTorch:
>>> import torch >>> torch.Tensor([[1, 2, 3], [4, 5, 6]]) 1 2 3 4 5 6 [torch.FloatTensor of size 2x3]The PyTorch code is basically the same as the numpy version, except in PyTorch we call multidimensional arrays tensors. Unsurprisingly, this is also the term used in TensorFlow and other frameworks, so get used to seeing multidimensional arrays referred to as tensors. We can and do refer to the tensor order, which is basically how many indexed dimensions the tensor has. This gets a little confusing because sometimes we speak of the dimension of a vector, in which case we’re referring to the length of the vector. But when we speak of the order of a tensor, we mean how many indices it has. A vector has one index, meaning every element can be “addressed” by a single index value, so it’s an order 1 tensor or 1-tensor for short. A matrix has two indices, one for each dimension, so it’s a 2-tensor. Higher order tensors can be referred to as a k-tensor, where k is the order, a non-negative integer. On the other end, a single number is a 0-tensor, also called a scalar, since it has no indices.

This is an excerpt from Manning's book Grokking Deep Learning.
If you’ve been reading about deep learning for long, you’ve probably come across one of the major frameworks such as PyTorch, TensorFlow, Theano (recently deprecated), Keras, Lasagne, or DyNet. Framework development has been extremely rapid over the past few years, and, despite all frameworks being free, open source software, there’s a light spirit of competition and comradery around each framework.
Why should you choose PyTorch? There are many good options, but if you’re coming from a NumPy background, PyTorch will feel the most familiar. Furthermore, the framework you built in chapter 13 closely resembles the API of PyTorch. I did it this way specifically with the intent of preparing you for an actual framework. If you choose PyTorch, you’ll feel right at home. That said, choosing a deep learning framework is sort of like joining a house at Hogwarts: they’re all great (but PyTorch is definitely Gryffindor).
Now the next question: how should you learn PyTorch? The best way is to take a deep learning course that teaches you deep learning using the framework. This will jog your memory about the concepts you’re already familiar with while showing you where each piece lives in PyTorch. (You’ll review stochastic gradient descent while also learning about where it’s located in PyTorch’s API.) The best place to do this at the time of writing is either Udacity’s deep learning Nanodegree (although I’m biased: I helped teach it) or fast.ai. In addition, https://pytorch.org/tutorials and https://github.com/pytorch/examples are golden resources.