concept storage in category deep learning

This is an excerpt from Manning's book Deep Learning with PyTorch.
At left in figure 1.2, we see that quite a bit of data processing is needed before the training data even reaches our model.4 First we need to physically get the data, most often from some sort of storage as the data source. Then we need to convert each sample from our data into a something PyTorch can actually handle: tensors. This bridge between our custom data (in whatever format it might be) and a standardized PyTorch tensor is the
Datasetclass PyTorch provides intorch.utils.data. As this process is wildly different from one problem to the next, we will have to implement this data sourcing ourselves. We will look in detail at how to represent various type of data we might want to work with as tensors in chapter 4.
It is time for us to look a bit closer at the implementation under the hood. Values in tensors are allocated in contiguous chunks of memory managed by
torch.Storageinstances. A storage is a one-dimensional array of numerical data: that is, a contiguous block of memory containing numbers of a given type, such asfloat(32 bits representing a floating-point number) orint64(64 bits representing an integer). A PyTorchTensorinstance is a view of such aStorageinstance that is capable of indexing into that storage using an offset and per-dimension strides.5
Let’s see how indexing into the storage works in practice with our 2D points. The storage for a given tensor is accessible using the
.storageproperty:# In[17]: points = torch.tensor([[4.0, 1.0], [5.0, 3.0], [2.0, 1.0]]) points.storage() # Out[17]: 4.0 1.0 5.0 3.0 2.0 1.0 [torch.FloatStorage of size 6]Even though the tensor reports itself as having three rows and two columns, the storage under the hood is a contiguous array of size 6. In this sense, the tensor just knows how to translate a pair of indices into a location in the storage.
# In[18]: points_storage = points.storage() points_storage[0] # Out[18]: 4.0 # In[19]: points.storage()[1] # Out[19]: 1.0