2 Introduction to Vectors, Matrices and Tensors from Machine Learning and Data Science point of view

At its core, machine learning, indeed all computer software, is about number crunching. One inputs a set of numbers to the machine and gets back a different set of numbers as output. However, this cannot be done randomly. It is important to organize these numbers appropriately, group them into meaningful objects that go in and come out of the machine. This is where vectors and matrices come in. These are concepts that mathematicians have been using for centuries - we are simply reusing them in machine learning. In this chapter, we will study vectors and matrices, primarily from a machine learning point of view. Starting from the basics, we will quickly graduate to advanced concepts, restricting ourselves to topics that have relevance to machine learning.

We provide Jupyter notebook based python implementations for most of the concepts discussed in this and other chapters. Complete fully functional code that can be downloaded and executed (after installing python and Jupyter notebook) can be found at http://mng.bz/KMQ4. The code relevant to this chapter can be found at http://mng.bz/d4nz.

2.1 Vectors and their role in Machine Learning and Data Science

2.1.1 Geometric View of Vectors and its significance in Machine Learning and Data Science

2.2 Python code to create and access vectors and sub-vectors, slice and dice vectors, via Numpy and PyTorch parallel code

2.2.1 Python Numpy code for introduction to Vectors

2.2.2 PyTorch code for introduction to Vectors

2.3 Matrices and their role in Machine Learning and Data Science

2.4 Python Code: Introduction to Matrices, Tensors and Images via Numpy and PyTorch parallel code

2.4.1 Python Numpy code for introduction to Tensors, Matrices and Images

2.4.2 PyTorch code for introduction to Tensors and Matrices

2.5 Basic Vector and Matrix operations in Machine Learning and Data Science

2.5.1 Matrix and Vector Transpose

2.5.2 Dot Product of two vectors and its role in Machine Learning and Data Science

2.5.3 Matrix Multiplication and Machine Learning, Data Science

2.5.4 Length of a Vector aka L2 norm and its role in Machine Learning

2.5.5 Geometric intuitions for Vector Length - Model Error in Machine Learning

2.5.6 Geometric intuitions for the Dot Product - Feature Similarity in Machine Learning and Data Science

2.6 Orthogonality of Vectors and its physical significance

2.7 Python code: Basic Vector and Matrix operations via Numpy

2.7.1 Python numpy code for Matrix Transpose

2.7.2 Python numpy code for Dot product

2.7.3 Python numpy code for Matrix vector multiplication

2.7.4 Python numpy code for Matrix Matrix Multiplication

2.7.5 Python numpy code for Transpose of Matrix Product

2.7.6 Python numpy code for Matrix Inverse