Span in Linear Algebra

The term “span” is a fundamental concept in linear algebra, particularly in the study of vector spaces. It refers to the set of all possible vectors that can be formed by taking linear combinations of a given set of vectors. This concept is crucial for understanding the structure and properties of vector spaces.

Definition and Basic Understanding

The span of a set of vectors is the collection of all vectors that can be expressed as linear combinations of those vectors. This means that if you have a set of vectors, the span includes all vectors that can be created by scaling and adding these vectors together. The span forms a vector subspace, which is a subset of the vector space that is closed under vector addition and scalar multiplication.

For example, if you have two vectors in a two-dimensional space, such as the Cartesian coordinate axes, the span of these vectors is the entire plane that contains them. This is because any point in the plane can be reached by some linear combination of these two vectors. If the vectors are the standard basis vectors (\mathbf{i}) and (\mathbf{j}), any vector (\mathbf{v}) in the plane can be expressed as (a\mathbf{i} + b\mathbf{j}), where (a) and (b) are scalars.

Visualizing Span

Span of Two Non-Parallel Vectors

When considering two non-parallel vectors, each vector individually spans a line. However, when combined, they span a larger set of points. For instance, the vector sum (\mathbf{v} + \mathbf{w}) lies on neither of the individual lines spanned by (\mathbf{v}) or (\mathbf{w}). This concept is illustrated in Figure 6.17.

Figure 6.17 The span of two non-parallel vectors. Each individual vector spans a line, but together they span more points, for instance, v + w lies on neither line.

In the plane, the span of two non-parallel vectors is the entire plane. This is true for any pair of non-parallel vectors, such as the standard basis vectors. For example, any point ((x, y)) can be reached as the linear combination (x \cdot (1, 0) + y \cdot (0, 1)).

Span in Higher Dimensions

In higher dimensions, the concept of span becomes more complex. For example, three vectors might not span a 3D space even if no pair of vectors is parallel. Figure 6.20 shows three non-parallel vectors that only span a 2D space because they all lie in the same plane.

Figure 6.20 Three non-parallel vectors that only span a 2D space.

To span a higher-dimensional space, a new vector must point in a direction not already included in the span of the existing vectors. In the plane, three vectors often have redundancy. As shown in Figure 6.21, a linear combination of two vectors can yield the third, indicating that the span of all three is no larger than the span of just two.

Figure 6.21 A linear combination of u and w returns v, so the span of u, v, and w should be no bigger than the span of u and w.

Linear Independence

The concept of linear independence is closely related to span. A set of vectors is linearly dependent if any vector in the set can be expressed as a linear combination of the others. For example, two parallel vectors are linearly dependent because they are scalar multiples of each other. Similarly, a set of three vectors ({\mathbf{u}, \mathbf{v}, \mathbf{w}}) is linearly dependent if one of the vectors can be expressed as a combination of the others. This is illustrated in Figure 6.21, where (\mathbf{v}) can be obtained from (\mathbf{u}) and (\mathbf{w}).

Understanding the span of vectors and their linear independence is essential for exploring the dimensions and capabilities of vector spaces, which are foundational in fields such as data science and deep learning.