Linear Combination

A linear combination is a fundamental concept in linear algebra, involving the creation of a new vector by taking a weighted sum of a set of vectors. The weights applied to these vectors are scalars, which can be any real or complex numbers. This operation is central to many areas of mathematics and its applications, including deep learning, geometry, and data science.

Overview

A linear combination of a collection of vectors is a sum of scalar multiples of those vectors. Mathematically, if you have vectors u and v, a linear combination could be expressed as 3u − 2v. More generally, a linear combination is an expression made up of a set of terms multiplied by scalars and added together. For example, a linear combination of vectors v and w is av + bw. This operation is crucial in understanding linear transformations, where vectors are transformed through operations involving linear combinations.

Linear Dependence

A set of vectors is said to be linearly dependent if there exists a set of scalar weights, not all of which are zero, such that their linear combination results in the zero vector. This implies that at least one of the vectors in the set can be expressed as a linear combination of the others. Understanding linear dependence is crucial for determining the independence of vectors, which is a key property in vector spaces.

Vector Spaces

Vector spaces are mathematical structures that allow for the operations of vector addition and scalar multiplication. These operations enable the formation of linear combinations, making vector spaces a foundational concept in linear algebra. In a vector space, any vector can be expressed as a linear combination of a set of basis vectors. Linear combinations are fundamental in the study of vector spaces, which can include objects like arrows in the plane, tuples of numbers, or even images.

Examples and Applications

• Images as Vectors: You can treat images as vectors and take a linear combination of them, as shown in Figure 6.1.

  Figure 6.1 A linear combination of two pictures produces a new picture.

• Midpoint Calculation: Linear combinations can be used to find midpoints between vectors. For example, the midpoint between the tips of two vectors u and v can be found using the linear combination ½u + ½v = ½(u + v), as illustrated in Figure 4.24.

  Figure 4.24 The midpoint between the tips of two vectors u and v can be found as the linear combination ½u + ½v = ½(u + v).

• Spanning Vectors: The span of two non-parallel vectors is the set of all possible linear combinations of those vectors. This means that each individual vector spans a line, but together they span more points. For instance, the vector v + w lies on neither line spanned by v or w alone, as shown in Figure 6.17.

  Figure 6.17 The span of two non-parallel vectors. Each individual vector spans a line, but together they span more points, for instance, v + w lies on neither line.

• Fourier Series: In the context of sound waves, linear combinations are used to generate complex sounds from simpler sinusoidal functions. An example of this is the Fourier series, which combines sine and cosine functions with different frequencies and coefficients to create complex waveforms.

Implementation

The concept of linear combinations can be implemented programmatically. Below is a Python function that computes the linear combination of given vectors and scalars:

from vectors import *
def linear_combination(scalars,*vectors):
   scaled = [scale(s,v) for s,v in zip(scalars,vectors)]
   return add(*scaled)

>>> linear_combination([1,2,3], (1,0,0), (0,1,0), (0,0,1))
(1, 2, 3)

This function takes a list of scalars and a variable number of vectors, scales each vector by its corresponding scalar, and then adds the scaled vectors together to produce the linear combination.