concept angle in category deep learning

appears as: gle, n angle
Math and Architectures of Deep Learning

This is an excerpt from Manning's book Math and Architectures of Deep Learning.

The dot product can be expressed using the cosine of the angle between the vectors. Given two vectors  and , if θ is the angle between them we have (see Figure 2.7)

 ·  = axbx + ayby                    for two dimensions

 ·  = T  = |||| ||||cos(θ) for all dimensions

(2.12)

Expressing the dot product using cosines makes it easier to see that it measures the agreement (aka correlation) between two vectors. If the vectors have the same direction, the angle between them is 0 and the cosine is 1 implying maximum agreement. The cosine progressively becomes smaller as the angle between the vectors increases until the two vectors become perpendicular to each other when the cosine becomes zero, implying no correlation - vectors are independent of each other. If the angle between them is 180° the cosine is -1 implying vectors are anti-correlated. Thus, the dot-product of two vectors is proportional to the directional agreement between them.

Figure 2.16: Note that the ellipse major axis is forming an angle of 45 degrees with X axis. Rotating coordinate system by this angle will align ellipse axes with coordinate axes. Subsequently, the first principal vector will also lie along this direction