Daniel J. Duffy - Numerical Methods in Computational Finance

We say that two vectors x and y are orthogonal if . We immediately see that the zero vector is orthogonal to every other vector. Another example is ; then .

Definition 5.4The set in an inner product space is orthonormal if:

(5.7)

Finding an orthonormal set in an inner product space is analogous to choosing a set of mutually perpendicular unit vectors in elementary vector analysis.

Let X be an inner product space. The set is orthonormal if for and for . The set is called an orthonormal basis .

Then .

The inner product space (continuous functions) has:

Orthnormal basis .

Orthogonality because .

An interesting application of inner products is to kernel theory to statistical learning in Learning with Kernels , Schölkopf and Smola (2002). In this case we do not work in an original (let's say n -dimensional) space X but in a feature space H . To this end, consider the map:

(5.8)

We embed data into H , and this approach offers several advantages, one of which is that we can define a similarity measure from the inner product in H :

(5.9)

The function is called a kernel .

We are almost finished with our introduction to vector spaces and linear transformations. We now discuss how matrices arise and their relationship with the current topics.

We discuss the notion of a matrix for a linear transformation. To this end, consider the linear transformation where V and W are finite-dimensional vector spaces, and let and be bases in V and W , respectively. Then:

(5.10)

for some scalars . We can represent these scalars in rectangular form which we call a matrix:

(5.11)

In this case we speak of a square matrix when otherwise, it is called a rectangular matrix . In short, each linear transformation determines a unique matrix with respect to the basis functions. Conversely, every such matrix determines a unique linear transformation from V to W defined by Equation (5.10)and the following mapping for a general vector :

(5.12)

We take simple two-dimensional problems to model reflection and (counterclockwise) rotation in the plane.

Case 1:(5.3)

Case 2:From (5.10)we getwhich allows us to find the matrix A.

Case 3:The trick is to build the matrix column by column, from top to bottom.

We first give a short history of how matrices were discovered.

The term matrix was introduced by the 19th-century English mathematician James Sylvester, but it was his friend the mathematician Arthur Cayley who developed the algebraic aspect of matrices in two papers in the 1850s. Cayley first applied them to the study of systems of linear equations, where they are still very useful. They are also important because, as Cayley recognised, certain sets of matrices form algebraic systems in which many of the ordinary laws of arithmetic (e.g., the associative and distributive laws) are valid but in which other laws (for example, the commutative law) are not valid. (Wikipedia)