http://eclr.humanities.manchester.ac.uk/api.php?action=feedcontributions&feedformat=atom&user=LGECLR - User contributions [en]2026-05-16T05:46:36ZUser contributionsMediaWiki 1.30.1http://eclr.humanities.manchester.ac.uk/index.php?title=Lnotes&diff=3045Lnotes2013-09-10T15:10:54Z<p>LG: </p>
<hr />
<div>= Matrices =<br />
<br />
In the PreSession Maths course, a matrix was defined as follows:<br />
<br />
<blockquote>A matrix is a rectangular array of numbers enclosed in parentheses, conventionally denoted by a capital letter. The number of rows (say <math>m</math>) and<br />
<br />
the number of columns (say <math>n</math>) determine the order of the matrix (<math>m</math> <math>\times</math> <math>n</math>).<br />
</blockquote><br />
Two examples were given:<br />
<br />
<math>\begin{aligned}<br />
P & =\left[\begin{array}{rrr}<br />
2 & 3 & 4\\<br />
3 & 1 & 5<br />
\end{array}\right],\ \ \ Q=\left[\begin{array}{rr}<br />
2 & 3\\<br />
4 & 3\\<br />
1 & 5<br />
\end{array}\right],\end{aligned}</math><br />
<br />
matrices of dimensions <math>2\times3</math> and <math>3\times2</math> respectively.<br />
<br />
Why study matrices for econometrics? Basically because a data set of several variables, e.g. on the weights and heights of 12 students, can be thought of as a matrix:<br />
<br />
<math>\begin{aligned}<br />
D & =\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]\end{aligned}</math><br />
<br />
The properties of matrices can then be used to facilitate answering all the usual questions of econometrics - list not given here!<br />
<br />
Calculations with matrices with explicit numerical elements, as in the examples above is called matrix ''arithmetic''. Matrix ''algebra'' is the algebra of matrices where the elements are not made explicit: this is what is really required for econometrics, as we shall see.<br />
<br />
As an example of this, a <math>2\times3</math> matrix might be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{ccc}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\end{aligned}</math><br />
<br />
and would equal <math>P</math> above if the collection of <math>a_{ij}</math> were given appropriate numerical values.<br />
<br />
A general <math>m\times n</math> matrix <math>A</math> can be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
There is also a ''typical element ''notation for matrices:<br />
<br />
<math>\begin{aligned}<br />
A & =\left\Vert a_{ij}\right\Vert ,\ \ \ \ \ i=1,...,m,j=1,...,n,\end{aligned}</math><br />
<br />
so that <math>a_{ij}</math> is the element at the intersection of the <math>i</math>th row and <math>j</math>th column in <math>A.</math><br />
<br />
When <math>m\neq n,</math> <math>A</math> is a ''rectangular ''matrix; when <math>m=n,</math> <math>A</math> is <math>m\times m</math> or <math>n\times n,</math> and <math>A</math> is a square matrix, having the same number of rows or columns.<br />
<br />
== Rows, columns and vectors ==<br />
<br />
Clearly, there is no reason why <math>m</math> or <math>n</math> cannot equal 1: so, an <math>m\times n</math> matrix with <math>n=1,</math> i.e. with one column, is usually called a column vector. Similarly, a matrix with one row is a row vector.<br />
<br />
There are a lot of advantages to thinking of matrices as collections of row or column vectors, as we shall see. As an example, define the <math>2\times1</math> column vectors:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a} & =\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\mathbf{,\ \ \ b}=\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
and arrange as the columns of the <math>2\times2</math> matrix<br />
<br />
<math>A=\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\label{eq:axy}</math><br />
<br />
In general, a column vector <math>\mathbf{x}</math> with <math>n</math> elements can be written as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\end{aligned}</math><br />
<br />
What happens when both <math>m</math> and <math>n</math> are equal to <math>1?</math> Then, <math>A</math> is a <math>1\times1</math> matrix, but it is also considered to be a real number, or ''scalar'' in the language of linear algebra:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[a_{11}\right]=a_{11}.\end{aligned}</math><br />
<br />
This is perhaps a little odd, but turns out to be a useful convention in a number of situations.<br />
<br />
== Transposition of vectors ==<br />
<br />
The ''rows'' of the matrix <math>A</math> in equation (1) can be seen as elements of column vectors, say:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c} & =\left[\begin{array}{r}<br />
6\\<br />
2<br />
\end{array}\right],\ \ \ \boldsymbol{d}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
This representation of row vectors as column vectors is a bit clumsy, so some transformation which converts a column vector into a row vector, and vice versa would be useful. The process of converting a column vector <math>\mathbf{a}</math> into a row vector is called ''transposition, ''and the transposed version of <math>\mathbf{c}</math> is denoted:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c}^{T} & =\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\end{aligned}</math><br />
<br />
the <math>^{T}</math> superscript denoting transposition. In practice, a prime, <math>^{\prime},</math> is used instead of <math>^{T}.</math> However, whilst the prime is much simpler to write than the <math>^{T}</math> sign, it is also much easier to lose track of in writing out long or complicated expressions. So, it is best initially to use <math>^{T}</math> to denote transposition rather than the prime <math>^{\prime}.</math><br />
<br />
<math>A</math> can then be written via its rows as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{r}<br />
\mathbf{c}^{T}\\<br />
\boldsymbol{d}^{T}<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
The same ideas can be applied to the matrices <math>P</math> and <math>Q.</math><br />
<br />
= Operations with matrices =<br />
<br />
== Addition, subtraction and scalar multiplication ==<br />
<br />
For vectors, addition and subtraction are defined only for vectors of the same dimensions. If:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right],\\<br />
\mathbf{x+y} & =\left[\begin{array}{c}<br />
x_{1}+y_{1}\\<br />
\vdots\\<br />
x_{n}+y_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{x-y}=\left[\begin{array}{c}<br />
x_{1}-y_{1}\\<br />
\vdots\\<br />
x_{n}-y_{n}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Clearly, the addition or subtraction operation is ''elementwise. ''If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> have different dimensions, there will be some elements left over once all the elements of the smaller dimensioned vector have been used up.<br />
<br />
Another operation is ''scalar multiplication'': if <math>\lambda</math> is a real number or scalar, the product <math>\lambda\mathbf{x}</math> is defined as: <math>\lambda\mathbf{x}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right],</math> so that every element of <math>\mathbf{x}</math> is multiplied by the same scalar <math>\lambda.</math><br />
<br />
The two types of operation can be combined into the ''linear combination'' of vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>: <math>\lambda\mathbf{x}+\mu\mathbf{y}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right]+\left[\begin{array}{c}<br />
\mu y_{1}\\<br />
\vdots\\<br />
\mu y_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\lambda x_{1}+\mu y_{1}\\<br />
\vdots\\<br />
\lambda x_{n}+\mu y_{n}<br />
\end{array}\right].</math> Equally, one can define the linear combination of vectors <math>\mathbf{x,y,}\ldots,\mathbf{z}</math> by scalars <math>\lambda,\mu,\ldots,\nu</math> as: <math>\lambda\mathbf{x}+\mu\mathbf{y}+\ldots+\nu\mathbf{z}</math> with typical element: <math>\lambda x_{i}+\mu y_{i}+\ldots+\nu z_{i},</math> provided that all the vectors have the same dimension.<br />
<br />
For matrices, these ideas carry over immediately: apply to each column of the matrices involved. For example, if <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right]</math> and <math>B=\left[\begin{array}{rrr}<br />
\mathbf{b}_{1} & \ldots & \mathbf{b}_{n}\end{array}\right],</math> both <math>m\times n,</math> then addition and subtraction are defined elementwise, as for vectors:<br />
<br />
<math>\begin{aligned}<br />
A+B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}+\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}+\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}+b_{ij}\right\Vert ,\\<br />
A-B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}-\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}-\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}-b_{ij}\right\Vert .\end{aligned}</math><br />
<br />
Scalar multiplication of <math>A</math> by <math>\lambda</math> involves multiplying every column vector of <math>A</math> by <math>\lambda,</math> and therefore multiplying every element of <math>A</math> by <math>\lambda</math>: <math>\lambda A=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1} & \ldots & \lambda\mathbf{a}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}\right\Vert .</math> With the same idea for <math>B,</math> the linear combination of <math>A</math> and <math>B</math> by <math>\lambda</math> and <math>\mu</math> is: <math>\lambda A+\mu B=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1}+\mu\mathbf{b}_{1} & \ldots & \lambda\mathbf{a}_{n}+\mu\mathbf{b}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}+\mu b_{ij}\right\Vert .</math><br />
<br />
For example, consider the matrices: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right]</math> with <math>\lambda=1,</math> <math>\mu=-2:</math> then:<br />
<br />
<math>\begin{aligned}<br />
\lambda A+\mu B & = & A-2B\\<br />
& = & \left[\begin{array}{cc}<br />
4 & 0\\<br />
1 & 7<br />
\end{array}\right].\end{aligned}</math><br />
<br />
== Matrix - vector products ==<br />
<br />
=== Inner product ===<br />
<br />
The simplest form of a matrix vector product is the case where <math>A</math> consists of one row, so that <math>A</math> is <math>1\times n</math>: <math>A=\mathbf{a}^{T}=\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right].</math> If <math>\mathbf{x}</math> is an <math>n\times1</math> vector: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],</math> the product <math>A\mathbf{x}=\mathbf{a}^{T}\mathbf{x}</math> is called the ''inner product'' and is defined as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a}^{T}\mathbf{x} & =a_{1}x_{1}+\ldots+a_{n}x_{n}.\end{aligned}</math><br />
<br />
One can see that the definition amounts to multiplying corresponding elements in <math>\mathbf{a}</math> and <math>\mathbf{x,}</math> and adding up the resultant products. Writing: <math>\mathbf{a}^{T}\mathbf{x=}\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=a_{1}x_{1}+\ldots+a_{n}x_{n}</math> motivates the familiar description of the ''across and down rule ''for this product: ''across and down'' is the ’multiply corresponding elements’ part of the definition.<br />
<br />
Notice that the result of the inner product is a real number, for example: <math>\mathbf{c}^{T}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{c}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=36+6=42.</math><br />
<br />
In general, in the product <math>\mathbf{a}^{T}\mathbf{x,}</math> <math>\mathbf{a}</math> and <math>\mathbf{x}</math> must have the same number of elements, <math>n</math> say, for the product to be defined. If <math>\mathbf{a}</math> and <math>\mathbf{x}</math> had different numbers of elements, there would be some elements of <math>\mathbf{a}</math> or <math>\mathbf{x}</math> left over or not used in the product: e.g.: <math>\mathbf{b}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{x=}\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> When the inner product of two vectors is defined, the vectors are said to be ''conformable''.<br />
<br />
== Orthogonality ==<br />
<br />
Two vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math> with the property that <math>\mathbf{x}^{T}\mathbf{y}=0</math> are said to be orthogonal to each other. For example, if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
-1\\<br />
1<br />
\end{array}\right],</math> it is clear that <math>\mathbf{x}^{T}\mathbf{y}=0.</math> This seems a rather innocuous definition, and yet the idea of orthogonality turns out to be extremely important in econometrics.<br />
<br />
If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> are thought of as points in <math>R^{2},</math> and arrows are drawn from the origin to <math>\mathbf{x}</math> and to <math>\mathbf{y,}</math> then the two arrows are perpendicular to each other - see Figure 1. If <math>\mathbf{y}</math> were defined as: <math>\mathbf{y}=\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right],</math> the position of the <math>\mathbf{y}</math> vector and the corresponding arrow would change, but the perpendicularity property would still hold.<br />
<br />
Figure 1<br />
<br />
[[File:orthy_example.png]]<br />
<br />
<br />
=== Matrix - vector products ===<br />
<br />
Since the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> has two rows, now denoted <math>\boldsymbol{\alpha}_{1}^{T}</math> and <math>\boldsymbol{\alpha}_{2}^{T},</math> there are two possible inner products with the vector:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]:\\<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x} & = & 42,\ \ \ \ \ \boldsymbol{\alpha}_{2}^{T}\mathbf{x}=33.\end{aligned}</math><br />
<br />
Assembling the two inner product values into a <math>2\times1</math> vector defines the product of the matrix <math>A</math> with the vector <math>\mathbf{x}</math>: <math>A\mathbf{x}=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x}\\<br />
\boldsymbol{\alpha}_{2}^{T}\mathbf{x}<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right].</math> Focussing only on the part: <math>\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right],</math> one can see that each element of <math>A\mathbf{x}</math> is obtained from an ''across and down'' argument.<br />
<br />
Sometimes this product is described as forming a ''linear combination ''of the columns of <math>A</math> using the scalar elements in <math>\mathbf{x}</math>: <math>A\mathbf{x}=6\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]+3\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].</math> More generally, if:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right],\ \ \ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
\lambda\\<br />
\mu<br />
\end{array}\right],\\<br />
A\mathbf{x} & = & \lambda\mathbf{a}+\mu\mathbf{b.}\end{aligned}</math><br />
<br />
The general version of these ideas for an <math>m\times n</math> matrix <math>A</math>: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]=\left[\begin{array}{rrrr}<br />
\mathbf{a}_{1} & \mathbf{a}_{2} & \ldots & \mathbf{a}_{n}\end{array}\right].</math> is straightforward. If <math>\mathbf{x}</math> is an <math>n\times1</math> vector, then the vector <math>A\mathbf{x}</math> is, by the ''across and down'' rule:<br />
<br />
<math>A\mathbf{x}=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
x_{2}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
a_{11}x_{1}+\ldots+a_{1n}x_{n}\\<br />
a_{21}x_{1}+\ldots+a_{2n}x_{n}\\<br />
\vdots\\<br />
a_{m1}x_{1}+\ldots+a_{mn}x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\sum\limits _{j=1}^{n}a_{1j}x_{j}\\<br />
\sum\limits _{j=1}^{n}a_{2j}x_{j}\\<br />
\vdots\\<br />
\sum\limits _{j=1}^{n}a_{mj}x_{j}<br />
\end{array}\right],\label{eq:ab}</math><br />
<br />
so that the typical element, the <math>i</math>th, is <math>\sum\limits _{j=1}^{n}a_{ij}x_{j}.</math> Equally, <math>A\mathbf{x}</math> is the linear combination <math>\mathbf{a}_{1}x_{1}+\ldots+\mathbf{a}_{n}x_{n}</math> of the columns of <math>A.</math><br />
<br />
== Matrix - matrix products ==<br />
<br />
Suppose that <math>A</math> is <math>m\times n,</math> with columns <math>\mathbf{a}_{1},\ldots,\mathbf{a}_{n},</math> and <math>B</math> is <math>n\times r,</math> with columns <math>\mathbf{b}_{1},\ldots,\mathbf{b}_{r}.</math> Clearly, each product <math>A\mathbf{b}_{1},...,A\mathbf{b}_{r}</math> exists, and is <math>m\times1.</math> These products can be arranged as the columns of a matrix as <math>\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]</math> and this matrix is ''defined'' to be the product <math>C</math> of the matrices <math>A</math> and <math>B</math>: <math>C=\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]=AB.</math> By construction, this must be an <math>m\times r</math> matrix, since each column is <math>m\times1</math> and there are <math>r</math> columns.<br />
<br />
This is not the usual presentation of the definition of the product of two matrices, which relies on the ''across and down rule'' mentioned earlier, and focusses on the elements of each matrix <math>A</math> and <math>B.</math> Set:<br />
<br />
<math>\begin{aligned}<br />
B & = & \left[\begin{array}{rrrr}<br />
\mathbf{b}_{1} & \mathbf{b}_{2} & \ldots & \mathbf{b}_{r}\end{array}\right]\ \ \ \ \ \ \ \text{(by columns)}\\<br />
& = & \left\Vert b_{ik}\right\Vert ,\ \ \ \ \ i=1,...,n,k=1,...,r\ \ \ \ \ \ \ \text{(typical element)}\\<br />
& = & \left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2r}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nr}<br />
\end{array}\right]\ \ \ \ \ \ \ \text{(the array)}\end{aligned}</math><br />
<br />
What does the typical element of the <math>m\times r</math> matrix <math>C</math> look like? Start with the <math>k</math>th column of <math>C,</math> which is <math>A\mathbf{b}_{k}.</math> The <math>i</math>th element in <math>A\mathbf{b}_{k}</math> is, from equation (2), the inner product of the elements of the <math>i</math>th row in <math>A</math>: <math>\left[\begin{array}{rrrr}<br />
a_{i1} & a_{i2} & \ldots & a_{in}\end{array}\right],</math> with the elements of <math>\mathbf{b}_{k},</math> so that the inner product is: <math>a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math><br />
<br />
So, the <math>ik</math>th element of <math>C</math> is: <math>c_{ik}=a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math> We can see this arising from an ''across and down'' calculation by writing:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\label{eq:c_ab}\\<br />
& = & \left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{rrrrrr}<br />
b_{11} & b_{12} & \ldots & b_{1k} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2k} & \ldots & b_{2r}\\<br />
\vdots & \vdots & & \vdots & & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nk} & \ldots & b_{nr}<br />
\end{array}\right]\\<br />
& = & \left\Vert \sum_{j=1}^{n}a_{ij}b_{jk}\right\Vert .\end{aligned}</math><br />
<br />
These ideas are simple, but a little tedious. Numerical examples are equally tedious! As an example, using: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],</math> we can find the matrix <math>B</math> such that<br />
<br />
# the first column of <math>AB</math> adds together the columns of <math>A,</math><br />
# the second column is the difference of the first and second columns of <math>A,</math><br />
# the third column is <math>2\times</math> the first column of <math>A,</math><br />
# the fourth column is zero.<br />
<br />
It is easy to check that <math>B</math> is: <math>B=\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]</math> and that:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\\<br />
& = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{cccc}<br />
8 & 4 & 12 & 0\\<br />
8 & -2 & 6 & 0<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Arithmetic calculations of matrix products almost always use the elementwise across and down formula. However, there are many situations in econometrics where algebraic rather than arithmetic arguments are required. In these cases, the viewpoint of matrix multiplication as linear combinations of columns is much more powerful.<br />
<br />
Clearly one can give many more examples of different dimensions and complexities - but the same basic rules apply. To multiply two matrices <math>A</math> and <math>B</math> together, the number of columns in <math>A</math> must match the number of rows in <math>B</math> - this is ''conformability'' in action again. The resulting product will have number of rows equal to the number in <math>A</math> and number of columns equal to the number in <math>B.</math><br />
<br />
If this conformability rule does not hold, then the product of <math>A</math> and <math>B</math> is not defined.<br />
<br />
== Matlab ==<br />
<br />
One should also say that as the dimensions of the matrices increases, so the tediousness of the calculations increase. The solution to this for numerical calculation is to appeal to the computer. Programs like Matlab and Excel (and a number of others, some of them free) resolve this difficulty easily.<br />
<br />
In Matlab, symbols for row or column vectors do not need any particular differentiation: they are distinguished by how they are defined. For example, the following Matlab commands define <code>rowvec </code>as a <math>1\times4</math> vector, and <code>colvec</code> as a <math>4\times1</math> vector, then display the contents of these variables, and do a calculation:<br />
<br />
&gt;&gt; rowvec = [1 2 3 4]; <br />
<br />
&gt;&gt; colvec = [1;2;3;4]; <br />
<br />
&gt;&gt; rowvec<br />
<br />
rowvec =<br />
<br />
1 2 3 4<br />
<br />
&gt;&gt; colvec<br />
<br />
colvec =<br />
<br />
1 2 3 4 <br />
<br />
&gt;&gt; rowvec*colvec<br />
<br />
ans =<br />
<br />
30 <br />
<br />
So, the semi-colon indicates the end of a row in a matrix or vector; it can be replaced by a carriage return. Notice the difference in how a row vector and a column vector is defined. One can see that the product <code>rowvec*colvec</code> is well defined, just because <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector.<br />
<br />
Matlab also allows elementwise multiplication of two vectors using the <math>\centerdot\ast</math> operator: if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
x_{1}\\<br />
x_{2}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
y_{1}\\<br />
y_{2}<br />
\end{array}\right],</math> then: <math>\mathbf{x}\centerdot\ast\mathbf{y}=\left[\begin{array}{r}<br />
x_{1}y_{1}\\<br />
x_{2}y_{2}<br />
\end{array}\right]</math> and one can see that the inner product of <math>\mathbf{x}</math> and <math>\mathbf{y}</math> can be obtained as the sum of the elements of <math>\mathbf{x}</math> and <math>\mathbf{y.}</math> In Matlab, this would be obtained as: <math>\text{sum}\left(\mathbf{x}\centerdot\ast\mathbf{y}\right).</math><br />
<br />
In the example above, this calculation fails since <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector:<br />
<br />
&gt;&gt; sum(rowvec .* colvec) ??? <br />
<br />
Error using ==&gt; times Matrix dimensions must agree. <br />
<br />
For this to work, <code>rowvec</code> would have to be transposed as <code>rowvec’</code>, so that transposition in Matlab is very natural.<br />
<br />
Allowing for such difficulties, matrix multiplication in Matlab is very simple:<br />
<br />
&gt;&gt; A = [6 2; 3 5];<br />
<br />
&gt;&gt; B = [1 1 2 0;1 -1 0 0];<br />
<br />
&gt;&gt; C = A * B; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
8 4 1<br />
<br />
2 0 8 <br />
<br />
-2 6 0 <br />
<br />
Notice how the matrices are defined here through their rows. The <code>disp() </code>command displays the contents of the object referred to.<br />
<br />
It is less natural in Matlab to define matrices by columns - a typical example of how mathematics and computing have conflicts of notation. However, once columns <math>\mathbf{a}</math> and <math>\mathbf{b}</math> have been defined, the concatenation operation <math>\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]</math> collects the columns into a matrix:<br />
<br />
&gt;&gt; a = [6;2]; <br />
<br />
&gt;&gt; b = [3;5]; <br />
<br />
&gt;&gt; C = [a b]; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
6 3 <br />
<br />
2 5 <br />
<br />
Notice that the <code>disp(C)</code> command does not label the result that is printed out. Simply typing <code>C</code> would preface the output by <code>C =</code>.<br />
<br />
== Pre and Post Multiplication ==<br />
<br />
If <math>C=AB,</math> as above, say that <math>B</math> is ''pre-multiplied ''by <math>A</math> to get <math>C,</math> and that <math>A</math> is ''post-multiplied ''by <math>B</math> to get <math>C.</math><br />
<br />
This distinction between ''pre ''and ''post ''multiplication is important, in the following sense. Suppose that <math>A</math> and <math>B</math> are matrices such that the products <math>AB</math> and <math>BA</math> are both defined. If <math>A</math> is <math>m\times n,</math> <math>B</math> must have <math>n</math> rows for <math>AB</math> to be defined. For <math>BA</math> to be defined, <math>B</math> must have <math>m</math> columns to match the <math>m</math> rows in <math>A.</math> So, <math>AB</math> and <math>BA</math> are both defined if <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times m.</math><br />
<br />
Even when both products are defined, there is no reason for the two products coincide. The first thing to notice is that <math>AB</math> is a square, <math>m\times m,</math> matrix, whilst <math>BA</math> is a square, <math>n\times n,</math> matrix. Different sized matrices cannot be equal. To illustrate, use the matrices: <math>B_{2}=\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right],\ \ \ C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]:</math><br />
<br />
<math>\begin{aligned}<br />
B_{2}C & = & \left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]=\left[\begin{array}{rrr}<br />
27 & -3 & -15\\<br />
27 & 29 & -11\\<br />
-15 & -1 & 8<br />
\end{array}\right],\\<br />
CB_{2} & = & \left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
49 & -11\\<br />
31 & 15<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Even when <math>m=n,</math> so that <math>AB</math> and <math>BA</math> are both <math>m\times m</math> matrices, the products can differ: for example:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right],\\<br />
AB & = & \left[\begin{array}{cc}<br />
8 & 4\\<br />
8 & -2<br />
\end{array}\right],\ \ \ \ \ BA=\left[\begin{array}{cc}<br />
9 & 7\\<br />
3 & -3<br />
\end{array}\right].\end{aligned}</math><br />
<br />
In cases where <math>AB=BA,</math> the matrices <math>A</math> and <math>B</math> are said to ''commute''.<br />
<br />
== Transposition ==<br />
<br />
A column vector <math>\mathbf{x}</math> can be converted to a row vector <math>\mathbf{x}^{T}</math> by transposition: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \ \ \mathbf{x}^{T}=\left[\begin{array}{rrr}<br />
x_{1} & \ldots & x_{n}\end{array}\right].</math> Transposing <math>\mathbf{x}^{T}</math> as <math>\left(\mathbf{x}^{T}\right)^{T}</math> reproduces the original vector <math>\mathbf{x.}</math> How do these ideas carry over to matrices?<br />
<br />
If the <math>m\times n</math> matrix <math>A</math> can be written as <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right],</math> the transpose of <math>A,</math> <math>A^{T},</math> is defined as the matrix whose ''rows'' are <math>\mathbf{a}_{i}^{T}</math>: <math>A^{T}=\left[\begin{array}{c}<br />
\mathbf{a}_{1}^{T}\\<br />
\vdots\\<br />
\mathbf{a}_{n}^{T}<br />
\end{array}\right].</math> In terms of elements, if: <math>\mathbf{a}_{i}=\left[\begin{array}{c}<br />
a_{1i}\\<br />
\vdots\\<br />
a_{ni}<br />
\end{array}\right]</math> then: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right],\ \ \ \ \ A^{T}=\left[\begin{array}{rrrrr}<br />
a_{11} & \ldots & a_{i1} & \ldots & a_{m1}\\<br />
a_{12} & \ldots & a_{i2} & \ldots & a_{m2}\\<br />
\vdots & & \vdots & & \vdots\\<br />
a_{1n} & \ldots & a_{in} & \ldots & a_{mn}<br />
\end{array}\right].</math> One can see that the first column of <math>A</math> has now become the first row of <math>A^{T}.</math> Notice too that <math>A^{T}</math> is an <math>n\times m</math> matrix if <math>A</math> is an <math>m\times n</math> matrix.<br />
<br />
Transposing <math>A^{T}</math> takes the first column of <math>A^{T}</math> and writes it as a row, which coincides with the first row of <math>A.</math> The same argument applies to the other columns of <math>A^{T},</math> so that <math>\left(A^{T}\right)^{T}=A.</math><br />
<br />
=== The product rule for transposition ===<br />
<br />
This states that if <math>C=AB,</math> then <math>C^{T}=B^{T}A^{T}.</math><br />
<br />
How to see this? Consider the following example: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & b_{13} & b_{14}\\<br />
b_{21} & b_{22} & b_{23} & b_{24}\\<br />
b_{31} & b_{32} & b_{33} & b_{34}<br />
\end{array}\right]</math> where:<br />
<br />
<math>c_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=\sum_{k=1}^{3}a_{2k}b_{k3}.\label{eq:c23}</math><br />
<br />
One can see that: <math>B^{T}A^{T}=\left[\begin{array}{rrr}<br />
b_{11} & b_{21} & b_{31}\\<br />
b_{12} & b_{22} & b_{32}\\<br />
b_{13} & b_{23} & b_{33}\\<br />
b_{14} & b_{24} & b_{34}<br />
\end{array}\right]\left[\begin{array}{rr}<br />
a_{11} & a_{21}\\<br />
a_{12} & a_{22}\\<br />
a_{13} & a_{23}<br />
\end{array}\right]</math> and that the <math>\left(3,2\right)</math> element of this product is actually <math>c_{23}</math>: <math>b_{13}a_{21}+b_{23}a_{22}+b_{33}a_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=c_{23}.</math> In summation notation, we see that from <math>B^{T}A^{T}</math>: <math>c_{23}=\sum_{k=1}^{3}b_{k3}a_{2k},</math> where the position of the index of summation is due to the transposition. So, in summation notation, the calculation of <math>c_{23}</math> from <math>B^{T}A^{T}</math> equals that from equation (4).<br />
<br />
More generally, the <math>\left(i,j\right)</math> element of <math>AB</math>: <math>\sum_{k=1}^{3}a_{ik}b_{kj}</math> is the <math>\left(j,i\right)</math> element of <math>B^{T}A^{T}.</math> But this means that <math>B^{T}A^{T}</math> must be the transpose of <math>AB,</math> since the elements in the <math>i</math>th row of <math>AB</math> are being written in the <math>i</math>th column of <math>B^{T}A^{T}.</math><br />
<br />
This ''Product Rule for Transposition'' can be applied again to find the transpose <math>\left(C^{T}\right)^{T}</math> of <math>C^{T}</math>: <math>\left(C^{T}\right)^{T}=\left(B^{T}A^{T}\right)^{T}=\left(A^{T}\right)^{T}\left(B^{T}\right)^{T}=AB=C.</math><br />
<br />
= Special Types of Matrix =<br />
<br />
== The zero matrix ==<br />
<br />
The most obvious special type of matrix is one whose elements are all zeros. In typical element notation, the zero matrix is: <math>0=\left\Vert 0\right\Vert .</math> Since there is no indexing on the elements, it is not obvious what the dimension of this matrix is, Sometimes one writes <math>0_{mn}</math> to indicate a zero matrix of dimension <math>m\times n.</math> The same ideas apply to vectors whose elements are all zero.<br />
<br />
The effect of the zero matrix in any product that is defined is simple: <math>0A=0,\ \ \ \ \ B0=0.</math> This is easy to check using the across and down rule.<br />
<br />
== The identity or unit matrix ==<br />
<br />
Vectors of the form:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{r}<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }2\ \text{dimensions}\\<br />
\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }3\ \text{dimensions}\\<br />
\left[\begin{array}{r}<br />
1\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1\\<br />
0\\<br />
\vdots\\<br />
0<br />
\end{array}\right],\ldots,\left[\begin{array}{r}<br />
0\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }n\ \text{dimensions}\end{aligned}</math><br />
<br />
are called coordinate vectors. They are often given a characteristic notation, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> in <math>n</math> dimensions. When arranged as columns of a matrix in the natural order, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> a matrix with a characteristic pattern elements emerges, with a special notation:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{rr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2}\end{array}\right] & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=I_{2}\\<br />
\left[\begin{array}{rrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \mathbf{e}_{3}\end{array}\right] & = & \left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 1 & 0\\<br />
0 & 0 & 1<br />
\end{array}\right]=I_{3}\\<br />
\left[\begin{array}{rrrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \ldots & \mathbf{e}_{n}\end{array}\right] & = & \left[\begin{array}{rrrr}<br />
1 & 0 & \ldots & 0\\<br />
0 & 1 & \ldots & 0\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
0 & 0 & \ldots & 1<br />
\end{array}\right]=I_{n}.\end{aligned}</math><br />
<br />
The ''diagonal'' of this matrix is where the 1 elements are located, and every other element is zero.<br />
<br />
Consider the effect of <math>I_{2}</math> on the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> by both pre and post multiplication:<br />
<br />
<math>\begin{aligned}<br />
I_{2}A & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\\<br />
AI_{2} & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\end{aligned}</math><br />
<br />
as is easily checked by the across and down rule.<br />
<br />
Because any matrix is left unchanged by pre or post multiplication by an appropriately dimensioned <math>I_{n},</math> <math>I_{n}</math> is called an ''identity matrix of dimension ''<math>n.</math> Sometimes it is called a ''unit matrix of dimension ''<math>n.</math> Notice that <math>I_{n}</math> is necessarily a square matrix.<br />
<br />
== Diagonal matrices ==<br />
<br />
The identity matrix is an example of a diagonal matrix, a matrix whose elements are all zero except for those on the diagonal. Usually diagonal matrices are taken to be square, for example: <math>D=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 2 & 0\\<br />
0 & 0 & 3<br />
\end{array}\right].</math> They also produce characteristic effects when pre or post multiplying another matrix.<br />
<br />
Consider the diagonal matrix: <math>B=\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]</math> and the products <math>AB,</math> <math>BA</math> for <math>A</math> as defined in the previous section:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & -4\\<br />
6 & -10<br />
\end{array}\right],\\<br />
BA & = & \left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & 4\\<br />
-6 & -10<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Comparing the results, we can deduce that post multiplication by a diagonal matrix multiplies each column of <math>A</math> by the corresponding diagonal element, whereas pre multiplication multiplies each row by the corresponding diagonal element.<br />
<br />
== Symmetric matrices ==<br />
<br />
Symmetric matrices are matrices having the property that <math>A=A^{T}.</math> Notice that such matrices must be square, since if <math>A</math> is <math>m\times n,</math> <math>A^{T}</math> is <math>n\times m,</math> and to have equality of <math>A</math> and <math>A^{T},</math> they must have the same dimension, so that <math>m=n</math> is required.<br />
<br />
Suppose that <math>A</math> is a <math>3\times3</math> symmetric matrix, with typical element <math>a_{ij}</math>: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right],</math> so that: <math>A^{T}=\left[\begin{array}{rrr}<br />
a_{11} & a_{21} & a_{31}\\<br />
a_{12} & a_{22} & a_{32}\\<br />
a_{13} & a_{23} & a_{33}<br />
\end{array}\right].</math> Equality of matrices is defined as equality of all elements. This is fine on the diagonals, since <math>A</math> and <math>A^{T}</math> have the same diagonal elements. For the off diagonal elements, we end up with the requirements: <math>a_{12}=a_{21},\ \ \ a_{13}=a_{31},\ \ \ a_{23}=a_{32}</math> or more generally: <math>a_{ij}=a_{ji}\ \ \ \ \ \text{for}\ i\neq j.</math><br />
<br />
The effect of this conclusion is that in a symmetric matrix, the ’triangle’ of above diagonal elements coincides with the triangle of below diagonal elements. It is as if the upper triangle is folded over the diagonal to become the lower triangle.<br />
<br />
A simple example is: <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
2 & 1<br />
\end{array}\right].</math> A more complicated example uses the <math>2\times3</math> matrix <math>C</math>: <math>C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]</math> and calculates the <math>3\times3</math> matrix:<br />
<br />
<math>\begin{aligned}<br />
C^{T}C & = & \left[\begin{array}{cc}<br />
6 & 3\\<br />
2 & 5\\<br />
-3 & -1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{rrr}<br />
45 & 27 & -21\\<br />
27 & 29 & -11\\<br />
-21 & -11 & 10<br />
\end{array}\right],\end{aligned}</math><br />
<br />
which is clearly symmetric.<br />
<br />
This illustrates the general proposition that if <math>A</math> is an <math>m\times n</math> matrix, the product <math>A^{T}A</math> is a symmetric <math>n\times n</math> matrix. Proof? Compute the transpose of <math>A^{T}A</math> using the product rule for transposition: <math>\left(A^{T}A\right)^{T}=A^{T}\left(A^{T}\right)^{T}=A^{T}A.</math> Since <math>A^{T}A</math> is equal to its transpose, it must be a symmetric matrix. Such symmetric matrices appear frequently in econometrics.<br />
<br />
It should be clear that diagonal matrices are symmetric, since all their off-diagonal elements are equal (zero), and thence the identity matrix <math>I_{n}</math> is also symmetric.<br />
<br />
== The outer product ==<br />
<br />
The inner product of two <math>n\times1</math> vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>, <math>\mathbf{x}^{T}\mathbf{y}</math>, is automatically a <math>1\times1</math> quantity, a scalar, although it can be interpreted as a <math>1\times1</math> matrix, a matrix with a single element.<br />
<br />
Suppose one considered the product of <math>\mathbf{x}</math> with <math>\mathbf{x}^{T}.</math> Is this defined? If <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times r,</math> then the product <math>AB</math> is <math>m\times r.</math> Applying this logic to <math>\mathbf{xx}^{T},</math> this is <math>\left(n\times1\right)\left(1\times n\right),</math> so the resulting product ''is ''defined, and is an <math>n\times n</math>'' matrix'' - the ''outer product'' of <math>\mathbf{x}</math> and <math>\mathbf{x}^{T},</math> the word ’outer’ being used to distinguish from the inner product.<br />
<br />
How does the across and down rule work here? Suppose that: <math>\mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> Then: <math>\mathbf{xx}^{T}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right].</math> Here, there is <math>1</math> element in row one of the ’matrix’ <math>\mathbf{x,}</math> and <math>1</math> element in column one of the matrix <math>\mathbf{x}^{T},</math> so the across and down rule still works - it is just that there is only one product per row and column combination. So: <math>\mathbf{xx}^{T}=\left[\begin{array}{cc}<br />
36 & 18\\<br />
18 & 9<br />
\end{array}\right],</math> and it is obvious from this that <math>\mathbf{xx}^{T}</math> is a symmetric matrix.<br />
<br />
One can see that this outer product need not be restricted to vectors of the same dimension. If <math>\mathbf{x}</math> is <math>n\times1</math> and <math>\mathbf{y}</math> is <math>m\times1,</math> then: <math>\mathbf{xy}^{T}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
y_{1} & \ldots & y_{m}\end{array}\right]=\left[\begin{array}{rrrr}<br />
x_{1}y_{1} & x_{1}y_{2} & \ldots & x_{1}y_{m}\\<br />
x_{2}y_{1} & x_{2}y_{2} & \ldots & x_{2}y_{m}\\<br />
\\<br />
x_{n}y_{1} & x_{n}y_{2} & \ldots & x_{n}y_{m}<br />
\end{array}\right].</math> So, <math>\mathbf{xy}^{T}</math> is <math>n\times m,</math> and consists of rows which are <math>\mathbf{y}^{T}</math> multiplied by an element of the <math>\mathbf{x}</math> vector.<br />
<br />
Another interesting and useful example involves a vector with every element equal to <math>1</math>: <math>\mathbf{1}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right].</math> Sometimes this is written as <math>\mathbf{1}_{n}</math> to indicate an <math>n\times1</math> vector, and is called the ''sum vector''. Why? Consider the impact of <math>\mathbf{1}_{2}</math> on the <math>2\times1</math> vector <math>\mathbf{x}</math> used above: <math>\mathbf{1}_{2}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=9,</math> i.e. an inner product of <math>\mathbf{x}</math> with the sum vector is the sum of the elements of <math>\mathbf{x.}</math> Dividing through by the number of elements in <math>\mathbf{x}</math> produces the average of the elements of <math>\mathbf{x}</math> - i.e. the ’sample mean’ of the elements of <math>\mathbf{x.}</math><br />
<br />
The outer product of <math>\mathbf{x}</math> with <math>\mathbf{1}_{2}</math> is also interesting:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{1}_{2}\mathbf{x}^{T} & = & \left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 3\\<br />
6 & 3<br />
\end{array}\right],\\<br />
\mathbf{x1}_{2}^{T} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 6\\<br />
3 & 3<br />
\end{array}\right],\end{aligned}</math><br />
<br />
showing that pre multiplication of an <math>\mathbf{x}^{T}</math> by <math>\mathbf{1}</math> repeats <math>\mathbf{x}^{T}</math> as rows of the product, whilst post multiplication of <math>\mathbf{x}</math> by <math>\mathbf{1}^{T}</math> repeats <math>\mathbf{x}</math> as the columns of the product.<br />
<br />
Finally: <math>\mathbf{1}_{n}\mathbf{1}_{n}^{T}=\left[\begin{array}{rrr}<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1<br />
\end{array}\right],</math> an <math>n\times n</math> matrix with every element equal to <math>1.</math> This type of matrix also appears in econometrics!<br />
<br />
== Triangular matrices ==<br />
<br />
A square ''lower triangular ''matrix has all elements above the main diagonal equal to zero, whilst a square ''upper triangular ''matrix has all elements below the main diagonal equal to zero. A simple example of a lower triangular matrix is: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & 0 & 0\\<br />
a_{21} & a_{22} & 0\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right].</math> Clearly, for this matrix, <math>A^{T}</math> is an upper triangular matrix.<br />
<br />
One can adapt the definition to rectangular matrices: for example, if two arbitrary rows are added to <math>A,</math> so that it becomes <math>5\times3,</math> it would still be considered lower triangular. Equally, if, for example, the third column of <math>A</math> above is removed, <math>A</math> is still considered lower triangular.<br />
<br />
Often, we use ''unit ''triangular matrices, where the diagonal elements are all equal to <math>1:</math> e.g.:<br />
<br />
<math>\left[\begin{array}{rrr}<br />
1 & 2 & 0\\<br />
0 & 1 & 1\\<br />
0 & 0 & 1<br />
\end{array}\right].\label{eq:lt_matrix}</math><br />
<br />
== Partitioned matrices ==<br />
<br />
Sometimes, especially with big matrices, it is useful to organise the elements of the matrix into components which are themselves matrices, for example: <math>B=\left[\begin{array}{rrrr}<br />
1 & 2 & 0 & 0\\<br />
8 & 3 & 0 & 0\\<br />
0 & 0 & 7 & 4\\<br />
0 & 0 & 6 & 5<br />
\end{array}\right]</math> Here it would be reasonable to write: <math>B=\left[\begin{array}{cc}<br />
B_{11} & 0\\<br />
0 & B_{22}<br />
\end{array}\right],</math> where <math>B_{ii},i=1,2,</math> represent <math>2\times2</math> matrices. <math>B</math> is an example of a ''partitioned matrix'': that is, an <math>m\times n</math> matrix <math>A</math> say: <math>A=\left\Vert a_{ij}\right\Vert ,</math> where the elements of <math>A</math> are organised into ''sub-matrices''. An example might be:<br />
<br />
<math>A=\left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right],\label{eq:partition_a}</math><br />
<br />
where the ''sub - matrices'' in the first row block have <math>r</math> rows, and therefore <math>m-r</math> rows in the second row block. The column blocks might be defined by (for example) 3 columns in the first column block, 4 in the second and <math>n-7</math> in the third column block.<br />
<br />
Another simple example might be: <math>A=\left[\begin{array}{rrr}<br />
A_{1} & A_{2} & A_{3}\end{array}\right],\ \ \ \ \ \mathbf{x=}\left[\begin{array}{c}<br />
\mathbf{x}_{1}\\<br />
\mathbf{x}_{2}\\<br />
\mathbf{x}_{3}<br />
\end{array}\right],</math> where <math>A</math> and therefore <math>A_{1},A_{2},A_{3}</math> have <math>m</math> rows, <math>A_{1}</math> has <math>n_{1}</math> columns, <math>A_{2}</math> has <math>n_{2}</math> columns, <math>A_{3}</math> has <math>n_{3}</math> columns. The ''subvectors'' in <math>\mathbf{x}</math> must have <math>n_{1},n_{2}</math> and <math>n_{3}</math> rows respectively, for the product <math>A\mathbf{x}</math> to exist.<br />
<br />
Suppose that <math>n_{1}+n_{2}+n_{3}=n,</math> so that <math>A</math> is <math>m\times n.</math> The <math>i</math>th element of <math>A\mathbf{x}</math> is: <math>\sum_{i=1}^{n}a_{ij}x_{j}</math> but the summation can be broken up into the first <math>n_{1}</math> terms: <math>\sum_{i=1}^{n_{1}}a_{ij}x_{j},</math> the next <math>n_{2}</math> terms: <math>\sum_{i=n_{1}+1}^{n_{1}+n_{2}}a_{ij}x_{j},</math> and the next <math>n_{3}</math> terms; <math>\sum_{i=n_{1}+n_{2}+1}^{n}a_{ij}x_{j}.</math><br />
<br />
The point about the use of partitioned matrices is that the product <math>A\mathbf{x}</math> can be represented as: <math>A\mathbf{x}=A_{1}\mathbf{x}_{1}+A_{2}\mathbf{x}_{2}+A\mathbf{x}_{3}</math> by applying the across and down rule to the submatrices and the subvectors, a much simpler representation than the use of summations.<br />
<br />
Each of the components is a conformable matrix-vector product: this is essential in any use of partitioned matrices to represent some matrix product. For example, using <math>A</math> from equation (6) and <math>B</math> as: <math>B=\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right],</math> it is easy to write:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right]\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{r}<br />
A_{11}B_{11}+A_{12}B_{21}+A_{13}B_{31}\\<br />
A_{21}B_{11}+A_{22}B_{21}+A_{23}B_{31}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
But, what are the row dimensions for the submatrices in <math>B?</math> What are the possible column dimensions for the submatrices in <math>B?</math><br />
<br />
= Matrices, vectors and econometrics =<br />
<br />
The data on weights and heights for 12 students in the data matrix: <math>D=\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]</math> would seem to be ideally suited for fitting a two variable regression model: <math>y_{i}=\alpha+\beta x_{i}+u_{i},\;\;\;\;\; i=1,...,12.</math> Here, the first column of <math>D</math> contains all the weight data, the data on the dependent variable <math>y_{i},</math> and so should be labelled <math>\mathbf{y.}</math> The second column of <math>D</math> contains all the data on the explanatory variable height, in the vector <math>\mathbf{x}</math> say, so that: <math>D=\left[\begin{array}{rr}<br />
\mathbf{y} & \mathbf{x}\end{array}\right].</math><br />
<br />
If we define a <math>12\times1</math> vector with every element <math>1</math>: <math>\mathbf{1}_{12}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right],</math> and a <math>12\times1</math> vector <math>\mathbf{u}</math> to contain the error terms; <math>\mathbf{u}=\left[\begin{array}{c}<br />
u_{1}\\<br />
\vdots\\<br />
u_{12}<br />
\end{array}\right]</math> the regression model can be written in terms of the three data vectors <math>\mathbf{y,1}_{12}</math> and <math>\mathbf{x}</math> as: <math>\mathbf{y}=\mathbf{1}_{12}\alpha+\mathbf{x}\beta+\mathbf{u.}</math> To see this, think of the <math>i</math>th elements of the vectors on the left and right hand sides.<br />
<br />
The standard next step is then to combine the data vectors for the explanatory variables into a matrix: <math>X=\left[\begin{array}{rr}<br />
\mathbf{1}_{12} & \mathbf{x}\end{array}\right],</math> and then define a <math>2\times1</math> vector <math>\boldsymbol{\delta}</math> to contain the parameters <math>\alpha,\beta</math> as: <math>\boldsymbol{\delta}=\left[\begin{array}{r}<br />
\alpha\\<br />
\beta<br />
\end{array}\right]</math> to give the data matrix representation of the regression model as: <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
<br />
For the purposes of developing the theory of regression, this is the most convenient form of the regression model. It can represent regression models with any number of explanatory variables, and thus any number of parameters. The obvious point is that a knowledge of vector and matrix operations is needed to use and understand this form.<br />
<br />
We shall see later that there are two particular matrix and vector quantities associated with a regression model. The first is the matrix <math>X^{T}X,</math> and the second the vector <math>X^{T}\mathbf{y.}</math> The following Matlab code snippet provides the numerical values of these quantities for the weight data:<br />
<br />
&gt;&gt; dset = load(’weights.mat’); <br />
<br />
&gt;&gt; xtx = dset.X’ * dset.X; <br />
<br />
&gt;&gt; xty = dset.X’ * dset.y; <br />
<br />
&gt;&gt; disp(xtx) <br />
<br />
12 802<br />
<br />
802 53792<br />
<br />
&gt;&gt; disp(xty)<br />
<br />
1850<br />
<br />
124528<br />
<br />
Hand calculation is of course possible, but not recommended.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Lnotes&diff=3043Lnotes2013-09-10T15:02:55Z<p>LG: </p>
<hr />
<div>= Matrices =<br />
<br />
In the PreSession Maths course, a matrix was defined as follows:<br />
<br />
<blockquote>A matrix is a rectangular array of numbers enclosed in parentheses, conventionally denoted by a capital letter. The number of rows (say <math>m</math>) and<br />
<br />
the number of columns (say <math>n</math>) determine the order of the matrix (<math>m</math> <math>\times</math> <math>n</math>).<br />
</blockquote><br />
Two examples were given:<br />
<br />
<math>\begin{aligned}<br />
P & =\left[\begin{array}{rrr}<br />
2 & 3 & 4\\<br />
3 & 1 & 5<br />
\end{array}\right],\ \ \ Q=\left[\begin{array}{rr}<br />
2 & 3\\<br />
4 & 3\\<br />
1 & 5<br />
\end{array}\right],\end{aligned}</math><br />
<br />
matrices of dimensions <math>2\times3</math> and <math>3\times2</math> respectively.<br />
<br />
Why study matrices for econometrics? Basically because a data set of several variables, e.g. on the weights and heights of 12 students, can be thought of as a matrix:<br />
<br />
<math>\begin{aligned}<br />
D & =\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]\end{aligned}</math><br />
<br />
The properties of matrices can then be used to facilitate answering all the usual questions of econometrics - list not given here!<br />
<br />
Calculations with matrices with explicit numerical elements, as in the examples above is called matrix ''arithmetic''. Matrix ''algebra'' is the algebra of matrices where the elements are not made explicit: this is what is really required for econometrics, as we shall see.<br />
<br />
As an example of this, a <math>2\times3</math> matrix might be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{ccc}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\end{aligned}</math><br />
<br />
and would equal <math>P</math> above if the collection of <math>a_{ij}</math> were given appropriate numerical values.<br />
<br />
A general <math>m\times n</math> matrix <math>A</math> can be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
There is also a ''typical element ''notation for matrices:<br />
<br />
<math>\begin{aligned}<br />
A & =\left\Vert a_{ij}\right\Vert ,\ \ \ \ \ i=1,...,m,j=1,...,n,\end{aligned}</math><br />
<br />
so that <math>a_{ij}</math> is the element at the intersection of the <math>i</math>th row and <math>j</math>th column in <math>A.</math><br />
<br />
When <math>m\neq n,</math> <math>A</math> is a ''rectangular ''matrix; when <math>m=n,</math> <math>A</math> is <math>m\times m</math> or <math>n\times n,</math> and <math>A</math> is a square matrix, having the same number of rows or columns.<br />
<br />
== Rows, columns and vectors ==<br />
<br />
Clearly, there is no reason why <math>m</math> or <math>n</math> cannot equal 1: so, an <math>m\times n</math> matrix with <math>n=1,</math> i.e. with one column, is usually called a column vector. Similarly, a matrix with one row is a row vector.<br />
<br />
There are a lot of advantages to thinking of matrices as collections of row or column vectors, as we shall see. As an example, define the <math>2\times1</math> column vectors:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a} & =\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\mathbf{,\ \ \ b}=\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
and arrange as the columns of the <math>2\times2</math> matrix<br />
<br />
<math>A=\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\label{eq:axy}</math><br />
<br />
In general, a column vector <math>\mathbf{x}</math> with <math>n</math> elements can be written as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\end{aligned}</math><br />
<br />
What happens when both <math>m</math> and <math>n</math> are equal to <math>1?</math> Then, <math>A</math> is a <math>1\times1</math> matrix, but it is also considered to be a real number, or ''scalar'' in the language of linear algebra:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[a_{11}\right]=a_{11}.\end{aligned}</math><br />
<br />
This is perhaps a little odd, but turns out to be a useful convention in a number of situations.<br />
<br />
== Transposition of vectors ==<br />
<br />
The ''rows'' of the matrix <math>A</math> in equation (1) can be seen as elements of column vectors, say:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c} & =\left[\begin{array}{r}<br />
6\\<br />
2<br />
\end{array}\right],\ \ \ \boldsymbol{d}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
This representation of row vectors as column vectors is a bit clumsy, so some transformation which converts a column vector into a row vector, and vice versa would be useful. The process of converting a column vector <math>\mathbf{a}</math> into a row vector is called ''transposition, ''and the transposed version of <math>\mathbf{c}</math> is denoted:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c}^{T} & =\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\end{aligned}</math><br />
<br />
the <math>^{T}</math> superscript denoting transposition. In practice, a prime, <math>^{\prime},</math> is used instead of <math>^{T}.</math> However, whilst the prime is much simpler to write than the <math>^{T}</math> sign, it is also much easier to lose track of in writing out long or complicated expressions. So, it is best initially to use <math>^{T}</math> to denote transposition rather than the prime <math>^{\prime}.</math><br />
<br />
<math>A</math> can then be written via its rows as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{r}<br />
\mathbf{c}^{T}\\<br />
\boldsymbol{d}^{T}<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
The same ideas can be applied to the matrices <math>P</math> and <math>Q.</math><br />
<br />
= Operations with matrices =<br />
<br />
== Addition, subtraction and scalar multiplication ==<br />
<br />
For vectors, addition and subtraction are defined only for vectors of the same dimensions. If:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right],\\<br />
\mathbf{x+y} & =\left[\begin{array}{c}<br />
x_{1}+y_{1}\\<br />
\vdots\\<br />
x_{n}+y_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{x-y}=\left[\begin{array}{c}<br />
x_{1}-y_{1}\\<br />
\vdots\\<br />
x_{n}-y_{n}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Clearly, the addition or subtraction operation is ''elementwise. ''If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> have different dimensions, there will be some elements left over once all the elements of the smaller dimensioned vector have been used up.<br />
<br />
Another operation is ''scalar multiplication'': if <math>\lambda</math> is a real number or scalar, the product <math>\lambda\mathbf{x}</math> is defined as: <math>\lambda\mathbf{x}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right],</math> so that every element of <math>\mathbf{x}</math> is multiplied by the same scalar <math>\lambda.</math><br />
<br />
The two types of operation can be combined into the ''linear combination'' of vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>: <math>\lambda\mathbf{x}+\mu\mathbf{y}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right]+\left[\begin{array}{c}<br />
\mu y_{1}\\<br />
\vdots\\<br />
\mu y_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\lambda x_{1}+\mu y_{1}\\<br />
\vdots\\<br />
\lambda x_{n}+\mu y_{n}<br />
\end{array}\right].</math> Equally, one can define the linear combination of vectors <math>\mathbf{x,y,}\ldots,\mathbf{z}</math> by scalars <math>\lambda,\mu,\ldots,\nu</math> as: <math>\lambda\mathbf{x}+\mu\mathbf{y}+\ldots+\nu\mathbf{z}</math> with typical element: <math>\lambda x_{i}+\mu y_{i}+\ldots+\nu z_{i},</math> provided that all the vectors have the same dimension.<br />
<br />
For matrices, these ideas carry over immediately: apply to each column of the matrices involved. For example, if <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right]</math> and <math>B=\left[\begin{array}{rrr}<br />
\mathbf{b}_{1} & \ldots & \mathbf{b}_{n}\end{array}\right],</math> both <math>m\times n,</math> then addition and subtraction are defined elementwise, as for vectors:<br />
<br />
<math>\begin{aligned}<br />
A+B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}+\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}+\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}+b_{ij}\right\Vert ,\\<br />
A-B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}-\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}-\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}-b_{ij}\right\Vert .\end{aligned}</math><br />
<br />
Scalar multiplication of <math>A</math> by <math>\lambda</math> involves multiplying every column vector of <math>A</math> by <math>\lambda,</math> and therefore multiplying every element of <math>A</math> by <math>\lambda</math>: <math>\lambda A=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1} & \ldots & \lambda\mathbf{a}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}\right\Vert .</math> With the same idea for <math>B,</math> the linear combination of <math>A</math> and <math>B</math> by <math>\lambda</math> and <math>\mu</math> is: <math>\lambda A+\mu B=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1}+\mu\mathbf{b}_{1} & \ldots & \lambda\mathbf{a}_{n}+\mu\mathbf{b}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}+\mu b_{ij}\right\Vert .</math><br />
<br />
For example, consider the matrices: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right]</math> with <math>\lambda=1,</math> <math>\mu=-2:</math> then:<br />
<br />
<math>\begin{aligned}<br />
\lambda A+\mu B & = & A-2B\\<br />
& = & \left[\begin{array}{cc}<br />
4 & 0\\<br />
1 & 7<br />
\end{array}\right].\end{aligned}</math><br />
<br />
== Matrix - vector products ==<br />
<br />
=== Inner product ===<br />
<br />
The simplest form of a matrix vector product is the case where <math>A</math> consists of one row, so that <math>A</math> is <math>1\times n</math>: <math>A=\mathbf{a}^{T}=\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right].</math> If <math>\mathbf{x}</math> is an <math>n\times1</math> vector: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],</math> the product <math>A\mathbf{x}=\mathbf{a}^{T}\mathbf{x}</math> is called the ''inner product'' and is defined as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a}^{T}\mathbf{x} & =a_{1}x_{1}+\ldots+a_{n}x_{n}.\end{aligned}</math><br />
<br />
One can see that the definition amounts to multiplying corresponding elements in <math>\mathbf{a}</math> and <math>\mathbf{x,}</math> and adding up the resultant products. Writing: <math>\mathbf{a}^{T}\mathbf{x=}\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=a_{1}x_{1}+\ldots+a_{n}x_{n}</math> motivates the familiar description of the ''across and down rule ''for this product: ''across and down'' is the ’multiply corresponding elements’ part of the definition.<br />
<br />
Notice that the result of the inner product is a real number, for example: <math>\mathbf{c}^{T}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{c}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=36+6=42.</math><br />
<br />
In general, in the product <math>\mathbf{a}^{T}\mathbf{x,}</math> <math>\mathbf{a}</math> and <math>\mathbf{x}</math> must have the same number of elements, <math>n</math> say, for the product to be defined. If <math>\mathbf{a}</math> and <math>\mathbf{x}</math> had different numbers of elements, there would be some elements of <math>\mathbf{a}</math> or <math>\mathbf{x}</math> left over or not used in the product: e.g.: <math>\mathbf{b}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{x=}\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> When the inner product of two vectors is defined, the vectors are said to be ''conformable''.<br />
<br />
== Orthogonality ==<br />
<br />
Two vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math> with the property that <math>\mathbf{x}^{T}\mathbf{y}=0</math> are said to be orthogonal to each other. For example, if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
-1\\<br />
1<br />
\end{array}\right],</math> it is clear that <math>\mathbf{x}^{T}\mathbf{y}=0.</math> This seems a rather innocuous definition, and yet the idea of orthogonality turns out to be extremely important in econometrics.<br />
<br />
If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> are thought of as points in <math>R^{2},</math> and arrows are drawn from the origin to <math>\mathbf{x}</math> and to <math>\mathbf{y,}</math> then the two arrows are perpendicular to each other - see Figure 1. If <math>\mathbf{y}</math> were defined as: <math>\mathbf{y}=\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right],</math> the position of the <math>\mathbf{y}</math> vector and the corresponding arrow would change, but the perpendicularity property would still hold.<br />
<br />
Figure 1<br />
<br />
[[File:orthy_example.png]]<br />
<br />
<br />
=== Matrix - vector products ===<br />
<br />
Since the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> has two rows, now denoted <math>\boldsymbol{\alpha}_{1}^{T}</math> and <math>\boldsymbol{\alpha}_{2}^{T},</math> there are two possible inner products with the vector:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]:\\<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x} & = & 42,\ \ \ \ \ \boldsymbol{\alpha}_{2}^{T}\mathbf{x}=33.\end{aligned}</math><br />
<br />
Assembling the two inner product values into a <math>2\times1</math> vector defines the product of the matrix <math>A</math> with the vector <math>\mathbf{x}</math>: <math>A\mathbf{x}=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x}\\<br />
\boldsymbol{\alpha}_{2}^{T}\mathbf{x}<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right].</math> Focussing only on the part: <math>\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right],</math> one can see that each element of <math>A\mathbf{x}</math> is obtained from an ''across and down'' argument.<br />
<br />
Sometimes this product is described as forming a ''linear combination ''of the columns of <math>A</math> using the scalar elements in <math>\mathbf{x}</math>: <math>A\mathbf{x}=6\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]+3\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].</math> More generally, if:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right],\ \ \ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
\lambda\\<br />
\mu<br />
\end{array}\right],\\<br />
A\mathbf{x} & = & \lambda\mathbf{a}+\mu\mathbf{b.}\end{aligned}</math><br />
<br />
The general version of these ideas for an <math>m\times n</math> matrix <math>A</math>: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]=\left[\begin{array}{rrrr}<br />
\mathbf{a}_{1} & \mathbf{a}_{2} & \ldots & \mathbf{a}_{n}\end{array}\right].</math> is straightforward. If <math>\mathbf{x}</math> is an <math>n\times1</math> vector, then the vector <math>A\mathbf{x}</math> is, by the ''across and down'' rule:<br />
<br />
<math>A\mathbf{x}=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
x_{2}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
a_{11}x_{1}+\ldots+a_{1n}x_{n}\\<br />
a_{21}x_{1}+\ldots+a_{2n}x_{n}\\<br />
\vdots\\<br />
a_{m1}x_{1}+\ldots+a_{mn}x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\sum\limits _{j=1}^{n}a_{1j}x_{j}\\<br />
\sum\limits _{j=1}^{n}a_{2j}x_{j}\\<br />
\vdots\\<br />
\sum\limits _{j=1}^{n}a_{mj}x_{j}<br />
\end{array}\right],\label{eq:ab}</math><br />
<br />
so that the typical element, the <math>i</math>th, is <math>\sum\limits _{j=1}^{n}a_{ij}x_{j}.</math> Equally, <math>A\mathbf{x}</math> is the linear combination <math>\mathbf{a}_{1}x_{1}+\ldots+\mathbf{a}_{n}x_{n}</math> of the columns of <math>A.</math><br />
<br />
== Matrix - matrix products ==<br />
<br />
Suppose that <math>A</math> is <math>m\times n,</math> with columns <math>\mathbf{a}_{1},\ldots,\mathbf{a}_{n},</math> and <math>B</math> is <math>n\times r,</math> with columns <math>\mathbf{b}_{1},\ldots,\mathbf{b}_{r}.</math> Clearly, each product <math>A\mathbf{b}_{1},...,A\mathbf{b}_{r}</math> exists, and is <math>m\times1.</math> These products can be arranged as the columns of a matrix as <math>\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]</math> and this matrix is ''defined'' to be the product <math>C</math> of the matrices <math>A</math> and <math>B</math>: <math>C=\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]=AB.</math> By construction, this must be an <math>m\times r</math> matrix, since each column is <math>m\times1</math> and there are <math>r</math> columns.<br />
<br />
This is not the usual presentation of the definition of the product of two matrices, which relies on the ''across and down rule'' mentioned earlier, and focusses on the elements of each matrix <math>A</math> and <math>B.</math> Set:<br />
<br />
<math>\begin{aligned}<br />
B & = & \left[\begin{array}{rrrr}<br />
\mathbf{b}_{1} & \mathbf{b}_{2} & \ldots & \mathbf{b}_{r}\end{array}\right]\ \ \ \ \ \ \ \text{(by columns)}\\<br />
& = & \left\Vert b_{ik}\right\Vert ,\ \ \ \ \ i=1,...,n,k=1,...,r\ \ \ \ \ \ \ \text{(typical element)}\\<br />
& = & \left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2r}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nr}<br />
\end{array}\right]\ \ \ \ \ \ \ \text{(the array)}\end{aligned}</math><br />
<br />
What does the typical element of the <math>m\times r</math> matrix <math>C</math> look like? Start with the <math>k</math>th column of <math>C,</math> which is <math>A\mathbf{b}_{k}.</math> The <math>i</math>th element in <math>A\mathbf{b}_{k}</math> is, from equation (2), the inner product of the elements of the <math>i</math>th row in <math>A</math>: <math>\left[\begin{array}{rrrr}<br />
a_{i1} & a_{i2} & \ldots & a_{in}\end{array}\right],</math> with the elements of <math>\mathbf{b}_{k},</math> so that the inner product is: <math>a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math><br />
<br />
So, the <math>ik</math>th element of <math>C</math> is: <math>c_{ik}=a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math> We can see this arising from an ''across and down'' calculation by writing:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\label{eq:c_ab}\\<br />
& = & \left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{rrrrrr}<br />
b_{11} & b_{12} & \ldots & b_{1k} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2k} & \ldots & b_{2r}\\<br />
\vdots & \vdots & & \vdots & & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nk} & \ldots & b_{nr}<br />
\end{array}\right]\\<br />
& = & \left\Vert \sum_{j=1}^{n}a_{ij}b_{jk}\right\Vert .\end{aligned}</math><br />
<br />
These ideas are simple, but a little tedious. Numerical examples are equally tedious! As an example, using: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],</math> we can find the matrix <math>B</math> such that<br />
<br />
# the first column of <math>AB</math> adds together the columns of <math>A,</math><br />
# the second column is the difference of the first and second columns of <math>A,</math><br />
# the third column is <math>2\times</math> the first column of <math>A,</math><br />
# the fourth column is zero.<br />
<br />
It is easy to check that <math>B</math> is: <math>B=\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]</math> and that:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\\<br />
& = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{cccc}<br />
8 & 4 & 12 & 0\\<br />
8 & -2 & 6 & 0<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Arithmetic calculations of matrix products almost always use the elementwise across and down formula. However, there are many situations in econometrics where algebraic rather than arithmetic arguments are required. In these cases, the viewpoint of matrix multiplication as linear combinations of columns is much more powerful.<br />
<br />
Clearly one can give many more examples of different dimensions and complexities - but the same basic rules apply. To multiply two matrices <math>A</math> and <math>B</math> together, the number of columns in <math>A</math> must match the number of rows in <math>B</math> - this is ''conformability'' in action again. The resulting product will have number of rows equal to the number in <math>A</math> and number of columns equal to the number in <math>B.</math><br />
<br />
If this conformability rule does not hold, then the product of <math>A</math> and <math>B</math> is not defined.<br />
<br />
== Matlab ==<br />
<br />
One should also say that as the dimensions of the matrices increases, so the tediousness of the calculations increase. The solution to this for numerical calculation is to appeal to the computer. Programs like Matlab and Excel (and a number of others, some of them free) resolve this difficulty easily.<br />
<br />
In Matlab, symbols for row or column vectors do not need any particular differentiation: they are distinguished by how they are defined. For example, the following Matlab commands define <code>rowvec </code>as a <math>1\times4</math> vector, and <code>colvec</code> as a <math>4\times1</math> vector, then display the contents of these variables, and do a calculation:<br />
<br />
&gt;&gt; rowvec = [1 2 3 4]; <br />
<br />
&gt;&gt; colvec = [1;2;3;4]; <br />
<br />
&gt;&gt; rowvec<br />
<br />
rowvec =<br />
<br />
1 2 3 4<br />
<br />
&gt;&gt; colvec<br />
<br />
colvec =<br />
<br />
1 2 3 4 <br />
<br />
&gt;&gt; rowvec*colvec<br />
<br />
ans =<br />
<br />
30 <br />
<br />
So, the semi-colon indicates the end of a row in a matrix or vector; it can be replaced by a carriage return. Notice the difference in how a row vector and a column vector is defined. One can see that the product <code>rowvec*colvec</code> is well defined, just because <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector.<br />
<br />
Matlab also allows elementwise multiplication of two vectors using the <math>\centerdot\ast</math> operator: if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
x_{1}\\<br />
x_{2}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
y_{1}\\<br />
y_{2}<br />
\end{array}\right],</math> then: <math>\mathbf{x}\centerdot\ast\mathbf{y}=\left[\begin{array}{r}<br />
x_{1}y_{1}\\<br />
x_{2}y_{2}<br />
\end{array}\right]</math> and one can see that the inner product of <math>\mathbf{x}</math> and <math>\mathbf{y}</math> can be obtained as the sum of the elements of <math>\mathbf{x}</math> and <math>\mathbf{y.}</math> In Matlab, this would be obtained as: <math>\text{sum}\left(\mathbf{x}\centerdot\ast\mathbf{y}\right).</math><br />
<br />
In the example above, this calculation fails since <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector:<br />
<br />
&gt;&gt; sum(rowvec .* colvec) ??? <br />
<br />
Error using ==&gt; times Matrix dimensions must agree. <br />
<br />
For this to work, <code>rowvec</code> would have to be transposed as <code>rowvec’</code>, so that transposition in Matlab is very natural.<br />
<br />
Allowing for such difficulties, matrix multiplication in Matlab is very simple:<br />
<br />
&gt;&gt; A = [6 2; 3 5];<br />
<br />
&gt;&gt; B = [1 1 2 0;1 -1 0 0];<br />
<br />
&gt;&gt; C = A * B; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
8 4 1<br />
<br />
2 0 8 <br />
<br />
-2 6 0 <br />
<br />
Notice how the matrices are defined here through their rows. The <code>disp() </code>command displays the contents of the object referred to.<br />
<br />
It is less natural in Matlab to define matrices by columns - a typical example of how mathematics and computing have conflicts of notation. However, once columns <math>\mathbf{a}</math> and <math>\mathbf{b}</math> have been defined, the concatenation operation <math>\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]</math> collects the columns into a matrix:<br />
<br />
&gt;&gt; a = [6;2]; <br />
<br />
&gt;&gt; b = [3;5]; <br />
<br />
&gt;&gt; C = [a b]; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
6 3 <br />
<br />
2 5 <br />
<br />
Notice that the <code>disp(C)</code> command does not label the result that is printed out. Simply typing <code>C</code> would preface the output by <code>C =</code>.<br />
<br />
== Pre and Post Multiplication ==<br />
<br />
If <math>C=AB,</math> as above, say that <math>B</math> is ''pre-multiplied ''by <math>A</math> to get <math>C,</math> and that <math>A</math> is ''post-multiplied ''by <math>B</math> to get <math>C.</math><br />
<br />
This distinction between ''pre ''and ''post ''multiplication is important, in the following sense. Suppose that <math>A</math> and <math>B</math> are matrices such that the products <math>AB</math> and <math>BA</math> are both defined. If <math>A</math> is <math>m\times n,</math> <math>B</math> must have <math>n</math> rows for <math>AB</math> to be defined. For <math>BA</math> to be defined, <math>B</math> must have <math>m</math> columns to match the <math>m</math> rows in <math>A.</math> So, <math>AB</math> and <math>BA</math> are both defined if <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times m.</math><br />
<br />
Even when both products are defined, there is no reason for the two products coincide. The first thing to notice is that <math>AB</math> is a square, <math>m\times m,</math> matrix, whilst <math>BA</math> is a square, <math>n\times n,</math> matrix. Different sized matrices cannot be equal. To illustrate, use the matrices: <math>B_{2}=\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right],\ \ \ C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]:</math><br />
<br />
<math>\begin{aligned}<br />
B_{2}C & = & \left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]=\left[\begin{array}{rrr}<br />
27 & -3 & -15\\<br />
27 & 29 & -11\\<br />
-15 & -1 & 8<br />
\end{array}\right],\\<br />
CB_{2} & = & \left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
49 & -11\\<br />
31 & 15<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Even when <math>m=n,</math> so that <math>AB</math> and <math>BA</math> are both <math>m\times m</math> matrices, the products can differ: for example:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right],\\<br />
AB & = & \left[\begin{array}{cc}<br />
8 & 4\\<br />
8 & -2<br />
\end{array}\right],\ \ \ \ \ BA=\left[\begin{array}{cc}<br />
9 & 7\\<br />
3 & -3<br />
\end{array}\right].\end{aligned}</math><br />
<br />
In cases where <math>AB=BA,</math> the matrices <math>A</math> and <math>B</math> are said to ''commute''.<br />
<br />
== Transposition ==<br />
<br />
A column vector <math>\mathbf{x}</math> can be converted to a row vector <math>\mathbf{x}^{T}</math> by transposition: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \ \ \mathbf{x}^{T}=\left[\begin{array}{rrr}<br />
x_{1} & \ldots & x_{n}\end{array}\right].</math> Transposing <math>\mathbf{x}^{T}</math> as <math>\left(\mathbf{x}^{T}\right)^{T}</math> reproduces the original vector <math>\mathbf{x.}</math> How do these ideas carry over to matrices?<br />
<br />
If the <math>m\times n</math> matrix <math>A</math> can be written as <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right],</math> the transpose of <math>A,</math> <math>A^{T},</math> is defined as the matrix whose ''rows'' are <math>\mathbf{a}_{i}^{T}</math>: <math>A^{T}=\left[\begin{array}{c}<br />
\mathbf{a}_{1}^{T}\\<br />
\vdots\\<br />
\mathbf{a}_{n}^{T}<br />
\end{array}\right].</math> In terms of elements, if: <math>\mathbf{a}_{i}=\left[\begin{array}{c}<br />
a_{1i}\\<br />
\vdots\\<br />
a_{ni}<br />
\end{array}\right]</math> then: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right],\ \ \ \ \ A^{T}=\left[\begin{array}{rrrrr}<br />
a_{11} & \ldots & a_{i1} & \ldots & a_{m1}\\<br />
a_{12} & \ldots & a_{i2} & \ldots & a_{m2}\\<br />
\vdots & & \vdots & & \vdots\\<br />
a_{1n} & \ldots & a_{in} & \ldots & a_{mn}<br />
\end{array}\right].</math> One can see that the first column of <math>A</math> has now become the first row of <math>A^{T}.</math> Notice too that <math>A^{T}</math> is an <math>n\times m</math> matrix if <math>A</math> is an <math>m\times n</math> matrix.<br />
<br />
Transposing <math>A^{T}</math> takes the first column of <math>A^{T}</math> and writes it as a row, which coincides with the first row of <math>A.</math> The same argument applies to the other columns of <math>A^{T},</math> so that <math>\left(A^{T}\right)^{T}=A.</math><br />
<br />
=== The product rule for transposition ===<br />
<br />
This states that if <math>C=AB,</math> then <math>C^{T}=B^{T}A^{T}.</math><br />
<br />
How to see this? Consider the following example: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & b_{13} & b_{14}\\<br />
b_{21} & b_{22} & b_{23} & b_{24}\\<br />
b_{31} & b_{32} & b_{33} & b_{34}<br />
\end{array}\right]</math> where:<br />
<br />
<math>c_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=\sum_{k=1}^{3}a_{2k}b_{k3}.\label{eq:c23}</math><br />
<br />
One can see that: <math>B^{T}A^{T}=\left[\begin{array}{rrr}<br />
b_{11} & b_{21} & b_{31}\\<br />
b_{12} & b_{22} & b_{32}\\<br />
b_{13} & b_{23} & b_{33}\\<br />
b_{14} & b_{24} & b_{34}<br />
\end{array}\right]\left[\begin{array}{rr}<br />
a_{11} & a_{21}\\<br />
a_{12} & a_{22}\\<br />
a_{13} & a_{23}<br />
\end{array}\right]</math> and that the <math>\left(3,2\right)</math> element of this product is actually <math>c_{23}</math>: <math>b_{13}a_{21}+b_{23}a_{22}+b_{33}a_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=c_{23}.</math> In summation notation, we see that from <math>B^{T}A^{T}</math>: <math>c_{23}=\sum_{k=1}^{3}b_{k3}a_{2k},</math> where the position of the index of summation is due to the transposition. So, in summation notation, the calculation of <math>c_{23}</math> from <math>B^{T}A^{T}</math> equals that from equation (6).<br />
<br />
More generally, the <math>\left(i,j\right)</math> element of <math>AB</math>: <math>\sum_{k=1}^{3}a_{ik}b_{kj}</math> is the <math>\left(j,i\right)</math> element of <math>B^{T}A^{T}.</math> But this means that <math>B^{T}A^{T}</math> must be the transpose of <math>AB,</math> since the elements in the <math>i</math>th row of <math>AB</math> are being written in the <math>i</math>th column of <math>B^{T}A^{T}.</math><br />
<br />
This ''Product Rule for Transposition'' can be applied again to find the transpose <math>\left(C^{T}\right)^{T}</math> of <math>C^{T}</math>: <math>\left(C^{T}\right)^{T}=\left(B^{T}A^{T}\right)^{T}=\left(A^{T}\right)^{T}\left(B^{T}\right)^{T}=AB=C.</math><br />
<br />
= Special Types of Matrix =<br />
<br />
== The zero matrix ==<br />
<br />
The most obvious special type of matrix is one whose elements are all zeros. In typical element notation, the zero matrix is: <math>0=\left\Vert 0\right\Vert .</math> Since there is no indexing on the elements, it is not obvious what the dimension of this matrix is, Sometimes one writes <math>0_{mn}</math> to indicate a zero matrix of dimension <math>m\times n.</math> The same ideas apply to vectors whose elements are all zero.<br />
<br />
The effect of the zero matrix in any product that is defined is simple: <math>0A=0,\ \ \ \ \ B0=0.</math> This is easy to check using the across and down rule.<br />
<br />
== The identity or unit matrix ==<br />
<br />
Vectors of the form:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{r}<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }2\ \text{dimensions}\\<br />
\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }3\ \text{dimensions}\\<br />
\left[\begin{array}{r}<br />
1\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1\\<br />
0\\<br />
\vdots\\<br />
0<br />
\end{array}\right],\ldots,\left[\begin{array}{r}<br />
0\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }n\ \text{dimensions}\end{aligned}</math><br />
<br />
are called coordinate vectors. They are often given a characteristic notation, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> in <math>n</math> dimensions. When arranged as columns of a matrix in the natural order, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> a matrix with a characteristic pattern elements emerges, with a special notation:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{rr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2}\end{array}\right] & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=I_{2}\\<br />
\left[\begin{array}{rrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \mathbf{e}_{3}\end{array}\right] & = & \left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 1 & 0\\<br />
0 & 0 & 1<br />
\end{array}\right]=I_{3}\\<br />
\left[\begin{array}{rrrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \ldots & \mathbf{e}_{n}\end{array}\right] & = & \left[\begin{array}{rrrr}<br />
1 & 0 & \ldots & 0\\<br />
0 & 1 & \ldots & 0\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
0 & 0 & \ldots & 1<br />
\end{array}\right]=I_{n}.\end{aligned}</math><br />
<br />
The ''diagonal'' of this matrix is where the 1 elements are located, and every other element is zero.<br />
<br />
Consider the effect of <math>I_{2}</math> on the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> by both pre and post multiplication:<br />
<br />
<math>\begin{aligned}<br />
I_{2}A & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\\<br />
AI_{2} & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\end{aligned}</math><br />
<br />
as is easily checked by the across and down rule.<br />
<br />
Because any matrix is left unchanged by pre or post multiplication by an appropriately dimensioned <math>I_{n},</math> <math>I_{n}</math> is called an ''identity matrix of dimension ''<math>n.</math> Sometimes it is called a ''unit matrix of dimension ''<math>n.</math> Notice that <math>I_{n}</math> is necessarily a square matrix.<br />
<br />
== Diagonal matrices ==<br />
<br />
The identity matrix is an example of a diagonal matrix, a matrix whose elements are all zero except for those on the diagonal. Usually diagonal matrices are taken to be square, for example: <math>D=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 2 & 0\\<br />
0 & 0 & 3<br />
\end{array}\right].</math> They also produce characteristic effects when pre or post multiplying another matrix.<br />
<br />
Consider the diagonal matrix: <math>B=\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]</math> and the products <math>AB,</math> <math>BA</math> for <math>A</math> as defined in the previous section:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & -4\\<br />
6 & -10<br />
\end{array}\right],\\<br />
BA & = & \left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & 4\\<br />
-6 & -10<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Comparing the results, we can deduce that post multiplication by a diagonal matrix multiplies each column of <math>A</math> by the corresponding diagonal element, whereas pre multiplication multiplies each row by the corresponding diagonal element.<br />
<br />
== Symmetric matrices ==<br />
<br />
Symmetric matrices are matrices having the property that <math>A=A^{T}.</math> Notice that such matrices must be square, since if <math>A</math> is <math>m\times n,</math> <math>A^{T}</math> is <math>n\times m,</math> and to have equality of <math>A</math> and <math>A^{T},</math> they must have the same dimension, so that <math>m=n</math> is required.<br />
<br />
Suppose that <math>A</math> is a <math>3\times3</math> symmetric matrix, with typical element <math>a_{ij}</math>: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right],</math> so that: <math>A^{T}=\left[\begin{array}{rrr}<br />
a_{11} & a_{21} & a_{31}\\<br />
a_{12} & a_{22} & a_{32}\\<br />
a_{13} & a_{23} & a_{33}<br />
\end{array}\right].</math> Equality of matrices is defined as equality of all elements. This is fine on the diagonals, since <math>A</math> and <math>A^{T}</math> have the same diagonal elements. For the off diagonal elements, we end up with the requirements: <math>a_{12}=a_{21},\ \ \ a_{13}=a_{31},\ \ \ a_{23}=a_{32}</math> or more generally: <math>a_{ij}=a_{ji}\ \ \ \ \ \text{for}\ i\neq j.</math><br />
<br />
The effect of this conclusion is that in a symmetric matrix, the ’triangle’ of above diagonal elements coincides with the triangle of below diagonal elements. It is as if the upper triangle is folded over the diagonal to become the lower triangle.<br />
<br />
A simple example is: <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
2 & 1<br />
\end{array}\right].</math> A more complicated example uses the <math>2\times3</math> matrix <math>C</math>: <math>C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]</math> and calculates the <math>3\times3</math> matrix:<br />
<br />
<math>\begin{aligned}<br />
C^{T}C & = & \left[\begin{array}{cc}<br />
6 & 3\\<br />
2 & 5\\<br />
-3 & -1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{rrr}<br />
45 & 27 & -21\\<br />
27 & 29 & -11\\<br />
-21 & -11 & 10<br />
\end{array}\right],\end{aligned}</math><br />
<br />
which is clearly symmetric.<br />
<br />
This illustrates the general proposition that if <math>A</math> is an <math>m\times n</math> matrix, the product <math>A^{T}A</math> is a symmetric <math>n\times n</math> matrix. Proof? Compute the transpose of <math>A^{T}A</math> using the product rule for transposition: <math>\left(A^{T}A\right)^{T}=A^{T}\left(A^{T}\right)^{T}=A^{T}A.</math> Since <math>A^{T}A</math> is equal to its transpose, it must be a symmetric matrix. Such symmetric matrices appear frequently in econometrics.<br />
<br />
It should be clear that diagonal matrices are symmetric, since all their off-diagonal elements are equal (zero), and thence the identity matrix <math>I_{n}</math> is also symmetric.<br />
<br />
== The outer product ==<br />
<br />
The inner product of two <math>n\times1</math> vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>, <math>\mathbf{x}^{T}\mathbf{y}</math>, is automatically a <math>1\times1</math> quantity, a scalar, although it can be interpreted as a <math>1\times1</math> matrix, a matrix with a single element.<br />
<br />
Suppose one considered the product of <math>\mathbf{x}</math> with <math>\mathbf{x}^{T}.</math> Is this defined? If <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times r,</math> then the product <math>AB</math> is <math>m\times r.</math> Applying this logic to <math>\mathbf{xx}^{T},</math> this is <math>\left(n\times1\right)\left(1\times n\right),</math> so the resulting product ''is ''defined, and is an <math>n\times n</math>'' matrix'' - the ''outer product'' of <math>\mathbf{x}</math> and <math>\mathbf{x}^{T},</math> the word ’outer’ being used to distinguish from the inner product.<br />
<br />
How does the across and down rule work here? Suppose that: <math>\mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> Then: <math>\mathbf{xx}^{T}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right].</math> Here, there is <math>1</math> element in row one of the ’matrix’ <math>\mathbf{x,}</math> and <math>1</math> element in column one of the matrix <math>\mathbf{x}^{T},</math> so the across and down rule still works - it is just that there is only one product per row and column combination. So: <math>\mathbf{xx}^{T}=\left[\begin{array}{cc}<br />
36 & 18\\<br />
18 & 9<br />
\end{array}\right],</math> and it is obvious from this that <math>\mathbf{xx}^{T}</math> is a symmetric matrix.<br />
<br />
One can see that this outer product need not be restricted to vectors of the same dimension. If <math>\mathbf{x}</math> is <math>n\times1</math> and <math>\mathbf{y}</math> is <math>m\times1,</math> then: <math>\mathbf{xy}^{T}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
y_{1} & \ldots & y_{m}\end{array}\right]=\left[\begin{array}{rrrr}<br />
x_{1}y_{1} & x_{1}y_{2} & \ldots & x_{1}y_{m}\\<br />
x_{2}y_{1} & x_{2}y_{2} & \ldots & x_{2}y_{m}\\<br />
\\<br />
x_{n}y_{1} & x_{n}y_{2} & \ldots & x_{n}y_{m}<br />
\end{array}\right].</math> So, <math>\mathbf{xy}^{T}</math> is <math>n\times m,</math> and consists of rows which are <math>\mathbf{y}^{T}</math> multiplied by an element of the <math>\mathbf{x}</math> vector.<br />
<br />
Another interesting and useful example involves a vector with every element equal to <math>1</math>: <math>\mathbf{1}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right].</math> Sometimes this is written as <math>\mathbf{1}_{n}</math> to indicate an <math>n\times1</math> vector, and is called the ''sum vector''. Why? Consider the impact of <math>\mathbf{1}_{2}</math> on the <math>2\times1</math> vector <math>\mathbf{x}</math> used above: <math>\mathbf{1}_{2}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=9,</math> i.e. an inner product of <math>\mathbf{x}</math> with the sum vector is the sum of the elements of <math>\mathbf{x.}</math> Dividing through by the number of elements in <math>\mathbf{x}</math> produces the average of the elements of <math>\mathbf{x}</math> - i.e. the ’sample mean’ of the elements of <math>\mathbf{x.}</math><br />
<br />
The outer product of <math>\mathbf{x}</math> with <math>\mathbf{1}_{2}</math> is also interesting:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{1}_{2}\mathbf{x}^{T} & = & \left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 3\\<br />
6 & 3<br />
\end{array}\right],\\<br />
\mathbf{x1}_{2}^{T} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 6\\<br />
3 & 3<br />
\end{array}\right],\end{aligned}</math><br />
<br />
showing that pre multiplication of an <math>\mathbf{x}^{T}</math> by <math>\mathbf{1}</math> repeats <math>\mathbf{x}^{T}</math> as rows of the product, whilst post multiplication of <math>\mathbf{x}</math> by <math>\mathbf{1}^{T}</math> repeats <math>\mathbf{x}</math> as the columns of the product.<br />
<br />
Finally: <math>\mathbf{1}_{n}\mathbf{1}_{n}^{T}=\left[\begin{array}{rrr}<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1<br />
\end{array}\right],</math> an <math>n\times n</math> matrix with every element equal to <math>1.</math> This type of matrix also appears in econometrics!<br />
<br />
== Triangular matrices ==<br />
<br />
A square ''lower triangular ''matrix has all elements above the main diagonal equal to zero, whilst a square ''upper triangular ''matrix has all elements below the main diagonal equal to zero. A simple example of a lower triangular matrix is: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & 0 & 0\\<br />
a_{21} & a_{22} & 0\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right].</math> Clearly, for this matrix, <math>A^{T}</math> is an upper triangular matrix.<br />
<br />
One can adapt the definition to rectangular matrices: for example, if two arbitrary rows are added to <math>A,</math> so that it becomes <math>5\times3,</math> it would still be considered lower triangular. Equally, if, for example, the third column of <math>A</math> above is removed, <math>A</math> is still considered lower triangular.<br />
<br />
Often, we use ''unit ''triangular matrices, where the diagonal elements are all equal to <math>1:</math> e.g.:<br />
<br />
<math>\left[\begin{array}{rrr}<br />
1 & 2 & 0\\<br />
0 & 1 & 1\\<br />
0 & 0 & 1<br />
\end{array}\right].\label{eq:lt_matrix}</math><br />
<br />
== Partitioned matrices ==<br />
<br />
Sometimes, especially with big matrices, it is useful to organise the elements of the matrix into components which are themselves matrices, for example: <math>B=\left[\begin{array}{rrrr}<br />
1 & 2 & 0 & 0\\<br />
8 & 3 & 0 & 0\\<br />
0 & 0 & 7 & 4\\<br />
0 & 0 & 6 & 5<br />
\end{array}\right]</math> Here it would be reasonable to write: <math>B=\left[\begin{array}{cc}<br />
B_{11} & 0\\<br />
0 & B_{22}<br />
\end{array}\right],</math> where <math>B_{ii},i=1,2,</math> represent <math>2\times2</math> matrices. <math>B</math> is an example of a ''partitioned matrix'': that is, an <math>m\times n</math> matrix <math>A</math> say: <math>A=\left\Vert a_{ij}\right\Vert ,</math> where the elements of <math>A</math> are organised into ''sub-matrices''. An example might be:<br />
<br />
<math>A=\left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right],\label{eq:partition_a}</math><br />
<br />
where the ''sub - matrices'' in the first row block have <math>r</math> rows, and therefore <math>m-r</math> rows in the second row block. The column blocks might be defined by (for example) 3 columns in the first column block, 4 in the second and <math>n-7</math> in the third column block.<br />
<br />
Another simple example might be: <math>A=\left[\begin{array}{rrr}<br />
A_{1} & A_{2} & A_{3}\end{array}\right],\ \ \ \ \ \mathbf{x=}\left[\begin{array}{c}<br />
\mathbf{x}_{1}\\<br />
\mathbf{x}_{2}\\<br />
\mathbf{x}_{3}<br />
\end{array}\right],</math> where <math>A</math> and therefore <math>A_{1},A_{2},A_{3}</math> have <math>m</math> rows, <math>A_{1}</math> has <math>n_{1}</math> columns, <math>A_{2}</math> has <math>n_{2}</math> columns, <math>A_{3}</math> has <math>n_{3}</math> columns. The ''subvectors'' in <math>\mathbf{x}</math> must have <math>n_{1},n_{2}</math> and <math>n_{3}</math> rows respectively, for the product <math>A\mathbf{x}</math> to exist.<br />
<br />
Suppose that <math>n_{1}+n_{2}+n_{3}=n,</math> so that <math>A</math> is <math>m\times n.</math> The <math>i</math>th element of <math>A\mathbf{x}</math> is: <math>\sum_{i=1}^{n}a_{ij}x_{j}</math> but the summation can be broken up into the first <math>n_{1}</math> terms: <math>\sum_{i=1}^{n_{1}}a_{ij}x_{j},</math> the next <math>n_{2}</math> terms: <math>\sum_{i=n_{1}+1}^{n_{1}+n_{2}}a_{ij}x_{j},</math> and the next <math>n_{3}</math> terms; <math>\sum_{i=n_{1}+n_{2}+1}^{n}a_{ij}x_{j}.</math><br />
<br />
The point about the use of partitioned matrices is that the product <math>A\mathbf{x}</math> can be represented as: <math>A\mathbf{x}=A_{1}\mathbf{x}_{1}+A_{2}\mathbf{x}_{2}+A\mathbf{x}_{3}</math> by applying the across and down rule to the submatrices and the subvectors, a much simpler representation than the use of summations.<br />
<br />
Each of the components is a conformable matrix-vector product: this is essential in any use of partitioned matrices to represent some matrix product. For example, using <math>A</math> from equation (8) and <math>B</math> as: <math>B=\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right],</math> it is easy to write:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right]\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{r}<br />
A_{11}B_{11}+A_{12}B_{21}+A_{13}B_{31}\\<br />
A_{21}B_{11}+A_{22}B_{21}+A_{23}B_{31}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
But, what are the row dimensions for the submatrices in <math>B?</math> What are the possible column dimensions for the submatrices in <math>B?</math><br />
<br />
= Matrices, vectors and econometrics =<br />
<br />
The data on weights and heights for 12 students in the data matrix: <math>D=\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]</math> would seem to be ideally suited for fitting a two variable regression model: <math>y_{i}=\alpha+\beta x_{i}+u_{i},\;\;\;\;\; i=1,...,12.</math> Here, the first column of <math>D</math> contains all the weight data, the data on the dependent variable <math>y_{i},</math> and so should be labelled <math>\mathbf{y.}</math> The second column of <math>D</math> contains all the data on the explanatory variable height, in the vector <math>\mathbf{x}</math> say, so that: <math>D=\left[\begin{array}{rr}<br />
\mathbf{y} & \mathbf{x}\end{array}\right].</math><br />
<br />
If we define a <math>12\times1</math> vector with every element <math>1</math>: <math>\mathbf{1}_{12}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right],</math> and a <math>12\times1</math> vector <math>\mathbf{u}</math> to contain the error terms; <math>\mathbf{u}=\left[\begin{array}{c}<br />
u_{1}\\<br />
\vdots\\<br />
u_{12}<br />
\end{array}\right]</math> the regression model can be written in terms of the three data vectors <math>\mathbf{y,1}_{12}</math> and <math>\mathbf{x}</math> as: <math>\mathbf{y}=\mathbf{1}_{12}\alpha+\mathbf{x}\beta+\mathbf{u.}</math> To see this, think of the <math>i</math>th elements of the vectors on the left and right hand sides.<br />
<br />
The standard next step is then to combine the data vectors for the explanatory variables into a matrix: <math>X=\left[\begin{array}{rr}<br />
\mathbf{1}_{12} & \mathbf{x}\end{array}\right],</math> and then define a <math>2\times1</math> vector <math>\boldsymbol{\delta}</math> to contain the parameters <math>\alpha,\beta</math> as: <math>\boldsymbol{\delta}=\left[\begin{array}{r}<br />
\alpha\\<br />
\beta<br />
\end{array}\right]</math> to give the data matrix representation of the regression model as: <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
<br />
For the purposes of developing the theory of regression, this is the most convenient form of the regression model. It can represent regression models with any number of explanatory variables, and thus any number of parameters. The obvious point is that a knowledge of vector and matrix operations is needed to use and understand this form.<br />
<br />
We shall see later that there are two particular matrix and vector quantities associated with a regression model. The first is the matrix <math>X^{T}X,</math> and the second the vector <math>X^{T}\mathbf{y.}</math> The following Matlab code snippet provides the numerical values of these quantities for the weight data:<br />
<br />
&gt;&gt; dset = load(’weights.mat’); <br />
<br />
&gt;&gt; xtx = dset.X’ * dset.X; <br />
<br />
&gt;&gt; xty = dset.X’ * dset.y; <br />
<br />
&gt;&gt; disp(xtx) <br />
<br />
12 802<br />
<br />
802 53792<br />
<br />
&gt;&gt; disp(xty)<br />
<br />
1850<br />
<br />
124528<br />
<br />
Hand calculation is of course possible, but not recommended.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Lnotes&diff=3042Lnotes2013-09-10T14:54:50Z<p>LG: Created page with "= Matrices = In the PreSession Maths course, a matrix was defined as follows: <blockquote>A matrix is a rectangular array of numbers enclosed in parentheses, con- ventional..."</p>
<hr />
<div>= Matrices =<br />
<br />
In the PreSession Maths course, a matrix was defined as follows:<br />
<br />
<blockquote>A matrix is a rectangular array of numbers enclosed in parentheses, con-<br />
<br />
ventionally denoted by a capital letter. The number of rows (say <math>m</math>) and<br />
<br />
the number of columns (say <math>n</math>) determine the order of the matrix (<math>m</math> <math>\times</math> <math>n</math>).<br />
</blockquote><br />
Two examples were given:<br />
<br />
<math>\begin{aligned}<br />
P & =\left[\begin{array}{rrr}<br />
2 & 3 & 4\\<br />
3 & 1 & 5<br />
\end{array}\right],\ \ \ Q=\left[\begin{array}{rr}<br />
2 & 3\\<br />
4 & 3\\<br />
1 & 5<br />
\end{array}\right],\end{aligned}</math><br />
<br />
matrices of dimensions <math>2\times3</math> and <math>3\times2</math> respectively.<br />
<br />
Why study matrices for econometrics? Basically because a data set of several variables, e.g. on the weights and heights of 12 students, can be thought of as a matrix:<br />
<br />
<math>\begin{aligned}<br />
D & =\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]\end{aligned}</math><br />
<br />
The properties of matrices can then be used to facilitate answering all the usual questions of econometrics - list not given here!<br />
<br />
Calculations with matrices with explicit numerical elements, as in the examples above is called matrix ''arithmetic''. Matrix ''algebra'' is the algebra of matrices where the elements are not made explicit: this is what is really required for econometrics, as we shall see.<br />
<br />
As an example of this, a <math>2\times3</math> matrix might be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{ccc}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\end{aligned}</math><br />
<br />
and would equal <math>P</math> above if the collection of <math>a_{ij}</math> were given appropriate numerical values.<br />
<br />
A general <math>m\times n</math> matrix <math>A</math> can be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
There is also a ''typical element ''notation for matrices:<br />
<br />
<math>\begin{aligned}<br />
A & =\left\Vert a_{ij}\right\Vert ,\ \ \ \ \ i=1,...,m,j=1,...,n,\end{aligned}</math><br />
<br />
so that <math>a_{ij}</math> is the element at the intersection of the <math>i</math>th row and <math>j</math>th column in <math>A.</math><br />
<br />
When <math>m\neq n,</math> <math>A</math> is a ''rectangular ''matrix; when <math>m=n,</math> <math>A</math> is <math>m\times m</math> or <math>n\times n,</math> and <math>A</math> is a square matrix, having the same number of rows or columns.<br />
<br />
== Rows, columns and vectors ==<br />
<br />
Clearly, there is no reason why <math>m</math> or <math>n</math> cannot equal 1: so, an <math>m\times n</math> matrix with <math>n=1,</math> i.e. with one column, is usually called a column vector. Similarly, a matrix with one row is a row vector.<br />
<br />
There are a lot of advantages to thinking of matrices as collections of row or column vectors, as we shall see. As an example, define the <math>2\times1</math> column vectors:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a} & =\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\mathbf{,\ \ \ b}=\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
and arrange as the columns of the <math>2\times2</math> matrix<br />
<br />
<math>A=\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\label{eq:axy}</math><br />
<br />
In general, a column vector <math>\mathbf{x}</math> with <math>n</math> elements can be written as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\end{aligned}</math><br />
<br />
What happens when both <math>m</math> and <math>n</math> are equal to <math>1?</math> Then, <math>A</math> is a <math>1\times1</math> matrix, but it is also considered to be a real number, or ''scalar'' in the language of linear algebra:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[a_{11}\right]=a_{11}.\end{aligned}</math><br />
<br />
This is perhaps a little odd, but turns out to be a useful convention in a number of situations.<br />
<br />
== Transposition of vectors ==<br />
<br />
The ''rows'' of the matrix <math>A</math> in equation ([eq:axy]) can be seen as elements of column vectors, say:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c} & =\left[\begin{array}{r}<br />
6\\<br />
2<br />
\end{array}\right],\ \ \ \boldsymbol{d}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
This representation of row vectors as column vectors is a bit clumsy, so some transformation which converts a column vector into a row vector, and vice versa would be useful. The process of converting a column vector <math>\mathbf{a}</math> into a row vector is called ''transposition, ''and the transposed version of <math>\mathbf{c}</math> is denoted:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c}^{T} & =\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\end{aligned}</math><br />
<br />
the <math>^{T}</math> superscript denoting transposition. In practice, a prime, <math>^{\prime},</math> is used instead of <math>^{T}.</math> However, whilst the prime is much simpler to write than the <math>^{T}</math> sign, it is also much easier to lose track of in writing out long or complicated expressions. So, it is best initially to use <math>^{T}</math> to denote transposition rather than the prime <math>^{\prime}.</math><br />
<br />
<math>A</math> can then be written via its rows as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{r}<br />
\mathbf{c}^{T}\\<br />
\boldsymbol{d}^{T}<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
The same ideas can be applied to the matrices <math>P</math> and <math>Q.</math><br />
<br />
= Operations with matrices =<br />
<br />
== Addition, subtraction and scalar multiplication ==<br />
<br />
For vectors, addition and subtraction are defined only for vectors of the same dimensions. If:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right],\\<br />
\mathbf{x+y} & =\left[\begin{array}{c}<br />
x_{1}+y_{1}\\<br />
\vdots\\<br />
x_{n}+y_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{x-y}=\left[\begin{array}{c}<br />
x_{1}-y_{1}\\<br />
\vdots\\<br />
x_{n}-y_{n}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Clearly, the addition or subtraction operation is ''elementwise. ''If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> have different dimensions, there will be some elements left over once all the elements of the smaller dimensioned vector have been used up.<br />
<br />
Another operation is ''scalar multiplication'': if <math>\lambda</math> is a real number or scalar, the product <math>\lambda\mathbf{x}</math> is defined as: <math>\lambda\mathbf{x}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right],</math> so that every element of <math>\mathbf{x}</math> is multiplied by the same scalar <math>\lambda.</math><br />
<br />
The two types of operation can be combined into the ''linear combination'' of vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>: <math>\lambda\mathbf{x}+\mu\mathbf{y}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right]+\left[\begin{array}{c}<br />
\mu y_{1}\\<br />
\vdots\\<br />
\mu y_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\lambda x_{1}+\mu y_{1}\\<br />
\vdots\\<br />
\lambda x_{n}+\mu y_{n}<br />
\end{array}\right].</math> Equally, one can define the linear combination of vectors <math>\mathbf{x,y,}\ldots,\mathbf{z}</math> by scalars <math>\lambda,\mu,\ldots,\nu</math> as: <math>\lambda\mathbf{x}+\mu\mathbf{y}+\ldots+\nu\mathbf{z}</math> with typical element: <math>\lambda x_{i}+\mu y_{i}+\ldots+\nu z_{i},</math> provided that all the vectors have the same dimension.<br />
<br />
For matrices, these ideas carry over immediately: apply to each column of the matrices involved. For example, if <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right]</math> and <math>B=\left[\begin{array}{rrr}<br />
\mathbf{b}_{1} & \ldots & \mathbf{b}_{n}\end{array}\right],</math> both <math>m\times n,</math> then addition and subtraction are defined elementwise, as for vectors:<br />
<br />
<math>\begin{aligned}<br />
A+B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}+\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}+\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}+b_{ij}\right\Vert ,\\<br />
A-B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}-\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}-\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}-b_{ij}\right\Vert .\end{aligned}</math><br />
<br />
Scalar multiplication of <math>A</math> by <math>\lambda</math> involves multiplying every column vector of <math>A</math> by <math>\lambda,</math> and therefore multiplying every element of <math>A</math> by <math>\lambda</math>: <math>\lambda A=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1} & \ldots & \lambda\mathbf{a}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}\right\Vert .</math> With the same idea for <math>B,</math> the linear combination of <math>A</math> and <math>B</math> by <math>\lambda</math> and <math>\mu</math> is: <math>\lambda A+\mu B=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1}+\mu\mathbf{b}_{1} & \ldots & \lambda\mathbf{a}_{n}+\mu\mathbf{b}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}+\mu b_{ij}\right\Vert .</math><br />
<br />
For example, consider the matrices: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right]</math> with <math>\lambda=1,</math> <math>\mu=-2:</math> then:<br />
<br />
<math>\begin{aligned}<br />
\lambda A+\mu B & = & A-2B\\<br />
& = & \left[\begin{array}{cc}<br />
4 & 0\\<br />
1 & 7<br />
\end{array}\right].\end{aligned}</math><br />
<br />
== Matrix - vector products ==<br />
<br />
=== Inner product ===<br />
<br />
The simplest form of a matrix vector product is the case where <math>A</math> consists of one row, so that <math>A</math> is <math>1\times n</math>: <math>A=\mathbf{a}^{T}=\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right].</math> If <math>\mathbf{x}</math> is an <math>n\times1</math> vector: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],</math> the product <math>A\mathbf{x}=\mathbf{a}^{T}\mathbf{x}</math> is called the ''inner product'' and is defined as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a}^{T}\mathbf{x} & =a_{1}x_{1}+\ldots+a_{n}x_{n}.\end{aligned}</math><br />
<br />
One can see that the definition amounts to multiplying corresponding elements in <math>\mathbf{a}</math> and <math>\mathbf{x,}</math> and adding up the resultant products. Writing: <math>\mathbf{a}^{T}\mathbf{x=}\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=a_{1}x_{1}+\ldots+a_{n}x_{n}</math> motivates the familiar description of the ''across and down rule ''for this product: ''across and down'' is the ’multiply corresponding elements’ part of the definition.<br />
<br />
Notice that the result of the inner product is a real number, for example: <math>\mathbf{c}^{T}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{c}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=36+6=42.</math><br />
<br />
In general, in the product <math>\mathbf{a}^{T}\mathbf{x,}</math> <math>\mathbf{a}</math> and <math>\mathbf{x}</math> must have the same number of elements, <math>n</math> say, for the product to be defined. If <math>\mathbf{a}</math> and <math>\mathbf{x}</math> had different numbers of elements, there would be some elements of <math>\mathbf{a}</math> or <math>\mathbf{x}</math> left over or not used in the product: e.g.: <math>\mathbf{b}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{x=}\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> When the inner product of two vectors is defined, the vectors are said to be ''conformable''.<br />
<br />
== Orthogonality ==<br />
<br />
Two vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math> with the property that <math>\mathbf{x}^{T}\mathbf{y}=0</math> are said to be orthogonal to each other. For example, if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
-1\\<br />
1<br />
\end{array}\right],</math> it is clear that <math>\mathbf{x}^{T}\mathbf{y}=0.</math> This seems a rather innocuous definition, and yet the idea of orthogonality turns out to be extremely important in econometrics.<br />
<br />
If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> are thought of as points in <math>R^{2},</math> and arrows are drawn from the origin to <math>\mathbf{x}</math> and to <math>\mathbf{y,}</math> then the two arrows are perpendicular to each other - see Figure [orthy<sub>e</sub>xample]. If <math>\mathbf{y}</math> were defined as: <math>\mathbf{y}=\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right],</math> the position of the <math>\mathbf{y}</math> vector and the corresponding arrow would change, but the perpendicularity property would still hold.<br />
<br />
[ht] [[Image:0C__courses_econometric_methods_yed_orthy_example.pdf|fig:]]<br />
<br />
[orthy<sub>e</sub>xample]<br />
<br />
=== Matrix - vector products ===<br />
<br />
Since the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> has two rows, now denoted <math>\boldsymbol{\alpha}_{1}^{T}</math> and <math>\boldsymbol{\alpha}_{2}^{T},</math> there are two possible inner products with the vector:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]:\\<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x} & = & 42,\ \ \ \ \ \boldsymbol{\alpha}_{2}^{T}\mathbf{x}=33.\end{aligned}</math><br />
<br />
Assembling the two inner product values into a <math>2\times1</math> vector defines the product of the matrix <math>A</math> with the vector <math>\mathbf{x}</math>: <math>A\mathbf{x}=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x}\\<br />
\boldsymbol{\alpha}_{2}^{T}\mathbf{x}<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right].</math> Focussing only on the part: <math>\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right],</math> one can see that each element of <math>A\mathbf{x}</math> is obtained from an ''across and down'' argument.<br />
<br />
Sometimes this product is described as forming a ''linear combination ''of the columns of <math>A</math> using the scalar elements in <math>\mathbf{x}</math>: <math>A\mathbf{x}=6\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]+3\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].</math> More generally, if:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right],\ \ \ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
\lambda\\<br />
\mu<br />
\end{array}\right],\\<br />
A\mathbf{x} & = & \lambda\mathbf{a}+\mu\mathbf{b.}\end{aligned}</math><br />
<br />
The general version of these ideas for an <math>m\times n</math> matrix <math>A</math>: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]=\left[\begin{array}{rrrr}<br />
\mathbf{a}_{1} & \mathbf{a}_{2} & \ldots & \mathbf{a}_{n}\end{array}\right].</math> is straightforward. If <math>\mathbf{x}</math> is an <math>n\times1</math> vector, then the vector <math>A\mathbf{x}</math> is, by the ''across and down'' rule:<br />
<br />
<math>A\mathbf{x}=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
x_{2}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
a_{11}x_{1}+\ldots+a_{1n}x_{n}\\<br />
a_{21}x_{1}+\ldots+a_{2n}x_{n}\\<br />
\vdots\\<br />
a_{m1}x_{1}+\ldots+a_{mn}x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\sum\limits _{j=1}^{n}a_{1j}x_{j}\\<br />
\sum\limits _{j=1}^{n}a_{2j}x_{j}\\<br />
\vdots\\<br />
\sum\limits _{j=1}^{n}a_{mj}x_{j}<br />
\end{array}\right],\label{eq:ab}</math><br />
<br />
so that the typical element, the <math>i</math>th, is <math>\sum\limits _{j=1}^{n}a_{ij}x_{j}.</math> Equally, <math>A\mathbf{x}</math> is the linear combination <math>\mathbf{a}_{1}x_{1}+\ldots+\mathbf{a}_{n}x_{n}</math> of the columns of <math>A.</math><br />
<br />
== Matrix - matrix products ==<br />
<br />
Suppose that <math>A</math> is <math>m\times n,</math> with columns <math>\mathbf{a}_{1},\ldots,\mathbf{a}_{n},</math> and <math>B</math> is <math>n\times r,</math> with columns <math>\mathbf{b}_{1},\ldots,\mathbf{b}_{r}.</math> Clearly, each product <math>A\mathbf{b}_{1},...,A\mathbf{b}_{r}</math> exists, and is <math>m\times1.</math> These products can be arranged as the columns of a matrix as <math>\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]</math> and this matrix is ''defined'' to be the product <math>C</math> of the matrices <math>A</math> and <math>B</math>: <math>C=\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]=AB.</math> By construction, this must be an <math>m\times r</math> matrix, since each column is <math>m\times1</math> and there are <math>r</math> columns.<br />
<br />
This is not the usual presentation of the definition of the product of two matrices, which relies on the ''across and down rule'' mentioned earlier, and focusses on the elements of each matrix <math>A</math> and <math>B.</math> Set:<br />
<br />
<math>\begin{aligned}<br />
B & = & \left[\begin{array}{rrrr}<br />
\mathbf{b}_{1} & \mathbf{b}_{2} & \ldots & \mathbf{b}_{r}\end{array}\right]\text{\,\,\,\,\,\,\,(by columns)}\\<br />
& = & \left\Vert b_{ik}\right\Vert ,\ \ \ \ \ i=1,...,n,k=1,...,r\text{ \,\,\,\,\,\,(typical element)}\\<br />
& = & \left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2r}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nr}<br />
\end{array}\right]\text{\,\,\,\,\,\,\,(the array)}\end{aligned}</math><br />
<br />
What does the typical element of the <math>m\times r</math> matrix <math>C</math> look like? Start with the <math>k</math>th column of <math>C,</math> which is <math>A\mathbf{b}_{k}.</math> The <math>i</math>th element in <math>A\mathbf{b}_{k}</math> is, from equation ([eq:ab]), the inner product of the elements of the <math>i</math>th row in <math>A</math>: <math>\left[\begin{array}{rrrr}<br />
a_{i1} & a_{i2} & \ldots & a_{in}\end{array}\right],</math> with the elements of <math>\mathbf{b}_{k},</math> so that the inner product is: <math>a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math><br />
<br />
So, the <math>ik</math>th element of <math>C</math> is: <math>c_{ik}=a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math> We can see this arising from an ''across and down'' calculation by writing:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\label{eq:c_ab}\\<br />
& = & \left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{rrrrrr}<br />
b_{11} & b_{12} & \ldots & b_{1k} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2k} & \ldots & b_{2r}\\<br />
\vdots & \vdots & & \vdots & & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nk} & \ldots & b_{nr}<br />
\end{array}\right]\\<br />
& = & \left\Vert \sum_{j=1}^{n}a_{ij}b_{jk}\right\Vert .\end{aligned}</math><br />
<br />
These ideas are simple, but a little tedious. Numerical examples are equally tedious! As an example, using: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],</math> we can find the matrix <math>B</math> such that<br />
<br />
# the first column of <math>AB</math> adds together the columns of <math>A,</math><br />
# the second column is the difference of the first and second columns of <math>A,</math><br />
# the third column is <math>2\times</math> the first column of <math>A,</math><br />
# the fourth column is zero.<br />
<br />
It is easy to check that <math>B</math> is: <math>B=\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]</math> and that:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\\<br />
& = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{cccc}<br />
8 & 4 & 12 & 0\\<br />
8 & -2 & 6 & 0<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Arithmetic calculations of matrix products almost always use the elementwise across and down formula. However, there are many situations in econometrics where algebraic rather than arithmetic arguments are required. In these cases, the viewpoint of matrix multiplication as linear combinations of columns is much more powerful.<br />
<br />
Clearly one can give many more examples of different dimensions and complexities - but the same basic rules apply. To multiply two matrices <math>A</math> and <math>B</math> together, the number of columns in <math>A</math> must match the number of rows in <math>B</math> - this is ''conformability'' in action again. The resulting product will have number of rows equal to the number in <math>A</math> and number of columns equal to the number in <math>B.</math><br />
<br />
If this conformability rule does not hold, then the product of <math>A</math> and <math>B</math> is not defined.<br />
<br />
== Matlab ==<br />
<br />
One should also say that as the dimensions of the matrices increases, so the tediousness of the calculations increase. The solution to this for numerical calculation is to appeal to the computer. Programs like Matlab and Excel (and a number of others, some of them free) resolve this difficulty easily.<br />
<br />
In Matlab, symbols for row or column vectors do not need any particular differentiation: they are distinguished by how they are defined. For example, the following Matlab commands define <code>rowvec </code>as a <math>1\times4</math> vector, and <code>colvec</code> as a <math>4\times1</math> vector, then display the contents of these variables, and do a calculation:<br />
<br />
&gt;&gt; rowvec = [1 2 3 4]; <br />
<br />
&gt;&gt; colvec = [1;2;3;4]; <br />
<br />
&gt;&gt; rowvec<br />
<br />
rowvec =<br />
<br />
1 2 3 4<br />
<br />
&gt;&gt; colvec<br />
<br />
colvec =<br />
<br />
1 2 3 4 <br />
<br />
&gt;&gt; rowvec*colvec<br />
<br />
ans =<br />
<br />
30 <br />
<br />
So, the semi-colon indicates the end of a row in a matrix or vector; it can be replaced by a carriage return. Notice the difference in how a row vector and a column vector is defined. One can see that the product <code>rowvec*colvec</code> is well defined, just because <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector.<br />
<br />
Matlab also allows elementwise multiplication of two vectors using the <math>\centerdot\ast</math> operator: if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
x_{1}\\<br />
x_{2}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
y_{1}\\<br />
y_{2}<br />
\end{array}\right],</math> then: <math>\mathbf{x}\centerdot\ast\mathbf{y}=\left[\begin{array}{r}<br />
x_{1}y_{1}\\<br />
x_{2}y_{2}<br />
\end{array}\right]</math> and one can see that the inner product of <math>\mathbf{x}</math> and <math>\mathbf{y}</math> can be obtained as the sum of the elements of <math>\mathbf{x}</math> and <math>\mathbf{y.}</math> In Matlab, this would be obtained as: <math>\text{sum}\left(\mathbf{x}\centerdot\ast\mathbf{y}\right).</math><br />
<br />
In the example above, this calculation fails since <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector:<br />
<br />
&gt;&gt; sum(rowvec .* colvec) ??? <br />
<br />
Error using ==&gt; times Matrix dimensions must agree. <br />
<br />
For this to work, <code>rowvec</code> would have to be transposed as <code>rowvec’</code>, so that transposition in Matlab is very natural.<br />
<br />
Allowing for such difficulties, matrix multiplication in Matlab is very simple:<br />
<br />
&gt;&gt; A = [6 2; 3 5];<br />
<br />
&gt;&gt; B = [1 1 2 0;1 -1 0 0];<br />
<br />
&gt;&gt; C = A * B; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
8 4 1<br />
<br />
2 0 8 <br />
<br />
-2 6 0 <br />
<br />
Notice how the matrices are defined here through their rows. The <code>disp() </code>command displays the contents of the object referred to.<br />
<br />
It is less natural in Matlab to define matrices by columns - a typical example of how mathematics and computing have conflicts of notation. However, once columns <math>\mathbf{a}</math> and <math>\mathbf{b}</math> have been defined, the concatenation operation <math>\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]</math> collects the columns into a matrix:<br />
<br />
&gt;&gt; a = [6;2]; <br />
<br />
&gt;&gt; b = [3;5]; <br />
<br />
&gt;&gt; C = [a b]; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
6 3 <br />
<br />
2 5 <br />
<br />
Notice that the <code>disp(C)</code> command does not label the result that is printed out. Simply typing <code>C</code> would preface the output by <code>C =</code>.<br />
<br />
== Pre and Post Multiplication ==<br />
<br />
If <math>C=AB,</math> as above, say that <math>B</math> is ''pre-multiplied ''by <math>A</math> to get <math>C,</math> and that <math>A</math> is ''post-multiplied ''by <math>B</math> to get <math>C.</math><br />
<br />
This distinction between ''pre ''and ''post ''multiplication is important, in the following sense. Suppose that <math>A</math> and <math>B</math> are matrices such that the products <math>AB</math> and <math>BA</math> are both defined. If <math>A</math> is <math>m\times n,</math> <math>B</math> must have <math>n</math> rows for <math>AB</math> to be defined. For <math>BA</math> to be defined, <math>B</math> must have <math>m</math> columns to match the <math>m</math> rows in <math>A.</math> So, <math>AB</math> and <math>BA</math> are both defined if <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times m.</math><br />
<br />
Even when both products are defined, there is no reason for the two products coincide. The first thing to notice is that <math>AB</math> is a square, <math>m\times m,</math> matrix, whilst <math>BA</math> is a square, <math>n\times n,</math> matrix. Different sized matrices cannot be equal. To illustrate, use the matrices: <math>B_{2}=\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right],\ \ \ C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]:</math><br />
<br />
<math>\begin{aligned}<br />
B_{2}C & = & \left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]=\left[\begin{array}{rrr}<br />
27 & -3 & -15\\<br />
27 & 29 & -11\\<br />
-15 & -1 & 8<br />
\end{array}\right],\\<br />
CB_{2} & = & \left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
49 & -11\\<br />
31 & 15<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Even when <math>m=n,</math> so that <math>AB</math> and <math>BA</math> are both <math>m\times m</math> matrices, the products can differ: for example:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right],\\<br />
AB & = & \left[\begin{array}{cc}<br />
8 & 4\\<br />
8 & -2<br />
\end{array}\right],\ \ \ \ \ BA=\left[\begin{array}{cc}<br />
9 & 7\\<br />
3 & -3<br />
\end{array}\right].\end{aligned}</math><br />
<br />
In cases where <math>AB=BA,</math> the matrices <math>A</math> and <math>B</math> are said to ''commute''.<br />
<br />
== Transposition ==<br />
<br />
A column vector <math>\mathbf{x}</math> can be converted to a row vector <math>\mathbf{x}^{T}</math> by transposition: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \ \ \mathbf{x}^{T}=\left[\begin{array}{rrr}<br />
x_{1} & \ldots & x_{n}\end{array}\right].</math> Transposing <math>\mathbf{x}^{T}</math> as <math>\left(\mathbf{x}^{T}\right)^{T}</math> reproduces the original vector <math>\mathbf{x.}</math> How do these ideas carry over to matrices?<br />
<br />
If the <math>m\times n</math> matrix <math>A</math> can be written as <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right],</math> the transpose of <math>A,</math> <math>A^{T},</math> is defined as the matrix whose ''rows'' are <math>\mathbf{a}_{i}^{T}</math>: <math>A^{T}=\left[\begin{array}{c}<br />
\mathbf{a}_{1}^{T}\\<br />
\vdots\\<br />
\mathbf{a}_{n}^{T}<br />
\end{array}\right].</math> In terms of elements, if: <math>\mathbf{a}_{i}=\left[\begin{array}{c}<br />
a_{1i}\\<br />
\vdots\\<br />
a_{ni}<br />
\end{array}\right]</math> then: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right],\ \ \ \ \ A^{T}=\left[\begin{array}{rrrrr}<br />
a_{11} & \ldots & a_{i1} & \ldots & a_{m1}\\<br />
a_{12} & \ldots & a_{i2} & \ldots & a_{m2}\\<br />
\vdots & & \vdots & & \vdots\\<br />
a_{1n} & \ldots & a_{in} & \ldots & a_{mn}<br />
\end{array}\right].</math> One can see that the first column of <math>A</math> has now become the first row of <math>A^{T}.</math> Notice too that <math>A^{T}</math> is an <math>n\times m</math> matrix if <math>A</math> is an <math>m\times n</math> matrix.<br />
<br />
Transposing <math>A^{T}</math> takes the first column of <math>A^{T}</math> and writes it as a row, which coincides with the first row of <math>A.</math> The same argument applies to the other columns of <math>A^{T},</math> so that <math>\left(A^{T}\right)^{T}=A.</math><br />
<br />
=== The product rule for transposition ===<br />
<br />
This states that if <math>C=AB,</math> then <math>C^{T}=B^{T}A^{T}.</math><br />
<br />
How to see this? Consider the following example: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & b_{13} & b_{14}\\<br />
b_{21} & b_{22} & b_{23} & b_{24}\\<br />
b_{31} & b_{32} & b_{33} & b_{34}<br />
\end{array}\right]</math> where:<br />
<br />
<math>c_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=\sum_{k=1}^{3}a_{2k}b_{k3}.\label{eq:c23}</math><br />
<br />
One can see that: <math>B^{T}A^{T}=\left[\begin{array}{rrr}<br />
b_{11} & b_{21} & b_{31}\\<br />
b_{12} & b_{22} & b_{32}\\<br />
b_{13} & b_{23} & b_{33}\\<br />
b_{14} & b_{24} & b_{34}<br />
\end{array}\right]\left[\begin{array}{rr}<br />
a_{11} & a_{21}\\<br />
a_{12} & a_{22}\\<br />
a_{13} & a_{23}<br />
\end{array}\right]</math> and that the <math>\left(3,2\right)</math> element of this product is actually <math>c_{23}</math>: <math>b_{13}a_{21}+b_{23}a_{22}+b_{33}a_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=c_{23}.</math> In summation notation, we see that from <math>B^{T}A^{T}</math>: <math>c_{23}=\sum_{k=1}^{3}b_{k3}a_{2k},</math> where the position of the index of summation is due to the transposition. So, in summation notation, the calculation of <math>c_{23}</math> from <math>B^{T}A^{T}</math> equals that from equation ([eq:c23]).<br />
<br />
More generally, the <math>\left(i,j\right)</math> element of <math>AB</math>: <math>\sum_{k=1}^{3}a_{ik}b_{kj}</math> is the <math>\left(j,i\right)</math> element of <math>B^{T}A^{T}.</math> But this means that <math>B^{T}A^{T}</math> must be the transpose of <math>AB,</math> since the elements in the <math>i</math>th row of <math>AB</math> are being written in the <math>i</math>th column of <math>B^{T}A^{T}.</math><br />
<br />
This ''Product Rule for Transposition'' can be applied again to find the transpose <math>\left(C^{T}\right)^{T}</math> of <math>C^{T}</math>: <math>\left(C^{T}\right)^{T}=\left(B^{T}A^{T}\right)^{T}=\left(A^{T}\right)^{T}\left(B^{T}\right)^{T}=AB=C.</math><br />
<br />
= Special Types of Matrix =<br />
<br />
== The zero matrix ==<br />
<br />
The most obvious special type of matrix is one whose elements are all zeros. In typical element notation, the zero matrix is: <math>0=\left\Vert 0\right\Vert .</math> Since there is no indexing on the elements, it is not obvious what the dimension of this matrix is, Sometimes one writes <math>0_{mn}</math> to indicate a zero matrix of dimension <math>m\times n.</math> The same ideas apply to vectors whose elements are all zero.<br />
<br />
The effect of the zero matrix in any product that is defined is simple: <math>0A=0,\ \ \ \ \ B0=0.</math> This is easy to check using the across and down rule.<br />
<br />
== The identity or unit matrix ==<br />
<br />
Vectors of the form:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{r}<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }2\ \text{dimensions}\\<br />
\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }3\ \text{dimensions}\\<br />
\left[\begin{array}{r}<br />
1\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1\\<br />
0\\<br />
\vdots\\<br />
0<br />
\end{array}\right],\ldots,\left[\begin{array}{r}<br />
0\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }n\ \text{dimensions}\end{aligned}</math><br />
<br />
are called coordinate vectors. They are often given a characteristic notation, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> in <math>n</math> dimensions. When arranged as columns of a matrix in the natural order, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> a matrix with a characteristic pattern elements emerges, with a special notation:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{rr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2}\end{array}\right] & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=I_{2}\\<br />
\left[\begin{array}{rrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \mathbf{e}_{3}\end{array}\right] & = & \left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 1 & 0\\<br />
0 & 0 & 1<br />
\end{array}\right]=I_{3}\\<br />
\left[\begin{array}{rrrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \ldots & \mathbf{e}_{n}\end{array}\right] & = & \left[\begin{array}{rrrr}<br />
1 & 0 & \ldots & 0\\<br />
0 & 1 & \ldots & 0\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
0 & 0 & \ldots & 1<br />
\end{array}\right]=I_{n}.\end{aligned}</math><br />
<br />
The ''diagonal'' of this matrix is where the 1 elements are located, and every other element is zero.<br />
<br />
Consider the effect of <math>I_{2}</math> on the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> by both pre and post multiplication:<br />
<br />
<math>\begin{aligned}<br />
I_{2}A & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\\<br />
AI_{2} & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\end{aligned}</math><br />
<br />
as is easily checked by the across and down rule.<br />
<br />
Because any matrix is left unchanged by pre or post multiplication by an appropriately dimensioned <math>I_{n},</math> <math>I_{n}</math> is called an ''identity matrix of dimension ''<math>n.</math> Sometimes it is called a ''unit matrix of dimension ''<math>n.</math> Notice that <math>I_{n}</math> is necessarily a square matrix.<br />
<br />
== Diagonal matrices ==<br />
<br />
The identity matrix is an example of a diagonal matrix, a matrix whose elements are all zero except for those on the diagonal. Usually diagonal matrices are taken to be square, for example: <math>D=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 2 & 0\\<br />
0 & 0 & 3<br />
\end{array}\right].</math> They also produce characteristic effects when pre or post multiplying another matrix.<br />
<br />
Consider the diagonal matrix: <math>B=\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]</math> and the products <math>AB,</math> <math>BA</math> for <math>A</math> as defined in the previous section:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & -4\\<br />
6 & -10<br />
\end{array}\right],\\<br />
BA & = & \left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & 4\\<br />
-6 & -10<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Comparing the results, we can deduce that post multiplication by a diagonal matrix multiplies each column of <math>A</math> by the corresponding diagonal element, whereas pre multiplication multiplies each row by the corresponding diagonal element.<br />
<br />
== Symmetric matrices ==<br />
<br />
Symmetric matrices are matrices having the property that <math>A=A^{T}.</math> Notice that such matrices must be square, since if <math>A</math> is <math>m\times n,</math> <math>A^{T}</math> is <math>n\times m,</math> and to have equality of <math>A</math> and <math>A^{T},</math> they must have the same dimension, so that <math>m=n</math> is required.<br />
<br />
Suppose that <math>A</math> is a <math>3\times3</math> symmetric matrix, with typical element <math>a_{ij}</math>: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right],</math> so that: <math>A^{T}=\left[\begin{array}{rrr}<br />
a_{11} & a_{21} & a_{31}\\<br />
a_{12} & a_{22} & a_{32}\\<br />
a_{13} & a_{23} & a_{33}<br />
\end{array}\right].</math> Equality of matrices is defined as equality of all elements. This is fine on the diagonals, since <math>A</math> and <math>A^{T}</math> have the same diagonal elements. For the off diagonal elements, we end up with the requirements: <math>a_{12}=a_{21},\ \ \ a_{13}=a_{31},\ \ \ a_{23}=a_{32}</math> or more generally: <math>a_{ij}=a_{ji}\ \ \ \ \ \text{for}\ i\neq j.</math><br />
<br />
The effect of this conclusion is that in a symmetric matrix, the ’triangle’ of above diagonal elements coincides with the triangle of below diagonal elements. It is as if the upper triangle is folded over the diagonal to become the lower triangle.<br />
<br />
A simple example is: <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
2 & 1<br />
\end{array}\right].</math> A more complicated example uses the <math>2\times3</math> matrix <math>C</math>: <math>C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]</math> and calculates the <math>3\times3</math> matrix:<br />
<br />
<math>\begin{aligned}<br />
C^{T}C & = & \left[\begin{array}{cc}<br />
6 & 3\\<br />
2 & 5\\<br />
-3 & -1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{rrr}<br />
45 & 27 & -21\\<br />
27 & 29 & -11\\<br />
-21 & -11 & 10<br />
\end{array}\right],\end{aligned}</math><br />
<br />
which is clearly symmetric.<br />
<br />
This illustrates the general proposition that if <math>A</math> is an <math>m\times n</math> matrix, the product <math>A^{T}A</math> is a symmetric <math>n\times n</math> matrix. Proof? Compute the transpose of <math>A^{T}A</math> using the product rule for transposition: <math>\left(A^{T}A\right)^{T}=A^{T}\left(A^{T}\right)^{T}=A^{T}A.</math> Since <math>A^{T}A</math> is equal to its transpose, it must be a symmetric matrix. Such symmetric matrices appear frequently in econometrics.<br />
<br />
It should be clear that diagonal matrices are symmetric, since all their off-diagonal elements are equal (zero), and thence the identity matrix <math>I_{n}</math> is also symmetric.<br />
<br />
== The outer product ==<br />
<br />
The inner product of two <math>n\times1</math> vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>, <math>\mathbf{x}^{T}\mathbf{y}</math>, is automatically a <math>1\times1</math> quantity, a scalar, although it can be interpreted as a <math>1\times1</math> matrix, a matrix with a single element.<br />
<br />
Suppose one considered the product of <math>\mathbf{x}</math> with <math>\mathbf{x}^{T}.</math> Is this defined? If <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times r,</math> then the product <math>AB</math> is <math>m\times r.</math> Applying this logic to <math>\mathbf{xx}^{T},</math> this is <math>\left(n\times1\right)\left(1\times n\right),</math> so the resulting product ''is ''defined, and is an <math>n\times n</math>'' matrix'' - the ''outer product'' of <math>\mathbf{x}</math> and <math>\mathbf{x}^{T},</math> the word ’outer’ being used to distinguish from the inner product.<br />
<br />
How does the across and down rule work here? Suppose that: <math>\mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> Then: <math>\mathbf{xx}^{T}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right].</math> Here, there is <math>1</math> element in row one of the ’matrix’ <math>\mathbf{x,}</math> and <math>1</math> element in column one of the matrix <math>\mathbf{x}^{T},</math> so the across and down rule still works - it is just that there is only one product per row and column combination. So: <math>\mathbf{xx}^{T}=\left[\begin{array}{cc}<br />
36 & 18\\<br />
18 & 9<br />
\end{array}\right],</math> and it is obvious from this that <math>\mathbf{xx}^{T}</math> is a symmetric matrix.<br />
<br />
One can see that this outer product need not be restricted to vectors of the same dimension. If <math>\mathbf{x}</math> is <math>n\times1</math> and <math>\mathbf{y}</math> is <math>m\times1,</math> then: <math>\mathbf{xy}^{T}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
y_{1} & \ldots & y_{m}\end{array}\right]=\left[\begin{array}{rrrr}<br />
x_{1}y_{1} & x_{1}y_{2} & \ldots & x_{1}y_{m}\\<br />
x_{2}y_{1} & x_{2}y_{2} & \ldots & x_{2}y_{m}\\<br />
\\<br />
x_{n}y_{1} & x_{n}y_{2} & \ldots & x_{n}y_{m}<br />
\end{array}\right].</math> So, <math>\mathbf{xy}^{T}</math> is <math>n\times m,</math> and consists of rows which are <math>\mathbf{y}^{T}</math> multiplied by an element of the <math>\mathbf{x}</math> vector.<br />
<br />
Another interesting and useful example involves a vector with every element equal to <math>1</math>: <math>\mathbf{1}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right].</math> Sometimes this is written as <math>\mathbf{1}_{n}</math> to indicate an <math>n\times1</math> vector, and is called the ''sum vector''. Why? Consider the impact of <math>\mathbf{1}_{2}</math> on the <math>2\times1</math> vector <math>\mathbf{x}</math> used above: <math>\mathbf{1}_{2}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=9,</math> i.e. an inner product of <math>\mathbf{x}</math> with the sum vector is the sum of the elements of <math>\mathbf{x.}</math> Dividing through by the number of elements in <math>\mathbf{x}</math> produces the average of the elements of <math>\mathbf{x}</math> - i.e. the ’sample mean’ of the elements of <math>\mathbf{x.}</math><br />
<br />
The outer product of <math>\mathbf{x}</math> with <math>\mathbf{1}_{2}</math> is also interesting:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{1}_{2}\mathbf{x}^{T} & = & \left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 3\\<br />
6 & 3<br />
\end{array}\right],\\<br />
\mathbf{x1}_{2}^{T} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 6\\<br />
3 & 3<br />
\end{array}\right],\end{aligned}</math><br />
<br />
showing that pre multiplication of an <math>\mathbf{x}^{T}</math> by <math>\mathbf{1}</math> repeats <math>\mathbf{x}^{T}</math> as rows of the product, whilst post multiplication of <math>\mathbf{x}</math> by <math>\mathbf{1}^{T}</math> repeats <math>\mathbf{x}</math> as the columns of the product.<br />
<br />
Finally: <math>\mathbf{1}_{n}\mathbf{1}_{n}^{T}=\left[\begin{array}{rrr}<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1<br />
\end{array}\right],</math> an <math>n\times n</math> matrix with every element equal to <math>1.</math> This type of matrix also appears in econometrics!<br />
<br />
== Triangular matrices ==<br />
<br />
A square ''lower triangular ''matrix has all elements above the main diagonal equal to zero, whilst a square ''upper triangular ''matrix has all elements below the main diagonal equal to zero. A simple example of a lower triangular matrix is: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & 0 & 0\\<br />
a_{21} & a_{22} & 0\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right].</math> Clearly, for this matrix, <math>A^{T}</math> is an upper triangular matrix.<br />
<br />
One can adapt the definition to rectangular matrices: for example, if two arbitrary rows are added to <math>A,</math> so that it becomes <math>5\times3,</math> it would still be considered lower triangular. Equally, if, for example, the third column of <math>A</math> above is removed, <math>A</math> is still considered lower triangular.<br />
<br />
Often, we use ''unit ''triangular matrices, where the diagonal elements are all equal to <math>1:</math> e.g.:<br />
<br />
<math>\left[\begin{array}{rrr}<br />
1 & 2 & 0\\<br />
0 & 1 & 1\\<br />
0 & 0 & 1<br />
\end{array}\right].\label{eq:lt_matrix}</math><br />
<br />
== Partitioned matrices ==<br />
<br />
Sometimes, especially with big matrices, it is useful to organise the elements of the matrix into components which are themselves matrices, for example: <math>B=\left[\begin{array}{rrrr}<br />
1 & 2 & 0 & 0\\<br />
8 & 3 & 0 & 0\\<br />
0 & 0 & 7 & 4\\<br />
0 & 0 & 6 & 5<br />
\end{array}\right]</math> Here it would be reasonable to write: <math>B=\left[\begin{array}{cc}<br />
B_{11} & 0\\<br />
0 & B_{22}<br />
\end{array}\right],</math> where <math>B_{ii},i=1,2,</math> represent <math>2\times2</math> matrices. <math>B</math> is an example of a ''partitioned matrix'': that is, an <math>m\times n</math> matrix <math>A</math> say: <math>A=\left\Vert a_{ij}\right\Vert ,</math> where the elements of <math>A</math> are organised into ''sub-matrices''. An example might be:<br />
<br />
<math>A=\left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right],\label{eq:partition_a}</math><br />
<br />
where the ''sub - matrices'' in the first row block have <math>r</math> rows, and therefore <math>m-r</math> rows in the second row block. The column blocks might be defined by (for example) 3 columns in the first column block, 4 in the second and <math>n-7</math> in the third column block.<br />
<br />
Another simple example might be: <math>A=\left[\begin{array}{rrr}<br />
A_{1} & A_{2} & A_{3}\end{array}\right],\ \ \ \ \ \mathbf{x=}\left[\begin{array}{c}<br />
\mathbf{x}_{1}\\<br />
\mathbf{x}_{2}\\<br />
\mathbf{x}_{3}<br />
\end{array}\right],</math> where <math>A</math> and therefore <math>A_{1},A_{2},A_{3}</math> have <math>m</math> rows, <math>A_{1}</math> has <math>n_{1}</math> columns, <math>A_{2}</math> has <math>n_{2}</math> columns, <math>A_{3}</math> has <math>n_{3}</math> columns. The ''subvectors'' in <math>\mathbf{x}</math> must have <math>n_{1},n_{2}</math> and <math>n_{3}</math> rows respectively, for the product <math>A\mathbf{x}</math> to exist.<br />
<br />
Suppose that <math>n_{1}+n_{2}+n_{3}=n,</math> so that <math>A</math> is <math>m\times n.</math> The <math>i</math>th element of <math>A\mathbf{x}</math> is: <math>\sum_{i=1}^{n}a_{ij}x_{j}</math> but the summation can be broken up into the first <math>n_{1}</math> terms: <math>\sum_{i=1}^{n_{1}}a_{ij}x_{j},</math> the next <math>n_{2}</math> terms: <math>\sum_{i=n_{1}+1}^{n_{1}+n_{2}}a_{ij}x_{j},</math> and the next <math>n_{3}</math> terms; <math>\sum_{i=n_{1}+n_{2}+1}^{n}a_{ij}x_{j}.</math><br />
<br />
The point about the use of partitioned matrices is that the product <math>A\mathbf{x}</math> can be represented as: <math>A\mathbf{x}=A_{1}\mathbf{x}_{1}+A_{2}\mathbf{x}_{2}+A\mathbf{x}_{3}</math> by applying the across and down rule to the submatrices and the subvectors, a much simpler representation than the use of summations.<br />
<br />
Each of the components is a conformable matrix-vector product: this is essential in any use of partitioned matrices to represent some matrix product. For example, using <math>A</math> from equation ([eq:partition<sub>a</sub>]) and <math>B</math> as: <math>B=\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right],</math> it is easy to write:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right]\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{r}<br />
A_{11}B_{11}+A_{12}B_{21}+A_{13}B_{31}\\<br />
A_{21}B_{11}+A_{22}B_{21}+A_{23}B_{31}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
But, what are the row dimensions for the submatrices in <math>B?</math> What are the possible column dimensions for the submatrices in <math>B?</math><br />
<br />
= Matrices, vectors and econometrics =<br />
<br />
The data on weights and heights for 12 students in the data matrix: <math>D=\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]</math> would seem to be ideally suited for fitting a two variable regression model: <math>y_{i}=\alpha+\beta x_{i}+u_{i},\;\;\;\;\; i=1,...,12.</math> Here, the first column of <math>D</math> contains all the weight data, the data on the dependent variable <math>y_{i},</math> and so should be labelled <math>\mathbf{y.}</math> The second column of <math>D</math> contains all the data on the explanatory variable height, in the vector <math>\mathbf{x}</math> say, so that: <math>D=\left[\begin{array}{rr}<br />
\mathbf{y} & \mathbf{x}\end{array}\right].</math><br />
<br />
If we define a <math>12\times1</math> vector with every element <math>1</math>: <math>\mathbf{1}_{12}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right],</math> and a <math>12\times1</math> vector <math>\mathbf{u}</math> to contain the error terms; <math>\mathbf{u}=\left[\begin{array}{c}<br />
u_{1}\\<br />
\vdots\\<br />
u_{12}<br />
\end{array}\right]</math> the regression model can be written in terms of the three data vectors <math>\mathbf{y,1}_{12}</math> and <math>\mathbf{x}</math> as: <math>\mathbf{y}=\mathbf{1}_{12}\alpha+\mathbf{x}\beta+\mathbf{u.}</math> To see this, think of the <math>i</math>th elements of the vectors on the left and right hand sides.<br />
<br />
The standard next step is then to combine the data vectors for the explanatory variables into a matrix: <math>X=\left[\begin{array}{rr}<br />
\mathbf{1}_{12} & \mathbf{x}\end{array}\right],</math> and then define a <math>2\times1</math> vector <math>\boldsymbol{\delta}</math> to contain the parameters <math>\alpha,\beta</math> as: <math>\boldsymbol{\delta}=\left[\begin{array}{r}<br />
\alpha\\<br />
\beta<br />
\end{array}\right]</math> to give the data matrix representation of the regression model as: <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
<br />
For the purposes of developing the theory of regression, this is the most convenient form of the regression model. It can represent regression models with any number of explanatory variables, and thus any number of parameters. The obvious point is that a knowledge of vector and matrix operations is needed to use and understand this form.<br />
<br />
We shall see later that there are two particular matrix and vector quantities associated with a regression model. The first is the matrix <math>X^{T}X,</math> and the second the vector <math>X^{T}\mathbf{y.}</math> The following Matlab code snippet provides the numerical values of these quantities for the weight data:<br />
<br />
&gt;&gt; dset = load(’weights.mat’); <br />
<br />
&gt;&gt; xtx = dset.X’ * dset.X; <br />
<br />
&gt;&gt; xty = dset.X’ * dset.y; <br />
<br />
&gt;&gt; disp(xtx) <br />
<br />
12 802<br />
<br />
802 53792<br />
<br />
&gt;&gt; disp(xty)<br />
<br />
1850<br />
<br />
124528<br />
<br />
Hand calculation is of course possible, but not recommended.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3041Maths2013-09-10T14:52:33Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
The lecture notes are also available here,<br />
<br />
[[Lnotes|Lnotes]]<br />
<br />
and the corresponding exercise sheet is<br />
<br />
[[Exercise Sheet 2|XS2]]<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material, and test their understanding using Maple TA.<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments: there is usually a delay whilst they are loaded. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+, -, *, /, ^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be too hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Exercise_Sheet_2&diff=3037Exercise Sheet 22013-09-10T14:41:57Z<p>LG: </p>
<hr />
<div>= Introduction =<br />
<br />
Blackboard is the University’s virtual learning environment. All of the material for this module, including formative assessments, and summative assessments, will be available in Blackboard.<br />
<br />
You will normally enter the Blackboard site for this course through [https://my.manchester.ac.uk/ MyManchester] at https://my.manchester.ac.uk/ .<br />
<br />
All of the Blackboard material for the course is organised into Lecture topics, so that lecture notes, lecture slides, and exercise sheets can be found in the folder for each Lecture topic.<br />
<br />
The questions on each exercise sheet may be answered in the traditional way, on paper, and handed in to be marked. Alternatively you can answer the majority (but not all) of the exercise sheet questions online, using Maple TA. All of the questions on this exercise sheet can be answered in Maple TA. The name of the matching Maple TA assignment is indicated in each question in this exercise sheet.<br />
<br />
= Using Maple TA =<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. In this module, most of the questions on the paper Exercise Sheets are also available as Maple TA assignments. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
http://place36.placementtester.com/manchester<br />and this link is also given in each Lecture folder, for convenience. Login with your registration number (first 7 digits only): the password is also your registration number. Once you have logged in, there is generally a wait whilst the assignments are loaded. On the page that follows, you can click on MyProfile and then Password Update to change your password.<br />
<br />
You should select the course<br />
<br />
ECON61001 Econometric Methods 2013-14<br />by clicking on the entry for this course. This will bring up a page of assignments. You can click on the assignment you want to do - the notation follows that in the exercise sheets. The assignments are organised by question group, as in the Exercise Sheets, or by individual question - a component of the question group. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between “Print assignment for off-line work” or Work assignment on-line right now. If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the Work ... online option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (<math> +, -, *, /, ^ </math>) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by <math>1/x-1</math> - is it <math>(1/x)-1</math> or <math>1/(x-1)</math>? Information about the entry of vectors and matrices in your answers is given in the next section, although the instructions are often repeated in questions.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and View Details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
Most of the Exercise Sheets have additional randomised questions associated with them in Maple TA. These are questions where Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These randomised questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be to hard, simply click on the Refresh button at the top of the page to get another question.<br />
<br />
= Using the equation editor in Maple TA =<br />
<br />
Usually it is quicker to enter your answers directly in Maple TA rather than using the Equation Editor. Using the Equation Editor is straightforward: see Figure 1. To select a symbol, click on the required panel and select the symbol required. Figure 2 shows the subscript and superscript panel, and Figure 3 shows the array selection panel. Usage is fairly self-evident - keep trying until you find the required symbol.<br />
<br />
Figure 1<br />
<br />
[[File:equation_editor001.png]]<br />
<br />
Figure 2<br />
<br />
[[File:equation_editor_subscripts.png]]<br />
<br />
Figure 3<br />
<br />
[[File:equation_editor_arrays.png]]<br />
<br />
= Entering vectors and matrices into Maple TA =<br />
<br />
Many questions on this exercise sheet require vectors, matrices or indexed expression as their answers. Some questions provide an array of the right size for the answer, so that none of the methods below are required.<br />
<br />
There are a variety of ways of entering these expressions into Maple TA - choose the way you feel most comfortable in using.<br />
<br />
# Vector entry.<br />
## You can use Maple “text entry” for column vectors, e.g. for the <math>3\times1</math> vector,: <math>\left(\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right),</math> enter (without the quotes) Vector(<math>[1,2,3]</math>).<br />
## A quicker version is <math><1,2,3></math>.<br />
## For a row vector, use <math><1|2|3></math>, as above, or Vector[row](<math>[1,2,3]</math>).<br />
## Or, you can right click in the “Equation Editor”, select the array object, choose the correct matrix size and enter the elements of the vector.<br />
# Matrix entry.<br />
## You can use Maple “text entry” for matrices, e.g. for a <math>2\times3</math> matrix, enter (without the quotes) Matrix(<math>[[1,2,3],[345]]</math>) for the matrix: <math>\left(\begin{array}{ccc}<br />
1 & 2 & 3\\<br />
4 & 5 & 6<br />
\end{array}\right).</math> For more columns, include more elements in each [.] block. For more rows, add more [.] blocks, each separated by a comma. This is “row - by - row” construction.<br />
## A quicker version is <math><1,2,3;4,5,6></math>.<br />
## Column by column construction is also possible using <math><1,4|2,5|3,6></math> or <math><<1,4>|<2,5>|<3,6>></math>.<br />
## A matrix with one row, say <math>\left[\begin{array}{ccc}<br />
1 & 2 & 3\end{array}\right]</math> can be entered as Matrix(<math>[1,2,3]</math>), or as <math><1|2|3></math>.<br />
## A matrix with one column, say <math>\left[\begin{array}{c}<br />
1\\<br />
4<br />
\end{array}\right],</math> can be entered as Matrix(<math>[[1],[4]]</math>) or as <math><1,4></math>.<br />
## Or, you can right click in the “Equation Editor”, select the array object, choose the correct matrix size and enter the elements of the product.<br />
# Entering indexed elements in vectors and matrices - the methods below can be used with any of the vector or matrix entry schemes.<br />
## To insert <math>x_{1}</math> into an answer, you can use <math>x[1]</math>, or you can use the Equation Editor. NB: select the right palette in the equation editor '''before''' entering any symbols!<br />
## To insert <math>x_{11},</math> use x[1,1], or use the Equation Editor, subject to the previous warning.<br />
<br />
On the whole, the entry methods using <math><...></math> are the quickest to use.<br />
<br />
Arithmetic operators are best shown explicitly in entering answers. These operators are the standard ones, <math>+,-,*,/</math> and .<br />
<br />
= Question 1: Vectors =<br />
<br />
# (XS1 Q1 Part 1) Given the vectors: <math>\mathbf{0}=\left[\begin{array}{r}<br />
0\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right],</math> find: <math>\mathbf{0}-\mathbf{x,x}+\mathbf{y,\mathbf{x-y},x-}2\mathbf{z,}3\mathbf{z}-2\mathbf{y,x}-2\mathbf{y}+\mathbf{z.}</math><br />
# (XS1 Q1 Part 2) Can you find a linear combination <math>\alpha\mathbf{x}+\beta\mathbf{y}</math> of <math>\mathbf{x=}\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right]</math> and <math>\mathbf{y=}\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]</math> to generate (in turn) the vectors: <math>\mathbf{w}_{1}=\left[\begin{array}{c}<br />
4\\<br />
-2<br />
\end{array}\right],\mathbf{w}_{2}=\left[\begin{array}{r}<br />
-2\\<br />
4<br />
\end{array}\right],\mathbf{w}_{3}=\left[\begin{array}{r}<br />
-2\\<br />
2<br />
\end{array}\right]?</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 1 Part 1 and Part 2.<br />
<br />
= Question 2: 3 - d vectors =<br />
<br />
# (XS1 Q2 Part 2) For the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
0<br />
\end{array}\right]</math> find <math>-2\mathbf{x,\ }3\mathbf{x}+2\mathbf{y,\ }4\mathbf{x}-2\mathbf{y.}</math><br />
# (XS1 Q2 Part 2) For the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right]</math> find <math>10\mathbf{x}-\alpha\mathbf{y,}\beta\mathbf{y.}</math><br />
# (XS1 Q2 Part 3) Use the vector <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right]</math> and the vector <math>\mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right]</math> to find <math>2\mathbf{x-z,}</math> <math>\mathbf{x}+3\mathbf{z.}</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 2 Part 1 and Part 2.<br />
<br />
= Question 3: Matrices =<br />
<br />
# (XS1 Q3 Part 1) Arrange the vectors: <math>\mathbf{0}=\left[\begin{array}{r}<br />
0\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right].</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> Find a suitable vector <math>\boldsymbol{\alpha}</math> so that the product <math>A\boldsymbol{\alpha}</math> is equal to the following linear combinations in turn: <math>\mathbf{0}-\mathbf{x,\ x}+\mathbf{y,\ \mathbf{x-y},\ 2z}-\mathbf{x,}</math>''' '''<math>\mathbf{y}-2\mathbf{z,\ x}+\mathbf{y}-\mathbf{z.}</math><br />
# (XS1 Q3 Part 2) Arrange the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
0<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> Find a suitable vector <math>\boldsymbol{\alpha}</math> so that the product <math>A\boldsymbol{\alpha}</math> is equal to the following linear combinations in turn: <math>-2\mathbf{x,\ }3\mathbf{x}+2\mathbf{y,\ }4\mathbf{x}-2\mathbf{y.}</math><br />
# (XS1 Q3 Part 3) Arrange the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math><br />
# (XS1 Q3 Part 4) Arrange the vectors <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> If <math>\boldsymbol{\alpha}=\left[\begin{array}{c}<br />
2\\<br />
1\\<br />
3<br />
\end{array}\right]</math> find <math>A\boldsymbol{\alpha.}</math><br />
# (XS1 Q3 Part 5) Arrange the vectors : <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],\ \ \ \mathbf{w}=\left[\begin{array}{c}<br />
-1\\<br />
-1\\<br />
1<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> If <math>\boldsymbol{\alpha}=\left[\begin{array}{c}<br />
0\\<br />
-1/2\\<br />
-1/2<br />
\end{array}\right]</math> find <math>A\boldsymbol{\alpha.}</math><br />
# (XS1 Q3 Part 6) Arrange the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],\ \ \ \mathbf{w}=\left[\begin{array}{c}<br />
-1\\<br />
-1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{v}=\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> If <math>\boldsymbol{\alpha}=\left[\begin{array}{c}<br />
0\\<br />
1/2\\<br />
1/2\\<br />
1<br />
\end{array}\right]</math> find <math>A\boldsymbol{\alpha}.</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 3 Part 1, Part 2, Part 4, Part 5 and Part 6.<br />
<br />
= Question 4: Inner products =<br />
<br />
# (XS1 Q4 Part 1) Find the inner product of the vectors: <math>\mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].</math> What is <math>\mathbf{x}^{T}\mathbf{x}</math> and <math>\mathbf{y^{T}y?}</math> Is it true that <math>\left(\mathbf{x}^{T}\mathbf{y}\right)^{2}\leq</math> <math>\mathbf{x}^{T}\mathbf{x\times y^{T}y?}</math> This is called the Cauchy-Schwartz Inequality; equality holds only when <math>\mathbf{x}=\lambda\mathbf{y,}</math> for some <math>\lambda.</math><br />
# (XS1 Q4 Part 2) Find a vector <math>\mathbf{z}</math> with the property that <math>\mathbf{x}^{T}\mathbf{z}=0</math> for <math>\mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right].</math> Draw a diagram showing <math>\mathbf{x}</math> and <math>\mathbf{z:}</math> how would you describe the relationship between <math>\mathbf{x}</math> and <math>\mathbf{z?}</math><br />
# (XS1 Q4 Part 3) For: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1/3\\<br />
1/3\\<br />
1/3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{c}<br />
6\\<br />
2\\<br />
-1<br />
\end{array}\right]</math> find <math>\mathbf{x}^{T}\mathbf{y,\ x}^{T}\mathbf{z,\ y}^{T}\mathbf{z.}</math> How would you describe the value of these inner products from a statistical perspective?<br />
# (XS1 Q4 Part 4) If <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{10}<br />
\end{array}\right],</math> what is <math>\mathbf{x}^{T}\mathbf{x?}</math><br />
# (XS1 Q4 Part 5) If <math>\mathbf{z}=\left[\begin{array}{c}<br />
3\\<br />
7\\<br />
1\\<br />
9<br />
\end{array}\right]</math> and <math>\mathbf{1}_{4}</math> is a <math>4\times1</math> vector with every element equal to <math>1,</math> what is the quantity <math>c=\dfrac{1}{4}\mathbf{1}_{4}^{T}\mathbf{z?}</math> Find the elements of the vector <math>\mathbf{z}-c\mathbf{1}_{4}.</math> From a statistical perspective, what are the elements of this vector? Find the value of the inner product <math>\mathbf{1}_{4}^{T}\left(\mathbf{z}-c\mathbf{1}_{4}\right).</math> What statistical information does this illustrate?<br />
# (XS1 Q4 Part 6) Using <math>\mathbf{z}=\left[\begin{array}{c}<br />
3\\<br />
7\\<br />
1\\<br />
9<br />
\end{array}\right],</math> <math>\mathbf{1}_{4}</math> and <math>c</math> from part (5), what is the inner product of <math>\left(\mathbf{z}-c\mathbf{1}_{4}\right)</math> with itself? If this distance is divided by <math>3</math> or <math>4,</math> what statistical quantity is the result?<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 4 Part 1, Part 2, Part 3, and Part 4.<br />
<br />
= Question 5: Across and down =<br />
<br />
# (XS1 Q5 Part 1) Suppose that <math>B</math> is an <math>r\times s</math> matrix with typical element <math>\left\Vert b_{ij}\right\Vert ,</math> and <math>\mathbf{z}</math> is an <math>s\times1</math> vector, with typical element <math>z_{j}.</math> What is the second element of the product <math>B\mathbf{z?}</math> What is the <math>k</math>th element?<br />
# (XS1 Q5 Part 2) If <math>A=\left[\begin{array}{rrr}<br />
2 & 1 & 3\\<br />
1 & 0 & 4<br />
\end{array}\right],</math> and <math>\mathbf{b}_{1}=\left[\begin{array}{r}<br />
4\\<br />
1\\<br />
2<br />
\end{array}\right],</math> <math>\mathbf{b}_{2}=\left[\begin{array}{r}<br />
1\\<br />
2\\<br />
-1<br />
\end{array}\right],</math> find <math>A\mathbf{b}_{1},</math> <math>A\mathbf{b}_{2}.</math><br />
# (XS1 Q5 Part 3) If <math>A=\left[\begin{array}{rrr}<br />
2 & 1 & 3\\<br />
1 & 0 & 4<br />
\end{array}\right],</math> and <math>\mathbf{b}_{1}=\left[\begin{array}{r}<br />
4\\<br />
1\\<br />
2<br />
\end{array}\right],</math> <math>\mathbf{b}_{2}=\left[\begin{array}{r}<br />
1\\<br />
2\\<br />
-1<br />
\end{array}\right],</math> make <math>\mathbf{b}_{1}</math> and <math>\mathbf{b}_{2}</math> the columns of a matrix <math>B</math> and find the matrix which is equal to <math>AB.</math><br />
# (XS1 Q5 Part 4) If <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
3 & 4<br />
\end{array}\right]</math> and <math>B=\left[\begin{array}{cc}<br />
1 & 4\\<br />
6 & 7<br />
\end{array}\right],</math> find <math>AB</math> and <math>BA.</math> What property do the matrices <math>A</math> and <math>B</math> have?<br />
# (XS1 Q5 Part 5) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{10}=\left[\begin{array}{rr}<br />
-3 & 2\\<br />
4 & 9\\<br />
1 & -2<br />
\end{array}\right],</math> find <math>A_{4}A_{10}</math> and <math>A_{10}A_{4}.</math> Why are <math>A_{4}A_{10}</math> and <math>A_{10}A_{4}</math> not equal?<br />
# (XS1 Q5 Part 6) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{11}=\left[\begin{array}{rrr}<br />
2 & 4 & 4\\<br />
3 & 1 & -1\\<br />
2 & 0 & 4<br />
\end{array}\right],</math> find <math>A_{4}A_{11}</math> and <math>A_{11}A_{4}.</math><br />
# (XS1 Q5 Part 7) If <math>A_{11}=\left[\begin{array}{rrr}<br />
2 & 4 & 4\\<br />
3 & 1 & -1\\<br />
2 & 0 & 4<br />
\end{array}\right]</math> and <math>A_{12}=\left[\begin{array}{rrr}<br />
3 & 0 & -1\\<br />
-2 & -1 & -1\\<br />
1 & 2 & 3<br />
\end{array}\right]</math> , find <math>A_{11}A_{12}</math> and <math>A_{12}A_{11}.</math><br />
# (XS1 Q5 Part 8) If <math>A_{1}=\left[\begin{array}{rrr}<br />
4 & 0 & -3\\<br />
1 & -1 & 4\\<br />
2 & 2 & -11<br />
\end{array}\right]</math> and <math>A_{8}=\left[\begin{array}{rrr}<br />
4 & 2 & 6\\<br />
1 & 3 & 4\\<br />
5 & 0 & 5<br />
\end{array}\right]</math> , find <math>2A_{8}-3A_{1}.</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 5 Part 1, Part 2, Part 4, and Part 8.<br />
<br />
= Question 6: Transposition =<br />
<br />
<ol><br />
<li><p>(XS1 Q6 Part 1) Find the transpose of the following matrices:</p><br />
<p><math>\begin{aligned}<br />
A_{1} & = & \left[\begin{array}{cc}<br />
-2 & 3\\<br />
3 & 5<br />
\end{array}\right];\ \ \ A_{2}=\left[\begin{array}{rrr}<br />
1 & 0 & 9\\<br />
6 & -2 & 15<br />
\end{array}\right];\ \ \ A_{3}=\left[\begin{array}{rr}<br />
1 & 1\\<br />
2 & -1\\<br />
0 & 0<br />
\end{array}\right];\\<br />
A_{4} & = & \left[\begin{array}{rrr}<br />
1 & 1 & -1\\<br />
1 & -1 & -1\\<br />
1 & -1 & 1<br />
\end{array}\right];\ \ \ A_{5}=\left[\begin{array}{rrr}<br />
1 & 1 & -1\\<br />
1 & -1 & -1\\<br />
-1 & -1 & 1<br />
\end{array}\right];\ \ \ A_{6}=\left[\begin{array}{rr}<br />
4 & 2\\<br />
3 & 5\\<br />
-2 & 3<br />
\end{array}\right].\end{aligned}</math></p></li><br />
<li><p>(XS1 Q6 Part 1) What is the dimension of <math>A^{T}</math> for each matrix in part (1)?</p></li><br />
<li><p>Check that <math>\left(A^{T}\right)^{T}=A</math> for each of the matrices in part (1).</p></li><br />
<li><p>(XS1 Q6 Part 1) What properties do the matrices <math>A_{1}</math> and <math>A_{5}</math> have that are not shared by the other matrices in part (1)?</p></li><br />
<li><p>(XS1 Q6 Part 5) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{10}=\left[\begin{array}{rr}<br />
-3 & 2\\<br />
4 & 9\\<br />
1 & -2<br />
\end{array}\right],</math> find <math>A_{4}^{T}</math> and <math>A_{10}^{T}.</math> Confirm that <math>\left(A_{4}A_{10}\right)^{T}=A_{10}^{T}A_{4}^{T}.</math></p></li></ol><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 6 Part 1, Part 4, and Part 5.<br />
<br />
= Question 7: Special Matrices =<br />
<br />
# (XS1 Q7 Part 1) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right],</math> confirm that <math>I_{2}A_{4}=A_{4}I_{3}.</math><br />
# (XS1 Q7 Part 2) If <math>D=\left[\begin{array}{cc}<br />
3 & 0\\<br />
0 & 5<br />
\end{array}\right],</math> <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{10}=\left[\begin{array}{rr}<br />
-3 & 2\\<br />
4 & 9\\<br />
1 & -2<br />
\end{array}\right],</math> find <math>DA_{4}</math> and <math>A_{10}D.</math> What pattern can you detect in the results?<br />
# (XS1 Q7 Part 3) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right],</math> find <math>A_{4}A_{4}^{T}</math> and <math>A_{4}^{T}A_{4}.</math> Are these two matrices equal? What property do these two matrices possess?<br />
# (XS1 Q7 Part 4) If <math>\mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right]</math> and <math>\mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],</math> find <math>\mathbf{xx}^{T}</math>, <math>\mathbf{yy}^{T}</math>. Are these symmetric? Do they equal <math>\mathbf{x}^{T}\mathbf{x,}</math> <math>\mathbf{y}^{T}\mathbf{y?}</math> Find <math>\mathbf{xy}^{T}:</math> is <math>\mathbf{xy}^{T}</math> equal to <math>\mathbf{yx}^{T}</math> or <math>\mathbf{y}^{T}\mathbf{x?}</math><br />
# (XS1 Q7 Part 5) If <math>L=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
2 & 1 & 0\\<br />
3 & 2 & 1<br />
\end{array}\right],</math> show that <math>L^{T}</math> is an upper triangular matrix.<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 7 Part 3 and 4.<br />
<br />
= Question 8: Partitioned matrices =<br />
<br />
# (XS1 Q8 Part 1) Write <math>A_{2}=\left[\begin{array}{rrrr}<br />
1 & 2 & 1 & -3\\<br />
6 & 0 & 10 & 9\\<br />
2 & 0 & 3 & 4<br />
\end{array}\right]</math> in the form <math>A_{2}=\left[\begin{array}{rr}<br />
B_{1} & B_{2}\end{array}\right],</math> where both <math>B_{1}</math> and <math>B_{2}</math> have two columns. What are <math>B_{1}</math> and <math>B_{2}?</math> Write <math>A_{9}=\left[\begin{array}{rr}<br />
2 & 0\\<br />
-1 & 3\\<br />
4 & -2\\<br />
7 & 7<br />
\end{array}\right]</math> as <math>A_{9}=\left[\begin{array}{r}<br />
C_{1}\\<br />
C_{2}<br />
\end{array}\right],</math> where both <math>C_{1}</math> and <math>C_{2}</math> have 2 rows. Express the product <math>A_{2}A_{9}</math> in terms of <math>B_{1}</math>, <math>B_{2},</math> <math>C_{1}</math> and <math>C_{2}.</math> Do the same for the product <math>A_{9}A_{2},</math> carefully stating the dimensions of any submatrices.<br />
# (XS1 Q8 Part 1) If <math>A_{2}=\left[\begin{array}{rrrr}<br />
1 & 2 & 1 & -3\\<br />
6 & 0 & 10 & 9\\<br />
2 & 0 & 3 & 4<br />
\end{array}\right]=\left[\begin{array}{rr}<br />
B_{1} & B_{2}\end{array}\right],</math> where both <math>B_{1}</math> and <math>B_{2}</math> have two columns, and <math>\mathbf{z}=\left[\begin{array}{c}<br />
5\\<br />
1\\<br />
-3\\<br />
2<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\mathbf{z}_{1}\\<br />
\mathbf{z}_{2}<br />
\end{array}\right]</math> where <math>\mathbf{z}_{i}</math> is <math>2\times1,</math> given an expression for <math>A_{2}\mathbf{z}</math> in terms of <math>B_{1},B_{2},\mathbf{z}_{1},\mathbf{z}_{2}.</math><br />
# (XS1 Q8 Part 3) If <math>A=\left[\begin{array}{cc}<br />
A_{11} & A_{12}\\<br />
A_{21} & A_{22}<br />
\end{array}\right]</math> and <math>\mathbf{z}=\left[\begin{array}{r}<br />
\mathbf{z}_{1}\\<br />
\mathbf{z}_{2}<br />
\end{array}\right],</math> where <math>A_{11}</math> is <math>2\times2,</math> <math>A_{12}</math> is <math>2\times1,</math> <math>A_{21}</math> is <math>1\times2,</math> <math>A_{22}</math> <math>1\times1</math> and <math>\mathbf{z}_{1}</math> is <math>1\times1,</math> <math>\mathbf{z}_{2}</math> <math>2\times1,</math> is the matrix <math>A</math> well defined? Does the product <math>A\mathbf{z}=\left[\begin{array}{cc}<br />
A_{11} & A_{12}\\<br />
A_{21} & A_{22}<br />
\end{array}\right]\left[\begin{array}{r}<br />
\mathbf{z}_{1}\\<br />
\mathbf{z}_{2}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
A_{11}\mathbf{z}_{1}+A_{12}\mathbf{z}_{2}\\<br />
A_{21}\mathbf{z}_{1}+A_{22}\mathbf{z}_{2}<br />
\end{array}\right]</math> exist in this form?<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 8 Part 1.<br />
<br />
= Question 9: Data matrices =<br />
<br />
A small data set on variables <math>y</math> and <math>x</math> is: <math>D=\left[\begin{array}{rr}<br />
14 & 2\\<br />
17 & 4\\<br />
8 & 3\\<br />
16 & 5\\<br />
3 & 2<br />
\end{array}\right].</math><br />
<br />
# (XS1 Q9 Part 1) Define a vector of observations on <math>y</math> and a matrix <math>X</math> so that the two variable regression model <math>y_{i}=\alpha+\beta x_{i}+u_{i}</math> can be represented, using the data in <math>D,</math> as <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
# (XS1 Q9 Part 1) Compute for your choice of <math>X,</math> <math>X^{T}X</math> and <math>X^{T}\mathbf{y.}</math> Check your answers in Matlab.<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 9 Part 1.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=File:Equation_editor001.png&diff=3036File:Equation editor001.png2013-09-10T14:39:32Z<p>LG: Maple TA screenshot of basic Equation Editor</p>
<hr />
<div>Maple TA screenshot of basic Equation Editor</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=File:Equation_editor_subscripts.png&diff=3035File:Equation editor subscripts.png2013-09-10T14:39:09Z<p>LG: Maple TA screenshot of Subscripts and superscripts part of Equation Editor</p>
<hr />
<div>Maple TA screenshot of Subscripts and superscripts part of Equation Editor</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=File:Equation_editor_arrays.png&diff=3034File:Equation editor arrays.png2013-09-10T14:38:22Z<p>LG: Maple TA screenshot of array part of Equation editor</p>
<hr />
<div>Maple TA screenshot of array part of Equation editor</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Exercise_Sheet_2&diff=3031Exercise Sheet 22013-09-10T14:31:08Z<p>LG: </p>
<hr />
<div>= Introduction =<br />
<br />
Blackboard is the University’s virtual learning environment. All of the material for this module, including formative assessments, and summative assessments, will be available in Blackboard.<br />
<br />
You will normally enter the Blackboard site for this course through [https://my.manchester.ac.uk/ MyManchester] at https://my.manchester.ac.uk/ .<br />
<br />
All of the Blackboard material for the course is organised into Lecture topics, so that lecture notes, lecture slides, and exercise sheets can be found in the folder for each Lecture topic.<br />
<br />
The questions on each exercise sheet may be answered in the traditional way, on paper, and handed in to be marked. Alternatively you can answer the majority (but not all) of the exercise sheet questions online, using Maple TA. All of the questions on this exercise sheet can be answered in Maple TA. The name of the matching Maple TA assignment is indicated in each question in this exercise sheet.<br />
<br />
= Using Maple TA =<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. In this module, most of the questions on the paper Exercise Sheets are also available as Maple TA assignments. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
http://place36.placementtester.com/manchester<br />and this link is also given in each Lecture folder, for convenience. Login with your registration number (first 7 digits only): the password is also your registration number. Once you have logged in, there is generally a wait whilst the assignments are loaded. On the page that follows, you can click on MyProfile and then Password Update to change your password.<br />
<br />
You should select the course<br />
<br />
ECON61001 Econometric Methods 2013-14<br />by clicking on the entry for this course. This will bring up a page of assignments. You can click on the assignment you want to do - the notation follows that in the exercise sheets. The assignments are organised by question group, as in the Exercise Sheets, or by individual question - a component of the question group. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between “Print assignment for off-line work” or Work assignment on-line right now. If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the Work ... online option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (<math> +, -, *, /, ^ </math>) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by <math>1/x-1</math> - is it <math>(1/x)-1</math> or <math>1/(x-1)</math>? Information about the entry of vectors and matrices in your answers is given in the next section, although the instructions are often repeated in questions.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and View Details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
Most of the Exercise Sheets have additional randomised questions associated with them in Maple TA. These are questions where Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These randomised questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be to hard, simply click on the Refresh button at the top of the page to get another question.<br />
<br />
= Using the equation editor in Maple TA =<br />
<br />
Usually it is quicker to enter your answers directly in Maple TA rather than using the Equation Editor. Using the Equation Editor is straightforward: see Figure [fig:The-Equation-Editor]. To select a symbol, click on the required panel and select the symbol required. Figure [fig:Subscripts-and-superscripts.] shows the subscript and superscript panel, and Figure [fig:Arrays.] shows the array selection panel. Usage is fairly self-evident - keep trying until you find the required symbol.<br />
<br />
[[Image:1C__maple_local_screenshots_equation_editor001.png|image]]<br />
<br />
[th]<br />
<br />
[[Image:2C__maple_local_screenshots_equation_editor_subscripts.png|image]]<br />
<br />
[th]<br />
<br />
[[Image:3C__maple_local_screenshots_equation_editor_arrays.png|image]]<br />
<br />
= Entering vectors and matrices into Maple TA =<br />
<br />
Many questions on this exercise sheet require vectors, matrices or indexed expression as their answers. Some questions provide an array of the right size for the answer, so that none of the methods below are required.<br />
<br />
There are a variety of ways of entering these expressions into Maple TA - choose the way you feel most comfortable in using.<br />
<br />
# Vector entry.<br />
## You can use Maple “text entry” for column vectors, e.g. for the <math>3\times1</math> vector,: <math>\left(\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right),</math> enter (without the quotes) Vector(<math>[1,2,3]</math>).<br />
## A quicker version is <math><1,2,3></math>.<br />
## For a row vector, use <math><1|2|3></math>, as above, or Vector[row](<math>[1,2,3]</math>).<br />
## Or, you can right click in the “Equation Editor”, select the array object, choose the correct matrix size and enter the elements of the vector.<br />
# Matrix entry.<br />
## You can use Maple “text entry” for matrices, e.g. for a <math>2\times3</math> matrix, enter (without the quotes) Matrix(<math>[[1,2,3],[345]]</math>) for the matrix: <math>\left(\begin{array}{ccc}<br />
1 & 2 & 3\\<br />
4 & 5 & 6<br />
\end{array}\right).</math> For more columns, include more elements in each [.] block. For more rows, add more [.] blocks, each separated by a comma. This is “row - by - row” construction.<br />
## A quicker version is <math><1,2,3;4,5,6></math>.<br />
## Column by column construction is also possible using <math><1,4|2,5|3,6></math> or <math><<1,4>|<2,5>|<3,6>></math>.<br />
## A matrix with one row, say <math>\left[\begin{array}{ccc}<br />
1 & 2 & 3\end{array}\right]</math> can be entered as Matrix(<math>[1,2,3]</math>), or as <math><1|2|3></math>.<br />
## A matrix with one column, say <math>\left[\begin{array}{c}<br />
1\\<br />
4<br />
\end{array}\right],</math> can be entered as Matrix(<math>[[1],[4]]</math>) or as <math><1,4></math>.<br />
## Or, you can right click in the “Equation Editor”, select the array object, choose the correct matrix size and enter the elements of the product.<br />
# Entering indexed elements in vectors and matrices - the methods below can be used with any of the vector or matrix entry schemes.<br />
## To insert <math>x_{1}</math> into an answer, you can use <math>x[1]</math>, or you can use the Equation Editor. NB: select the right palette in the equation editor '''before''' entering any symbols!<br />
## To insert <math>x_{11},</math> use x[1,1], or use the Equation Editor, subject to the previous warning.<br />
<br />
On the whole, the entry methods using <math><...></math> are the quickest to use.<br />
<br />
Arithmetic operators are best shown explicitly in entering answers. These operators are the standard ones, <math>+,-,*,/</math> and .<br />
<br />
= Question 1: Vectors =<br />
<br />
# (XS1 Q1 Part 1) Given the vectors: <math>\mathbf{0}=\left[\begin{array}{r}<br />
0\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right],</math> find: <math>\mathbf{0}-\mathbf{x,x}+\mathbf{y,\mathbf{x-y},x-}2\mathbf{z,}3\mathbf{z}-2\mathbf{y,x}-2\mathbf{y}+\mathbf{z.}</math><br />
# (XS1 Q1 Part 2) Can you find a linear combination <math>\alpha\mathbf{x}+\beta\mathbf{y}</math> of <math>\mathbf{x=}\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right]</math> and <math>\mathbf{y=}\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]</math> to generate (in turn) the vectors: <math>\mathbf{w}_{1}=\left[\begin{array}{c}<br />
4\\<br />
-2<br />
\end{array}\right],\mathbf{w}_{2}=\left[\begin{array}{r}<br />
-2\\<br />
4<br />
\end{array}\right],\mathbf{w}_{3}=\left[\begin{array}{r}<br />
-2\\<br />
2<br />
\end{array}\right]?</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 1 Part 1 and Part 2.<br />
<br />
= Question 2: 3 - d vectors =<br />
<br />
# (XS1 Q2 Part 2) For the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
0<br />
\end{array}\right]</math> find <math>-2\mathbf{x,\ }3\mathbf{x}+2\mathbf{y,\ }4\mathbf{x}-2\mathbf{y.}</math><br />
# (XS1 Q2 Part 2) For the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right]</math> find <math>10\mathbf{x}-\alpha\mathbf{y,}\beta\mathbf{y.}</math><br />
# (XS1 Q2 Part 3) Use the vector <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right]</math> and the vector <math>\mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right]</math> to find <math>2\mathbf{x-z,}</math> <math>\mathbf{x}+3\mathbf{z.}</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 2 Part 1 and Part 2.<br />
<br />
= Question 3: Matrices =<br />
<br />
# (XS1 Q3 Part 1) Arrange the vectors: <math>\mathbf{0}=\left[\begin{array}{r}<br />
0\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right].</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> Find a suitable vector <math>\boldsymbol{\alpha}</math> so that the product <math>A\boldsymbol{\alpha}</math> is equal to the following linear combinations in turn: <math>\mathbf{0}-\mathbf{x,\ x}+\mathbf{y,\ \mathbf{x-y},\ 2z}-\mathbf{x,}</math>''' '''<math>\mathbf{y}-2\mathbf{z,\ x}+\mathbf{y}-\mathbf{z.}</math><br />
# (XS1 Q3 Part 2) Arrange the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
0<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> Find a suitable vector <math>\boldsymbol{\alpha}</math> so that the product <math>A\boldsymbol{\alpha}</math> is equal to the following linear combinations in turn: <math>-2\mathbf{x,\ }3\mathbf{x}+2\mathbf{y,\ }4\mathbf{x}-2\mathbf{y.}</math><br />
# (XS1 Q3 Part 3) Arrange the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math><br />
# (XS1 Q3 Part 4) Arrange the vectors <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> If <math>\boldsymbol{\alpha}=\left[\begin{array}{c}<br />
2\\<br />
1\\<br />
3<br />
\end{array}\right]</math> find <math>A\boldsymbol{\alpha.}</math><br />
# (XS1 Q3 Part 5) Arrange the vectors : <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],\ \ \ \mathbf{w}=\left[\begin{array}{c}<br />
-1\\<br />
-1\\<br />
1<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> If <math>\boldsymbol{\alpha}=\left[\begin{array}{c}<br />
0\\<br />
-1/2\\<br />
-1/2<br />
\end{array}\right]</math> find <math>A\boldsymbol{\alpha.}</math><br />
# (XS1 Q3 Part 6) Arrange the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],\ \ \ \mathbf{w}=\left[\begin{array}{c}<br />
-1\\<br />
-1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{v}=\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> If <math>\boldsymbol{\alpha}=\left[\begin{array}{c}<br />
0\\<br />
1/2\\<br />
1/2\\<br />
1<br />
\end{array}\right]</math> find <math>A\boldsymbol{\alpha}.</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 3 Part 1, Part 2, Part 4, Part 5 and Part 6.<br />
<br />
= Question 4: Inner products =<br />
<br />
# (XS1 Q4 Part 1) Find the inner product of the vectors: <math>\mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].</math> What is <math>\mathbf{x}^{T}\mathbf{x}</math> and <math>\mathbf{y^{T}y?}</math> Is it true that <math>\left(\mathbf{x}^{T}\mathbf{y}\right)^{2}\leq</math> <math>\mathbf{x}^{T}\mathbf{x\times y^{T}y?}</math> This is called the Cauchy-Schwartz Inequality; equality holds only when <math>\mathbf{x}=\lambda\mathbf{y,}</math> for some <math>\lambda.</math><br />
# (XS1 Q4 Part 2) Find a vector <math>\mathbf{z}</math> with the property that <math>\mathbf{x}^{T}\mathbf{z}=0</math> for <math>\mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right].</math> Draw a diagram showing <math>\mathbf{x}</math> and <math>\mathbf{z:}</math> how would you describe the relationship between <math>\mathbf{x}</math> and <math>\mathbf{z?}</math><br />
# (XS1 Q4 Part 3) For: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1/3\\<br />
1/3\\<br />
1/3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{c}<br />
6\\<br />
2\\<br />
-1<br />
\end{array}\right]</math> find <math>\mathbf{x}^{T}\mathbf{y,\ x}^{T}\mathbf{z,\ y}^{T}\mathbf{z.}</math> How would you describe the value of these inner products from a statistical perspective?<br />
# (XS1 Q4 Part 4) If <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{10}<br />
\end{array}\right],</math> what is <math>\mathbf{x}^{T}\mathbf{x?}</math><br />
# (XS1 Q4 Part 5) If <math>\mathbf{z}=\left[\begin{array}{c}<br />
3\\<br />
7\\<br />
1\\<br />
9<br />
\end{array}\right]</math> and <math>\mathbf{1}_{4}</math> is a <math>4\times1</math> vector with every element equal to <math>1,</math> what is the quantity <math>c=\dfrac{1}{4}\mathbf{1}_{4}^{T}\mathbf{z?}</math> Find the elements of the vector <math>\mathbf{z}-c\mathbf{1}_{4}.</math> From a statistical perspective, what are the elements of this vector? Find the value of the inner product <math>\mathbf{1}_{4}^{T}\left(\mathbf{z}-c\mathbf{1}_{4}\right).</math> What statistical information does this illustrate?<br />
# (XS1 Q4 Part 6) Using <math>\mathbf{z}=\left[\begin{array}{c}<br />
3\\<br />
7\\<br />
1\\<br />
9<br />
\end{array}\right],</math> <math>\mathbf{1}_{4}</math> and <math>c</math> from part (5), what is the inner product of <math>\left(\mathbf{z}-c\mathbf{1}_{4}\right)</math> with itself? If this distance is divided by <math>3</math> or <math>4,</math> what statistical quantity is the result?<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 4 Part 1, Part 2, Part 3, and Part 4.<br />
<br />
= Question 5: Across and down =<br />
<br />
# (XS1 Q5 Part 1) Suppose that <math>B</math> is an <math>r\times s</math> matrix with typical element <math>\left\Vert b_{ij}\right\Vert ,</math> and <math>\mathbf{z}</math> is an <math>s\times1</math> vector, with typical element <math>z_{j}.</math> What is the second element of the product <math>B\mathbf{z?}</math> What is the <math>k</math>th element?<br />
# (XS1 Q5 Part 2) If <math>A=\left[\begin{array}{rrr}<br />
2 & 1 & 3\\<br />
1 & 0 & 4<br />
\end{array}\right],</math> and <math>\mathbf{b}_{1}=\left[\begin{array}{r}<br />
4\\<br />
1\\<br />
2<br />
\end{array}\right],</math> <math>\mathbf{b}_{2}=\left[\begin{array}{r}<br />
1\\<br />
2\\<br />
-1<br />
\end{array}\right],</math> find <math>A\mathbf{b}_{1},</math> <math>A\mathbf{b}_{2}.</math><br />
# (XS1 Q5 Part 3) If <math>A=\left[\begin{array}{rrr}<br />
2 & 1 & 3\\<br />
1 & 0 & 4<br />
\end{array}\right],</math> and <math>\mathbf{b}_{1}=\left[\begin{array}{r}<br />
4\\<br />
1\\<br />
2<br />
\end{array}\right],</math> <math>\mathbf{b}_{2}=\left[\begin{array}{r}<br />
1\\<br />
2\\<br />
-1<br />
\end{array}\right],</math> make <math>\mathbf{b}_{1}</math> and <math>\mathbf{b}_{2}</math> the columns of a matrix <math>B</math> and find the matrix which is equal to <math>AB.</math><br />
# (XS1 Q5 Part 4) If <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
3 & 4<br />
\end{array}\right]</math> and <math>B=\left[\begin{array}{cc}<br />
1 & 4\\<br />
6 & 7<br />
\end{array}\right],</math> find <math>AB</math> and <math>BA.</math> What property do the matrices <math>A</math> and <math>B</math> have?<br />
# (XS1 Q5 Part 5) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{10}=\left[\begin{array}{rr}<br />
-3 & 2\\<br />
4 & 9\\<br />
1 & -2<br />
\end{array}\right],</math> find <math>A_{4}A_{10}</math> and <math>A_{10}A_{4}.</math> Why are <math>A_{4}A_{10}</math> and <math>A_{10}A_{4}</math> not equal?<br />
# (XS1 Q5 Part 6) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{11}=\left[\begin{array}{rrr}<br />
2 & 4 & 4\\<br />
3 & 1 & -1\\<br />
2 & 0 & 4<br />
\end{array}\right],</math> find <math>A_{4}A_{11}</math> and <math>A_{11}A_{4}.</math><br />
# (XS1 Q5 Part 7) If <math>A_{11}=\left[\begin{array}{rrr}<br />
2 & 4 & 4\\<br />
3 & 1 & -1\\<br />
2 & 0 & 4<br />
\end{array}\right]</math> and <math>A_{12}=\left[\begin{array}{rrr}<br />
3 & 0 & -1\\<br />
-2 & -1 & -1\\<br />
1 & 2 & 3<br />
\end{array}\right]</math> , find <math>A_{11}A_{12}</math> and <math>A_{12}A_{11}.</math><br />
# (XS1 Q5 Part 8) If <math>A_{1}=\left[\begin{array}{rrr}<br />
4 & 0 & -3\\<br />
1 & -1 & 4\\<br />
2 & 2 & -11<br />
\end{array}\right]</math> and <math>A_{8}=\left[\begin{array}{rrr}<br />
4 & 2 & 6\\<br />
1 & 3 & 4\\<br />
5 & 0 & 5<br />
\end{array}\right]</math> , find <math>2A_{8}-3A_{1}.</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 5 Part 1, Part 2, Part 4, and Part 8.<br />
<br />
= Question 6: Transposition =<br />
<br />
<ol><br />
<li><p>(XS1 Q6 Part 1) Find the transpose of the following matrices:</p><br />
<p><math>\begin{aligned}<br />
A_{1} & = & \left[\begin{array}{cc}<br />
-2 & 3\\<br />
3 & 5<br />
\end{array}\right];\ \ \ A_{2}=\left[\begin{array}{rrr}<br />
1 & 0 & 9\\<br />
6 & -2 & 15<br />
\end{array}\right];\ \ \ A_{3}=\left[\begin{array}{rr}<br />
1 & 1\\<br />
2 & -1\\<br />
0 & 0<br />
\end{array}\right];\\<br />
A_{4} & = & \left[\begin{array}{rrr}<br />
1 & 1 & -1\\<br />
1 & -1 & -1\\<br />
1 & -1 & 1<br />
\end{array}\right];\ \ \ A_{5}=\left[\begin{array}{rrr}<br />
1 & 1 & -1\\<br />
1 & -1 & -1\\<br />
-1 & -1 & 1<br />
\end{array}\right];\ \ \ A_{6}=\left[\begin{array}{rr}<br />
4 & 2\\<br />
3 & 5\\<br />
-2 & 3<br />
\end{array}\right].\end{aligned}</math></p></li><br />
<li><p>(XS1 Q6 Part 1) What is the dimension of <math>A^{T}</math> for each matrix in part (1)?</p></li><br />
<li><p>Check that <math>\left(A^{T}\right)^{T}=A</math> for each of the matrices in part (1).</p></li><br />
<li><p>(XS1 Q6 Part 1) What properties do the matrices <math>A_{1}</math> and <math>A_{5}</math> have that are not shared by the other matrices in part (1)?</p></li><br />
<li><p>(XS1 Q6 Part 5) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{10}=\left[\begin{array}{rr}<br />
-3 & 2\\<br />
4 & 9\\<br />
1 & -2<br />
\end{array}\right],</math> find <math>A_{4}^{T}</math> and <math>A_{10}^{T}.</math> Confirm that <math>\left(A_{4}A_{10}\right)^{T}=A_{10}^{T}A_{4}^{T}.</math></p></li></ol><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 6 Part 1, Part 4, and Part 5.<br />
<br />
= Question 7: Special Matrices =<br />
<br />
# (XS1 Q7 Part 1) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right],</math> confirm that <math>I_{2}A_{4}=A_{4}I_{3}.</math><br />
# (XS1 Q7 Part 2) If <math>D=\left[\begin{array}{cc}<br />
3 & 0\\<br />
0 & 5<br />
\end{array}\right],</math> <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{10}=\left[\begin{array}{rr}<br />
-3 & 2\\<br />
4 & 9\\<br />
1 & -2<br />
\end{array}\right],</math> find <math>DA_{4}</math> and <math>A_{10}D.</math> What pattern can you detect in the results?<br />
# (XS1 Q7 Part 3) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right],</math> find <math>A_{4}A_{4}^{T}</math> and <math>A_{4}^{T}A_{4}.</math> Are these two matrices equal? What property do these two matrices possess?<br />
# (XS1 Q7 Part 4) If <math>\mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right]</math> and <math>\mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],</math> find <math>\mathbf{xx}^{T}</math>, <math>\mathbf{yy}^{T}</math>. Are these symmetric? Do they equal <math>\mathbf{x}^{T}\mathbf{x,}</math> <math>\mathbf{y}^{T}\mathbf{y?}</math> Find <math>\mathbf{xy}^{T}:</math> is <math>\mathbf{xy}^{T}</math> equal to <math>\mathbf{yx}^{T}</math> or <math>\mathbf{y}^{T}\mathbf{x?}</math><br />
# (XS1 Q7 Part 5) If <math>L=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
2 & 1 & 0\\<br />
3 & 2 & 1<br />
\end{array}\right],</math> show that <math>L^{T}</math> is an upper triangular matrix.<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 7 Part 3 and 4.<br />
<br />
= Question 8: Partitioned matrices =<br />
<br />
# (XS1 Q8 Part 1) Write <math>A_{2}=\left[\begin{array}{rrrr}<br />
1 & 2 & 1 & -3\\<br />
6 & 0 & 10 & 9\\<br />
2 & 0 & 3 & 4<br />
\end{array}\right]</math> in the form <math>A_{2}=\left[\begin{array}{rr}<br />
B_{1} & B_{2}\end{array}\right],</math> where both <math>B_{1}</math> and <math>B_{2}</math> have two columns. What are <math>B_{1}</math> and <math>B_{2}?</math> Write <math>A_{9}=\left[\begin{array}{rr}<br />
2 & 0\\<br />
-1 & 3\\<br />
4 & -2\\<br />
7 & 7<br />
\end{array}\right]</math> as <math>A_{9}=\left[\begin{array}{r}<br />
C_{1}\\<br />
C_{2}<br />
\end{array}\right],</math> where both <math>C_{1}</math> and <math>C_{2}</math> have 2 rows. Express the product <math>A_{2}A_{9}</math> in terms of <math>B_{1}</math>, <math>B_{2},</math> <math>C_{1}</math> and <math>C_{2}.</math> Do the same for the product <math>A_{9}A_{2},</math> carefully stating the dimensions of any submatrices.<br />
# (XS1 Q8 Part 1) If <math>A_{2}=\left[\begin{array}{rrrr}<br />
1 & 2 & 1 & -3\\<br />
6 & 0 & 10 & 9\\<br />
2 & 0 & 3 & 4<br />
\end{array}\right]=\left[\begin{array}{rr}<br />
B_{1} & B_{2}\end{array}\right],</math> where both <math>B_{1}</math> and <math>B_{2}</math> have two columns, and <math>\mathbf{z}=\left[\begin{array}{c}<br />
5\\<br />
1\\<br />
-3\\<br />
2<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\mathbf{z}_{1}\\<br />
\mathbf{z}_{2}<br />
\end{array}\right]</math> where <math>\mathbf{z}_{i}</math> is <math>2\times1,</math> given an expression for <math>A_{2}\mathbf{z}</math> in terms of <math>B_{1},B_{2},\mathbf{z}_{1},\mathbf{z}_{2}.</math><br />
# (XS1 Q8 Part 3) If <math>A=\left[\begin{array}{cc}<br />
A_{11} & A_{12}\\<br />
A_{21} & A_{22}<br />
\end{array}\right]</math> and <math>\mathbf{z}=\left[\begin{array}{r}<br />
\mathbf{z}_{1}\\<br />
\mathbf{z}_{2}<br />
\end{array}\right],</math> where <math>A_{11}</math> is <math>2\times2,</math> <math>A_{12}</math> is <math>2\times1,</math> <math>A_{21}</math> is <math>1\times2,</math> <math>A_{22}</math> <math>1\times1</math> and <math>\mathbf{z}_{1}</math> is <math>1\times1,</math> <math>\mathbf{z}_{2}</math> <math>2\times1,</math> is the matrix <math>A</math> well defined? Does the product <math>A\mathbf{z}=\left[\begin{array}{cc}<br />
A_{11} & A_{12}\\<br />
A_{21} & A_{22}<br />
\end{array}\right]\left[\begin{array}{r}<br />
\mathbf{z}_{1}\\<br />
\mathbf{z}_{2}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
A_{11}\mathbf{z}_{1}+A_{12}\mathbf{z}_{2}\\<br />
A_{21}\mathbf{z}_{1}+A_{22}\mathbf{z}_{2}<br />
\end{array}\right]</math> exist in this form?<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 8 Part 1.<br />
<br />
= Question 9: Data matrices =<br />
<br />
A small data set on variables <math>y</math> and <math>x</math> is: <math>D=\left[\begin{array}{rr}<br />
14 & 2\\<br />
17 & 4\\<br />
8 & 3\\<br />
16 & 5\\<br />
3 & 2<br />
\end{array}\right].</math><br />
<br />
# (XS1 Q9 Part 1) Define a vector of observations on <math>y</math> and a matrix <math>X</math> so that the two variable regression model <math>y_{i}=\alpha+\beta x_{i}+u_{i}</math> can be represented, using the data in <math>D,</math> as <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
# (XS1 Q9 Part 1) Compute for your choice of <math>X,</math> <math>X^{T}X</math> and <math>X^{T}\mathbf{y.}</math> Check your answers in Matlab.<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 9 Part 1.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Exercise_Sheet_2&diff=3030Exercise Sheet 22013-09-10T14:30:36Z<p>LG: Created page with "= Introduction = Blackboard is the University’s virtual learning environment. All of the material for this module, including formative assessments, and summative assessment..."</p>
<hr />
<div>= Introduction =<br />
<br />
Blackboard is the University’s virtual learning environment. All of the material for this module, including formative assessments, and summative assessments, will be available in Blackboard.<br />
<br />
You will normally enter the Blackboard site for this course through [https://my.manchester.ac.uk/ MyManchester] at https://my.manchester.ac.uk/ .<br />
<br />
All of the Blackboard material for the course is organised into Lecture topics, so that lecture notes, lecture slides, and exercise sheets can be found in the folder for each Lecture topic.<br />
<br />
The questions on each exercise sheet may be answered in the traditional way, on paper, and handed in to be marked. Alternatively you can answer the majority (but not all) of the exercise sheet questions online, using Maple TA. All of the questions on this exercise sheet can be answered in Maple TA. The name of the matching Maple TA assignment is indicated in each question in this exercise sheet.<br />
<br />
= Using Maple TA =<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. In this module, most of the questions on the paper Exercise Sheets are also available as Maple TA assignments. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
http://place36.placementtester.com/manchester<br />and this link is also given in each Lecture folder, for convenience. Login with your registration number (first 7 digits only): the password is also your registration number. Once you have logged in, there is generally a wait whilst the assignments are loaded. On the page that follows, you can click on MyProfile and then Password Update to change your password.<br />
<br />
You should select the course<br />
<br />
ECON61001 Econometric Methods 2013-14<br />by clicking on the entry for this course. This will bring up a page of assignments. You can click on the assignment you want to do - the notation follows that in the exercise sheets. The assignments are organised by question group, as in the Exercise Sheets, or by individual question - a component of the question group. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between “Print assignment for off-line work” or Work assignment on-line right now. If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the Work ... online option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (<math>+,-,*,/,^</math>) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by <math>1/x-1</math> - is it <math>(1/x)-1</math> or <math>1/(x-1)</math>? Information about the entry of vectors and matrices in your answers is given in the next section, although the instructions are often repeated in questions.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and View Details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
Most of the Exercise Sheets have additional randomised questions associated with them in Maple TA. These are questions where Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These randomised questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be to hard, simply click on the Refresh button at the top of the page to get another question.<br />
<br />
= Using the equation editor in Maple TA =<br />
<br />
Usually it is quicker to enter your answers directly in Maple TA rather than using the Equation Editor. Using the Equation Editor is straightforward: see Figure [fig:The-Equation-Editor]. To select a symbol, click on the required panel and select the symbol required. Figure [fig:Subscripts-and-superscripts.] shows the subscript and superscript panel, and Figure [fig:Arrays.] shows the array selection panel. Usage is fairly self-evident - keep trying until you find the required symbol.<br />
<br />
[[Image:1C__maple_local_screenshots_equation_editor001.png|image]]<br />
<br />
[th]<br />
<br />
[[Image:2C__maple_local_screenshots_equation_editor_subscripts.png|image]]<br />
<br />
[th]<br />
<br />
[[Image:3C__maple_local_screenshots_equation_editor_arrays.png|image]]<br />
<br />
= Entering vectors and matrices into Maple TA =<br />
<br />
Many questions on this exercise sheet require vectors, matrices or indexed expression as their answers. Some questions provide an array of the right size for the answer, so that none of the methods below are required.<br />
<br />
There are a variety of ways of entering these expressions into Maple TA - choose the way you feel most comfortable in using.<br />
<br />
# Vector entry.<br />
## You can use Maple “text entry” for column vectors, e.g. for the <math>3\times1</math> vector,: <math>\left(\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right),</math> enter (without the quotes) Vector(<math>[1,2,3]</math>).<br />
## A quicker version is <math><1,2,3></math>.<br />
## For a row vector, use <math><1|2|3></math>, as above, or Vector[row](<math>[1,2,3]</math>).<br />
## Or, you can right click in the “Equation Editor”, select the array object, choose the correct matrix size and enter the elements of the vector.<br />
# Matrix entry.<br />
## You can use Maple “text entry” for matrices, e.g. for a <math>2\times3</math> matrix, enter (without the quotes) Matrix(<math>[[1,2,3],[345]]</math>) for the matrix: <math>\left(\begin{array}{ccc}<br />
1 & 2 & 3\\<br />
4 & 5 & 6<br />
\end{array}\right).</math> For more columns, include more elements in each [.] block. For more rows, add more [.] blocks, each separated by a comma. This is “row - by - row” construction.<br />
## A quicker version is <math><1,2,3;4,5,6></math>.<br />
## Column by column construction is also possible using <math><1,4|2,5|3,6></math> or <math><<1,4>|<2,5>|<3,6>></math>.<br />
## A matrix with one row, say <math>\left[\begin{array}{ccc}<br />
1 & 2 & 3\end{array}\right]</math> can be entered as Matrix(<math>[1,2,3]</math>), or as <math><1|2|3></math>.<br />
## A matrix with one column, say <math>\left[\begin{array}{c}<br />
1\\<br />
4<br />
\end{array}\right],</math> can be entered as Matrix(<math>[[1],[4]]</math>) or as <math><1,4></math>.<br />
## Or, you can right click in the “Equation Editor”, select the array object, choose the correct matrix size and enter the elements of the product.<br />
# Entering indexed elements in vectors and matrices - the methods below can be used with any of the vector or matrix entry schemes.<br />
## To insert <math>x_{1}</math> into an answer, you can use <math>x[1]</math>, or you can use the Equation Editor. NB: select the right palette in the equation editor '''before''' entering any symbols!<br />
## To insert <math>x_{11},</math> use x[1,1], or use the Equation Editor, subject to the previous warning.<br />
<br />
On the whole, the entry methods using <math><...></math> are the quickest to use.<br />
<br />
Arithmetic operators are best shown explicitly in entering answers. These operators are the standard ones, <math>+,-,*,/</math> and .<br />
<br />
= Question 1: Vectors =<br />
<br />
# (XS1 Q1 Part 1) Given the vectors: <math>\mathbf{0}=\left[\begin{array}{r}<br />
0\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right],</math> find: <math>\mathbf{0}-\mathbf{x,x}+\mathbf{y,\mathbf{x-y},x-}2\mathbf{z,}3\mathbf{z}-2\mathbf{y,x}-2\mathbf{y}+\mathbf{z.}</math><br />
# (XS1 Q1 Part 2) Can you find a linear combination <math>\alpha\mathbf{x}+\beta\mathbf{y}</math> of <math>\mathbf{x=}\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right]</math> and <math>\mathbf{y=}\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]</math> to generate (in turn) the vectors: <math>\mathbf{w}_{1}=\left[\begin{array}{c}<br />
4\\<br />
-2<br />
\end{array}\right],\mathbf{w}_{2}=\left[\begin{array}{r}<br />
-2\\<br />
4<br />
\end{array}\right],\mathbf{w}_{3}=\left[\begin{array}{r}<br />
-2\\<br />
2<br />
\end{array}\right]?</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 1 Part 1 and Part 2.<br />
<br />
= Question 2: 3 - d vectors =<br />
<br />
# (XS1 Q2 Part 2) For the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
0<br />
\end{array}\right]</math> find <math>-2\mathbf{x,\ }3\mathbf{x}+2\mathbf{y,\ }4\mathbf{x}-2\mathbf{y.}</math><br />
# (XS1 Q2 Part 2) For the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right]</math> find <math>10\mathbf{x}-\alpha\mathbf{y,}\beta\mathbf{y.}</math><br />
# (XS1 Q2 Part 3) Use the vector <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right]</math> and the vector <math>\mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right]</math> to find <math>2\mathbf{x-z,}</math> <math>\mathbf{x}+3\mathbf{z.}</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 2 Part 1 and Part 2.<br />
<br />
= Question 3: Matrices =<br />
<br />
# (XS1 Q3 Part 1) Arrange the vectors: <math>\mathbf{0}=\left[\begin{array}{r}<br />
0\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right].</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> Find a suitable vector <math>\boldsymbol{\alpha}</math> so that the product <math>A\boldsymbol{\alpha}</math> is equal to the following linear combinations in turn: <math>\mathbf{0}-\mathbf{x,\ x}+\mathbf{y,\ \mathbf{x-y},\ 2z}-\mathbf{x,}</math>''' '''<math>\mathbf{y}-2\mathbf{z,\ x}+\mathbf{y}-\mathbf{z.}</math><br />
# (XS1 Q3 Part 2) Arrange the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
0<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> Find a suitable vector <math>\boldsymbol{\alpha}</math> so that the product <math>A\boldsymbol{\alpha}</math> is equal to the following linear combinations in turn: <math>-2\mathbf{x,\ }3\mathbf{x}+2\mathbf{y,\ }4\mathbf{x}-2\mathbf{y.}</math><br />
# (XS1 Q3 Part 3) Arrange the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
0<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{r}<br />
4\\<br />
2<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math><br />
# (XS1 Q3 Part 4) Arrange the vectors <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> If <math>\boldsymbol{\alpha}=\left[\begin{array}{c}<br />
2\\<br />
1\\<br />
3<br />
\end{array}\right]</math> find <math>A\boldsymbol{\alpha.}</math><br />
# (XS1 Q3 Part 5) Arrange the vectors : <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],\ \ \ \mathbf{w}=\left[\begin{array}{c}<br />
-1\\<br />
-1\\<br />
1<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> If <math>\boldsymbol{\alpha}=\left[\begin{array}{c}<br />
0\\<br />
-1/2\\<br />
-1/2<br />
\end{array}\right]</math> find <math>A\boldsymbol{\alpha.}</math><br />
# (XS1 Q3 Part 6) Arrange the vectors: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],\ \ \ \mathbf{w}=\left[\begin{array}{c}<br />
-1\\<br />
-1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{v}=\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right]</math> as the columns of a matrix <math>A.</math> What is the dimension of <math>A?</math> If <math>\boldsymbol{\alpha}=\left[\begin{array}{c}<br />
0\\<br />
1/2\\<br />
1/2\\<br />
1<br />
\end{array}\right]</math> find <math>A\boldsymbol{\alpha}.</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 3 Part 1, Part 2, Part 4, Part 5 and Part 6.<br />
<br />
= Question 4: Inner products =<br />
<br />
# (XS1 Q4 Part 1) Find the inner product of the vectors: <math>\mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].</math> What is <math>\mathbf{x}^{T}\mathbf{x}</math> and <math>\mathbf{y^{T}y?}</math> Is it true that <math>\left(\mathbf{x}^{T}\mathbf{y}\right)^{2}\leq</math> <math>\mathbf{x}^{T}\mathbf{x\times y^{T}y?}</math> This is called the Cauchy-Schwartz Inequality; equality holds only when <math>\mathbf{x}=\lambda\mathbf{y,}</math> for some <math>\lambda.</math><br />
# (XS1 Q4 Part 2) Find a vector <math>\mathbf{z}</math> with the property that <math>\mathbf{x}^{T}\mathbf{z}=0</math> for <math>\mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right].</math> Draw a diagram showing <math>\mathbf{x}</math> and <math>\mathbf{z:}</math> how would you describe the relationship between <math>\mathbf{x}</math> and <math>\mathbf{z?}</math><br />
# (XS1 Q4 Part 3) For: <math>\mathbf{x}=\left[\begin{array}{c}<br />
1/3\\<br />
1/3\\<br />
1/3<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{z}=\left[\begin{array}{c}<br />
6\\<br />
2\\<br />
-1<br />
\end{array}\right]</math> find <math>\mathbf{x}^{T}\mathbf{y,\ x}^{T}\mathbf{z,\ y}^{T}\mathbf{z.}</math> How would you describe the value of these inner products from a statistical perspective?<br />
# (XS1 Q4 Part 4) If <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{10}<br />
\end{array}\right],</math> what is <math>\mathbf{x}^{T}\mathbf{x?}</math><br />
# (XS1 Q4 Part 5) If <math>\mathbf{z}=\left[\begin{array}{c}<br />
3\\<br />
7\\<br />
1\\<br />
9<br />
\end{array}\right]</math> and <math>\mathbf{1}_{4}</math> is a <math>4\times1</math> vector with every element equal to <math>1,</math> what is the quantity <math>c=\dfrac{1}{4}\mathbf{1}_{4}^{T}\mathbf{z?}</math> Find the elements of the vector <math>\mathbf{z}-c\mathbf{1}_{4}.</math> From a statistical perspective, what are the elements of this vector? Find the value of the inner product <math>\mathbf{1}_{4}^{T}\left(\mathbf{z}-c\mathbf{1}_{4}\right).</math> What statistical information does this illustrate?<br />
# (XS1 Q4 Part 6) Using <math>\mathbf{z}=\left[\begin{array}{c}<br />
3\\<br />
7\\<br />
1\\<br />
9<br />
\end{array}\right],</math> <math>\mathbf{1}_{4}</math> and <math>c</math> from part (5), what is the inner product of <math>\left(\mathbf{z}-c\mathbf{1}_{4}\right)</math> with itself? If this distance is divided by <math>3</math> or <math>4,</math> what statistical quantity is the result?<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 4 Part 1, Part 2, Part 3, and Part 4.<br />
<br />
= Question 5: Across and down =<br />
<br />
# (XS1 Q5 Part 1) Suppose that <math>B</math> is an <math>r\times s</math> matrix with typical element <math>\left\Vert b_{ij}\right\Vert ,</math> and <math>\mathbf{z}</math> is an <math>s\times1</math> vector, with typical element <math>z_{j}.</math> What is the second element of the product <math>B\mathbf{z?}</math> What is the <math>k</math>th element?<br />
# (XS1 Q5 Part 2) If <math>A=\left[\begin{array}{rrr}<br />
2 & 1 & 3\\<br />
1 & 0 & 4<br />
\end{array}\right],</math> and <math>\mathbf{b}_{1}=\left[\begin{array}{r}<br />
4\\<br />
1\\<br />
2<br />
\end{array}\right],</math> <math>\mathbf{b}_{2}=\left[\begin{array}{r}<br />
1\\<br />
2\\<br />
-1<br />
\end{array}\right],</math> find <math>A\mathbf{b}_{1},</math> <math>A\mathbf{b}_{2}.</math><br />
# (XS1 Q5 Part 3) If <math>A=\left[\begin{array}{rrr}<br />
2 & 1 & 3\\<br />
1 & 0 & 4<br />
\end{array}\right],</math> and <math>\mathbf{b}_{1}=\left[\begin{array}{r}<br />
4\\<br />
1\\<br />
2<br />
\end{array}\right],</math> <math>\mathbf{b}_{2}=\left[\begin{array}{r}<br />
1\\<br />
2\\<br />
-1<br />
\end{array}\right],</math> make <math>\mathbf{b}_{1}</math> and <math>\mathbf{b}_{2}</math> the columns of a matrix <math>B</math> and find the matrix which is equal to <math>AB.</math><br />
# (XS1 Q5 Part 4) If <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
3 & 4<br />
\end{array}\right]</math> and <math>B=\left[\begin{array}{cc}<br />
1 & 4\\<br />
6 & 7<br />
\end{array}\right],</math> find <math>AB</math> and <math>BA.</math> What property do the matrices <math>A</math> and <math>B</math> have?<br />
# (XS1 Q5 Part 5) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{10}=\left[\begin{array}{rr}<br />
-3 & 2\\<br />
4 & 9\\<br />
1 & -2<br />
\end{array}\right],</math> find <math>A_{4}A_{10}</math> and <math>A_{10}A_{4}.</math> Why are <math>A_{4}A_{10}</math> and <math>A_{10}A_{4}</math> not equal?<br />
# (XS1 Q5 Part 6) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{11}=\left[\begin{array}{rrr}<br />
2 & 4 & 4\\<br />
3 & 1 & -1\\<br />
2 & 0 & 4<br />
\end{array}\right],</math> find <math>A_{4}A_{11}</math> and <math>A_{11}A_{4}.</math><br />
# (XS1 Q5 Part 7) If <math>A_{11}=\left[\begin{array}{rrr}<br />
2 & 4 & 4\\<br />
3 & 1 & -1\\<br />
2 & 0 & 4<br />
\end{array}\right]</math> and <math>A_{12}=\left[\begin{array}{rrr}<br />
3 & 0 & -1\\<br />
-2 & -1 & -1\\<br />
1 & 2 & 3<br />
\end{array}\right]</math> , find <math>A_{11}A_{12}</math> and <math>A_{12}A_{11}.</math><br />
# (XS1 Q5 Part 8) If <math>A_{1}=\left[\begin{array}{rrr}<br />
4 & 0 & -3\\<br />
1 & -1 & 4\\<br />
2 & 2 & -11<br />
\end{array}\right]</math> and <math>A_{8}=\left[\begin{array}{rrr}<br />
4 & 2 & 6\\<br />
1 & 3 & 4\\<br />
5 & 0 & 5<br />
\end{array}\right]</math> , find <math>2A_{8}-3A_{1}.</math><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 5 Part 1, Part 2, Part 4, and Part 8.<br />
<br />
= Question 6: Transposition =<br />
<br />
<ol><br />
<li><p>(XS1 Q6 Part 1) Find the transpose of the following matrices:</p><br />
<p><math>\begin{aligned}<br />
A_{1} & = & \left[\begin{array}{cc}<br />
-2 & 3\\<br />
3 & 5<br />
\end{array}\right];\ \ \ A_{2}=\left[\begin{array}{rrr}<br />
1 & 0 & 9\\<br />
6 & -2 & 15<br />
\end{array}\right];\ \ \ A_{3}=\left[\begin{array}{rr}<br />
1 & 1\\<br />
2 & -1\\<br />
0 & 0<br />
\end{array}\right];\\<br />
A_{4} & = & \left[\begin{array}{rrr}<br />
1 & 1 & -1\\<br />
1 & -1 & -1\\<br />
1 & -1 & 1<br />
\end{array}\right];\ \ \ A_{5}=\left[\begin{array}{rrr}<br />
1 & 1 & -1\\<br />
1 & -1 & -1\\<br />
-1 & -1 & 1<br />
\end{array}\right];\ \ \ A_{6}=\left[\begin{array}{rr}<br />
4 & 2\\<br />
3 & 5\\<br />
-2 & 3<br />
\end{array}\right].\end{aligned}</math></p></li><br />
<li><p>(XS1 Q6 Part 1) What is the dimension of <math>A^{T}</math> for each matrix in part (1)?</p></li><br />
<li><p>Check that <math>\left(A^{T}\right)^{T}=A</math> for each of the matrices in part (1).</p></li><br />
<li><p>(XS1 Q6 Part 1) What properties do the matrices <math>A_{1}</math> and <math>A_{5}</math> have that are not shared by the other matrices in part (1)?</p></li><br />
<li><p>(XS1 Q6 Part 5) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{10}=\left[\begin{array}{rr}<br />
-3 & 2\\<br />
4 & 9\\<br />
1 & -2<br />
\end{array}\right],</math> find <math>A_{4}^{T}</math> and <math>A_{10}^{T}.</math> Confirm that <math>\left(A_{4}A_{10}\right)^{T}=A_{10}^{T}A_{4}^{T}.</math></p></li></ol><br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 6 Part 1, Part 4, and Part 5.<br />
<br />
= Question 7: Special Matrices =<br />
<br />
# (XS1 Q7 Part 1) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right],</math> confirm that <math>I_{2}A_{4}=A_{4}I_{3}.</math><br />
# (XS1 Q7 Part 2) If <math>D=\left[\begin{array}{cc}<br />
3 & 0\\<br />
0 & 5<br />
\end{array}\right],</math> <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right]</math> and <math>A_{10}=\left[\begin{array}{rr}<br />
-3 & 2\\<br />
4 & 9\\<br />
1 & -2<br />
\end{array}\right],</math> find <math>DA_{4}</math> and <math>A_{10}D.</math> What pattern can you detect in the results?<br />
# (XS1 Q7 Part 3) If <math>A_{4}=\left[\begin{array}{rrr}<br />
-3 & 9 & 4\\<br />
0 & 5 & 2<br />
\end{array}\right],</math> find <math>A_{4}A_{4}^{T}</math> and <math>A_{4}^{T}A_{4}.</math> Are these two matrices equal? What property do these two matrices possess?<br />
# (XS1 Q7 Part 4) If <math>\mathbf{x}=\left[\begin{array}{r}<br />
-2\\<br />
3<br />
\end{array}\right]</math> and <math>\mathbf{y}=\left[\begin{array}{c}<br />
1\\<br />
-1\\<br />
-1<br />
\end{array}\right],</math> find <math>\mathbf{xx}^{T}</math>, <math>\mathbf{yy}^{T}</math>. Are these symmetric? Do they equal <math>\mathbf{x}^{T}\mathbf{x,}</math> <math>\mathbf{y}^{T}\mathbf{y?}</math> Find <math>\mathbf{xy}^{T}:</math> is <math>\mathbf{xy}^{T}</math> equal to <math>\mathbf{yx}^{T}</math> or <math>\mathbf{y}^{T}\mathbf{x?}</math><br />
# (XS1 Q7 Part 5) If <math>L=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
2 & 1 & 0\\<br />
3 & 2 & 1<br />
\end{array}\right],</math> show that <math>L^{T}</math> is an upper triangular matrix.<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 7 Part 3 and 4.<br />
<br />
= Question 8: Partitioned matrices =<br />
<br />
# (XS1 Q8 Part 1) Write <math>A_{2}=\left[\begin{array}{rrrr}<br />
1 & 2 & 1 & -3\\<br />
6 & 0 & 10 & 9\\<br />
2 & 0 & 3 & 4<br />
\end{array}\right]</math> in the form <math>A_{2}=\left[\begin{array}{rr}<br />
B_{1} & B_{2}\end{array}\right],</math> where both <math>B_{1}</math> and <math>B_{2}</math> have two columns. What are <math>B_{1}</math> and <math>B_{2}?</math> Write <math>A_{9}=\left[\begin{array}{rr}<br />
2 & 0\\<br />
-1 & 3\\<br />
4 & -2\\<br />
7 & 7<br />
\end{array}\right]</math> as <math>A_{9}=\left[\begin{array}{r}<br />
C_{1}\\<br />
C_{2}<br />
\end{array}\right],</math> where both <math>C_{1}</math> and <math>C_{2}</math> have 2 rows. Express the product <math>A_{2}A_{9}</math> in terms of <math>B_{1}</math>, <math>B_{2},</math> <math>C_{1}</math> and <math>C_{2}.</math> Do the same for the product <math>A_{9}A_{2},</math> carefully stating the dimensions of any submatrices.<br />
# (XS1 Q8 Part 1) If <math>A_{2}=\left[\begin{array}{rrrr}<br />
1 & 2 & 1 & -3\\<br />
6 & 0 & 10 & 9\\<br />
2 & 0 & 3 & 4<br />
\end{array}\right]=\left[\begin{array}{rr}<br />
B_{1} & B_{2}\end{array}\right],</math> where both <math>B_{1}</math> and <math>B_{2}</math> have two columns, and <math>\mathbf{z}=\left[\begin{array}{c}<br />
5\\<br />
1\\<br />
-3\\<br />
2<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\mathbf{z}_{1}\\<br />
\mathbf{z}_{2}<br />
\end{array}\right]</math> where <math>\mathbf{z}_{i}</math> is <math>2\times1,</math> given an expression for <math>A_{2}\mathbf{z}</math> in terms of <math>B_{1},B_{2},\mathbf{z}_{1},\mathbf{z}_{2}.</math><br />
# (XS1 Q8 Part 3) If <math>A=\left[\begin{array}{cc}<br />
A_{11} & A_{12}\\<br />
A_{21} & A_{22}<br />
\end{array}\right]</math> and <math>\mathbf{z}=\left[\begin{array}{r}<br />
\mathbf{z}_{1}\\<br />
\mathbf{z}_{2}<br />
\end{array}\right],</math> where <math>A_{11}</math> is <math>2\times2,</math> <math>A_{12}</math> is <math>2\times1,</math> <math>A_{21}</math> is <math>1\times2,</math> <math>A_{22}</math> <math>1\times1</math> and <math>\mathbf{z}_{1}</math> is <math>1\times1,</math> <math>\mathbf{z}_{2}</math> <math>2\times1,</math> is the matrix <math>A</math> well defined? Does the product <math>A\mathbf{z}=\left[\begin{array}{cc}<br />
A_{11} & A_{12}\\<br />
A_{21} & A_{22}<br />
\end{array}\right]\left[\begin{array}{r}<br />
\mathbf{z}_{1}\\<br />
\mathbf{z}_{2}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
A_{11}\mathbf{z}_{1}+A_{12}\mathbf{z}_{2}\\<br />
A_{21}\mathbf{z}_{1}+A_{22}\mathbf{z}_{2}<br />
\end{array}\right]</math> exist in this form?<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 8 Part 1.<br />
<br />
= Question 9: Data matrices =<br />
<br />
A small data set on variables <math>y</math> and <math>x</math> is: <math>D=\left[\begin{array}{rr}<br />
14 & 2\\<br />
17 & 4\\<br />
8 & 3\\<br />
16 & 5\\<br />
3 & 2<br />
\end{array}\right].</math><br />
<br />
# (XS1 Q9 Part 1) Define a vector of observations on <math>y</math> and a matrix <math>X</math> so that the two variable regression model <math>y_{i}=\alpha+\beta x_{i}+u_{i}</math> can be represented, using the data in <math>D,</math> as <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
# (XS1 Q9 Part 1) Compute for your choice of <math>X,</math> <math>X^{T}X</math> and <math>X^{T}\mathbf{y.}</math> Check your answers in Matlab.<br />
<br />
Additional “randomised” questions in Maple TA are in ExSheet2 Randomised Questions. The individual questions are XS2 Random Question 9 Part 1.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3029Maths2013-09-10T14:26:51Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
The lecture notes are also available here,<br />
<br />
[[Lecture Notes 2|Lnotes]]<br />
<br />
and the corresponding exercise sheet is<br />
<br />
[[Exercise Sheet 2|XS2]]<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material, and test their understanding using Maple TA.<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments: there is usually a delay whilst they are loaded. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+, -, *, /, ^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be too hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=LNotes&diff=3028LNotes2013-09-10T14:15:01Z<p>LG: </p>
<hr />
<div>= Matrices =<br />
<br />
In the PreSession Maths course, a matrix was defined as follows:<br />
<br />
<blockquote>A matrix is a rectangular array of numbers enclosed in parentheses, conventionally denoted by a capital letter. The number of rows (say <math>m</math>) and<br />
<br />
the number of columns (say <math>n</math>) determine the order of the matrix (<math>m</math> <math>\times</math> <math>n</math>).<br />
</blockquote><br />
Two examples were given:<br />
<br />
<math>\begin{aligned}<br />
P & =\left[\begin{array}{rrr}<br />
2 & 3 & 4\\<br />
3 & 1 & 5<br />
\end{array}\right],\ \ \ Q=\left[\begin{array}{rr}<br />
2 & 3\\<br />
4 & 3\\<br />
1 & 5<br />
\end{array}\right],\end{aligned}</math><br />
<br />
matrices of dimensions <math>2\times3</math> and <math>3\times2</math> respectively.<br />
<br />
Why study matrices for econometrics? Basically because a data set of several variables, e.g. on the weights and heights of 12 students, can be thought of as a matrix:<br />
<br />
<math>\begin{aligned}<br />
D & =\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]\end{aligned}</math><br />
<br />
The properties of matrices can then be used to facilitate answering all the usual questions of econometrics - list not given here!<br />
<br />
Calculations with matrices with explicit numerical elements, as in the examples above is called matrix ''arithmetic''. Matrix ''algebra'' is the algebra of matrices where the elements are not made explicit: this is what is really required for econometrics, as we shall see.<br />
<br />
As an example of this, a <math>2\times3</math> matrix might be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{ccc}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\end{aligned}</math><br />
<br />
and would equal <math>P</math> above if the collection of <math>a_{ij}</math> were given appropriate numerical values.<br />
<br />
A general <math>m\times n</math> matrix <math>A</math> can be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
There is also a ''typical element ''notation for matrices:<br />
<br />
<math>\begin{aligned}<br />
A & =\left\Vert a_{ij}\right\Vert ,\ \ \ \ \ i=1,...,m,j=1,...,n,\end{aligned}</math><br />
<br />
so that <math>a_{ij}</math> is the element at the intersection of the <math>i</math>th row and <math>j</math>th column in <math>A.</math><br />
<br />
When <math>m\neq n,</math> <math>A</math> is a ''rectangular ''matrix; when <math>m=n,</math> <math>A</math> is <math>m\times m</math> or <math>n\times n,</math> and <math>A</math> is a square matrix, having the same number of rows or columns.<br />
<br />
== Rows, columns and vectors ==<br />
<br />
Clearly, there is no reason why <math>m</math> or <math>n</math> cannot equal 1: so, an <math>m\times n</math> matrix with <math>n=1,</math> i.e. with one column, is usually called a column vector. Similarly, a matrix with one row is a row vector.<br />
<br />
There are a lot of advantages to thinking of matrices as collections of row or column vectors, as we shall see. As an example, define the <math>2\times1</math> column vectors:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a} & =\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\mathbf{,\ \ \ b}=\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
and arrange as the columns of the <math>2\times2</math> matrix<br />
<br />
<math>A=\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\label{eq:axy}</math><br />
<br />
In general, a column vector <math>\mathbf{x}</math> with <math>n</math> elements can be written as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\end{aligned}</math><br />
<br />
What happens when both <math>m</math> and <math>n</math> are equal to <math>1?</math> Then, <math>A</math> is a <math>1\times1</math> matrix, but it is also considered to be a real number, or ''scalar'' in the language of linear algebra:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[a_{11}\right]=a_{11}.\end{aligned}</math><br />
<br />
This is perhaps a little odd, but turns out to be a useful convention in a number of situations.<br />
<br />
== Transposition of vectors ==<br />
<br />
The ''rows'' of the matrix <math>A</math> in equation (1) can be seen as elements of column vectors, say:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c} & =\left[\begin{array}{r}<br />
6\\<br />
2<br />
\end{array}\right],\ \ \ \boldsymbol{d}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
This representation of row vectors as column vectors is a bit clumsy, so some transformation which converts a column vector into a row vector, and vice versa would be useful. The process of converting a column vector <math>\mathbf{a}</math> into a row vector is called ''transposition, ''and the transposed version of <math>\mathbf{c}</math> is denoted:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c}^{T} & =\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\end{aligned}</math><br />
<br />
the <math>^{T}</math> superscript denoting transposition. In practice, a prime, <math>^{\prime},</math> is used instead of <math>^{T}.</math> However, whilst the prime is much simpler to write than the <math>^{T}</math> sign, it is also much easier to lose track of in writing out long or complicated expressions. So, it is best initially to use <math>^{T}</math> to denote transposition rather than the prime <math>^{\prime}.</math><br />
<br />
<math>A</math> can then be written via its rows as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{r}<br />
\mathbf{c}^{T}\\<br />
\boldsymbol{d}^{T}<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
The same ideas can be applied to the matrices <math>P</math> and <math>Q.</math><br />
<br />
= Operations with matrices =<br />
<br />
== Addition, subtraction and scalar multiplication ==<br />
<br />
For vectors, addition and subtraction are defined only for vectors of the same dimensions. If:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right],\\<br />
\mathbf{x+y} & =\left[\begin{array}{c}<br />
x_{1}+y_{1}\\<br />
\vdots\\<br />
x_{n}+y_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{x-y}=\left[\begin{array}{c}<br />
x_{1}-y_{1}\\<br />
\vdots\\<br />
x_{n}-y_{n}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Clearly, the addition or subtraction operation is ''elementwise. ''If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> have different dimensions, there will be some elements left over once all the elements of the smaller dimensioned vector have been used up.<br />
<br />
Another operation is ''scalar multiplication'': if <math>\lambda</math> is a real number or scalar, the product <math>\lambda\mathbf{x}</math> is defined as: <math>\lambda\mathbf{x}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right],</math> so that every element of <math>\mathbf{x}</math> is multiplied by the same scalar <math>\lambda.</math><br />
<br />
The two types of operation can be combined into the ''linear combination'' of vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>: <math>\lambda\mathbf{x}+\mu\mathbf{y}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right]+\left[\begin{array}{c}<br />
\mu y_{1}\\<br />
\vdots\\<br />
\mu y_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\lambda x_{1}+\mu y_{1}\\<br />
\vdots\\<br />
\lambda x_{n}+\mu y_{n}<br />
\end{array}\right].</math> Equally, one can define the linear combination of vectors <math>\mathbf{x,y,}\ldots,\mathbf{z}</math> by scalars <math>\lambda,\mu,\ldots,\nu</math> as: <math>\lambda\mathbf{x}+\mu\mathbf{y}+\ldots+\nu\mathbf{z}</math> with typical element: <math>\lambda x_{i}+\mu y_{i}+\ldots+\nu z_{i},</math> provided that all the vectors have the same dimension.<br />
<br />
For matrices, these ideas carry over immediately: apply to each column of the matrices involved. For example, if <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right]</math> and <math>B=\left[\begin{array}{rrr}<br />
\mathbf{b}_{1} & \ldots & \mathbf{b}_{n}\end{array}\right],</math> both <math>m\times n,</math> then addition and subtraction are defined elementwise, as for vectors:<br />
<br />
<math>\begin{aligned}<br />
A+B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}+\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}+\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}+b_{ij}\right\Vert ,\\<br />
A-B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}-\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}-\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}-b_{ij}\right\Vert .\end{aligned}</math><br />
<br />
Scalar multiplication of <math>A</math> by <math>\lambda</math> involves multiplying every column vector of <math>A</math> by <math>\lambda,</math> and therefore multiplying every element of <math>A</math> by <math>\lambda</math>: <math>\lambda A=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1} & \ldots & \lambda\mathbf{a}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}\right\Vert .</math> With the same idea for <math>B,</math> the linear combination of <math>A</math> and <math>B</math> by <math>\lambda</math> and <math>\mu</math> is: <math>\lambda A+\mu B=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1}+\mu\mathbf{b}_{1} & \ldots & \lambda\mathbf{a}_{n}+\mu\mathbf{b}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}+\mu b_{ij}\right\Vert .</math><br />
<br />
For example, consider the matrices: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right]</math> with <math>\lambda=1,</math> <math>\mu=-2:</math> then:<br />
<br />
<math>\begin{aligned}<br />
\lambda A+\mu B & = & A-2B\\<br />
& = & \left[\begin{array}{cc}<br />
4 & 0\\<br />
1 & 7<br />
\end{array}\right].\end{aligned}</math><br />
<br />
== Matrix - vector products ==<br />
<br />
=== Inner product ===<br />
<br />
The simplest form of a matrix vector product is the case where <math>A</math> consists of one row, so that <math>A</math> is <math>1\times n</math>: <math>A=\mathbf{a}^{T}=\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right].</math> If <math>\mathbf{x}</math> is an <math>n\times1</math> vector: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],</math> the product <math>A\mathbf{x}=\mathbf{a}^{T}\mathbf{x}</math> is called the ''inner product'' and is defined as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a}^{T}\mathbf{x} & =a_{1}x_{1}+\ldots+a_{n}x_{n}.\end{aligned}</math><br />
<br />
One can see that the definition amounts to multiplying corresponding elements in <math>\mathbf{a}</math> and <math>\mathbf{x,}</math> and adding up the resultant products. Writing: <math>\mathbf{a}^{T}\mathbf{x=}\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=a_{1}x_{1}+\ldots+a_{n}x_{n}</math> motivates the familiar description of the ''across and down rule ''for this product: ''across and down'' is the ’multiply corresponding elements’ part of the definition.<br />
<br />
Notice that the result of the inner product is a real number, for example: <math>\mathbf{c}^{T}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{c}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=36+6=42.</math><br />
<br />
In general, in the product <math>\mathbf{a}^{T}\mathbf{x,}</math> <math>\mathbf{a}</math> and <math>\mathbf{x}</math> must have the same number of elements, <math>n</math> say, for the product to be defined. If <math>\mathbf{a}</math> and <math>\mathbf{x}</math> had different numbers of elements, there would be some elements of <math>\mathbf{a}</math> or <math>\mathbf{x}</math> left over or not used in the product: e.g.: <math>\mathbf{b}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{x=}\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> When the inner product of two vectors is defined, the vectors are said to be ''conformable''.<br />
<br />
== Orthogonality ==<br />
<br />
Two vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math> with the property that <math>\mathbf{x}^{T}\mathbf{y}=0</math> are said to be orthogonal to each other. For example, if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
-1\\<br />
1<br />
\end{array}\right],</math> it is clear that <math>\mathbf{x}^{T}\mathbf{y}=0.</math> This seems a rather innocuous definition, and yet the idea of orthogonality turns out to be extremely important in econometrics.<br />
<br />
If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> are thought of as points in <math>R^{2},</math> and arrows are drawn from the origin to <math>\mathbf{x}</math> and to <math>\mathbf{y,}</math> then the two arrows are perpendicular to each other - see Figure [orthy<sub>e</sub>xample]. If <math>\mathbf{y}</math> were defined as: <math>\mathbf{y}=\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right],</math> the position of the <math>\mathbf{y}</math> vector and the corresponding arrow would change, but the perpendicularity property would still hold.<br />
<br />
Figure 1:<br />
<br />
[[File:orthy_example.png]]<br />
<br />
<br />
=== Matrix - vector products ===<br />
<br />
Since the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> has two rows, now denoted <math>\boldsymbol{\alpha}_{1}^{T}</math> and <math>\boldsymbol{\alpha}_{2}^{T},</math> there are two possible inner products with the vector:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]:\\<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x} & = & 42,\ \ \ \ \ \boldsymbol{\alpha}_{2}^{T}\mathbf{x}=33.\end{aligned}</math><br />
<br />
Assembling the two inner product values into a <math>2\times1</math> vector defines the product of the matrix <math>A</math> with the vector <math>\mathbf{x}</math>: <math>A\mathbf{x}=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x}\\<br />
\boldsymbol{\alpha}_{2}^{T}\mathbf{x}<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right].</math> Focussing only on the part: <math>\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right],</math> one can see that each element of <math>A\mathbf{x}</math> is obtained from an ''across and down'' argument.<br />
<br />
Sometimes this product is described as forming a ''linear combination ''of the columns of <math>A</math> using the scalar elements in <math>\mathbf{x}</math>: <math>A\mathbf{x}=6\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]+3\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].</math> More generally, if:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right],\ \ \ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
\lambda\\<br />
\mu<br />
\end{array}\right],\\<br />
A\mathbf{x} & = & \lambda\mathbf{a}+\mu\mathbf{b.}\end{aligned}</math><br />
<br />
The general version of these ideas for an <math>m\times n</math> matrix <math>A</math>: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]=\left[\begin{array}{rrrr}<br />
\mathbf{a}_{1} & \mathbf{a}_{2} & \ldots & \mathbf{a}_{n}\end{array}\right].</math> is straightforward. If <math>\mathbf{x}</math> is an <math>n\times1</math> vector, then the vector <math>A\mathbf{x}</math> is, by the ''across and down'' rule:<br />
<br />
<math>A\mathbf{x}=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
x_{2}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
a_{11}x_{1}+\ldots+a_{1n}x_{n}\\<br />
a_{21}x_{1}+\ldots+a_{2n}x_{n}\\<br />
\vdots\\<br />
a_{m1}x_{1}+\ldots+a_{mn}x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\sum\limits _{j=1}^{n}a_{1j}x_{j}\\<br />
\sum\limits _{j=1}^{n}a_{2j}x_{j}\\<br />
\vdots\\<br />
\sum\limits _{j=1}^{n}a_{mj}x_{j}<br />
\end{array}\right],\label{eq:ab}</math><br />
<br />
so that the typical element, the <math>i</math>th, is <math>\sum\limits _{j=1}^{n}a_{ij}x_{j}.</math> Equally, <math>A\mathbf{x}</math> is the linear combination <math>\mathbf{a}_{1}x_{1}+\ldots+\mathbf{a}_{n}x_{n}</math> of the columns of <math>A.</math><br />
<br />
== Matrix - matrix products ==<br />
<br />
Suppose that <math>A</math> is <math>m\times n,</math> with columns <math>\mathbf{a}_{1},\ldots,\mathbf{a}_{n},</math> and <math>B</math> is <math>n\times r,</math> with columns <math>\mathbf{b}_{1},\ldots,\mathbf{b}_{r}.</math> Clearly, each product <math>A\mathbf{b}_{1},...,A\mathbf{b}_{r}</math> exists, and is <math>m\times1.</math> These products can be arranged as the columns of a matrix as <math>\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]</math> and this matrix is ''defined'' to be the product <math>C</math> of the matrices <math>A</math> and <math>B</math>: <math>C=\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]=AB.</math> By construction, this must be an <math>m\times r</math> matrix, since each column is <math>m\times1</math> and there are <math>r</math> columns.<br />
<br />
This is not the usual presentation of the definition of the product of two matrices, which relies on the ''across and down rule'' mentioned earlier, and focusses on the elements of each matrix <math>A</math> and <math>B.</math> Set:<br />
<br />
<math>\begin{aligned}<br />
B & = & \left[\begin{array}{rrrr}<br />
\mathbf{b}_{1} & \mathbf{b}_{2} & \ldots & \mathbf{b}_{r}\end{array}\right]\ \ \ \ \ \ \ \text{(by columns)}\\<br />
& = & \left\Vert b_{ik}\right\Vert ,\ \ \ \ \ i=1,...,n,k=1,...,r\ \ \ \ \ \ \ \text{(typical element)}\\<br />
& = & \left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2r}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nr}<br />
\end{array}\right]\ \ \ \ \ \ \ \text{(the array)}\end{aligned}</math><br />
<br />
What does the typical element of the <math>m\times r</math> matrix <math>C</math> look like? Start with the <math>k</math>th column of <math>C,</math> which is <math>A\mathbf{b}_{k}.</math> The <math>i</math>th element in <math>A\mathbf{b}_{k}</math> is, from equation (2), the inner product of the elements of the <math>i</math>th row in <math>A</math>: <math>\left[\begin{array}{rrrr}<br />
a_{i1} & a_{i2} & \ldots & a_{in}\end{array}\right],</math> with the elements of <math>\mathbf{b}_{k},</math> so that the inner product is: <math>a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math><br />
<br />
So, the <math>ik</math>th element of <math>C</math> is: <math>c_{ik}=a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math> We can see this arising from an ''across and down'' calculation by writing:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\label{eq:c_ab}\\<br />
& = & \left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{rrrrrr}<br />
b_{11} & b_{12} & \ldots & b_{1k} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2k} & \ldots & b_{2r}\\<br />
\vdots & \vdots & & \vdots & & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nk} & \ldots & b_{nr}<br />
\end{array}\right]\\<br />
& = & \left\Vert \sum_{j=1}^{n}a_{ij}b_{jk}\right\Vert .\end{aligned}</math><br />
<br />
These ideas are simple, but a little tedious. Numerical examples are equally tedious! As an example, using: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],</math> we can find the matrix <math>B</math> such that<br />
<br />
# the first column of <math>AB</math> adds together the columns of <math>A,</math><br />
# the second column is the difference of the first and second columns of <math>A,</math><br />
# the third column is <math>2\times</math> the first column of <math>A,</math><br />
# the fourth column is zero.<br />
<br />
It is easy to check that <math>B</math> is: <math>B=\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]</math> and that:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\\<br />
& = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{cccc}<br />
8 & 4 & 12 & 0\\<br />
8 & -2 & 6 & 0<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Arithmetic calculations of matrix products almost always use the elementwise across and down formula. However, there are many situations in econometrics where algebraic rather than arithmetic arguments are required. In these cases, the viewpoint of matrix multiplication as linear combinations of columns is much more powerful.<br />
<br />
Clearly one can give many more examples of different dimensions and complexities - but the same basic rules apply. To multiply two matrices <math>A</math> and <math>B</math> together, the number of columns in <math>A</math> must match the number of rows in <math>B</math> - this is ''conformability'' in action again. The resulting product will have number of rows equal to the number in <math>A</math> and number of columns equal to the number in <math>B.</math><br />
<br />
If this conformability rule does not hold, then the product of <math>A</math> and <math>B</math> is not defined.<br />
<br />
== Matlab ==<br />
<br />
One should also say that as the dimensions of the matrices increases, so the tediousness of the calculations increase. The solution to this for numerical calculation is to appeal to the computer. Programs like Matlab and Excel (and a number of others, some of them free) resolve this difficulty easily.<br />
<br />
In Matlab, symbols for row or column vectors do not need any particular differentiation: they are distinguished by how they are defined. For example, the following Matlab commands define <code>rowvec </code>as a <math>1\times4</math> vector, and <code>colvec</code> as a <math>4\times1</math> vector, then display the contents of these variables, and do a calculation:<br />
<br />
&gt;&gt; rowvec = [1 2 3 4]; <br />
<br />
&gt;&gt; colvec = [1;2;3;4]; <br />
<br />
&gt;&gt; rowvec<br />
<br />
rowvec =<br />
<br />
1 2 3 4<br />
<br />
&gt;&gt; colvec<br />
<br />
colvec =<br />
<br />
1 2 3 4 <br />
<br />
&gt;&gt; rowvec*colvec<br />
<br />
ans =<br />
<br />
30 <br />
<br />
So, the semi-colon indicates the end of a row in a matrix or vector; it can be replaced by a carriage return. Notice the difference in how a row vector and a column vector is defined. One can see that the product <code>rowvec*colvec</code> is well defined, just because <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector.<br />
<br />
Matlab also allows elementwise multiplication of two vectors using the <math>\centerdot\ast</math> operator: if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
x_{1}\\<br />
x_{2}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
y_{1}\\<br />
y_{2}<br />
\end{array}\right],</math> then: <math>\mathbf{x}\centerdot\ast\mathbf{y}=\left[\begin{array}{r}<br />
x_{1}y_{1}\\<br />
x_{2}y_{2}<br />
\end{array}\right]</math> and one can see that the inner product of <math>\mathbf{x}</math> and <math>\mathbf{y}</math> can be obtained as the sum of the elements of <math>\mathbf{x}</math> and <math>\mathbf{y.}</math> In Matlab, this would be obtained as: <math>\text{sum}\left(\mathbf{x}\centerdot\ast\mathbf{y}\right).</math><br />
<br />
In the example above, this calculation fails since <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector:<br />
<br />
&gt;&gt; sum(rowvec .* colvec) ??? <br />
<br />
Error using ==&gt; times Matrix dimensions must agree. <br />
<br />
For this to work, <code>rowvec</code> would have to be transposed as <code>rowvec’</code>, so that transposition in Matlab is very natural.<br />
<br />
Allowing for such difficulties, matrix multiplication in Matlab is very simple:<br />
<br />
&gt;&gt; A = [6 2; 3 5];<br />
<br />
&gt;&gt; B = [1 1 2 0;1 -1 0 0];<br />
<br />
&gt;&gt; C = A * B; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
8 4 1<br />
<br />
2 0 8 <br />
<br />
-2 6 0 <br />
<br />
Notice how the matrices are defined here through their rows. The <code>disp() </code>command displays the contents of the object referred to.<br />
<br />
It is less natural in Matlab to define matrices by columns - a typical example of how mathematics and computing have conflicts of notation. However, once columns <math>\mathbf{a}</math> and <math>\mathbf{b}</math> have been defined, the concatenation operation <math>\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]</math> collects the columns into a matrix:<br />
<br />
&gt;&gt; a = [6;2]; <br />
<br />
&gt;&gt; b = [3;5]; <br />
<br />
&gt;&gt; C = [a b]; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
6 3 <br />
<br />
2 5 <br />
<br />
Notice that the <code>disp(C)</code> command does not label the result that is printed out. Simply typing <code>C</code> would preface the output by <code>C =</code>.<br />
<br />
== Pre and Post Multiplication ==<br />
<br />
If <math>C=AB,</math> as above, say that <math>B</math> is ''pre-multiplied ''by <math>A</math> to get <math>C,</math> and that <math>A</math> is ''post-multiplied ''by <math>B</math> to get <math>C.</math><br />
<br />
This distinction between ''pre ''and ''post ''multiplication is important, in the following sense. Suppose that <math>A</math> and <math>B</math> are matrices such that the products <math>AB</math> and <math>BA</math> are both defined. If <math>A</math> is <math>m\times n,</math> <math>B</math> must have <math>n</math> rows for <math>AB</math> to be defined. For <math>BA</math> to be defined, <math>B</math> must have <math>m</math> columns to match the <math>m</math> rows in <math>A.</math> So, <math>AB</math> and <math>BA</math> are both defined if <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times m.</math><br />
<br />
Even when both products are defined, there is no reason for the two products coincide. The first thing to notice is that <math>AB</math> is a square, <math>m\times m,</math> matrix, whilst <math>BA</math> is a square, <math>n\times n,</math> matrix. Different sized matrices cannot be equal. To illustrate, use the matrices: <math>B_{2}=\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right],\ \ \ C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]:</math><br />
<br />
<math>\begin{aligned}<br />
B_{2}C & = & \left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]=\left[\begin{array}{rrr}<br />
27 & -3 & -15\\<br />
27 & 29 & -11\\<br />
-15 & -1 & 8<br />
\end{array}\right],\\<br />
CB_{2} & = & \left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
49 & -11\\<br />
31 & 15<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Even when <math>m=n,</math> so that <math>AB</math> and <math>BA</math> are both <math>m\times m</math> matrices, the products can differ: for example:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right],\\<br />
AB & = & \left[\begin{array}{cc}<br />
8 & 4\\<br />
8 & -2<br />
\end{array}\right],\ \ \ \ \ BA=\left[\begin{array}{cc}<br />
9 & 7\\<br />
3 & -3<br />
\end{array}\right].\end{aligned}</math><br />
<br />
In cases where <math>AB=BA,</math> the matrices <math>A</math> and <math>B</math> are said to ''commute''.<br />
<br />
== Transposition ==<br />
<br />
A column vector <math>\mathbf{x}</math> can be converted to a row vector <math>\mathbf{x}^{T}</math> by transposition: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \ \ \mathbf{x}^{T}=\left[\begin{array}{rrr}<br />
x_{1} & \ldots & x_{n}\end{array}\right].</math> Transposing <math>\mathbf{x}^{T}</math> as <math>\left(\mathbf{x}^{T}\right)^{T}</math> reproduces the original vector <math>\mathbf{x.}</math> How do these ideas carry over to matrices?<br />
<br />
If the <math>m\times n</math> matrix <math>A</math> can be written as <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right],</math> the transpose of <math>A,</math> <math>A^{T},</math> is defined as the matrix whose ''rows'' are <math>\mathbf{a}_{i}^{T}</math>: <math>A^{T}=\left[\begin{array}{c}<br />
\mathbf{a}_{1}^{T}\\<br />
\vdots\\<br />
\mathbf{a}_{n}^{T}<br />
\end{array}\right].</math> In terms of elements, if: <math>\mathbf{a}_{i}=\left[\begin{array}{c}<br />
a_{1i}\\<br />
\vdots\\<br />
a_{ni}<br />
\end{array}\right]</math> then: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right],\ \ \ \ \ A^{T}=\left[\begin{array}{rrrrr}<br />
a_{11} & \ldots & a_{i1} & \ldots & a_{m1}\\<br />
a_{12} & \ldots & a_{i2} & \ldots & a_{m2}\\<br />
\vdots & & \vdots & & \vdots\\<br />
a_{1n} & \ldots & a_{in} & \ldots & a_{mn}<br />
\end{array}\right].</math> One can see that the first column of <math>A</math> has now become the first row of <math>A^{T}.</math> Notice too that <math>A^{T}</math> is an <math>n\times m</math> matrix if <math>A</math> is an <math>m\times n</math> matrix.<br />
<br />
Transposing <math>A^{T}</math> takes the first column of <math>A^{T}</math> and writes it as a row, which coincides with the first row of <math>A.</math> The same argument applies to the other columns of <math>A^{T},</math> so that <math>\left(A^{T}\right)^{T}=A.</math><br />
<br />
=== The product rule for transposition ===<br />
<br />
This states that if <math>C=AB,</math> then <math>C^{T}=B^{T}A^{T}.</math><br />
<br />
How to see this? Consider the following example: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & b_{13} & b_{14}\\<br />
b_{21} & b_{22} & b_{23} & b_{24}\\<br />
b_{31} & b_{32} & b_{33} & b_{34}<br />
\end{array}\right]</math> where:<br />
<br />
<math>c_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=\sum_{k=1}^{3}a_{2k}b_{k3}.\label{eq:c23}</math><br />
<br />
One can see that: <math>B^{T}A^{T}=\left[\begin{array}{rrr}<br />
b_{11} & b_{21} & b_{31}\\<br />
b_{12} & b_{22} & b_{32}\\<br />
b_{13} & b_{23} & b_{33}\\<br />
b_{14} & b_{24} & b_{34}<br />
\end{array}\right]\left[\begin{array}{rr}<br />
a_{11} & a_{21}\\<br />
a_{12} & a_{22}\\<br />
a_{13} & a_{23}<br />
\end{array}\right]</math> and that the <math>\left(3,2\right)</math> element of this product is actually <math>c_{23}</math>: <math>b_{13}a_{21}+b_{23}a_{22}+b_{33}a_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=c_{23}.</math> In summation notation, we see that from <math>B^{T}A^{T}</math>: <math>c_{23}=\sum_{k=1}^{3}b_{k3}a_{2k},</math> where the position of the index of summation is due to the transposition. So, in summation notation, the calculation of <math>c_{23}</math> from <math>B^{T}A^{T}</math> equals that from equation (6).<br />
<br />
More generally, the <math>\left(i,j\right)</math> element of <math>AB</math>: <math>\sum_{k=1}^{3}a_{ik}b_{kj}</math> is the <math>\left(j,i\right)</math> element of <math>B^{T}A^{T}.</math> But this means that <math>B^{T}A^{T}</math> must be the transpose of <math>AB,</math> since the elements in the <math>i</math>th row of <math>AB</math> are being written in the <math>i</math>th column of <math>B^{T}A^{T}.</math><br />
<br />
This ''Product Rule for Transposition'' can be applied again to find the transpose <math>\left(C^{T}\right)^{T}</math> of <math>C^{T}</math>: <math>\left(C^{T}\right)^{T}=\left(B^{T}A^{T}\right)^{T}=\left(A^{T}\right)^{T}\left(B^{T}\right)^{T}=AB=C.</math><br />
<br />
= Special Types of Matrix =<br />
<br />
== The zero matrix ==<br />
<br />
The most obvious special type of matrix is one whose elements are all zeros. In typical element notation, the zero matrix is: <math>0=\left\Vert 0\right\Vert .</math> Since there is no indexing on the elements, it is not obvious what the dimension of this matrix is, Sometimes one writes <math>0_{mn}</math> to indicate a zero matrix of dimension <math>m\times n.</math> The same ideas apply to vectors whose elements are all zero.<br />
<br />
The effect of the zero matrix in any product that is defined is simple: <math>0A=0,\ \ \ \ \ B0=0.</math> This is easy to check using the across and down rule.<br />
<br />
== The identity or unit matrix ==<br />
<br />
Vectors of the form:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{r}<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }2\ \text{dimensions}\\<br />
\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }3\ \text{dimensions}\\<br />
\left[\begin{array}{r}<br />
1\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1\\<br />
0\\<br />
\vdots\\<br />
0<br />
\end{array}\right],\ldots,\left[\begin{array}{r}<br />
0\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }n\ \text{dimensions}\end{aligned}</math><br />
<br />
are called coordinate vectors. They are often given a characteristic notation, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> in <math>n</math> dimensions. When arranged as columns of a matrix in the natural order, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> a matrix with a characteristic pattern elements emerges, with a special notation:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{rr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2}\end{array}\right] & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=I_{2}\\<br />
\left[\begin{array}{rrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \mathbf{e}_{3}\end{array}\right] & = & \left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 1 & 0\\<br />
0 & 0 & 1<br />
\end{array}\right]=I_{3}\\<br />
\left[\begin{array}{rrrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \ldots & \mathbf{e}_{n}\end{array}\right] & = & \left[\begin{array}{rrrr}<br />
1 & 0 & \ldots & 0\\<br />
0 & 1 & \ldots & 0\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
0 & 0 & \ldots & 1<br />
\end{array}\right]=I_{n}.\end{aligned}</math><br />
<br />
The ''diagonal'' of this matrix is where the 1 elements are located, and every other element is zero.<br />
<br />
Consider the effect of <math>I_{2}</math> on the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> by both pre and post multiplication:<br />
<br />
<math>\begin{aligned}<br />
I_{2}A & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\\<br />
AI_{2} & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\end{aligned}</math><br />
<br />
as is easily checked by the across and down rule.<br />
<br />
Because any matrix is left unchanged by pre or post multiplication by an appropriately dimensioned <math>I_{n},</math> <math>I_{n}</math> is called an ''identity matrix of dimension ''<math>n.</math> Sometimes it is called a ''unit matrix of dimension ''<math>n.</math> Notice that <math>I_{n}</math> is necessarily a square matrix.<br />
<br />
== Diagonal matrices ==<br />
<br />
The identity matrix is an example of a diagonal matrix, a matrix whose elements are all zero except for those on the diagonal. Usually diagonal matrices are taken to be square, for example: <math>D=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 2 & 0\\<br />
0 & 0 & 3<br />
\end{array}\right].</math> They also produce characteristic effects when pre or post multiplying another matrix.<br />
<br />
Consider the diagonal matrix: <math>B=\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]</math> and the products <math>AB,</math> <math>BA</math> for <math>A</math> as defined in the previous section:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & -4\\<br />
6 & -10<br />
\end{array}\right],\\<br />
BA & = & \left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & 4\\<br />
-6 & -10<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Comparing the results, we can deduce that post multiplication by a diagonal matrix multiplies each column of <math>A</math> by the corresponding diagonal element, whereas pre multiplication multiplies each row by the corresponding diagonal element.<br />
<br />
== Symmetric matrices ==<br />
<br />
Symmetric matrices are matrices having the property that <math>A=A^{T}.</math> Notice that such matrices must be square, since if <math>A</math> is <math>m\times n,</math> <math>A^{T}</math> is <math>n\times m,</math> and to have equality of <math>A</math> and <math>A^{T},</math> they must have the same dimension, so that <math>m=n</math> is required.<br />
<br />
Suppose that <math>A</math> is a <math>3\times3</math> symmetric matrix, with typical element <math>a_{ij}</math>: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right],</math> so that: <math>A^{T}=\left[\begin{array}{rrr}<br />
a_{11} & a_{21} & a_{31}\\<br />
a_{12} & a_{22} & a_{32}\\<br />
a_{13} & a_{23} & a_{33}<br />
\end{array}\right].</math> Equality of matrices is defined as equality of all elements. This is fine on the diagonals, since <math>A</math> and <math>A^{T}</math> have the same diagonal elements. For the off diagonal elements, we end up with the requirements: <math>a_{12}=a_{21},\ \ \ a_{13}=a_{31},\ \ \ a_{23}=a_{32}</math> or more generally: <math>a_{ij}=a_{ji}\ \ \ \ \ \text{for}\ i\neq j.</math><br />
<br />
The effect of this conclusion is that in a symmetric matrix, the ’triangle’ of above diagonal elements coincides with the triangle of below diagonal elements. It is as if the upper triangle is folded over the diagonal to become the lower triangle.<br />
<br />
A simple example is: <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
2 & 1<br />
\end{array}\right].</math> A more complicated example uses the <math>2\times3</math> matrix <math>C</math>: <math>C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]</math> and calculates the <math>3\times3</math> matrix:<br />
<br />
<math>\begin{aligned}<br />
C^{T}C & = & \left[\begin{array}{cc}<br />
6 & 3\\<br />
2 & 5\\<br />
-3 & -1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{rrr}<br />
45 & 27 & -21\\<br />
27 & 29 & -11\\<br />
-21 & -11 & 10<br />
\end{array}\right],\end{aligned}</math><br />
<br />
which is clearly symmetric.<br />
<br />
This illustrates the general proposition that if <math>A</math> is an <math>m\times n</math> matrix, the product <math>A^{T}A</math> is a symmetric <math>n\times n</math> matrix. Proof? Compute the transpose of <math>A^{T}A</math> using the product rule for transposition: <math>\left(A^{T}A\right)^{T}=A^{T}\left(A^{T}\right)^{T}=A^{T}A.</math> Since <math>A^{T}A</math> is equal to its transpose, it must be a symmetric matrix. Such symmetric matrices appear frequently in econometrics.<br />
<br />
It should be clear that diagonal matrices are symmetric, since all their off-diagonal elements are equal (zero), and thence the identity matrix <math>I_{n}</math> is also symmetric.<br />
<br />
== The outer product ==<br />
<br />
The inner product of two <math>n\times1</math> vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>, <math>\mathbf{x}^{T}\mathbf{y}</math>, is automatically a <math>1\times1</math> quantity, a scalar, although it can be interpreted as a <math>1\times1</math> matrix, a matrix with a single element.<br />
<br />
Suppose one considered the product of <math>\mathbf{x}</math> with <math>\mathbf{x}^{T}.</math> Is this defined? If <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times r,</math> then the product <math>AB</math> is <math>m\times r.</math> Applying this logic to <math>\mathbf{xx}^{T},</math> this is <math>\left(n\times1\right)\left(1\times n\right),</math> so the resulting product ''is ''defined, and is an <math>n\times n</math>'' matrix'' - the ''outer product'' of <math>\mathbf{x}</math> and <math>\mathbf{x}^{T},</math> the word ’outer’ being used to distinguish from the inner product.<br />
<br />
How does the across and down rule work here? Suppose that: <math>\mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> Then: <math>\mathbf{xx}^{T}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right].</math> Here, there is <math>1</math> element in row one of the ’matrix’ <math>\mathbf{x,}</math> and <math>1</math> element in column one of the matrix <math>\mathbf{x}^{T},</math> so the across and down rule still works - it is just that there is only one product per row and column combination. So: <math>\mathbf{xx}^{T}=\left[\begin{array}{cc}<br />
36 & 18\\<br />
18 & 9<br />
\end{array}\right],</math> and it is obvious from this that <math>\mathbf{xx}^{T}</math> is a symmetric matrix.<br />
<br />
One can see that this outer product need not be restricted to vectors of the same dimension. If <math>\mathbf{x}</math> is <math>n\times1</math> and <math>\mathbf{y}</math> is <math>m\times1,</math> then: <math>\mathbf{xy}^{T}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
y_{1} & \ldots & y_{m}\end{array}\right]=\left[\begin{array}{rrrr}<br />
x_{1}y_{1} & x_{1}y_{2} & \ldots & x_{1}y_{m}\\<br />
x_{2}y_{1} & x_{2}y_{2} & \ldots & x_{2}y_{m}\\<br />
\\<br />
x_{n}y_{1} & x_{n}y_{2} & \ldots & x_{n}y_{m}<br />
\end{array}\right].</math> So, <math>\mathbf{xy}^{T}</math> is <math>n\times m,</math> and consists of rows which are <math>\mathbf{y}^{T}</math> multiplied by an element of the <math>\mathbf{x}</math> vector.<br />
<br />
Another interesting and useful example involves a vector with every element equal to <math>1</math>: <math>\mathbf{1}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right].</math> Sometimes this is written as <math>\mathbf{1}_{n}</math> to indicate an <math>n\times1</math> vector, and is called the ''sum vector''. Why? Consider the impact of <math>\mathbf{1}_{2}</math> on the <math>2\times1</math> vector <math>\mathbf{x}</math> used above: <math>\mathbf{1}_{2}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=9,</math> i.e. an inner product of <math>\mathbf{x}</math> with the sum vector is the sum of the elements of <math>\mathbf{x.}</math> Dividing through by the number of elements in <math>\mathbf{x}</math> produces the average of the elements of <math>\mathbf{x}</math> - i.e. the ’sample mean’ of the elements of <math>\mathbf{x.}</math><br />
<br />
The outer product of <math>\mathbf{x}</math> with <math>\mathbf{1}_{2}</math> is also interesting:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{1}_{2}\mathbf{x}^{T} & = & \left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 3\\<br />
6 & 3<br />
\end{array}\right],\\<br />
\mathbf{x1}_{2}^{T} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 6\\<br />
3 & 3<br />
\end{array}\right],\end{aligned}</math><br />
<br />
showing that pre multiplication of an <math>\mathbf{x}^{T}</math> by <math>\mathbf{1}</math> repeats <math>\mathbf{x}^{T}</math> as rows of the product, whilst post multiplication of <math>\mathbf{x}</math> by <math>\mathbf{1}^{T}</math> repeats <math>\mathbf{x}</math> as the columns of the product.<br />
<br />
Finally: <math>\mathbf{1}_{n}\mathbf{1}_{n}^{T}=\left[\begin{array}{rrr}<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1<br />
\end{array}\right],</math> an <math>n\times n</math> matrix with every element equal to <math>1.</math> This type of matrix also appears in econometrics!<br />
<br />
== Triangular matrices ==<br />
<br />
A square ''lower triangular ''matrix has all elements above the main diagonal equal to zero, whilst a square ''upper triangular ''matrix has all elements below the main diagonal equal to zero. A simple example of a lower triangular matrix is: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & 0 & 0\\<br />
a_{21} & a_{22} & 0\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right].</math> Clearly, for this matrix, <math>A^{T}</math> is an upper triangular matrix.<br />
<br />
One can adapt the definition to rectangular matrices: for example, if two arbitrary rows are added to <math>A,</math> so that it becomes <math>5\times3,</math> it would still be considered lower triangular. Equally, if, for example, the third column of <math>A</math> above is removed, <math>A</math> is still considered lower triangular.<br />
<br />
Often, we use ''unit ''triangular matrices, where the diagonal elements are all equal to <math>1:</math> e.g.:<br />
<br />
<math>\left[\begin{array}{rrr}<br />
1 & 2 & 0\\<br />
0 & 1 & 1\\<br />
0 & 0 & 1<br />
\end{array}\right].\label{eq:lt_matrix}</math><br />
<br />
== Partitioned matrices ==<br />
<br />
Sometimes, especially with big matrices, it is useful to organise the elements of the matrix into components which are themselves matrices, for example: <math>B=\left[\begin{array}{rrrr}<br />
1 & 2 & 0 & 0\\<br />
8 & 3 & 0 & 0\\<br />
0 & 0 & 7 & 4\\<br />
0 & 0 & 6 & 5<br />
\end{array}\right]</math> Here it would be reasonable to write: <math>B=\left[\begin{array}{cc}<br />
B_{11} & 0\\<br />
0 & B_{22}<br />
\end{array}\right],</math> where <math>B_{ii},i=1,2,</math> represent <math>2\times2</math> matrices. <math>B</math> is an example of a ''partitioned matrix'': that is, an <math>m\times n</math> matrix <math>A</math> say: <math>A=\left\Vert a_{ij}\right\Vert ,</math> where the elements of <math>A</math> are organised into ''sub-matrices''. An example might be:<br />
<br />
<math>A=\left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right],\label{eq:partition_a}</math><br />
<br />
where the ''sub - matrices'' in the first row block have <math>r</math> rows, and therefore <math>m-r</math> rows in the second row block. The column blocks might be defined by (for example) 3 columns in the first column block, 4 in the second and <math>n-7</math> in the third column block.<br />
<br />
Another simple example might be: <math>A=\left[\begin{array}{rrr}<br />
A_{1} & A_{2} & A_{3}\end{array}\right],\ \ \ \ \ \mathbf{x=}\left[\begin{array}{c}<br />
\mathbf{x}_{1}\\<br />
\mathbf{x}_{2}\\<br />
\mathbf{x}_{3}<br />
\end{array}\right],</math> where <math>A</math> and therefore <math>A_{1},A_{2},A_{3}</math> have <math>m</math> rows, <math>A_{1}</math> has <math>n_{1}</math> columns, <math>A_{2}</math> has <math>n_{2}</math> columns, <math>A_{3}</math> has <math>n_{3}</math> columns. The ''subvectors'' in <math>\mathbf{x}</math> must have <math>n_{1},n_{2}</math> and <math>n_{3}</math> rows respectively, for the product <math>A\mathbf{x}</math> to exist.<br />
<br />
Suppose that <math>n_{1}+n_{2}+n_{3}=n,</math> so that <math>A</math> is <math>m\times n.</math> The <math>i</math>th element of <math>A\mathbf{x}</math> is: <math>\sum_{i=1}^{n}a_{ij}x_{j}</math> but the summation can be broken up into the first <math>n_{1}</math> terms: <math>\sum_{i=1}^{n_{1}}a_{ij}x_{j},</math> the next <math>n_{2}</math> terms: <math>\sum_{i=n_{1}+1}^{n_{1}+n_{2}}a_{ij}x_{j},</math> and the next <math>n_{3}</math> terms; <math>\sum_{i=n_{1}+n_{2}+1}^{n}a_{ij}x_{j}.</math><br />
<br />
The point about the use of partitioned matrices is that the product <math>A\mathbf{x}</math> can be represented as: <math>A\mathbf{x}=A_{1}\mathbf{x}_{1}+A_{2}\mathbf{x}_{2}+A\mathbf{x}_{3}</math> by applying the across and down rule to the submatrices and the subvectors, a much simpler representation than the use of summations.<br />
<br />
Each of the components is a conformable matrix-vector product: this is essential in any use of partitioned matrices to represent some matrix product. For example, using <math>A</math> from equation (8) and <math>B</math> as: <math>B=\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right],</math> it is easy to write:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right]\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{r}<br />
A_{11}B_{11}+A_{12}B_{21}+A_{13}B_{31}\\<br />
A_{21}B_{11}+A_{22}B_{21}+A_{23}B_{31}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
But, what are the row dimensions for the submatrices in <math>B?</math> What are the possible column dimensions for the submatrices in <math>B?</math><br />
<br />
= Matrices, vectors and econometrics =<br />
<br />
The data on weights and heights for 12 students in the data matrix: <math>D=\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]</math> would seem to be ideally suited for fitting a two variable regression model: <math>y_{i}=\alpha+\beta x_{i}+u_{i},\;\;\;\;\; i=1,...,12.</math> Here, the first column of <math>D</math> contains all the weight data, the data on the dependent variable <math>y_{i},</math> and so should be labelled <math>\mathbf{y.}</math> The second column of <math>D</math> contains all the data on the explanatory variable height, in the vector <math>\mathbf{x}</math> say, so that: <math>D=\left[\begin{array}{rr}<br />
\mathbf{y} & \mathbf{x}\end{array}\right].</math><br />
<br />
If we define a <math>12\times1</math> vector with every element <math>1</math>: <math>\mathbf{1}_{12}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right],</math> and a <math>12\times1</math> vector <math>\mathbf{u}</math> to contain the error terms; <math>\mathbf{u}=\left[\begin{array}{c}<br />
u_{1}\\<br />
\vdots\\<br />
u_{12}<br />
\end{array}\right]</math> the regression model can be written in terms of the three data vectors <math>\mathbf{y,1}_{12}</math> and <math>\mathbf{x}</math> as: <math>\mathbf{y}=\mathbf{1}_{12}\alpha+\mathbf{x}\beta+\mathbf{u.}</math> To see this, think of the <math>i</math>th elements of the vectors on the left and right hand sides.<br />
<br />
The standard next step is then to combine the data vectors for the explanatory variables into a matrix: <math>X=\left[\begin{array}{rr}<br />
\mathbf{1}_{12} & \mathbf{x}\end{array}\right],</math> and then define a <math>2\times1</math> vector <math>\boldsymbol{\delta}</math> to contain the parameters <math>\alpha,\beta</math> as: <math>\boldsymbol{\delta}=\left[\begin{array}{r}<br />
\alpha\\<br />
\beta<br />
\end{array}\right]</math> to give the data matrix representation of the regression model as: <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
<br />
For the purposes of developing the theory of regression, this is the most convenient form of the regression model. It can represent regression models with any number of explanatory variables, and thus any number of parameters. The obvious point is that a knowledge of vector and matrix operations is needed to use and understand this form.<br />
<br />
We shall see later that there are two particular matrix and vector quantities associated with a regression model. The first is the matrix <math>X^{T}X,</math> and the second the vector <math>X^{T}\mathbf{y.}</math> The following Matlab code snippet provides the numerical values of these quantities for the weight data:<br />
<br />
&gt;&gt; dset = load(’weights.mat’); <br />
<br />
&gt;&gt; xtx = dset.X’ * dset.X; <br />
<br />
&gt;&gt; xty = dset.X’ * dset.y; <br />
<br />
&gt;&gt; disp(xtx) <br />
<br />
12 802<br />
<br />
802 53792<br />
<br />
&gt;&gt; disp(xty)<br />
<br />
1850<br />
<br />
124528<br />
<br />
Hand calculation is of course possible, but not recommended.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=LNotes&diff=3027LNotes2013-09-10T14:11:24Z<p>LG: </p>
<hr />
<div>= Matrices =<br />
<br />
In the PreSession Maths course, a matrix was defined as follows:<br />
<br />
<blockquote>A matrix is a rectangular array of numbers enclosed in parentheses, conventionally denoted by a capital letter. The number of rows (say <math>m</math>) and<br />
<br />
the number of columns (say <math>n</math>) determine the order of the matrix (<math>m</math> <math>\times</math> <math>n</math>).<br />
</blockquote><br />
Two examples were given:<br />
<br />
<math>\begin{aligned}<br />
P & =\left[\begin{array}{rrr}<br />
2 & 3 & 4\\<br />
3 & 1 & 5<br />
\end{array}\right],\ \ \ Q=\left[\begin{array}{rr}<br />
2 & 3\\<br />
4 & 3\\<br />
1 & 5<br />
\end{array}\right],\end{aligned}</math><br />
<br />
matrices of dimensions <math>2\times3</math> and <math>3\times2</math> respectively.<br />
<br />
Why study matrices for econometrics? Basically because a data set of several variables, e.g. on the weights and heights of 12 students, can be thought of as a matrix:<br />
<br />
<math>\begin{aligned}<br />
D & =\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]\end{aligned}</math><br />
<br />
The properties of matrices can then be used to facilitate answering all the usual questions of econometrics - list not given here!<br />
<br />
Calculations with matrices with explicit numerical elements, as in the examples above is called matrix ''arithmetic''. Matrix ''algebra'' is the algebra of matrices where the elements are not made explicit: this is what is really required for econometrics, as we shall see.<br />
<br />
As an example of this, a <math>2\times3</math> matrix might be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{ccc}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\end{aligned}</math><br />
<br />
and would equal <math>P</math> above if the collection of <math>a_{ij}</math> were given appropriate numerical values.<br />
<br />
A general <math>m\times n</math> matrix <math>A</math> can be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
There is also a ''typical element ''notation for matrices:<br />
<br />
<math>\begin{aligned}<br />
A & =\left\Vert a_{ij}\right\Vert ,\ \ \ \ \ i=1,...,m,j=1,...,n,\end{aligned}</math><br />
<br />
so that <math>a_{ij}</math> is the element at the intersection of the <math>i</math>th row and <math>j</math>th column in <math>A.</math><br />
<br />
When <math>m\neq n,</math> <math>A</math> is a ''rectangular ''matrix; when <math>m=n,</math> <math>A</math> is <math>m\times m</math> or <math>n\times n,</math> and <math>A</math> is a square matrix, having the same number of rows or columns.<br />
<br />
== Rows, columns and vectors ==<br />
<br />
Clearly, there is no reason why <math>m</math> or <math>n</math> cannot equal 1: so, an <math>m\times n</math> matrix with <math>n=1,</math> i.e. with one column, is usually called a column vector. Similarly, a matrix with one row is a row vector.<br />
<br />
There are a lot of advantages to thinking of matrices as collections of row or column vectors, as we shall see. As an example, define the <math>2\times1</math> column vectors:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a} & =\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\mathbf{,\ \ \ b}=\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
and arrange as the columns of the <math>2\times2</math> matrix<br />
<br />
<math>A=\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\label{eq:axy}</math><br />
<br />
In general, a column vector <math>\mathbf{x}</math> with <math>n</math> elements can be written as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\end{aligned}</math><br />
<br />
What happens when both <math>m</math> and <math>n</math> are equal to <math>1?</math> Then, <math>A</math> is a <math>1\times1</math> matrix, but it is also considered to be a real number, or ''scalar'' in the language of linear algebra:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[a_{11}\right]=a_{11}.\end{aligned}</math><br />
<br />
This is perhaps a little odd, but turns out to be a useful convention in a number of situations.<br />
<br />
== Transposition of vectors ==<br />
<br />
The ''rows'' of the matrix <math>A</math> in equation (1) can be seen as elements of column vectors, say:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c} & =\left[\begin{array}{r}<br />
6\\<br />
2<br />
\end{array}\right],\ \ \ \boldsymbol{d}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
This representation of row vectors as column vectors is a bit clumsy, so some transformation which converts a column vector into a row vector, and vice versa would be useful. The process of converting a column vector <math>\mathbf{a}</math> into a row vector is called ''transposition, ''and the transposed version of <math>\mathbf{c}</math> is denoted:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c}^{T} & =\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\end{aligned}</math><br />
<br />
the <math>^{T}</math> superscript denoting transposition. In practice, a prime, <math>^{\prime},</math> is used instead of <math>^{T}.</math> However, whilst the prime is much simpler to write than the <math>^{T}</math> sign, it is also much easier to lose track of in writing out long or complicated expressions. So, it is best initially to use <math>^{T}</math> to denote transposition rather than the prime <math>^{\prime}.</math><br />
<br />
<math>A</math> can then be written via its rows as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{r}<br />
\mathbf{c}^{T}\\<br />
\boldsymbol{d}^{T}<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
The same ideas can be applied to the matrices <math>P</math> and <math>Q.</math><br />
<br />
= Operations with matrices =<br />
<br />
== Addition, subtraction and scalar multiplication ==<br />
<br />
For vectors, addition and subtraction are defined only for vectors of the same dimensions. If:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right],\\<br />
\mathbf{x+y} & =\left[\begin{array}{c}<br />
x_{1}+y_{1}\\<br />
\vdots\\<br />
x_{n}+y_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{x-y}=\left[\begin{array}{c}<br />
x_{1}-y_{1}\\<br />
\vdots\\<br />
x_{n}-y_{n}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Clearly, the addition or subtraction operation is ''elementwise. ''If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> have different dimensions, there will be some elements left over once all the elements of the smaller dimensioned vector have been used up.<br />
<br />
Another operation is ''scalar multiplication'': if <math>\lambda</math> is a real number or scalar, the product <math>\lambda\mathbf{x}</math> is defined as: <math>\lambda\mathbf{x}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right],</math> so that every element of <math>\mathbf{x}</math> is multiplied by the same scalar <math>\lambda.</math><br />
<br />
The two types of operation can be combined into the ''linear combination'' of vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>: <math>\lambda\mathbf{x}+\mu\mathbf{y}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right]+\left[\begin{array}{c}<br />
\mu y_{1}\\<br />
\vdots\\<br />
\mu y_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\lambda x_{1}+\mu y_{1}\\<br />
\vdots\\<br />
\lambda x_{n}+\mu y_{n}<br />
\end{array}\right].</math> Equally, one can define the linear combination of vectors <math>\mathbf{x,y,}\ldots,\mathbf{z}</math> by scalars <math>\lambda,\mu,\ldots,\nu</math> as: <math>\lambda\mathbf{x}+\mu\mathbf{y}+\ldots+\nu\mathbf{z}</math> with typical element: <math>\lambda x_{i}+\mu y_{i}+\ldots+\nu z_{i},</math> provided that all the vectors have the same dimension.<br />
<br />
For matrices, these ideas carry over immediately: apply to each column of the matrices involved. For example, if <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right]</math> and <math>B=\left[\begin{array}{rrr}<br />
\mathbf{b}_{1} & \ldots & \mathbf{b}_{n}\end{array}\right],</math> both <math>m\times n,</math> then addition and subtraction are defined elementwise, as for vectors:<br />
<br />
<math>\begin{aligned}<br />
A+B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}+\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}+\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}+b_{ij}\right\Vert ,\\<br />
A-B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}-\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}-\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}-b_{ij}\right\Vert .\end{aligned}</math><br />
<br />
Scalar multiplication of <math>A</math> by <math>\lambda</math> involves multiplying every column vector of <math>A</math> by <math>\lambda,</math> and therefore multiplying every element of <math>A</math> by <math>\lambda</math>: <math>\lambda A=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1} & \ldots & \lambda\mathbf{a}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}\right\Vert .</math> With the same idea for <math>B,</math> the linear combination of <math>A</math> and <math>B</math> by <math>\lambda</math> and <math>\mu</math> is: <math>\lambda A+\mu B=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1}+\mu\mathbf{b}_{1} & \ldots & \lambda\mathbf{a}_{n}+\mu\mathbf{b}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}+\mu b_{ij}\right\Vert .</math><br />
<br />
For example, consider the matrices: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right]</math> with <math>\lambda=1,</math> <math>\mu=-2:</math> then:<br />
<br />
<math>\begin{aligned}<br />
\lambda A+\mu B & = & A-2B\\<br />
& = & \left[\begin{array}{cc}<br />
4 & 0\\<br />
1 & 7<br />
\end{array}\right].\end{aligned}</math><br />
<br />
== Matrix - vector products ==<br />
<br />
=== Inner product ===<br />
<br />
The simplest form of a matrix vector product is the case where <math>A</math> consists of one row, so that <math>A</math> is <math>1\times n</math>: <math>A=\mathbf{a}^{T}=\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right].</math> If <math>\mathbf{x}</math> is an <math>n\times1</math> vector: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],</math> the product <math>A\mathbf{x}=\mathbf{a}^{T}\mathbf{x}</math> is called the ''inner product'' and is defined as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a}^{T}\mathbf{x} & =a_{1}x_{1}+\ldots+a_{n}x_{n}.\end{aligned}</math><br />
<br />
One can see that the definition amounts to multiplying corresponding elements in <math>\mathbf{a}</math> and <math>\mathbf{x,}</math> and adding up the resultant products. Writing: <math>\mathbf{a}^{T}\mathbf{x=}\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=a_{1}x_{1}+\ldots+a_{n}x_{n}</math> motivates the familiar description of the ''across and down rule ''for this product: ''across and down'' is the ’multiply corresponding elements’ part of the definition.<br />
<br />
Notice that the result of the inner product is a real number, for example: <math>\mathbf{c}^{T}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{c}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=36+6=42.</math><br />
<br />
In general, in the product <math>\mathbf{a}^{T}\mathbf{x,}</math> <math>\mathbf{a}</math> and <math>\mathbf{x}</math> must have the same number of elements, <math>n</math> say, for the product to be defined. If <math>\mathbf{a}</math> and <math>\mathbf{x}</math> had different numbers of elements, there would be some elements of <math>\mathbf{a}</math> or <math>\mathbf{x}</math> left over or not used in the product: e.g.: <math>\mathbf{b}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{x=}\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> When the inner product of two vectors is defined, the vectors are said to be ''conformable''.<br />
<br />
== Orthogonality ==<br />
<br />
Two vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math> with the property that <math>\mathbf{x}^{T}\mathbf{y}=0</math> are said to be orthogonal to each other. For example, if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
-1\\<br />
1<br />
\end{array}\right],</math> it is clear that <math>\mathbf{x}^{T}\mathbf{y}=0.</math> This seems a rather innocuous definition, and yet the idea of orthogonality turns out to be extremely important in econometrics.<br />
<br />
If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> are thought of as points in <math>R^{2},</math> and arrows are drawn from the origin to <math>\mathbf{x}</math> and to <math>\mathbf{y,}</math> then the two arrows are perpendicular to each other - see Figure [orthy<sub>e</sub>xample]. If <math>\mathbf{y}</math> were defined as: <math>\mathbf{y}=\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right],</math> the position of the <math>\mathbf{y}</math> vector and the corresponding arrow would change, but the perpendicularity property would still hold.<br />
<br />
Figure 1:<br />
<br />
[[File:orthy_example.png]]<br />
<br />
<br />
=== Matrix - vector products ===<br />
<br />
Since the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> has two rows, now denoted <math>\boldsymbol{\alpha}_{1}^{T}</math> and <math>\boldsymbol{\alpha}_{2}^{T},</math> there are two possible inner products with the vector:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]:\\<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x} & = & 42,\ \ \ \ \ \boldsymbol{\alpha}_{2}^{T}\mathbf{x}=33.\end{aligned}</math><br />
<br />
Assembling the two inner product values into a <math>2\times1</math> vector defines the product of the matrix <math>A</math> with the vector <math>\mathbf{x}</math>: <math>A\mathbf{x}=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x}\\<br />
\boldsymbol{\alpha}_{2}^{T}\mathbf{x}<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right].</math> Focussing only on the part: <math>\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right],</math> one can see that each element of <math>A\mathbf{x}</math> is obtained from an ''across and down'' argument.<br />
<br />
Sometimes this product is described as forming a ''linear combination ''of the columns of <math>A</math> using the scalar elements in <math>\mathbf{x}</math>: <math>A\mathbf{x}=6\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]+3\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].</math> More generally, if:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right],\ \ \ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
\lambda\\<br />
\mu<br />
\end{array}\right],\\<br />
A\mathbf{x} & = & \lambda\mathbf{a}+\mu\mathbf{b.}\end{aligned}</math><br />
<br />
The general version of these ideas for an <math>m\times n</math> matrix <math>A</math>: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]=\left[\begin{array}{rrrr}<br />
\mathbf{a}_{1} & \mathbf{a}_{2} & \ldots & \mathbf{a}_{n}\end{array}\right].</math> is straightforward. If <math>\mathbf{x}</math> is an <math>n\times1</math> vector, then the vector <math>A\mathbf{x}</math> is, by the ''across and down'' rule:<br />
<br />
<math>A\mathbf{x}=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
x_{2}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
a_{11}x_{1}+\ldots+a_{1n}x_{n}\\<br />
a_{21}x_{1}+\ldots+a_{2n}x_{n}\\<br />
\vdots\\<br />
a_{m1}x_{1}+\ldots+a_{mn}x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\sum\limits _{j=1}^{n}a_{1j}x_{j}\\<br />
\sum\limits _{j=1}^{n}a_{2j}x_{j}\\<br />
\vdots\\<br />
\sum\limits _{j=1}^{n}a_{mj}x_{j}<br />
\end{array}\right],\label{eq:ab}</math><br />
<br />
so that the typical element, the <math>i</math>th, is <math>\sum\limits _{j=1}^{n}a_{ij}x_{j}.</math> Equally, <math>A\mathbf{x}</math> is the linear combination <math>\mathbf{a}_{1}x_{1}+\ldots+\mathbf{a}_{n}x_{n}</math> of the columns of <math>A.</math><br />
<br />
== Matrix - matrix products ==<br />
<br />
Suppose that <math>A</math> is <math>m\times n,</math> with columns <math>\mathbf{a}_{1},\ldots,\mathbf{a}_{n},</math> and <math>B</math> is <math>n\times r,</math> with columns <math>\mathbf{b}_{1},\ldots,\mathbf{b}_{r}.</math> Clearly, each product <math>A\mathbf{b}_{1},...,A\mathbf{b}_{r}</math> exists, and is <math>m\times1.</math> These products can be arranged as the columns of a matrix as <math>\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]</math> and this matrix is ''defined'' to be the product <math>C</math> of the matrices <math>A</math> and <math>B</math>: <math>C=\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]=AB.</math> By construction, this must be an <math>m\times r</math> matrix, since each column is <math>m\times1</math> and there are <math>r</math> columns.<br />
<br />
This is not the usual presentation of the definition of the product of two matrices, which relies on the ''across and down rule'' mentioned earlier, and focusses on the elements of each matrix <math>A</math> and <math>B.</math> Set:<br />
<br />
<math>\begin{aligned}<br />
B & = & \left[\begin{array}{rrrr}<br />
\mathbf{b}_{1} & \mathbf{b}_{2} & \ldots & \mathbf{b}_{r}\end{array}\right]\text{\ \ \ \ \ \ \ (by columns)}\\<br />
& = & \left\Vert b_{ik}\right\Vert ,\ \ \ \ \ i=1,...,n,k=1,...,r\text{ \ \ \ \ \ \ (typical element)}\\<br />
& = & \left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2r}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nr}<br />
\end{array}\right]\text{\ \ \ \ \ \ \ (the array)}\end{aligned}</math><br />
<br />
What does the typical element of the <math>m\times r</math> matrix <math>C</math> look like? Start with the <math>k</math>th column of <math>C,</math> which is <math>A\mathbf{b}_{k}.</math> The <math>i</math>th element in <math>A\mathbf{b}_{k}</math> is, from equation (2), the inner product of the elements of the <math>i</math>th row in <math>A</math>: <math>\left[\begin{array}{rrrr}<br />
a_{i1} & a_{i2} & \ldots & a_{in}\end{array}\right],</math> with the elements of <math>\mathbf{b}_{k},</math> so that the inner product is: <math>a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math><br />
<br />
So, the <math>ik</math>th element of <math>C</math> is: <math>c_{ik}=a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math> We can see this arising from an ''across and down'' calculation by writing:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\label{eq:c_ab}\\<br />
& = & \left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{rrrrrr}<br />
b_{11} & b_{12} & \ldots & b_{1k} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2k} & \ldots & b_{2r}\\<br />
\vdots & \vdots & & \vdots & & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nk} & \ldots & b_{nr}<br />
\end{array}\right]\\<br />
& = & \left\Vert \sum_{j=1}^{n}a_{ij}b_{jk}\right\Vert .\end{aligned}</math><br />
<br />
These ideas are simple, but a little tedious. Numerical examples are equally tedious! As an example, using: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],</math> we can find the matrix <math>B</math> such that<br />
<br />
# the first column of <math>AB</math> adds together the columns of <math>A,</math><br />
# the second column is the difference of the first and second columns of <math>A,</math><br />
# the third column is <math>2\times</math> the first column of <math>A,</math><br />
# the fourth column is zero.<br />
<br />
It is easy to check that <math>B</math> is: <math>B=\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]</math> and that:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\\<br />
& = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{cccc}<br />
8 & 4 & 12 & 0\\<br />
8 & -2 & 6 & 0<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Arithmetic calculations of matrix products almost always use the elementwise across and down formula. However, there are many situations in econometrics where algebraic rather than arithmetic arguments are required. In these cases, the viewpoint of matrix multiplication as linear combinations of columns is much more powerful.<br />
<br />
Clearly one can give many more examples of different dimensions and complexities - but the same basic rules apply. To multiply two matrices <math>A</math> and <math>B</math> together, the number of columns in <math>A</math> must match the number of rows in <math>B</math> - this is ''conformability'' in action again. The resulting product will have number of rows equal to the number in <math>A</math> and number of columns equal to the number in <math>B.</math><br />
<br />
If this conformability rule does not hold, then the product of <math>A</math> and <math>B</math> is not defined.<br />
<br />
== Matlab ==<br />
<br />
One should also say that as the dimensions of the matrices increases, so the tediousness of the calculations increase. The solution to this for numerical calculation is to appeal to the computer. Programs like Matlab and Excel (and a number of others, some of them free) resolve this difficulty easily.<br />
<br />
In Matlab, symbols for row or column vectors do not need any particular differentiation: they are distinguished by how they are defined. For example, the following Matlab commands define <code>rowvec </code>as a <math>1\times4</math> vector, and <code>colvec</code> as a <math>4\times1</math> vector, then display the contents of these variables, and do a calculation:<br />
<br />
&gt;&gt; rowvec = [1 2 3 4]; <br />
<br />
&gt;&gt; colvec = [1;2;3;4]; <br />
<br />
&gt;&gt; rowvec<br />
<br />
rowvec =<br />
<br />
1 2 3 4<br />
<br />
&gt;&gt; colvec<br />
<br />
colvec =<br />
<br />
1 2 3 4 <br />
<br />
&gt;&gt; rowvec*colvec<br />
<br />
ans =<br />
<br />
30 <br />
<br />
So, the semi-colon indicates the end of a row in a matrix or vector; it can be replaced by a carriage return. Notice the difference in how a row vector and a column vector is defined. One can see that the product <code>rowvec*colvec</code> is well defined, just because <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector.<br />
<br />
Matlab also allows elementwise multiplication of two vectors using the <math>\centerdot\ast</math> operator: if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
x_{1}\\<br />
x_{2}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
y_{1}\\<br />
y_{2}<br />
\end{array}\right],</math> then: <math>\mathbf{x}\centerdot\ast\mathbf{y}=\left[\begin{array}{r}<br />
x_{1}y_{1}\\<br />
x_{2}y_{2}<br />
\end{array}\right]</math> and one can see that the inner product of <math>\mathbf{x}</math> and <math>\mathbf{y}</math> can be obtained as the sum of the elements of <math>\mathbf{x}</math> and <math>\mathbf{y.}</math> In Matlab, this would be obtained as: <math>\text{sum}\left(\mathbf{x}\centerdot\ast\mathbf{y}\right).</math><br />
<br />
In the example above, this calculation fails since <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector:<br />
<br />
&gt;&gt; sum(rowvec .* colvec) ??? <br />
<br />
Error using ==&gt; times Matrix dimensions must agree. <br />
<br />
For this to work, <code>rowvec</code> would have to be transposed as <code>rowvec’</code>, so that transposition in Matlab is very natural.<br />
<br />
Allowing for such difficulties, matrix multiplication in Matlab is very simple:<br />
<br />
&gt;&gt; A = [6 2; 3 5];<br />
<br />
&gt;&gt; B = [1 1 2 0;1 -1 0 0];<br />
<br />
&gt;&gt; C = A * B; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
8 4 1<br />
<br />
2 0 8 <br />
<br />
-2 6 0 <br />
<br />
Notice how the matrices are defined here through their rows. The <code>disp() </code>command displays the contents of the object referred to.<br />
<br />
It is less natural in Matlab to define matrices by columns - a typical example of how mathematics and computing have conflicts of notation. However, once columns <math>\mathbf{a}</math> and <math>\mathbf{b}</math> have been defined, the concatenation operation <math>\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]</math> collects the columns into a matrix:<br />
<br />
&gt;&gt; a = [6;2]; <br />
<br />
&gt;&gt; b = [3;5]; <br />
<br />
&gt;&gt; C = [a b]; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
6 3 <br />
<br />
2 5 <br />
<br />
Notice that the <code>disp(C)</code> command does not label the result that is printed out. Simply typing <code>C</code> would preface the output by <code>C =</code>.<br />
<br />
== Pre and Post Multiplication ==<br />
<br />
If <math>C=AB,</math> as above, say that <math>B</math> is ''pre-multiplied ''by <math>A</math> to get <math>C,</math> and that <math>A</math> is ''post-multiplied ''by <math>B</math> to get <math>C.</math><br />
<br />
This distinction between ''pre ''and ''post ''multiplication is important, in the following sense. Suppose that <math>A</math> and <math>B</math> are matrices such that the products <math>AB</math> and <math>BA</math> are both defined. If <math>A</math> is <math>m\times n,</math> <math>B</math> must have <math>n</math> rows for <math>AB</math> to be defined. For <math>BA</math> to be defined, <math>B</math> must have <math>m</math> columns to match the <math>m</math> rows in <math>A.</math> So, <math>AB</math> and <math>BA</math> are both defined if <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times m.</math><br />
<br />
Even when both products are defined, there is no reason for the two products coincide. The first thing to notice is that <math>AB</math> is a square, <math>m\times m,</math> matrix, whilst <math>BA</math> is a square, <math>n\times n,</math> matrix. Different sized matrices cannot be equal. To illustrate, use the matrices: <math>B_{2}=\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right],\ \ \ C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]:</math><br />
<br />
<math>\begin{aligned}<br />
B_{2}C & = & \left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]=\left[\begin{array}{rrr}<br />
27 & -3 & -15\\<br />
27 & 29 & -11\\<br />
-15 & -1 & 8<br />
\end{array}\right],\\<br />
CB_{2} & = & \left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
49 & -11\\<br />
31 & 15<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Even when <math>m=n,</math> so that <math>AB</math> and <math>BA</math> are both <math>m\times m</math> matrices, the products can differ: for example:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right],\\<br />
AB & = & \left[\begin{array}{cc}<br />
8 & 4\\<br />
8 & -2<br />
\end{array}\right],\ \ \ \ \ BA=\left[\begin{array}{cc}<br />
9 & 7\\<br />
3 & -3<br />
\end{array}\right].\end{aligned}</math><br />
<br />
In cases where <math>AB=BA,</math> the matrices <math>A</math> and <math>B</math> are said to ''commute''.<br />
<br />
== Transposition ==<br />
<br />
A column vector <math>\mathbf{x}</math> can be converted to a row vector <math>\mathbf{x}^{T}</math> by transposition: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \ \ \mathbf{x}^{T}=\left[\begin{array}{rrr}<br />
x_{1} & \ldots & x_{n}\end{array}\right].</math> Transposing <math>\mathbf{x}^{T}</math> as <math>\left(\mathbf{x}^{T}\right)^{T}</math> reproduces the original vector <math>\mathbf{x.}</math> How do these ideas carry over to matrices?<br />
<br />
If the <math>m\times n</math> matrix <math>A</math> can be written as <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right],</math> the transpose of <math>A,</math> <math>A^{T},</math> is defined as the matrix whose ''rows'' are <math>\mathbf{a}_{i}^{T}</math>: <math>A^{T}=\left[\begin{array}{c}<br />
\mathbf{a}_{1}^{T}\\<br />
\vdots\\<br />
\mathbf{a}_{n}^{T}<br />
\end{array}\right].</math> In terms of elements, if: <math>\mathbf{a}_{i}=\left[\begin{array}{c}<br />
a_{1i}\\<br />
\vdots\\<br />
a_{ni}<br />
\end{array}\right]</math> then: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right],\ \ \ \ \ A^{T}=\left[\begin{array}{rrrrr}<br />
a_{11} & \ldots & a_{i1} & \ldots & a_{m1}\\<br />
a_{12} & \ldots & a_{i2} & \ldots & a_{m2}\\<br />
\vdots & & \vdots & & \vdots\\<br />
a_{1n} & \ldots & a_{in} & \ldots & a_{mn}<br />
\end{array}\right].</math> One can see that the first column of <math>A</math> has now become the first row of <math>A^{T}.</math> Notice too that <math>A^{T}</math> is an <math>n\times m</math> matrix if <math>A</math> is an <math>m\times n</math> matrix.<br />
<br />
Transposing <math>A^{T}</math> takes the first column of <math>A^{T}</math> and writes it as a row, which coincides with the first row of <math>A.</math> The same argument applies to the other columns of <math>A^{T},</math> so that <math>\left(A^{T}\right)^{T}=A.</math><br />
<br />
=== The product rule for transposition ===<br />
<br />
This states that if <math>C=AB,</math> then <math>C^{T}=B^{T}A^{T}.</math><br />
<br />
How to see this? Consider the following example: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & b_{13} & b_{14}\\<br />
b_{21} & b_{22} & b_{23} & b_{24}\\<br />
b_{31} & b_{32} & b_{33} & b_{34}<br />
\end{array}\right]</math> where:<br />
<br />
<math>c_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=\sum_{k=1}^{3}a_{2k}b_{k3}.\label{eq:c23}</math><br />
<br />
One can see that: <math>B^{T}A^{T}=\left[\begin{array}{rrr}<br />
b_{11} & b_{21} & b_{31}\\<br />
b_{12} & b_{22} & b_{32}\\<br />
b_{13} & b_{23} & b_{33}\\<br />
b_{14} & b_{24} & b_{34}<br />
\end{array}\right]\left[\begin{array}{rr}<br />
a_{11} & a_{21}\\<br />
a_{12} & a_{22}\\<br />
a_{13} & a_{23}<br />
\end{array}\right]</math> and that the <math>\left(3,2\right)</math> element of this product is actually <math>c_{23}</math>: <math>b_{13}a_{21}+b_{23}a_{22}+b_{33}a_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=c_{23}.</math> In summation notation, we see that from <math>B^{T}A^{T}</math>: <math>c_{23}=\sum_{k=1}^{3}b_{k3}a_{2k},</math> where the position of the index of summation is due to the transposition. So, in summation notation, the calculation of <math>c_{23}</math> from <math>B^{T}A^{T}</math> equals that from equation (6).<br />
<br />
More generally, the <math>\left(i,j\right)</math> element of <math>AB</math>: <math>\sum_{k=1}^{3}a_{ik}b_{kj}</math> is the <math>\left(j,i\right)</math> element of <math>B^{T}A^{T}.</math> But this means that <math>B^{T}A^{T}</math> must be the transpose of <math>AB,</math> since the elements in the <math>i</math>th row of <math>AB</math> are being written in the <math>i</math>th column of <math>B^{T}A^{T}.</math><br />
<br />
This ''Product Rule for Transposition'' can be applied again to find the transpose <math>\left(C^{T}\right)^{T}</math> of <math>C^{T}</math>: <math>\left(C^{T}\right)^{T}=\left(B^{T}A^{T}\right)^{T}=\left(A^{T}\right)^{T}\left(B^{T}\right)^{T}=AB=C.</math><br />
<br />
= Special Types of Matrix =<br />
<br />
== The zero matrix ==<br />
<br />
The most obvious special type of matrix is one whose elements are all zeros. In typical element notation, the zero matrix is: <math>0=\left\Vert 0\right\Vert .</math> Since there is no indexing on the elements, it is not obvious what the dimension of this matrix is, Sometimes one writes <math>0_{mn}</math> to indicate a zero matrix of dimension <math>m\times n.</math> The same ideas apply to vectors whose elements are all zero.<br />
<br />
The effect of the zero matrix in any product that is defined is simple: <math>0A=0,\ \ \ \ \ B0=0.</math> This is easy to check using the across and down rule.<br />
<br />
== The identity or unit matrix ==<br />
<br />
Vectors of the form:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{r}<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }2\ \text{dimensions}\\<br />
\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }3\ \text{dimensions}\\<br />
\left[\begin{array}{r}<br />
1\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1\\<br />
0\\<br />
\vdots\\<br />
0<br />
\end{array}\right],\ldots,\left[\begin{array}{r}<br />
0\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }n\ \text{dimensions}\end{aligned}</math><br />
<br />
are called coordinate vectors. They are often given a characteristic notation, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> in <math>n</math> dimensions. When arranged as columns of a matrix in the natural order, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> a matrix with a characteristic pattern elements emerges, with a special notation:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{rr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2}\end{array}\right] & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=I_{2}\\<br />
\left[\begin{array}{rrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \mathbf{e}_{3}\end{array}\right] & = & \left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 1 & 0\\<br />
0 & 0 & 1<br />
\end{array}\right]=I_{3}\\<br />
\left[\begin{array}{rrrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \ldots & \mathbf{e}_{n}\end{array}\right] & = & \left[\begin{array}{rrrr}<br />
1 & 0 & \ldots & 0\\<br />
0 & 1 & \ldots & 0\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
0 & 0 & \ldots & 1<br />
\end{array}\right]=I_{n}.\end{aligned}</math><br />
<br />
The ''diagonal'' of this matrix is where the 1 elements are located, and every other element is zero.<br />
<br />
Consider the effect of <math>I_{2}</math> on the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> by both pre and post multiplication:<br />
<br />
<math>\begin{aligned}<br />
I_{2}A & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\\<br />
AI_{2} & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\end{aligned}</math><br />
<br />
as is easily checked by the across and down rule.<br />
<br />
Because any matrix is left unchanged by pre or post multiplication by an appropriately dimensioned <math>I_{n},</math> <math>I_{n}</math> is called an ''identity matrix of dimension ''<math>n.</math> Sometimes it is called a ''unit matrix of dimension ''<math>n.</math> Notice that <math>I_{n}</math> is necessarily a square matrix.<br />
<br />
== Diagonal matrices ==<br />
<br />
The identity matrix is an example of a diagonal matrix, a matrix whose elements are all zero except for those on the diagonal. Usually diagonal matrices are taken to be square, for example: <math>D=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 2 & 0\\<br />
0 & 0 & 3<br />
\end{array}\right].</math> They also produce characteristic effects when pre or post multiplying another matrix.<br />
<br />
Consider the diagonal matrix: <math>B=\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]</math> and the products <math>AB,</math> <math>BA</math> for <math>A</math> as defined in the previous section:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & -4\\<br />
6 & -10<br />
\end{array}\right],\\<br />
BA & = & \left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & 4\\<br />
-6 & -10<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Comparing the results, we can deduce that post multiplication by a diagonal matrix multiplies each column of <math>A</math> by the corresponding diagonal element, whereas pre multiplication multiplies each row by the corresponding diagonal element.<br />
<br />
== Symmetric matrices ==<br />
<br />
Symmetric matrices are matrices having the property that <math>A=A^{T}.</math> Notice that such matrices must be square, since if <math>A</math> is <math>m\times n,</math> <math>A^{T}</math> is <math>n\times m,</math> and to have equality of <math>A</math> and <math>A^{T},</math> they must have the same dimension, so that <math>m=n</math> is required.<br />
<br />
Suppose that <math>A</math> is a <math>3\times3</math> symmetric matrix, with typical element <math>a_{ij}</math>: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right],</math> so that: <math>A^{T}=\left[\begin{array}{rrr}<br />
a_{11} & a_{21} & a_{31}\\<br />
a_{12} & a_{22} & a_{32}\\<br />
a_{13} & a_{23} & a_{33}<br />
\end{array}\right].</math> Equality of matrices is defined as equality of all elements. This is fine on the diagonals, since <math>A</math> and <math>A^{T}</math> have the same diagonal elements. For the off diagonal elements, we end up with the requirements: <math>a_{12}=a_{21},\ \ \ a_{13}=a_{31},\ \ \ a_{23}=a_{32}</math> or more generally: <math>a_{ij}=a_{ji}\ \ \ \ \ \text{for}\ i\neq j.</math><br />
<br />
The effect of this conclusion is that in a symmetric matrix, the ’triangle’ of above diagonal elements coincides with the triangle of below diagonal elements. It is as if the upper triangle is folded over the diagonal to become the lower triangle.<br />
<br />
A simple example is: <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
2 & 1<br />
\end{array}\right].</math> A more complicated example uses the <math>2\times3</math> matrix <math>C</math>: <math>C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]</math> and calculates the <math>3\times3</math> matrix:<br />
<br />
<math>\begin{aligned}<br />
C^{T}C & = & \left[\begin{array}{cc}<br />
6 & 3\\<br />
2 & 5\\<br />
-3 & -1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{rrr}<br />
45 & 27 & -21\\<br />
27 & 29 & -11\\<br />
-21 & -11 & 10<br />
\end{array}\right],\end{aligned}</math><br />
<br />
which is clearly symmetric.<br />
<br />
This illustrates the general proposition that if <math>A</math> is an <math>m\times n</math> matrix, the product <math>A^{T}A</math> is a symmetric <math>n\times n</math> matrix. Proof? Compute the transpose of <math>A^{T}A</math> using the product rule for transposition: <math>\left(A^{T}A\right)^{T}=A^{T}\left(A^{T}\right)^{T}=A^{T}A.</math> Since <math>A^{T}A</math> is equal to its transpose, it must be a symmetric matrix. Such symmetric matrices appear frequently in econometrics.<br />
<br />
It should be clear that diagonal matrices are symmetric, since all their off-diagonal elements are equal (zero), and thence the identity matrix <math>I_{n}</math> is also symmetric.<br />
<br />
== The outer product ==<br />
<br />
The inner product of two <math>n\times1</math> vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>, <math>\mathbf{x}^{T}\mathbf{y}</math>, is automatically a <math>1\times1</math> quantity, a scalar, although it can be interpreted as a <math>1\times1</math> matrix, a matrix with a single element.<br />
<br />
Suppose one considered the product of <math>\mathbf{x}</math> with <math>\mathbf{x}^{T}.</math> Is this defined? If <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times r,</math> then the product <math>AB</math> is <math>m\times r.</math> Applying this logic to <math>\mathbf{xx}^{T},</math> this is <math>\left(n\times1\right)\left(1\times n\right),</math> so the resulting product ''is ''defined, and is an <math>n\times n</math>'' matrix'' - the ''outer product'' of <math>\mathbf{x}</math> and <math>\mathbf{x}^{T},</math> the word ’outer’ being used to distinguish from the inner product.<br />
<br />
How does the across and down rule work here? Suppose that: <math>\mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> Then: <math>\mathbf{xx}^{T}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right].</math> Here, there is <math>1</math> element in row one of the ’matrix’ <math>\mathbf{x,}</math> and <math>1</math> element in column one of the matrix <math>\mathbf{x}^{T},</math> so the across and down rule still works - it is just that there is only one product per row and column combination. So: <math>\mathbf{xx}^{T}=\left[\begin{array}{cc}<br />
36 & 18\\<br />
18 & 9<br />
\end{array}\right],</math> and it is obvious from this that <math>\mathbf{xx}^{T}</math> is a symmetric matrix.<br />
<br />
One can see that this outer product need not be restricted to vectors of the same dimension. If <math>\mathbf{x}</math> is <math>n\times1</math> and <math>\mathbf{y}</math> is <math>m\times1,</math> then: <math>\mathbf{xy}^{T}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
y_{1} & \ldots & y_{m}\end{array}\right]=\left[\begin{array}{rrrr}<br />
x_{1}y_{1} & x_{1}y_{2} & \ldots & x_{1}y_{m}\\<br />
x_{2}y_{1} & x_{2}y_{2} & \ldots & x_{2}y_{m}\\<br />
\\<br />
x_{n}y_{1} & x_{n}y_{2} & \ldots & x_{n}y_{m}<br />
\end{array}\right].</math> So, <math>\mathbf{xy}^{T}</math> is <math>n\times m,</math> and consists of rows which are <math>\mathbf{y}^{T}</math> multiplied by an element of the <math>\mathbf{x}</math> vector.<br />
<br />
Another interesting and useful example involves a vector with every element equal to <math>1</math>: <math>\mathbf{1}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right].</math> Sometimes this is written as <math>\mathbf{1}_{n}</math> to indicate an <math>n\times1</math> vector, and is called the ''sum vector''. Why? Consider the impact of <math>\mathbf{1}_{2}</math> on the <math>2\times1</math> vector <math>\mathbf{x}</math> used above: <math>\mathbf{1}_{2}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=9,</math> i.e. an inner product of <math>\mathbf{x}</math> with the sum vector is the sum of the elements of <math>\mathbf{x.}</math> Dividing through by the number of elements in <math>\mathbf{x}</math> produces the average of the elements of <math>\mathbf{x}</math> - i.e. the ’sample mean’ of the elements of <math>\mathbf{x.}</math><br />
<br />
The outer product of <math>\mathbf{x}</math> with <math>\mathbf{1}_{2}</math> is also interesting:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{1}_{2}\mathbf{x}^{T} & = & \left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 3\\<br />
6 & 3<br />
\end{array}\right],\\<br />
\mathbf{x1}_{2}^{T} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 6\\<br />
3 & 3<br />
\end{array}\right],\end{aligned}</math><br />
<br />
showing that pre multiplication of an <math>\mathbf{x}^{T}</math> by <math>\mathbf{1}</math> repeats <math>\mathbf{x}^{T}</math> as rows of the product, whilst post multiplication of <math>\mathbf{x}</math> by <math>\mathbf{1}^{T}</math> repeats <math>\mathbf{x}</math> as the columns of the product.<br />
<br />
Finally: <math>\mathbf{1}_{n}\mathbf{1}_{n}^{T}=\left[\begin{array}{rrr}<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1<br />
\end{array}\right],</math> an <math>n\times n</math> matrix with every element equal to <math>1.</math> This type of matrix also appears in econometrics!<br />
<br />
== Triangular matrices ==<br />
<br />
A square ''lower triangular ''matrix has all elements above the main diagonal equal to zero, whilst a square ''upper triangular ''matrix has all elements below the main diagonal equal to zero. A simple example of a lower triangular matrix is: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & 0 & 0\\<br />
a_{21} & a_{22} & 0\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right].</math> Clearly, for this matrix, <math>A^{T}</math> is an upper triangular matrix.<br />
<br />
One can adapt the definition to rectangular matrices: for example, if two arbitrary rows are added to <math>A,</math> so that it becomes <math>5\times3,</math> it would still be considered lower triangular. Equally, if, for example, the third column of <math>A</math> above is removed, <math>A</math> is still considered lower triangular.<br />
<br />
Often, we use ''unit ''triangular matrices, where the diagonal elements are all equal to <math>1:</math> e.g.:<br />
<br />
<math>\left[\begin{array}{rrr}<br />
1 & 2 & 0\\<br />
0 & 1 & 1\\<br />
0 & 0 & 1<br />
\end{array}\right].\label{eq:lt_matrix}</math><br />
<br />
== Partitioned matrices ==<br />
<br />
Sometimes, especially with big matrices, it is useful to organise the elements of the matrix into components which are themselves matrices, for example: <math>B=\left[\begin{array}{rrrr}<br />
1 & 2 & 0 & 0\\<br />
8 & 3 & 0 & 0\\<br />
0 & 0 & 7 & 4\\<br />
0 & 0 & 6 & 5<br />
\end{array}\right]</math> Here it would be reasonable to write: <math>B=\left[\begin{array}{cc}<br />
B_{11} & 0\\<br />
0 & B_{22}<br />
\end{array}\right],</math> where <math>B_{ii},i=1,2,</math> represent <math>2\times2</math> matrices. <math>B</math> is an example of a ''partitioned matrix'': that is, an <math>m\times n</math> matrix <math>A</math> say: <math>A=\left\Vert a_{ij}\right\Vert ,</math> where the elements of <math>A</math> are organised into ''sub-matrices''. An example might be:<br />
<br />
<math>A=\left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right],\label{eq:partition_a}</math><br />
<br />
where the ''sub - matrices'' in the first row block have <math>r</math> rows, and therefore <math>m-r</math> rows in the second row block. The column blocks might be defined by (for example) 3 columns in the first column block, 4 in the second and <math>n-7</math> in the third column block.<br />
<br />
Another simple example might be: <math>A=\left[\begin{array}{rrr}<br />
A_{1} & A_{2} & A_{3}\end{array}\right],\ \ \ \ \ \mathbf{x=}\left[\begin{array}{c}<br />
\mathbf{x}_{1}\\<br />
\mathbf{x}_{2}\\<br />
\mathbf{x}_{3}<br />
\end{array}\right],</math> where <math>A</math> and therefore <math>A_{1},A_{2},A_{3}</math> have <math>m</math> rows, <math>A_{1}</math> has <math>n_{1}</math> columns, <math>A_{2}</math> has <math>n_{2}</math> columns, <math>A_{3}</math> has <math>n_{3}</math> columns. The ''subvectors'' in <math>\mathbf{x}</math> must have <math>n_{1},n_{2}</math> and <math>n_{3}</math> rows respectively, for the product <math>A\mathbf{x}</math> to exist.<br />
<br />
Suppose that <math>n_{1}+n_{2}+n_{3}=n,</math> so that <math>A</math> is <math>m\times n.</math> The <math>i</math>th element of <math>A\mathbf{x}</math> is: <math>\sum_{i=1}^{n}a_{ij}x_{j}</math> but the summation can be broken up into the first <math>n_{1}</math> terms: <math>\sum_{i=1}^{n_{1}}a_{ij}x_{j},</math> the next <math>n_{2}</math> terms: <math>\sum_{i=n_{1}+1}^{n_{1}+n_{2}}a_{ij}x_{j},</math> and the next <math>n_{3}</math> terms; <math>\sum_{i=n_{1}+n_{2}+1}^{n}a_{ij}x_{j}.</math><br />
<br />
The point about the use of partitioned matrices is that the product <math>A\mathbf{x}</math> can be represented as: <math>A\mathbf{x}=A_{1}\mathbf{x}_{1}+A_{2}\mathbf{x}_{2}+A\mathbf{x}_{3}</math> by applying the across and down rule to the submatrices and the subvectors, a much simpler representation than the use of summations.<br />
<br />
Each of the components is a conformable matrix-vector product: this is essential in any use of partitioned matrices to represent some matrix product. For example, using <math>A</math> from equation (8) and <math>B</math> as: <math>B=\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right],</math> it is easy to write:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right]\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{r}<br />
A_{11}B_{11}+A_{12}B_{21}+A_{13}B_{31}\\<br />
A_{21}B_{11}+A_{22}B_{21}+A_{23}B_{31}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
But, what are the row dimensions for the submatrices in <math>B?</math> What are the possible column dimensions for the submatrices in <math>B?</math><br />
<br />
= Matrices, vectors and econometrics =<br />
<br />
The data on weights and heights for 12 students in the data matrix: <math>D=\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]</math> would seem to be ideally suited for fitting a two variable regression model: <math>y_{i}=\alpha+\beta x_{i}+u_{i},\;\;\;\;\; i=1,...,12.</math> Here, the first column of <math>D</math> contains all the weight data, the data on the dependent variable <math>y_{i},</math> and so should be labelled <math>\mathbf{y.}</math> The second column of <math>D</math> contains all the data on the explanatory variable height, in the vector <math>\mathbf{x}</math> say, so that: <math>D=\left[\begin{array}{rr}<br />
\mathbf{y} & \mathbf{x}\end{array}\right].</math><br />
<br />
If we define a <math>12\times1</math> vector with every element <math>1</math>: <math>\mathbf{1}_{12}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right],</math> and a <math>12\times1</math> vector <math>\mathbf{u}</math> to contain the error terms; <math>\mathbf{u}=\left[\begin{array}{c}<br />
u_{1}\\<br />
\vdots\\<br />
u_{12}<br />
\end{array}\right]</math> the regression model can be written in terms of the three data vectors <math>\mathbf{y,1}_{12}</math> and <math>\mathbf{x}</math> as: <math>\mathbf{y}=\mathbf{1}_{12}\alpha+\mathbf{x}\beta+\mathbf{u.}</math> To see this, think of the <math>i</math>th elements of the vectors on the left and right hand sides.<br />
<br />
The standard next step is then to combine the data vectors for the explanatory variables into a matrix: <math>X=\left[\begin{array}{rr}<br />
\mathbf{1}_{12} & \mathbf{x}\end{array}\right],</math> and then define a <math>2\times1</math> vector <math>\boldsymbol{\delta}</math> to contain the parameters <math>\alpha,\beta</math> as: <math>\boldsymbol{\delta}=\left[\begin{array}{r}<br />
\alpha\\<br />
\beta<br />
\end{array}\right]</math> to give the data matrix representation of the regression model as: <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
<br />
For the purposes of developing the theory of regression, this is the most convenient form of the regression model. It can represent regression models with any number of explanatory variables, and thus any number of parameters. The obvious point is that a knowledge of vector and matrix operations is needed to use and understand this form.<br />
<br />
We shall see later that there are two particular matrix and vector quantities associated with a regression model. The first is the matrix <math>X^{T}X,</math> and the second the vector <math>X^{T}\mathbf{y.}</math> The following Matlab code snippet provides the numerical values of these quantities for the weight data:<br />
<br />
&gt;&gt; dset = load(’weights.mat’); <br />
<br />
&gt;&gt; xtx = dset.X’ * dset.X; <br />
<br />
&gt;&gt; xty = dset.X’ * dset.y; <br />
<br />
&gt;&gt; disp(xtx) <br />
<br />
12 802<br />
<br />
802 53792<br />
<br />
&gt;&gt; disp(xty)<br />
<br />
1850<br />
<br />
124528<br />
<br />
Hand calculation is of course possible, but not recommended.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=LNotes&diff=3026LNotes2013-09-10T14:10:16Z<p>LG: </p>
<hr />
<div>= Matrices =<br />
<br />
In the PreSession Maths course, a matrix was defined as follows:<br />
<br />
<blockquote>A matrix is a rectangular array of numbers enclosed in parentheses, conventionally denoted by a capital letter. The number of rows (say <math>m</math>) and<br />
<br />
the number of columns (say <math>n</math>) determine the order of the matrix (<math>m</math> <math>\times</math> <math>n</math>).<br />
</blockquote><br />
Two examples were given:<br />
<br />
<math>\begin{aligned}<br />
P & =\left[\begin{array}{rrr}<br />
2 & 3 & 4\\<br />
3 & 1 & 5<br />
\end{array}\right],\ \ \ Q=\left[\begin{array}{rr}<br />
2 & 3\\<br />
4 & 3\\<br />
1 & 5<br />
\end{array}\right],\end{aligned}</math><br />
<br />
matrices of dimensions <math>2\times3</math> and <math>3\times2</math> respectively.<br />
<br />
Why study matrices for econometrics? Basically because a data set of several variables, e.g. on the weights and heights of 12 students, can be thought of as a matrix:<br />
<br />
<math>\begin{aligned}<br />
D & =\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]\end{aligned}</math><br />
<br />
The properties of matrices can then be used to facilitate answering all the usual questions of econometrics - list not given here!<br />
<br />
Calculations with matrices with explicit numerical elements, as in the examples above is called matrix ''arithmetic''. Matrix ''algebra'' is the algebra of matrices where the elements are not made explicit: this is what is really required for econometrics, as we shall see.<br />
<br />
As an example of this, a <math>2\times3</math> matrix might be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{ccc}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\end{aligned}</math><br />
<br />
and would equal <math>P</math> above if the collection of <math>a_{ij}</math> were given appropriate numerical values.<br />
<br />
A general <math>m\times n</math> matrix <math>A</math> can be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
There is also a ''typical element ''notation for matrices:<br />
<br />
<math>\begin{aligned}<br />
A & =\left\Vert a_{ij}\right\Vert ,\ \ \ \ \ i=1,...,m,j=1,...,n,\end{aligned}</math><br />
<br />
so that <math>a_{ij}</math> is the element at the intersection of the <math>i</math>th row and <math>j</math>th column in <math>A.</math><br />
<br />
When <math>m\neq n,</math> <math>A</math> is a ''rectangular ''matrix; when <math>m=n,</math> <math>A</math> is <math>m\times m</math> or <math>n\times n,</math> and <math>A</math> is a square matrix, having the same number of rows or columns.<br />
<br />
== Rows, columns and vectors ==<br />
<br />
Clearly, there is no reason why <math>m</math> or <math>n</math> cannot equal 1: so, an <math>m\times n</math> matrix with <math>n=1,</math> i.e. with one column, is usually called a column vector. Similarly, a matrix with one row is a row vector.<br />
<br />
There are a lot of advantages to thinking of matrices as collections of row or column vectors, as we shall see. As an example, define the <math>2\times1</math> column vectors:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a} & =\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\mathbf{,\ \ \ b}=\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
and arrange as the columns of the <math>2\times2</math> matrix<br />
<br />
<math>A=\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\label{eq:axy}</math><br />
<br />
In general, a column vector <math>\mathbf{x}</math> with <math>n</math> elements can be written as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\end{aligned}</math><br />
<br />
What happens when both <math>m</math> and <math>n</math> are equal to <math>1?</math> Then, <math>A</math> is a <math>1\times1</math> matrix, but it is also considered to be a real number, or ''scalar'' in the language of linear algebra:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[a_{11}\right]=a_{11}.\end{aligned}</math><br />
<br />
This is perhaps a little odd, but turns out to be a useful convention in a number of situations.<br />
<br />
== Transposition of vectors ==<br />
<br />
The ''rows'' of the matrix <math>A</math> in equation (1) can be seen as elements of column vectors, say:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c} & =\left[\begin{array}{r}<br />
6\\<br />
2<br />
\end{array}\right],\ \ \ \boldsymbol{d}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
This representation of row vectors as column vectors is a bit clumsy, so some transformation which converts a column vector into a row vector, and vice versa would be useful. The process of converting a column vector <math>\mathbf{a}</math> into a row vector is called ''transposition, ''and the transposed version of <math>\mathbf{c}</math> is denoted:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c}^{T} & =\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\end{aligned}</math><br />
<br />
the <math>^{T}</math> superscript denoting transposition. In practice, a prime, <math>^{\prime},</math> is used instead of <math>^{T}.</math> However, whilst the prime is much simpler to write than the <math>^{T}</math> sign, it is also much easier to lose track of in writing out long or complicated expressions. So, it is best initially to use <math>^{T}</math> to denote transposition rather than the prime <math>^{\prime}.</math><br />
<br />
<math>A</math> can then be written via its rows as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{r}<br />
\mathbf{c}^{T}\\<br />
\boldsymbol{d}^{T}<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
The same ideas can be applied to the matrices <math>P</math> and <math>Q.</math><br />
<br />
= Operations with matrices =<br />
<br />
== Addition, subtraction and scalar multiplication ==<br />
<br />
For vectors, addition and subtraction are defined only for vectors of the same dimensions. If:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right],\\<br />
\mathbf{x+y} & =\left[\begin{array}{c}<br />
x_{1}+y_{1}\\<br />
\vdots\\<br />
x_{n}+y_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{x-y}=\left[\begin{array}{c}<br />
x_{1}-y_{1}\\<br />
\vdots\\<br />
x_{n}-y_{n}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Clearly, the addition or subtraction operation is ''elementwise. ''If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> have different dimensions, there will be some elements left over once all the elements of the smaller dimensioned vector have been used up.<br />
<br />
Another operation is ''scalar multiplication'': if <math>\lambda</math> is a real number or scalar, the product <math>\lambda\mathbf{x}</math> is defined as: <math>\lambda\mathbf{x}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right],</math> so that every element of <math>\mathbf{x}</math> is multiplied by the same scalar <math>\lambda.</math><br />
<br />
The two types of operation can be combined into the ''linear combination'' of vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>: <math>\lambda\mathbf{x}+\mu\mathbf{y}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right]+\left[\begin{array}{c}<br />
\mu y_{1}\\<br />
\vdots\\<br />
\mu y_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\lambda x_{1}+\mu y_{1}\\<br />
\vdots\\<br />
\lambda x_{n}+\mu y_{n}<br />
\end{array}\right].</math> Equally, one can define the linear combination of vectors <math>\mathbf{x,y,}\ldots,\mathbf{z}</math> by scalars <math>\lambda,\mu,\ldots,\nu</math> as: <math>\lambda\mathbf{x}+\mu\mathbf{y}+\ldots+\nu\mathbf{z}</math> with typical element: <math>\lambda x_{i}+\mu y_{i}+\ldots+\nu z_{i},</math> provided that all the vectors have the same dimension.<br />
<br />
For matrices, these ideas carry over immediately: apply to each column of the matrices involved. For example, if <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right]</math> and <math>B=\left[\begin{array}{rrr}<br />
\mathbf{b}_{1} & \ldots & \mathbf{b}_{n}\end{array}\right],</math> both <math>m\times n,</math> then addition and subtraction are defined elementwise, as for vectors:<br />
<br />
<math>\begin{aligned}<br />
A+B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}+\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}+\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}+b_{ij}\right\Vert ,\\<br />
A-B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}-\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}-\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}-b_{ij}\right\Vert .\end{aligned}</math><br />
<br />
Scalar multiplication of <math>A</math> by <math>\lambda</math> involves multiplying every column vector of <math>A</math> by <math>\lambda,</math> and therefore multiplying every element of <math>A</math> by <math>\lambda</math>: <math>\lambda A=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1} & \ldots & \lambda\mathbf{a}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}\right\Vert .</math> With the same idea for <math>B,</math> the linear combination of <math>A</math> and <math>B</math> by <math>\lambda</math> and <math>\mu</math> is: <math>\lambda A+\mu B=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1}+\mu\mathbf{b}_{1} & \ldots & \lambda\mathbf{a}_{n}+\mu\mathbf{b}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}+\mu b_{ij}\right\Vert .</math><br />
<br />
For example, consider the matrices: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right]</math> with <math>\lambda=1,</math> <math>\mu=-2:</math> then:<br />
<br />
<math>\begin{aligned}<br />
\lambda A+\mu B & = & A-2B\\<br />
& = & \left[\begin{array}{cc}<br />
4 & 0\\<br />
1 & 7<br />
\end{array}\right].\end{aligned}</math><br />
<br />
== Matrix - vector products ==<br />
<br />
=== Inner product ===<br />
<br />
The simplest form of a matrix vector product is the case where <math>A</math> consists of one row, so that <math>A</math> is <math>1\times n</math>: <math>A=\mathbf{a}^{T}=\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right].</math> If <math>\mathbf{x}</math> is an <math>n\times1</math> vector: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],</math> the product <math>A\mathbf{x}=\mathbf{a}^{T}\mathbf{x}</math> is called the ''inner product'' and is defined as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a}^{T}\mathbf{x} & =a_{1}x_{1}+\ldots+a_{n}x_{n}.\end{aligned}</math><br />
<br />
One can see that the definition amounts to multiplying corresponding elements in <math>\mathbf{a}</math> and <math>\mathbf{x,}</math> and adding up the resultant products. Writing: <math>\mathbf{a}^{T}\mathbf{x=}\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=a_{1}x_{1}+\ldots+a_{n}x_{n}</math> motivates the familiar description of the ''across and down rule ''for this product: ''across and down'' is the ’multiply corresponding elements’ part of the definition.<br />
<br />
Notice that the result of the inner product is a real number, for example: <math>\mathbf{c}^{T}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{c}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=36+6=42.</math><br />
<br />
In general, in the product <math>\mathbf{a}^{T}\mathbf{x,}</math> <math>\mathbf{a}</math> and <math>\mathbf{x}</math> must have the same number of elements, <math>n</math> say, for the product to be defined. If <math>\mathbf{a}</math> and <math>\mathbf{x}</math> had different numbers of elements, there would be some elements of <math>\mathbf{a}</math> or <math>\mathbf{x}</math> left over or not used in the product: e.g.: <math>\mathbf{b}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{x=}\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> When the inner product of two vectors is defined, the vectors are said to be ''conformable''.<br />
<br />
== Orthogonality ==<br />
<br />
Two vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math> with the property that <math>\mathbf{x}^{T}\mathbf{y}=0</math> are said to be orthogonal to each other. For example, if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
-1\\<br />
1<br />
\end{array}\right],</math> it is clear that <math>\mathbf{x}^{T}\mathbf{y}=0.</math> This seems a rather innocuous definition, and yet the idea of orthogonality turns out to be extremely important in econometrics.<br />
<br />
If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> are thought of as points in <math>R^{2},</math> and arrows are drawn from the origin to <math>\mathbf{x}</math> and to <math>\mathbf{y,}</math> then the two arrows are perpendicular to each other - see Figure [orthy<sub>e</sub>xample]. If <math>\mathbf{y}</math> were defined as: <math>\mathbf{y}=\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right],</math> the position of the <math>\mathbf{y}</math> vector and the corresponding arrow would change, but the perpendicularity property would still hold.<br />
<br />
Figure 1:<br />
[[Media:orthy_example.png]]<br />
[[File:orthy_example.png]]<br />
<br />
<br />
=== Matrix - vector products ===<br />
<br />
Since the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> has two rows, now denoted <math>\boldsymbol{\alpha}_{1}^{T}</math> and <math>\boldsymbol{\alpha}_{2}^{T},</math> there are two possible inner products with the vector:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]:\\<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x} & = & 42,\ \ \ \ \ \boldsymbol{\alpha}_{2}^{T}\mathbf{x}=33.\end{aligned}</math><br />
<br />
Assembling the two inner product values into a <math>2\times1</math> vector defines the product of the matrix <math>A</math> with the vector <math>\mathbf{x}</math>: <math>A\mathbf{x}=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x}\\<br />
\boldsymbol{\alpha}_{2}^{T}\mathbf{x}<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right].</math> Focussing only on the part: <math>\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right],</math> one can see that each element of <math>A\mathbf{x}</math> is obtained from an ''across and down'' argument.<br />
<br />
Sometimes this product is described as forming a ''linear combination ''of the columns of <math>A</math> using the scalar elements in <math>\mathbf{x}</math>: <math>A\mathbf{x}=6\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]+3\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].</math> More generally, if:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right],\ \ \ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
\lambda\\<br />
\mu<br />
\end{array}\right],\\<br />
A\mathbf{x} & = & \lambda\mathbf{a}+\mu\mathbf{b.}\end{aligned}</math><br />
<br />
The general version of these ideas for an <math>m\times n</math> matrix <math>A</math>: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]=\left[\begin{array}{rrrr}<br />
\mathbf{a}_{1} & \mathbf{a}_{2} & \ldots & \mathbf{a}_{n}\end{array}\right].</math> is straightforward. If <math>\mathbf{x}</math> is an <math>n\times1</math> vector, then the vector <math>A\mathbf{x}</math> is, by the ''across and down'' rule:<br />
<br />
<math>A\mathbf{x}=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
x_{2}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
a_{11}x_{1}+\ldots+a_{1n}x_{n}\\<br />
a_{21}x_{1}+\ldots+a_{2n}x_{n}\\<br />
\vdots\\<br />
a_{m1}x_{1}+\ldots+a_{mn}x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\sum\limits _{j=1}^{n}a_{1j}x_{j}\\<br />
\sum\limits _{j=1}^{n}a_{2j}x_{j}\\<br />
\vdots\\<br />
\sum\limits _{j=1}^{n}a_{mj}x_{j}<br />
\end{array}\right],\label{eq:ab}</math><br />
<br />
so that the typical element, the <math>i</math>th, is <math>\sum\limits _{j=1}^{n}a_{ij}x_{j}.</math> Equally, <math>A\mathbf{x}</math> is the linear combination <math>\mathbf{a}_{1}x_{1}+\ldots+\mathbf{a}_{n}x_{n}</math> of the columns of <math>A.</math><br />
<br />
== Matrix - matrix products ==<br />
<br />
Suppose that <math>A</math> is <math>m\times n,</math> with columns <math>\mathbf{a}_{1},\ldots,\mathbf{a}_{n},</math> and <math>B</math> is <math>n\times r,</math> with columns <math>\mathbf{b}_{1},\ldots,\mathbf{b}_{r}.</math> Clearly, each product <math>A\mathbf{b}_{1},...,A\mathbf{b}_{r}</math> exists, and is <math>m\times1.</math> These products can be arranged as the columns of a matrix as <math>\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]</math> and this matrix is ''defined'' to be the product <math>C</math> of the matrices <math>A</math> and <math>B</math>: <math>C=\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]=AB.</math> By construction, this must be an <math>m\times r</math> matrix, since each column is <math>m\times1</math> and there are <math>r</math> columns.<br />
<br />
This is not the usual presentation of the definition of the product of two matrices, which relies on the ''across and down rule'' mentioned earlier, and focusses on the elements of each matrix <math>A</math> and <math>B.</math> Set:<br />
<br />
<math>\begin{aligned}<br />
B & = & \left[\begin{array}{rrrr}<br />
\mathbf{b}_{1} & \mathbf{b}_{2} & \ldots & \mathbf{b}_{r}\end{array}\right]\text{\ \ \ \ \ \ \ (by columns)}\\<br />
& = & \left\Vert b_{ik}\right\Vert ,\ \ \ \ \ i=1,...,n,k=1,...,r\text{ \ \ \ \ \ \ (typical element)}\\<br />
& = & \left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2r}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nr}<br />
\end{array}\right]\text{\ \ \ \ \ \ \ (the array)}\end{aligned}</math><br />
<br />
What does the typical element of the <math>m\times r</math> matrix <math>C</math> look like? Start with the <math>k</math>th column of <math>C,</math> which is <math>A\mathbf{b}_{k}.</math> The <math>i</math>th element in <math>A\mathbf{b}_{k}</math> is, from equation (2), the inner product of the elements of the <math>i</math>th row in <math>A</math>: <math>\left[\begin{array}{rrrr}<br />
a_{i1} & a_{i2} & \ldots & a_{in}\end{array}\right],</math> with the elements of <math>\mathbf{b}_{k},</math> so that the inner product is: <math>a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math><br />
<br />
So, the <math>ik</math>th element of <math>C</math> is: <math>c_{ik}=a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math> We can see this arising from an ''across and down'' calculation by writing:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\label{eq:c_ab}\\<br />
& = & \left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{rrrrrr}<br />
b_{11} & b_{12} & \ldots & b_{1k} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2k} & \ldots & b_{2r}\\<br />
\vdots & \vdots & & \vdots & & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nk} & \ldots & b_{nr}<br />
\end{array}\right]\\<br />
& = & \left\Vert \sum_{j=1}^{n}a_{ij}b_{jk}\right\Vert .\end{aligned}</math><br />
<br />
These ideas are simple, but a little tedious. Numerical examples are equally tedious! As an example, using: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],</math> we can find the matrix <math>B</math> such that<br />
<br />
# the first column of <math>AB</math> adds together the columns of <math>A,</math><br />
# the second column is the difference of the first and second columns of <math>A,</math><br />
# the third column is <math>2\times</math> the first column of <math>A,</math><br />
# the fourth column is zero.<br />
<br />
It is easy to check that <math>B</math> is: <math>B=\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]</math> and that:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\\<br />
& = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{cccc}<br />
8 & 4 & 12 & 0\\<br />
8 & -2 & 6 & 0<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Arithmetic calculations of matrix products almost always use the elementwise across and down formula. However, there are many situations in econometrics where algebraic rather than arithmetic arguments are required. In these cases, the viewpoint of matrix multiplication as linear combinations of columns is much more powerful.<br />
<br />
Clearly one can give many more examples of different dimensions and complexities - but the same basic rules apply. To multiply two matrices <math>A</math> and <math>B</math> together, the number of columns in <math>A</math> must match the number of rows in <math>B</math> - this is ''conformability'' in action again. The resulting product will have number of rows equal to the number in <math>A</math> and number of columns equal to the number in <math>B.</math><br />
<br />
If this conformability rule does not hold, then the product of <math>A</math> and <math>B</math> is not defined.<br />
<br />
== Matlab ==<br />
<br />
One should also say that as the dimensions of the matrices increases, so the tediousness of the calculations increase. The solution to this for numerical calculation is to appeal to the computer. Programs like Matlab and Excel (and a number of others, some of them free) resolve this difficulty easily.<br />
<br />
In Matlab, symbols for row or column vectors do not need any particular differentiation: they are distinguished by how they are defined. For example, the following Matlab commands define <code>rowvec </code>as a <math>1\times4</math> vector, and <code>colvec</code> as a <math>4\times1</math> vector, then display the contents of these variables, and do a calculation:<br />
<br />
&gt;&gt; rowvec = [1 2 3 4]; <br />
<br />
&gt;&gt; colvec = [1;2;3;4]; <br />
<br />
&gt;&gt; rowvec<br />
<br />
rowvec =<br />
<br />
1 2 3 4<br />
<br />
&gt;&gt; colvec<br />
<br />
colvec =<br />
<br />
1 2 3 4 <br />
<br />
&gt;&gt; rowvec*colvec<br />
<br />
ans =<br />
<br />
30 <br />
<br />
So, the semi-colon indicates the end of a row in a matrix or vector; it can be replaced by a carriage return. Notice the difference in how a row vector and a column vector is defined. One can see that the product <code>rowvec*colvec</code> is well defined, just because <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector.<br />
<br />
Matlab also allows elementwise multiplication of two vectors using the <math>\centerdot\ast</math> operator: if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
x_{1}\\<br />
x_{2}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
y_{1}\\<br />
y_{2}<br />
\end{array}\right],</math> then: <math>\mathbf{x}\centerdot\ast\mathbf{y}=\left[\begin{array}{r}<br />
x_{1}y_{1}\\<br />
x_{2}y_{2}<br />
\end{array}\right]</math> and one can see that the inner product of <math>\mathbf{x}</math> and <math>\mathbf{y}</math> can be obtained as the sum of the elements of <math>\mathbf{x}</math> and <math>\mathbf{y.}</math> In Matlab, this would be obtained as: <math>\text{sum}\left(\mathbf{x}\centerdot\ast\mathbf{y}\right).</math><br />
<br />
In the example above, this calculation fails since <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector:<br />
<br />
&gt;&gt; sum(rowvec .* colvec) ??? <br />
<br />
Error using ==&gt; times Matrix dimensions must agree. <br />
<br />
For this to work, <code>rowvec</code> would have to be transposed as <code>rowvec’</code>, so that transposition in Matlab is very natural.<br />
<br />
Allowing for such difficulties, matrix multiplication in Matlab is very simple:<br />
<br />
&gt;&gt; A = [6 2; 3 5];<br />
<br />
&gt;&gt; B = [1 1 2 0;1 -1 0 0];<br />
<br />
&gt;&gt; C = A * B; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
8 4 1<br />
<br />
2 0 8 <br />
<br />
-2 6 0 <br />
<br />
Notice how the matrices are defined here through their rows. The <code>disp() </code>command displays the contents of the object referred to.<br />
<br />
It is less natural in Matlab to define matrices by columns - a typical example of how mathematics and computing have conflicts of notation. However, once columns <math>\mathbf{a}</math> and <math>\mathbf{b}</math> have been defined, the concatenation operation <math>\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]</math> collects the columns into a matrix:<br />
<br />
&gt;&gt; a = [6;2]; <br />
<br />
&gt;&gt; b = [3;5]; <br />
<br />
&gt;&gt; C = [a b]; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
6 3 <br />
<br />
2 5 <br />
<br />
Notice that the <code>disp(C)</code> command does not label the result that is printed out. Simply typing <code>C</code> would preface the output by <code>C =</code>.<br />
<br />
== Pre and Post Multiplication ==<br />
<br />
If <math>C=AB,</math> as above, say that <math>B</math> is ''pre-multiplied ''by <math>A</math> to get <math>C,</math> and that <math>A</math> is ''post-multiplied ''by <math>B</math> to get <math>C.</math><br />
<br />
This distinction between ''pre ''and ''post ''multiplication is important, in the following sense. Suppose that <math>A</math> and <math>B</math> are matrices such that the products <math>AB</math> and <math>BA</math> are both defined. If <math>A</math> is <math>m\times n,</math> <math>B</math> must have <math>n</math> rows for <math>AB</math> to be defined. For <math>BA</math> to be defined, <math>B</math> must have <math>m</math> columns to match the <math>m</math> rows in <math>A.</math> So, <math>AB</math> and <math>BA</math> are both defined if <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times m.</math><br />
<br />
Even when both products are defined, there is no reason for the two products coincide. The first thing to notice is that <math>AB</math> is a square, <math>m\times m,</math> matrix, whilst <math>BA</math> is a square, <math>n\times n,</math> matrix. Different sized matrices cannot be equal. To illustrate, use the matrices: <math>B_{2}=\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right],\ \ \ C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]:</math><br />
<br />
<math>\begin{aligned}<br />
B_{2}C & = & \left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]=\left[\begin{array}{rrr}<br />
27 & -3 & -15\\<br />
27 & 29 & -11\\<br />
-15 & -1 & 8<br />
\end{array}\right],\\<br />
CB_{2} & = & \left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
49 & -11\\<br />
31 & 15<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Even when <math>m=n,</math> so that <math>AB</math> and <math>BA</math> are both <math>m\times m</math> matrices, the products can differ: for example:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right],\\<br />
AB & = & \left[\begin{array}{cc}<br />
8 & 4\\<br />
8 & -2<br />
\end{array}\right],\ \ \ \ \ BA=\left[\begin{array}{cc}<br />
9 & 7\\<br />
3 & -3<br />
\end{array}\right].\end{aligned}</math><br />
<br />
In cases where <math>AB=BA,</math> the matrices <math>A</math> and <math>B</math> are said to ''commute''.<br />
<br />
== Transposition ==<br />
<br />
A column vector <math>\mathbf{x}</math> can be converted to a row vector <math>\mathbf{x}^{T}</math> by transposition: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \ \ \mathbf{x}^{T}=\left[\begin{array}{rrr}<br />
x_{1} & \ldots & x_{n}\end{array}\right].</math> Transposing <math>\mathbf{x}^{T}</math> as <math>\left(\mathbf{x}^{T}\right)^{T}</math> reproduces the original vector <math>\mathbf{x.}</math> How do these ideas carry over to matrices?<br />
<br />
If the <math>m\times n</math> matrix <math>A</math> can be written as <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right],</math> the transpose of <math>A,</math> <math>A^{T},</math> is defined as the matrix whose ''rows'' are <math>\mathbf{a}_{i}^{T}</math>: <math>A^{T}=\left[\begin{array}{c}<br />
\mathbf{a}_{1}^{T}\\<br />
\vdots\\<br />
\mathbf{a}_{n}^{T}<br />
\end{array}\right].</math> In terms of elements, if: <math>\mathbf{a}_{i}=\left[\begin{array}{c}<br />
a_{1i}\\<br />
\vdots\\<br />
a_{ni}<br />
\end{array}\right]</math> then: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right],\ \ \ \ \ A^{T}=\left[\begin{array}{rrrrr}<br />
a_{11} & \ldots & a_{i1} & \ldots & a_{m1}\\<br />
a_{12} & \ldots & a_{i2} & \ldots & a_{m2}\\<br />
\vdots & & \vdots & & \vdots\\<br />
a_{1n} & \ldots & a_{in} & \ldots & a_{mn}<br />
\end{array}\right].</math> One can see that the first column of <math>A</math> has now become the first row of <math>A^{T}.</math> Notice too that <math>A^{T}</math> is an <math>n\times m</math> matrix if <math>A</math> is an <math>m\times n</math> matrix.<br />
<br />
Transposing <math>A^{T}</math> takes the first column of <math>A^{T}</math> and writes it as a row, which coincides with the first row of <math>A.</math> The same argument applies to the other columns of <math>A^{T},</math> so that <math>\left(A^{T}\right)^{T}=A.</math><br />
<br />
=== The product rule for transposition ===<br />
<br />
This states that if <math>C=AB,</math> then <math>C^{T}=B^{T}A^{T}.</math><br />
<br />
How to see this? Consider the following example: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & b_{13} & b_{14}\\<br />
b_{21} & b_{22} & b_{23} & b_{24}\\<br />
b_{31} & b_{32} & b_{33} & b_{34}<br />
\end{array}\right]</math> where:<br />
<br />
<math>c_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=\sum_{k=1}^{3}a_{2k}b_{k3}.\label{eq:c23}</math><br />
<br />
One can see that: <math>B^{T}A^{T}=\left[\begin{array}{rrr}<br />
b_{11} & b_{21} & b_{31}\\<br />
b_{12} & b_{22} & b_{32}\\<br />
b_{13} & b_{23} & b_{33}\\<br />
b_{14} & b_{24} & b_{34}<br />
\end{array}\right]\left[\begin{array}{rr}<br />
a_{11} & a_{21}\\<br />
a_{12} & a_{22}\\<br />
a_{13} & a_{23}<br />
\end{array}\right]</math> and that the <math>\left(3,2\right)</math> element of this product is actually <math>c_{23}</math>: <math>b_{13}a_{21}+b_{23}a_{22}+b_{33}a_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=c_{23}.</math> In summation notation, we see that from <math>B^{T}A^{T}</math>: <math>c_{23}=\sum_{k=1}^{3}b_{k3}a_{2k},</math> where the position of the index of summation is due to the transposition. So, in summation notation, the calculation of <math>c_{23}</math> from <math>B^{T}A^{T}</math> equals that from equation (6).<br />
<br />
More generally, the <math>\left(i,j\right)</math> element of <math>AB</math>: <math>\sum_{k=1}^{3}a_{ik}b_{kj}</math> is the <math>\left(j,i\right)</math> element of <math>B^{T}A^{T}.</math> But this means that <math>B^{T}A^{T}</math> must be the transpose of <math>AB,</math> since the elements in the <math>i</math>th row of <math>AB</math> are being written in the <math>i</math>th column of <math>B^{T}A^{T}.</math><br />
<br />
This ''Product Rule for Transposition'' can be applied again to find the transpose <math>\left(C^{T}\right)^{T}</math> of <math>C^{T}</math>: <math>\left(C^{T}\right)^{T}=\left(B^{T}A^{T}\right)^{T}=\left(A^{T}\right)^{T}\left(B^{T}\right)^{T}=AB=C.</math><br />
<br />
= Special Types of Matrix =<br />
<br />
== The zero matrix ==<br />
<br />
The most obvious special type of matrix is one whose elements are all zeros. In typical element notation, the zero matrix is: <math>0=\left\Vert 0\right\Vert .</math> Since there is no indexing on the elements, it is not obvious what the dimension of this matrix is, Sometimes one writes <math>0_{mn}</math> to indicate a zero matrix of dimension <math>m\times n.</math> The same ideas apply to vectors whose elements are all zero.<br />
<br />
The effect of the zero matrix in any product that is defined is simple: <math>0A=0,\ \ \ \ \ B0=0.</math> This is easy to check using the across and down rule.<br />
<br />
== The identity or unit matrix ==<br />
<br />
Vectors of the form:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{r}<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }2\ \text{dimensions}\\<br />
\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }3\ \text{dimensions}\\<br />
\left[\begin{array}{r}<br />
1\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1\\<br />
0\\<br />
\vdots\\<br />
0<br />
\end{array}\right],\ldots,\left[\begin{array}{r}<br />
0\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }n\ \text{dimensions}\end{aligned}</math><br />
<br />
are called coordinate vectors. They are often given a characteristic notation, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> in <math>n</math> dimensions. When arranged as columns of a matrix in the natural order, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> a matrix with a characteristic pattern elements emerges, with a special notation:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{rr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2}\end{array}\right] & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=I_{2}\\<br />
\left[\begin{array}{rrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \mathbf{e}_{3}\end{array}\right] & = & \left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 1 & 0\\<br />
0 & 0 & 1<br />
\end{array}\right]=I_{3}\\<br />
\left[\begin{array}{rrrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \ldots & \mathbf{e}_{n}\end{array}\right] & = & \left[\begin{array}{rrrr}<br />
1 & 0 & \ldots & 0\\<br />
0 & 1 & \ldots & 0\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
0 & 0 & \ldots & 1<br />
\end{array}\right]=I_{n}.\end{aligned}</math><br />
<br />
The ''diagonal'' of this matrix is where the 1 elements are located, and every other element is zero.<br />
<br />
Consider the effect of <math>I_{2}</math> on the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> by both pre and post multiplication:<br />
<br />
<math>\begin{aligned}<br />
I_{2}A & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\\<br />
AI_{2} & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\end{aligned}</math><br />
<br />
as is easily checked by the across and down rule.<br />
<br />
Because any matrix is left unchanged by pre or post multiplication by an appropriately dimensioned <math>I_{n},</math> <math>I_{n}</math> is called an ''identity matrix of dimension ''<math>n.</math> Sometimes it is called a ''unit matrix of dimension ''<math>n.</math> Notice that <math>I_{n}</math> is necessarily a square matrix.<br />
<br />
== Diagonal matrices ==<br />
<br />
The identity matrix is an example of a diagonal matrix, a matrix whose elements are all zero except for those on the diagonal. Usually diagonal matrices are taken to be square, for example: <math>D=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 2 & 0\\<br />
0 & 0 & 3<br />
\end{array}\right].</math> They also produce characteristic effects when pre or post multiplying another matrix.<br />
<br />
Consider the diagonal matrix: <math>B=\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]</math> and the products <math>AB,</math> <math>BA</math> for <math>A</math> as defined in the previous section:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & -4\\<br />
6 & -10<br />
\end{array}\right],\\<br />
BA & = & \left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & 4\\<br />
-6 & -10<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Comparing the results, we can deduce that post multiplication by a diagonal matrix multiplies each column of <math>A</math> by the corresponding diagonal element, whereas pre multiplication multiplies each row by the corresponding diagonal element.<br />
<br />
== Symmetric matrices ==<br />
<br />
Symmetric matrices are matrices having the property that <math>A=A^{T}.</math> Notice that such matrices must be square, since if <math>A</math> is <math>m\times n,</math> <math>A^{T}</math> is <math>n\times m,</math> and to have equality of <math>A</math> and <math>A^{T},</math> they must have the same dimension, so that <math>m=n</math> is required.<br />
<br />
Suppose that <math>A</math> is a <math>3\times3</math> symmetric matrix, with typical element <math>a_{ij}</math>: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right],</math> so that: <math>A^{T}=\left[\begin{array}{rrr}<br />
a_{11} & a_{21} & a_{31}\\<br />
a_{12} & a_{22} & a_{32}\\<br />
a_{13} & a_{23} & a_{33}<br />
\end{array}\right].</math> Equality of matrices is defined as equality of all elements. This is fine on the diagonals, since <math>A</math> and <math>A^{T}</math> have the same diagonal elements. For the off diagonal elements, we end up with the requirements: <math>a_{12}=a_{21},\ \ \ a_{13}=a_{31},\ \ \ a_{23}=a_{32}</math> or more generally: <math>a_{ij}=a_{ji}\ \ \ \ \ \text{for}\ i\neq j.</math><br />
<br />
The effect of this conclusion is that in a symmetric matrix, the ’triangle’ of above diagonal elements coincides with the triangle of below diagonal elements. It is as if the upper triangle is folded over the diagonal to become the lower triangle.<br />
<br />
A simple example is: <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
2 & 1<br />
\end{array}\right].</math> A more complicated example uses the <math>2\times3</math> matrix <math>C</math>: <math>C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]</math> and calculates the <math>3\times3</math> matrix:<br />
<br />
<math>\begin{aligned}<br />
C^{T}C & = & \left[\begin{array}{cc}<br />
6 & 3\\<br />
2 & 5\\<br />
-3 & -1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{rrr}<br />
45 & 27 & -21\\<br />
27 & 29 & -11\\<br />
-21 & -11 & 10<br />
\end{array}\right],\end{aligned}</math><br />
<br />
which is clearly symmetric.<br />
<br />
This illustrates the general proposition that if <math>A</math> is an <math>m\times n</math> matrix, the product <math>A^{T}A</math> is a symmetric <math>n\times n</math> matrix. Proof? Compute the transpose of <math>A^{T}A</math> using the product rule for transposition: <math>\left(A^{T}A\right)^{T}=A^{T}\left(A^{T}\right)^{T}=A^{T}A.</math> Since <math>A^{T}A</math> is equal to its transpose, it must be a symmetric matrix. Such symmetric matrices appear frequently in econometrics.<br />
<br />
It should be clear that diagonal matrices are symmetric, since all their off-diagonal elements are equal (zero), and thence the identity matrix <math>I_{n}</math> is also symmetric.<br />
<br />
== The outer product ==<br />
<br />
The inner product of two <math>n\times1</math> vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>, <math>\mathbf{x}^{T}\mathbf{y}</math>, is automatically a <math>1\times1</math> quantity, a scalar, although it can be interpreted as a <math>1\times1</math> matrix, a matrix with a single element.<br />
<br />
Suppose one considered the product of <math>\mathbf{x}</math> with <math>\mathbf{x}^{T}.</math> Is this defined? If <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times r,</math> then the product <math>AB</math> is <math>m\times r.</math> Applying this logic to <math>\mathbf{xx}^{T},</math> this is <math>\left(n\times1\right)\left(1\times n\right),</math> so the resulting product ''is ''defined, and is an <math>n\times n</math>'' matrix'' - the ''outer product'' of <math>\mathbf{x}</math> and <math>\mathbf{x}^{T},</math> the word ’outer’ being used to distinguish from the inner product.<br />
<br />
How does the across and down rule work here? Suppose that: <math>\mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> Then: <math>\mathbf{xx}^{T}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right].</math> Here, there is <math>1</math> element in row one of the ’matrix’ <math>\mathbf{x,}</math> and <math>1</math> element in column one of the matrix <math>\mathbf{x}^{T},</math> so the across and down rule still works - it is just that there is only one product per row and column combination. So: <math>\mathbf{xx}^{T}=\left[\begin{array}{cc}<br />
36 & 18\\<br />
18 & 9<br />
\end{array}\right],</math> and it is obvious from this that <math>\mathbf{xx}^{T}</math> is a symmetric matrix.<br />
<br />
One can see that this outer product need not be restricted to vectors of the same dimension. If <math>\mathbf{x}</math> is <math>n\times1</math> and <math>\mathbf{y}</math> is <math>m\times1,</math> then: <math>\mathbf{xy}^{T}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
y_{1} & \ldots & y_{m}\end{array}\right]=\left[\begin{array}{rrrr}<br />
x_{1}y_{1} & x_{1}y_{2} & \ldots & x_{1}y_{m}\\<br />
x_{2}y_{1} & x_{2}y_{2} & \ldots & x_{2}y_{m}\\<br />
\\<br />
x_{n}y_{1} & x_{n}y_{2} & \ldots & x_{n}y_{m}<br />
\end{array}\right].</math> So, <math>\mathbf{xy}^{T}</math> is <math>n\times m,</math> and consists of rows which are <math>\mathbf{y}^{T}</math> multiplied by an element of the <math>\mathbf{x}</math> vector.<br />
<br />
Another interesting and useful example involves a vector with every element equal to <math>1</math>: <math>\mathbf{1}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right].</math> Sometimes this is written as <math>\mathbf{1}_{n}</math> to indicate an <math>n\times1</math> vector, and is called the ''sum vector''. Why? Consider the impact of <math>\mathbf{1}_{2}</math> on the <math>2\times1</math> vector <math>\mathbf{x}</math> used above: <math>\mathbf{1}_{2}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=9,</math> i.e. an inner product of <math>\mathbf{x}</math> with the sum vector is the sum of the elements of <math>\mathbf{x.}</math> Dividing through by the number of elements in <math>\mathbf{x}</math> produces the average of the elements of <math>\mathbf{x}</math> - i.e. the ’sample mean’ of the elements of <math>\mathbf{x.}</math><br />
<br />
The outer product of <math>\mathbf{x}</math> with <math>\mathbf{1}_{2}</math> is also interesting:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{1}_{2}\mathbf{x}^{T} & = & \left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 3\\<br />
6 & 3<br />
\end{array}\right],\\<br />
\mathbf{x1}_{2}^{T} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 6\\<br />
3 & 3<br />
\end{array}\right],\end{aligned}</math><br />
<br />
showing that pre multiplication of an <math>\mathbf{x}^{T}</math> by <math>\mathbf{1}</math> repeats <math>\mathbf{x}^{T}</math> as rows of the product, whilst post multiplication of <math>\mathbf{x}</math> by <math>\mathbf{1}^{T}</math> repeats <math>\mathbf{x}</math> as the columns of the product.<br />
<br />
Finally: <math>\mathbf{1}_{n}\mathbf{1}_{n}^{T}=\left[\begin{array}{rrr}<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1<br />
\end{array}\right],</math> an <math>n\times n</math> matrix with every element equal to <math>1.</math> This type of matrix also appears in econometrics!<br />
<br />
== Triangular matrices ==<br />
<br />
A square ''lower triangular ''matrix has all elements above the main diagonal equal to zero, whilst a square ''upper triangular ''matrix has all elements below the main diagonal equal to zero. A simple example of a lower triangular matrix is: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & 0 & 0\\<br />
a_{21} & a_{22} & 0\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right].</math> Clearly, for this matrix, <math>A^{T}</math> is an upper triangular matrix.<br />
<br />
One can adapt the definition to rectangular matrices: for example, if two arbitrary rows are added to <math>A,</math> so that it becomes <math>5\times3,</math> it would still be considered lower triangular. Equally, if, for example, the third column of <math>A</math> above is removed, <math>A</math> is still considered lower triangular.<br />
<br />
Often, we use ''unit ''triangular matrices, where the diagonal elements are all equal to <math>1:</math> e.g.:<br />
<br />
<math>\left[\begin{array}{rrr}<br />
1 & 2 & 0\\<br />
0 & 1 & 1\\<br />
0 & 0 & 1<br />
\end{array}\right].\label{eq:lt_matrix}</math><br />
<br />
== Partitioned matrices ==<br />
<br />
Sometimes, especially with big matrices, it is useful to organise the elements of the matrix into components which are themselves matrices, for example: <math>B=\left[\begin{array}{rrrr}<br />
1 & 2 & 0 & 0\\<br />
8 & 3 & 0 & 0\\<br />
0 & 0 & 7 & 4\\<br />
0 & 0 & 6 & 5<br />
\end{array}\right]</math> Here it would be reasonable to write: <math>B=\left[\begin{array}{cc}<br />
B_{11} & 0\\<br />
0 & B_{22}<br />
\end{array}\right],</math> where <math>B_{ii},i=1,2,</math> represent <math>2\times2</math> matrices. <math>B</math> is an example of a ''partitioned matrix'': that is, an <math>m\times n</math> matrix <math>A</math> say: <math>A=\left\Vert a_{ij}\right\Vert ,</math> where the elements of <math>A</math> are organised into ''sub-matrices''. An example might be:<br />
<br />
<math>A=\left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right],\label{eq:partition_a}</math><br />
<br />
where the ''sub - matrices'' in the first row block have <math>r</math> rows, and therefore <math>m-r</math> rows in the second row block. The column blocks might be defined by (for example) 3 columns in the first column block, 4 in the second and <math>n-7</math> in the third column block.<br />
<br />
Another simple example might be: <math>A=\left[\begin{array}{rrr}<br />
A_{1} & A_{2} & A_{3}\end{array}\right],\ \ \ \ \ \mathbf{x=}\left[\begin{array}{c}<br />
\mathbf{x}_{1}\\<br />
\mathbf{x}_{2}\\<br />
\mathbf{x}_{3}<br />
\end{array}\right],</math> where <math>A</math> and therefore <math>A_{1},A_{2},A_{3}</math> have <math>m</math> rows, <math>A_{1}</math> has <math>n_{1}</math> columns, <math>A_{2}</math> has <math>n_{2}</math> columns, <math>A_{3}</math> has <math>n_{3}</math> columns. The ''subvectors'' in <math>\mathbf{x}</math> must have <math>n_{1},n_{2}</math> and <math>n_{3}</math> rows respectively, for the product <math>A\mathbf{x}</math> to exist.<br />
<br />
Suppose that <math>n_{1}+n_{2}+n_{3}=n,</math> so that <math>A</math> is <math>m\times n.</math> The <math>i</math>th element of <math>A\mathbf{x}</math> is: <math>\sum_{i=1}^{n}a_{ij}x_{j}</math> but the summation can be broken up into the first <math>n_{1}</math> terms: <math>\sum_{i=1}^{n_{1}}a_{ij}x_{j},</math> the next <math>n_{2}</math> terms: <math>\sum_{i=n_{1}+1}^{n_{1}+n_{2}}a_{ij}x_{j},</math> and the next <math>n_{3}</math> terms; <math>\sum_{i=n_{1}+n_{2}+1}^{n}a_{ij}x_{j}.</math><br />
<br />
The point about the use of partitioned matrices is that the product <math>A\mathbf{x}</math> can be represented as: <math>A\mathbf{x}=A_{1}\mathbf{x}_{1}+A_{2}\mathbf{x}_{2}+A\mathbf{x}_{3}</math> by applying the across and down rule to the submatrices and the subvectors, a much simpler representation than the use of summations.<br />
<br />
Each of the components is a conformable matrix-vector product: this is essential in any use of partitioned matrices to represent some matrix product. For example, using <math>A</math> from equation (8) and <math>B</math> as: <math>B=\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right],</math> it is easy to write:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right]\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{r}<br />
A_{11}B_{11}+A_{12}B_{21}+A_{13}B_{31}\\<br />
A_{21}B_{11}+A_{22}B_{21}+A_{23}B_{31}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
But, what are the row dimensions for the submatrices in <math>B?</math> What are the possible column dimensions for the submatrices in <math>B?</math><br />
<br />
= Matrices, vectors and econometrics =<br />
<br />
The data on weights and heights for 12 students in the data matrix: <math>D=\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]</math> would seem to be ideally suited for fitting a two variable regression model: <math>y_{i}=\alpha+\beta x_{i}+u_{i},\;\;\;\;\; i=1,...,12.</math> Here, the first column of <math>D</math> contains all the weight data, the data on the dependent variable <math>y_{i},</math> and so should be labelled <math>\mathbf{y.}</math> The second column of <math>D</math> contains all the data on the explanatory variable height, in the vector <math>\mathbf{x}</math> say, so that: <math>D=\left[\begin{array}{rr}<br />
\mathbf{y} & \mathbf{x}\end{array}\right].</math><br />
<br />
If we define a <math>12\times1</math> vector with every element <math>1</math>: <math>\mathbf{1}_{12}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right],</math> and a <math>12\times1</math> vector <math>\mathbf{u}</math> to contain the error terms; <math>\mathbf{u}=\left[\begin{array}{c}<br />
u_{1}\\<br />
\vdots\\<br />
u_{12}<br />
\end{array}\right]</math> the regression model can be written in terms of the three data vectors <math>\mathbf{y,1}_{12}</math> and <math>\mathbf{x}</math> as: <math>\mathbf{y}=\mathbf{1}_{12}\alpha+\mathbf{x}\beta+\mathbf{u.}</math> To see this, think of the <math>i</math>th elements of the vectors on the left and right hand sides.<br />
<br />
The standard next step is then to combine the data vectors for the explanatory variables into a matrix: <math>X=\left[\begin{array}{rr}<br />
\mathbf{1}_{12} & \mathbf{x}\end{array}\right],</math> and then define a <math>2\times1</math> vector <math>\boldsymbol{\delta}</math> to contain the parameters <math>\alpha,\beta</math> as: <math>\boldsymbol{\delta}=\left[\begin{array}{r}<br />
\alpha\\<br />
\beta<br />
\end{array}\right]</math> to give the data matrix representation of the regression model as: <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
<br />
For the purposes of developing the theory of regression, this is the most convenient form of the regression model. It can represent regression models with any number of explanatory variables, and thus any number of parameters. The obvious point is that a knowledge of vector and matrix operations is needed to use and understand this form.<br />
<br />
We shall see later that there are two particular matrix and vector quantities associated with a regression model. The first is the matrix <math>X^{T}X,</math> and the second the vector <math>X^{T}\mathbf{y.}</math> The following Matlab code snippet provides the numerical values of these quantities for the weight data:<br />
<br />
&gt;&gt; dset = load(’weights.mat’); <br />
<br />
&gt;&gt; xtx = dset.X’ * dset.X; <br />
<br />
&gt;&gt; xty = dset.X’ * dset.y; <br />
<br />
&gt;&gt; disp(xtx) <br />
<br />
12 802<br />
<br />
802 53792<br />
<br />
&gt;&gt; disp(xty)<br />
<br />
1850<br />
<br />
124528<br />
<br />
Hand calculation is of course possible, but not recommended.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=LNotes&diff=3025LNotes2013-09-10T14:09:38Z<p>LG: </p>
<hr />
<div>= Matrices =<br />
<br />
In the PreSession Maths course, a matrix was defined as follows:<br />
<br />
<blockquote>A matrix is a rectangular array of numbers enclosed in parentheses, con-<br />
<br />
ventionally denoted by a capital letter. The number of rows (say <math>m</math>) and<br />
<br />
the number of columns (say <math>n</math>) determine the order of the matrix (<math>m</math> <math>\times</math> <math>n</math>).<br />
</blockquote><br />
Two examples were given:<br />
<br />
<math>\begin{aligned}<br />
P & =\left[\begin{array}{rrr}<br />
2 & 3 & 4\\<br />
3 & 1 & 5<br />
\end{array}\right],\ \ \ Q=\left[\begin{array}{rr}<br />
2 & 3\\<br />
4 & 3\\<br />
1 & 5<br />
\end{array}\right],\end{aligned}</math><br />
<br />
matrices of dimensions <math>2\times3</math> and <math>3\times2</math> respectively.<br />
<br />
Why study matrices for econometrics? Basically because a data set of several variables, e.g. on the weights and heights of 12 students, can be thought of as a matrix:<br />
<br />
<math>\begin{aligned}<br />
D & =\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]\end{aligned}</math><br />
<br />
The properties of matrices can then be used to facilitate answering all the usual questions of econometrics - list not given here!<br />
<br />
Calculations with matrices with explicit numerical elements, as in the examples above is called matrix ''arithmetic''. Matrix ''algebra'' is the algebra of matrices where the elements are not made explicit: this is what is really required for econometrics, as we shall see.<br />
<br />
As an example of this, a <math>2\times3</math> matrix might be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{ccc}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\end{aligned}</math><br />
<br />
and would equal <math>P</math> above if the collection of <math>a_{ij}</math> were given appropriate numerical values.<br />
<br />
A general <math>m\times n</math> matrix <math>A</math> can be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
There is also a ''typical element ''notation for matrices:<br />
<br />
<math>\begin{aligned}<br />
A & =\left\Vert a_{ij}\right\Vert ,\ \ \ \ \ i=1,...,m,j=1,...,n,\end{aligned}</math><br />
<br />
so that <math>a_{ij}</math> is the element at the intersection of the <math>i</math>th row and <math>j</math>th column in <math>A.</math><br />
<br />
When <math>m\neq n,</math> <math>A</math> is a ''rectangular ''matrix; when <math>m=n,</math> <math>A</math> is <math>m\times m</math> or <math>n\times n,</math> and <math>A</math> is a square matrix, having the same number of rows or columns.<br />
<br />
== Rows, columns and vectors ==<br />
<br />
Clearly, there is no reason why <math>m</math> or <math>n</math> cannot equal 1: so, an <math>m\times n</math> matrix with <math>n=1,</math> i.e. with one column, is usually called a column vector. Similarly, a matrix with one row is a row vector.<br />
<br />
There are a lot of advantages to thinking of matrices as collections of row or column vectors, as we shall see. As an example, define the <math>2\times1</math> column vectors:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a} & =\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\mathbf{,\ \ \ b}=\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
and arrange as the columns of the <math>2\times2</math> matrix<br />
<br />
<math>A=\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\label{eq:axy}</math><br />
<br />
In general, a column vector <math>\mathbf{x}</math> with <math>n</math> elements can be written as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\end{aligned}</math><br />
<br />
What happens when both <math>m</math> and <math>n</math> are equal to <math>1?</math> Then, <math>A</math> is a <math>1\times1</math> matrix, but it is also considered to be a real number, or ''scalar'' in the language of linear algebra:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[a_{11}\right]=a_{11}.\end{aligned}</math><br />
<br />
This is perhaps a little odd, but turns out to be a useful convention in a number of situations.<br />
<br />
== Transposition of vectors ==<br />
<br />
The ''rows'' of the matrix <math>A</math> in equation (1) can be seen as elements of column vectors, say:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c} & =\left[\begin{array}{r}<br />
6\\<br />
2<br />
\end{array}\right],\ \ \ \boldsymbol{d}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
This representation of row vectors as column vectors is a bit clumsy, so some transformation which converts a column vector into a row vector, and vice versa would be useful. The process of converting a column vector <math>\mathbf{a}</math> into a row vector is called ''transposition, ''and the transposed version of <math>\mathbf{c}</math> is denoted:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c}^{T} & =\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\end{aligned}</math><br />
<br />
the <math>^{T}</math> superscript denoting transposition. In practice, a prime, <math>^{\prime},</math> is used instead of <math>^{T}.</math> However, whilst the prime is much simpler to write than the <math>^{T}</math> sign, it is also much easier to lose track of in writing out long or complicated expressions. So, it is best initially to use <math>^{T}</math> to denote transposition rather than the prime <math>^{\prime}.</math><br />
<br />
<math>A</math> can then be written via its rows as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{r}<br />
\mathbf{c}^{T}\\<br />
\boldsymbol{d}^{T}<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
The same ideas can be applied to the matrices <math>P</math> and <math>Q.</math><br />
<br />
= Operations with matrices =<br />
<br />
== Addition, subtraction and scalar multiplication ==<br />
<br />
For vectors, addition and subtraction are defined only for vectors of the same dimensions. If:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right],\\<br />
\mathbf{x+y} & =\left[\begin{array}{c}<br />
x_{1}+y_{1}\\<br />
\vdots\\<br />
x_{n}+y_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{x-y}=\left[\begin{array}{c}<br />
x_{1}-y_{1}\\<br />
\vdots\\<br />
x_{n}-y_{n}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Clearly, the addition or subtraction operation is ''elementwise. ''If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> have different dimensions, there will be some elements left over once all the elements of the smaller dimensioned vector have been used up.<br />
<br />
Another operation is ''scalar multiplication'': if <math>\lambda</math> is a real number or scalar, the product <math>\lambda\mathbf{x}</math> is defined as: <math>\lambda\mathbf{x}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right],</math> so that every element of <math>\mathbf{x}</math> is multiplied by the same scalar <math>\lambda.</math><br />
<br />
The two types of operation can be combined into the ''linear combination'' of vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>: <math>\lambda\mathbf{x}+\mu\mathbf{y}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right]+\left[\begin{array}{c}<br />
\mu y_{1}\\<br />
\vdots\\<br />
\mu y_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\lambda x_{1}+\mu y_{1}\\<br />
\vdots\\<br />
\lambda x_{n}+\mu y_{n}<br />
\end{array}\right].</math> Equally, one can define the linear combination of vectors <math>\mathbf{x,y,}\ldots,\mathbf{z}</math> by scalars <math>\lambda,\mu,\ldots,\nu</math> as: <math>\lambda\mathbf{x}+\mu\mathbf{y}+\ldots+\nu\mathbf{z}</math> with typical element: <math>\lambda x_{i}+\mu y_{i}+\ldots+\nu z_{i},</math> provided that all the vectors have the same dimension.<br />
<br />
For matrices, these ideas carry over immediately: apply to each column of the matrices involved. For example, if <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right]</math> and <math>B=\left[\begin{array}{rrr}<br />
\mathbf{b}_{1} & \ldots & \mathbf{b}_{n}\end{array}\right],</math> both <math>m\times n,</math> then addition and subtraction are defined elementwise, as for vectors:<br />
<br />
<math>\begin{aligned}<br />
A+B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}+\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}+\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}+b_{ij}\right\Vert ,\\<br />
A-B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}-\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}-\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}-b_{ij}\right\Vert .\end{aligned}</math><br />
<br />
Scalar multiplication of <math>A</math> by <math>\lambda</math> involves multiplying every column vector of <math>A</math> by <math>\lambda,</math> and therefore multiplying every element of <math>A</math> by <math>\lambda</math>: <math>\lambda A=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1} & \ldots & \lambda\mathbf{a}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}\right\Vert .</math> With the same idea for <math>B,</math> the linear combination of <math>A</math> and <math>B</math> by <math>\lambda</math> and <math>\mu</math> is: <math>\lambda A+\mu B=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1}+\mu\mathbf{b}_{1} & \ldots & \lambda\mathbf{a}_{n}+\mu\mathbf{b}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}+\mu b_{ij}\right\Vert .</math><br />
<br />
For example, consider the matrices: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right]</math> with <math>\lambda=1,</math> <math>\mu=-2:</math> then:<br />
<br />
<math>\begin{aligned}<br />
\lambda A+\mu B & = & A-2B\\<br />
& = & \left[\begin{array}{cc}<br />
4 & 0\\<br />
1 & 7<br />
\end{array}\right].\end{aligned}</math><br />
<br />
== Matrix - vector products ==<br />
<br />
=== Inner product ===<br />
<br />
The simplest form of a matrix vector product is the case where <math>A</math> consists of one row, so that <math>A</math> is <math>1\times n</math>: <math>A=\mathbf{a}^{T}=\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right].</math> If <math>\mathbf{x}</math> is an <math>n\times1</math> vector: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],</math> the product <math>A\mathbf{x}=\mathbf{a}^{T}\mathbf{x}</math> is called the ''inner product'' and is defined as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a}^{T}\mathbf{x} & =a_{1}x_{1}+\ldots+a_{n}x_{n}.\end{aligned}</math><br />
<br />
One can see that the definition amounts to multiplying corresponding elements in <math>\mathbf{a}</math> and <math>\mathbf{x,}</math> and adding up the resultant products. Writing: <math>\mathbf{a}^{T}\mathbf{x=}\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=a_{1}x_{1}+\ldots+a_{n}x_{n}</math> motivates the familiar description of the ''across and down rule ''for this product: ''across and down'' is the ’multiply corresponding elements’ part of the definition.<br />
<br />
Notice that the result of the inner product is a real number, for example: <math>\mathbf{c}^{T}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{c}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=36+6=42.</math><br />
<br />
In general, in the product <math>\mathbf{a}^{T}\mathbf{x,}</math> <math>\mathbf{a}</math> and <math>\mathbf{x}</math> must have the same number of elements, <math>n</math> say, for the product to be defined. If <math>\mathbf{a}</math> and <math>\mathbf{x}</math> had different numbers of elements, there would be some elements of <math>\mathbf{a}</math> or <math>\mathbf{x}</math> left over or not used in the product: e.g.: <math>\mathbf{b}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{x=}\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> When the inner product of two vectors is defined, the vectors are said to be ''conformable''.<br />
<br />
== Orthogonality ==<br />
<br />
Two vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math> with the property that <math>\mathbf{x}^{T}\mathbf{y}=0</math> are said to be orthogonal to each other. For example, if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
-1\\<br />
1<br />
\end{array}\right],</math> it is clear that <math>\mathbf{x}^{T}\mathbf{y}=0.</math> This seems a rather innocuous definition, and yet the idea of orthogonality turns out to be extremely important in econometrics.<br />
<br />
If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> are thought of as points in <math>R^{2},</math> and arrows are drawn from the origin to <math>\mathbf{x}</math> and to <math>\mathbf{y,}</math> then the two arrows are perpendicular to each other - see Figure [orthy<sub>e</sub>xample]. If <math>\mathbf{y}</math> were defined as: <math>\mathbf{y}=\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right],</math> the position of the <math>\mathbf{y}</math> vector and the corresponding arrow would change, but the perpendicularity property would still hold.<br />
<br />
Figure 1:<br />
[[Media:orthy_example.png]]<br />
[[File:orthy_example.png]]<br />
<br />
<br />
=== Matrix - vector products ===<br />
<br />
Since the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> has two rows, now denoted <math>\boldsymbol{\alpha}_{1}^{T}</math> and <math>\boldsymbol{\alpha}_{2}^{T},</math> there are two possible inner products with the vector:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]:\\<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x} & = & 42,\ \ \ \ \ \boldsymbol{\alpha}_{2}^{T}\mathbf{x}=33.\end{aligned}</math><br />
<br />
Assembling the two inner product values into a <math>2\times1</math> vector defines the product of the matrix <math>A</math> with the vector <math>\mathbf{x}</math>: <math>A\mathbf{x}=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x}\\<br />
\boldsymbol{\alpha}_{2}^{T}\mathbf{x}<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right].</math> Focussing only on the part: <math>\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right],</math> one can see that each element of <math>A\mathbf{x}</math> is obtained from an ''across and down'' argument.<br />
<br />
Sometimes this product is described as forming a ''linear combination ''of the columns of <math>A</math> using the scalar elements in <math>\mathbf{x}</math>: <math>A\mathbf{x}=6\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]+3\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].</math> More generally, if:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right],\ \ \ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
\lambda\\<br />
\mu<br />
\end{array}\right],\\<br />
A\mathbf{x} & = & \lambda\mathbf{a}+\mu\mathbf{b.}\end{aligned}</math><br />
<br />
The general version of these ideas for an <math>m\times n</math> matrix <math>A</math>: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]=\left[\begin{array}{rrrr}<br />
\mathbf{a}_{1} & \mathbf{a}_{2} & \ldots & \mathbf{a}_{n}\end{array}\right].</math> is straightforward. If <math>\mathbf{x}</math> is an <math>n\times1</math> vector, then the vector <math>A\mathbf{x}</math> is, by the ''across and down'' rule:<br />
<br />
<math>A\mathbf{x}=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
x_{2}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
a_{11}x_{1}+\ldots+a_{1n}x_{n}\\<br />
a_{21}x_{1}+\ldots+a_{2n}x_{n}\\<br />
\vdots\\<br />
a_{m1}x_{1}+\ldots+a_{mn}x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\sum\limits _{j=1}^{n}a_{1j}x_{j}\\<br />
\sum\limits _{j=1}^{n}a_{2j}x_{j}\\<br />
\vdots\\<br />
\sum\limits _{j=1}^{n}a_{mj}x_{j}<br />
\end{array}\right],\label{eq:ab}</math><br />
<br />
so that the typical element, the <math>i</math>th, is <math>\sum\limits _{j=1}^{n}a_{ij}x_{j}.</math> Equally, <math>A\mathbf{x}</math> is the linear combination <math>\mathbf{a}_{1}x_{1}+\ldots+\mathbf{a}_{n}x_{n}</math> of the columns of <math>A.</math><br />
<br />
== Matrix - matrix products ==<br />
<br />
Suppose that <math>A</math> is <math>m\times n,</math> with columns <math>\mathbf{a}_{1},\ldots,\mathbf{a}_{n},</math> and <math>B</math> is <math>n\times r,</math> with columns <math>\mathbf{b}_{1},\ldots,\mathbf{b}_{r}.</math> Clearly, each product <math>A\mathbf{b}_{1},...,A\mathbf{b}_{r}</math> exists, and is <math>m\times1.</math> These products can be arranged as the columns of a matrix as <math>\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]</math> and this matrix is ''defined'' to be the product <math>C</math> of the matrices <math>A</math> and <math>B</math>: <math>C=\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]=AB.</math> By construction, this must be an <math>m\times r</math> matrix, since each column is <math>m\times1</math> and there are <math>r</math> columns.<br />
<br />
This is not the usual presentation of the definition of the product of two matrices, which relies on the ''across and down rule'' mentioned earlier, and focusses on the elements of each matrix <math>A</math> and <math>B.</math> Set:<br />
<br />
<math>\begin{aligned}<br />
B & = & \left[\begin{array}{rrrr}<br />
\mathbf{b}_{1} & \mathbf{b}_{2} & \ldots & \mathbf{b}_{r}\end{array}\right]\text{\ \ \ \ \ \ \ (by columns)}\\<br />
& = & \left\Vert b_{ik}\right\Vert ,\ \ \ \ \ i=1,...,n,k=1,...,r\text{ \ \ \ \ \ \ (typical element)}\\<br />
& = & \left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2r}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nr}<br />
\end{array}\right]\text{\ \ \ \ \ \ \ (the array)}\end{aligned}</math><br />
<br />
What does the typical element of the <math>m\times r</math> matrix <math>C</math> look like? Start with the <math>k</math>th column of <math>C,</math> which is <math>A\mathbf{b}_{k}.</math> The <math>i</math>th element in <math>A\mathbf{b}_{k}</math> is, from equation (2), the inner product of the elements of the <math>i</math>th row in <math>A</math>: <math>\left[\begin{array}{rrrr}<br />
a_{i1} & a_{i2} & \ldots & a_{in}\end{array}\right],</math> with the elements of <math>\mathbf{b}_{k},</math> so that the inner product is: <math>a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math><br />
<br />
So, the <math>ik</math>th element of <math>C</math> is: <math>c_{ik}=a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math> We can see this arising from an ''across and down'' calculation by writing:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\label{eq:c_ab}\\<br />
& = & \left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{rrrrrr}<br />
b_{11} & b_{12} & \ldots & b_{1k} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2k} & \ldots & b_{2r}\\<br />
\vdots & \vdots & & \vdots & & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nk} & \ldots & b_{nr}<br />
\end{array}\right]\\<br />
& = & \left\Vert \sum_{j=1}^{n}a_{ij}b_{jk}\right\Vert .\end{aligned}</math><br />
<br />
These ideas are simple, but a little tedious. Numerical examples are equally tedious! As an example, using: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],</math> we can find the matrix <math>B</math> such that<br />
<br />
# the first column of <math>AB</math> adds together the columns of <math>A,</math><br />
# the second column is the difference of the first and second columns of <math>A,</math><br />
# the third column is <math>2\times</math> the first column of <math>A,</math><br />
# the fourth column is zero.<br />
<br />
It is easy to check that <math>B</math> is: <math>B=\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]</math> and that:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\\<br />
& = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{cccc}<br />
8 & 4 & 12 & 0\\<br />
8 & -2 & 6 & 0<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Arithmetic calculations of matrix products almost always use the elementwise across and down formula. However, there are many situations in econometrics where algebraic rather than arithmetic arguments are required. In these cases, the viewpoint of matrix multiplication as linear combinations of columns is much more powerful.<br />
<br />
Clearly one can give many more examples of different dimensions and complexities - but the same basic rules apply. To multiply two matrices <math>A</math> and <math>B</math> together, the number of columns in <math>A</math> must match the number of rows in <math>B</math> - this is ''conformability'' in action again. The resulting product will have number of rows equal to the number in <math>A</math> and number of columns equal to the number in <math>B.</math><br />
<br />
If this conformability rule does not hold, then the product of <math>A</math> and <math>B</math> is not defined.<br />
<br />
== Matlab ==<br />
<br />
One should also say that as the dimensions of the matrices increases, so the tediousness of the calculations increase. The solution to this for numerical calculation is to appeal to the computer. Programs like Matlab and Excel (and a number of others, some of them free) resolve this difficulty easily.<br />
<br />
In Matlab, symbols for row or column vectors do not need any particular differentiation: they are distinguished by how they are defined. For example, the following Matlab commands define <code>rowvec </code>as a <math>1\times4</math> vector, and <code>colvec</code> as a <math>4\times1</math> vector, then display the contents of these variables, and do a calculation:<br />
<br />
&gt;&gt; rowvec = [1 2 3 4]; <br />
<br />
&gt;&gt; colvec = [1;2;3;4]; <br />
<br />
&gt;&gt; rowvec<br />
<br />
rowvec =<br />
<br />
1 2 3 4<br />
<br />
&gt;&gt; colvec<br />
<br />
colvec =<br />
<br />
1 2 3 4 <br />
<br />
&gt;&gt; rowvec*colvec<br />
<br />
ans =<br />
<br />
30 <br />
<br />
So, the semi-colon indicates the end of a row in a matrix or vector; it can be replaced by a carriage return. Notice the difference in how a row vector and a column vector is defined. One can see that the product <code>rowvec*colvec</code> is well defined, just because <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector.<br />
<br />
Matlab also allows elementwise multiplication of two vectors using the <math>\centerdot\ast</math> operator: if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
x_{1}\\<br />
x_{2}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
y_{1}\\<br />
y_{2}<br />
\end{array}\right],</math> then: <math>\mathbf{x}\centerdot\ast\mathbf{y}=\left[\begin{array}{r}<br />
x_{1}y_{1}\\<br />
x_{2}y_{2}<br />
\end{array}\right]</math> and one can see that the inner product of <math>\mathbf{x}</math> and <math>\mathbf{y}</math> can be obtained as the sum of the elements of <math>\mathbf{x}</math> and <math>\mathbf{y.}</math> In Matlab, this would be obtained as: <math>\text{sum}\left(\mathbf{x}\centerdot\ast\mathbf{y}\right).</math><br />
<br />
In the example above, this calculation fails since <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector:<br />
<br />
&gt;&gt; sum(rowvec .* colvec) ??? <br />
<br />
Error using ==&gt; times Matrix dimensions must agree. <br />
<br />
For this to work, <code>rowvec</code> would have to be transposed as <code>rowvec’</code>, so that transposition in Matlab is very natural.<br />
<br />
Allowing for such difficulties, matrix multiplication in Matlab is very simple:<br />
<br />
&gt;&gt; A = [6 2; 3 5];<br />
<br />
&gt;&gt; B = [1 1 2 0;1 -1 0 0];<br />
<br />
&gt;&gt; C = A * B; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
8 4 1<br />
<br />
2 0 8 <br />
<br />
-2 6 0 <br />
<br />
Notice how the matrices are defined here through their rows. The <code>disp() </code>command displays the contents of the object referred to.<br />
<br />
It is less natural in Matlab to define matrices by columns - a typical example of how mathematics and computing have conflicts of notation. However, once columns <math>\mathbf{a}</math> and <math>\mathbf{b}</math> have been defined, the concatenation operation <math>\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]</math> collects the columns into a matrix:<br />
<br />
&gt;&gt; a = [6;2]; <br />
<br />
&gt;&gt; b = [3;5]; <br />
<br />
&gt;&gt; C = [a b]; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
6 3 <br />
<br />
2 5 <br />
<br />
Notice that the <code>disp(C)</code> command does not label the result that is printed out. Simply typing <code>C</code> would preface the output by <code>C =</code>.<br />
<br />
== Pre and Post Multiplication ==<br />
<br />
If <math>C=AB,</math> as above, say that <math>B</math> is ''pre-multiplied ''by <math>A</math> to get <math>C,</math> and that <math>A</math> is ''post-multiplied ''by <math>B</math> to get <math>C.</math><br />
<br />
This distinction between ''pre ''and ''post ''multiplication is important, in the following sense. Suppose that <math>A</math> and <math>B</math> are matrices such that the products <math>AB</math> and <math>BA</math> are both defined. If <math>A</math> is <math>m\times n,</math> <math>B</math> must have <math>n</math> rows for <math>AB</math> to be defined. For <math>BA</math> to be defined, <math>B</math> must have <math>m</math> columns to match the <math>m</math> rows in <math>A.</math> So, <math>AB</math> and <math>BA</math> are both defined if <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times m.</math><br />
<br />
Even when both products are defined, there is no reason for the two products coincide. The first thing to notice is that <math>AB</math> is a square, <math>m\times m,</math> matrix, whilst <math>BA</math> is a square, <math>n\times n,</math> matrix. Different sized matrices cannot be equal. To illustrate, use the matrices: <math>B_{2}=\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right],\ \ \ C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]:</math><br />
<br />
<math>\begin{aligned}<br />
B_{2}C & = & \left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]=\left[\begin{array}{rrr}<br />
27 & -3 & -15\\<br />
27 & 29 & -11\\<br />
-15 & -1 & 8<br />
\end{array}\right],\\<br />
CB_{2} & = & \left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
49 & -11\\<br />
31 & 15<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Even when <math>m=n,</math> so that <math>AB</math> and <math>BA</math> are both <math>m\times m</math> matrices, the products can differ: for example:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right],\\<br />
AB & = & \left[\begin{array}{cc}<br />
8 & 4\\<br />
8 & -2<br />
\end{array}\right],\ \ \ \ \ BA=\left[\begin{array}{cc}<br />
9 & 7\\<br />
3 & -3<br />
\end{array}\right].\end{aligned}</math><br />
<br />
In cases where <math>AB=BA,</math> the matrices <math>A</math> and <math>B</math> are said to ''commute''.<br />
<br />
== Transposition ==<br />
<br />
A column vector <math>\mathbf{x}</math> can be converted to a row vector <math>\mathbf{x}^{T}</math> by transposition: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \ \ \mathbf{x}^{T}=\left[\begin{array}{rrr}<br />
x_{1} & \ldots & x_{n}\end{array}\right].</math> Transposing <math>\mathbf{x}^{T}</math> as <math>\left(\mathbf{x}^{T}\right)^{T}</math> reproduces the original vector <math>\mathbf{x.}</math> How do these ideas carry over to matrices?<br />
<br />
If the <math>m\times n</math> matrix <math>A</math> can be written as <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right],</math> the transpose of <math>A,</math> <math>A^{T},</math> is defined as the matrix whose ''rows'' are <math>\mathbf{a}_{i}^{T}</math>: <math>A^{T}=\left[\begin{array}{c}<br />
\mathbf{a}_{1}^{T}\\<br />
\vdots\\<br />
\mathbf{a}_{n}^{T}<br />
\end{array}\right].</math> In terms of elements, if: <math>\mathbf{a}_{i}=\left[\begin{array}{c}<br />
a_{1i}\\<br />
\vdots\\<br />
a_{ni}<br />
\end{array}\right]</math> then: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right],\ \ \ \ \ A^{T}=\left[\begin{array}{rrrrr}<br />
a_{11} & \ldots & a_{i1} & \ldots & a_{m1}\\<br />
a_{12} & \ldots & a_{i2} & \ldots & a_{m2}\\<br />
\vdots & & \vdots & & \vdots\\<br />
a_{1n} & \ldots & a_{in} & \ldots & a_{mn}<br />
\end{array}\right].</math> One can see that the first column of <math>A</math> has now become the first row of <math>A^{T}.</math> Notice too that <math>A^{T}</math> is an <math>n\times m</math> matrix if <math>A</math> is an <math>m\times n</math> matrix.<br />
<br />
Transposing <math>A^{T}</math> takes the first column of <math>A^{T}</math> and writes it as a row, which coincides with the first row of <math>A.</math> The same argument applies to the other columns of <math>A^{T},</math> so that <math>\left(A^{T}\right)^{T}=A.</math><br />
<br />
=== The product rule for transposition ===<br />
<br />
This states that if <math>C=AB,</math> then <math>C^{T}=B^{T}A^{T}.</math><br />
<br />
How to see this? Consider the following example: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & b_{13} & b_{14}\\<br />
b_{21} & b_{22} & b_{23} & b_{24}\\<br />
b_{31} & b_{32} & b_{33} & b_{34}<br />
\end{array}\right]</math> where:<br />
<br />
<math>c_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=\sum_{k=1}^{3}a_{2k}b_{k3}.\label{eq:c23}</math><br />
<br />
One can see that: <math>B^{T}A^{T}=\left[\begin{array}{rrr}<br />
b_{11} & b_{21} & b_{31}\\<br />
b_{12} & b_{22} & b_{32}\\<br />
b_{13} & b_{23} & b_{33}\\<br />
b_{14} & b_{24} & b_{34}<br />
\end{array}\right]\left[\begin{array}{rr}<br />
a_{11} & a_{21}\\<br />
a_{12} & a_{22}\\<br />
a_{13} & a_{23}<br />
\end{array}\right]</math> and that the <math>\left(3,2\right)</math> element of this product is actually <math>c_{23}</math>: <math>b_{13}a_{21}+b_{23}a_{22}+b_{33}a_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=c_{23}.</math> In summation notation, we see that from <math>B^{T}A^{T}</math>: <math>c_{23}=\sum_{k=1}^{3}b_{k3}a_{2k},</math> where the position of the index of summation is due to the transposition. So, in summation notation, the calculation of <math>c_{23}</math> from <math>B^{T}A^{T}</math> equals that from equation (6).<br />
<br />
More generally, the <math>\left(i,j\right)</math> element of <math>AB</math>: <math>\sum_{k=1}^{3}a_{ik}b_{kj}</math> is the <math>\left(j,i\right)</math> element of <math>B^{T}A^{T}.</math> But this means that <math>B^{T}A^{T}</math> must be the transpose of <math>AB,</math> since the elements in the <math>i</math>th row of <math>AB</math> are being written in the <math>i</math>th column of <math>B^{T}A^{T}.</math><br />
<br />
This ''Product Rule for Transposition'' can be applied again to find the transpose <math>\left(C^{T}\right)^{T}</math> of <math>C^{T}</math>: <math>\left(C^{T}\right)^{T}=\left(B^{T}A^{T}\right)^{T}=\left(A^{T}\right)^{T}\left(B^{T}\right)^{T}=AB=C.</math><br />
<br />
= Special Types of Matrix =<br />
<br />
== The zero matrix ==<br />
<br />
The most obvious special type of matrix is one whose elements are all zeros. In typical element notation, the zero matrix is: <math>0=\left\Vert 0\right\Vert .</math> Since there is no indexing on the elements, it is not obvious what the dimension of this matrix is, Sometimes one writes <math>0_{mn}</math> to indicate a zero matrix of dimension <math>m\times n.</math> The same ideas apply to vectors whose elements are all zero.<br />
<br />
The effect of the zero matrix in any product that is defined is simple: <math>0A=0,\ \ \ \ \ B0=0.</math> This is easy to check using the across and down rule.<br />
<br />
== The identity or unit matrix ==<br />
<br />
Vectors of the form:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{r}<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }2\ \text{dimensions}\\<br />
\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }3\ \text{dimensions}\\<br />
\left[\begin{array}{r}<br />
1\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1\\<br />
0\\<br />
\vdots\\<br />
0<br />
\end{array}\right],\ldots,\left[\begin{array}{r}<br />
0\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }n\ \text{dimensions}\end{aligned}</math><br />
<br />
are called coordinate vectors. They are often given a characteristic notation, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> in <math>n</math> dimensions. When arranged as columns of a matrix in the natural order, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> a matrix with a characteristic pattern elements emerges, with a special notation:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{rr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2}\end{array}\right] & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=I_{2}\\<br />
\left[\begin{array}{rrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \mathbf{e}_{3}\end{array}\right] & = & \left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 1 & 0\\<br />
0 & 0 & 1<br />
\end{array}\right]=I_{3}\\<br />
\left[\begin{array}{rrrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \ldots & \mathbf{e}_{n}\end{array}\right] & = & \left[\begin{array}{rrrr}<br />
1 & 0 & \ldots & 0\\<br />
0 & 1 & \ldots & 0\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
0 & 0 & \ldots & 1<br />
\end{array}\right]=I_{n}.\end{aligned}</math><br />
<br />
The ''diagonal'' of this matrix is where the 1 elements are located, and every other element is zero.<br />
<br />
Consider the effect of <math>I_{2}</math> on the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> by both pre and post multiplication:<br />
<br />
<math>\begin{aligned}<br />
I_{2}A & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\\<br />
AI_{2} & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\end{aligned}</math><br />
<br />
as is easily checked by the across and down rule.<br />
<br />
Because any matrix is left unchanged by pre or post multiplication by an appropriately dimensioned <math>I_{n},</math> <math>I_{n}</math> is called an ''identity matrix of dimension ''<math>n.</math> Sometimes it is called a ''unit matrix of dimension ''<math>n.</math> Notice that <math>I_{n}</math> is necessarily a square matrix.<br />
<br />
== Diagonal matrices ==<br />
<br />
The identity matrix is an example of a diagonal matrix, a matrix whose elements are all zero except for those on the diagonal. Usually diagonal matrices are taken to be square, for example: <math>D=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 2 & 0\\<br />
0 & 0 & 3<br />
\end{array}\right].</math> They also produce characteristic effects when pre or post multiplying another matrix.<br />
<br />
Consider the diagonal matrix: <math>B=\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]</math> and the products <math>AB,</math> <math>BA</math> for <math>A</math> as defined in the previous section:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & -4\\<br />
6 & -10<br />
\end{array}\right],\\<br />
BA & = & \left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & 4\\<br />
-6 & -10<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Comparing the results, we can deduce that post multiplication by a diagonal matrix multiplies each column of <math>A</math> by the corresponding diagonal element, whereas pre multiplication multiplies each row by the corresponding diagonal element.<br />
<br />
== Symmetric matrices ==<br />
<br />
Symmetric matrices are matrices having the property that <math>A=A^{T}.</math> Notice that such matrices must be square, since if <math>A</math> is <math>m\times n,</math> <math>A^{T}</math> is <math>n\times m,</math> and to have equality of <math>A</math> and <math>A^{T},</math> they must have the same dimension, so that <math>m=n</math> is required.<br />
<br />
Suppose that <math>A</math> is a <math>3\times3</math> symmetric matrix, with typical element <math>a_{ij}</math>: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right],</math> so that: <math>A^{T}=\left[\begin{array}{rrr}<br />
a_{11} & a_{21} & a_{31}\\<br />
a_{12} & a_{22} & a_{32}\\<br />
a_{13} & a_{23} & a_{33}<br />
\end{array}\right].</math> Equality of matrices is defined as equality of all elements. This is fine on the diagonals, since <math>A</math> and <math>A^{T}</math> have the same diagonal elements. For the off diagonal elements, we end up with the requirements: <math>a_{12}=a_{21},\ \ \ a_{13}=a_{31},\ \ \ a_{23}=a_{32}</math> or more generally: <math>a_{ij}=a_{ji}\ \ \ \ \ \text{for}\ i\neq j.</math><br />
<br />
The effect of this conclusion is that in a symmetric matrix, the ’triangle’ of above diagonal elements coincides with the triangle of below diagonal elements. It is as if the upper triangle is folded over the diagonal to become the lower triangle.<br />
<br />
A simple example is: <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
2 & 1<br />
\end{array}\right].</math> A more complicated example uses the <math>2\times3</math> matrix <math>C</math>: <math>C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]</math> and calculates the <math>3\times3</math> matrix:<br />
<br />
<math>\begin{aligned}<br />
C^{T}C & = & \left[\begin{array}{cc}<br />
6 & 3\\<br />
2 & 5\\<br />
-3 & -1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{rrr}<br />
45 & 27 & -21\\<br />
27 & 29 & -11\\<br />
-21 & -11 & 10<br />
\end{array}\right],\end{aligned}</math><br />
<br />
which is clearly symmetric.<br />
<br />
This illustrates the general proposition that if <math>A</math> is an <math>m\times n</math> matrix, the product <math>A^{T}A</math> is a symmetric <math>n\times n</math> matrix. Proof? Compute the transpose of <math>A^{T}A</math> using the product rule for transposition: <math>\left(A^{T}A\right)^{T}=A^{T}\left(A^{T}\right)^{T}=A^{T}A.</math> Since <math>A^{T}A</math> is equal to its transpose, it must be a symmetric matrix. Such symmetric matrices appear frequently in econometrics.<br />
<br />
It should be clear that diagonal matrices are symmetric, since all their off-diagonal elements are equal (zero), and thence the identity matrix <math>I_{n}</math> is also symmetric.<br />
<br />
== The outer product ==<br />
<br />
The inner product of two <math>n\times1</math> vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>, <math>\mathbf{x}^{T}\mathbf{y}</math>, is automatically a <math>1\times1</math> quantity, a scalar, although it can be interpreted as a <math>1\times1</math> matrix, a matrix with a single element.<br />
<br />
Suppose one considered the product of <math>\mathbf{x}</math> with <math>\mathbf{x}^{T}.</math> Is this defined? If <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times r,</math> then the product <math>AB</math> is <math>m\times r.</math> Applying this logic to <math>\mathbf{xx}^{T},</math> this is <math>\left(n\times1\right)\left(1\times n\right),</math> so the resulting product ''is ''defined, and is an <math>n\times n</math>'' matrix'' - the ''outer product'' of <math>\mathbf{x}</math> and <math>\mathbf{x}^{T},</math> the word ’outer’ being used to distinguish from the inner product.<br />
<br />
How does the across and down rule work here? Suppose that: <math>\mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> Then: <math>\mathbf{xx}^{T}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right].</math> Here, there is <math>1</math> element in row one of the ’matrix’ <math>\mathbf{x,}</math> and <math>1</math> element in column one of the matrix <math>\mathbf{x}^{T},</math> so the across and down rule still works - it is just that there is only one product per row and column combination. So: <math>\mathbf{xx}^{T}=\left[\begin{array}{cc}<br />
36 & 18\\<br />
18 & 9<br />
\end{array}\right],</math> and it is obvious from this that <math>\mathbf{xx}^{T}</math> is a symmetric matrix.<br />
<br />
One can see that this outer product need not be restricted to vectors of the same dimension. If <math>\mathbf{x}</math> is <math>n\times1</math> and <math>\mathbf{y}</math> is <math>m\times1,</math> then: <math>\mathbf{xy}^{T}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
y_{1} & \ldots & y_{m}\end{array}\right]=\left[\begin{array}{rrrr}<br />
x_{1}y_{1} & x_{1}y_{2} & \ldots & x_{1}y_{m}\\<br />
x_{2}y_{1} & x_{2}y_{2} & \ldots & x_{2}y_{m}\\<br />
\\<br />
x_{n}y_{1} & x_{n}y_{2} & \ldots & x_{n}y_{m}<br />
\end{array}\right].</math> So, <math>\mathbf{xy}^{T}</math> is <math>n\times m,</math> and consists of rows which are <math>\mathbf{y}^{T}</math> multiplied by an element of the <math>\mathbf{x}</math> vector.<br />
<br />
Another interesting and useful example involves a vector with every element equal to <math>1</math>: <math>\mathbf{1}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right].</math> Sometimes this is written as <math>\mathbf{1}_{n}</math> to indicate an <math>n\times1</math> vector, and is called the ''sum vector''. Why? Consider the impact of <math>\mathbf{1}_{2}</math> on the <math>2\times1</math> vector <math>\mathbf{x}</math> used above: <math>\mathbf{1}_{2}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=9,</math> i.e. an inner product of <math>\mathbf{x}</math> with the sum vector is the sum of the elements of <math>\mathbf{x.}</math> Dividing through by the number of elements in <math>\mathbf{x}</math> produces the average of the elements of <math>\mathbf{x}</math> - i.e. the ’sample mean’ of the elements of <math>\mathbf{x.}</math><br />
<br />
The outer product of <math>\mathbf{x}</math> with <math>\mathbf{1}_{2}</math> is also interesting:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{1}_{2}\mathbf{x}^{T} & = & \left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 3\\<br />
6 & 3<br />
\end{array}\right],\\<br />
\mathbf{x1}_{2}^{T} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 6\\<br />
3 & 3<br />
\end{array}\right],\end{aligned}</math><br />
<br />
showing that pre multiplication of an <math>\mathbf{x}^{T}</math> by <math>\mathbf{1}</math> repeats <math>\mathbf{x}^{T}</math> as rows of the product, whilst post multiplication of <math>\mathbf{x}</math> by <math>\mathbf{1}^{T}</math> repeats <math>\mathbf{x}</math> as the columns of the product.<br />
<br />
Finally: <math>\mathbf{1}_{n}\mathbf{1}_{n}^{T}=\left[\begin{array}{rrr}<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1<br />
\end{array}\right],</math> an <math>n\times n</math> matrix with every element equal to <math>1.</math> This type of matrix also appears in econometrics!<br />
<br />
== Triangular matrices ==<br />
<br />
A square ''lower triangular ''matrix has all elements above the main diagonal equal to zero, whilst a square ''upper triangular ''matrix has all elements below the main diagonal equal to zero. A simple example of a lower triangular matrix is: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & 0 & 0\\<br />
a_{21} & a_{22} & 0\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right].</math> Clearly, for this matrix, <math>A^{T}</math> is an upper triangular matrix.<br />
<br />
One can adapt the definition to rectangular matrices: for example, if two arbitrary rows are added to <math>A,</math> so that it becomes <math>5\times3,</math> it would still be considered lower triangular. Equally, if, for example, the third column of <math>A</math> above is removed, <math>A</math> is still considered lower triangular.<br />
<br />
Often, we use ''unit ''triangular matrices, where the diagonal elements are all equal to <math>1:</math> e.g.:<br />
<br />
<math>\left[\begin{array}{rrr}<br />
1 & 2 & 0\\<br />
0 & 1 & 1\\<br />
0 & 0 & 1<br />
\end{array}\right].\label{eq:lt_matrix}</math><br />
<br />
== Partitioned matrices ==<br />
<br />
Sometimes, especially with big matrices, it is useful to organise the elements of the matrix into components which are themselves matrices, for example: <math>B=\left[\begin{array}{rrrr}<br />
1 & 2 & 0 & 0\\<br />
8 & 3 & 0 & 0\\<br />
0 & 0 & 7 & 4\\<br />
0 & 0 & 6 & 5<br />
\end{array}\right]</math> Here it would be reasonable to write: <math>B=\left[\begin{array}{cc}<br />
B_{11} & 0\\<br />
0 & B_{22}<br />
\end{array}\right],</math> where <math>B_{ii},i=1,2,</math> represent <math>2\times2</math> matrices. <math>B</math> is an example of a ''partitioned matrix'': that is, an <math>m\times n</math> matrix <math>A</math> say: <math>A=\left\Vert a_{ij}\right\Vert ,</math> where the elements of <math>A</math> are organised into ''sub-matrices''. An example might be:<br />
<br />
<math>A=\left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right],\label{eq:partition_a}</math><br />
<br />
where the ''sub - matrices'' in the first row block have <math>r</math> rows, and therefore <math>m-r</math> rows in the second row block. The column blocks might be defined by (for example) 3 columns in the first column block, 4 in the second and <math>n-7</math> in the third column block.<br />
<br />
Another simple example might be: <math>A=\left[\begin{array}{rrr}<br />
A_{1} & A_{2} & A_{3}\end{array}\right],\ \ \ \ \ \mathbf{x=}\left[\begin{array}{c}<br />
\mathbf{x}_{1}\\<br />
\mathbf{x}_{2}\\<br />
\mathbf{x}_{3}<br />
\end{array}\right],</math> where <math>A</math> and therefore <math>A_{1},A_{2},A_{3}</math> have <math>m</math> rows, <math>A_{1}</math> has <math>n_{1}</math> columns, <math>A_{2}</math> has <math>n_{2}</math> columns, <math>A_{3}</math> has <math>n_{3}</math> columns. The ''subvectors'' in <math>\mathbf{x}</math> must have <math>n_{1},n_{2}</math> and <math>n_{3}</math> rows respectively, for the product <math>A\mathbf{x}</math> to exist.<br />
<br />
Suppose that <math>n_{1}+n_{2}+n_{3}=n,</math> so that <math>A</math> is <math>m\times n.</math> The <math>i</math>th element of <math>A\mathbf{x}</math> is: <math>\sum_{i=1}^{n}a_{ij}x_{j}</math> but the summation can be broken up into the first <math>n_{1}</math> terms: <math>\sum_{i=1}^{n_{1}}a_{ij}x_{j},</math> the next <math>n_{2}</math> terms: <math>\sum_{i=n_{1}+1}^{n_{1}+n_{2}}a_{ij}x_{j},</math> and the next <math>n_{3}</math> terms; <math>\sum_{i=n_{1}+n_{2}+1}^{n}a_{ij}x_{j}.</math><br />
<br />
The point about the use of partitioned matrices is that the product <math>A\mathbf{x}</math> can be represented as: <math>A\mathbf{x}=A_{1}\mathbf{x}_{1}+A_{2}\mathbf{x}_{2}+A\mathbf{x}_{3}</math> by applying the across and down rule to the submatrices and the subvectors, a much simpler representation than the use of summations.<br />
<br />
Each of the components is a conformable matrix-vector product: this is essential in any use of partitioned matrices to represent some matrix product. For example, using <math>A</math> from equation (8) and <math>B</math> as: <math>B=\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right],</math> it is easy to write:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right]\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{r}<br />
A_{11}B_{11}+A_{12}B_{21}+A_{13}B_{31}\\<br />
A_{21}B_{11}+A_{22}B_{21}+A_{23}B_{31}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
But, what are the row dimensions for the submatrices in <math>B?</math> What are the possible column dimensions for the submatrices in <math>B?</math><br />
<br />
= Matrices, vectors and econometrics =<br />
<br />
The data on weights and heights for 12 students in the data matrix: <math>D=\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]</math> would seem to be ideally suited for fitting a two variable regression model: <math>y_{i}=\alpha+\beta x_{i}+u_{i},\;\;\;\;\; i=1,...,12.</math> Here, the first column of <math>D</math> contains all the weight data, the data on the dependent variable <math>y_{i},</math> and so should be labelled <math>\mathbf{y.}</math> The second column of <math>D</math> contains all the data on the explanatory variable height, in the vector <math>\mathbf{x}</math> say, so that: <math>D=\left[\begin{array}{rr}<br />
\mathbf{y} & \mathbf{x}\end{array}\right].</math><br />
<br />
If we define a <math>12\times1</math> vector with every element <math>1</math>: <math>\mathbf{1}_{12}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right],</math> and a <math>12\times1</math> vector <math>\mathbf{u}</math> to contain the error terms; <math>\mathbf{u}=\left[\begin{array}{c}<br />
u_{1}\\<br />
\vdots\\<br />
u_{12}<br />
\end{array}\right]</math> the regression model can be written in terms of the three data vectors <math>\mathbf{y,1}_{12}</math> and <math>\mathbf{x}</math> as: <math>\mathbf{y}=\mathbf{1}_{12}\alpha+\mathbf{x}\beta+\mathbf{u.}</math> To see this, think of the <math>i</math>th elements of the vectors on the left and right hand sides.<br />
<br />
The standard next step is then to combine the data vectors for the explanatory variables into a matrix: <math>X=\left[\begin{array}{rr}<br />
\mathbf{1}_{12} & \mathbf{x}\end{array}\right],</math> and then define a <math>2\times1</math> vector <math>\boldsymbol{\delta}</math> to contain the parameters <math>\alpha,\beta</math> as: <math>\boldsymbol{\delta}=\left[\begin{array}{r}<br />
\alpha\\<br />
\beta<br />
\end{array}\right]</math> to give the data matrix representation of the regression model as: <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
<br />
For the purposes of developing the theory of regression, this is the most convenient form of the regression model. It can represent regression models with any number of explanatory variables, and thus any number of parameters. The obvious point is that a knowledge of vector and matrix operations is needed to use and understand this form.<br />
<br />
We shall see later that there are two particular matrix and vector quantities associated with a regression model. The first is the matrix <math>X^{T}X,</math> and the second the vector <math>X^{T}\mathbf{y.}</math> The following Matlab code snippet provides the numerical values of these quantities for the weight data:<br />
<br />
&gt;&gt; dset = load(’weights.mat’); <br />
<br />
&gt;&gt; xtx = dset.X’ * dset.X; <br />
<br />
&gt;&gt; xty = dset.X’ * dset.y; <br />
<br />
&gt;&gt; disp(xtx) <br />
<br />
12 802<br />
<br />
802 53792<br />
<br />
&gt;&gt; disp(xty)<br />
<br />
1850<br />
<br />
124528<br />
<br />
Hand calculation is of course possible, but not recommended.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=LNotes&diff=3024LNotes2013-09-10T13:55:59Z<p>LG: </p>
<hr />
<div>= Matrices =<br />
<br />
In the PreSession Maths course, a matrix was defined as follows:<br />
<br />
<blockquote>A matrix is a rectangular array of numbers enclosed in parentheses, con-<br />
<br />
ventionally denoted by a capital letter. The number of rows (say <math>m</math>) and<br />
<br />
the number of columns (say <math>n</math>) determine the order of the matrix (<math>m</math> <math>\times</math> <math>n</math>).<br />
</blockquote><br />
Two examples were given:<br />
<br />
<math>\begin{aligned}<br />
P & =\left[\begin{array}{rrr}<br />
2 & 3 & 4\\<br />
3 & 1 & 5<br />
\end{array}\right],\ \ \ Q=\left[\begin{array}{rr}<br />
2 & 3\\<br />
4 & 3\\<br />
1 & 5<br />
\end{array}\right],\end{aligned}</math><br />
<br />
matrices of dimensions <math>2\times3</math> and <math>3\times2</math> respectively.<br />
<br />
Why study matrices for econometrics? Basically because a data set of several variables, e.g. on the weights and heights of 12 students, can be thought of as a matrix:<br />
<br />
<math>\begin{aligned}<br />
D & =\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]\end{aligned}</math><br />
<br />
The properties of matrices can then be used to facilitate answering all the usual questions of econometrics - list not given here!<br />
<br />
Calculations with matrices with explicit numerical elements, as in the examples above is called matrix ''arithmetic''. Matrix ''algebra'' is the algebra of matrices where the elements are not made explicit: this is what is really required for econometrics, as we shall see.<br />
<br />
As an example of this, a <math>2\times3</math> matrix might be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{ccc}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\end{aligned}</math><br />
<br />
and would equal <math>P</math> above if the collection of <math>a_{ij}</math> were given appropriate numerical values.<br />
<br />
A general <math>m\times n</math> matrix <math>A</math> can be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
There is also a ''typical element ''notation for matrices:<br />
<br />
<math>\begin{aligned}<br />
A & =\left\Vert a_{ij}\right\Vert ,\ \ \ \ \ i=1,...,m,j=1,...,n,\end{aligned}</math><br />
<br />
so that <math>a_{ij}</math> is the element at the intersection of the <math>i</math>th row and <math>j</math>th column in <math>A.</math><br />
<br />
When <math>m\neq n,</math> <math>A</math> is a ''rectangular ''matrix; when <math>m=n,</math> <math>A</math> is <math>m\times m</math> or <math>n\times n,</math> and <math>A</math> is a square matrix, having the same number of rows or columns.<br />
<br />
== Rows, columns and vectors ==<br />
<br />
Clearly, there is no reason why <math>m</math> or <math>n</math> cannot equal 1: so, an <math>m\times n</math> matrix with <math>n=1,</math> i.e. with one column, is usually called a column vector. Similarly, a matrix with one row is a row vector.<br />
<br />
There are a lot of advantages to thinking of matrices as collections of row or column vectors, as we shall see. As an example, define the <math>2\times1</math> column vectors:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a} & =\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\mathbf{,\ \ \ b}=\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
and arrange as the columns of the <math>2\times2</math> matrix<br />
<br />
<math>A=\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\label{eq:axy}</math><br />
<br />
In general, a column vector <math>\mathbf{x}</math> with <math>n</math> elements can be written as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\end{aligned}</math><br />
<br />
What happens when both <math>m</math> and <math>n</math> are equal to <math>1?</math> Then, <math>A</math> is a <math>1\times1</math> matrix, but it is also considered to be a real number, or ''scalar'' in the language of linear algebra:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[a_{11}\right]=a_{11}.\end{aligned}</math><br />
<br />
This is perhaps a little odd, but turns out to be a useful convention in a number of situations.<br />
<br />
== Transposition of vectors ==<br />
<br />
The ''rows'' of the matrix <math>A</math> in equation (1) can be seen as elements of column vectors, say:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c} & =\left[\begin{array}{r}<br />
6\\<br />
2<br />
\end{array}\right],\ \ \ \boldsymbol{d}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
This representation of row vectors as column vectors is a bit clumsy, so some transformation which converts a column vector into a row vector, and vice versa would be useful. The process of converting a column vector <math>\mathbf{a}</math> into a row vector is called ''transposition, ''and the transposed version of <math>\mathbf{c}</math> is denoted:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c}^{T} & =\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\end{aligned}</math><br />
<br />
the <math>^{T}</math> superscript denoting transposition. In practice, a prime, <math>^{\prime},</math> is used instead of <math>^{T}.</math> However, whilst the prime is much simpler to write than the <math>^{T}</math> sign, it is also much easier to lose track of in writing out long or complicated expressions. So, it is best initially to use <math>^{T}</math> to denote transposition rather than the prime <math>^{\prime}.</math><br />
<br />
<math>A</math> can then be written via its rows as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{r}<br />
\mathbf{c}^{T}\\<br />
\boldsymbol{d}^{T}<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
The same ideas can be applied to the matrices <math>P</math> and <math>Q.</math><br />
<br />
= Operations with matrices =<br />
<br />
== Addition, subtraction and scalar multiplication ==<br />
<br />
For vectors, addition and subtraction are defined only for vectors of the same dimensions. If:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right],\\<br />
\mathbf{x+y} & =\left[\begin{array}{c}<br />
x_{1}+y_{1}\\<br />
\vdots\\<br />
x_{n}+y_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{x-y}=\left[\begin{array}{c}<br />
x_{1}-y_{1}\\<br />
\vdots\\<br />
x_{n}-y_{n}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Clearly, the addition or subtraction operation is ''elementwise. ''If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> have different dimensions, there will be some elements left over once all the elements of the smaller dimensioned vector have been used up.<br />
<br />
Another operation is ''scalar multiplication'': if <math>\lambda</math> is a real number or scalar, the product <math>\lambda\mathbf{x}</math> is defined as: <math>\lambda\mathbf{x}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right],</math> so that every element of <math>\mathbf{x}</math> is multiplied by the same scalar <math>\lambda.</math><br />
<br />
The two types of operation can be combined into the ''linear combination'' of vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>: <math>\lambda\mathbf{x}+\mu\mathbf{y}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right]+\left[\begin{array}{c}<br />
\mu y_{1}\\<br />
\vdots\\<br />
\mu y_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\lambda x_{1}+\mu y_{1}\\<br />
\vdots\\<br />
\lambda x_{n}+\mu y_{n}<br />
\end{array}\right].</math> Equally, one can define the linear combination of vectors <math>\mathbf{x,y,}\ldots,\mathbf{z}</math> by scalars <math>\lambda,\mu,\ldots,\nu</math> as: <math>\lambda\mathbf{x}+\mu\mathbf{y}+\ldots+\nu\mathbf{z}</math> with typical element: <math>\lambda x_{i}+\mu y_{i}+\ldots+\nu z_{i},</math> provided that all the vectors have the same dimension.<br />
<br />
For matrices, these ideas carry over immediately: apply to each column of the matrices involved. For example, if <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right]</math> and <math>B=\left[\begin{array}{rrr}<br />
\mathbf{b}_{1} & \ldots & \mathbf{b}_{n}\end{array}\right],</math> both <math>m\times n,</math> then addition and subtraction are defined elementwise, as for vectors:<br />
<br />
<math>\begin{aligned}<br />
A+B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}+\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}+\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}+b_{ij}\right\Vert ,\\<br />
A-B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}-\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}-\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}-b_{ij}\right\Vert .\end{aligned}</math><br />
<br />
Scalar multiplication of <math>A</math> by <math>\lambda</math> involves multiplying every column vector of <math>A</math> by <math>\lambda,</math> and therefore multiplying every element of <math>A</math> by <math>\lambda</math>: <math>\lambda A=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1} & \ldots & \lambda\mathbf{a}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}\right\Vert .</math> With the same idea for <math>B,</math> the linear combination of <math>A</math> and <math>B</math> by <math>\lambda</math> and <math>\mu</math> is: <math>\lambda A+\mu B=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1}+\mu\mathbf{b}_{1} & \ldots & \lambda\mathbf{a}_{n}+\mu\mathbf{b}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}+\mu b_{ij}\right\Vert .</math><br />
<br />
For example, consider the matrices: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right]</math> with <math>\lambda=1,</math> <math>\mu=-2:</math> then:<br />
<br />
<math>\begin{aligned}<br />
\lambda A+\mu B & = & A-2B\\<br />
& = & \left[\begin{array}{cc}<br />
4 & 0\\<br />
1 & 7<br />
\end{array}\right].\end{aligned}</math><br />
<br />
== Matrix - vector products ==<br />
<br />
=== Inner product ===<br />
<br />
The simplest form of a matrix vector product is the case where <math>A</math> consists of one row, so that <math>A</math> is <math>1\times n</math>: <math>A=\mathbf{a}^{T}=\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right].</math> If <math>\mathbf{x}</math> is an <math>n\times1</math> vector: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],</math> the product <math>A\mathbf{x}=\mathbf{a}^{T}\mathbf{x}</math> is called the ''inner product'' and is defined as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a}^{T}\mathbf{x} & =a_{1}x_{1}+\ldots+a_{n}x_{n}.\end{aligned}</math><br />
<br />
One can see that the definition amounts to multiplying corresponding elements in <math>\mathbf{a}</math> and <math>\mathbf{x,}</math> and adding up the resultant products. Writing: <math>\mathbf{a}^{T}\mathbf{x=}\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=a_{1}x_{1}+\ldots+a_{n}x_{n}</math> motivates the familiar description of the ''across and down rule ''for this product: ''across and down'' is the ’multiply corresponding elements’ part of the definition.<br />
<br />
Notice that the result of the inner product is a real number, for example: <math>\mathbf{c}^{T}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{c}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=36+6=42.</math><br />
<br />
In general, in the product <math>\mathbf{a}^{T}\mathbf{x,}</math> <math>\mathbf{a}</math> and <math>\mathbf{x}</math> must have the same number of elements, <math>n</math> say, for the product to be defined. If <math>\mathbf{a}</math> and <math>\mathbf{x}</math> had different numbers of elements, there would be some elements of <math>\mathbf{a}</math> or <math>\mathbf{x}</math> left over or not used in the product: e.g.: <math>\mathbf{b}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{x=}\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> When the inner product of two vectors is defined, the vectors are said to be ''conformable''.<br />
<br />
== Orthogonality ==<br />
<br />
Two vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math> with the property that <math>\mathbf{x}^{T}\mathbf{y}=0</math> are said to be orthogonal to each other. For example, if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
-1\\<br />
1<br />
\end{array}\right],</math> it is clear that <math>\mathbf{x}^{T}\mathbf{y}=0.</math> This seems a rather innocuous definition, and yet the idea of orthogonality turns out to be extremely important in econometrics.<br />
<br />
If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> are thought of as points in <math>R^{2},</math> and arrows are drawn from the origin to <math>\mathbf{x}</math> and to <math>\mathbf{y,}</math> then the two arrows are perpendicular to each other - see Figure [orthy<sub>e</sub>xample]. If <math>\mathbf{y}</math> were defined as: <math>\mathbf{y}=\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right],</math> the position of the <math>\mathbf{y}</math> vector and the corresponding arrow would change, but the perpendicularity property would still hold.<br />
Figure 1:<br />
[[Media:orthy_example.png]]<br />
<br />
<br />
=== Matrix - vector products ===<br />
<br />
Since the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> has two rows, now denoted <math>\boldsymbol{\alpha}_{1}^{T}</math> and <math>\boldsymbol{\alpha}_{2}^{T},</math> there are two possible inner products with the vector:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]:\\<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x} & = & 42,\ \ \ \ \ \boldsymbol{\alpha}_{2}^{T}\mathbf{x}=33.\end{aligned}</math><br />
<br />
Assembling the two inner product values into a <math>2\times1</math> vector defines the product of the matrix <math>A</math> with the vector <math>\mathbf{x}</math>: <math>A\mathbf{x}=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x}\\<br />
\boldsymbol{\alpha}_{2}^{T}\mathbf{x}<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right].</math> Focussing only on the part: <math>\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right],</math> one can see that each element of <math>A\mathbf{x}</math> is obtained from an ''across and down'' argument.<br />
<br />
Sometimes this product is described as forming a ''linear combination ''of the columns of <math>A</math> using the scalar elements in <math>\mathbf{x}</math>: <math>A\mathbf{x}=6\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]+3\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].</math> More generally, if:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right],\ \ \ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
\lambda\\<br />
\mu<br />
\end{array}\right],\\<br />
A\mathbf{x} & = & \lambda\mathbf{a}+\mu\mathbf{b.}\end{aligned}</math><br />
<br />
The general version of these ideas for an <math>m\times n</math> matrix <math>A</math>: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]=\left[\begin{array}{rrrr}<br />
\mathbf{a}_{1} & \mathbf{a}_{2} & \ldots & \mathbf{a}_{n}\end{array}\right].</math> is straightforward. If <math>\mathbf{x}</math> is an <math>n\times1</math> vector, then the vector <math>A\mathbf{x}</math> is, by the ''across and down'' rule:<br />
<br />
<math>A\mathbf{x}=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
x_{2}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
a_{11}x_{1}+\ldots+a_{1n}x_{n}\\<br />
a_{21}x_{1}+\ldots+a_{2n}x_{n}\\<br />
\vdots\\<br />
a_{m1}x_{1}+\ldots+a_{mn}x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\sum\limits _{j=1}^{n}a_{1j}x_{j}\\<br />
\sum\limits _{j=1}^{n}a_{2j}x_{j}\\<br />
\vdots\\<br />
\sum\limits _{j=1}^{n}a_{mj}x_{j}<br />
\end{array}\right],\label{eq:ab}</math><br />
<br />
so that the typical element, the <math>i</math>th, is <math>\sum\limits _{j=1}^{n}a_{ij}x_{j}.</math> Equally, <math>A\mathbf{x}</math> is the linear combination <math>\mathbf{a}_{1}x_{1}+\ldots+\mathbf{a}_{n}x_{n}</math> of the columns of <math>A.</math><br />
<br />
== Matrix - matrix products ==<br />
<br />
Suppose that <math>A</math> is <math>m\times n,</math> with columns <math>\mathbf{a}_{1},\ldots,\mathbf{a}_{n},</math> and <math>B</math> is <math>n\times r,</math> with columns <math>\mathbf{b}_{1},\ldots,\mathbf{b}_{r}.</math> Clearly, each product <math>A\mathbf{b}_{1},...,A\mathbf{b}_{r}</math> exists, and is <math>m\times1.</math> These products can be arranged as the columns of a matrix as <math>\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]</math> and this matrix is ''defined'' to be the product <math>C</math> of the matrices <math>A</math> and <math>B</math>: <math>C=\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]=AB.</math> By construction, this must be an <math>m\times r</math> matrix, since each column is <math>m\times1</math> and there are <math>r</math> columns.<br />
<br />
This is not the usual presentation of the definition of the product of two matrices, which relies on the ''across and down rule'' mentioned earlier, and focusses on the elements of each matrix <math>A</math> and <math>B.</math> Set:<br />
<br />
<math>\begin{aligned}<br />
B & = & \left[\begin{array}{rrrr}<br />
\mathbf{b}_{1} & \mathbf{b}_{2} & \ldots & \mathbf{b}_{r}\end{array}\right]\text{\,\,\,\,\,\,\,(by columns)}\\<br />
& = & \left\Vert b_{ik}\right\Vert ,\ \ \ \ \ i=1,...,n,k=1,...,r\text{ \,\,\,\,\,\,(typical element)}\\<br />
& = & \left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2r}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nr}<br />
\end{array}\right]\text{\,\,\,\,\,\,\,(the array)}\end{aligned}</math><br />
<br />
What does the typical element of the <math>m\times r</math> matrix <math>C</math> look like? Start with the <math>k</math>th column of <math>C,</math> which is <math>A\mathbf{b}_{k}.</math> The <math>i</math>th element in <math>A\mathbf{b}_{k}</math> is, from equation (2), the inner product of the elements of the <math>i</math>th row in <math>A</math>: <math>\left[\begin{array}{rrrr}<br />
a_{i1} & a_{i2} & \ldots & a_{in}\end{array}\right],</math> with the elements of <math>\mathbf{b}_{k},</math> so that the inner product is: <math>a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math><br />
<br />
So, the <math>ik</math>th element of <math>C</math> is: <math>c_{ik}=a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math> We can see this arising from an ''across and down'' calculation by writing:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\label{eq:c_ab}\\<br />
& = & \left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{rrrrrr}<br />
b_{11} & b_{12} & \ldots & b_{1k} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2k} & \ldots & b_{2r}\\<br />
\vdots & \vdots & & \vdots & & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nk} & \ldots & b_{nr}<br />
\end{array}\right]\\<br />
& = & \left\Vert \sum_{j=1}^{n}a_{ij}b_{jk}\right\Vert .\end{aligned}</math><br />
<br />
These ideas are simple, but a little tedious. Numerical examples are equally tedious! As an example, using: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],</math> we can find the matrix <math>B</math> such that<br />
<br />
# the first column of <math>AB</math> adds together the columns of <math>A,</math><br />
# the second column is the difference of the first and second columns of <math>A,</math><br />
# the third column is <math>2\times</math> the first column of <math>A,</math><br />
# the fourth column is zero.<br />
<br />
It is easy to check that <math>B</math> is: <math>B=\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]</math> and that:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\\<br />
& = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{cccc}<br />
8 & 4 & 12 & 0\\<br />
8 & -2 & 6 & 0<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Arithmetic calculations of matrix products almost always use the elementwise across and down formula. However, there are many situations in econometrics where algebraic rather than arithmetic arguments are required. In these cases, the viewpoint of matrix multiplication as linear combinations of columns is much more powerful.<br />
<br />
Clearly one can give many more examples of different dimensions and complexities - but the same basic rules apply. To multiply two matrices <math>A</math> and <math>B</math> together, the number of columns in <math>A</math> must match the number of rows in <math>B</math> - this is ''conformability'' in action again. The resulting product will have number of rows equal to the number in <math>A</math> and number of columns equal to the number in <math>B.</math><br />
<br />
If this conformability rule does not hold, then the product of <math>A</math> and <math>B</math> is not defined.<br />
<br />
== Matlab ==<br />
<br />
One should also say that as the dimensions of the matrices increases, so the tediousness of the calculations increase. The solution to this for numerical calculation is to appeal to the computer. Programs like Matlab and Excel (and a number of others, some of them free) resolve this difficulty easily.<br />
<br />
In Matlab, symbols for row or column vectors do not need any particular differentiation: they are distinguished by how they are defined. For example, the following Matlab commands define <code>rowvec </code>as a <math>1\times4</math> vector, and <code>colvec</code> as a <math>4\times1</math> vector, then display the contents of these variables, and do a calculation:<br />
<br />
&gt;&gt; rowvec = [1 2 3 4]; <br />
<br />
&gt;&gt; colvec = [1;2;3;4]; <br />
<br />
&gt;&gt; rowvec<br />
<br />
rowvec =<br />
<br />
1 2 3 4<br />
<br />
&gt;&gt; colvec<br />
<br />
colvec =<br />
<br />
1 2 3 4 <br />
<br />
&gt;&gt; rowvec*colvec<br />
<br />
ans =<br />
<br />
30 <br />
<br />
So, the semi-colon indicates the end of a row in a matrix or vector; it can be replaced by a carriage return. Notice the difference in how a row vector and a column vector is defined. One can see that the product <code>rowvec*colvec</code> is well defined, just because <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector.<br />
<br />
Matlab also allows elementwise multiplication of two vectors using the <math>\centerdot\ast</math> operator: if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
x_{1}\\<br />
x_{2}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
y_{1}\\<br />
y_{2}<br />
\end{array}\right],</math> then: <math>\mathbf{x}\centerdot\ast\mathbf{y}=\left[\begin{array}{r}<br />
x_{1}y_{1}\\<br />
x_{2}y_{2}<br />
\end{array}\right]</math> and one can see that the inner product of <math>\mathbf{x}</math> and <math>\mathbf{y}</math> can be obtained as the sum of the elements of <math>\mathbf{x}</math> and <math>\mathbf{y.}</math> In Matlab, this would be obtained as: <math>\text{sum}\left(\mathbf{x}\centerdot\ast\mathbf{y}\right).</math><br />
<br />
In the example above, this calculation fails since <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector:<br />
<br />
&gt;&gt; sum(rowvec .* colvec) ??? <br />
<br />
Error using ==&gt; times Matrix dimensions must agree. <br />
<br />
For this to work, <code>rowvec</code> would have to be transposed as <code>rowvec’</code>, so that transposition in Matlab is very natural.<br />
<br />
Allowing for such difficulties, matrix multiplication in Matlab is very simple:<br />
<br />
&gt;&gt; A = [6 2; 3 5];<br />
<br />
&gt;&gt; B = [1 1 2 0;1 -1 0 0];<br />
<br />
&gt;&gt; C = A * B; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
8 4 1<br />
<br />
2 0 8 <br />
<br />
-2 6 0 <br />
<br />
Notice how the matrices are defined here through their rows. The <code>disp() </code>command displays the contents of the object referred to.<br />
<br />
It is less natural in Matlab to define matrices by columns - a typical example of how mathematics and computing have conflicts of notation. However, once columns <math>\mathbf{a}</math> and <math>\mathbf{b}</math> have been defined, the concatenation operation <math>\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]</math> collects the columns into a matrix:<br />
<br />
&gt;&gt; a = [6;2]; <br />
<br />
&gt;&gt; b = [3;5]; <br />
<br />
&gt;&gt; C = [a b]; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
6 3 <br />
<br />
2 5 <br />
<br />
Notice that the <code>disp(C)</code> command does not label the result that is printed out. Simply typing <code>C</code> would preface the output by <code>C =</code>.<br />
<br />
== Pre and Post Multiplication ==<br />
<br />
If <math>C=AB,</math> as above, say that <math>B</math> is ''pre-multiplied ''by <math>A</math> to get <math>C,</math> and that <math>A</math> is ''post-multiplied ''by <math>B</math> to get <math>C.</math><br />
<br />
This distinction between ''pre ''and ''post ''multiplication is important, in the following sense. Suppose that <math>A</math> and <math>B</math> are matrices such that the products <math>AB</math> and <math>BA</math> are both defined. If <math>A</math> is <math>m\times n,</math> <math>B</math> must have <math>n</math> rows for <math>AB</math> to be defined. For <math>BA</math> to be defined, <math>B</math> must have <math>m</math> columns to match the <math>m</math> rows in <math>A.</math> So, <math>AB</math> and <math>BA</math> are both defined if <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times m.</math><br />
<br />
Even when both products are defined, there is no reason for the two products coincide. The first thing to notice is that <math>AB</math> is a square, <math>m\times m,</math> matrix, whilst <math>BA</math> is a square, <math>n\times n,</math> matrix. Different sized matrices cannot be equal. To illustrate, use the matrices: <math>B_{2}=\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right],\ \ \ C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]:</math><br />
<br />
<math>\begin{aligned}<br />
B_{2}C & = & \left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]=\left[\begin{array}{rrr}<br />
27 & -3 & -15\\<br />
27 & 29 & -11\\<br />
-15 & -1 & 8<br />
\end{array}\right],\\<br />
CB_{2} & = & \left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
49 & -11\\<br />
31 & 15<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Even when <math>m=n,</math> so that <math>AB</math> and <math>BA</math> are both <math>m\times m</math> matrices, the products can differ: for example:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right],\\<br />
AB & = & \left[\begin{array}{cc}<br />
8 & 4\\<br />
8 & -2<br />
\end{array}\right],\ \ \ \ \ BA=\left[\begin{array}{cc}<br />
9 & 7\\<br />
3 & -3<br />
\end{array}\right].\end{aligned}</math><br />
<br />
In cases where <math>AB=BA,</math> the matrices <math>A</math> and <math>B</math> are said to ''commute''.<br />
<br />
== Transposition ==<br />
<br />
A column vector <math>\mathbf{x}</math> can be converted to a row vector <math>\mathbf{x}^{T}</math> by transposition: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \ \ \mathbf{x}^{T}=\left[\begin{array}{rrr}<br />
x_{1} & \ldots & x_{n}\end{array}\right].</math> Transposing <math>\mathbf{x}^{T}</math> as <math>\left(\mathbf{x}^{T}\right)^{T}</math> reproduces the original vector <math>\mathbf{x.}</math> How do these ideas carry over to matrices?<br />
<br />
If the <math>m\times n</math> matrix <math>A</math> can be written as <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right],</math> the transpose of <math>A,</math> <math>A^{T},</math> is defined as the matrix whose ''rows'' are <math>\mathbf{a}_{i}^{T}</math>: <math>A^{T}=\left[\begin{array}{c}<br />
\mathbf{a}_{1}^{T}\\<br />
\vdots\\<br />
\mathbf{a}_{n}^{T}<br />
\end{array}\right].</math> In terms of elements, if: <math>\mathbf{a}_{i}=\left[\begin{array}{c}<br />
a_{1i}\\<br />
\vdots\\<br />
a_{ni}<br />
\end{array}\right]</math> then: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right],\ \ \ \ \ A^{T}=\left[\begin{array}{rrrrr}<br />
a_{11} & \ldots & a_{i1} & \ldots & a_{m1}\\<br />
a_{12} & \ldots & a_{i2} & \ldots & a_{m2}\\<br />
\vdots & & \vdots & & \vdots\\<br />
a_{1n} & \ldots & a_{in} & \ldots & a_{mn}<br />
\end{array}\right].</math> One can see that the first column of <math>A</math> has now become the first row of <math>A^{T}.</math> Notice too that <math>A^{T}</math> is an <math>n\times m</math> matrix if <math>A</math> is an <math>m\times n</math> matrix.<br />
<br />
Transposing <math>A^{T}</math> takes the first column of <math>A^{T}</math> and writes it as a row, which coincides with the first row of <math>A.</math> The same argument applies to the other columns of <math>A^{T},</math> so that <math>\left(A^{T}\right)^{T}=A.</math><br />
<br />
=== The product rule for transposition ===<br />
<br />
This states that if <math>C=AB,</math> then <math>C^{T}=B^{T}A^{T}.</math><br />
<br />
How to see this? Consider the following example: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & b_{13} & b_{14}\\<br />
b_{21} & b_{22} & b_{23} & b_{24}\\<br />
b_{31} & b_{32} & b_{33} & b_{34}<br />
\end{array}\right]</math> where:<br />
<br />
<math>c_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=\sum_{k=1}^{3}a_{2k}b_{k3}.\label{eq:c23}</math><br />
<br />
One can see that: <math>B^{T}A^{T}=\left[\begin{array}{rrr}<br />
b_{11} & b_{21} & b_{31}\\<br />
b_{12} & b_{22} & b_{32}\\<br />
b_{13} & b_{23} & b_{33}\\<br />
b_{14} & b_{24} & b_{34}<br />
\end{array}\right]\left[\begin{array}{rr}<br />
a_{11} & a_{21}\\<br />
a_{12} & a_{22}\\<br />
a_{13} & a_{23}<br />
\end{array}\right]</math> and that the <math>\left(3,2\right)</math> element of this product is actually <math>c_{23}</math>: <math>b_{13}a_{21}+b_{23}a_{22}+b_{33}a_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=c_{23}.</math> In summation notation, we see that from <math>B^{T}A^{T}</math>: <math>c_{23}=\sum_{k=1}^{3}b_{k3}a_{2k},</math> where the position of the index of summation is due to the transposition. So, in summation notation, the calculation of <math>c_{23}</math> from <math>B^{T}A^{T}</math> equals that from equation (6).<br />
<br />
More generally, the <math>\left(i,j\right)</math> element of <math>AB</math>: <math>\sum_{k=1}^{3}a_{ik}b_{kj}</math> is the <math>\left(j,i\right)</math> element of <math>B^{T}A^{T}.</math> But this means that <math>B^{T}A^{T}</math> must be the transpose of <math>AB,</math> since the elements in the <math>i</math>th row of <math>AB</math> are being written in the <math>i</math>th column of <math>B^{T}A^{T}.</math><br />
<br />
This ''Product Rule for Transposition'' can be applied again to find the transpose <math>\left(C^{T}\right)^{T}</math> of <math>C^{T}</math>: <math>\left(C^{T}\right)^{T}=\left(B^{T}A^{T}\right)^{T}=\left(A^{T}\right)^{T}\left(B^{T}\right)^{T}=AB=C.</math><br />
<br />
= Special Types of Matrix =<br />
<br />
== The zero matrix ==<br />
<br />
The most obvious special type of matrix is one whose elements are all zeros. In typical element notation, the zero matrix is: <math>0=\left\Vert 0\right\Vert .</math> Since there is no indexing on the elements, it is not obvious what the dimension of this matrix is, Sometimes one writes <math>0_{mn}</math> to indicate a zero matrix of dimension <math>m\times n.</math> The same ideas apply to vectors whose elements are all zero.<br />
<br />
The effect of the zero matrix in any product that is defined is simple: <math>0A=0,\ \ \ \ \ B0=0.</math> This is easy to check using the across and down rule.<br />
<br />
== The identity or unit matrix ==<br />
<br />
Vectors of the form:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{r}<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }2\ \text{dimensions}\\<br />
\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }3\ \text{dimensions}\\<br />
\left[\begin{array}{r}<br />
1\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1\\<br />
0\\<br />
\vdots\\<br />
0<br />
\end{array}\right],\ldots,\left[\begin{array}{r}<br />
0\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }n\ \text{dimensions}\end{aligned}</math><br />
<br />
are called coordinate vectors. They are often given a characteristic notation, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> in <math>n</math> dimensions. When arranged as columns of a matrix in the natural order, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> a matrix with a characteristic pattern elements emerges, with a special notation:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{rr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2}\end{array}\right] & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=I_{2}\\<br />
\left[\begin{array}{rrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \mathbf{e}_{3}\end{array}\right] & = & \left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 1 & 0\\<br />
0 & 0 & 1<br />
\end{array}\right]=I_{3}\\<br />
\left[\begin{array}{rrrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \ldots & \mathbf{e}_{n}\end{array}\right] & = & \left[\begin{array}{rrrr}<br />
1 & 0 & \ldots & 0\\<br />
0 & 1 & \ldots & 0\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
0 & 0 & \ldots & 1<br />
\end{array}\right]=I_{n}.\end{aligned}</math><br />
<br />
The ''diagonal'' of this matrix is where the 1 elements are located, and every other element is zero.<br />
<br />
Consider the effect of <math>I_{2}</math> on the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> by both pre and post multiplication:<br />
<br />
<math>\begin{aligned}<br />
I_{2}A & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\\<br />
AI_{2} & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\end{aligned}</math><br />
<br />
as is easily checked by the across and down rule.<br />
<br />
Because any matrix is left unchanged by pre or post multiplication by an appropriately dimensioned <math>I_{n},</math> <math>I_{n}</math> is called an ''identity matrix of dimension ''<math>n.</math> Sometimes it is called a ''unit matrix of dimension ''<math>n.</math> Notice that <math>I_{n}</math> is necessarily a square matrix.<br />
<br />
== Diagonal matrices ==<br />
<br />
The identity matrix is an example of a diagonal matrix, a matrix whose elements are all zero except for those on the diagonal. Usually diagonal matrices are taken to be square, for example: <math>D=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 2 & 0\\<br />
0 & 0 & 3<br />
\end{array}\right].</math> They also produce characteristic effects when pre or post multiplying another matrix.<br />
<br />
Consider the diagonal matrix: <math>B=\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]</math> and the products <math>AB,</math> <math>BA</math> for <math>A</math> as defined in the previous section:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & -4\\<br />
6 & -10<br />
\end{array}\right],\\<br />
BA & = & \left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & 4\\<br />
-6 & -10<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Comparing the results, we can deduce that post multiplication by a diagonal matrix multiplies each column of <math>A</math> by the corresponding diagonal element, whereas pre multiplication multiplies each row by the corresponding diagonal element.<br />
<br />
== Symmetric matrices ==<br />
<br />
Symmetric matrices are matrices having the property that <math>A=A^{T}.</math> Notice that such matrices must be square, since if <math>A</math> is <math>m\times n,</math> <math>A^{T}</math> is <math>n\times m,</math> and to have equality of <math>A</math> and <math>A^{T},</math> they must have the same dimension, so that <math>m=n</math> is required.<br />
<br />
Suppose that <math>A</math> is a <math>3\times3</math> symmetric matrix, with typical element <math>a_{ij}</math>: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right],</math> so that: <math>A^{T}=\left[\begin{array}{rrr}<br />
a_{11} & a_{21} & a_{31}\\<br />
a_{12} & a_{22} & a_{32}\\<br />
a_{13} & a_{23} & a_{33}<br />
\end{array}\right].</math> Equality of matrices is defined as equality of all elements. This is fine on the diagonals, since <math>A</math> and <math>A^{T}</math> have the same diagonal elements. For the off diagonal elements, we end up with the requirements: <math>a_{12}=a_{21},\ \ \ a_{13}=a_{31},\ \ \ a_{23}=a_{32}</math> or more generally: <math>a_{ij}=a_{ji}\ \ \ \ \ \text{for}\ i\neq j.</math><br />
<br />
The effect of this conclusion is that in a symmetric matrix, the ’triangle’ of above diagonal elements coincides with the triangle of below diagonal elements. It is as if the upper triangle is folded over the diagonal to become the lower triangle.<br />
<br />
A simple example is: <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
2 & 1<br />
\end{array}\right].</math> A more complicated example uses the <math>2\times3</math> matrix <math>C</math>: <math>C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]</math> and calculates the <math>3\times3</math> matrix:<br />
<br />
<math>\begin{aligned}<br />
C^{T}C & = & \left[\begin{array}{cc}<br />
6 & 3\\<br />
2 & 5\\<br />
-3 & -1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{rrr}<br />
45 & 27 & -21\\<br />
27 & 29 & -11\\<br />
-21 & -11 & 10<br />
\end{array}\right],\end{aligned}</math><br />
<br />
which is clearly symmetric.<br />
<br />
This illustrates the general proposition that if <math>A</math> is an <math>m\times n</math> matrix, the product <math>A^{T}A</math> is a symmetric <math>n\times n</math> matrix. Proof? Compute the transpose of <math>A^{T}A</math> using the product rule for transposition: <math>\left(A^{T}A\right)^{T}=A^{T}\left(A^{T}\right)^{T}=A^{T}A.</math> Since <math>A^{T}A</math> is equal to its transpose, it must be a symmetric matrix. Such symmetric matrices appear frequently in econometrics.<br />
<br />
It should be clear that diagonal matrices are symmetric, since all their off-diagonal elements are equal (zero), and thence the identity matrix <math>I_{n}</math> is also symmetric.<br />
<br />
== The outer product ==<br />
<br />
The inner product of two <math>n\times1</math> vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>, <math>\mathbf{x}^{T}\mathbf{y}</math>, is automatically a <math>1\times1</math> quantity, a scalar, although it can be interpreted as a <math>1\times1</math> matrix, a matrix with a single element.<br />
<br />
Suppose one considered the product of <math>\mathbf{x}</math> with <math>\mathbf{x}^{T}.</math> Is this defined? If <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times r,</math> then the product <math>AB</math> is <math>m\times r.</math> Applying this logic to <math>\mathbf{xx}^{T},</math> this is <math>\left(n\times1\right)\left(1\times n\right),</math> so the resulting product ''is ''defined, and is an <math>n\times n</math>'' matrix'' - the ''outer product'' of <math>\mathbf{x}</math> and <math>\mathbf{x}^{T},</math> the word ’outer’ being used to distinguish from the inner product.<br />
<br />
How does the across and down rule work here? Suppose that: <math>\mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> Then: <math>\mathbf{xx}^{T}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right].</math> Here, there is <math>1</math> element in row one of the ’matrix’ <math>\mathbf{x,}</math> and <math>1</math> element in column one of the matrix <math>\mathbf{x}^{T},</math> so the across and down rule still works - it is just that there is only one product per row and column combination. So: <math>\mathbf{xx}^{T}=\left[\begin{array}{cc}<br />
36 & 18\\<br />
18 & 9<br />
\end{array}\right],</math> and it is obvious from this that <math>\mathbf{xx}^{T}</math> is a symmetric matrix.<br />
<br />
One can see that this outer product need not be restricted to vectors of the same dimension. If <math>\mathbf{x}</math> is <math>n\times1</math> and <math>\mathbf{y}</math> is <math>m\times1,</math> then: <math>\mathbf{xy}^{T}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
y_{1} & \ldots & y_{m}\end{array}\right]=\left[\begin{array}{rrrr}<br />
x_{1}y_{1} & x_{1}y_{2} & \ldots & x_{1}y_{m}\\<br />
x_{2}y_{1} & x_{2}y_{2} & \ldots & x_{2}y_{m}\\<br />
\\<br />
x_{n}y_{1} & x_{n}y_{2} & \ldots & x_{n}y_{m}<br />
\end{array}\right].</math> So, <math>\mathbf{xy}^{T}</math> is <math>n\times m,</math> and consists of rows which are <math>\mathbf{y}^{T}</math> multiplied by an element of the <math>\mathbf{x}</math> vector.<br />
<br />
Another interesting and useful example involves a vector with every element equal to <math>1</math>: <math>\mathbf{1}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right].</math> Sometimes this is written as <math>\mathbf{1}_{n}</math> to indicate an <math>n\times1</math> vector, and is called the ''sum vector''. Why? Consider the impact of <math>\mathbf{1}_{2}</math> on the <math>2\times1</math> vector <math>\mathbf{x}</math> used above: <math>\mathbf{1}_{2}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=9,</math> i.e. an inner product of <math>\mathbf{x}</math> with the sum vector is the sum of the elements of <math>\mathbf{x.}</math> Dividing through by the number of elements in <math>\mathbf{x}</math> produces the average of the elements of <math>\mathbf{x}</math> - i.e. the ’sample mean’ of the elements of <math>\mathbf{x.}</math><br />
<br />
The outer product of <math>\mathbf{x}</math> with <math>\mathbf{1}_{2}</math> is also interesting:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{1}_{2}\mathbf{x}^{T} & = & \left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 3\\<br />
6 & 3<br />
\end{array}\right],\\<br />
\mathbf{x1}_{2}^{T} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 6\\<br />
3 & 3<br />
\end{array}\right],\end{aligned}</math><br />
<br />
showing that pre multiplication of an <math>\mathbf{x}^{T}</math> by <math>\mathbf{1}</math> repeats <math>\mathbf{x}^{T}</math> as rows of the product, whilst post multiplication of <math>\mathbf{x}</math> by <math>\mathbf{1}^{T}</math> repeats <math>\mathbf{x}</math> as the columns of the product.<br />
<br />
Finally: <math>\mathbf{1}_{n}\mathbf{1}_{n}^{T}=\left[\begin{array}{rrr}<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1<br />
\end{array}\right],</math> an <math>n\times n</math> matrix with every element equal to <math>1.</math> This type of matrix also appears in econometrics!<br />
<br />
== Triangular matrices ==<br />
<br />
A square ''lower triangular ''matrix has all elements above the main diagonal equal to zero, whilst a square ''upper triangular ''matrix has all elements below the main diagonal equal to zero. A simple example of a lower triangular matrix is: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & 0 & 0\\<br />
a_{21} & a_{22} & 0\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right].</math> Clearly, for this matrix, <math>A^{T}</math> is an upper triangular matrix.<br />
<br />
One can adapt the definition to rectangular matrices: for example, if two arbitrary rows are added to <math>A,</math> so that it becomes <math>5\times3,</math> it would still be considered lower triangular. Equally, if, for example, the third column of <math>A</math> above is removed, <math>A</math> is still considered lower triangular.<br />
<br />
Often, we use ''unit ''triangular matrices, where the diagonal elements are all equal to <math>1:</math> e.g.:<br />
<br />
<math>\left[\begin{array}{rrr}<br />
1 & 2 & 0\\<br />
0 & 1 & 1\\<br />
0 & 0 & 1<br />
\end{array}\right].\label{eq:lt_matrix}</math><br />
<br />
== Partitioned matrices ==<br />
<br />
Sometimes, especially with big matrices, it is useful to organise the elements of the matrix into components which are themselves matrices, for example: <math>B=\left[\begin{array}{rrrr}<br />
1 & 2 & 0 & 0\\<br />
8 & 3 & 0 & 0\\<br />
0 & 0 & 7 & 4\\<br />
0 & 0 & 6 & 5<br />
\end{array}\right]</math> Here it would be reasonable to write: <math>B=\left[\begin{array}{cc}<br />
B_{11} & 0\\<br />
0 & B_{22}<br />
\end{array}\right],</math> where <math>B_{ii},i=1,2,</math> represent <math>2\times2</math> matrices. <math>B</math> is an example of a ''partitioned matrix'': that is, an <math>m\times n</math> matrix <math>A</math> say: <math>A=\left\Vert a_{ij}\right\Vert ,</math> where the elements of <math>A</math> are organised into ''sub-matrices''. An example might be:<br />
<br />
<math>A=\left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right],\label{eq:partition_a}</math><br />
<br />
where the ''sub - matrices'' in the first row block have <math>r</math> rows, and therefore <math>m-r</math> rows in the second row block. The column blocks might be defined by (for example) 3 columns in the first column block, 4 in the second and <math>n-7</math> in the third column block.<br />
<br />
Another simple example might be: <math>A=\left[\begin{array}{rrr}<br />
A_{1} & A_{2} & A_{3}\end{array}\right],\ \ \ \ \ \mathbf{x=}\left[\begin{array}{c}<br />
\mathbf{x}_{1}\\<br />
\mathbf{x}_{2}\\<br />
\mathbf{x}_{3}<br />
\end{array}\right],</math> where <math>A</math> and therefore <math>A_{1},A_{2},A_{3}</math> have <math>m</math> rows, <math>A_{1}</math> has <math>n_{1}</math> columns, <math>A_{2}</math> has <math>n_{2}</math> columns, <math>A_{3}</math> has <math>n_{3}</math> columns. The ''subvectors'' in <math>\mathbf{x}</math> must have <math>n_{1},n_{2}</math> and <math>n_{3}</math> rows respectively, for the product <math>A\mathbf{x}</math> to exist.<br />
<br />
Suppose that <math>n_{1}+n_{2}+n_{3}=n,</math> so that <math>A</math> is <math>m\times n.</math> The <math>i</math>th element of <math>A\mathbf{x}</math> is: <math>\sum_{i=1}^{n}a_{ij}x_{j}</math> but the summation can be broken up into the first <math>n_{1}</math> terms: <math>\sum_{i=1}^{n_{1}}a_{ij}x_{j},</math> the next <math>n_{2}</math> terms: <math>\sum_{i=n_{1}+1}^{n_{1}+n_{2}}a_{ij}x_{j},</math> and the next <math>n_{3}</math> terms; <math>\sum_{i=n_{1}+n_{2}+1}^{n}a_{ij}x_{j}.</math><br />
<br />
The point about the use of partitioned matrices is that the product <math>A\mathbf{x}</math> can be represented as: <math>A\mathbf{x}=A_{1}\mathbf{x}_{1}+A_{2}\mathbf{x}_{2}+A\mathbf{x}_{3}</math> by applying the across and down rule to the submatrices and the subvectors, a much simpler representation than the use of summations.<br />
<br />
Each of the components is a conformable matrix-vector product: this is essential in any use of partitioned matrices to represent some matrix product. For example, using <math>A</math> from equation (8) and <math>B</math> as: <math>B=\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right],</math> it is easy to write:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right]\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{r}<br />
A_{11}B_{11}+A_{12}B_{21}+A_{13}B_{31}\\<br />
A_{21}B_{11}+A_{22}B_{21}+A_{23}B_{31}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
But, what are the row dimensions for the submatrices in <math>B?</math> What are the possible column dimensions for the submatrices in <math>B?</math><br />
<br />
= Matrices, vectors and econometrics =<br />
<br />
The data on weights and heights for 12 students in the data matrix: <math>D=\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]</math> would seem to be ideally suited for fitting a two variable regression model: <math>y_{i}=\alpha+\beta x_{i}+u_{i},\;\;\;\;\; i=1,...,12.</math> Here, the first column of <math>D</math> contains all the weight data, the data on the dependent variable <math>y_{i},</math> and so should be labelled <math>\mathbf{y.}</math> The second column of <math>D</math> contains all the data on the explanatory variable height, in the vector <math>\mathbf{x}</math> say, so that: <math>D=\left[\begin{array}{rr}<br />
\mathbf{y} & \mathbf{x}\end{array}\right].</math><br />
<br />
If we define a <math>12\times1</math> vector with every element <math>1</math>: <math>\mathbf{1}_{12}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right],</math> and a <math>12\times1</math> vector <math>\mathbf{u}</math> to contain the error terms; <math>\mathbf{u}=\left[\begin{array}{c}<br />
u_{1}\\<br />
\vdots\\<br />
u_{12}<br />
\end{array}\right]</math> the regression model can be written in terms of the three data vectors <math>\mathbf{y,1}_{12}</math> and <math>\mathbf{x}</math> as: <math>\mathbf{y}=\mathbf{1}_{12}\alpha+\mathbf{x}\beta+\mathbf{u.}</math> To see this, think of the <math>i</math>th elements of the vectors on the left and right hand sides.<br />
<br />
The standard next step is then to combine the data vectors for the explanatory variables into a matrix: <math>X=\left[\begin{array}{rr}<br />
\mathbf{1}_{12} & \mathbf{x}\end{array}\right],</math> and then define a <math>2\times1</math> vector <math>\boldsymbol{\delta}</math> to contain the parameters <math>\alpha,\beta</math> as: <math>\boldsymbol{\delta}=\left[\begin{array}{r}<br />
\alpha\\<br />
\beta<br />
\end{array}\right]</math> to give the data matrix representation of the regression model as: <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
<br />
For the purposes of developing the theory of regression, this is the most convenient form of the regression model. It can represent regression models with any number of explanatory variables, and thus any number of parameters. The obvious point is that a knowledge of vector and matrix operations is needed to use and understand this form.<br />
<br />
We shall see later that there are two particular matrix and vector quantities associated with a regression model. The first is the matrix <math>X^{T}X,</math> and the second the vector <math>X^{T}\mathbf{y.}</math> The following Matlab code snippet provides the numerical values of these quantities for the weight data:<br />
<br />
&gt;&gt; dset = load(’weights.mat’); <br />
<br />
&gt;&gt; xtx = dset.X’ * dset.X; <br />
<br />
&gt;&gt; xty = dset.X’ * dset.y; <br />
<br />
&gt;&gt; disp(xtx) <br />
<br />
12 802<br />
<br />
802 53792<br />
<br />
&gt;&gt; disp(xty)<br />
<br />
1850<br />
<br />
124528<br />
<br />
Hand calculation is of course possible, but not recommended.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=File:Orthy_example.png&diff=3023File:Orthy example.png2013-09-10T13:49:36Z<p>LG: For Lecture 2, Figure 1</p>
<hr />
<div>For Lecture 2, Figure 1</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=LNotes&diff=3022LNotes2013-09-10T13:46:43Z<p>LG: Created page with "= Matrices = In the PreSession Maths course, a matrix was defined as follows: <blockquote>A matrix is a rectangular array of numbers enclosed in parentheses, con- ventional..."</p>
<hr />
<div>= Matrices =<br />
<br />
In the PreSession Maths course, a matrix was defined as follows:<br />
<br />
<blockquote>A matrix is a rectangular array of numbers enclosed in parentheses, con-<br />
<br />
ventionally denoted by a capital letter. The number of rows (say <math>m</math>) and<br />
<br />
the number of columns (say <math>n</math>) determine the order of the matrix (<math>m</math> <math>\times</math> <math>n</math>).<br />
</blockquote><br />
Two examples were given:<br />
<br />
<math>\begin{aligned}<br />
P & =\left[\begin{array}{rrr}<br />
2 & 3 & 4\\<br />
3 & 1 & 5<br />
\end{array}\right],\ \ \ Q=\left[\begin{array}{rr}<br />
2 & 3\\<br />
4 & 3\\<br />
1 & 5<br />
\end{array}\right],\end{aligned}</math><br />
<br />
matrices of dimensions <math>2\times3</math> and <math>3\times2</math> respectively.<br />
<br />
Why study matrices for econometrics? Basically because a data set of several variables, e.g. on the weights and heights of 12 students, can be thought of as a matrix:<br />
<br />
<math>\begin{aligned}<br />
D & =\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]\end{aligned}</math><br />
<br />
The properties of matrices can then be used to facilitate answering all the usual questions of econometrics - list not given here!<br />
<br />
Calculations with matrices with explicit numerical elements, as in the examples above is called matrix ''arithmetic''. Matrix ''algebra'' is the algebra of matrices where the elements are not made explicit: this is what is really required for econometrics, as we shall see.<br />
<br />
As an example of this, a <math>2\times3</math> matrix might be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{ccc}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\end{aligned}</math><br />
<br />
and would equal <math>P</math> above if the collection of <math>a_{ij}</math> were given appropriate numerical values.<br />
<br />
A general <math>m\times n</math> matrix <math>A</math> can be written as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
There is also a ''typical element ''notation for matrices:<br />
<br />
<math>\begin{aligned}<br />
A & =\left\Vert a_{ij}\right\Vert ,\ \ \ \ \ i=1,...,m,j=1,...,n,\end{aligned}</math><br />
<br />
so that <math>a_{ij}</math> is the element at the intersection of the <math>i</math>th row and <math>j</math>th column in <math>A.</math><br />
<br />
When <math>m\neq n,</math> <math>A</math> is a ''rectangular ''matrix; when <math>m=n,</math> <math>A</math> is <math>m\times m</math> or <math>n\times n,</math> and <math>A</math> is a square matrix, having the same number of rows or columns.<br />
<br />
== Rows, columns and vectors ==<br />
<br />
Clearly, there is no reason why <math>m</math> or <math>n</math> cannot equal 1: so, an <math>m\times n</math> matrix with <math>n=1,</math> i.e. with one column, is usually called a column vector. Similarly, a matrix with one row is a row vector.<br />
<br />
There are a lot of advantages to thinking of matrices as collections of row or column vectors, as we shall see. As an example, define the <math>2\times1</math> column vectors:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a} & =\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\mathbf{,\ \ \ b}=\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
and arrange as the columns of the <math>2\times2</math> matrix<br />
<br />
<math>A=\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\label{eq:axy}</math><br />
<br />
In general, a column vector <math>\mathbf{x}</math> with <math>n</math> elements can be written as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\end{aligned}</math><br />
<br />
What happens when both <math>m</math> and <math>n</math> are equal to <math>1?</math> Then, <math>A</math> is a <math>1\times1</math> matrix, but it is also considered to be a real number, or ''scalar'' in the language of linear algebra:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[a_{11}\right]=a_{11}.\end{aligned}</math><br />
<br />
This is perhaps a little odd, but turns out to be a useful convention in a number of situations.<br />
<br />
== Transposition of vectors ==<br />
<br />
The ''rows'' of the matrix <math>A</math> in equation (1) can be seen as elements of column vectors, say:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c} & =\left[\begin{array}{r}<br />
6\\<br />
2<br />
\end{array}\right],\ \ \ \boldsymbol{d}=\left[\begin{array}{r}<br />
3\\<br />
5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
This representation of row vectors as column vectors is a bit clumsy, so some transformation which converts a column vector into a row vector, and vice versa would be useful. The process of converting a column vector <math>\mathbf{a}</math> into a row vector is called ''transposition, ''and the transposed version of <math>\mathbf{c}</math> is denoted:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{c}^{T} & =\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\end{aligned}</math><br />
<br />
the <math>^{T}</math> superscript denoting transposition. In practice, a prime, <math>^{\prime},</math> is used instead of <math>^{T}.</math> However, whilst the prime is much simpler to write than the <math>^{T}</math> sign, it is also much easier to lose track of in writing out long or complicated expressions. So, it is best initially to use <math>^{T}</math> to denote transposition rather than the prime <math>^{\prime}.</math><br />
<br />
<math>A</math> can then be written via its rows as:<br />
<br />
<math>\begin{aligned}<br />
A & =\left[\begin{array}{r}<br />
\mathbf{c}^{T}\\<br />
\boldsymbol{d}^{T}<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right].\end{aligned}</math><br />
<br />
The same ideas can be applied to the matrices <math>P</math> and <math>Q.</math><br />
<br />
= Operations with matrices =<br />
<br />
== Addition, subtraction and scalar multiplication ==<br />
<br />
For vectors, addition and subtraction are defined only for vectors of the same dimensions. If:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & =\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{y}=\left[\begin{array}{c}<br />
y_{1}\\<br />
\vdots\\<br />
y_{n}<br />
\end{array}\right],\\<br />
\mathbf{x+y} & =\left[\begin{array}{c}<br />
x_{1}+y_{1}\\<br />
\vdots\\<br />
x_{n}+y_{n}<br />
\end{array}\right],\,\,\,\,\mathbf{x-y}=\left[\begin{array}{c}<br />
x_{1}-y_{1}\\<br />
\vdots\\<br />
x_{n}-y_{n}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Clearly, the addition or subtraction operation is ''elementwise. ''If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> have different dimensions, there will be some elements left over once all the elements of the smaller dimensioned vector have been used up.<br />
<br />
Another operation is ''scalar multiplication'': if <math>\lambda</math> is a real number or scalar, the product <math>\lambda\mathbf{x}</math> is defined as: <math>\lambda\mathbf{x}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right],</math> so that every element of <math>\mathbf{x}</math> is multiplied by the same scalar <math>\lambda.</math><br />
<br />
The two types of operation can be combined into the ''linear combination'' of vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>: <math>\lambda\mathbf{x}+\mu\mathbf{y}=\left[\begin{array}{c}<br />
\lambda x_{1}\\<br />
\vdots\\<br />
\lambda x_{n}<br />
\end{array}\right]+\left[\begin{array}{c}<br />
\mu y_{1}\\<br />
\vdots\\<br />
\mu y_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\lambda x_{1}+\mu y_{1}\\<br />
\vdots\\<br />
\lambda x_{n}+\mu y_{n}<br />
\end{array}\right].</math> Equally, one can define the linear combination of vectors <math>\mathbf{x,y,}\ldots,\mathbf{z}</math> by scalars <math>\lambda,\mu,\ldots,\nu</math> as: <math>\lambda\mathbf{x}+\mu\mathbf{y}+\ldots+\nu\mathbf{z}</math> with typical element: <math>\lambda x_{i}+\mu y_{i}+\ldots+\nu z_{i},</math> provided that all the vectors have the same dimension.<br />
<br />
For matrices, these ideas carry over immediately: apply to each column of the matrices involved. For example, if <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right]</math> and <math>B=\left[\begin{array}{rrr}<br />
\mathbf{b}_{1} & \ldots & \mathbf{b}_{n}\end{array}\right],</math> both <math>m\times n,</math> then addition and subtraction are defined elementwise, as for vectors:<br />
<br />
<math>\begin{aligned}<br />
A+B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}+\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}+\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}+b_{ij}\right\Vert ,\\<br />
A-B & = & \left[\begin{array}{rrr}<br />
\mathbf{a}_{1}-\mathbf{b}_{1} & \ldots & \mathbf{a}_{n}-\mathbf{b}_{n}\end{array}\right]=\left\Vert a_{ij}-b_{ij}\right\Vert .\end{aligned}</math><br />
<br />
Scalar multiplication of <math>A</math> by <math>\lambda</math> involves multiplying every column vector of <math>A</math> by <math>\lambda,</math> and therefore multiplying every element of <math>A</math> by <math>\lambda</math>: <math>\lambda A=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1} & \ldots & \lambda\mathbf{a}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}\right\Vert .</math> With the same idea for <math>B,</math> the linear combination of <math>A</math> and <math>B</math> by <math>\lambda</math> and <math>\mu</math> is: <math>\lambda A+\mu B=\left[\begin{array}{rrr}<br />
\lambda\mathbf{a}_{1}+\mu\mathbf{b}_{1} & \ldots & \lambda\mathbf{a}_{n}+\mu\mathbf{b}_{n}\end{array}\right]=\left\Vert \lambda a_{ij}+\mu b_{ij}\right\Vert .</math><br />
<br />
For example, consider the matrices: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right]</math> with <math>\lambda=1,</math> <math>\mu=-2:</math> then:<br />
<br />
<math>\begin{aligned}<br />
\lambda A+\mu B & = & A-2B\\<br />
& = & \left[\begin{array}{cc}<br />
4 & 0\\<br />
1 & 7<br />
\end{array}\right].\end{aligned}</math><br />
<br />
== Matrix - vector products ==<br />
<br />
=== Inner product ===<br />
<br />
The simplest form of a matrix vector product is the case where <math>A</math> consists of one row, so that <math>A</math> is <math>1\times n</math>: <math>A=\mathbf{a}^{T}=\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right].</math> If <math>\mathbf{x}</math> is an <math>n\times1</math> vector: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],</math> the product <math>A\mathbf{x}=\mathbf{a}^{T}\mathbf{x}</math> is called the ''inner product'' and is defined as:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{a}^{T}\mathbf{x} & =a_{1}x_{1}+\ldots+a_{n}x_{n}.\end{aligned}</math><br />
<br />
One can see that the definition amounts to multiplying corresponding elements in <math>\mathbf{a}</math> and <math>\mathbf{x,}</math> and adding up the resultant products. Writing: <math>\mathbf{a}^{T}\mathbf{x=}\left[\begin{array}{rrr}<br />
a_{1} & \ldots & a_{n}\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=a_{1}x_{1}+\ldots+a_{n}x_{n}</math> motivates the familiar description of the ''across and down rule ''for this product: ''across and down'' is the ’multiply corresponding elements’ part of the definition.<br />
<br />
Notice that the result of the inner product is a real number, for example: <math>\mathbf{c}^{T}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right],\ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{c}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
6 & 2\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=36+6=42.</math><br />
<br />
In general, in the product <math>\mathbf{a}^{T}\mathbf{x,}</math> <math>\mathbf{a}</math> and <math>\mathbf{x}</math> must have the same number of elements, <math>n</math> say, for the product to be defined. If <math>\mathbf{a}</math> and <math>\mathbf{x}</math> had different numbers of elements, there would be some elements of <math>\mathbf{a}</math> or <math>\mathbf{x}</math> left over or not used in the product: e.g.: <math>\mathbf{b}=\left[\begin{array}{c}<br />
1\\<br />
2\\<br />
3<br />
\end{array}\right],\ \ \ \mathbf{x=}\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> When the inner product of two vectors is defined, the vectors are said to be ''conformable''.<br />
<br />
== Orthogonality ==<br />
<br />
Two vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math> with the property that <math>\mathbf{x}^{T}\mathbf{y}=0</math> are said to be orthogonal to each other. For example, if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
-1\\<br />
1<br />
\end{array}\right],</math> it is clear that <math>\mathbf{x}^{T}\mathbf{y}=0.</math> This seems a rather innocuous definition, and yet the idea of orthogonality turns out to be extremely important in econometrics.<br />
<br />
If <math>\mathbf{x}</math> and <math>\mathbf{y}</math> are thought of as points in <math>R^{2},</math> and arrows are drawn from the origin to <math>\mathbf{x}</math> and to <math>\mathbf{y,}</math> then the two arrows are perpendicular to each other - see Figure [orthy<sub>e</sub>xample]. If <math>\mathbf{y}</math> were defined as: <math>\mathbf{y}=\left[\begin{array}{r}<br />
1\\<br />
-1<br />
\end{array}\right],</math> the position of the <math>\mathbf{y}</math> vector and the corresponding arrow would change, but the perpendicularity property would still hold.<br />
<br />
[ht] [[Image:0C__courses_econometric_methods_yed_orthy_example.pdf|fig:]]<br />
<br />
[orthy<sub>e</sub>xample]<br />
<br />
=== Matrix - vector products ===<br />
<br />
Since the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> has two rows, now denoted <math>\boldsymbol{\alpha}_{1}^{T}</math> and <math>\boldsymbol{\alpha}_{2}^{T},</math> there are two possible inner products with the vector:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{x} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]:\\<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x} & = & 42,\ \ \ \ \ \boldsymbol{\alpha}_{2}^{T}\mathbf{x}=33.\end{aligned}</math><br />
<br />
Assembling the two inner product values into a <math>2\times1</math> vector defines the product of the matrix <math>A</math> with the vector <math>\mathbf{x}</math>: <math>A\mathbf{x}=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
\boldsymbol{\alpha}_{1}^{T}\mathbf{x}\\<br />
\boldsymbol{\alpha}_{2}^{T}\mathbf{x}<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right].</math> Focussing only on the part: <math>\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=\left[\begin{array}{r}<br />
42\\<br />
33<br />
\end{array}\right],</math> one can see that each element of <math>A\mathbf{x}</math> is obtained from an ''across and down'' argument.<br />
<br />
Sometimes this product is described as forming a ''linear combination ''of the columns of <math>A</math> using the scalar elements in <math>\mathbf{x}</math>: <math>A\mathbf{x}=6\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]+3\left[\begin{array}{r}<br />
2\\<br />
5<br />
\end{array}\right].</math> More generally, if:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right],\ \ \ \ \ \mathbf{x}=\left[\begin{array}{r}<br />
\lambda\\<br />
\mu<br />
\end{array}\right],\\<br />
A\mathbf{x} & = & \lambda\mathbf{a}+\mu\mathbf{b.}\end{aligned}</math><br />
<br />
The general version of these ideas for an <math>m\times n</math> matrix <math>A</math>: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]=\left[\begin{array}{rrrr}<br />
\mathbf{a}_{1} & \mathbf{a}_{2} & \ldots & \mathbf{a}_{n}\end{array}\right].</math> is straightforward. If <math>\mathbf{x}</math> is an <math>n\times1</math> vector, then the vector <math>A\mathbf{x}</math> is, by the ''across and down'' rule:<br />
<br />
<math>A\mathbf{x}=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{c}<br />
x_{1}\\<br />
x_{2}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
a_{11}x_{1}+\ldots+a_{1n}x_{n}\\<br />
a_{21}x_{1}+\ldots+a_{2n}x_{n}\\<br />
\vdots\\<br />
a_{m1}x_{1}+\ldots+a_{mn}x_{n}<br />
\end{array}\right]=\left[\begin{array}{c}<br />
\sum\limits _{j=1}^{n}a_{1j}x_{j}\\<br />
\sum\limits _{j=1}^{n}a_{2j}x_{j}\\<br />
\vdots\\<br />
\sum\limits _{j=1}^{n}a_{mj}x_{j}<br />
\end{array}\right],\label{eq:ab}</math><br />
<br />
so that the typical element, the <math>i</math>th, is <math>\sum\limits _{j=1}^{n}a_{ij}x_{j}.</math> Equally, <math>A\mathbf{x}</math> is the linear combination <math>\mathbf{a}_{1}x_{1}+\ldots+\mathbf{a}_{n}x_{n}</math> of the columns of <math>A.</math><br />
<br />
== Matrix - matrix products ==<br />
<br />
Suppose that <math>A</math> is <math>m\times n,</math> with columns <math>\mathbf{a}_{1},\ldots,\mathbf{a}_{n},</math> and <math>B</math> is <math>n\times r,</math> with columns <math>\mathbf{b}_{1},\ldots,\mathbf{b}_{r}.</math> Clearly, each product <math>A\mathbf{b}_{1},...,A\mathbf{b}_{r}</math> exists, and is <math>m\times1.</math> These products can be arranged as the columns of a matrix as <math>\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]</math> and this matrix is ''defined'' to be the product <math>C</math> of the matrices <math>A</math> and <math>B</math>: <math>C=\left[\begin{array}{rrrr}<br />
A\mathbf{b}_{1} & A\mathbf{b}_{2} & \ldots & A\mathbf{b}_{r}\end{array}\right]=AB.</math> By construction, this must be an <math>m\times r</math> matrix, since each column is <math>m\times1</math> and there are <math>r</math> columns.<br />
<br />
This is not the usual presentation of the definition of the product of two matrices, which relies on the ''across and down rule'' mentioned earlier, and focusses on the elements of each matrix <math>A</math> and <math>B.</math> Set:<br />
<br />
<math>\begin{aligned}<br />
B & = & \left[\begin{array}{rrrr}<br />
\mathbf{b}_{1} & \mathbf{b}_{2} & \ldots & \mathbf{b}_{r}\end{array}\right]\text{\,\,\,\,\,\,\,(by columns)}\\<br />
& = & \left\Vert b_{ik}\right\Vert ,\ \ \ \ \ i=1,...,n,k=1,...,r\text{ \,\,\,\,\,\,(typical element)}\\<br />
& = & \left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2r}\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nr}<br />
\end{array}\right]\text{\,\,\,\,\,\,\,(the array)}\end{aligned}</math><br />
<br />
What does the typical element of the <math>m\times r</math> matrix <math>C</math> look like? Start with the <math>k</math>th column of <math>C,</math> which is <math>A\mathbf{b}_{k}.</math> The <math>i</math>th element in <math>A\mathbf{b}_{k}</math> is, from equation ([eq:ab]), the inner product of the elements of the <math>i</math>th row in <math>A</math>: <math>\left[\begin{array}{rrrr}<br />
a_{i1} & a_{i2} & \ldots & a_{in}\end{array}\right],</math> with the elements of <math>\mathbf{b}_{k},</math> so that the inner product is: <math>a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math><br />
<br />
So, the <math>ik</math>th element of <math>C</math> is: <math>c_{ik}=a_{i1}b_{1k}+a_{i2}b_{2k}+\ldots+a_{in}b_{nk}=\sum_{j=1}^{n}a_{ij}b_{jk}.</math> We can see this arising from an ''across and down'' calculation by writing:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\label{eq:c_ab}\\<br />
& = & \left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
a_{21} & a_{22} & \ldots & a_{2n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right]\left[\begin{array}{rrrrrr}<br />
b_{11} & b_{12} & \ldots & b_{1k} & \ldots & b_{1r}\\<br />
b_{21} & b_{22} & \ldots & b_{2k} & \ldots & b_{2r}\\<br />
\vdots & \vdots & & \vdots & & \vdots\\<br />
b_{n1} & b_{n2} & \ldots & b_{nk} & \ldots & b_{nr}<br />
\end{array}\right]\\<br />
& = & \left\Vert \sum_{j=1}^{n}a_{ij}b_{jk}\right\Vert .\end{aligned}</math><br />
<br />
These ideas are simple, but a little tedious. Numerical examples are equally tedious! As an example, using: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],</math> we can find the matrix <math>B</math> such that<br />
<br />
# the first column of <math>AB</math> adds together the columns of <math>A,</math><br />
# the second column is the difference of the first and second columns of <math>A,</math><br />
# the third column is <math>2\times</math> the first column of <math>A,</math><br />
# the fourth column is zero.<br />
<br />
It is easy to check that <math>B</math> is: <math>B=\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]</math> and that:<br />
<br />
<math>\begin{aligned}<br />
C & = & AB\\<br />
& = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{rrrr}<br />
1 & 1 & 2 & 0\\<br />
1 & -1 & 0 & 0<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{cccc}<br />
8 & 4 & 12 & 0\\<br />
8 & -2 & 6 & 0<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Arithmetic calculations of matrix products almost always use the elementwise across and down formula. However, there are many situations in econometrics where algebraic rather than arithmetic arguments are required. In these cases, the viewpoint of matrix multiplication as linear combinations of columns is much more powerful.<br />
<br />
Clearly one can give many more examples of different dimensions and complexities - but the same basic rules apply. To multiply two matrices <math>A</math> and <math>B</math> together, the number of columns in <math>A</math> must match the number of rows in <math>B</math> - this is ''conformability'' in action again. The resulting product will have number of rows equal to the number in <math>A</math> and number of columns equal to the number in <math>B.</math><br />
<br />
If this conformability rule does not hold, then the product of <math>A</math> and <math>B</math> is not defined.<br />
<br />
== Matlab ==<br />
<br />
One should also say that as the dimensions of the matrices increases, so the tediousness of the calculations increase. The solution to this for numerical calculation is to appeal to the computer. Programs like Matlab and Excel (and a number of others, some of them free) resolve this difficulty easily.<br />
<br />
In Matlab, symbols for row or column vectors do not need any particular differentiation: they are distinguished by how they are defined. For example, the following Matlab commands define <code>rowvec </code>as a <math>1\times4</math> vector, and <code>colvec</code> as a <math>4\times1</math> vector, then display the contents of these variables, and do a calculation:<br />
<br />
&gt;&gt; rowvec = [1 2 3 4]; <br />
<br />
&gt;&gt; colvec = [1;2;3;4]; <br />
<br />
&gt;&gt; rowvec<br />
<br />
rowvec =<br />
<br />
1 2 3 4<br />
<br />
&gt;&gt; colvec<br />
<br />
colvec =<br />
<br />
1 2 3 4 <br />
<br />
&gt;&gt; rowvec*colvec<br />
<br />
ans =<br />
<br />
30 <br />
<br />
So, the semi-colon indicates the end of a row in a matrix or vector; it can be replaced by a carriage return. Notice the difference in how a row vector and a column vector is defined. One can see that the product <code>rowvec*colvec</code> is well defined, just because <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector.<br />
<br />
Matlab also allows elementwise multiplication of two vectors using the <math>\centerdot\ast</math> operator: if: <math>\mathbf{x}=\left[\begin{array}{r}<br />
x_{1}\\<br />
x_{2}<br />
\end{array}\right],\ \ \ \mathbf{y}=\left[\begin{array}{r}<br />
y_{1}\\<br />
y_{2}<br />
\end{array}\right],</math> then: <math>\mathbf{x}\centerdot\ast\mathbf{y}=\left[\begin{array}{r}<br />
x_{1}y_{1}\\<br />
x_{2}y_{2}<br />
\end{array}\right]</math> and one can see that the inner product of <math>\mathbf{x}</math> and <math>\mathbf{y}</math> can be obtained as the sum of the elements of <math>\mathbf{x}</math> and <math>\mathbf{y.}</math> In Matlab, this would be obtained as: <math>\text{sum}\left(\mathbf{x}\centerdot\ast\mathbf{y}\right).</math><br />
<br />
In the example above, this calculation fails since <code>rowvec </code>is a <math>1\times4</math> vector, and <code>colvec</code> is a <math>4\times1</math> vector:<br />
<br />
&gt;&gt; sum(rowvec .* colvec) ??? <br />
<br />
Error using ==&gt; times Matrix dimensions must agree. <br />
<br />
For this to work, <code>rowvec</code> would have to be transposed as <code>rowvec’</code>, so that transposition in Matlab is very natural.<br />
<br />
Allowing for such difficulties, matrix multiplication in Matlab is very simple:<br />
<br />
&gt;&gt; A = [6 2; 3 5];<br />
<br />
&gt;&gt; B = [1 1 2 0;1 -1 0 0];<br />
<br />
&gt;&gt; C = A * B; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
8 4 1<br />
<br />
2 0 8 <br />
<br />
-2 6 0 <br />
<br />
Notice how the matrices are defined here through their rows. The <code>disp() </code>command displays the contents of the object referred to.<br />
<br />
It is less natural in Matlab to define matrices by columns - a typical example of how mathematics and computing have conflicts of notation. However, once columns <math>\mathbf{a}</math> and <math>\mathbf{b}</math> have been defined, the concatenation operation <math>\left[\begin{array}{rr}<br />
\mathbf{a} & \mathbf{b}\end{array}\right]</math> collects the columns into a matrix:<br />
<br />
&gt;&gt; a = [6;2]; <br />
<br />
&gt;&gt; b = [3;5]; <br />
<br />
&gt;&gt; C = [a b]; <br />
<br />
&gt;&gt; disp(C)<br />
<br />
6 3 <br />
<br />
2 5 <br />
<br />
Notice that the <code>disp(C)</code> command does not label the result that is printed out. Simply typing <code>C</code> would preface the output by <code>C =</code>.<br />
<br />
== Pre and Post Multiplication ==<br />
<br />
If <math>C=AB,</math> as above, say that <math>B</math> is ''pre-multiplied ''by <math>A</math> to get <math>C,</math> and that <math>A</math> is ''post-multiplied ''by <math>B</math> to get <math>C.</math><br />
<br />
This distinction between ''pre ''and ''post ''multiplication is important, in the following sense. Suppose that <math>A</math> and <math>B</math> are matrices such that the products <math>AB</math> and <math>BA</math> are both defined. If <math>A</math> is <math>m\times n,</math> <math>B</math> must have <math>n</math> rows for <math>AB</math> to be defined. For <math>BA</math> to be defined, <math>B</math> must have <math>m</math> columns to match the <math>m</math> rows in <math>A.</math> So, <math>AB</math> and <math>BA</math> are both defined if <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times m.</math><br />
<br />
Even when both products are defined, there is no reason for the two products coincide. The first thing to notice is that <math>AB</math> is a square, <math>m\times m,</math> matrix, whilst <math>BA</math> is a square, <math>n\times n,</math> matrix. Different sized matrices cannot be equal. To illustrate, use the matrices: <math>B_{2}=\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right],\ \ \ C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]:</math><br />
<br />
<math>\begin{aligned}<br />
B_{2}C & = & \left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]=\left[\begin{array}{rrr}<br />
27 & -3 & -15\\<br />
27 & 29 & -11\\<br />
-15 & -1 & 8<br />
\end{array}\right],\\<br />
CB_{2} & = & \left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & -3\\<br />
2 & 5\\<br />
-3 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
49 & -11\\<br />
31 & 15<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Even when <math>m=n,</math> so that <math>AB</math> and <math>BA</math> are both <math>m\times m</math> matrices, the products can differ: for example:<br />
<br />
<math>\begin{aligned}<br />
A & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{cc}<br />
1 & 1\\<br />
1 & -1<br />
\end{array}\right],\\<br />
AB & = & \left[\begin{array}{cc}<br />
8 & 4\\<br />
8 & -2<br />
\end{array}\right],\ \ \ \ \ BA=\left[\begin{array}{cc}<br />
9 & 7\\<br />
3 & -3<br />
\end{array}\right].\end{aligned}</math><br />
<br />
In cases where <math>AB=BA,</math> the matrices <math>A</math> and <math>B</math> are said to ''commute''.<br />
<br />
== Transposition ==<br />
<br />
A column vector <math>\mathbf{x}</math> can be converted to a row vector <math>\mathbf{x}^{T}</math> by transposition: <math>\mathbf{x}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right],\ \ \ \ \ \mathbf{x}^{T}=\left[\begin{array}{rrr}<br />
x_{1} & \ldots & x_{n}\end{array}\right].</math> Transposing <math>\mathbf{x}^{T}</math> as <math>\left(\mathbf{x}^{T}\right)^{T}</math> reproduces the original vector <math>\mathbf{x.}</math> How do these ideas carry over to matrices?<br />
<br />
If the <math>m\times n</math> matrix <math>A</math> can be written as <math>A=\left[\begin{array}{rrr}<br />
\mathbf{a}_{1} & \ldots & \mathbf{a}_{n}\end{array}\right],</math> the transpose of <math>A,</math> <math>A^{T},</math> is defined as the matrix whose ''rows'' are <math>\mathbf{a}_{i}^{T}</math>: <math>A^{T}=\left[\begin{array}{c}<br />
\mathbf{a}_{1}^{T}\\<br />
\vdots\\<br />
\mathbf{a}_{n}^{T}<br />
\end{array}\right].</math> In terms of elements, if: <math>\mathbf{a}_{i}=\left[\begin{array}{c}<br />
a_{1i}\\<br />
\vdots\\<br />
a_{ni}<br />
\end{array}\right]</math> then: <math>A=\left[\begin{array}{rrrr}<br />
a_{11} & a_{12} & \ldots & a_{1n}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{i1} & a_{i2} & \ldots & a_{in}\\<br />
\vdots & \vdots & & \vdots\\<br />
a_{m1} & a_{m2} & \ldots & a_{mn}<br />
\end{array}\right],\ \ \ \ \ A^{T}=\left[\begin{array}{rrrrr}<br />
a_{11} & \ldots & a_{i1} & \ldots & a_{m1}\\<br />
a_{12} & \ldots & a_{i2} & \ldots & a_{m2}\\<br />
\vdots & & \vdots & & \vdots\\<br />
a_{1n} & \ldots & a_{in} & \ldots & a_{mn}<br />
\end{array}\right].</math> One can see that the first column of <math>A</math> has now become the first row of <math>A^{T}.</math> Notice too that <math>A^{T}</math> is an <math>n\times m</math> matrix if <math>A</math> is an <math>m\times n</math> matrix.<br />
<br />
Transposing <math>A^{T}</math> takes the first column of <math>A^{T}</math> and writes it as a row, which coincides with the first row of <math>A.</math> The same argument applies to the other columns of <math>A^{T},</math> so that <math>\left(A^{T}\right)^{T}=A.</math><br />
<br />
=== The product rule for transposition ===<br />
<br />
This states that if <math>C=AB,</math> then <math>C^{T}=B^{T}A^{T}.</math><br />
<br />
How to see this? Consider the following example: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}<br />
\end{array}\right],\ \ \ B=\left[\begin{array}{rrrr}<br />
b_{11} & b_{12} & b_{13} & b_{14}\\<br />
b_{21} & b_{22} & b_{23} & b_{24}\\<br />
b_{31} & b_{32} & b_{33} & b_{34}<br />
\end{array}\right]</math> where:<br />
<br />
<math>c_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=\sum_{k=1}^{3}a_{2k}b_{k3}.\label{eq:c23}</math><br />
<br />
One can see that: <math>B^{T}A^{T}=\left[\begin{array}{rrr}<br />
b_{11} & b_{21} & b_{31}\\<br />
b_{12} & b_{22} & b_{32}\\<br />
b_{13} & b_{23} & b_{33}\\<br />
b_{14} & b_{24} & b_{34}<br />
\end{array}\right]\left[\begin{array}{rr}<br />
a_{11} & a_{21}\\<br />
a_{12} & a_{22}\\<br />
a_{13} & a_{23}<br />
\end{array}\right]</math> and that the <math>\left(3,2\right)</math> element of this product is actually <math>c_{23}</math>: <math>b_{13}a_{21}+b_{23}a_{22}+b_{33}a_{23}=a_{21}b_{13}+a_{22}b_{23}+a_{23}b_{33}=c_{23}.</math> In summation notation, we see that from <math>B^{T}A^{T}</math>: <math>c_{23}=\sum_{k=1}^{3}b_{k3}a_{2k},</math> where the position of the index of summation is due to the transposition. So, in summation notation, the calculation of <math>c_{23}</math> from <math>B^{T}A^{T}</math> equals that from equation ([eq:c23]).<br />
<br />
More generally, the <math>\left(i,j\right)</math> element of <math>AB</math>: <math>\sum_{k=1}^{3}a_{ik}b_{kj}</math> is the <math>\left(j,i\right)</math> element of <math>B^{T}A^{T}.</math> But this means that <math>B^{T}A^{T}</math> must be the transpose of <math>AB,</math> since the elements in the <math>i</math>th row of <math>AB</math> are being written in the <math>i</math>th column of <math>B^{T}A^{T}.</math><br />
<br />
This ''Product Rule for Transposition'' can be applied again to find the transpose <math>\left(C^{T}\right)^{T}</math> of <math>C^{T}</math>: <math>\left(C^{T}\right)^{T}=\left(B^{T}A^{T}\right)^{T}=\left(A^{T}\right)^{T}\left(B^{T}\right)^{T}=AB=C.</math><br />
<br />
= Special Types of Matrix =<br />
<br />
== The zero matrix ==<br />
<br />
The most obvious special type of matrix is one whose elements are all zeros. In typical element notation, the zero matrix is: <math>0=\left\Vert 0\right\Vert .</math> Since there is no indexing on the elements, it is not obvious what the dimension of this matrix is, Sometimes one writes <math>0_{mn}</math> to indicate a zero matrix of dimension <math>m\times n.</math> The same ideas apply to vectors whose elements are all zero.<br />
<br />
The effect of the zero matrix in any product that is defined is simple: <math>0A=0,\ \ \ \ \ B0=0.</math> This is easy to check using the across and down rule.<br />
<br />
== The identity or unit matrix ==<br />
<br />
Vectors of the form:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{r}<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }2\ \text{dimensions}\\<br />
\left[\begin{array}{c}<br />
1\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
1\\<br />
0<br />
\end{array}\right],\left[\begin{array}{c}<br />
0\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }3\ \text{dimensions}\\<br />
\left[\begin{array}{r}<br />
1\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
0<br />
\end{array}\right],\left[\begin{array}{r}<br />
0\\<br />
1\\<br />
0\\<br />
\vdots\\<br />
0<br />
\end{array}\right],\ldots,\left[\begin{array}{r}<br />
0\\<br />
0\\<br />
\vdots\\<br />
0\\<br />
1<br />
\end{array}\right]\ \ \ \ \ \text{in }n\ \text{dimensions}\end{aligned}</math><br />
<br />
are called coordinate vectors. They are often given a characteristic notation, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> in <math>n</math> dimensions. When arranged as columns of a matrix in the natural order, <math>\mathbf{e}_{1},\ldots,\mathbf{e}_{n},</math> a matrix with a characteristic pattern elements emerges, with a special notation:<br />
<br />
<math>\begin{aligned}<br />
\left[\begin{array}{rr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2}\end{array}\right] & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=I_{2}\\<br />
\left[\begin{array}{rrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \mathbf{e}_{3}\end{array}\right] & = & \left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 1 & 0\\<br />
0 & 0 & 1<br />
\end{array}\right]=I_{3}\\<br />
\left[\begin{array}{rrrr}<br />
\mathbf{e}_{1} & \mathbf{e}_{2} & \ldots & \mathbf{e}_{n}\end{array}\right] & = & \left[\begin{array}{rrrr}<br />
1 & 0 & \ldots & 0\\<br />
0 & 1 & \ldots & 0\\<br />
\vdots & \vdots & \ddots & \vdots\\<br />
0 & 0 & \ldots & 1<br />
\end{array}\right]=I_{n}.\end{aligned}</math><br />
<br />
The ''diagonal'' of this matrix is where the 1 elements are located, and every other element is zero.<br />
<br />
Consider the effect of <math>I_{2}</math> on the matrix: <math>A=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]</math> by both pre and post multiplication:<br />
<br />
<math>\begin{aligned}<br />
I_{2}A & = & \left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\\<br />
AI_{2} & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
1 & 0\\<br />
0 & 1<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=A,\end{aligned}</math><br />
<br />
as is easily checked by the across and down rule.<br />
<br />
Because any matrix is left unchanged by pre or post multiplication by an appropriately dimensioned <math>I_{n},</math> <math>I_{n}</math> is called an ''identity matrix of dimension ''<math>n.</math> Sometimes it is called a ''unit matrix of dimension ''<math>n.</math> Notice that <math>I_{n}</math> is necessarily a square matrix.<br />
<br />
== Diagonal matrices ==<br />
<br />
The identity matrix is an example of a diagonal matrix, a matrix whose elements are all zero except for those on the diagonal. Usually diagonal matrices are taken to be square, for example: <math>D=\left[\begin{array}{rrr}<br />
1 & 0 & 0\\<br />
0 & 2 & 0\\<br />
0 & 0 & 3<br />
\end{array}\right].</math> They also produce characteristic effects when pre or post multiplying another matrix.<br />
<br />
Consider the diagonal matrix: <math>B=\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]</math> and the products <math>AB,</math> <math>BA</math> for <math>A</math> as defined in the previous section:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]\left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & -4\\<br />
6 & -10<br />
\end{array}\right],\\<br />
BA & = & \left[\begin{array}{cc}<br />
2 & 0\\<br />
0 & -2<br />
\end{array}\right]\left[\begin{array}{cc}<br />
6 & 2\\<br />
3 & 5<br />
\end{array}\right]=\left[\begin{array}{cc}<br />
12 & 4\\<br />
-6 & -10<br />
\end{array}\right].\end{aligned}</math><br />
<br />
Comparing the results, we can deduce that post multiplication by a diagonal matrix multiplies each column of <math>A</math> by the corresponding diagonal element, whereas pre multiplication multiplies each row by the corresponding diagonal element.<br />
<br />
== Symmetric matrices ==<br />
<br />
Symmetric matrices are matrices having the property that <math>A=A^{T}.</math> Notice that such matrices must be square, since if <math>A</math> is <math>m\times n,</math> <math>A^{T}</math> is <math>n\times m,</math> and to have equality of <math>A</math> and <math>A^{T},</math> they must have the same dimension, so that <math>m=n</math> is required.<br />
<br />
Suppose that <math>A</math> is a <math>3\times3</math> symmetric matrix, with typical element <math>a_{ij}</math>: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & a_{12} & a_{13}\\<br />
a_{21} & a_{22} & a_{23}\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right],</math> so that: <math>A^{T}=\left[\begin{array}{rrr}<br />
a_{11} & a_{21} & a_{31}\\<br />
a_{12} & a_{22} & a_{32}\\<br />
a_{13} & a_{23} & a_{33}<br />
\end{array}\right].</math> Equality of matrices is defined as equality of all elements. This is fine on the diagonals, since <math>A</math> and <math>A^{T}</math> have the same diagonal elements. For the off diagonal elements, we end up with the requirements: <math>a_{12}=a_{21},\ \ \ a_{13}=a_{31},\ \ \ a_{23}=a_{32}</math> or more generally: <math>a_{ij}=a_{ji}\ \ \ \ \ \text{for}\ i\neq j.</math><br />
<br />
The effect of this conclusion is that in a symmetric matrix, the ’triangle’ of above diagonal elements coincides with the triangle of below diagonal elements. It is as if the upper triangle is folded over the diagonal to become the lower triangle.<br />
<br />
A simple example is: <math>A=\left[\begin{array}{cc}<br />
1 & 2\\<br />
2 & 1<br />
\end{array}\right].</math> A more complicated example uses the <math>2\times3</math> matrix <math>C</math>: <math>C=\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]</math> and calculates the <math>3\times3</math> matrix:<br />
<br />
<math>\begin{aligned}<br />
C^{T}C & = & \left[\begin{array}{cc}<br />
6 & 3\\<br />
2 & 5\\<br />
-3 & -1<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
6 & 2 & -3\\<br />
3 & 5 & -1<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{rrr}<br />
45 & 27 & -21\\<br />
27 & 29 & -11\\<br />
-21 & -11 & 10<br />
\end{array}\right],\end{aligned}</math><br />
<br />
which is clearly symmetric.<br />
<br />
This illustrates the general proposition that if <math>A</math> is an <math>m\times n</math> matrix, the product <math>A^{T}A</math> is a symmetric <math>n\times n</math> matrix. Proof? Compute the transpose of <math>A^{T}A</math> using the product rule for transposition: <math>\left(A^{T}A\right)^{T}=A^{T}\left(A^{T}\right)^{T}=A^{T}A.</math> Since <math>A^{T}A</math> is equal to its transpose, it must be a symmetric matrix. Such symmetric matrices appear frequently in econometrics.<br />
<br />
It should be clear that diagonal matrices are symmetric, since all their off-diagonal elements are equal (zero), and thence the identity matrix <math>I_{n}</math> is also symmetric.<br />
<br />
== The outer product ==<br />
<br />
The inner product of two <math>n\times1</math> vectors <math>\mathbf{x}</math> and <math>\mathbf{y}</math>, <math>\mathbf{x}^{T}\mathbf{y}</math>, is automatically a <math>1\times1</math> quantity, a scalar, although it can be interpreted as a <math>1\times1</math> matrix, a matrix with a single element.<br />
<br />
Suppose one considered the product of <math>\mathbf{x}</math> with <math>\mathbf{x}^{T}.</math> Is this defined? If <math>A</math> is <math>m\times n</math> and <math>B</math> is <math>n\times r,</math> then the product <math>AB</math> is <math>m\times r.</math> Applying this logic to <math>\mathbf{xx}^{T},</math> this is <math>\left(n\times1\right)\left(1\times n\right),</math> so the resulting product ''is ''defined, and is an <math>n\times n</math>'' matrix'' - the ''outer product'' of <math>\mathbf{x}</math> and <math>\mathbf{x}^{T},</math> the word ’outer’ being used to distinguish from the inner product.<br />
<br />
How does the across and down rule work here? Suppose that: <math>\mathbf{x}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right].</math> Then: <math>\mathbf{xx}^{T}=\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right].</math> Here, there is <math>1</math> element in row one of the ’matrix’ <math>\mathbf{x,}</math> and <math>1</math> element in column one of the matrix <math>\mathbf{x}^{T},</math> so the across and down rule still works - it is just that there is only one product per row and column combination. So: <math>\mathbf{xx}^{T}=\left[\begin{array}{cc}<br />
36 & 18\\<br />
18 & 9<br />
\end{array}\right],</math> and it is obvious from this that <math>\mathbf{xx}^{T}</math> is a symmetric matrix.<br />
<br />
One can see that this outer product need not be restricted to vectors of the same dimension. If <math>\mathbf{x}</math> is <math>n\times1</math> and <math>\mathbf{y}</math> is <math>m\times1,</math> then: <math>\mathbf{xy}^{T}=\left[\begin{array}{c}<br />
x_{1}\\<br />
\vdots\\<br />
x_{n}<br />
\end{array}\right]\left[\begin{array}{rrr}<br />
y_{1} & \ldots & y_{m}\end{array}\right]=\left[\begin{array}{rrrr}<br />
x_{1}y_{1} & x_{1}y_{2} & \ldots & x_{1}y_{m}\\<br />
x_{2}y_{1} & x_{2}y_{2} & \ldots & x_{2}y_{m}\\<br />
\\<br />
x_{n}y_{1} & x_{n}y_{2} & \ldots & x_{n}y_{m}<br />
\end{array}\right].</math> So, <math>\mathbf{xy}^{T}</math> is <math>n\times m,</math> and consists of rows which are <math>\mathbf{y}^{T}</math> multiplied by an element of the <math>\mathbf{x}</math> vector.<br />
<br />
Another interesting and useful example involves a vector with every element equal to <math>1</math>: <math>\mathbf{1}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right].</math> Sometimes this is written as <math>\mathbf{1}_{n}</math> to indicate an <math>n\times1</math> vector, and is called the ''sum vector''. Why? Consider the impact of <math>\mathbf{1}_{2}</math> on the <math>2\times1</math> vector <math>\mathbf{x}</math> used above: <math>\mathbf{1}_{2}^{T}\mathbf{x}=\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]\left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]=9,</math> i.e. an inner product of <math>\mathbf{x}</math> with the sum vector is the sum of the elements of <math>\mathbf{x.}</math> Dividing through by the number of elements in <math>\mathbf{x}</math> produces the average of the elements of <math>\mathbf{x}</math> - i.e. the ’sample mean’ of the elements of <math>\mathbf{x.}</math><br />
<br />
The outer product of <math>\mathbf{x}</math> with <math>\mathbf{1}_{2}</math> is also interesting:<br />
<br />
<math>\begin{aligned}<br />
\mathbf{1}_{2}\mathbf{x}^{T} & = & \left[\begin{array}{r}<br />
1\\<br />
1<br />
\end{array}\right]\left[\begin{array}{rr}<br />
6 & 3\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 3\\<br />
6 & 3<br />
\end{array}\right],\\<br />
\mathbf{x1}_{2}^{T} & = & \left[\begin{array}{r}<br />
6\\<br />
3<br />
\end{array}\right]\left[\begin{array}{rr}<br />
1 & 1\end{array}\right]=\left[\begin{array}{cc}<br />
6 & 6\\<br />
3 & 3<br />
\end{array}\right],\end{aligned}</math><br />
<br />
showing that pre multiplication of an <math>\mathbf{x}^{T}</math> by <math>\mathbf{1}</math> repeats <math>\mathbf{x}^{T}</math> as rows of the product, whilst post multiplication of <math>\mathbf{x}</math> by <math>\mathbf{1}^{T}</math> repeats <math>\mathbf{x}</math> as the columns of the product.<br />
<br />
Finally: <math>\mathbf{1}_{n}\mathbf{1}_{n}^{T}=\left[\begin{array}{rrr}<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1\\<br />
1 & \ldots & 1<br />
\end{array}\right],</math> an <math>n\times n</math> matrix with every element equal to <math>1.</math> This type of matrix also appears in econometrics!<br />
<br />
== Triangular matrices ==<br />
<br />
A square ''lower triangular ''matrix has all elements above the main diagonal equal to zero, whilst a square ''upper triangular ''matrix has all elements below the main diagonal equal to zero. A simple example of a lower triangular matrix is: <math>A=\left[\begin{array}{rrr}<br />
a_{11} & 0 & 0\\<br />
a_{21} & a_{22} & 0\\<br />
a_{31} & a_{32} & a_{33}<br />
\end{array}\right].</math> Clearly, for this matrix, <math>A^{T}</math> is an upper triangular matrix.<br />
<br />
One can adapt the definition to rectangular matrices: for example, if two arbitrary rows are added to <math>A,</math> so that it becomes <math>5\times3,</math> it would still be considered lower triangular. Equally, if, for example, the third column of <math>A</math> above is removed, <math>A</math> is still considered lower triangular.<br />
<br />
Often, we use ''unit ''triangular matrices, where the diagonal elements are all equal to <math>1:</math> e.g.:<br />
<br />
<math>\left[\begin{array}{rrr}<br />
1 & 2 & 0\\<br />
0 & 1 & 1\\<br />
0 & 0 & 1<br />
\end{array}\right].\label{eq:lt_matrix}</math><br />
<br />
== Partitioned matrices ==<br />
<br />
Sometimes, especially with big matrices, it is useful to organise the elements of the matrix into components which are themselves matrices, for example: <math>B=\left[\begin{array}{rrrr}<br />
1 & 2 & 0 & 0\\<br />
8 & 3 & 0 & 0\\<br />
0 & 0 & 7 & 4\\<br />
0 & 0 & 6 & 5<br />
\end{array}\right]</math> Here it would be reasonable to write: <math>B=\left[\begin{array}{cc}<br />
B_{11} & 0\\<br />
0 & B_{22}<br />
\end{array}\right],</math> where <math>B_{ii},i=1,2,</math> represent <math>2\times2</math> matrices. <math>B</math> is an example of a ''partitioned matrix'': that is, an <math>m\times n</math> matrix <math>A</math> say: <math>A=\left\Vert a_{ij}\right\Vert ,</math> where the elements of <math>A</math> are organised into ''sub-matrices''. An example might be:<br />
<br />
<math>A=\left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right],\label{eq:partition_a}</math><br />
<br />
where the ''sub - matrices'' in the first row block have <math>r</math> rows, and therefore <math>m-r</math> rows in the second row block. The column blocks might be defined by (for example) 3 columns in the first column block, 4 in the second and <math>n-7</math> in the third column block.<br />
<br />
Another simple example might be: <math>A=\left[\begin{array}{rrr}<br />
A_{1} & A_{2} & A_{3}\end{array}\right],\ \ \ \ \ \mathbf{x=}\left[\begin{array}{c}<br />
\mathbf{x}_{1}\\<br />
\mathbf{x}_{2}\\<br />
\mathbf{x}_{3}<br />
\end{array}\right],</math> where <math>A</math> and therefore <math>A_{1},A_{2},A_{3}</math> have <math>m</math> rows, <math>A_{1}</math> has <math>n_{1}</math> columns, <math>A_{2}</math> has <math>n_{2}</math> columns, <math>A_{3}</math> has <math>n_{3}</math> columns. The ''subvectors'' in <math>\mathbf{x}</math> must have <math>n_{1},n_{2}</math> and <math>n_{3}</math> rows respectively, for the product <math>A\mathbf{x}</math> to exist.<br />
<br />
Suppose that <math>n_{1}+n_{2}+n_{3}=n,</math> so that <math>A</math> is <math>m\times n.</math> The <math>i</math>th element of <math>A\mathbf{x}</math> is: <math>\sum_{i=1}^{n}a_{ij}x_{j}</math> but the summation can be broken up into the first <math>n_{1}</math> terms: <math>\sum_{i=1}^{n_{1}}a_{ij}x_{j},</math> the next <math>n_{2}</math> terms: <math>\sum_{i=n_{1}+1}^{n_{1}+n_{2}}a_{ij}x_{j},</math> and the next <math>n_{3}</math> terms; <math>\sum_{i=n_{1}+n_{2}+1}^{n}a_{ij}x_{j}.</math><br />
<br />
The point about the use of partitioned matrices is that the product <math>A\mathbf{x}</math> can be represented as: <math>A\mathbf{x}=A_{1}\mathbf{x}_{1}+A_{2}\mathbf{x}_{2}+A\mathbf{x}_{3}</math> by applying the across and down rule to the submatrices and the subvectors, a much simpler representation than the use of summations.<br />
<br />
Each of the components is a conformable matrix-vector product: this is essential in any use of partitioned matrices to represent some matrix product. For example, using <math>A</math> from equation ([eq:partition<sub>a</sub>]) and <math>B</math> as: <math>B=\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right],</math> it is easy to write:<br />
<br />
<math>\begin{aligned}<br />
AB & = & \left[\begin{array}{rrr}<br />
A_{11} & A_{12} & A_{23}\\<br />
A_{21} & A_{22} & A_{23}<br />
\end{array}\right]\left[\begin{array}{c}<br />
B_{11}\\<br />
B_{21}\\<br />
B_{31}<br />
\end{array}\right]\\<br />
& = & \left[\begin{array}{r}<br />
A_{11}B_{11}+A_{12}B_{21}+A_{13}B_{31}\\<br />
A_{21}B_{11}+A_{22}B_{21}+A_{23}B_{31}<br />
\end{array}\right].\end{aligned}</math><br />
<br />
But, what are the row dimensions for the submatrices in <math>B?</math> What are the possible column dimensions for the submatrices in <math>B?</math><br />
<br />
= Matrices, vectors and econometrics =<br />
<br />
The data on weights and heights for 12 students in the data matrix: <math>D=\left[\begin{array}{cc}<br />
155 & 70\\<br />
150 & 63\\<br />
180 & 72\\<br />
135 & 60\\<br />
156 & 66\\<br />
168 & 70\\<br />
178 & 74\\<br />
160 & 65\\<br />
132 & 62\\<br />
145 & 67\\<br />
139 & 65\\<br />
152 & 68<br />
\end{array}\right]</math> would seem to be ideally suited for fitting a two variable regression model: <math>y_{i}=\alpha+\beta x_{i}+u_{i},\;\;\;\;\; i=1,...,12.</math> Here, the first column of <math>D</math> contains all the weight data, the data on the dependent variable <math>y_{i},</math> and so should be labelled <math>\mathbf{y.}</math> The second column of <math>D</math> contains all the data on the explanatory variable height, in the vector <math>\mathbf{x}</math> say, so that: <math>D=\left[\begin{array}{rr}<br />
\mathbf{y} & \mathbf{x}\end{array}\right].</math><br />
<br />
If we define a <math>12\times1</math> vector with every element <math>1</math>: <math>\mathbf{1}_{12}=\left[\begin{array}{c}<br />
1\\<br />
\vdots\\<br />
1<br />
\end{array}\right],</math> and a <math>12\times1</math> vector <math>\mathbf{u}</math> to contain the error terms; <math>\mathbf{u}=\left[\begin{array}{c}<br />
u_{1}\\<br />
\vdots\\<br />
u_{12}<br />
\end{array}\right]</math> the regression model can be written in terms of the three data vectors <math>\mathbf{y,1}_{12}</math> and <math>\mathbf{x}</math> as: <math>\mathbf{y}=\mathbf{1}_{12}\alpha+\mathbf{x}\beta+\mathbf{u.}</math> To see this, think of the <math>i</math>th elements of the vectors on the left and right hand sides.<br />
<br />
The standard next step is then to combine the data vectors for the explanatory variables into a matrix: <math>X=\left[\begin{array}{rr}<br />
\mathbf{1}_{12} & \mathbf{x}\end{array}\right],</math> and then define a <math>2\times1</math> vector <math>\boldsymbol{\delta}</math> to contain the parameters <math>\alpha,\beta</math> as: <math>\boldsymbol{\delta}=\left[\begin{array}{r}<br />
\alpha\\<br />
\beta<br />
\end{array}\right]</math> to give the data matrix representation of the regression model as: <math>\mathbf{y}=X\boldsymbol{\delta}+\mathbf{u.}</math><br />
<br />
For the purposes of developing the theory of regression, this is the most convenient form of the regression model. It can represent regression models with any number of explanatory variables, and thus any number of parameters. The obvious point is that a knowledge of vector and matrix operations is needed to use and understand this form.<br />
<br />
We shall see later that there are two particular matrix and vector quantities associated with a regression model. The first is the matrix <math>X^{T}X,</math> and the second the vector <math>X^{T}\mathbf{y.}</math> The following Matlab code snippet provides the numerical values of these quantities for the weight data:<br />
<br />
&gt;&gt; dset = load(’weights.mat’); <br />
<br />
&gt;&gt; xtx = dset.X’ * dset.X; <br />
<br />
&gt;&gt; xty = dset.X’ * dset.y; <br />
<br />
&gt;&gt; disp(xtx) <br />
<br />
12 802<br />
<br />
802 53792<br />
<br />
&gt;&gt; disp(xty)<br />
<br />
1850<br />
<br />
124528<br />
<br />
Hand calculation is of course possible, but not recommended.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3021Maths2013-09-10T12:55:30Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Trial for latex import:<br />
<br />
[[LNotes|Lnotes]]<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material, and test their understanding using Maple TA.<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments: there is usually a delay whilst they are loaded. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+, -, *, /, ^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be too hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3020Maths2013-09-10T12:28:53Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Trial for latex import:<br />
<br />
[[Math|Lnotes]]<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material, and test their understanding using Maple TA.<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments: there is usually a delay whilst they are loaded. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+, -, *, /, ^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be too hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3019Maths2013-09-10T12:28:29Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Trial for latex import:<br />
<br />
[[Math|Math]]<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material, and test their understanding using Maple TA.<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments: there is usually a delay whilst they are loaded. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+, -, *, /, ^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be too hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Math&diff=3018Math2013-09-10T11:01:00Z<p>LG: Created page with " = Introduction = In this Section we will demonstrate how to use instrumental variables (IV) estimation to estimate the parameters in a linear regression model. The material..."</p>
<hr />
<div><br />
<br />
= Introduction =<br />
<br />
In this Section we will demonstrate how to use instrumental variables (IV) estimation to estimate the parameters in a linear regression model. The material will follow the notation in the Heij ''et al.'' textbook<ref>Heij C, de Boer P., Franses P.H., Kloek T. and van Dijk H.K (2004) Econometric Methods with Applications in Business and Economics, Oxford University Press, New York [http://www.amazon.co.uk/Econometric-Methods-Applications-Business-Economics/dp/0199268010/ref=sr_1_1?s=books&ie=UTF8&qid=1354473313&sr=1-1]. This is an all-round good textbook that presents econometrics using matrix algebra.<br />
</ref>.<br />
<br />
<math>\mathbf{y}=\mathbf{X\beta }+\mathbf{\varepsilon }</math><br />
<br />
The issue is that we may suspect (or know) that the explanatory variable is correlated with the (unobserved) error term<br />
<br />
<math>p\lim \left( \frac{1}{n}\mathbf{X}^{\prime }\mathbf{\varepsilon }\right) \neq 0.</math><br />
<br />
Reasons for such a situation include measurement error in <math>x</math>, endogenous explanatory variables, omitted relevant variables or a combination of the above. The consequence is that the OLS parameter estimate of <math>\mathbf{\beta}</math> is biased and inconsistent. Fortunately it is well established that an IV estimation of <math>\mathbf{\beta}</math> can potentially deliver consistent parameter estimates. This does, however, require the availability of sufficient instruments <math>\mathbf{Z}</math>.<br />
<br />
Before continuing it is advisable to be clear about the dimensions of certain variables. Let’s assume that <math>\mathbf{y}</math> is a <math>(n \times 1)</math> vector containing the <math>n</math> observations for the dependent variable. <math>\mathbf{X}</math> is a <math>(n \times k)</math> matrix with the <math>k</math> explanatory variables in the columns, usually containing a vector of 1s in the first column, representing a regression constant. Now, let <math>\mathbf{Z}</math> be a <math>(n \times p)</math> matrix with instruments. Importantly, <math>p \ge k</math>, and further <math>\mathbf{X}</math> and <math>\mathbf{Z}</math> may have columns in common. If so, these are explanatory variables from <math>\mathbf{X}</math> that are judged to be certainly uncorrelated with the error term (like the constant).<br />
<br />
It is well established that the instrumental variables in <math>\mathbf{Z}</math> need to meet certain restrictions in order to deliver useful IV estimators of <math>\mathbf{\beta}</math>. They need to be uncorrelated to the error terms and they need to be correlated with the explanatory variables in <math>\mathbf{X}</math> that are deemed to be endogenous (related to the error term). Further they should have no relevance for the dependent variable, other than through its relation to the potentially endogenous variable (exclusion assumption).<br />
<br />
A number of MATLAB functions can be found [[ExampleCodeIV|here]].<br />
<br />
= IV estimator =<br />
<br />
It is well established that the IV estimator can be estimated as follows<br />
<br />
<math>\mathbf{\widehat{\beta}}_{IV} = \left(\mathbf{X}'\mathbf{P}_Z \mathbf{X}\right)^{-1} \mathbf{X}'\mathbf{P}_Z \mathbf{y}</math><br />
<br />
where <math>\mathbf{P}_Z</math> is the projection matrix of <math>\mathbf{Z}</math>. When performing inference the Variance-Covariance matrix of <math>\mathbf{\widehat{\beta}}_{IV}</math> is of obvious interest and it is calculated as follows<br />
<br />
<math>Var\left(\mathbf{\widehat{\beta}}_{IV} \right) = \sigma ^{2}\left( \mathbf{X}^{\prime }\mathbf{P}_{Z}\mathbf{X}\right)^{-1}</math><br />
<br />
where the estimate for the error variance comes from<br />
<br />
<math>\begin{aligned}<br />
s_{IV}^{2} &=&\frac{1}{n-k}\widehat{\mathbf{\varepsilon }}_{IV}^{\prime }\widehat{\mathbf{\varepsilon }}_{IV} \\<br />
&=&\frac{1}{n-k}\left( \mathbf{y-X}\widehat{\mathbf{\beta }}_{IV}\right)<br />
^{\prime }\left( \mathbf{y-X}\widehat{\mathbf{\beta }}_{IV}\right)\end{aligned}</math><br />
<br />
== MATLAB implementation ==<br />
<br />
The following code extract assumes that the vector <code>y</code> contains the <math>(n \times 1)</math> vector with the dependent variable, the <math>(n \times k)</math> matrix <code>x</code> contains all explanatory variables and <code>z</code> is a <math>(n \times p)</math> matrix <math>(p ge k)</math> with instruments.<br />
<br />
<source>pz = z*inv(z'*z)*z'; % Projection matrix<br />
xpzxi = inv(x'*pz*x); % this is also (Xhat'Xhat)^(-1)<br />
<br />
biv = xpzxi*x'*pz*y; % IV estimate<br />
res = y - x*biv; % IV residuals<br />
ssq = res'*res/(n-k); % Sample variance for IV residuals<br />
s = sqrt(ssq); % Sample Standard deviation for IV res<br />
bse = ssq*xpzxi; % Variance covariance matrix for IV estimates<br />
bse = sqrt(diag(bse)); % Extract diagonal and take square root -> standard errors for IV estimators</source><br />
= IV related Testing procedures =<br />
<br />
One feature of IV estimations is that in general it is an inferior estimator of <math>\mathbf{\beta}</math> if all explanatory variables are exogenous. In that case, assuming that all other Gauss-Markov assumptions are met, the OLS estimator is the BLUE estimator. In other words, IV estimators have larger standard errors for the coefficient estimates. Therefore, one would really like to avoid having to rely on IV estimators, unless, of course, they are the only estimators that deliver consistent estimates.<br />
<br />
For this reason any application of IV, should be accompanied by evidence that establishes that it was necessary. Once that is established, one should also establish that the instruments chosen meet the necessary requirements (of being correlated with the endogenous variable and being exogenous to the regression error term).<br />
<br />
== Testing for exogeneity ==<br />
<br />
The null hypothesis to be tested here is whether<br />
<br />
<math>p\lim \left( \frac{1}{n}\mathbf{X}^{\prime }\mathbf{\varepsilon }\right) \neq 0.</math><br />
<br />
and therefore whether an IV estimation is required or no. The procedure described is as in Heij ''et al.''. It consists of the following three steps.<br />
<br />
<ol><br />
<li><p>Estimate <math>\mathbf{y}=\mathbf{X\beta }+\mathbf{\varepsilon}</math> by OLS and save the residuals <math>\widehat{\mathbf{\varepsilon}}</math>.</p></li><br />
<li><p>Estimate</p><br />
<p><math>\mathbf{x}_{j}=\mathbf{Z\gamma }_{j}\mathbf{+v}_{j}</math></p><br />
<p>by OLS for all <math>\widetilde{k}</math> elements in <math>\mathbf{X}</math> that are possibly endogenous and save <math>\widehat{\mathbf{v}}_{j}</math>. Collect these in the <math>\left(<br />
n\times \widetilde{k}\right) </math> matrix <math>\widehat{\mathbf{V}}</math>.</p></li><br />
<li><p>Estimate the auxilliary regression</p><br />
<p><math>\widehat{\mathbf{\varepsilon }}=\mathbf{X\delta }_{0}+\widehat{\mathbf{V}} \mathbf{\delta }_{1}+\mathbf{u}</math></p><br />
<p>and test the following hypothesis</p><br />
<p><math>\begin{aligned}<br />
H_{0} &:&\mathbf{\delta }_{1}=0~~\mathbf{X}\text{ is exogenous} \\<br />
H_{A} &:&\mathbf{\delta }_{1}\neq 0~~\mathbf{X}\text{ is endogenous}<br />
\end{aligned}</math></p><br />
<p>using the usual test statistic <math>\chi ^{2}=nR^{2}</math> which, under <math>H_{0}</math>, is <math><br />
\chi ^{2}\left( \widetilde{k}\right) </math> distributed.</p></li></ol><br />
<br />
Implementing this test does not require anything else but the application of OLS regressions. In the following excerpt we assume that the dependent variable is contained in vector <code>y</code>, the elements in <math>X</math> that are assumed to be exogenous are contained in <code>x1</code>, those elements that are suspected that they may be endogenous are in <code>x2</code> and the instrument matrix is saved in <code>z</code>. As before, it is assumed that <code>z</code> should contain all elements of <code>x1</code>.<br />
<br />
The code also uses the OLSest function for the step 3 regression. However, that could easily be circumvented as for the regressions in Step 1 and 2.<br />
<br />
<source>x = [x1 x2]; % Combine to one matrix x<br />
xxi = inv(x'*x);<br />
b = xxi*x'*y; % Step 1: OLS estimator<br />
res = y - x*b; % Step 1: saved residuals<br />
<br />
zzi = inv(z'*z); % Step 2: inv(Z'Z) which is used in Step 2<br />
gam = zzi*z'*x2; % Step 2: Estimate OLS coefficients of step 2 regressions<br />
% This works even if we have more than one element in x2<br />
% we get as many columns of gam as we have elements in x2<br />
vhat = x2 - z*gam; % Step 2: residuals (has as many columns as in x2<br />
<br />
[b,bse,res,n,rss,r2] = OLSest(res,[x vhat],0); % Step 3 regression<br />
teststat = size(res,1)*r2; % Step 3: Calculate nR^2 test stat<br />
pval = 1 - chi2cdf(teststat,size(x2,2)); % Step 3: Calculate p-value</source><br />
A function that implements this test can be found [[ExampleCodeIV#Hausmann|here]].<br />
<br />
== Sargan test for instrument validity ==<br />
<br />
One crucial property of instruments is that they ought to be uncorrelated to the regression error terms <math>\mathbf{\varepsilon}</math>. Instrument endogeneity is set as the null hypothesis of this test with the alternative hypothesis being that the instruments are endogenous.<br />
<br />
<ol><br />
<li><p>Estimate the regression model by IV and save <math>\widehat{\mathbf{\varepsilon }}_{IV}=\mathbf{y}-\mathbf{X}\widehat{\mathbf{\beta }}_{IV}</math></p></li><br />
<li><p>Regress</p><br />
<p><math>\widehat{\mathbf{\varepsilon }}_{IV}=\mathbf{Z\gamma +u}</math></p></li><br />
<li><p>Calculate <math>LM=nR^{2}</math> from the auxilliary regresion in step 2. <math>LM</math> is (under <math>H_{0}</math>) <math>\chi ^{2}</math> distributed with <math>\left( p-k\right) </math> degrees of freedom.</p></li></ol><br />
<br />
The MATLAB implementation of this test relies on the availability of the IV parameter estimates. They can be calculated as indicated above. In [[ExampleCodeIV#IVest|this section]] you can find a function called <code>IVest</code> that can deliver the required IV residuals by calling:<br />
<br />
<source>[biv,bseiv,resiv,r2iv] = IVest(y,x,z);</source><br />
The third output are the IV residuals (refer to [[ExampleCodeIV#IVest|IVest]] for details) which can then be used as the dependent variable in the second step regression:<br />
<br />
<source>[b,bse,res,n,rss,r2] = OLSest(resiv,z,0); % Step 2: calculate Step 2 regression<br />
teststat = size(resiv,1)*r2; % Step 3: Calculates the nR^2 test statistic<br />
pval = 1 - chi2cdf(teststat,(size(z,2)-size(x,2))); % Step 3: Calculate p-value</source><br />
It should be noted that this test is only applicable for an over-identified case when the <code>z</code> contains more columns than <code>x</code>. A function that implements this test can be found [[ExampleCodeIV#Sargan|here]].<br />
<br />
== Instrument relevance ==<br />
<br />
The last instrument property that is required is that the instruments are correlated to the potentially endogenous variables. This is tested using a standard OLS regression that uses the endogenous variables as the dependent variable and all instrument variables (i.e. <code>z</code>) as the explanatory variables. We then need to check whether the restriction that all (non-constant) variables in <code>z</code> are relevant (F-test). If they are relevant, then the instruments are relevant. This is, actually, exactly what the Step 2 regressions of the Hausmann test do.<br />
<br />
=Footnotes=<br />
<br />
<references /></div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3017Maths2013-09-10T10:48:21Z<p>LG: /* Important Notice */</p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
[[Math|Math]]<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material, and test their understanding using Maple TA.<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments: there is usually a delay whilst they are loaded. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+, -, *, /, ^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be too hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3016Maths2013-09-10T10:45:56Z<p>LG: /* Using Maple TA */</p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material, and test their understanding using Maple TA.<br />
<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments: there is usually a delay whilst they are loaded. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+, -, *, /, ^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be too hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3015Maths2013-09-10T10:45:29Z<p>LG: /* Using Maple TA */</p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material, and test their understanding using Maple TA.<br />
<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments: there is usually a delay whilst they are loaded. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+, -, *, /, ^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "$1/x - 1$" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be too hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=File:Lecture_2.pdf&diff=3014File:Lecture 2.pdf2013-09-09T14:13:46Z<p>LG: LG uploaded a new version of &quot;File:Lecture 2.pdf&quot;: These notes cover the introductory matrix material, and should be studied before the lectures on ECON61001 start.</p>
<hr />
<div>These notes cover the introductory matrix material, and should be studied before the lectures on ECON61001 start.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=File:Lecture_2.pdf&diff=3013File:Lecture 2.pdf2013-09-09T14:13:10Z<p>LG: LG uploaded a new version of &quot;File:Lecture 2.pdf&quot;</p>
<hr />
<div>These notes cover the introductory matrix material, and should be studied before the lectures on ECON61001 start.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3012Maths2013-09-09T13:19:33Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material, and test their understanding using Maple TA.<br />
<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments: there is usually a delay whilst they are loaded. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+, -, *, /, ^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be too hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3011Maths2013-09-09T13:17:43Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material, and test their understanding using Maple TA.<br />
<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments: there is usually a delay whilst they are loaded. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+,-,*./,^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in a separate document, although the instructions are repeated in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be to hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3010Maths2013-09-09T13:16:36Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material, and test their understanding using Maple TA.<br />
<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+,-,*./,^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in a separate document, although the instructions are repeated in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be to hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3009Maths2013-09-09T13:15:50Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture 2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material.<br />
<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+,-,*./,^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in a separate document, although the instructions are repeated in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be to hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3008Maths2013-09-09T13:15:17Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:Lecture2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material.<br />
<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+,-,*./,^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in a separate document, although the instructions are repeated in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be to hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3007Maths2013-09-09T13:14:35Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:lecture2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho.pdf]]<br />
<br />
respectively.<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material.<br />
<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+,-,*./,^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in a separate document, although the instructions are repeated in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be to hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3006Maths2013-09-09T13:14:08Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this material are <br />
<br />
[[Media:lecture2.pdf]]<br />
<br />
and <br />
<br />
[[Media:L2_slide_ho,pdf]]<br />
<br />
respectively.<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. The link to this material is<br />
<br />
[[Media:Xs2.pdf]]<br />
<br />
Students on the MA program are welcome to try out this material.<br />
<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester] http://place36.placementtester.com/manchester<br />
<br />
Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments. You can click on the assignment you want to do - the notation follows that in the exercise sheet. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of each of the question groups. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+,-,*./,^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in a separate document, although the instructions are repeated in the Exercise Sheet.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be to hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3005Maths2013-09-09T13:05:31Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. The pdf files containing this materail are <br />
<br />
<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. <br />
<br />
Students on the MA program are welcome to try out this material.<br />
<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. In this module, the questions on the paper Exercise Sheets are also available as Maple TA assignments. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester]<br />
<br />
and this link is also given in each Lecture folder, for convenience. Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments. You can click on the assignment you want to do - the notation follows that in the exercise sheets. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of the question group. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+,-,*./,^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in a separate document, although the instructions are often repeated in questions.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be to hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=File:Xs2.pdf&diff=3004File:Xs2.pdf2013-09-09T13:03:55Z<p>LG: Exercise questions for Lecture 2 of ECON61001 - they cover the introductory matrix material. You should answer these questions using Maple TA - as soon as you have answered a question, you can grade it straight away. This will show whether your answer ...</p>
<hr />
<div>Exercise questions for Lecture 2 of ECON61001 - they cover the introductory matrix material. You should answer these questions using Maple TA - as soon as you have answered a question, you can grade it straight away. This will show whether your answer is correct, and also provide a sketch answer.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=File:L2_slide_ho.pdf&diff=3003File:L2 slide ho.pdf2013-09-09T13:02:18Z<p>LG: Presentation slides for the introductory matrix material for ECON61001.</p>
<hr />
<div>Presentation slides for the introductory matrix material for ECON61001.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=File:Lecture_2.pdf&diff=3002File:Lecture 2.pdf2013-09-09T13:01:30Z<p>LG: These notes cover the introductory matrix material, and should be studied before the lectures on ECON61001 start.</p>
<hr />
<div>These notes cover the introductory matrix material, and should be studied before the lectures on ECON61001 start.</div>LGhttp://eclr.humanities.manchester.ac.uk/index.php?title=Maths&diff=3001Maths2013-09-09T12:59:46Z<p>LG: </p>
<hr />
<div><br />
== Important Notice ==<br />
<br />
<br />
As preparation for the lectures in ECON61001 Econometric Methods, MSc students are expected to read and understand the material in Lecture Notes 2 (Matrices - 1) or in Lecture Slides 2, as part of Econ60901 PreSession Maths. (Give pdf links?)<br />
<br />
Students are also expected to tackle the corresponding questions on Econ61001 Exercise Sheet 2, either on paper or online using Maple TA, as part of the PreSession Maths course. <br />
<br />
Students on the MA program are welcome to try out this material.<br />
<br />
<br />
== Using Maple TA ==<br />
<br />
<br />
Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and automatically assessing student responses and performance. In this module, the questions on the paper Exercise Sheets are also available as Maple TA assignments. The idea is that you can answer the questions at your leisure, and have them correctly graded by Maple TA. Once you have answered a question, the correct answers and/or sketch answers are immediately available. You can go back and attempt the same question as many times as you like.<br />
<br />
Maple TA is located at<br />
<br />
[http://place36.placementtester.com/manchester]<br />
<br />
and this link is also given in each Lecture folder, for convenience. Login with your registration number (first 7 digits only): the password is also your registration number. On the page that follows, you can click on MyProfile and then Password Update to change your password. <br />
<br />
You should select the course<br />
<br />
ECON60901 PreSession Maths<br />
<br />
by clicking on the entry for this course. This will bring up a page of assignments. You can click on the assignment you want to do - the notation follows that in the exercise sheets. The assignments are organised as question groups, ExSheet 2 or ExSheet 2 Randomised Questions, or by individual question - a component of the question group. Picking a question group means that you have to answer all the questions in that group before you can grade your answers. Picking an individual question enables you to grade your answers immediately.<br />
<br />
When you click an an assignment, you are given choice between "Print assignment for off-line work" or "Work assignment on-line right now". If you choose to print, wait for the questions to be printed. When you have answered the questions, you can login and click on the assignment again, and choose the "Work ... online" option to enter your answers.<br />
<br />
Usually, you are given information on the type of response (number, formula etc) you are expected to give. If not, a textual response is required. In general, it is better to show arithmetic operators (+,-,*./,^) explicitly in your answers. Use of brackets to make clear meaning is also encouraged: what exactly is meant by "1/x - 1" - is it (1/x) - 1 or 1/(x - 1)? Additional information about the entry of vectors and matrices in your answers is given in a separate document, although the instructions are often repeated in questions.<br />
<br />
When you have finished one page of questions, click Next to go to the next part of the main question. You can also use the drop down menu of the Question item. When you have finished, click Grade and view details to see the marked version of your answers. This screen also contains a Comments section, which gives sketch answers. You can also click on Quit and Save, or on Print.<br />
<br />
You can save your work and return to it later if you clicked on Quit and Save when doing the assignment, but before clicking on Grade. To return, simply find the assignment in the Class Homepage list, and click on it.<br />
<br />
To inspect completed and marked assignments, start from the Assignments page by clicking on Class Homepage if necessary. Click on Gradebook, and select View Past Results. Select the assignment you want to inspect, click on Search, find the assignment in the list at the bottom of the page, and click on Details.<br />
<br />
The questions on ExSheet 2 Randomised Questions are randomised in the sense that Maple TA generates the numbers, which are different every time the question is attempted. These questions are intended for additional practice, should this be required, or for revision. These "randomised" questions are sometimes easier than, and sometimes harder than, the corresponding Exercise Sheet questions. If you find one of these randomised questions to be to hard, simply click on the "Refresh" button at the top of the page to get another question.</div>LG