Linear Combinations - Revision history

Rb at 15:22, 29 August 2013

2013-08-29T15:22:44Z

Rb at 15:21, 29 August 2013

2013-08-29T15:21:43Z

Rb at 15:19, 29 August 2013

2013-08-29T15:19:02Z

Rb: Created page with " In this section, some properties of linear functions of random variables $X$ and $Y$ are considered. In this [[ Joint_Probability_Distributions#Functio..."

2013-08-29T15:16:04Z

Created page with " In this section, some properties of linear functions of random variables <math>X</math> and <math>Y</math> are considered. In this [[ Joint_Probability_Distributions#Functio..."

New page

In this section, some properties of linear functions of random variables <math>X</math> and <math>Y</math> are considered. In this [[ Joint_Probability_Distributions#Functions_of_Two_Random_Variables|Section]], a new random variable was defined as a function of two other variables. Let us, here, define a random variable <math>V</math> as a function of random variables <math>X</math> and <math>Y</math>,

<math>V=g\left( X,Y\right) ,</math>

with no restriction on the nature of the function or transformation <math>g</math>. In this section, the function <math>g</math> is restricted to be a '''linear''' function of <math>X</math> and <math>Y</math>:

<math>V=aX+bY+c,</math>

where <math>a,b</math> and <math>c</math> are constants. <math>V</math> is also called a '''linear combination''' of <math>X</math> and <math>Y</math>.

The properties developed in this section are specific to linear functions: they do not hold in general for nonlinear functions or transformations.

= The Expected Value of a Linear Combination =

This result is easy to remember: it amounts to saying that ''the expected value of a linear combination is the linear combination of the expected values''.

Even more simply, ''the expected value of a sum is a sum of expected values.''

If <math>V=aX+bY+c</math>, where <math>a,b,c</math> are constants, then

<math>E\left[ V\right] =E\left[ aX+bY+c\right] =aE\left[ X\right] +bE\left[ Y\right] +c.</math>

This result is a natural generalisation of that given in this previous [[ Joint_Probability_Distributions#Functions_of_Two_Random_Variables|Section]].

<ul>
<li>'''Proof '''(discrete random variables case only). Using the result in this previous [[ Joint_Probability_Distributions#Functions_of_Two_Random_Variables|Section]],
<math>\begin{aligned}
E\left[ V\right] &=&E\left[ g\left( X,Y\right) \right] \\
&=&E\left[ aX+bY+c\right] \\
&=&\sum_{x}\sum_{y}\left( ax+by+c\right) p\left( x,y\right) .
\end{aligned}</math>
From this point on, the proof just involves manipulation of the summation signs:
<math>\begin{aligned}
\sum_{x}\sum_{y}\left( ax+by+c\right) p\left( x,y\right)&=&a\sum_{x}\sum_{y}xp\left( x,y\right)+b\sum_{x}\sum_{y}yp\left(x,y\right) +c\sum_{x}\sum_{y}p\left( x,y\right) \\
&=&a\sum_{x}\left[ x\left( \sum_{y}p\left( x,y\right) \right) \right]+b\sum_{y}\left[ y\left( \sum_{x}p\left( x,y\right) \right) \right] +c \\
&=&a\sum_{x}\left[ xp_{X}\left( x\right) \right] +b\sum_{y}\left[yp_{Y}\left( y\right) \right] +c \\
&=&aE\left[ X\right] +bE\left[ Y\right] +c.
\end{aligned}</math></li></ul>

Notice the steps used:

* <math>\sum\limits_{y}xp\left( x,y\right) =x\sum\limits_{y}p\left(x,y\right) =xp_{X}\left( x\right) </math>, <math>\sum\limits_{x}yp\left( x,y\right)=y\sum\limits_{x}p\left( x,y\right) =xp_{X}\left( x\right) </math>, as <math>x</math> is constant with respect to <math>y</math> summation and <math>y</math> is constant with respect to <math>x</math> summation;
* we used: <math>\sum\limits_{x}\sum\limits_{y}p\left( x,y\right) =1</math>,
* we used the definitions of marginal distributions from Section;
* we usedthe definitions of expected value for discrete random variables.

Notice that nothing need be known about the joint probability distribution <math>p\left( x,y\right) </math> of <math>X</math> and <math>Y</math>. The result is also valid for continuous random variables, nothing need be known about <math>p\left( x,y\right) </math>.

=== Examples ===

<ol>
<li>In a [[ Joint_Probability_Distributions#Marginal_Probabilities|previous example]] we defined
<math>T=W+H,</math>
and had <math>E\left[ H\right] =1.3</math>, <math>E\left[ W\right] =0.9</math>, giving
<math>\begin{aligned}
E\left[ T\right] &=&E\left[ W\right] +E\left[ H\right] \\
&=&2.2
\end{aligned}</math>
confirming the result obtained earlier.</li>
<li>Suppose that the random variables <math>X</math> and <math>Y</math> have <math>E\left[ X\right]=0.5</math> and <math>E\left[ Y\right] =3.5</math>, and let
<math>V=5X-Y.</math>
Then,
<math>\begin{aligned}
E\left[ V\right] &=&5E\left[ X\right] -E\left[ Y\right] \\
&=&\left( 5\right) \left( 0.5\right) -3.5 \\
&=&-1.
\end{aligned}</math></li></ol>

== Generalisation ==

Let <math>X_{1},...,X_{n}</math> be random variables and <math>a_{1},....,a_{n},c</math> be constants, and define the random variable <math>W</math> by

<math>\begin{aligned}
W &=&a_{1}X_{1}+...+a_{n}X_{n}+c \\
&=&\sum_{i=1}^{n}a_{i}X_{i}+c.\end{aligned}</math>

Then,

<math>E\left[ W\right] =\sum_{i=1}^{n}a_{i}E\left[ X_{i}\right] +c.</math>

The proof uses the linear combination result for two variables '''repeatedly''':

<math>\begin{aligned}
E\left[ W\right] &=&a_{1}E\left[ X_{1}\right] +E\left[a_{2}X_{2}+...+a_{n}X_{n}+c\right] \\
&=&a_{1}E\left[ X_{1}\right] +a_{2}E\left[ X_{2}\right] +E\left[a_{3}X_{3}+...+a_{n}X_{n}+c\right] \\
&=&... \\
&=&a_{1}E\left[ X_{1}\right] +a_{2}E\left[ X_{2}\right] +...+a_{n}E\left[X_{n}\right] +c.\end{aligned}</math>

=== Examples ===

Let <math>E\left[ X_{1}\right] =2,E\left[ X_{2}\right] =-1,E\left[ X_{3}\right] =3</math>, <math>W=2X_{1}+5X_{2}-3X_{3}+4</math> and then

<math>\begin{aligned}
E\left[ W\right] &=&E\left[ 2X_{1}+5X_{2}-3X_{3}+4\right] \\
&=&2E\left[ X_{1}\right] +5E\left[ X_{2}\right] -3E\left[ X_{3}\right] +4 \\
&=&\left( 2\right) \left( 2\right) +\left( 5\right) \left( -1\right) -\left(3\right) \left( 3\right) +4 \\
&=&-6.\end{aligned}</math>

= The Variance of a Linear Combination =

== Two Variable Case ==

Let <math>V</math> be the random variable defined above as:

<math>V=aX+bY+c.</math>

What is <math>\limfunc{var}\left[ V\right] ?</math> To find this, it is helpful to use notation that will simplify the proof. By definition,

<math>\limfunc{var}\left[ V\right] =E\left[ \left( V-E\left[ V\right] \right) ^{2}\right] .</math>

Put

<math>\tilde{V}=V-E\left[ V\right]</math>

so that

<math>\limfunc{var}\left[ V\right] =E\left[ \tilde{V}^{2}\right].</math>

We saw that

<math>E\left[ V\right] =aE\left[ X\right] +bE\left[ Y\right] +c,</math>

so that

<math>\begin{aligned}
\tilde{V} &=&\left( aX+bY+c\right) -\left( aE\left[ X\right] +bE\left[ Y\right] +c\right) \\
&=&a\left( X-E\left[ X\right] \right) +b\left( Y-E\left[ Y\right] \right) \\
&=&a\tilde{X}+b\tilde{Y}\end{aligned}</math>

and then

<math>\limfunc{var}\left[ V\right] =E\left[ \tilde{V}^{2}\right] =E\left[ \left( a\tilde{X}+b\tilde{Y}\right) ^{2}\right].</math>

Notice that this does not depend on the constant <math>c</math>.

To make further progress, recall that in the current notation,

<math>\begin{aligned}
\limfunc{var}\left[ X\right] &=&E\left[ \tilde{X}^{2}\right] ,\;\;\;\limfunc{var}\left[ Y\right] =E\left[ \tilde{Y}^{2}\right] , \\
\limfunc{cov}\left[ X,Y\right] &=&E\left[ \left( X-E\left[ X\right] \right)\left( Y-E\left[ Y\right] \right) \right] \\
&=&E\left[ \tilde{X}\tilde{Y}\right] .\end{aligned}</math>

Then,

<math>\begin{aligned}
\limfunc{var}\left[ V\right] &=&E\left[ \left( a\tilde{X}+b\tilde{Y}\right)^{2}\right] \\
&=&E\left[ a^{2}\tilde{X}^{2}+2ab\tilde{X}\tilde{Y}+b^{2}\tilde{Y}^{2}\right]\\
&=&a^{2}E\left[ \tilde{X}^{2}\right] +2abE\left[ \tilde{X}\tilde{Y}\right]+b^{2}E\left[ \tilde{Y}^{2}\right] \\
&=&a^{2}\limfunc{var}\left[ X\right] +2ab\limfunc{cov}\left[ X,Y\right]+b^{2}\limfunc{var}\left[ Y\right] ,\end{aligned}</math>

using the linear combination result for expected values.

Summarising,

<ul>
<li>if <math>V=aX+bY+c</math>, then
<math>\limfunc{var}\left[ V\right] =a^{2}\limfunc{var}\left[ X\right] +2ab\limfunc{cov}\left[ X,Y\right] +b^{2}\limfunc{var}\left[ Y\right] .</math></li>
<li>If <math>X</math> and <math>Y</math> are '''uncorrelated''', so that <math>\limfunc{cov}\left[X,Y\right] =0</math>,
<math>\limfunc{var}\left[ V\right] =a^{2}\limfunc{var}\left[ X\right] +b^{2}\limfunc{var}\left[ Y\right] .</math></li>
<li>If <math>X</math> and <math>Y</math> are independent, the same result holds.</li></ul>

=== Examples ===

<ol>
<li>Suppose that <math>X</math> and <math>Y</math> are independent random variables with <math>\limfunc{var}\left[ X\right] =0.25</math>, <math>\limfunc{var}\left[ Y\right] =2.5</math>. If
<math>V=X+Y,</math>
then
<math>\begin{aligned}
\limfunc{var}\left[ V\right] &=&\limfunc{var}\left[ X\right] +\limfunc{var}\left[ Y\right] \\
&=&0.25+2.5 \\
&=&2.75.
\end{aligned}</math></li>
<li>A [[ Joint_Probability_Distributions#Marginal_Probabilities|previous example]] used the following random variables <math>W </math> and <math>H</math>, and defined <math>T</math> by
<math>T=W+H.</math>
There we found that <math>\limfunc{var}\left[ W\right] =0.49, \limfunc{var}\left[ H\right] =0.61</math>, while we also found <math>\limfunc{cov}\left[ W,H\right] =0.03</math>. Then,
<math>\begin{aligned}
\limfunc{var}\left[ T\right] &=&\limfunc{var}\left[ W+H\right] \\
&=&\limfunc{var}\left[ W\right] +2\limfunc{cov}\left[ W,H\right] +\limfunc{var}\left[ H\right]
\end{aligned}</math>
(since this is a case with <math>a=b=1)</math>. So,
<math>\limfunc{var}\left[ T\right] =0.49+\left( 2\right) \left( 0.03\right)+0.61=1.16.</math></li>
<li>For the same joint distribution, the difference between the income of husbands and wives is
<math>D=H-W.</math>
This case has <math>a=1</math> and <math>b=-1</math>, so that
<math>\begin{aligned}
\limfunc{var}\left[ D\right] &=&\left( 1\right) ^{2}\limfunc{var}\left[ H \right] +2\left( 1\right) \left( -1\right) \limfunc{cov}\left[ W,H\right] +\left( -1\right) ^{2}\limfunc{var}\left[ W\right] \\
&=&0.61-\left( 2\right) \left( 0.03\right) +0.49 \\
&=&1.04.
\end{aligned}</math></li></ol>

== Generalisation ==

To extend the result to the case of a linear combination of <math>n</math> random variables <math>X_{1},...,X_{n}</math> is messy because of the large number of covariance terms involved. So, we simplify by supposing that <math>X_{1},...,X_{n}</math> are uncorrelated random variables, with all covariances equal to zero: <math>\limfunc{cov}\left[ X_{i},X_{j}\right] =0,i\neq j</math>. Then,

<ul>
<li>for <math>X_{1},...,X_{n}</math> uncorrelated,
<math>\limfunc{var}\left[ \sum_{i=1}^{n}a_{i}X_{i}\right] =\sum_{i=1}^{n}a_{i}^{2}\limfunc{var}\left[ X_{i}\right] .</math></li>
<li>This also applies when <math>X_{1},...,X_{n}</math> are '''independent''' random variables.</li></ul>

== Standard Deviations ==

None of these results apply '''directly''' to standard deviations. Consider the simple case where <math>X</math> and <math>Y</math> are independent random variables and

<math>W=X+Y.</math>

Then,

<math>\begin{aligned}
\limfunc{var}\left[ W\right] &=&\sigma _{W}^{2} \\
&=&\limfunc{var}\left[ X\right] +\limfunc{var}\left[ Y\right] \\
&=&\sigma _{X}^{2}+\sigma _{Y}^{2}\end{aligned}</math>

and then

<math>\sigma _{W}=\sqrt{\sigma _{X}^{2}+\sigma _{Y}^{2}.}</math>

In general it is true that

<math>\sigma _{W}\neq \sigma _{X}+\sigma _{Y}.</math>

To illustrate, if <math>X_{1},X_{2}</math> and <math>X_{3}</math> are independent random variables with <math>\limfunc{var}\left[ X_{1}\right] =3,\limfunc{var}\left[ X_{2}\right]=1 </math> and <math>\limfunc{var}\left[ X_{3}\right] =5</math>, and if

<math>P=2X_{1}+5X_{2}-3X_{3},</math>

then

<math>\begin{aligned}
\limfunc{var}\left[ P\right] &=&2^{2}\limfunc{var}\left[ X_{1}\right] +5^{2}\limfunc{var}\left[ X_{2}\right] +\left( -3\right) ^{2}\limfunc{var}\left[X_{3}\right] \\
&=&\left( 4\right) \left( 3\right) +\left( 25\right) \left( 1\right) +\left(9\right) \left( 5\right) \\
&=&82, \\
\sigma _{P} &=&\sqrt{82}=9.06.\end{aligned}</math>

= Linear Combinations of Normal Random Variables =

The results in this section so far relate to characteristics of the probability distribution of a linear combination like

<math>V=aX+bY,</math>

not to the probability distribution itself. Indeed, part of the attraction of these results is that they can be obtained without having to find the probability distribution of <math>V</math>.

However, if we knew that <math>X</math> and <math>Y</math> had normal distributions then it would follow that <math>V</math> is also normally distributed. This innocuous sounding result is '''EXTREMELY IMPORTANT!''' It is also rather unusual: there are not many distributions for which this type of result holds.

More specifically,

<ul>
<li>if <math>X\sim N\left( \mu _{X},\sigma _{X}^{2}\right) </math> and <math>Y\sim N\left(\mu _{Y},\sigma _{Y}^{2}\right) </math> and <math>W=aX+bY+c</math>, then
<math>W\sim N\left( \mu _{W},\sigma _{W}^{2}\right)</math>
with
<math>\begin{aligned}
\mu _{W} &=&a\mu _{X}+b\mu _{Y}+c, \\
\sigma _{W}^{2} &=&a^{2}\sigma _{X}^{2}+2ab\sigma _{XY}+b^{2}\sigma _{Y}^{2}
\end{aligned}</math>
Note that '''independence '''of <math>X</math> and <math>Y</math> has '''not''' been assumed.</li>
<li>If <math>X_{1},...X_{n}</math> are uncorrelated random variables with <math>X_{i}\sim N\left( \mu _{i},\sigma _{i}^{2}\right) </math> and <math>W=\sum \limits_{i=1}^{n}a_{i}X_{i}</math>, where <math>a_{1},...,a_{n}</math> are constants, then
<math>W\sim N\left( \sum\limits_{i=1}^{n}a_{i}\mu_{i},\sum\limits_{i=1}^{n}a_{i}^{2}\sigma _{i}^{2}\right).</math></li>
<li>You should note that we did apply the general rules for the expected value and variance above, but '''in addition''', in this case, we also know what the type of distribution is, not only the values for <math>E[W]</math> and <math>Var[W]</math>.</li></ul>

Of course, standard normal distribution tables can be used in the usual way to compute probabilities of events involving <math>W</math>. This is illustrated in the following example.

=== Example ===

If <math>X\sim N\left( 20,5\right) ,Y\sim N\left( 30,11\right) </math>, <math>X</math> and <math>Y</math> independent, and

<math>D=X-Y,</math>

then

<math>D\sim N\left( -10,16\right)</math>

and

<math>\begin{aligned}
\Pr \left( D>0\right) &=&\Pr \left( Z>\dfrac{0-\left( -10\right) }{4}\right)\\
&=&\Pr \left( Z>2.5\right) \\
&=&0.00621,\end{aligned}</math>

where <math>Z\sim N\left( 0,1\right) </math>.

= Exercise 5 =

<ol>
<li>Consider the joint probability distributions
<math>\begin{tabular}{|ll|lll|}
\hline
& & \multicolumn{3}{|l|}{Values of $Y:$} \\
Probabilities & & $-1$ & $0$ & $1$ \\ \hline
Values of $X:$ & $-1$ & $0.0$ & $0.1$ & $0.0$ \\
& $0$ & $0.2$ & $0.3$ & $0.3$ \\
& $1$ & $0.0$ & $0.0$ & $0.1$ \\ \hline
\end{tabular}
,</math>
<math>\begin{tabular}{|ll|lll|}
\hline
& & \multicolumn{3}{|l|}{Values of $Y:$} \\
Probabilities & & $-1$ & $0$ & $1$ \\ \hline
Values of $X:$ & $-1$ & $0.02$ & $0.10$ & $0.08$ \\
& $0$ & $0.06$ & $0.30$ & $0.24$ \\
& $1$ & $0.02$ & $0.10$ & $0.08$ \\ \hline
\end{tabular}
.</math>
In each case,
<ol>
<li>find the marginal probability distributions of <math>X</math> and <math>Y;</math></li>
<li>find out whether <math>X</math> and <math>Y</math> are independent.</li></ol>
</li>
<li>You are an investment consultant employed by an investor who intends to invest in the stock market or in a deposit account with a building society. The percentage annual rate of return for the stock market is denoted by the random variable <math>S</math>. For simplicity we assume that this rate of return will be one of four values: -10%, 0%,10% or 20%. The annual rate of interest on the deposit account (denoted <math>R)</math> will be 4%, 6% or 8%. From previous experience, you believe that the joint probability distribution for these variables is:
<math>\begin{tabular}{|ll|cccc|}
\hline
& & \multicolumn{4}{|c|}{Values of $S:$} \\
Probabilities & & $-10$ & $0$ & $10$ & $20$ \\ \hline
Values of $R:$ & $4$ & $0$ & $0$ & $0.1$ & $0.1$ \\
& $6$ & $0$ & $0.1$ & $0.3$ & $0.1$ \\
& $8$ & $0.1$ & $0.1$ & $0.1$ & $0$ \\ \hline
\end{tabular}</math>
<ol>
<li><ol>
<li>Find the marginal probability distributions for <math>R</math> and <math>S</math>. What is the overall probability that the rate of return for the stock market will be positive? What is the probability that the rate of return for the stock market will exceed that from the building society deposit account?</li>
<li>Calculate the mean and variance using each of these marginal distributions. What does this information imply about the relative merits of the two types of investment?</li></ol>
</li>
<li>Calculate the (population) covariance and (population) correlation between <math>R</math> and <math>S</math>. How would you interpret the value of the correlation?</li>
<li><ol>
<li>One proposal you make to the investor is to split her savings equally between the stock market and the building society account. Find (using any appropriate method) the mean and variance of the random variable
<math>A=0.5R+0.5S.</math></li>
<li>Why might the investor prefer the proposed new 50/50 strategy to the simple strategies of investing all her money in the building society or in the stock market?</li></ol>
</li></ol>
</li>
<li>The random variables <math>X</math> and <math>Y</math> have <math>\mu _{X}=10</math>, <math>\sigma
_{X}=3,\mu _{Y}=-1,\sigma _{Y}=4</math>.
<ol>
<li>Find the mean and standard deviation of <math>V=X+Y</math> when
<ol>
<li><math>X</math> and <math>Y</math> are independent<math>;</math></li>
<li><math>\sigma _{XY}=-8</math>.</li></ol>
</li>
<li>Find the mean and standard deviation of
<math>W=3X-2Y+8</math>
when <math>X</math> and <math>Y</math> are independent.</li></ol>
</li>
<li>In the manufacture of disposable syringes, the manufacturing process produces cylinders of diameter <math>X_{1}\sim N\left( 20.2,0.04\right) </math> and plungers of diameter <math>X_{2}\sim N\left( 19.7,0.0225\right) </math>.
<ol>
<li>If the components are combined so that <math>X_{1}</math> is independent of <math>X_{2}</math>, what proportion of plungers will not fit?</li>
<li>Suppose now that the components are not independently combined, but that larger plungers tend to be combined with larger cylinders, leading to <math>\limfunc{cov}\left[ X_{1},X_{2}\right] =0.02</math>. What proportion of plungers will not fit now?</li></ol>
</li></ol>

= Exercises in EXCEL =

EXCEL can be used to perform the routine calculations involved in the material of Sections 9 and 10. For example, summation in EXCEL can be used to obtain the marginal probability distributions obtained in Section 9. Similarly, the tedious calculations involved in obtaining the separate means, variances, the covariance and the correlation from a given joint probability distribution can be carried out in EXCEL. Note, however, that there are no functions available within EXCEL specifically designed to compute the expected value and standard deviation for a random variable.

EXCEL can be used to obtain probabilities for a Normal random variable, as discussed in Section 8.10. Thus, having obtained the mean and variance of a linear combination of normal random variables, the NORMDIST function can be used to obtain probabilities for values of this linear combination. This yields more accurate probabilities than those which can be obtained using statistical tables.

= Footnotes =

@@ Line 140: / Line 140: @@
 <math>\begin{aligned}
 var\left[ X\right] &=&E\left[ \tilde{X}^{2}\right] ,\;\;\;var\left[ Y\right] =E\left[ \tilde{Y}^{2}\right] , \\
-\limfunc{cov}\left[ X,Y\right] &=&E\left[ \left( X-E\left[ X\right] \right)\left( Y-E\left[ Y\right] \right) \right] \\
+Cov\left[ X,Y\right] &=&E\left[ \left( X-E\left[ X\right] \right)\left( Y-E\left[ Y\right] \right) \right] \\
 &=&E\left[ \tilde{X}\tilde{Y}\right] .\end{aligned}</math>
@@ Line 149: / Line 149: @@
 &=&E\left[ a^{2}\tilde{X}^{2}+2ab\tilde{X}\tilde{Y}+b^{2}\tilde{Y}^{2}\right]\\
 &=&a^{2}E\left[ \tilde{X}^{2}\right] +2abE\left[ \tilde{X}\tilde{Y}\right]+b^{2}E\left[ \tilde{Y}^{2}\right] \\
-&=&a^{2}var\left[ X\right] +2ab\limfunc{cov}\left[ X,Y\right]+b^{2}var\left[ Y\right] ,\end{aligned}</math>
+&=&a^{2}var\left[ X\right] +2abCov\left[ X,Y\right]+b^{2}var\left[ Y\right] ,\end{aligned}</math>
 using the linear combination result for expected values.
@@ Line 157: / Line 157: @@
 <ul>
 <li><p>if <math>V=aX+bY+c</math>, then</p>
-<p><math>var\left[ V\right] =a^{2}var\left[ X\right] +2ab\limfunc{cov}\left[ X,Y\right] +b^{2}var\left[ Y\right] .</math></p></li>
+<p><math>var\left[ V\right] =a^{2}var\left[ X\right] +2abCov\left[ X,Y\right] +b^{2}var\left[ Y\right] .</math></p></li>
-<li><p>If <math>X</math> and <math>Y</math> are '''uncorrelated''', so that <math>\limfunc{cov}\left[X,Y\right] =0</math>,</p>
+<li><p>If <math>X</math> and <math>Y</math> are '''uncorrelated''', so that <math>Cov\left[X,Y\right] =0</math>,</p>
 <p><math>var\left[ V\right] =a^{2}var\left[ X\right] +b^{2}var\left[ Y\right] .</math></p></li>
 <li><p>If <math>X</math> and <math>Y</math> are independent, the same result holds.</p></li></ul>
@@ Line 175: / Line 175: @@
 <li><p>A [[ Joint_Probability_Distributions#Marginal_Probabilities|previous example]] used the following random variables <math>W </math> and <math>H</math>, and defined <math>T</math> by</p>
 <p><math>T=W+H.</math></p>
-<p>There we found that <math>var\left[ W\right] =0.49, var\left[ H\right] =0.61</math>, while we also found <math>\limfunc{cov}\left[ W,H\right] =0.03</math>. Then,</p>
+<p>There we found that <math>var\left[ W\right] =0.49, var\left[ H\right] =0.61</math>, while we also found <math>Cov\left[ W,H\right] =0.03</math>. Then,</p>
 <p><math>\begin{aligned}
      var\left[ T\right] &=&var\left[ W+H\right] \\
-     &=&var\left[ W\right] +2\limfunc{cov}\left[ W,H\right] +var\left[ H\right]
+     &=&var\left[ W\right] +2Cov\left[ W,H\right] +var\left[ H\right]
      \end{aligned}</math></p>
 <p>(since this is a case with <math>a=b=1)</math>. So,</p>
@@ Line 186: / Line 186: @@
 <p>This case has <math>a=1</math> and <math>b=-1</math>, so that</p>
 <p><math>\begin{aligned}
-     var\left[ D\right] &=&\left( 1\right) ^{2}var\left[ H \right] +2\left( 1\right) \left( -1\right) \limfunc{cov}\left[ W,H\right] +\left( -1\right) ^{2}var\left[ W\right] \\
+     var\left[ D\right] &=&\left( 1\right) ^{2}var\left[ H \right] +2\left( 1\right) \left( -1\right) Cov\left[ W,H\right] +\left( -1\right) ^{2}var\left[ W\right] \\
      &=&0.61-\left( 2\right) \left( 0.03\right) +0.49 \\
      &=&1.04.
@@ Line 193: / Line 193: @@
 == Generalisation ==
-To extend the result to the case of a linear combination of <math>n</math> random variables <math>X_{1},...,X_{n}</math> is messy because of the large number of covariance terms involved. So, we simplify by supposing that <math>X_{1},...,X_{n}</math> are uncorrelated random variables, with all covariances equal to zero: <math>\limfunc{cov}\left[ X_{i},X_{j}\right] =0,i\neq j</math>. Then,
+To extend the result to the case of a linear combination of <math>n</math> random variables <math>X_{1},...,X_{n}</math> is messy because of the large number of covariance terms involved. So, we simplify by supposing that <math>X_{1},...,X_{n}</math> are uncorrelated random variables, with all covariances equal to zero: <math>Cov\left[ X_{i},X_{j}\right] =0,i\neq j</math>. Then,
 <ul>

@@ Line 107: / Line 107: @@
 <math>V=aX+bY+c.</math>
-What is <math>\limfunc{var}\left[ V\right] ?</math> To find this, it is helpful to use notation that will simplify the proof. By definition,
+What is <math>var\left[ V\right] ?</math> To find this, it is helpful to use notation that will simplify the proof. By definition,
-<math>\limfunc{var}\left[ V\right] =E\left[ \left( V-E\left[ V\right] \right) ^{2}\right] .</math>
+<math>var\left[ V\right] =E\left[ \left( V-E\left[ V\right] \right) ^{2}\right] .</math>
 Put
@@ Line 117: / Line 117: @@
 so that
-<math>\limfunc{var}\left[ V\right] =E\left[ \tilde{V}^{2}\right].</math>
+<math>var\left[ V\right] =E\left[ \tilde{V}^{2}\right].</math>
 We saw that
@@ Line 132: / Line 132: @@
 and then
-<math>\limfunc{var}\left[ V\right] =E\left[ \tilde{V}^{2}\right] =E\left[ \left( a\tilde{X}+b\tilde{Y}\right) ^{2}\right].</math>
+<math>var\left[ V\right] =E\left[ \tilde{V}^{2}\right] =E\left[ \left( a\tilde{X}+b\tilde{Y}\right) ^{2}\right].</math>
 Notice that this does not depend on the constant <math>c</math>.
@@ Line 139: / Line 139: @@
 <math>\begin{aligned}
-\limfunc{var}\left[ X\right] &=&E\left[ \tilde{X}^{2}\right] ,\;\;\;\limfunc{var}\left[ Y\right] =E\left[ \tilde{Y}^{2}\right] , \\
+var\left[ X\right] &=&E\left[ \tilde{X}^{2}\right] ,\;\;\;var\left[ Y\right] =E\left[ \tilde{Y}^{2}\right] , \\
 \limfunc{cov}\left[ X,Y\right] &=&E\left[ \left( X-E\left[ X\right] \right)\left( Y-E\left[ Y\right] \right) \right] \\
 &=&E\left[ \tilde{X}\tilde{Y}\right] .\end{aligned}</math>
@@ Line 146: / Line 146: @@
 <math>\begin{aligned}
-\limfunc{var}\left[ V\right] &=&E\left[ \left( a\tilde{X}+b\tilde{Y}\right)^{2}\right] \\
+var\left[ V\right] &=&E\left[ \left( a\tilde{X}+b\tilde{Y}\right)^{2}\right] \\
 &=&E\left[ a^{2}\tilde{X}^{2}+2ab\tilde{X}\tilde{Y}+b^{2}\tilde{Y}^{2}\right]\\
 &=&a^{2}E\left[ \tilde{X}^{2}\right] +2abE\left[ \tilde{X}\tilde{Y}\right]+b^{2}E\left[ \tilde{Y}^{2}\right] \\
-&=&a^{2}\limfunc{var}\left[ X\right] +2ab\limfunc{cov}\left[ X,Y\right]+b^{2}\limfunc{var}\left[ Y\right] ,\end{aligned}</math>
+&=&a^{2}var\left[ X\right] +2ab\limfunc{cov}\left[ X,Y\right]+b^{2}var\left[ Y\right] ,\end{aligned}</math>
 using the linear combination result for expected values.
@@ Line 157: / Line 157: @@
 <ul>
 <li><p>if <math>V=aX+bY+c</math>, then</p>
-<p><math>\limfunc{var}\left[ V\right] =a^{2}\limfunc{var}\left[ X\right] +2ab\limfunc{cov}\left[ X,Y\right] +b^{2}\limfunc{var}\left[ Y\right] .</math></p></li>
+<p><math>var\left[ V\right] =a^{2}var\left[ X\right] +2ab\limfunc{cov}\left[ X,Y\right] +b^{2}var\left[ Y\right] .</math></p></li>
 <li><p>If <math>X</math> and <math>Y</math> are '''uncorrelated''', so that <math>\limfunc{cov}\left[X,Y\right] =0</math>,</p>
-<p><math>\limfunc{var}\left[ V\right] =a^{2}\limfunc{var}\left[ X\right] +b^{2}\limfunc{var}\left[ Y\right] .</math></p></li>
+<p><math>var\left[ V\right] =a^{2}var\left[ X\right] +b^{2}var\left[ Y\right] .</math></p></li>
 <li><p>If <math>X</math> and <math>Y</math> are independent, the same result holds.</p></li></ul>
@@ Line 165: / Line 165: @@
 <ol>
-<li><p>Suppose that <math>X</math> and <math>Y</math> are independent random variables with <math>\limfunc{var}\left[ X\right] =0.25</math>, <math>\limfunc{var}\left[ Y\right] =2.5</math>. If</p>
+<li><p>Suppose that <math>X</math> and <math>Y</math> are independent random variables with <math>var\left[ X\right] =0.25</math>, <math>var\left[ Y\right] =2.5</math>. If</p>
 <p><math>V=X+Y,</math></p>
 <p>then</p>
 <p><math>\begin{aligned}
-     \limfunc{var}\left[ V\right] &=&\limfunc{var}\left[ X\right] +\limfunc{var}\left[ Y\right] \\
+     var\left[ V\right] &=&var\left[ X\right] +var\left[ Y\right] \\
      &=&0.25+2.5 \\
      &=&2.75.
@@ Line 175: / Line 175: @@
 <li><p>A [[ Joint_Probability_Distributions#Marginal_Probabilities|previous example]] used the following random variables <math>W </math> and <math>H</math>, and defined <math>T</math> by</p>
 <p><math>T=W+H.</math></p>
-<p>There we found that <math>\limfunc{var}\left[ W\right] =0.49, \limfunc{var}\left[ H\right] =0.61</math>, while we also found <math>\limfunc{cov}\left[ W,H\right] =0.03</math>. Then,</p>
+<p>There we found that <math>var\left[ W\right] =0.49, var\left[ H\right] =0.61</math>, while we also found <math>\limfunc{cov}\left[ W,H\right] =0.03</math>. Then,</p>
 <p><math>\begin{aligned}
-     \limfunc{var}\left[ T\right] &=&\limfunc{var}\left[ W+H\right] \\
+     var\left[ T\right] &=&var\left[ W+H\right] \\
-     &=&\limfunc{var}\left[ W\right] +2\limfunc{cov}\left[ W,H\right] +\limfunc{var}\left[ H\right]
+     &=&var\left[ W\right] +2\limfunc{cov}\left[ W,H\right] +var\left[ H\right]
      \end{aligned}</math></p>
 <p>(since this is a case with <math>a=b=1)</math>. So,</p>
-<p><math>\limfunc{var}\left[ T\right] =0.49+\left( 2\right) \left( 0.03\right)+0.61=1.16.</math></p></li>
+<p><math>var\left[ T\right] =0.49+\left( 2\right) \left( 0.03\right)+0.61=1.16.</math></p></li>
 <li><p>For the same joint distribution, the difference between the income of husbands and wives is</p>
 <p><math>D=H-W.</math></p>
 <p>This case has <math>a=1</math> and <math>b=-1</math>, so that</p>
 <p><math>\begin{aligned}
-     \limfunc{var}\left[ D\right] &=&\left( 1\right) ^{2}\limfunc{var}\left[ H \right] +2\left( 1\right) \left( -1\right) \limfunc{cov}\left[ W,H\right] +\left( -1\right) ^{2}\limfunc{var}\left[ W\right] \\
+     var\left[ D\right] &=&\left( 1\right) ^{2}var\left[ H \right] +2\left( 1\right) \left( -1\right) \limfunc{cov}\left[ W,H\right] +\left( -1\right) ^{2}var\left[ W\right] \\
      &=&0.61-\left( 2\right) \left( 0.03\right) +0.49 \\
      &=&1.04.
@@ Line 197: / Line 197: @@
 <ul>
 <li><p>for <math>X_{1},...,X_{n}</math> uncorrelated,</p>
-<p><math>\limfunc{var}\left[ \sum_{i=1}^{n}a_{i}X_{i}\right] =\sum_{i=1}^{n}a_{i}^{2}\limfunc{var}\left[ X_{i}\right] .</math></p></li>
+<p><math>var\left[ \sum_{i=1}^{n}a_{i}X_{i}\right] =\sum_{i=1}^{n}a_{i}^{2}var\left[ X_{i}\right] .</math></p></li>
 <li><p>This also applies when <math>X_{1},...,X_{n}</math> are '''independent''' random variables.</p></li></ul>

@@ Line 209: / Line 209: @@
 <math>\begin{aligned}
-\limfunc{var}\left[ W\right] &=&\sigma _{W}^{2} \\
+var\left[ W\right] &=&\sigma _{W}^{2} \\
-&=&\limfunc{var}\left[ X\right] +\limfunc{var}\left[ Y\right] \\
+&=&var\left[ X\right] +var\left[ Y\right] \\
 &=&\sigma _{X}^{2}+\sigma _{Y}^{2}\end{aligned}</math>
@@ Line 221: / Line 221: @@
 <math>\sigma _{W}\neq \sigma _{X}+\sigma _{Y}.</math>
-To illustrate, if <math>X_{1},X_{2}</math> and <math>X_{3}</math> are independent random variables with <math>\limfunc{var}\left[ X_{1}\right] =3,\limfunc{var}\left[ X_{2}\right]=1 </math> and <math>\limfunc{var}\left[ X_{3}\right] =5</math>, and if
+To illustrate, if <math>X_{1},X_{2}</math> and <math>X_{3}</math> are independent random variables with <math>var\left[ X_{1}\right] =3,var\left[ X_{2}\right]=1 </math> and <math>var\left[ X_{3}\right] =5</math>, and if
 <math>P=2X_{1}+5X_{2}-3X_{3},</math>
@@ Line 228: / Line 228: @@
 <math>\begin{aligned}
-\limfunc{var}\left[ P\right] &=&2^{2}\limfunc{var}\left[ X_{1}\right] +5^{2}\limfunc{var}\left[ X_{2}\right] +\left( -3\right) ^{2}\limfunc{var}\left[X_{3}\right] \\
+var\left[ P\right] &=&2^{2}var\left[ X_{1}\right] +5^{2}var\left[ X_{2}\right] +\left( -3\right) ^{2}var\left[X_{3}\right] \\
 &=&\left( 4\right) \left( 3\right) +\left( 25\right) \left( 1\right) +\left(9\right) \left( 5\right) \\
 &=&82, \\
@@ Line 278: / Line 278: @@
 where <math>Z\sim N\left( 0,1\right) </math>.
 = Footnotes =

Linear Combinations - Revision history

Rb at 15:22, 29 August 2013

Rb at 15:21, 29 August 2013

Rb at 15:19, 29 August 2013

Rb: Created page with " In this section, some properties of linear functions of random variables X and Y are considered. In this [[ Joint_Probability_Distributions#Functio..."

Rb: Created page with " In this section, some properties of linear functions of random variables $X$ and $Y$ are considered. In this [[ Joint_Probability_Distributions#Functio..."