Regression - Revision history

Rb: /* Additional resources */

2014-09-05T08:48:09Z

‎Additional resources

2014-09-04T13:44:36Z

‎Regression in Matrix Form

2014-09-04T11:56:58Z

‎Additional resources

2014-09-04T11:04:51Z

‎Regression in Matrix Form

2014-09-04T11:00:06Z

‎Regression in Matrix Form

2014-09-04T10:59:41Z

‎Regression in Matrix Form

2014-09-04T10:58:40Z

‎Regression in Matrix Form

2014-09-04T10:57:57Z

‎Regression

2014-09-04T10:47:21Z

2014-09-04T10:46:47Z

Undo revision 3287 by Rb (talk)

@@ Line 248: / Line 248: @@
 * Link to a full (55min) undergraduate, introductory lecture on regression [http://www.youtube.com/watch?v=AHAlqJTrPHE&list=PLW7MJJThJQQs3djo1EL6KCRFeCa6wpfYY]. After minute 22 this clip has material that is not covered here (<math>R^2</math> and statistical inference in regressions).
 * To understand how much matrix algebra you need to understand in order to handle the multiple regression model in matrix form you may want to look at these four clips:
-** The setup of a multiple linear regression framework in matrix form [http://youtu.be/3VQnW6yq1zs]
+** The setup of a multiple linear regression framework in matrix form [http://youtu.be/3VQnW6yq1zs?hd=1]
-** Matrix Addition [http://youtu.be/tKdbgIObtZk]
+** Matrix Addition [http://youtu.be/tKdbgIObtZk?hd=1]
-** Matrix Multiplication and Transpose [http://youtu.be/FL8zdDGcDiY]
+** Matrix Multiplication and Transpose [http://youtu.be/FL8zdDGcDiY?hd=1]
-** Matrix Inversion, Rank and linear dependence []
+** Matrix Inversion, Rank and linear dependence [http://youtu.be/xnHJ9QA1U8I?hd=1]
 = Examples =

@@ Line 239: / Line 239: @@
 But the real value will become obvious when we state the OLS estimator for the four coefficients that are hidden inside <math>\mathbf{\beta}</math>. It is the simply beautiful:
-<math>\mathbf{b} = (\mathbf{X}'\mathbf{X})^{-1} \mathbf{X}'\mathbf{Y}</math>
+<math>\mathbf{b} = (\mathbf{X}'\mathbf{X})^{-1} \mathbf{X}'\mathbf{y}</math>
 Just a few notes. ''First'', hidden behind <math>\mathbf{b}</math> are the sample estimates for the four coefficients in our model, <math>b_1</math>, <math>b_2</math>, <math>b_3</math> and <math>b_4</math> which are the estimates for the unknown <math>\beta_1</math>, <math>\beta_2</math>, <math>\beta_3</math> and <math>\beta_4</math>. , clearly you need to master some matrix algebra to be able to deal with this (see links in Additional Resources). But ''finally'' and most importantly, the above formula for the OLS esimators does not change when we add additional explanatory variables. Let’s say we wanted to add another variable to our model. We would merely have to recognise that we would have an additional column in <math>\mathbf{X}</math> and one additional parameter in <math>\mathbf{\beta}</math>. Neat!

@@ Line 245: / Line 245: @@
 == Additional resources ==
-* Khan Academy: Setup of the OLS problem and how to proof that the above formulae fr <math>a</math> and <math>b</math> ([https://www.khanacademy.org/math/probability/regression/regression-correlation/v/squared-error-of-regression-line] and four follow on clips - click on &quot;up next&quot; at the end of each clip). But be careful, in his video Salman Khan uses <math>m</math> for what we call <math>b</math> and <math>b</math> for what we call <math>a</math>. Life is never easy!
+* Khan Academy: Setup of the OLS problem and how to proof that the above formulae for <math>a</math> and <math>b</math> ([https://www.khanacademy.org/math/probability/regression/regression-correlation/v/squared-error-of-regression-line] and four follow on clips - click on &quot;up next&quot; at the end of each clip). But be careful, in his video Salman Khan uses <math>m</math> for what we call <math>b</math> and <math>b</math> for what we call <math>a</math>. Life is never easy!
 * Link to a full (55min) undergraduate, introductory lecture on regression [http://www.youtube.com/watch?v=AHAlqJTrPHE&list=PLW7MJJThJQQs3djo1EL6KCRFeCa6wpfYY]. After minute 22 this clip has material that is not covered here (<math>R^2</math> and statistical inference in regressions).
 = Examples =

@@ Line 200: / Line 200: @@
 y_2 &=& \beta_1 x_{1,2} + \beta_2 x_{2,2} + \beta_3 x_{3,2} + \beta_4 x_{4,2} + u_2 \\
 ... &=& ... \\
-y_12 &=& \beta_1 x_{1,12} + \beta_2 x_{2,12} + \beta_3 x_{3,12} + \beta_4 x_{4,12} + u_12\end{aligned}</math>
+y_{12} &=& \beta_1 x_{1,12} + \beta_2 x_{2,12} + \beta_3 x_{3,12} + \beta_4 x_{4,12} + u_12\end{aligned}</math>
 The only elements that remain constant for all observations are the coefficients. So and now we will introduce matrix notation. Let’s define the following vectors and matrices:

@@ Line 237: / Line 237: @@
 <math>\mathbf{y} = \mathbf{X} \mathbf{\beta} + \mathbf{u}</math>
-But the real value will become obvious when we state the OLS estimator for the four coefficients that are hidden inside <math>\mathbf{\beta}</math>. It is the simply beautiful
+But the real value will become obvious when we state the OLS estimator for the four coefficients that are hidden inside <math>\mathbf{\beta}</math>. It is the simply beautiful:
 <math>\mathbf{b} = (\mathbf{X}'\mathbf{X})^{-1} \mathbf{X}'\mathbf{Y}</math>

@@ Line 233: / Line 233: @@
                \end{array}
              \right);</math>
 == Additional resources ==

@@ Line 122: / Line 122: @@
 <math>res_i=y_i-\widehat{y}_i = y_i - a - bx_i.</math>
 ''Regression'' analysis is the statistical technique that finds the ''optimal'' values for <math>a</math> and <math>b</math>. We will soon see how to determine the best values for <math>a</math> and <math>b</math>. Before we do so we want to point out what ''optimal'' means in this context. In fact it implies that we want to minimise the values for <math>res_i</math> for all <math>i=1,...,n</math> (where <math>n</math> is the number of observations at hand) in some sense. in fact what we want to minimise is the ''sum of squared residuals''
@@ Line 230: / Line 237: @@
 * Khan Academy: Setup of the OLS problem and how to proof that the above formulae fr <math>a</math> and <math>b</math> ([https://www.khanacademy.org/math/probability/regression/regression-correlation/v/squared-error-of-regression-line] and four follow on clips - click on &quot;up next&quot; at the end of each clip). But be careful, in his video Salman Khan uses <math>m</math> for what we call <math>b</math> and <math>b</math> for what we call <math>a</math>. Life is never easy!
 * Link to a full (55min) undergraduate, introductory lecture on regression [http://www.youtube.com/watch?v=AHAlqJTrPHE&list=PLW7MJJThJQQs3djo1EL6KCRFeCa6wpfYY]. After minute 22 this clip has material that is not covered here (<math>R^2</math> and statistical inference in regressions).
 = Examples =

@@ Line 197: / Line 197: @@
 The only elements that remain constant for all observations are the coefficients. So and now we will introduce matrix notation. Let’s define the following vectors and matrices:
-<math>y_i = \beta_1 x_{1i} + \beta_2 x_{2i} + \beta_3 x_{3i} + \beta_4 x_{4i} + u_i</math>
+<math>\mathbf{y} = \left(\begin{array}{c}
+                y_1 \\
 == Additional resources ==