IV in R

This section is currently being written, especially the testing part.

Introduction

In this Section we will demonstrate how to use instrumental variables (IV) estimation (or better Two-Stage-Least Squares, 2SLS) to estimate the parameters in a linear regression model. If you want some more theoretical background on why we may need to use these techniques you may want to refer to any decent Econometrics textbook, or perhaps to this page.

Here we will be very short on the problem setup and big on the implementation! When you estimate a linear regression model, say

$y = \alpha_0 + \alpha_1 x_1 + \alpha_2 x_2 + \alpha_3 x_3 + u$

the most crucial of all assumptions you got to make is that the explanatory variables $x_1$ to $x_3$ are uncorrelated to the error term $u$. Of course, the error term $u$ is unobservable and hence it is impossible to empirically test this assumption (notwithstanding a related test introduced below) and the applied econometrician ought to think very carefully whether there may be any reason that makes it likely that this assumption might be breached. The seasoned econometrician would immediately rattle down simultaneity bias, measurement error and omitted relevant variables as the three most common causes for this to happen.

In some such situations you can actually fix the problem, e.g. by including the relevant variable into the model, but in others that is impossible and you need to accept that there is a high probability that, say, $x_3$ is correlated with $u$. We would then call $x_3$ an endogenous variable and all those explanatory variables that do not have that issue are called exogenous.. If you still persist with estimating that model by Ordinary Least Squares (OLS) you will have to accept that your estimated coefficients come from a random distribution that on average will not produce the correct (yet unknown) value, in technical lingo, the estimators are biased.

To the rescue come instrumental variables (IV) estimation. What we need to use this technique is what is called an instrumental variable. And if only $x_3$ is potentially correlated we need at least one such variable, but more could be useful as well. You always need at least as many instruments as you have endogenous variables. These instruments need to have the following crucial properties, they need to be correlated to the endogenous variable, uncorrelated to the error term and shouldn't be part of the model explaining the dependent variable $y$.

The basic idea of IV/2SLS

Here is a brief outline of what happens when you use IV, in the form of a TSLS regression.

1. Take all of your endogenous variables and run regressions with these as the dependent variable and all other exogenous and all instrumental variables as explanatory variables. From these regressions you get the predicted values for all your endogenous variables, e.g. $\hat{x}_3$. These regression(s) are called first stage regressions. The idea is that, as all explanatory variables in this first stage regression are assumed to be uncorrelated with the error term, the variable $\hat{x}_3$ is also uncorrelated with the unobserved error term $u$. All variation in $x_3$ that was correlated with the error term $u$ must have ended up in the error term of this auxilliary regression.

1. In the second stage of the procedure we return to our original regression model and replace $x_3$ with $\hat{x}_3$ and then estimate values for the parameters $\alpha_0$ to $\alpha_3$ by OLS. These estimators, at least for sufficiently large samples, will deliver unbiased estimates (technical lingo: consistent).

This sounds pretty easy. There is a slight complication, the standard errors that the second stage OLS regression delivers are incorrect and we need to calculate different standard errors. But that will happen automatically in the procedure below.

Implementation in R

The R Package needed is the AER package that we already recommended for use in the context of estimating robust standard errors. Included in that package is a function called ivreg which we will use. We explain how to use it by walking through an example.

Example

We will use the Women's Wages dataset to illustrate the use of the IV regression. The dependent variable which we use here is the log wage lwage and we are interested in whether the years of education, educ, has a positive influence on this log wage (here we mirror the analysis in Wooldridge's Example 15.1). An extremely simple model would be to estimate the following OLS regression which models lwage as a function of a constant and educ.

    reg_ex0 <- lm(lwage~educ,data=mydata)
    print(summary(reg_ex0))

which delivers

    Call:
    lm(formula = lwage ~ educ, data = mydata)
    Residuals:
         Min       1Q   Median       3Q      Max 
    -3.10256 -0.31473  0.06434  0.40081  2.10029 
    Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
    (Intercept)  -0.1852     0.1852  -1.000    0.318    
    educ          0.1086     0.0144   7.545 2.76e-13 ***
    ---
    Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
    Residual standard error: 0.68 on 426 degrees of freedom
      (325 observations deleted due to missingness)
    Multiple R-squared:  0.1179,	Adjusted R-squared:  0.1158 
    F-statistic: 56.93 on 1 and 426 DF,  p-value: 2.761e-13

This seems to indicate that every additional year of education increases the wage by almost 11% (recall the interpretation of a coefficient in a log-lin model!). The issue with this sort of model is that education is most likely to be correlated with individual characteristics that are important for the person's wage, but not modelled (and hence captured by the error term).

What we need is an instrument that meets the conditions outlined above and here and as in Wooldridge's example we use the father's education as an instrument. The way to do this is as follows:

    reg_iv0 <- ivreg(lwage~educ|fatheduc,data=mydata)
    print(summary(reg_iv0))

The ivreg function works very similar to the lm command (as usual use ?ivreg to get more detailed help). In fact the only difference is the specification of the instrument |fatheduc. The instruments follow the model specification. Behind the vertical lines we find the instrument used to instrument the educ variable.

The result is

    Call:
    ivreg(formula = lwage ~ educ | fatheduc, data = mydata)
    
    Residuals:
        Min      1Q  Median      3Q     Max 
    -3.0870 -0.3393  0.0525  0.4042  2.0677 
    
    Coefficients:
                Estimate Std. Error t value Pr(>|t|)  
    (Intercept)  0.44110    0.44610   0.989   0.3233  
    educ         0.05917    0.03514   1.684   0.0929 .
    ---
    Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
    
    Residual standard error: 0.6894 on 426 degrees of freedom
    Multiple R-Squared: 0.09344,	Adjusted R-squared: 0.09131 
    Wald test: 2.835 on 1 and 426 DF,  p-value: 0.09294 

Clearly, the effect of an additional year of education, has significantly dropped and is now only marginally significant. It is, of course, often a feature of IV estimation that the estimated standard errors are significantly smaller than the OLS estimators. The size of the standard error depends a lot on the strength of the relation between the endogenous explanatory variables which we can be checked by looking at the Rsquared of the regression of educ on fatheduc.

In order to illustrate the full functionality of the ivreg procedure we re-estimate the model with extra explanatory variables and more instruments than endogenous variables which means that really we are applying a 2SLS estimation (This is the example estimated in Wooldridge's Example 15.5). Let's start by estimating this model by OLS (as we need this result later, but result not shown here)

    reg_1 <- lm(lwage~educ+age+exper+expersq, data=mydata) # OLS estimation
    reg_1_sm <- summary(reg_1)
    print(reg_1_sm)

The estimated coefficient for educ is 0.1075 with standard error 0.0142 (the rest of the results are not shown). Then we estimate the TSLS regression with fatheduc and matheduc as instruments.

    reg_iv1 <- ivreg(lwage~educ+exper+expersq|fatheduc+motheduc+exper+expersq,data=mydata)
    print(summary(reg_iv1))

Before the vertical line we can see the model that is to be estimeted, lwage~educ+exper+expersq. All the action is after the vertical line. First we see the instrumental variables used to instrument educ, fatheduc+motheduc; this is followed by all the explanatory variables that are considered exogenous, exper+expersq.

When you have a model with a lot of variables this way of calling an IV estimation can be quite unwieldy as you have to replicate all the exogenous variables (in red). A slightly different, more economical way of asking R to do the same thing is as follows

    reg_iv1 <- ivreg(lwage~educ+exper+expersq|.-educ+fatheduc+motheduc,data=mydata)
    print(summary(reg_iv1))

After the vertical line you are basically telling R which variable to remove from the instrument set (the endogenous variable, .-educ) and which to add (+fatheduc+motheduc). Make sure you don't forget the decimal point straight after the vertical line when you use this way of specifying the instruments. What you get is the following

    Call:
    ivreg(formula = lwage ~ educ + age + exper + expersq | . - educ + 
        fatheduc, data = mydata)
    
    Residuals:
         Min       1Q   Median       3Q      Max 
    -3.09354 -0.32798  0.05094  0.37402  2.35375 
    
    Coefficients:
                  Estimate Std. Error t value Pr(>|t|)   
    (Intercept) -0.0513505  0.4936538  -0.104  0.91720   
    educ         0.0701490  0.0346051   2.027  0.04328 * 
    age         -0.0002287  0.0049140  -0.047  0.96290   
    exper        0.0436778  0.0134180   3.255  0.00122 **
    expersq     -0.0008790  0.0004064  -2.163  0.03111 * 
    ---
    Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
    
    Residual standard error: 0.6727 on 423 degrees of freedom
    Multiple R-Squared: 0.143,	Adjusted R-squared: 0.1349 
    Wald test: 6.225 on 4 and 423 DF,  p-value: 7.106e-05

IV related Testing procedures

One feature of IV estimations is that in general it is an inferior estimator of [math]\mathbf{\beta}[/math] if all explanatory variables are exogenous. In that case, assuming that all other Gauss-Markov assumptions are met, the OLS estimator is the BLUE estimator. In other words, IV estimators have larger standard errors for the coefficient estimates. Therefore, one would really like to avoid having to rely on IV estimators, unless, of course, they are the only estimators that deliver consistent estimates.

For this reason any application of IV, should be accompanied by evidence that establishes that it was necessary. Once that is established, one should also establish that the instruments chosen meet the necessary requirements (of being correlated with the endogenous variable and being exogenous to the regression error term).

Testing for exogeneity

You really only want to use IV/TSLS if you are really dealing with endogenous explanatory variables. If the variable you suspected wasn't endogenous, then IV only has disadvantages compared to OLS. Most crucially it will deliver much larger standard errors. For this reason you really want to make sure that you do have an endogeneity problem.

The celebrated test to use in this case is the Hausman test. Here we use a slightly different implementation to the original Hausman test, the so-called Hausman-Wu test.

In the end it is pretty straighforward and you only need simple regressions to implement it. In a first step you run the first step regression(s) of the TSLS procedure. In our case this is

    # First Stage
    first_stage <- lm(educ~age+exper+expersq+fatheduc+motheduc,data=mydata)

In a second step you add the residual(s) from this first step into the original model

    # Hausman
    Hausman_reg <- lm(lwage~educ+age+exper+expersq+first_stage$residuals,data=mydata)
    print(summary(Hausman_reg))

Now we need to compare this result to the one we got from the original model reg_2. If the educ is indeed endogenous, then the first stage regression should have isolated the variation of $x_3$ that was correlated with error term in the residual of the first stage regression. In that case the included first_stage$residuals should be relevant. As there may potentially be more than one endogenous variable and hence more than one first stage residual we use an F-test to test the null hypothesis that these residuals are irrelevant (and hence endogeneity not being a problem).

    HausWutest <- waldtest(Hausman_reg,.~.-first_stage$residuals)
    print(HausWutest)

    Wald test

    Model 1: lwage ~ educ + age + exper + expersq + first_stage$residuals
    Model 2: lwage ~ educ + age + exper + expersq
      Res.Df Df      F  Pr(>F)  
    1    422                    
    2    423 -1 2.9051 0.08903 .
    ---
    Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

The result is a p-value of 0.089. So at an $\alpha = 0.05$ we just fail to reject the null of $x_3$ being exogenous.

Sargan test for instrument validity

One crucial property of instruments is that they ought to be uncorrelated to the regression error terms [math]\mathbf{\varepsilon}[/math]. Instrument endogeneity is set as the null hypothesis of this test with the alternative hypothesis being that the instruments are endogenous.

1. Estimate the regression model by IV and save [math]\widehat{\mathbf{\varepsilon }}% _{IV}=\mathbf{y}-\mathbf{X}\widehat{\mathbf{\beta }}_{IV}[/math]

2. Regress

   [math]\widehat{\mathbf{\varepsilon }}_{IV}=\mathbf{Z\gamma +u}[/math]

3. Calculate [math]LM=nR^{2}[/math] from the auxilliary regresion in step 2. [math]LM[/math] is (under [math]H_{0}[/math]) [math]\chi ^{2}[/math] distributed with [math]\left( p-k\right) [/math] degrees of freedom.

MATLAB implementation of this test relies on the availability of the IV parameter estimates. They can be calculated as indicated above. In this section you can find a function called IVest that can deliver the required IV residuals by calling:

[biv,bseiv,resiv,r2iv] = IVest(y,x,z);

The third output are the IV residuals (refer to IVest for details) which can then be used as the dependent variable in the second step regression:

[b,bse,res,n,rss,r2] = OLSest(resiv,z,0);               % Step 2: calculate Step 2 regression
teststat = size(resiv,1)*r2;                            % Step 3: Calculates the nR^2 test statistic
pval = 1 - chi2cdf(teststat,(size(z,2)-size(x,2)));     % Step 3: Calculate p-value

It should be noted that this test is only applicable for an over-identified case when the z contains more columns than x. A function that implements this test can be found here.

Instrument relevance

The last instrument property that is required is that the instruments are correlated to the potentially endogenous variables. This is tested using a standard OLS regression that uses the endogenous variables as the dependent variable and all instrument variables (i.e. z) as the explanatory variables. We then need to check whether the restriction that all (non-constant) variables in z are relevant (F-test). If they are relevant, then the instruments are relevant. This is fact exactly what the Step 2 regressions of the Hausmann test do.