Do instrumental variables introduce problematic multicollinearity? If not, why not? If so, how is it deal with?
Do instrumental variables introduce problematic multicollinearity? If not, why not? If so, how is it deal with?
Loading saved threads...
Joshua Schroijen · External communityPost link
External question — Cross Validated Stack Exchange
Author: Joshua Schroijen
Original post: https://stats.stackexchange.com/questions/664973
License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/
Adaptation: HTML converted to plain text; contact email addresses removed.
I've been confused about something. There are, among others, two important rules of linear regression modelling:
The independent variables should be uncorrelated with the residuals
Independent variables should be uncorrelated with one another
Usually, to address violations of rule 1 we are told to introduce instrumental variables. Instrumental variables cause/explain variance in the dependent variable by causing/explaining variance in an independent variable which by itself is violating rule 1.
But doesn't the construction of the instrumental variable, which will be colinear/correlated with its corresponding independent variable by it, imply a violation of rule 2?
How is the use of instrumental variables not just trading one problem for another?
Quote
Report
Jonathan · External communityPost link
External answer — Cross Validated Stack Exchange
Author: Jonathan
Original post: https://stats.stackexchange.com/a/665004
License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/
Adaptation: HTML converted to plain text; contact email addresses removed.
Before answering your question, it is necessary to note that
there is no rule that the independent variables have to be uncorrelated
. We do require
no perfect collinearity
, however. That is, it must not be possible to construct one independent variable as a linear combination of other independent variables. Because the wording of your first rule is also problematic
$^*$
, I will not directly discuss violations of these rules.
Rather, a better starting point for sorting out your confusion is that you have identified two potential problems that we might want to deal with in a regression
Confounding (or omitted variable bias)
Multicollinearity
And you have correctly spotted that dealing with one would seem to aggravate the other.
My short answer is that
confounding is a problem to be dealt with, but multicollinearity is not
. Multicollinearity is better understood as a concept that may help us explain what would happen to the coefficients (and standard errors) from a regression if we were able to adjust the correlation between the independent variables up or down
My slightly longer answer is that regression can be approached from different perspectives, and from no perspective (that I can come up with) would we be simultaneously concerned with multicollinearity and the use of instrumental variables.
From one perspective, we are interested in estimating the effect of
$X$
on
$Y$
. We will then add additional independent variables to this regression because they are confounders (causes of both
$X$
and
$Y$
), or we might use an instrumental variable that is an important cause of
$X$
because we do not believe that it is possible to include all confounders in the regression. From this perspective, most variables are included precisely because they are correlated with
$X$
for some particular reason. Sometimes we will find that, having accounted for all confounders, there is very little variation left in in
$X$
. This could be described as a problem of multicollinearity, but it cannot be solved, because all independent variables and/or instruments that have been included in the regression are necessary if we want to estimate the true effect of
$X$
on
$Y$
.
From another perspective, we have a larger set of independent variables and we are assuming that all these variables are independent of the error term (all unobserved variables that explain
$Y$
but are not part of the data set). Therefore, there can be no need to use instrumental variables. We start out with the objective of interpreting the coefficient on every single independent variable. However, there might be a high degree of multicollinearity among the independent variables. This would signal that it might be very hard to get precise or stable estimates of the independent effects. Therefore, we might opt for omitting some of the independent variables. From the first perspective, that would make no sense, because it would introduce confounding, but from this perspective we would think of the coefficients on the remaining independent variables as representing something like ‘combined’ rather than independent effects.
Have a look at this related question/answer
on the (non-)trade-off between dealing with omitted variable bias and multicollinearity
, which focuses on the, arguably, more typical case of adding control variables to the regression rather than using instruments. Instrumental variables are independent variables in a sense, but they are not exactly the same. I believe that adds unnecessary complexity to resolving your confusion.
$^*$
The
errors
have to be uncorrelated with the independent variables; the
residuals
and independent variables are uncorrelated by construction.
Quote
Report
Post Reply
Quoted from Forex.com.bd-Editorial External answer — Cross Validated Stack Exchange Author: Jonathan Source score (net votes, not local likes): 6 Original post: https://stats.stackexchange.com/a/665004 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. Before answering your question, it is necessary to note that there is no rule that the independent variables have to be uncorrelated . We do require no perfect collinearity , however. That is, it must not be possible to construct one independent variable as a linear combination of other independent variables. Because the wording of your first rule is also problematic $^*$ , I will not directly discuss violations of these rules. Rather, a better starting point for sorting out your confusion is that you have identified two potential problems that we might want to deal with in a regression Confounding (or omitted variable bias) Multicollinearity And you have correctly spotted that dealing with one would seem to aggravate the other. My short answer is that confounding is a problem to be dealt with, but multicollinearity is not . Multicollinearity is better understood as a concept that may help us explain what would happen to the coefficients (and standard errors) from a regression if we were able to adjust the correlation between the independent variables up or down My slightly longer answer is that regression can be approached from different perspectives, and from no perspective (that I can come up with) would we be simultaneously concerned with multicollinearity and the use of instrumental variables. From one perspective, we are interested in estimating the effect of $X$ on $Y$ . We will then add additional independent variables to this regression because they are confounders (causes of both $X$ and $Y$ ), or we might use an instrumental variable that is an important cause of $X$ because we do not believe that it is possible to include all confounders in the regression. From this perspective, most variables are included precisely because they are correlated with $X$ for some particular reason. Sometimes we will find that, having accounted for all confounders, there is very little variation left in in $X$ . This could be described as a problem of multicollinearity, but it cannot be solved, because all independent variables and/or instruments that have been included in the regression are necessary if we want to estimate the true effect of $X$ on $Y$ . From another perspective, we have a larger set of independent variables and we are assuming that all these variables are independent of the error term (all unobserved variables that explain $Y$ but are not part of the data set). Therefore, there can be no need to use instrumental variables. We start out with the objective of interpreting the coefficient on every single independent variable. However, there might be a high degree of multicollinearity among the independent variables. This would signal that it might be very hard to get precise or stable estimates of the independent effects. Therefore, we might opt for omitting some of the independent variables. From the first perspective, that would make no sense, because it would introduce confounding, but from this perspective we would think of the coefficients on the remaining independent variables as representing something like ‘combined’ rather than independent effects. Have a look at this related question/answer on the (non-)trade-off between dealing with omitted variable bias and multicollinearity , which focuses on the, arguably, more typical case of adding control variables to the regression rather than using instruments. Instrumental variables are independent variables in a sense, but they are not exactly the same. I believe that adds unnecessary complexity to resolving your confusion. $^*$ The errors have to be uncorrelated with the independent variables; the residuals and independent variables are uncorrelated by construction.
Checking account access…