Multicollinearity

CORE DEFINITION

Multicollinearity refers to the situation in multiple linear regression where independent variables are highly correlated, leading to inaccurate regression estimates. In such cases, the coefficients of the multiple regression may change drastically due to small changes in the model or data. In a sample dataset, multicollinearity does not affect the overall predictive power or reliability of the model; it only affects the parameters of individual predictors. In short, a multiple regression model with collinear predictors can indicate the overall predictive reliability of the model, but it may not provide valid results for individual predictors, nor can it determine which predictors are redundant. It is important to note that in regression analysis, the phrase "no multicollinearity" is often used to refer to the absence of "perfect multicollinearity," meaning that there is an exact linear relationship among predictors. In such a case, the model matrix X is not of full rank, and therefore its moment matrix X^T X is not invertible.