Homoskedastic Linear Model

Gauss Markov Assumptions

• Linearity : $Y = X\beta$
•  Strict Exogeneity : $E(\epsilon_i X) = 0$
• Unconditional mean of error $E(\epsilon_i) = 0$
• Cross moment of residuals and regressors is zero, X is orthogonal to $\epsilon$ : $E(X_i \epsilon_i) = 0$
• No multicollinearity - $rank(X) = k$
•  Spherical error variance : $E(\epsilon_i^2 X) = \sigma^2 ; E(\epsilon \epsilon’ X) = \sigma^2 I_n$
•  $\epsilon X \sim N(0,\sigma^2 I_n)$
• ${(Y_i,X_i): i = 1, …, n}$ are i.i.d.

This gives us $\hat{\beta} = (X'X)^{-1}X'Y$ $V(\beta) = \sigma^2(X'X)^{-1}$

where, under homoskedasticity, $\hat{\sigma^2} = \frac{e’e}{n-k}$, where $e = y - X\beta$

MLE

Density of error:

Generalised least squares

If covariance matrix of errors is known: $E(\epsilon \epsilon’ |X) = \Omega$ $\hat{\beta}_{GLS} = (X'\Omega^{-1}X)^{-1}X'\Omega^{-1}Y$ $\mathbb{V}(\hat{\beta}_{GLS}) = (X'\Omega^{-1}X)^{-1}$ Restrited OLS - optimise: $L(b,\lambda) = (Y-Xb)’(Y-Xb)+2\lambda(Rb-r)$

Huber-White Sandwich ‘Robust’ SEs

Under homoskedasticity, $E[(\hat{\beta} - \beta)(\hat{\beta} - \beta)'] = (X'X)^{-1}X'E(\epsilon \epsilon')X(X'X)^{-1}$ which simplifies to $V(\beta) = \sigma^2(X’X)^{-1}$ because of the assumption $E(\epsilon \epsilon’) = \sigma^2 I$. If this is not true (i.e. heteroskedasticity is present), the variance covariance formula is $E[(\hat{\beta} - \beta)(\hat{\beta} - \beta)'] = (X'X)^{-1}X'ee'X(X'X)^{-1} = \sigma_\epsilon^2 (X'X)^{-1}X' \Omega X(X'X)^{-1}$

$V(\hat{\beta}) = Q^{-1}\Omega Q^{-1}$ Where, $Q=\mathbb{E}X_iXi’, \Omega = \mathbb{E}\hat{u_i}^2 X_i X_i’$

Fitted values and residuals

Define 2 matrices that are positive semidifinite, symmetric,idempotent:

• $P_x = X(X’X)^{-1}X’$ - Hat Matrix - projector into $span(X)$
• $M_x = I_n - P_x = I_n - X(X’X)^{-1}X’$ - Annihilator Matrix - projector into $span^{\bot}(X)$

Fitted Value: $\hat{Y} = P_x Y$ Residual: $e = M_x Y$

Model Fit : $R^2 , F$

R-squared

ESS = Explained Sum of Squares

TSS = Total Sum of Squares

RSS = Residual Sum of Squares

Adjusted $R^2$

Mean Squared Error (MSE) = $\mathbb{E}(y-X_i’\hat{\beta})$

F statistic

$\text{F Stat} = (R\hat{\beta}-r)' (s^2 R(X'X)^{-1}R')^{-1} (R\hat{\beta}-r) / q$ $\text{F Stat} = \frac{(TSS-RSS)/(k-1)}{RSS/(n-k)} = \frac{R^2/(k-1)}{(1-R^2)/(n-k)} \sim F_{k-1,n-k}$

Wald Statistic

$W_n = nh(\hat{\beta_n})' \left( \frac{\partial h(\hat{\beta_n})}{\partial \beta'} \hat{V_n} \frac{\partial h(\hat{\beta_n})'}{\partial \beta} \right) nh(\hat{\beta_n})$ reject $H_0$ if $W_q > \chi^2_{q,1-\alpha} = F/q$

Bonferroni correction - multiple hypothesis correction, J hypotheses : $\tau = \alpha/J$ Holms-Bonferroni : $\alpha/J \ldots \alpha/(J-n)$ each step

Instrumental Variables

Exogeneity violated when $E(X_i \epsilon_i) \neq 0$. OLS estimates no longer consistent.

IV requirements:

• $Cov(Z,X) \neq 0$ - Relevance
• $Cov(Z,\epsilon) = 0 ; Z \bot \epsilon$ - Exogeneity / Exclusion restriction
• Affects Y only through X
• $dim(Z_i) \geq dim(X_i)$

Terminology

• First Stage : Regress X on Z
• Reduced form : Regress Y on Z

$\mathbb{V}(\hat{\beta}_{iv}) = Q_{zx}^{-1} \Omega Q_{xz}^{-1} ; \enspace \Omega = \mathbb{E}z_i z_i' u_i^2$ If $dim(Z_i) > dim(X_i)$ (more instruments than endogenous regressors), $\hat{\beta}_{2SLS} =(X'P_zX)^{-1}X'P_zY = (X'Z(Z'Z)^{-1}Z'X)^{-1}X'Z(Z'Z)^{-1}Z'Y$ $\mathbb{V}(\hat{\beta}_{2sls}) = (Q_{xz} Q_{zz}^{-1} Q_{zx})^{-1} Q_{xz} Q_{zz}^{-1} \Omega Q_{zz}^{-1} Q_{zx} (Q_{xz} Q_{zz}^{-1} Q_{zx})^{-1} ; \enspace \Omega = \mathbb{E}z_i z_i' u_i^2$

GMM

If $dim(Z_i) > dim(X_i)$, $\hat{\beta}_{gmm}(W) = (X'ZWZ'X)^{-1}X'ZWZ'Y$

efficient GMM : $\mathbb{V}(\hat{\beta}_{gmm}) = (Q’\Omega^{-1}Q)^{-1}$

Sargan’s Over-ID Test

$H_0 : E (Z_i(Y_i - X_i’\beta)) = 0$

$S = \sum((Y_i - X_i'\hat{\beta}_{gmm})Z_i)'(\sum Z_i Z_i')^{-1} \sum((Y_i - X_i'\hat{\beta}_{gmm})Z_i) \sim \chi^2_{l-k}$ Reject if $S > \chi^2_{l-k}$