MSE Bias Variance tradeoff in estimating the variance of noise for MLE linear regression

External question — Cross Validated Stack Exchange Author: Jason Original post: https://stats.stackexchange.com/questions/463777 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. $\newcommand{\Var}{\operatorname{Var}}$ I am just grasping the bias variance trade-off as it is explained by the MSE heuristic. We have that if $y = f(x) + \varepsilon$ for $\varepsilon \sim N(0,\sigma^2)$ we can show that \begin{aligned} \operatorname{MSE}(y, \hat y) &= \Var(\varepsilon) + \operatorname{Bias}^2(\hat y) + \Var(\hat y) \\ &= \Var(\varepsilon) + \Var(\hat y) \qquad \text{ because we assume OLS is unbiased} \end{aligned} I am confused because when we solve linear regression via MLE, we assume the same $y = f(x) + \varepsilon$ for $\varepsilon$ normally distributed with variance $$\Var(\varepsilon) = \operatorname{MSE}(y, \hat y)$$ What happened to the $\Var(\hat y)$ term? Thank you!

External answer — Cross Validated Stack Exchange Author: mlofton Original post: https://stats.stackexchange.com/a/463788 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. Hi: In your second formula, the bias is assumed to be zero so there is no bias term. The variance term is the MSE