How do machine learning topics fit into a traditional undergraduate statistics course on estimation?

External question — Cross Validated Stack Exchange Author: ExcitedSnail Original post: https://stats.stackexchange.com/questions/658050 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. I'm recently teaching an undergraduate introduction to statistics course, but as required by program director, need to add some machine learning materials to it. I'm wondering what is the appropriate way of injecting machine learning materials (bias-variance trade-off, over-fitting and under-fitting, training and testing, regularization and model selection, performance evaluation) to the topic of estimation (traditional materials include method of moments estimator, MLE, unbiasedness, efficiency, quantification of uncertainty), and what's a good way of organizing this part? I feel it is difficult to organize this mixture of ML and statistics elements in a well-organized and logical way. It would be great if you could suggest a reasonable arrangement of the two strands of topics.

External answer — Cross Validated Stack Exchange Author: cinch Original post: https://stats.stackexchange.com/a/658058 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. To integrate those ML topics into undergraduate traditional statistics topics of estimation, begin with statistical estimation theory (MLE, MAP, MME) to set the foundational stage and then apply it to more complex ML models such as parametric neural networks and non-parametric decision trees to show how ML extends traditional statistical methods. Statistical estimations like MLE/MAP point estimation, full Bayesian inference and hypothesis testing can be extended to ML via topics like MLE equivalence with ML's common CE/MSE loss function, MAP equivalence with ML's regularized loss function, MLE vs Maximum entropy reduction (decision tree's information gain objective), elastic net, variational inference, MCMC, UCB or Thompson sampling based A/B testing, etc. Statistical estimation's uncertainty quantification like confidence interval and credible interval can be extended to ML via topics like forest/bagging ensemble uncertainty estimation via bootstrap sampling, neural network Monte Carlo Dropout uncertainty estimation, etc. And statistical model performance evaluation like bias-variance tradeoff, AIC/BIC complexity, $R^2$ , $\chi^2$ , residuals and RMSE/MAE can be extended to ML via topics like PAC uniform convergence, VC-dimension, k-fold cross validation, etc.