Machine Learning and Statistical Learning

Author

Ewen Gallic

You are on the course page for Machine Learning and Statistical Learning, taught in the Master 2 programme in Economics at Aix Marseille School of Economics (Aix-Marseille Université). This course was intended for students enrolled in the Econometrics, Big Data, Statistics track.

Aix Marseille University · Master in Economics/Econometrics

Week Topic Slides Exercise Code Notebook
1 Introduction
1 Optimization problem: gradient descent
2 Variants of the gradient descent algorithm
2 Hands-on session: preparing train and test datasets, overfitting
2 Logistic regression
3 K-nearest neighbors
3 Support Vector Machines
3 Hands-on: dealing with images
4 Working with imbalanced datasets
4 Explainability: Shap Values
4 Machine learning and ethics

By the end of the course, students will be able to:

  • Understand how machine learning algorithms work.
  • Understand the basic theory underlying machine learning.
  • Estimate and evaluate supervised learning models, from linear and quantile regression to logistic regression, KNN, LDA, and SVM.
  • Apply machine learning methods to structured data and images, and interpret model predictions using explainability tools such as Shap values.
  • Code (in Python or R) simple automatic learning algorithms, and reflect on the ethical implications of machine learning.

This course provides a broad introduction to statistical learning and machine learning, with the aim of giving students the knowledge necessary to understand machine learning methods. It opens with the key concepts underlying statistical learning (estimation, the accuracy/interpretability trade-off, the curse of dimensionality), before turning to supervised methods: regression models (linear and quantile regression) and classification (logistic regression, KNN, LDA, SVM, and image data). The course closes with a session on model explainability (Shap values) and a discussion of machine learning and ethics. Unsupervised methods are covered separately, in Pierre Michel’s course.

Organization

The course consists of 24 hours of teaching, with sessions alternating theoretical presentations and hands-on applications carried out on computers (Python or R).

Prerequisites

Mandatory: Notions in econometrics, basic Python programming.

Bibliography and textbooks

  • James, G., Witten, D., Hastie, T., and Tibshirani, R. (2013). An Introduction to Statistical Learning. New York: Springer. doi:10.1007/978-1-4614-7138-7
  • Berk, R. A. (2016). Statistical Learning from a Regression Perspective. Springer Texts in Statistics. doi:10.1007/978-3-319-44048-4
  • Murphy, K. (2012). Machine Learning: A Probabilistic Perspective. The MIT Press. ISBN: 9780262018029