European Central Bank Machine Learning Training

Machine learning for econometricians

• Course by Emmanuel Flachaire, Professor of Economics at Aix-Marseille Université.

• Hands-on sessions by Ewen Gallic, Assistant Professor at Aix-Marseille Université.

• Training organised by :

  • Mario Porqueddu, Senior Economist, Prices and Costs Division, European Central Bank.
  • Linda de Leeuw, Directorate General Economics, Prices & Costs Division, European Central Bank

Basic Requirements

Intermediate knowledge of econometrics and basic knowledge of programming with R.

Course description

These lectures have been conceived by econometricians for econometricians. The sessions proceed step by step, recalling the fundamental statistical concepts at the heart of the modern learning techniques. Their relative merits are illustrated by means of practical exercises and applications in R.

The course will present supervised Machine Learning techniques to econometricians. In particular, the lecturers will:

1. present various concepts intensively used in the Machine Learning literature such as cross-validation, resampling, optimization routines
2. describe and explain popular machine learning techniques such as bagging, random forests, boosting, SVM, neural nets and deep learning
3. highlight their usefulness in econometrics for misspecification detection and causal inference.

References

• James, Witten, Hastie and Tibshirani (2021). An Introduction to Statistical Learning. Springer
• Hastie, Tibshirani and Friedman (2009). The Elements of Statistical Learning. Springer Verlag
• Berk (2008). Statistical Learning from a Regression Perspective. Springer Verlag
• Grolemund, G. and Wickham, H. (2018). R for Data Science. O’Reilly

Course (Emmanuel Flachaire)

Session	Subject	Schedule	Supplementary Materials
Session 1	Introduction, Optimization, Overfitting, Cross-Validation	29 November 9.30-11.30
Session 2	Ridge and Lasso Regression	30 November 9.30-11.30	R: Ridge/Lasso
Session 3	CART, Bagging, Random Forests	6 December 9.30-11.30	R: Trees
Session 4	Boosting, Support Vector Machine	13 December 9.30-11.30	R: SVM
Session 5	Neural Networks, Deep Learning	20 December 9.30-11.30	R: Deep Learning
Session 6	Misspecification detection, Causal Inference	21 December 9.30-11.30	R: Misspecif. R: Causal Inf.

Access the slides (PDF)

Hands-on (Ewen Gallic)

The hands-on session will use the programming language R. To run the codes you will need to download and install R and RStudio. More details on how to install R are available here.

Session	Subject	Schedule
Session 1	Gradient descent and overfitting	1 December 9.30-11.30
Session 2	Random forest and SVM	17 December 15.30-17.30
Session 3	Deep learning	21 December 14.30-16.30

The materials for the hands-on sessions are provided through an html book. A PDF version is also available. The R codes of each chapter can be downloaded (see below in the detailed programme).

Notebook PDF version

• Session 1:

  • Optimisation:

    R Codes

    Required packages: tidyverse, numDeriv, plot3D

    If these are not installed, please run the following instructions in R:

    install.packages("tidyverse", "numDeriv", "plot3D")

  • Overfitting:

    R Codes

    Required packages: tidyverse, readxl, kableExtra, randomForest, ISLR, glmnet

    If these are not installed, please run the following instructions in R:

    install.packages(
       c("tidyverse", "readxl", "kableExtra", "randomForest", "ISLR", "glmnet")
    )

• Session 2: Random forest and SVM: 17 December 15.30-17.30

  • Trees and Ensemble Methods:

    R Codes

    Required packages: tidyverse, lubridate, arsenal, cowplot, corrplot, rpart, rpart.plot, ggtext, foreach, ipred, lattice, caret, randomForest

    If these are not installed, please run the following instructions in R:

    install.packages(c("tidyverse", "lubridate", "arsenal", "cowplot",
       "corrplot", "rpart", "rpart.plot", "ggtext", "foreach",
       "ipred", "lattice", "caret", "randomForest"))

  • Support Vector Machines:

    R Codes

    Required packages: tidyverse, e1071

    If these are not installed, please run the following instructions in R:

    install.packages(c("tidyverse", "e1071"))

• Session 3: Deep learning: 21 December 14.30-16.30

  • Single notebook:

    R Codes

    Required packages: tidyverse, fastDummies, reshape2, keras

    install.packages("tidyverse")
    install.packages("fastDummies")
    install.packages("reshape2")

    To install Keras: please run the following instructions in R:

    install.packages("tensorflow")
    install.packages("keras")
    library(keras)
    tensorflow::install_tensorflow()
    tensorflow::tf_config()
    install_keras()

    On one of my machines, I had to install Miniconda:

    unlink(reticulate::miniconda_path(), recursive = TRUE)
    reticulate::install_miniconda(path = reticulate::miniconda_path(), 
                          update = TRUE, force = FALSE)
    keras::install_keras()