index.md

layout	default
title	Machine Learning @ VU

{::nomarkdown}

0. Preliminaries

1. Introduction

2. Linear models and search

3. Evaluation

4.Probabilistic models

5.Data pre-processing

6.Beyond linear models

7.Deep learning

8.Sequences

9.Trees and ensembles

10.Transformers

11.Deep generative modeling

12.Embedding models

13.Reinforcement learning

Extra resources

Social impact dossier

Support vector machines

Expectation Maximization

{:/nomarkdown}

This page contains all public information about the course Machine Learning at the VU University Amsterdam. We provide the following materials:

• Lecture slides and videos.
• Worksheets These are very brief Jupyter notebooks to help you get the software installed and to show the basics. They introduce the libraries Numpy, Matplotlib, Pandas, Sklearn and Keras.
• Homework The homework consists of small pen-and-paper exercises to help you test that you've really understood the more technical points of the lectures. Answers are provided. If you are a registered student, please refer to the Canvas page instead. All material authored by Peter Bloem unless noted otherwise.

Reuse is allowed under a creative commons license, details below.

All content

{::nomarkdown}

<td><a href="https://youtu.be/k0_56JyYaOM">2020</a> <a href="https://youtu.be/f2HIW37Ohho">2019</a> <a href="https://youtu.be/DM1APCpqF8g">2018</a></td>

<td rowspan="2"><a href="./homework/week3.noanswers.pdf">plain</a>, <a href="./homework/week3.answers.pdf">answers</a></td>
<td rowspan="2"><a href="https://github.com/mlvu/worksheets/blob/master/Worksheet%203%2C%20Pandas.ipynb">pandas</a></td>

<td><a href="https://youtu.be/Aad5UDrdHPg">2020</a> <a href="https://youtu.be/H4c4qpHdGq8">2019</a> <a href="https://youtu.be/csk2HSWS5r8">2018</a></td>

<td><a href="https://www.youtube.com/playlist?list=PLCof9EqayQgs-MP4aQQ-2teemZANWKBjh">youtube</a> <a href="./lectures/32.LinearModels2.annotated.pdf">pdf</a></td>

<td><a href="https://youtu.be/1NVgspM98W0">2020</a> <a href="https://youtu.be/g2lziWxf_9Q">2019</a> <a href="https://youtu.be/F6gFYAwXmAs">2018</a></td>

<td> <a href="https://www.youtube.com/playlist?list=PLCof9EqayQgvCGzTPoRXPEYUWvFl8Cj71">playlist</a> <a href="./lectures/41.DeepLearning1.annotated.pdf">slides</a></td> 
<td rowspan="2"><a href="./homework/week4.noanswers.pdf">plain</a>, <a href="./homework/week4.answers.pdf">answers</a></td> 
<td rowspan="2"><a href="https://github.com/mlvu/worksheets/blob/master/Worksheet%204%2C%20Keras.ipynb">keras</a></td> 
<td><a href="https://youtu.be/DidHjsp_OV0">2020</a> <a href="https://youtu.be/VZwrbIBNzzA">2019</a> <a href="https://youtu.be/jOrYBnEPpYU">2018</a></td>

			homework	worksheets	previous
w0	0. Preliminaries 0.1 Logs, sums, argmax 0.2 Linear Algebra 0.3 Calculus 0.4 Probability	pdf
w1	1. Introduction 1.1 What is machine learning? 1.2 Classification 1.3 Other abstract tasks 1.4 Social impact 1 1.5 Generalization	youtube pdf	plain, answers	getting set up, numpy	2020 2019 2018
	2. Linear models and search 2.1 Linear regression 2.2 Searching for a good model 2.3 Gradient descent 2.4 Linear classification	youtube pdf			2020 2019 2018
w2	3. Model evaluation 3.1 Machine learning experiments 3.2 Model evaluation 3.3 ROC curves, area under the curve 3.4 Social impact 2 3.5 Statistics for ML experiments 3.6 No free lunch	youtube pdf	plain, answers	sklearn	2020 2019 2018
	4. Probabilistic Models 4.1 Learning with probability 4.2 The (naive) Bayes classifier 4.3 Logistic regression 4.4 Information theory	youtube pdf
w3	5. Data pre-processing 5.1 Missing values and outliers 5.2 Class imbalance and feature design 5.3 Normalization 5.4 Principal component analysis	youtube pdf
	6. Beyond Linear models 6.1 Neural networks 6.2 Backpropagation 6.3 Support vector machines 6.4 Lagrange multipliers 6.5 The kernel trick*
w4	7. Deep Learning 7.1 Deep learning and automatic differentiation 7.2 Tensor backpropagation 7.3 Convolutions 7.4 Making it work
	8. Sequences 8.1 Markov models 8.2 Deep learning on sequences 8.3 Recurrent neural nets and LSTMs	youtube pdf	2020 2019 2018
w5	9. Tree Model and Ensembles 9.1 Decision trees 9.2 Regression trees and numeric features 9.3 Ensembling 9.4 Boosting	youtube pdf	plain, answers	pytorch	2020 2019 2018
	10. Transformers 10.1 Self-attention 10.2 Building transformers 10.3 Building chatbots	youtube pdf
w6	11. Deep generative models 11.1 Generator networks 11.2 Generative adversarial networks 11.3 Autoencoders 11.4 Variational autoencoders 11.5 Social impact 3	playlist slides	plain answers		2020 2019 2018
	12. Embedding models 12.1 Recommender systems 12.2 Improving recommender systems 12.3 PCA revisited 12.4 Graph models 12.5 Validation of embedding models	youtube pdf			2020 2019 2018
w7	13. Reinforcement Learning 13.1 The abstract task 13.2 Policy gradients and random search 13.3 (Deep) Q learning 13.4 AlphaGo 13.5 Social impact 4	youtube pdf			2019 2018
	14. Review	video slides			2019 2018
w8	Exam. See below for practice exams.

Additional content

<colgroup>
	<col class="week">
	<col class="lecture">
	<col class="links">
	<col class="homework">
	<col class="worksheets">
	<col class="previous">
</colgroup>

Expectation-maximization Normal distributions Maximum likelihood estimators Expectation-maximization A formal analysis of EM	playlist slides	2020 2019 2018
Support Vector Machines Lagrange multipliers The kernel trick KKT conditions and the SVM dual
Social Impact Dossier Why and how 1. Xinjiang, COMPAS and Gender classification 2. Physiognomy and sexuality 3. Profiling

{:/nomarkdown}

Feel free to open a github issue if you're working through the material and you spot a mistake, run into a problem or have any other kind of question.

Required reading

Each week comes with a small amount of reading material. Most resources are publicly available free of charge. If you are a VU student, check Canvas for PDFs of the copyrighted works.

Week 1	Deep Learning, Goodfellow et al. Section 5.1
Week 2	Machine Learning, Peter Flach. Section 2.2 Everything you did and didn't know about PCA, Alex Williams
Week 3	Neural Networks and Deep Learning, Chapter 6
Week 4	What is the expectation maximization algorithm? Do et al.
Week 5	Intuitively Understanding Variational Autoencoders, Irhum Shafkat Machine Learning, Tom Mitchell. Chapter 3.
Week 6	Understanding LSTM Networks, Chris Olah
Week 7	Reinforcement Learning: Pong from pixels, Andrej Karpathy

Practice exams

Each exam consists of 40 multiple choice questions.

• Final exam 2018, answers
• Resit 2018, answers

Keynote files and re-use license

All material that is original to this course may be used under a CC BY 4.0 license. That means you are free to use the material, and adapt it, so long as appropriate credit is given. You may redistribute only under the same license.

How to credit:

• For individual slides, please add a link to mlvu.github.io, on the slide, or in the published slide annotations.
• If you are using a slide deck for a lecture as is, please indicate the source of the slides as mlvu.github.io clearly at the start of the lecture. Leaving the existing URL in place on the opening slide suffices.
• If you use many of the slides, a single attribution can be made once at the start of the slide deck. Crediting me by name (Peter Bloem) is appreciated, but not strictly necessary.

If you would like to use the material, but do not want to attribute in this way for some reason, please get in touch. I'm sure we can work something out.

Some parts of the material are taken from other sources. The source should always be credited on the slide itself (let me know if this isn't the case). Please adapt and redistribute these only under the original licenses.

Keynote files

The original keynote files for the lectures (of the 2019 version) can be found here, and may be used under the terms of the license above. These can be converted to ppt, but the formulas may not survive the conversion process.

The formulas were typeset using a fantastic tool called LaTeXiT. Copy pasting a formula from Keynote to LaTeXiT should reveal the original LaTeX. You'll need to use this preample for the typesetting to work.

Use of AI

In no part of these materials, including writing of lecture notes and mking of slides, was generative AI involved (except for a few examples explicitly labelled that way). I don't disagree with the use of AI per se, but for some projects, including this one, it's important that everything is done by hand.