Advanced Course in Machine Learning : Exam
Passing the course
Passing the course requires a combination of exam and home exercises. If you have completed the weekly exercises (receiving at lesat 50% of the points), you can take part in the regular course exam, or any of the separate exams until Spring 2017 -- they are treated as renewal exams for the course.
If you have not completeld the weekly exercises but want to get the grade, you need to perform a separate set of small project exercises. The project details are available in the Project tab.
Exam material
The course exam is based on the material presented on the lectures and the course book Machine learning: A probabilistic perspective. We did not cover the whole book, and hence detailed list of sections covering the course exam is provided below. Phrases like "3.1 until 3.1.3" mean that you should start from the beginning on Section 3.1 and read still Section 3.1.3". In addition to the lecture material and the book, you should check the model solutions for the exercises when preparing for the exam.
Note that the page numbers below refer to the 5th printing of the book; the author says the page numbers can vary a bit from print to print. However, the Section numbers should be stable -- use those as the definitive guide and think of the page ranges as an additional hint.
Some topics are not covered by the book. This includes multidimensional scaling, stochastic neighbor embedding, Isomap, and largely also NMF and recurrent neural networks only briefly mentioned in the book. For these topics the level of detail presented on the lecture slides and exercises is sufficient.
Chapter 1: Introduction
Read the whole chapter, but treat all examples not covered during the course as further motivation.
Pages 1-25
Chapter 2: Probabbility
Section 2.1 - 2.6 and 2.8. You do not need to remember the exact formulations for the density functions except for Bernoulli and multivariate normal.
Pages 27-53, 26-61
Chapter 3: Generative models for discrete data
Sections 3.1, 3.2, 3.5.3. We did not really go through 3.2 on the lectures, but it presents one interesting way for understanding what the prior, likelihood and posterior mean.
Pages 67-74, 88
Chapter 4: Gaussian models
Section 4.1, 4.2 (except 4.2.8), 4.3, 4.3.1, 4.3.4.2, 4.4.1. I recommend reading the rest of 4.4 as well, but you do not need to be able to replicate mathematical derivations at this level. This is simply useful background for understanding linear latent variable models.
Pages 99-110, 112-113, 121-122
Chapter 5: Bayesian statistics
Section 5.1, 5.2, 5.3 until 5.3.1, 5.7 until 5.7.1.5
Pages 151-159, 178-182
Chapter 6: Frequentist statistics
Section 6.1, 6.2, 6.3 until 6.3.2, 6.4, 6.5, 6.6.4
Pages 193-199, 202-214, 217-218
Chapter 7: Linear regression
Section 7.1, 7.2, 7.3, 7.4, 7.5 except 7.5.3.
Pages 219-230, 232
Chapter 8: Logistic regression
Section 8.1, 8.2, 8.3, 8.5 until 8.5.3, 8.6 until 8.6.1
Pages 247-257, 264-268, 270-271 (table on 273)
Chapter 9: Generalized linear models
Section 9.5. I also recommend quickly checking 9.1 and 9.3 -- we skipped these during the lectures but some of the latter sections refer to GLMs so you should know roughly what they are, even though you need not understand the exponential family.
Pages 298-300
Chapter 10: Directed graphical models
Section 10.3, 10.4 -- even though we did not cover Bayesian networks as such, these sections cover inference for generative models in general
Pages 321-325
Chapter 11: Mixture models and the EM algorithm
Section 11.1, 11.2, 11.3, 11.4 until 11.4.2.8, 11.4.7 and its subsections,11.5, 11.6. For EM algorithm it is enough to understand the basic procedure; the theoretical foundation is not so important but might help in understanding what the algorithm actually does.
Pages 339-359, 365-367, 372-376
Chapter 12: Latent linear models
Section 12.1, 12.2 (cursory reading of the SVD part is enough), 12.3 until 12.3.2, 12.5, 12.6
Pages 383-403, 406-418
Chapter 13: Sparse linear models
Section 13.1, 13.2 (understanding the idea of how spike-and-slab is related to l_0 is enough -- no need to be able to replicate the derivations), 13.3 until 13.3.4, 13.4 until 13.4.2, 13.8 until 13.8.2
Pages 423-440, 443-444, 470-474
Chapter 14: Kernels
Section 14.1, 14.2 until 14.2.7 (no need to understand the details of the kernels we did not cover on the lectures), 14.3, 14.4 until 14.4.3.3 (we skipped K-medoids, but you should read it as one additional example of kernel method derivation), 14.5
Pages 481-486, 488-495, 498-507
Chapter 16: Adaptive basis function models
Section 16.1, 16.2, 16.4 until 16.4.4, 16.4.8, 16.5 until 16.5.6.4, 16.6 except 16.6.2, 16.7
Pages 545-554, 556-562, 564-578, 582-587
Chapter 19: Undirected graphical models
We skipped this, but Section 19.5.1 is useful for understanding RBMs
Pages 678-679
Chapter 25: Clustering
Section 25.1 (no need to remember the details for the evaluation methods), 25.4, 25.5 until 25.5.1.3 Note that the generative clustering models were covered already in Chapter 11
Pages 877-881, 892-900
Chapter 27: Latent variable models for discrete data
Section 27.6.2, 27.7 until 27.7.2.4 (no need to remember the details for RBMs for non-binary variables)
Pages 983-994
Chapter 28: Deep learning
Section 28.1, 28.2, 28.3, 28.4, 28.5
Pages 999-1011
What to bring to the exam?
As with all the exams at the department, you should bring writing materials (pencil etc. but not your own paper) and some means of identification (student card, passport etc.).
Additionally, for the exams of this course (including both the course exam and the separate exams), you may bring a "cheat sheet" which is one hand-written A4 sheet, to which you can write whatever information you think might be useful in the exam, using both sides if you wish. Even if you don't think you'll really need a cheat sheet in the exam, you may wish to create one just to help clarify to yourself what you think the important things are. You are not allowed to bring any other written material.
You will not need, and should not bring, a calculator. Use of any electronic devices, including of course mobile phones, is prohibited.
What will be asked in the exam?
In the exam, you may be asked to
- briefly define and explain key concepts and terms
- explain algorithms, techniques and other broader topics in more detail; you should explain all possible aspects of the topic, but exact mathematical details are typically not critical unless explicitly asked for (however, presenting them often clarifies the answer)
- manual execution of an algorithm on a small data set
- derivation of the mathematical details for some algorithm or model
- problem solving: given a task you need to come up with a practical solution using the techniques presented on the course
Printer-friendly version