Lecturer	Nando de Freitas
Degrees	Schedule C1 — Computer Science Schedule C1 — Mathematics and Computer Science Schedule C — MSc in Advanced Computer Science
Term	Hilary Term 2015  (24 lectures)

Overview

Machine learning techniques enable us to automatically extract features from data so as to solve predictive tasks, such as speech recognition, object recognition, machine translation, question-answering, anomaly detection, medical diagnosis and prognosis, automatic algorithm configuration, personalisation, robot control, time series forecasting, and much more. Learning systems adapt so that they can solve new tasks, related to previously encountered tasks, more efficiently.

The course focuses on the exciting field of deep learning. By drawing inspiration from neuroscience and statistics, it introduces the basic background on neural networks, back propagation, Boltzmann machines, autoencoders, convolutional neural networks and recurrent neural networks. It illustrates how deep learning is impacting our understanding of intelligence and contributing to the practical design of intelligent machines.

Learning outcomes

On completion of the course students will be expected to:

• Understand what is learning and why it is essential to the design of intelligent machines.
• Know how to fit models to data.
• Understand numerical computation, statistics and optimization in the context of learning.
• Have a good understanding of the problems that arise when dealing with very small and very big data sets, and how to solve them.
• Understand the basic mathematics necessary for constructing novel machine learning solutions.
• Be able to design and implement various machine learning algorithms in a wide range of real-world applications.
• Understand the background on deep learning and be able to implement deep learning models for language, vision, speech, decision making, and more.

Prerequisites

Machine Learning is a mathematical discipline, and students will benefit from a good background in probability, linear algebra and calculus. Programming experience is essential.

Synopsis

• 1. Introduction (1 lecture)

• 2. Linear prediction (1 lecture)

• 3. Maximum likelihood (1 lecture)

• 4. Regularizers, basis functions and cross-validation (1 lecture)

• 5. Optimisation (1 lecture)

• 6. Logistic regression (1 lecture)

• 7. Feedforward neural networks  (1 lecture)

• 8. Back-propagation (1 lecture)

• 9. Convolutional neural networks (1 lecture)

• 10. Max-margin learning and siamese networks (1 lecture)

• 11. Boltzmann machines and log-bilinear models (1 lecture)

• 12. Autoencoders (1 lecture) 

• 13. Helmholtz machines and learning by simulation (1 lecture) 

• 14. Recurrent neural networks and LSTMs (1 lecture)

• 15. Reinforcement learning with direct policy search (1 lecture)

• 16. Reinforcement learning with action-value functions (1 lecture) 

Syllabus

Mathematics of machine learning. Overview of supervised, unsupervised, multi-task, transfer, active and reinforcement learning techniques.

Reading list

Recommended texts

• Kevin P. Murphy. Machine Learning: A Probabilistic Perspective, MIT Press 2012.
• Christopher M. Bishop. Pattern Recognition and Machine Learning, Springer 2007.
• T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer 2011.
• S. Haykin. Neural networks and learning machines. Pearson 2008.

Taking our courses

This form is not to be used by students studying for a degree in the Department of Computer Science, or for Visiting Students who are registered for Computer Science courses

Other matriculated University of Oxford students who are interested in taking this, or other, courses in the Department of Computer Science, must complete this online form by 17.00 on Friday of 0th week of term in which the course is taught. Late requests, and requests sent by email, will not be considered. All requests must be approved by the relevant Computer Science departmental committee and can only be submitted using this form.