Machine Learning:  2013-2014

Overview

It has long been a central concern of Computer Science to create machines capable of learning from experience. From predicting movies a customer will like, to learning to translate human languages from examples, systems that learn are becoming increasing prevalent and effective in our everyday lives. This course will introduce the field of Machine Learning, in particular focusing on the core concepts of supervised and unsupervised learning. In supervised learning we will discuss algorithms which are trained on input data labelled with a desired output, for instance an image of a face and the name of the person whose face it is, and learn a function mapping from the input to the output. Unsupervised learning aims to discover latent structure in an input signal where no output labels are available, an example of which is grouping webpages based on the topics they discuss. Students will learn the algorithms which underpin many popular Machine Learning techniques, as well as developing an understanding of the theoretical relationships between these algorithms. The practicals will concern the application of machine learning to a range of real-world problems, with particular reference to working with data on the web.

Learning outcomes

On completion of the course students will be expected to:

• Have a good understanding of the fundamental issues and challenges of machine learning: data, model selection, model complexity, etc.
• Have an understanding of the strengths and weaknesses of many popular machine learning approaches.
• Appreciate the underlying mathematical relationships within and across Machine Learning algorithms and the paradigms of supervised and un-supervised learning.
• Be able to design and implement various machine learning algorithms in a range of real-world applications.

Prerequisites

Modern Machine Learning is a mathematical discipline and as such students will benefit from a familiarity with probability theory and linear algebra, as well as some programming skills. The course is self-contained and does not require any previous knowledge of Artificial Intelligence.

Synopsis

• Introduction to Machine Learning (1 lecture): classification, regression and clustering; supervised and unsupervised learning.

• Mathematical preliminaries (1 lecture): Introduction to probability: random variables, independence, Bayes' theorem, conditional and marginal probabilities. Maximum likelihood (MLE) and Maximum a posteriori (MAP) density estimation and probability distributions.

Part 1: Classification and Regression

• Introduction to classification (1 lectures): The Naive Bayes classifier.
• Probabilistic classification and regression (3 lectures): Generative vs. Discriminative models. Logistic regression, optimisation and regularisation.
• Model Selection (1 lecture): Model selection, metrics, bias variance tradeoffs.
• Non-linear classification (3 lectures): The Perceptron, Support Vector Machines and Kernels.

Part 2: Clustering and Dimensionality Reduction

• Clustering (2 lectures): Mixture models and the EM algorithm.
• Topic Models (1 lecture): Latent Dirichlet Allocation.
• Dimensionality Reduction (1 lecture): Principle Component Analysis.

Part 3: Structured Learning

• Sequential Data (2 lectures): Hidden Markov Models

Syllabus

Reading list

Primary Text

• Kevin P. Murphy. Machine Learning: A Probabilistic Perspective, MIT Press 2012.

Secondary Texts

• Christopher M. Bishop. Pattern Recognition and Machine Learning, Springer 2007.
• I.H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, Second Edition Morgan Kaufmann, 2005.
• T.M. Mitchell, Machine Learning, McGraw-Hill, 1997.

Taking our courses

This form is not to be used by students studying for a degree in the Department of Computer Science, or for Visiting Students who are registered for Computer Science courses

Other matriculated University of Oxford students who are interested in taking this, or other, courses in the Department of Computer Science, must complete this online form by 17.00 on Friday of 0th week of term in which the course is taught. Late requests, and requests sent by email, will not be considered. All requests must be approved by the relevant Computer Science departmental committee and can only be submitted using this form.