Introduction to Machine Learning

CMSC 478/678

Spring 2011

Course Description

Research in the field of machine learning seeks to understand how computer programs can be built that automatically improve their performance at some task with experience. Among the many successes of this field are a program named TD-gammon that learned to play backgammon by playing games against itself. TD-gammon plays as well as the best human players. Another celebrated success is a program that learned how to drive a car on public highways by monitoring the actions of human drivers and then drove almost completely autonomously across the entire country. Machine learning algorithms are central to many applications, including speech recognition, document classification, information extraction from web pages, computer intrusion detection, and spam and splog (spam blog) detection. This course will cover core topics in machine learning, including the following: decision trees, nearest neighbor, Bayesian classification, regression, discriminant analysis, ensemble methods, the bias/variance tradeoff, experiment design, support vector machines, Bayesian networks, neural networks, clustering, and reinforcement learning.

Course Goals

Students who complete this course should be able to identify problems where machine learning techniques can be useful, frame the problem and the solution in terms of machine learning, choose appropriate algorithms, implement the solution, and evaluate the results. The lectures will provide students with knowledge of various types of machine learning problems/settings (e.g., supervised vs. unsupervised) and algorithms, while the homeworks and project will, at least in part, give students experience applying this knowledge using tools that are popular (and very useful) to machine learning researchers, such as Matlab and the WEKA toolkit. Finally, students should be able to read and understand current research papers in machine learning, primarily in the area of supervised learning.

Course Mechanics

Grades will be based on a midterm exam, a final exam, a project, and four homework assignments. These items will be weighted as follows: All assignments (homeworks and the various components of the course project) must be turned in at the beginning of class on the date that they are due. I understand that students have many demands on their time that vary in intensity over the course of the semester. Therefore, you will be allowed 3 late days without penalty for the entire semester. You can turn in 3 different assignments one day late each, or one assignment 3 days late, and so on. Once these late days are used, a penalty of 25% will be imposed for each day (or fraction thereof) an assignment is late (25% for one day, 50% for two, 75% for three, 100% for four or more).

Hard copy is required for all assignments. If you are not on campus for the start of class (what?!?!) and want to email your assignment to me or the TA to establish that it was done on time, that's OK. However, you must hand in hard copy before the assignment will be graded.

This class is open to both advanced undergraduate students and graduate students. The homeworks and exams will contain some questions to be answered by all students (those enrolled in either 478 or 678), and one or more questions to be answered just by those students enrolled in 678.

The project is meant to give students deeper exposure to some topic in machine learning than they would get from the lectures, readings, and discussions alone. Those projects that are most successful often blend the student's own research with machine learning, e.g. by applying machine learning techniques to a problem in some other area, or by bringing an insight from some other area to a problem in machine learning. However, projects need not involve ties with ongoing research. Many good projects in the past have investigated the application of existing algorithms to a domain/dataset of interest to the student, such as Texas Hold'em, the stock market, sporting events, and so on. Students can come up with their own project ideas or they can come see me and we'll brainstorm project ideas. Note that the project accounts for a significant fraction of the final grade, so there will be milestones throughout the semester to ensure that everyone is making good progress. See the syllabus below for more information.

Projects may be done by individuals or teams of two people. However, teams of two will be expected to do significantly more work than what is expected of an individual project. More information on projects, along with potential project ideas, can be found here.

Prerequisites

CMSC 471 (Introduction to Artificial Intelligence) or its equivalent is a prerequisite for this course. All students should have some prior exposure to probability, statistics, linear algebra, and differential calculus. Undergraduate courses in these areas suffice. If you do not meet the formal and informal prerequisites, come talk to me about whether it makes sense for you to enroll.

Who, Where and When?

The instructor for the course is Professor Tim Oates. I'm in room 336 of the ITE building and can be reached at (410) 455-3082 and at oates@cs.umbc.edu. We will meet on Mondays and Wednesdays from 2:30 - 3:45 in Math/Psych 103. My office hours are on Wednesdays from 1:00 until 2:00. I respond very quickly to email, and I'm happy to meet with you outside of my office hours if you can't make it. The TA for this course is Max Morawski (though he is half-time on this course and half-time on another). His email address is mm11@umbc.edu.

Textbook

There are many machine learning textbooks, but no single textbook deals with all of the topics this course will cover at the right level. Most textbooks assume too much background knowledge on the part of the reader, or are too terse in their descriptions, or lack rigor. The book I will draw most of my material from is Pattern Recognition and Machine Learning by Chris Bishop. However, I will pull materials from a variety of other sources and provide copies as the semester progresses.

Academic Integrity

By enrolling in this course, each student assumes the responsibilities of an active participant in UMBC's scholarly community in which everyone's academic work and behavior are held to the highest standards of honesty. Cheating, fabrication, plagiarism, and helping others to commit these acts are all forms of academic dishonesty, and they are wrong. Academic misconduct could result in disciplinary action that may include, but is not limited to, suspension or dismissal. To read the full Student Academic Conduct Policy, consult the UMBC Student Handbook, the Faculty Handbook, or the UMBC Policies section of the UMBC Directory (or for graduate courses, the Graduate School website).

For this course in particular it is both OK and a good idea for students to study together, including discussing homework problems. However, whatever a student turns in must be his/her own. A good rule of thumb is that it is OK to talk about problems, but it is not OK to share written materials or code. If you incorporate written materials or code from any source in the project, an appropriate citation is required.

Syllabus

This syllabus is tentative. This is the first time this course has been taught with the Bishop textbook, so there will invariably be topics that run longer than expected to ensure that we all understand them thoroughly, and there may be topics that are cut short to make up for lost time. Also, topics may be reordered. However, due dates and exam dates will not change.

Class
Date
Topic
Events/Readings
1 Wed  26 Jan Course overview - supervised learning, hypothesis spaces Read Bishop Chapter 1 through the end of section 1.1
2 Mon  31 Jan Decision trees - information gain, overfitting, pruning Decision tree slides (Thanks to Tom Mitchell)
Book chapter on deicision trees
Bishop chapter 1 slides (Thanks to Chris Bishop)
3 Wed  2 Feb Decision trees, nearest neighbor Nearest neighbor slides (Thanks to Andrew Moore)
4 Mon  7 Feb Nearest neighbor
5 Wed  9 Feb Review of probability and statistics See slides from class #2, Bishop chapter 1
Homework 1 assigned
6 Mon  14 Feb Bayesian classifiers, naive Bayes Naive Bayes slides (Thanks to Tom Mitchell)
Naive Bayes readings (Sections 1 and 2 of the chapter from Tom Mitchell)
7 Wed  16 Feb Bayesian classifiers continued
8 Mon  21 Feb Perceptrons, logistic regression Perceptron, logistic, LDA, and LR slides (Thanks to Tom Dietterich)
Homework 1 due
9 Wed  23 Feb Linear discriminant analysis, linear regression Homework 2 assigned
Slides (1 - 32) (Thanks to Andrew Moore)
10 Mon  28 Feb Linear regression and basis functions Bishop section 3.1
11 Wed  2 Mar Bias/variance theory Slides (Thanks to Tom Dietterich)
Bishop section 3.2
12 Mon  7 Mar Bias/variance theory, ensemble methods - bagging, boosting Homework 2 due
13 Wed  9 Mar Ensemble methods
14 Mon  14 Mar Mid-term exam
15 Wed  16 Mar Overfitting Slides (Thanks to Andrew Moore)
Mon  21 Mar Spring break
Wed  23 Mar Spring break
16 Mon  28 Mar Experiment design - cross validation, leave one out Slides, thanks to Tom Dietterich
Project proposal due
17 Wed  30 Mar Support vector machines - large margin classifiers, the kernel trick Homework 3 assigned
Tutorial paper on SVMs
Slides thanks to Andrew Moore
18 Mon  4 Apr Support vector machines continued Section 7.1 in Bishop book
19 Wed  6 Apr Bayesian networks - representation, conditional independence Slides thanks to Andrew Moore
Sections 8.1 and 8.2 in Bishop book
20 Mon  11 Apr Bayesian networks - inference, variable elimination Homework 3 due
21 Wed  13 Apr Neural networks
22 Mon  18 Apr Neural networks Project midway report due
23 Wed  20 Apr Clustering - agglomerative, divisive, k-means Homework 4 assigned
ionosphere.data, ionosphere.names
24 Mon  25 Apr Clustering
25 Wed  27 Apr Reinforcement learning - Markov decision processes, value iteration Readings: chapters 3, 4, and 6 from the Sutton and Barto RL Book
Slides for the RL problem; dynamic programming; temporal difference learning
26 Mon  2 May Reinforcement learning - policy iteration, Q-learning
27 Wed  4 May Project presentations Homework 4 due
28 Mon  9 May Project presentations
29 Wed  11 May Project presentations
Fri  20 May Final Exam 1:00 - 3:00PM Final project writeup due
Final exam topics