CSCI4155/CSCI6505 (2017)

Machine Learning 2017

Instructors

Prof: Dr. Thomas Trappenberg (tt@cs.dal.ca)

Office: Room 4216 in Mona Campbell Building

Office hour: Write email if you want to meet

Teaching Assistants:

Lead TA: Dr. Hossein Parvar (hparvar@dal.ca)

TA: Farzaneh Fard (fard@cs.dal.ca)

Course Description

This course is an introduction to machine learning, including their practical use and theoretical foundation. We will emphasize probabilistic and deep learning methods, and will be using Python with advanced implementations such as sklearn and Google's Tensorflow. We will start by showing how to apply pre-programmed algorithms in Python to get some practical experience before unpacking some of the theory behind them. The course includes introductory reviews of scientific programming with Python. The course requires knowledge of mathematical concepts such as calculus and linear algebra as well as the formalism of describing uncertainty with probability theory.

Course Textbook

A manuscript for the course will be provided on Brightspace that outlines our specific content for the class. This is a new manuscript and I would appreciate notification of any errors.

In addition, there are many good textbook on machine learning, some which are listed below. Recommended textbooks for further studies are:

• Kevin Murphy; Machine Learning: A Probabilistic Perspective; MIT Press

• Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press

• Ethem Alpaydim; Introduction to Machine Learning; MIT Press

• Trevor Hastie, Robert Tibshirani, and Jerome Friedman; The Elements of Statistical Learning: Data Mining, Inference, and Prediction; Springer

Course Slides

Slides1
Slides2

Programs

GaussianClassification.py.zip
LinearRegression.ipynb.zip
BiLinearRegression.zip
PerceptronBooleanBatch.zip
MLPBooleanBatch.zip
MLPBooleanSGDComponentwise.zip
RL.zip

Assignments

Assignments are posted on the Brightspace. You can download the first assignment here, but you must submit it on Brightspace.

Grad Presentations

Graduate students must present a research paper from the ICLR 2017 conference. The accepted papers together with comments from reviewers and possible others can be found at

https://openreview.net/group?id=ICLR.cc/2017/conference

Every grad student needs to pick a paper that is of interest to them and present a 15 minutes summary and discussion of the paper in the tutorial classes during the last few weeks of the course. I will open a registration for specific time slots soon.

Grading Scheme

CSCI 4155: Assignments 50%, Midterm 20%, Final 30%
CSCI 6505: Assignments 30%, Midterm 20%, Final 30%, Presentation 20%

Background resources

The course assumes background in math, probability theory and programming. Some reviews will be included in the lectures to ensure we are on the same page, though if necessary you need to study this further on your own. As an example of the assumed level of background and to prepare if necessary l recommend the following sections from Khan academy:

• Matrices: https://www.khanacademy.org/math/precalculus/precalc-matrices
• Differential calculus: https://www.khanacademy.org/math/differential-calculus
• Probability theory: https://www.khanacademy.org/math/probability/random-variables-topic

You need to install the Python programming environment (Version 3.5 or higher). Make sure your installation includes Numpy, Matplotlib, Spyder, sklearn, tensorflow, and Lea. On Windows we recommend WinPython which should include everything except Lea (https://bitbucket.org/piedenis/lea/wiki/Installation ). On Macs we recommend Anaconda which includes all but tensorflow and Lea. Please consult the helpdesk if you have problems with the installation.

There are many different levels of tutorials available at https://wiki.python.org/moin/BeginnersGuide/Programmers

A good overview of Machine learning to prepare for this course is http://www.cs.ubc.ca/~murphyk/MLbook/pml-intro-22may12.pdf. An older overview of machine learning from my perspective can be found at A Brief Introduction to Probabilistic Machine Learning and Its Relation to Neuroscience, in Growing Adaptive Machines, T. Kowaliw et al. (eds.), Studies in Computational Intelligence 557, DOI: 10.1007/978-3-642-55337-0_2, Springer

Schedule

Below is the plan for this term. Note that dates/content can change depending on demand.

Further resources

Some useful websites to visualize http://cs231n.github.io/convolutional-networks/ and http://scs.ryerson.ca/~aharley/vis/conv/

DeepLearningBook.pdf.zip

Great intro to Neural Networks: http://www.3blue1brown.com/videos/2017/10/9/but-what-is-a-neural-network-deep-learning-part-1

Academic Integrity & Plagiarism

(Based on the sample statement provided at http://academicintegrity.dal.ca. Written by Dr. Alex Brodsky.)

Please familiarize yourself with the university policy on Intellectual Honesty. Every suspected case will be reported.

At Dalhousie University, we respect the values of academic integrity: honesty, trust, fairness, responsibility and respect. As a student, adherence to the values of academic integrity and related policies is a requirement of being part of the academic community at Dalhousie University.

What does academic integrity mean?

Academic integrity means being honest in the fulfillment of your academic responsibilities thus establishing mutual trust. Fairness is essential to the interactions of the academic community and is achieved through respect for the opinions and ideas of others. Violations of intellectual honesty are offensive to the entire academic community, not just to the individual faculty member and students in whose class an offence occurs. (see Intellectual Honesty section of University Calendar)

How can you achieve academic integrity?

• Make sure you understand Dalhousies policies on academic integrity.

• Give appropriate credit to the sources used in your assignment such as written or oral work, computer codes/programs, artistic or architectural works, scientific projects, performances, web page designs, graphical representations, diagrams, videos, and images. Use RefWorks to keep track of your research and edit and format bibliographies in the citation style required by the instructor (http://www.library.dal.ca/How/RefWorks)

• Do not download the work of another from the Internet and submit it as your own.

• Do not submit work that has been completed through collaboration or previously submitted for another assignment without permission from your instructor. • Do not write an examination or test for someone else.

• Do not falsify data or lab results.

These examples should be considered only as a guide and not an exhaustive list.

What will happen if an allegation of an academic offence is made against you?

I am required to report a suspected offence. The full process is outlined in the Discipline flow chart, which can be found at: http://academicintegrity.dal.ca/Files/AcademicDisciplineProcess.pdf and in- cludes the following:

1. Each Faculty has an Academic Integrity Officer (AIO) who receives allegations from instructors.

2. The AIO decides whether to proceed with the allegation and you will be notified of the process.

3. If the case proceeds, you will receive an INC (incomplete) grade until the matter is resolved.

4. If you are found guilty of an academic offence, a penalty will be assigned ranging from a warning to a suspension or expulsion from the University and can include a notation on your transcript, failure of the assignment or failure of the course. All penalties are academic in nature.

Where can you turn for help?

• If you are ever unsure about ANYTHING, contact myself.

• The Academic Integrity website (http://academicintegrity.dal.ca) has links to policies, defini tions, online tutorials, tips on citing and paraphrasing.

• The Writing Center provides assistance with proofreading, writing styles, citations.

• Dalhousie Libraries have workshops, online tutorials, citation guides, Assignment Calculator, Ref- Works, etc.

• The Dalhousie Student Advocacy Service assists students with academic appeals and student discipline procedures.

• The Senate Office provides links to a list of Academic Integrity Officers, discipline flow chart, and Senate Discipline Committee.

Request for special accommodation

Students may request accommodation as a result of barriers related to disability, religious obligation, or any characteristic under the Nova Scotia Human Rights Act. Students who require academic accommodation for either classroom participation or the writing of tests and exams should make their request to the Advising and Access Services Center (AASC) prior to or at the outset of the regular academic year. Please visit www.dal.ca/access for more information and to obtain the Request for Accommodation – Form A.

A note taker may be required as part of a student’s accommodation. There is an honorarium of $75/course/term (with some exceptions). If you are interested, please contact AASC at 494-2836 for more information.

Please note that your classroom may contain specialized accessible furniture and equipment. It is important that these items remain in the classroom, untouched, so that students who require their usage will be able to participate in the class.