Skip to content

Course Administrative Guide

The review of the course syllabus, grading policy, assignment schedule, and available resources to ensure everyone understands the expectations and logistics for our journey into machine learning and reproducible research.

Course Overview

  • Name: Machine Learning
  • Course Code: W13AMA-SM2338G
  • Prerequisites:
    • Fundamental understanding of linear algebra
    • Basic understanding of multivariable calculus concepts and optimization algorithm
    • Ability to write a non-trivial computer program in Python or equivalent
    • Working knowledge of GitHub.

Learning Objectives

Foundational Concepts

  • Explain the fundamental principles of machine learning and deep learning
  • Describe the historical development of neural networks and deep learning
  • Define and explain the function of a perceptron
  • Illustrate the structure and operation of artificial neurons
  • Differentiate between various activation functions and their use cases

Neural Network Architectures

  • Design and implement basic feedforward neural networks
  • Analyze the architecture of Convolutional Neural Networks (CNNs) and their applications in image processing
  • Examine the structure of Recurrent Neural Networks (RNNs) and their use in sequential data processing
  • Evaluate the improvements offered by Long Short-Term Memory (LSTM) networks over traditional RNNs

Training and Optimization

  • Explain the concept of backpropagation and its role in neural network training
  • Compare and contrast different gradient descent methods (e.g., stochastic, mini-batch, batch)
  • Implement various optimization algorithms to improve model performance

Practical Implementation

  • Utilize NumPy to build neural networks from scratch
  • Develop deep learning models using high-level frameworks such as Keras and PyTorch
  • Apply CNN architectures to solve image classification problems
  • Implement RNN and LSTM models for natural language processing tasks

Advanced Topics

  • Discuss regularization techniques to prevent overfitting in deep learning models
  • Explore transfer learning and its applications in various domains
  • Evaluate the ethical implications and potential biases in deep learning systems

Project Work

  • Design, implement, and evaluate a deep learning solution for a real-world problem
  • Present and defend the choices made in model architecture and training process

Critical Analysis

  • Compare the performance of different deep learning architectures on various tasks
  • Critically analyze current research papers in the field of deep learning
  • Identify limitations and potential future directions in deep learning research

Grading Policy

  • Assignments: 70%
  • Project: 30%

Grading Scale:

Percents Grade
>90% 5
[80%, 90%) 4.5
[70%, 80%) 4
[60%, 70%) 3.5
[50%, 60%) 3

Assignments and Projects

  • Weekly programming assignments due every Wednesday by 11:59 PM.
  • Late submissions: 10% penalty per day, up to 3 days
  • Final project proposal and team formation (max. 3 persons) due Week 8 (21st of November, 2024),
  • Final project presentation during the last week of classes (30rd of January, 2025),

Academic Integrity

  • All submitted work must be original
  • Collaboration on assignments is encouraged, but submitted code must be individual work
  • Plagiarism or cheating will result in a failing grade for the assignment and possible disciplinary action

Attendance Policy

  • Attendance is not mandatory for all lectures and lab sessions

Projects

References

Machine Learning

  1. The Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani, and Jerome Friedman.

  2. Pattern Recognition and Machine Learning by Christopher Bishop.

  3. Introduction to Machine Learning with Python by Andreas C. Müller & Sarah Guido.

  4. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow by Aurélien Géron.

Deep Learning

  1. Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville

  2. Neural Networks and Deep Learning by Michael Nielsen.

  3. Deep Learning with Python by François Chollet.

  4. Grokking Deep Learning by Andrew W. Trask.

Specialized Topics

  1. Natural Language Processing with Transformers by Lewis Tunstall, Leandro von Werra, and Thomas Wolf.

  2. Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto.

  3. Computer Vision: Algorithms and Applications by Richard Szeliski.

  4. Probabilistic Graphical Models: Principles and Techniques by Daphne Koller and Nir Friedman.