Skip to content

Week #6 - Multilayer Perceptrons

In today's lecture, we'll dive into multilayer perceptrons (MLPs), understanding how these more sophisticated neural networks overcome the limitations of single-layer models by utilizing hidden layers, backpropagation for learning, and their ability to solve complex non-linear problems that form the foundation of modern deep learning architectures.

Learning objectives:

  • Explain the architecture of multilayer perceptrons and how hidden layers enable non-linear function approximation
  • Understand the mathematics behind the backpropagation algorithm and its role in training MLPs
  • Explore the Universal Approximation Theorem and its implications for MLP capabilities and limitations
  • Implement and train an MLP to solve the XOR problem, demonstrating the advantage over single-layer perceptrons
  • Explore activation functions (sigmoid, tanh, ReLU) and their impact on network performance and training dynamics
  • Apply MLPs to real-world classification and regression problems, including techniques for avoiding overfitting

Laboratory TBA

Resources

  • https://brilliant.org/wiki/artificial-neural-network/
  • https://brilliant.org/wiki/feedforward-neural-networks/
  • https://brilliant.org/wiki/backpropagation/