Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Introduction and ANN Architecture.

• Comparison between biological and artificial neurons.
• The fundamental model of an ANN.
• Key activation functions utilized in ANNs.
• Standard categories of network architectures.

Mathematical Basis and Learning Mechanisms.

• Review of vector and matrix algebra.
• Introduction to state-space concepts.
• Core principles of optimization.
• Mechanisms of error-correction learning.
• Principles of memory-based learning.
• The concept of Hebbian learning.
• Modes of competitive learning.

Single Layer Perceptrons.

• Architectural design and learning processes of perceptrons.
• Pattern classification: Introduction to Bayes' classifiers.
• Utilizing perceptrons as pattern classifiers.
• The convergence behavior of perceptrons.
• Inherent limitations of perceptron models.

Feedforward ANNs.

• Structures of Multi-layer feedforward networks.
• The Back propagation algorithm.
• Back propagation: Training dynamics and convergence.
• Functional approximation techniques using back propagation.
• Practical implementation and design considerations for back propagation learning.

Radial Basis Function Networks.

• Concepts of pattern separability and interpolation.
• Foundations of Regularization Theory.
• Application of regularization to RBF networks.
• Design and training procedures for RBF networks.
• Approximation capabilities of RBF models.

Competitive Learning and Self-Organizing ANNs.

• General procedures for clustering.
• Learning Vector Quantization (LVQ).
• Variations of competitive learning algorithms and architectures.
• Development of Self-Organizing Feature Maps.
• Key properties of feature maps.

Fuzzy Neural Networks.

• Overview of Neuro-fuzzy systems.
• Background on fuzzy sets and logic.
• Methodologies for designing fuzzy systems.
• Architecture design for fuzzy ANNs.

Real-World Applications

• A discussion of various Neural Network applications, highlighting their specific advantages and associated challenges.

DAY -2 MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets – consistent case
  • Guarantees for finite hypothesis sets – inconsistent case
  • General considerations
    • Deterministic vs. Stochastic scenarios
    • Bayes error noise
    • Estimation and approximation errors
    • Model selection strategies
• Rademacher Complexity and VC-Dimension
• The Bias-Variance tradeoff
• Regularisation techniques
• Managing Over-fitting
• Validation methods
• Support Vector Machines
• Kriging (Gaussian Process regression)
• PCA and Kernel PCA
• Self-Organisation Maps (SOM)
• Kernel induced vector space
  • Mercer Kernels and Kernel-induced similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

These topics are presented in the context of the material covered on Day 1 and Day 2

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA, and Whitening
• Self-Taught Learning
• Deep Networks
• Linear Decoders
• Convolution and Pooling
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demonstrations and Applications