GUG TY CSE syllabus

GUG · Third Year B.Tech. Computer Science and Engineering · NEP 2020

Deep Learning

STPECCSE604A · Programme Elective II · Semester VI · 3 credits · IE 10 + mid-semester 10 + ESE 80

One of the Programme Elective II options. Programme Elective II comes with its lab.

All notes for this subject
  1. Unit 1: Foundations of Neural Networks

    9 hours

    Biological and artificial neurons, models of a neuron, networks as directed graphs, feedback and architectures; Rosenblatt's perceptron and its convergence, the perceptron and the Bayes classifier; perceptron, ADALINE and backpropagation networks; BAM and Hopfield associative memories.

    Examined in the 80-mark end-semester paper, and in the mid-semester exam if it is taught by then

    • McCulloch–Pitts neuron and thresholding logic
    • Perceptron and feed-forward networks
    • Forward and backpropagation
  2. Unit 2: Multilayer Perceptrons and Training

    9 hours

    Multilayer feed-forward networks; batch and online learning; backpropagation and its variants, the XOR problem; heuristics for better backpropagation; gradient-based learning, hidden units and architecture design; empirical risk minimisation and the bias-variance trade-off; linear and softmax outputs, tanh and ReLU hidden units.

    Examined in the 80-mark end-semester paper, and in the mid-semester exam if it is taught by then

    • Activation functions in deep networks
    • Loss functions
    • Gradient descent and vanishing and exploding gradients
    • Hyperparameters: learning rate, momentum and regularisation
  3. Unit 3: Deep Learning Basics and Regularisation

    9 hours

    Deep feed-forward networks and greedy layer-wise training; ill-conditioning, local minima, plateaus and saddle points; AdaGrad, RMSProp and Adam; L2 penalties, dataset augmentation, semi-supervised learning, dropout, batch normalisation, early stopping, bagging and adversarial training; the curse of dimensionality.

    Examined in the 80-mark end-semester paper, and in the mid-semester exam if it is taught by then

    • Representation learning and deep architectures
    • Weight initialisation and greedy layer-wise pre-training
    • Optimisers: momentum, Nesterov, AdaGrad, RMSProp and Adam
    • Regularisation: dropout, batch normalisation and early stopping
  4. Unit 4: Convolutional Neural Networks

    9 hours

    The convolution operation, pooling and variants of convolution; layers, filters and parameter sharing; LeNet, AlexNet, VGG, ResNet and DenseNet; the vanishing gradient and weight initialisation; transfer learning and fine-tuning; image classification and object detection.

    Examined in the 80-mark end-semester paper, and in the mid-semester exam if it is taught by then

    • CNN architecture
    • Convolution, padding and strides
    • Pooling and fully connected layers
    • LeNet and AlexNet
    • CNN architectures: VGG, GoogLeNet, ResNet and DenseNet
    • Transfer learning and data augmentation
    • Object detection and image segmentation
  5. Unit 5: Sequence Models

    9 hours

    Unfolding computational graphs; recurrent and bidirectional RNNs; encoder-decoder sequence-to-sequence architectures; backpropagation through time; LSTM and GRU; recursive neural networks; applications in speech recognition and NLP.

    Examined in the 80-mark end-semester paper, and in the mid-semester exam if it is taught by then

    • Recurrent neural networks
    • Bidirectional and deep RNNs
    • Encoder–decoder sequence-to-sequence models
    • Backpropagation through time
    • Long-term dependencies, LSTM and GRU
    • Recursive networks and echo state networks

Unit contents are summarised from the official GUG syllabus . AiCanCode.org is not affiliated with Gondwana University, Gadchiroli.