GUG · Third Year B.Tech. Computer Science and Engineering · NEP 2020
Deep Learning
STPECCSE604A · Programme Elective II · Semester VI · 3 credits · IE 10 + mid-semester 10 + ESE 80
One of the Programme Elective II options. Programme Elective II comes with its lab.
Unit 1: Foundations of Neural Networks
9 hoursBiological and artificial neurons, models of a neuron, networks as directed graphs, feedback and architectures; Rosenblatt's perceptron and its convergence, the perceptron and the Bayes classifier; perceptron, ADALINE and backpropagation networks; BAM and Hopfield associative memories.
Examined in the 80-mark end-semester paper, and in the mid-semester exam if it is taught by then
- McCulloch–Pitts neuron and thresholding logic
- Perceptron and feed-forward networks
- Forward and backpropagation
Unit 2: Multilayer Perceptrons and Training
9 hoursMultilayer feed-forward networks; batch and online learning; backpropagation and its variants, the XOR problem; heuristics for better backpropagation; gradient-based learning, hidden units and architecture design; empirical risk minimisation and the bias-variance trade-off; linear and softmax outputs, tanh and ReLU hidden units.
Examined in the 80-mark end-semester paper, and in the mid-semester exam if it is taught by then
- Activation functions in deep networks
- Loss functions
- Gradient descent and vanishing and exploding gradients
- Hyperparameters: learning rate, momentum and regularisation
Unit 3: Deep Learning Basics and Regularisation
9 hoursDeep feed-forward networks and greedy layer-wise training; ill-conditioning, local minima, plateaus and saddle points; AdaGrad, RMSProp and Adam; L2 penalties, dataset augmentation, semi-supervised learning, dropout, batch normalisation, early stopping, bagging and adversarial training; the curse of dimensionality.
Examined in the 80-mark end-semester paper, and in the mid-semester exam if it is taught by then
- Representation learning and deep architectures
- Weight initialisation and greedy layer-wise pre-training
- Optimisers: momentum, Nesterov, AdaGrad, RMSProp and Adam
- Regularisation: dropout, batch normalisation and early stopping
Unit 4: Convolutional Neural Networks
9 hoursThe convolution operation, pooling and variants of convolution; layers, filters and parameter sharing; LeNet, AlexNet, VGG, ResNet and DenseNet; the vanishing gradient and weight initialisation; transfer learning and fine-tuning; image classification and object detection.
Examined in the 80-mark end-semester paper, and in the mid-semester exam if it is taught by then
- CNN architecture
- Convolution, padding and strides
- Pooling and fully connected layers
- LeNet and AlexNet
- CNN architectures: VGG, GoogLeNet, ResNet and DenseNet
- Transfer learning and data augmentation
- Object detection and image segmentation
Unit 5: Sequence Models
9 hoursUnfolding computational graphs; recurrent and bidirectional RNNs; encoder-decoder sequence-to-sequence architectures; backpropagation through time; LSTM and GRU; recursive neural networks; applications in speech recognition and NLP.
Examined in the 80-mark end-semester paper, and in the mid-semester exam if it is taught by then
- Recurrent neural networks
- Bidirectional and deep RNNs
- Encoder–decoder sequence-to-sequence models
- Backpropagation through time
- Long-term dependencies, LSTM and GRU
- Recursive networks and echo state networks
Unit contents are summarised from the official GUG syllabus . AiCanCode.org is not affiliated with Gondwana University, Gadchiroli.