SPPU TE AI&DS syllabus

SPPU · Third Year Artificial Intelligence and Data Science · 2024 Pattern

Reinforcement Learning

PEC-362C-AID · Elective III · Semester VI · 3 credits · CCE 30 + end-semester 70

One of the Elective III options.

All notes for this subject
  1. Unit 1: Introduction to Reinforcement Learning

    9 hours

    RL compared with supervised and unsupervised learning; agent–environment interaction; rewards, returns and discounting; episodic and continuing tasks; model-based and model-free, on- and off-policy; elements of RL; challenges; multi-armed bandits and exploration strategies.

    Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)

    • Agent-environment interface
    • Rewards, returns and discounting
    • Types of reinforcement learning
    • Multi-armed bandits
    • Exploration strategies
  2. Unit 2: Markov Decision Processes

    9 hours

    Markov chains and reward processes; MDP components; policies and value functions; Bellman expectation and optimality equations; optimal policies; the advantage function.

    Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)

  3. Unit 3: Monte Carlo and TD Methods

    9 hours

    Monte Carlo prediction and control; exploring starts; importance sampling and off-policy control; TD learning and bootstrapping; SARSA; SARSA versus Q-learning.

    Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)

    • Monte Carlo prediction
    • Monte Carlo control
    • Importance sampling
    • Temporal difference learning
    • SARSA
  4. Unit 4: Deep RL and Policy Gradients

    9 hours

    DQN, double DQN, dueling and recurrent DQN; hierarchical DQN; REINFORCE; actor–critic and A2C/A3C; PPO; value- versus policy-based methods; continuous actions; Gymnasium and Stable-Baselines3.

    Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)

    • Deep Q-networks
    • Double and dueling DQN
    • Policy gradients: REINFORCE
    • Actor-critic methods
    • Proximal policy optimisation
  5. Unit 5: RL from Human Feedback

    9 hours

    RLHF, reward modelling and preference learning; PPO in RLHF; alignment and safety; multi-agent RL and self-play; offline RL; RL in agentic AI.

    Marks: 14 in the end-semester exam · CCE: Seminar, open-book test or quiz (6 marks, Unit 5)

    • Reinforcement learning from human feedback
    • Reward modelling
    • Multi-agent reinforcement learning
    • Offline reinforcement learning

Unit contents are summarised from the official SPPU syllabus . AiCanCode.org is not affiliated with Savitribai Phule Pune University.