SPPU · Third Year Artificial Intelligence and Data Science · 2024 Pattern
Reinforcement Learning
PEC-362C-AID · Elective III · Semester VI · 3 credits · CCE 30 + end-semester 70
One of the Elective III options.
Unit 1: Introduction to Reinforcement Learning
9 hoursRL compared with supervised and unsupervised learning; agent–environment interaction; rewards, returns and discounting; episodic and continuing tasks; model-based and model-free, on- and off-policy; elements of RL; challenges; multi-armed bandits and exploration strategies.
Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)
- Agent-environment interface
- Rewards, returns and discounting
- Types of reinforcement learning
- Multi-armed bandits
- Exploration strategies
Unit 2: Markov Decision Processes
9 hoursMarkov chains and reward processes; MDP components; policies and value functions; Bellman expectation and optimality equations; optimal policies; the advantage function.
Marks: 14 in the end-semester exam · CCE: Unit test (12 marks, Units 1 & 2)
- Markov chains and reward processes
- Markov decision processes
- Value functions
- Bellman equations
- Optimal policies
Unit 3: Monte Carlo and TD Methods
9 hoursMonte Carlo prediction and control; exploring starts; importance sampling and off-policy control; TD learning and bootstrapping; SARSA; SARSA versus Q-learning.
Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)
- Monte Carlo prediction
- Monte Carlo control
- Importance sampling
- Temporal difference learning
- SARSA
Unit 4: Deep RL and Policy Gradients
9 hoursDQN, double DQN, dueling and recurrent DQN; hierarchical DQN; REINFORCE; actor–critic and A2C/A3C; PPO; value- versus policy-based methods; continuous actions; Gymnasium and Stable-Baselines3.
Marks: 14 in the end-semester exam · CCE: Assignments or case study (12 marks, Units 3 & 4)
- Deep Q-networks
- Double and dueling DQN
- Policy gradients: REINFORCE
- Actor-critic methods
- Proximal policy optimisation
Unit 5: RL from Human Feedback
9 hoursRLHF, reward modelling and preference learning; PPO in RLHF; alignment and safety; multi-agent RL and self-play; offline RL; RL in agentic AI.
Marks: 14 in the end-semester exam · CCE: Seminar, open-book test or quiz (6 marks, Unit 5)
- Reinforcement learning from human feedback
- Reward modelling
- Multi-agent reinforcement learning
- Offline reinforcement learning
Unit contents are summarised from the official SPPU syllabus . AiCanCode.org is not affiliated with Savitribai Phule Pune University.