SPPU · Final Year Artificial Intelligence and Data Science · 2020 Course
Reinforcement Learning
417533D · Elective VI · Semester VIII · 3 credits · In-semester 30 + end-semester 70
One of the Elective VI options.
Unit 1: Introduction
6 hoursBasics, origins and history of RL; its links with other fields and branches of ML; limitations and scope.
Marks: 15 in the in-semester exam
- Types of reinforcement learning
- History and scope of reinforcement learning
Unit 2: Markov Decision Processes
6 hoursThe Markov property; states, actions, transitions, rewards and policy; infinite horizons and utilities; Bellman equations; policy iteration; POMDPs, state estimation and value iteration in POMDPs.
Marks: 15 in the in-semester exam
- Markov decision processes
- Bellman equations
- Policy iteration
- Partially observable MDPs
Unit 3: Dynamic Programming
6 hoursPlanning in MDPs; principle of optimality; iterative policy evaluation; policy and value iteration; Banach fixed-point theorem and contraction proofs; convergence.
Marks: 17–18 in the end-semester exam
- Dynamic programming for MDPs
- Value iteration
- Contraction mapping and convergence
Unit 4: Monte Carlo Methods
6 hoursFirst-visit and every-visit prediction; Monte Carlo control; on- and off-policy learning; Monte Carlo tree search; discounting-aware and per-decision importance sampling.
Marks: 17–18 in the end-semester exam
- Monte Carlo prediction
- Monte Carlo control
- Importance sampling
- Monte Carlo tree search
Unit 5: Q-learning and Deep Q-Networks
6 hoursModel-based and model-free learning; temporal difference learning; Q-learning; DQN with CNNs and target networks; double and dueling DQN.
Marks: 17–18 in the end-semester exam
- Temporal difference learning
- Deep Q-networks
- Double and dueling DQN
Unit 6: Planning and Learning
6 hoursModels and planning; Dyna; wrong models; prioritised sweeping; expected and sample updates; trajectory sampling; real-time DP; decision-time planning, heuristic search and rollouts.
Marks: 17–18 in the end-semester exam
- Dyna: integrated planning and learning
- Prioritised sweeping
- Rollout algorithms
Unit contents are summarised from the official SPPU syllabus . AiCanCode.org is not affiliated with Savitribai Phule Pune University.