SPPU BE AI&DS syllabus

SPPU · Final Year Artificial Intelligence and Data Science · 2020 Course

Reinforcement Learning

417533D · Elective VI · Semester VIII · 3 credits · In-semester 30 + end-semester 70

One of the Elective VI options.

All notes for this subject
  1. Unit 1: Introduction

    6 hours

    Basics, origins and history of RL; its links with other fields and branches of ML; limitations and scope.

    Marks: 15 in the in-semester exam

    • Types of reinforcement learning
    • History and scope of reinforcement learning
  2. Unit 2: Markov Decision Processes

    6 hours

    The Markov property; states, actions, transitions, rewards and policy; infinite horizons and utilities; Bellman equations; policy iteration; POMDPs, state estimation and value iteration in POMDPs.

    Marks: 15 in the in-semester exam

  3. Unit 3: Dynamic Programming

    6 hours

    Planning in MDPs; principle of optimality; iterative policy evaluation; policy and value iteration; Banach fixed-point theorem and contraction proofs; convergence.

    Marks: 17–18 in the end-semester exam

    • Dynamic programming for MDPs
    • Value iteration
    • Contraction mapping and convergence
  4. Unit 4: Monte Carlo Methods

    6 hours

    First-visit and every-visit prediction; Monte Carlo control; on- and off-policy learning; Monte Carlo tree search; discounting-aware and per-decision importance sampling.

    Marks: 17–18 in the end-semester exam

    • Monte Carlo prediction
    • Monte Carlo control
    • Importance sampling
    • Monte Carlo tree search
  5. Unit 5: Q-learning and Deep Q-Networks

    6 hours

    Model-based and model-free learning; temporal difference learning; Q-learning; DQN with CNNs and target networks; double and dueling DQN.

    Marks: 17–18 in the end-semester exam

    • Temporal difference learning
    • Deep Q-networks
    • Double and dueling DQN
  6. Unit 6: Planning and Learning

    6 hours

    Models and planning; Dyna; wrong models; prioritised sweeping; expected and sample updates; trajectory sampling; real-time DP; decision-time planning, heuristic search and rollouts.

    Marks: 17–18 in the end-semester exam

    • Dyna: integrated planning and learning
    • Prioritised sweeping
    • Rollout algorithms

Unit contents are summarised from the official SPPU syllabus . AiCanCode.org is not affiliated with Savitribai Phule Pune University.