Mumbai University BE IT syllabus

Mumbai University · Final Year Information Technology · Rev-2019 'C' Scheme

Reinforcement Learning

ITDO8012 · Department Optional Course 5 · Semester VIII · 3 credits · Internal tests 20 (average of two) + end-semester 80

End-semester paper (3 hours): Six questions of 20 marks; Q1 is compulsory and you answer four.

One of the Department Optional Course 5 options.

All notes for this subject
  1. Module 1: Introduction to Reinforcement Learning

    4 hours

    Key features and elements of RL; types of RL; rewards; Q-learning and SARSA.

    Examined in the 80-mark end-semester paper; the class tests cover what has been taught by then

    • Elements of reinforcement learning
    • Types of reinforcement learning and rewards
    • Q-learning and SARSA
  2. Module 2: Bandit Problems and Online Learning

    7 hours

    The n-armed bandit; action-value methods; non-stationary problems; optimistic initial values; UCB action selection; gradient bandits.

    Examined in the 80-mark end-semester paper; the class tests cover what has been taught by then

    • The n-armed bandit problem
    • Action-value methods
    • Optimistic initial values and UCB action selection
    • Gradient bandits
  3. Module 3: Markov Decision Processes

    7 hours

    The agent–environment interface; goals, rewards and returns; the Markov property; MDPs; value and optimal value functions.

    Examined in the 80-mark end-semester paper; the class tests cover what has been taught by then

    • The agent–environment interface
    • Returns and the Markov property
    • Markov decision processes
    • Value functions and optimal value functions
  4. Module 4: Dynamic Programming

    7 hours

    Policy evaluation and improvement; policy and value iteration; asynchronous DP; generalised policy iteration.

    Examined in the 80-mark end-semester paper; the class tests cover what has been taught by then

    • Policy evaluation and improvement
    • Policy iteration
    • Value iteration
    • Asynchronous DP and generalised policy iteration
  5. Module 5: Monte Carlo and Temporal-difference Learning

    7 hours

    Monte Carlo prediction, action values and control; TD prediction; TD control with Q-learning; off-policy prediction by importance sampling.

    Examined in the 80-mark end-semester paper; the class tests cover what has been taught by then

    • Monte Carlo prediction and control
    • Temporal-difference prediction
    • Off-policy prediction with importance sampling
  6. Module 6: Applications and Case Studies

    5 hours

    Elevator dispatching; dynamic channel allocation; job-shop scheduling.

    Examined in the 80-mark end-semester paper; the class tests cover what has been taught by then

    • RL case studies: elevator dispatching, channel allocation and job-shop scheduling

Module contents are summarised from the official Mumbai University syllabus . AiCanCode.org is not affiliated with the University of Mumbai.