(:requiretuid:)

Reinforcement Learning: From Foundations to Deep Approaches

Quick Facts

Description:

"The fundamental challenge in artificial intelligence and machine learning is learning to make good decisions under uncertainty," -- Emma Brunskill.

Reinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning. RL differs from supervised learning in not needing labelled input/output pairs be presented, and in not needing sub-optimal actions to be explicitly corrected. Instead the focus is on finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge).

About this course:

This course will take you through the foundation of reinforcement learning methods till recent deep reinforcement learning advances. By the end of this course, you will have a solid knowledge about the field, and you will be able to solve problems with different reinforcement learning algorithms. This course serves as an excellent background for people wanting to carry out reinforcement learning research independently, e.g. within the scope of a Bachelor's or Master's thesis.

Contents

(Lecture) Reinforcement Learning: From Foundations to Deep Approaches:

• Theory of Markov Decision Processes
• Dynamic Programming
• Policy Evaluation
• Tabular Reinforcement Learning
• Reinforcement Learning with Function Approximation
• Deadly Triad
• Policy Search and Policy Gradient Methods
• Deep Reinforcement Learning
• Deep Actor-Critic
• Frontiers of Reinforcement Learning

Requirements:

• Basic knowledge of linear algebra, statistics and probabilities
• Basic programming skills in Python
• Previous registration for the Statistical Machine Learning lecture is helpful, but not mandatory

Reading material:

There is no official textbook for the class, but a number of the supporting readings will come from:

• Reinforcement Learning: An Introduction, Sutton and Barto, 2nd Edition.

Some other additional references that may be useful are listed below:

• Reinforcement Learning: State-of-the-Art, Marco Wiering and Martijn van Otterlo, Eds.
• Artificial Intelligence: A Modern Approach, Stuart J. Russell and Peter Norvig.
• Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville.

Reinforcement Learning Library:

Course Format:

This course serves as an introduction to reinforcement learning methods. It is split into two main parts:

• Classical RL
• Deep RL

For each part, you will get a project assignment (two project assignments in total) consisting of some critical thinking questions and a programming assignment. Bonus question will be included in each assignment.

A report should be submitted till the designated deadline. The assignments are mandatory! Missing submission of an assignment will not allow you to participate in the final exam!

Late-submission policy: Every day after the designated deadline will lead to a reduction to 10% of the original score. Submissions three days later than the designated deadline will not be considered.

Grade decomposition:

• Assignment 1: 20% (+ 5% Bonus)
• Assignment 2: 20% (+ 5% Bonus)
• Final exam: 60%

Grades will be normalized accordingly.

Moodle Class

All further information and announcements regarding the lecture, seminar and project will be made public over the Moodle system of TU Darmstadt: https://moodle.tu-darmstadt.de/course/view.php?id=30326