关于此课程： Welcome to the Reinforcement Learning course. Here you will find out about: - foundations of RL methods: value/policy iteration, q-learning, policy gradient, etc. --- with math & batteries included - using deep neural networks for RL tasks --- also known as "the hype train" - state of the art RL algorithms --- and how to apply duct tape to them for practical problems. - and, of course, teaching your neural network to play games --- because that's what everyone thinks RL is about. We'll also use it for seq2seq and contextual bandits. Jump in. It's gonna be fun!
此课程适用人群： The course is designed for engineers and scientists, (1) who already know the basics of machine learning and want to broaden their horizons (2) who plan to apply reinforcement learning to their problems, or (3) who want to understand the methods and details standing behind the breaking AI news.
制作方： 国立高等经济大学
教学方： Pavel Shvechikov, Researcher at HSE and Sberbank AI Lab
HSE Faculty of Computer Science
教学方： Alexander Panin, Lecturer
HSE Faculty of Computer Science
| 基本信息 | 课程 4（共 7 门，Advanced Machine Learning Specialization ） |
| 级别 | 高级设置 |
| 承诺学习时间 | 6 weeks of study, 3-6 hours/week for base track, 6-9 with all the horrors of honors section |
| 语言 | English |
| 如何通过 | 通过所有计分作业以完成课程。 |
| 用户评分 |
授课大纲
第 1 周
Intro: why should i care?
In this module we gonna define and "taste" what reinforcement learning is about. We'll also learn one simple algorithm that can solve reinforcement learning problems with embarrassing efficiency.
13 个视频, 7 个阅读材料
- 阅读: What you're getting into
- 阅读: Setting up course environment
- 阅读: Note: this course vs github course
- 阅读: Course teaser placeholder
- 视频: Reinforcement learning vs all
- 阅读: Primers
- 笔记本: Numpy primer
- 笔记本: Tensorflow primer
- 视频: Multi-armed bandit
- 视频: Decision process & applications
- 笔记本: Openai Gym
- 视频: Markov Decision Process
- 视频: Crossentropy method
- 笔记本: Crossentropy method
- 阅读: About honors track
- 视频: Approximate crossentropy method
- 视频: More on approximate crossentropy method
- 笔记本: Going deeper
- 阅读: Extras
- 视频: Evolution strategies: core idea
- 视频: Evolution strategies: math problems
- 视频: Evolution strategies: log-derivative trick
- 视频: Evolution strategies: duct tape
- 视频: Blackbox optimization: drawbacks
已评分: Grader: OpenAI Gym
已评分: Grader: CEM
已评分: Grader: Going deeper
第 2 周
At the heart of RL: Dynamic Programming
This week we'll consider the reinforcement learning formalisms in a more rigorous, mathematical way. You'll learn how to effectively compute the return your agent gets for a particular action - and how to pick best actions based on that return.
5 个视频, 2 个阅读材料
- 阅读: Advanced Reward Design
- 视频: State and Action Value Functions
- 视频: Measuring Policy Optimality
- 视频: Policy: evaluation & improvement
- 视频: Policy and value iteration
- 阅读: Discrete Stochastic Dynamic Programming
- 笔记本: Value Iteration
已评分: Reward design
已评分: Optimality in RL
已评分: Policy Iteration
已评分: Value iteration
第 3 周
Model-free methods
This week we'll find out how to apply last week's ideas to the real world problems: ones where you don't have a perfect model of your environment.
6 个视频, 1 个阅读材料
- 视频: Monte-Carlo & Temporal Difference; Q-learning
- 视频: Exploration vs Exploitation
- 笔记本: Q-learning
- 视频: Footnote: Monte-Carlo vs Temporal Difference
- 视频: Accounting for exploration. Expected Value SARSA.
- 笔记本: Expected Value SARSA
- 视频: On-policy vs off-policy; Experience replay
- 笔记本: Q-learning with experience replay
- 阅读: Extras
已评分: Grader: Q-learning
已评分: Expected Value SARSA
已评分: Model-free reinforcement learning
已评分: Grader: exp. replay
第 4 周
Approximate Value Based Methods
This week we'll learn to scale things even farther up by training agents based on neural networks.
9 个视频, 3 个阅读材料
- 阅读: TD vs MC
- 视频: Loss functions in value based RL
- 视频: Difficulties with Approximate Methods
- 笔记本: Approximate Qlearning - CartPole
- 视频: DQN – bird's eye view
- 视频: DQN – the internals
- 阅读: Extras
- 视频: DQN: statistical issues
- 视频: Double Q-learning
- 视频: More DQN tricks
- 笔记本: DQN - Breakout
- 视频: Partial observability
- 阅读: DQN follow-ups
已评分: MC & TD
已评分: SARSA and QLeaning
已评分: Approximate Qlearning - CartPole
已评分: DQN
已评分: DQN - Breakout
第 5 周
Policy-based methods
We spent 3 previous modules working on the value-based methods: learning state values, action values and whatnot. Now's the time to see an alternative approach that doesn't require you to predict all future rewards to learn something.
11 个视频, 1 个阅读材料
- 视频: All Kinds of Policies
- 视频: Policy gradient formalism
- 视频: The log-derivative trick
- 视频: REINFORCE
- 笔记本: REINFORCE
- 视频: Advantage actor-critic
- 视频: Duct tape zone
- 视频: Policy-based vs Value-based
- 视频: Case study: A3C
- 视频: A3C case study (2/2)
- 视频: Combining supervised & reinforcement learning
- 笔记本: Atari Advantage Actor Critic
- 阅读: Extras
已评分: Grader: REINFORCE
已评分: A policy-based quiz
已评分: Grader: A3C
第 6 周
Exploration
In this final week you'll learn how to build better exploration strategies with a focus on contextual bandit setup. In honor track, you'll also learn how to apply reinforcement learning to train structured deep learning models.
10 个视频, 4 个阅读材料
- 视频: Regret: measuring the quality of exploration
- 视频: The message just repeats. 'Regret, Regret, Regret.'
- 视频: Intuitive explanation
- 视频: Thompson Sampling
- 视频: Optimism in face of uncertainty
- 视频: UCB-1
- 视频: Bayesian UCB
- 笔记本: Bandits
- 阅读: Extras: exploration
- 视频: Introduction to planning
- 视频: Monte Carlo Tree Search
- 笔记本: MCTS
- 阅读: Extras: planning
- 阅读: Materials
- 笔记本: RL for seq2seq
- 阅读: Outro
已评分: Exploration
已评分: Bandits & exploration
已评分: MCTS
已评分: Grader: MCTS
常见问题解答
运作方式
课程作业
每门课程都像是一本互动的教科书，具有预先录制的视频、测验和项目。
来自同学的帮助
与其他成千上万的学生相联系，对想法进行辩论，讨论课程材料，并寻求帮助来掌握概念。
证书
获得正式认证的作业，并与朋友、同事和雇主分享您的成功。
制作方
国立高等经济大学
National Research University - Higher School of Economics (HSE) is one of the top research universities in Russia. Established in 1992 to promote new research and teaching in economics and related disciplines, it now offers programs at all levels of university education across an extraordinary range of fields of study including business, sociology, cultural studies, philosophy, political science, international relations, law, Asian studies, media and communications, IT, mathematics, engineering, and more.
Learn more on www.hse.ru
价格
| 购买课程 | |
|---|---|
| 访问课程材料 | 可用 |
| 访问评分的材料 | 可用 |
| 收到最终成绩 | 可用 |
| 获得可共享的课程证书 | 可用 |
评分和审阅
The course is dense and is accompanied by quality support (references to other courses, articles,...). It is punctuated with quizzes (which are unfortunately often quite ambiguous) and exercises on jupyter (which are well guided). This course seems to me, alone, insufficient and it is necessary to go to consult some references proposed to have a better understanding of certain topics. It is regrettable that the course goes sometimes too fast (some examples described in detail would be very useful for understanding) and that teachers are not always easy to understand.
Had a lot of fun doing this course. Although some of my fellow classmates are complaining that there are a few bugs in assignments, fixing those bugs itself can be a learning experience. The assignments,in general, are fun, particularly the honor's assignments.
Great instructors and course material, but there are enough bugs in the quizzes and assignments to be annoying.
AH
Learned a lot. The pace is quick and the assignment is challenging sometimes