关于此课程： Welcome to the Reinforcement Learning course. Here you will find out about: - foundations of RL methods: value/policy iteration, q-learning, policy gradient, etc. --- with math & batteries included - using deep neural networks for RL tasks --- also known as "the hype train" - state of the art RL algorithms --- and how to apply duct tape to them for practical problems. - and, of course, teaching your neural network to play games --- because that's what everyone thinks RL is about. We'll also use it for seq2seq and contextual bandits. Jump in. It's gonna be fun!
此课程适用人群： The course is designed for engineers and scientists, (1) who already know the basics of machine learning and want to broaden their horizons (2) who plan to apply reinforcement learning to their problems, or (3) who want to understand the methods and details standing behind the breaking AI news.
制作方： National Research University Higher School of Economics
教学方： Pavel Shvechikov, Researcher at HSE and Sberbank AI Lab
HSE Faculty of Computer Science教学方： Alexander Panin, Lecturer
HSE Faculty of Computer Science
| 基本信息 | 课程 4（共 7 门，Advanced Machine Learning Specialization ） |
| 级别 | 高级设置 |
| 承诺学习时间 | 6 weeks of study, 3-6 hours/week for base track, 6-9 with all the horrors of honors section |
| 语言 | English |
| 如何通过 | 通过所有计分作业以完成课程。 |
| 用户评分 |
授课大纲
第 1 周
Intro: why should i care?
In this module we gonna define and "taste" what reinforcement learning is about. We'll also learn one simple algorithm that can solve reinforcement learning problems with embarrassing efficiency.
13 个视频, 7 个阅读材料
- Reading: What you're getting into
- Reading: Setting up course environment
- Reading: Note: this course vs github course
- Reading: Course teaser placeholder
- 视频: Reinforcement learning vs all
- Reading: Primers
- Notebook: Numpy primer
- Notebook: Tensorflow primer
- 视频: Multi-armed bandit
- 视频: Decision process & applications
- Notebook: Openai Gym
- 视频: Markov Decision Process
- 视频: Crossentropy method
- Notebook: Crossentropy method
- Reading: About honors track
- 视频: Approximate crossentropy method
- 视频: More on approximate crossentropy method
- Notebook: Going deeper
- Reading: Extras
- 视频: Evolution strategies: core idea
- 视频: Evolution strategies: math problems
- 视频: Evolution strategies: log-derivative trick
- 视频: Evolution strategies: duct tape
- 视频: Blackbox optimization: drawbacks
已评分: Grader: OpenAI Gym
已评分: Grader: CEM
已评分: Grader: Going deeper
第 2 周
At the heart of RL: Dynamic Programming
This week we'll consider the reinforcement learning formalisms in a more rigorous, mathematical way. You'll learn how to effectively compute the return your agent gets for a particular action - and how to pick best actions based on that return.
5 个视频, 2 个阅读材料
- Reading: Advanced Reward Design
- 视频: State and Action Value Functions
- 视频: Measuring Policy Optimality
- 视频: Policy: evaluation & improvement
- 视频: Policy and value iteration
- Reading: Discrete Stochastic Dynamic Programming
- Notebook: Value Iteration
已评分: Reward design
已评分: Optimality in RL
已评分: Policy Iteration
已评分: Value iteration
第 3 周
Model-free methods
This week we'll find out how to apply last week's ideas to the real world problems: ones where you don't have a perfect model of your environment.
6 个视频, 1 个阅读材料
- 视频: Monte-Carlo & Temporal Difference; Q-learning
- 视频: Exploration vs Exploitation
- Notebook: Q-learning
- 视频: Footnote: Monte-Carlo vs Temporal Difference
- 视频: Accounting for exploration. Expected Value SARSA.
- Notebook: Expected Value SARSA
- 视频: On-policy vs off-policy; Experience replay
- Notebook: Q-learning with experience replay
- Reading: Extras
已评分: Grader: Q-learning
已评分: Expected Value SARSA
已评分: Model-free reinforcement learning
已评分: Grader: exp. replay
第 4 周
Approximate Value Based Methods
This week we'll learn to scale things even farther up by training agents based on neural networks.
9 个视频, 3 个阅读材料
- Reading: TD vs MC
- 视频: Loss functions in value based RL
- 视频: Difficulties with Approximate Methods
- Notebook: Approximate Qlearning - CartPole
- 视频: DQN – bird's eye view
- 视频: DQN – the internals
- Reading: Extras
- 视频: DQN: statistical issues
- 视频: Double Q-learning
- 视频: More DQN tricks
- Notebook: DQN - Breakout
- 视频: Partial observability
- Reading: DQN follow-ups
已评分: MC & TD
已评分: SARSA and QLeaning
已评分: Approximate Qlearning - CartPole
已评分: DQN
已评分: DQN - Breakout
第 5 周
Policy-based methods
We spent 3 previous modules working on the value-based methods: learning state values, action values and whatnot. Now's the time to see an alternative approach that doesn't require you to predict all future rewards to learn something.
11 个视频, 1 个阅读材料
- 视频: All Kinds of Policies
- 视频: Policy gradient formalism
- 视频: The log-derivative trick
- 视频: REINFORCE
- Notebook: REINFORCE
- 视频: Advantage actor-critic
- 视频: Duct tape zone
- 视频: Policy-based vs Value-based
- 视频: Case study: A3C
- 视频: A3C case study (2/2)
- 视频: Combining supervised & reinforcement learning
- Notebook: Atari Advantage Actor Critic
- Reading: Extras
已评分: Grader: REINFORCE
已评分: A policy-based quiz
已评分: Grader: A3C
第 6 周
Exploration
In this final week you'll learn how to build better exploration strategies with a focus on contextual bandit setup. In honor track, you'll also learn how to apply reinforcement learning to train structured deep learning models.
10 个视频, 4 个阅读材料
- 视频: Regret: measuring the quality of exploration
- 视频: The message just repeats. 'Regret, Regret, Regret.'
- 视频: Intuitive explanation
- 视频: Thompson Sampling
- 视频: Optimism in face of uncertainty
- 视频: UCB-1
- 视频: Bayesian UCB
- Notebook: Bandits
- Reading: Extras: exploration
- 视频: Introduction to planning
- 视频: Monte Carlo Tree Search
- Notebook: MCTS
- Reading: Extras: planning
- Reading: Materials
- Notebook: RL for seq2seq
- Reading: Outro
已评分: Exploration
已评分: Bandits & exploration
已评分: MCTS
已评分: Grader: MCTS
常见问题解答
运作方式
Coursework
Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.
Help from Your Peers
Connect with thousands of other learners and debate ideas, discuss course material, and get help mastering concepts.
Certificates
Earn official recognition for your work, and share your success with friends, colleagues, and employers.
制作方
National Research University Higher School of Economics
National Research University - Higher School of Economics (HSE) is one of the top research universities in Russia. Established in 1992 to promote new research and teaching in economics and related disciplines, it now offers programs at all levels of university education across an extraordinary range of fields of study including business, sociology, cultural studies, philosophy, political science, international relations, law, Asian studies, media and communications, IT, mathematics, engineering, and more.
Learn more on www.hse.ru
价格
| 购买课程 | |
|---|---|
| 访问课程材料 | 可用 |
| 访问评分的材料 | 可用 |
| 收到最终成绩 | 可用 |
| 获得可共享的课程证书 | 可用 |
评分和审阅
Awesome. Worth it!
This is one of the best courses I have taken on Coursera, The course is very rich in content and methodically developed. Both lectures have done a wonderful job in delivering lecture with full energy and make difficult concepts graspable. Both programming assignments and quizzes are well thought out. My sincere appreciation and thanks.
This is a great introduction to reinforcement learning.I faced some problems in submitting assignments but the course content and the extra materials are really good.I think it is to RL what Andrew Ng's Course is to ML . You will implement algorithms like cross entropy method to DQNs and A3C .Assignments uses Open AI Gym so you can get some good practical results .Overall I loved this course and would like to recommend to any one who is getting started in Reinforcement Learning.
Thank you.
S
Доведите ноутбуки и grader до ума, не позорьтесь пожалуйста!