Explore
For Enterprise
Join for Free
Log In

艺术与人文
商务
计算机科学
数据科学
信息技术
健康
数学和逻辑
个人发展
物理科学与工程
社会科学
语言学习
学位
证书

浏览 Coursera 的全部课程

浏览
搜索
企业版
登录
免费加入

Orthogonalization

Loading...

Structuring Machine Learning Projects

deeplearning.ai

4.8（30,438 个评分） | 160K 名学生已注册

课程 3（共 5 门，深度学习专项课程）

You will learn how to build a successful machine learning project. If you aspire to be a technical leader in AI, and know how to set direction for your team's work, this course will show you how. Much of this content has never been taught elsewhere, and is drawn from my experience building and shipping many deep learning products. This course also has two "flight simulators" that let you practice decision-making as a machine learning project leader. This provides "industry experience" that you might otherwise get only after years of ML work experience. After 2 weeks, you will: - Understand how to diagnose errors in a machine learning system, and - Be able to prioritize the most promising directions for reducing error - Understand complex ML settings, such as mismatched training/test sets, and comparing to and/or surpassing human-level performance - Know how to apply end-to-end learning, transfer learning, and multi-task learning I've seen teams waste months or years through not understanding the principles taught in this course. I hope this two week course will save you months of time. This is a standalone course, and you can take this so long as you have basic machine learning knowledge. This is the third course in the Deep Learning Specialization.

查看授课大纲

您将学习的技能

Machine Learning, Deep Learning, Inductive Transfer, Multi-Task Learning

审阅

4.8（30,438 个评分）

5 stars
25,065 ratings
4 stars
4,243 ratings
3 stars
943 ratings
2 stars
147 ratings
1 star
40 ratings

WG

Mar 19, 2019

Though it might not seem imminently useful, the course notes I've referred back to the most come from this class. This course is could be summarized as a machine learning master giving useful advice.

AP

Jun 21, 2019

Really a good course with mostly the theoretical knowledge on some aspects to reuse your model as well as some error analysis. Thoroughly taught with lots of real-life examples, thanks to Andrew Ng.

从本节课中

ML Strategy (1)

Why ML Strategy2:42

Orthogonalization10:38

教学方

Andrew Ng
CEO/Founder Landing AI; Co-founder, Coursera; Adjunct Professor, Stanford University; formerly Chief Scientist,Baidu and founding lead of Google Brain
Head Teaching Assistant - Kian Katanforoosh
Lecturer of Computer Science at Stanford University, deeplearning.ai, Ecole CentraleSupelec
Teaching Assistant - Younes Bensouda Mourri
Mathematical & Computational Sciences, Stanford University, deeplearning.ai

脚本

One of the challenges with building machine learning systems is that there's

so many things you could try, so many things you could change.

Including, for example, so many hyperparameters you could tune.

One of the things I've noticed is about the most effective machine learning people

is they're very clear-eyed about what to tune

in order to try to achieve one effect.

This is a process we call orthogonalization.

Let me tell you what I mean.

Here's a picture of an old school television,

with a lot of knobs that you could tune to adjust the picture in various ways.

So for these old TV sets, maybe there was one knob to adjust how

tall vertically your image is and another knob to adjust how wide it is.

Maybe another knob to adjust how trapezoidal it is,

another knob to adjust how much to move the picture left and

right, another one to adjust how much the picture's rotated, and so on.

And what TV designers had spent a lot of time doing was to build the circuitry,

really often analog circuitry back then,

to make sure each of the knobs had a relatively interpretable function.

Such as one knob to tune this, one knob to tune this, one knob to tune this,

and so on.

In contrast, imagine if you had a knob that tunes 0.1 x how tall the image is,

+ 0.3 x how wide the image is,- 1.7 x how trapezoidal the image is,

+ 0.8 times the position of the image on the horizontal axis, and so on.

If you tune this knob, then the height of the image, the width of the image,

how trapezoidal it is, how much it shifts, it all changes all at the same time.

If you have a knob like that, it'd be almost impossible to tune the TV so

that the picture gets centered in the display area.

So in this context, orthogonalization refers to that the TV designers

had designed the knobs so that each knob kind of does only one thing.

And this makes it much easier to tune the TV, so

that the picture gets centered where you want it to be.

Here's another example of orthogonalization.

If you think about learning to drive a car, a car has three main controls,

which are steering, the steering wheel decides how much you go left or

right, acceleration, and braking.

So these three controls, or really one control for steering and

another two controls for your speed.

It makes it relatively interpretable,

what your different actions through different controls will do to your car.

But now imagine if someone were to build a car so that there was a joystick,

where one axis of the joystick controls 0.3 x your steering

angle,- 0.8 x your speed.

And you had a different control that controls 2

x the steering angle, + 0.9 x the speed of your car.

In theory, by tuning these two knobs,

you could get your car to steer at the angle and at the speed you want.

But it's much harder than if you had just one single control for

controlling the steering angle, and a separate, distinct set of controls for

controlling the speed.

So the concept of orthogonalization refers to that,

if you think of one dimension of what you want to do as controlling

a steering angle, and another dimension as controlling your speed.

Then you want one knob to just affect the steering angle as much as possible,

and another knob, in the case of the car, is really acceleration and

braking, that controls your speed.

But if you had a control that mixes the two together,

like a control like this one that affects both your steering angle and your speed,

something that changes both at the same time,

then it becomes much harder to set the car to the speed and angle you want.

And by having orthogonal, orthogonal means at 90 degrees to each other.

By having orthogonal controls that are ideally aligned with the things you

actually want to control, it makes it much easier to tune the knobs you have to tune.

To tune the steering wheel angle, and

your accelerator, your braking, to get the car to do what you want.

So how does this relate to machine learning?

For a supervised learning system to do well, you usually need to

tune the knobs of your system to make sure that four things hold true.

First, is that you usually have to make sure that you're at least doing well

on the training set.

So performance on the training set needs to pass some acceptability assessment.

For some applications,

this might mean doing comparably to human level performance.

But this will depend on your application, and

we'll talk more about comparing to human level performance next week.

But after doing well on the training sets,

you then hope that this leads to also doing well on the dev set.

And you then hope that this also does well on the test set.

And finally, you hope that doing well on the test set on the cost

function results in your system performing in the real world.

So you hope that this resolves in happy cat

picture app users, for example.

So to relate back to the TV tuning example, if the picture of your TV was

either too wide or too narrow, you wanted one knob to tune in order to adjust that.

You don't want to have to carefully adjust five different knobs,

which also affect different things.

You want one knob to just affect the width of your TV image.

So in a similar way, if your algorithm is not fitting the training set well on

the cost function, you want one knob, yes, that's my attempt to draw a knob.

Or maybe one specific set of knobs that you can use,

to make sure you can tune your algorithm to make it fit well on the training set.

So the knobs you use to tune this are, you might train a bigger network.

Or you might switch to a better optimization algorithm,

like the Adam optimization algorithm, and so

on, into some other options we'll discuss later this week and next week.

In contrast, if you find that the algorithm is not fitting the dev set well,

then there's a separate set of knobs.

Yes, that's my not very artistic rendering of another knob,

you want to have a distinct set of knobs to try.

So for example, if your algorithm is not doing well on the dev set, it's doing well

on the training set but not on the dev set, then you have a set of knobs around

regularization that you can use to try to make it satisfy the second criteria.

So the analogy is, now that you've tuned the width of your TV set,

if the height of the image isn't quite right,

then you want a different knob in order to tune the height of the TV image.

And you want to do this hopefully without affecting the width of your TV

image too much.

And getting a bigger training set would be another knob you could use,

that helps your learning algorithm generalize better to the dev set.

Now, having adjusted the width and height of your TV image, well,

what if it doesn't meet the third criteria?

What if you do well on the dev set but not on the test set?

If that happens,

then the knob you tune is, you probably want to get a bigger dev set.

Because if it does well on the dev set but not the test set, it probably means you've

overtuned to your dev set, and you need to go back and find a bigger dev set.

And finally, if it does well on the test set, but it isn't delivering to you

a happy cat picture app user, then what that means is that you want to go back and

change either the dev set or the cost function.

Because if doing well on the test set according to some cost function

doesn't correspond to your algorithm doing what you need it to do in the real world,

then it means that either your dev test set distribution isn't set correctly,

or your cost function isn't measuring the right thing.

I know I'm going over these examples quite quickly, but we'll go much more

into detail on these specific knobs later this week and next week.

So if you aren't following all the details right now, don't worry about it.

But I want to give you a sense of this orthogonalization process,

that you want to be very clear about which of these maybe four issues,

the different things you could tune, are trying to address.

And when I train a neural network, I tend not to use early stopping.

It's not a bad technique, quite a lot of people do it.

But I personally find early stopping difficult to think about.

Because this is an op that simultaneously affects how well you fit the training set,

because if you stop early, you fit the training set less well.

It also simultaneously is often done to improve your dev set performance.

So this is one knob that is less orthogonalized,

because it simultaneously affects two things.

It's like a knob that simultaneously affects both the width and

the height of your TV image.

And it doesn't mean that it's bad, not to use, you can use it if you want.

But when you have more orthogonalized controls,

such as these other ones that I'm writing down here,

then it just makes the process of tuning your network much easier.

So I hope that gives you a sense of what orthogonalization means.

Just like when you look at the TV image, it's nice if you can say, my TV image

is too wide, so I'm going to tune this knob, or it's too tall, so I'm going to

tune that knob, or it's too trapezoidal, so I'm going to have to tune that knob.

In machine learning, it's nice if you can look at your system and

say, this piece of it is wrong.

It does not do well on the training set, it does not do well on the dev set,

it does not do well on the test set, or it's doing well on the test set but

just not in the real world.

But figure out exactly what's wrong, and then have exactly one knob, or

a specific set of knobs that helps to just solve that problem

that is limiting the performance of machine learning system.

So what we're going to do this week and next week is go through how to diagnose

what exactly is the bottleneck to your system's performance.

As well as identify the specific set of knobs you could use to tune your system to

improve that aspect of its performance.

So let's start going more into the details of this process.

探索我们的目录

免费加入并获得个性化推荐、更新和优惠。

Coursera 致力于普及全世界最好的教育，它与全球一流大学和机构合作提供在线课程。

© 2019 Coursera Inc. 保留所有权利。

通过 App Store 下载

通过 Google Play 获取

Coursera
关于
管理团队
工作机会
目录
证书
MasterTrack™ 证书
学位
企业版
政府版

社区
Learners
Partners
Developers
Beta Testers
Translators

连接
博客
Facebook
领英
Twitter
Instagram
YouTube
技术博客

更多
条款
隐私
帮助
内容访问
媒体
联系我们
目录
附属公司