You will learn how to build a successful machine learning project. If you aspire to be a technical leader in AI, and know how to set direction for your team's work, this course will show you how. Much of this content has never been taught elsewhere, and is drawn from my experience building and shipping many deep learning products. This course also has two "flight simulators" that let you practice decision-making as a machine learning project leader. This provides "industry experience" that you might otherwise get only after years of ML work experience. After 2 weeks, you will: - Understand how to diagnose errors in a machine learning system, and - Be able to prioritize the most promising directions for reducing error - Understand complex ML settings, such as mismatched training/test sets, and comparing to and/or surpassing human-level performance - Know how to apply end-to-end learning, transfer learning, and multi-task learning I've seen teams waste months or years through not understanding the principles taught in this course. I hope this two week course will save you months of time. This is a standalone course, and you can take this so long as you have basic machine learning knowledge. This is the third course in the Deep Learning Specialization.
Orthogonalization
Loading...
来自 deeplearning.ai 的课程
Structuring Machine Learning Projects
26194 个评分
与讲师见面
Andrew NgCEO/Founder Landing AI; Co-founder, Coursera; Adjunct Professor, Stanford University; formerly Chief Scientist,Baidu and founding lead of Google Brain
Head Teaching Assistant - Kian KatanforooshLecturer of Computer Science at Stanford University, deeplearning.ai, Ecole CentraleSupelec
Teaching Assistant - Younes Bensouda MourriMathematical & Computational Sciences, Stanford University, deeplearning.ai
建立机器学习系统的挑战之一是 One of the challenges with building
machine learning systems is that there's
有太多可以尝试和改变的东西 so many things you could try, so
many things you could change.
包括 例如很多需要调整的超参数 Including, for example, so many
hyperparameters you could tune.
我注意到 最高效的机器学习人员 One of the things I've noticed is about
the most effective machine learning people
都非常清楚需要调整什么 is they're very clear-eyed about what to tune
来实习一个预期效果 in order to try to achieve one effect.
我们称这一过程为正交化 This is a process we call orthogonalization.
接下来 我来详细讲解下 Let me tell you what I mean.
这里有一张老式电视机的图片
上面有很多用于调整图像的旋钮 with a lot of knobs that you could tune
to adjust the picture in various ways.
因此 有可能某个旋钮可以调整 So for these old TV sets,
maybe there was one knob to adjust how
图像的垂直高度 另一个可以调整图像的宽度 tall vertically your image is and
another knob to adjust how wide it is.
也许 一个旋钮能调整梯形化程度 Maybe another knob to adjust how trapezoidal it is,
另一个可以左右移动图像 another knob to adjust how much
to move the picture left and right,
还有一个可以调整图像的旋转程度 等等 another one to adjust howmuch the picture's rotated, and so on.
而实际上电视设计者将大量的时间 花在了电路系统的设计上
And what TV designers had spent a lot of
time doing was to build the circuitry,
其实与当时的电路系统类似 really often analog circuitry back then,
就是为了确保每个旋钮都有一个实际的功能 to make sure each of the knobs had
a relatively interpretable function.
就好像其中一个旋钮控制这个 一个旋钮控制这个 一个旋钮来控制这个
Such as one knob to tune this, one knob
to tune this, one knob to tune this,
等等很多项
相反 想象一下如果你有一个旋钮 可以控制0.1倍的图像高度
In contrast, imagine if you had a knob
that tunes 0.1 x how tall the image is,
加上0.3倍的图像宽度 减去1.7倍的图像梯形化程度 + 0.3 x how wide the image is,-1.7x
how trapezoidal the image is,
再加上0.8倍的图像在横坐标上的位置 诸如此类 + 0.8 times the position of the image
on the horizontal axis, and so on.
如果你对这个旋钮进行调整 那么这个图像的高度 宽度 If you tune this knob, then the height
of the image, the width of the image,
梯形化程度 和它横移的位置 这些变量）都会同时发生改变
how trapezoidal it is, how much
it shifts, it all changes all at the same time.
如果你真的有这样的一个旋钮 那用户几乎不可能通过对电视荧幕进行调整 所以
If you have a knob like that, it'd be
almost impossible to tune the TV so
来实现将画面在电视屏幕位置居中的功能 that the picture gets
centered in the display area.
所以在这个情境中 正交化就是指电视设计师 So in this context, orthogonalization
refers to that the TV designers
在进行设计的时候 确保每个旋钮只能进行一个参数的调整
had designed the knobs so that each
knob kind of does only one thing
而这让电视屏幕的调整变得更加容易 And this makes it much easier to tune the TV, so
使得电视画面能够显示在你希望的位置上 that the picture gets centered
where you want it to be.
再来讲另一个正交化的例子 Here's another example of orthogonalization.
比如说想象一个学习开车的过程 一辆车主要有三个可控制的主体
If you think about learning to drive
a car, a car has three main controls,
分别是方向盘 which are steering, the steering
wheel decides how much you go left or
方向盘可以决定你向左或者向右转的程度 (另外两个操作是)加速和减速
所以有三个操控 或者说 So these three controls, or
really one control for steering and
一个对方向盘的操控 和另外两个对速度的操控 another two controls for your speed.
这让指令相对可以解读 It makes it relatively interpretable
你的不同动作能对车子产生不同的操控 what your different actions through
different controls will do to your car.
现在试想下 有人想设计一辆带有操纵杆的汽车 But now imagine if someone were to build
a car so that there was a joystick,
操纵杆的一个轴 只控制 where one axis of the joystick
controls 0.3 x your steering
(0.3*转向角-0.8*时速) angle,- 0.8 x your speed.
另一个不同的控制杆控制 And you had a different control that controls 2
(2*转向角+0.9*汽车时速) x the steering angle, + 0.9 x the speed of your car.
理论上 转动这两个旋钮 In theory, by tuning these two knobs,
可以让汽车转到既定方向 达到既定时速 you could get your car to steer at the
angle and at the speed you want.
但比起一个操纵杆控制转向角 But it's much harder than if
you had just one single control for
另一个单独的控制杆控制速度 controlling the steering angle, and
a separate, distinct set of controls for
这样的设计更困难 controlling the speed.
因此 正交化的概念在这个例子中指 So the concept of orthogonalization refers to that,
把一个维度看作控制转向角 if you think of one dimension of
what you want to do as controlling
把另一个维度看作控制车速 a steering angle, and another
dimension as controlling your speed.
那么 你需要一个尽可能影响转向角的旋钮 Then you want one knob to just
affect the steering angle as much as possible,
另一个旋钮 在汽车的例子中指油门和刹车 and another knob, in the case of the car,
is really acceleration and
以控制车速 braking, that controls your speed.
但是 如果你有一个能同时影响这两者的操纵杆 But if you had a control that mixes the two together,
像这个可以同时影响转向角和速度的操纵杆 like a control like this one that affects
both your steering angle and your speed,
能使两者同时变化 something that changes both at the same time,
那么 汽车在转向和变速时会很困难 then it becomes much harder to set
the car to the speed and angle you want.
通过使用正交化 正交化指(两个变量)成90度角 And by having orthogonal, orthogonal
means at 90 degrees to each other.
通过正交控制 能与你真正想要控制的事物保持一致 By having orthogonal controls that
are ideally aligned with the things you
它使调整相应旋钮变得更简便 actually want to control, it makes it much
easier to tune the knobs you have to tune.
改变方向盘角度 To tune the steering wheel angle, and
(控制)油门 刹车 让汽车完成你的各项指令 your accelerator, your braking,
to get the car to do what you want.
这与机器学习又有什么关系呢 So how does this relate to machine learning?
为保证有监督机制的的学习系统良好地运行 通常你需要 For a supervised learning system
to do well, you usually need to
调整系统旋钮 保证四件事准确无误 tune the knobs of your system to make
sure that four things hold true.
第一 通常确保至少训练组 First, is that you usually have to make
sure that you're at least doing well
运行良好 on the training set.
因此 训练组的运行情况需要进行一些可行性评估 So performance on the training set needs
to pass some acceptability assessment.
对于一些应用 For some applications,
这个能需要与人类的一些性能进行比较 this might mean doing comparably
to human level performance.
但这取决于你的应用 But this will depend on your application, and
下周我们将讨论更多关于与人类性能比较的内容 we'll talk more about comparing to
human level performance next week.
如果训练组表现良好 But after doing well on the training sets,
你将希望这能使开发组运行良好 you then hope that this leads to
also doing well on the dev set.
同时 你也会希望测试组运行良好 And you then hope that thisalso does well on the test set.
最终 你希望能在成本函数的测试组里运行良好 And finally, you hope that doingwell on the test set on the cost
因为结果会影响系统在实际情况中的表现 function results in your system
performing in the real world.
因此 希望这能顺利地实现 So you hope that this resolves in happy cat
举个例子 图片应用程序的使用者 picture app users, for example.
回到电视图像调整的例子 如果图像 So to relate back to the TV tuning
example, if the picture of your TV was
过宽或过窄 你需要一个旋钮来调整它 either too wide or too narrow, youwanted one knob to tune in order to adjust that.
你应该不希望仔细调整五个不同的旋钮 You don't want to have to carefullyadjust five different knobs,
而他们又会影响不同的参数 which also affect different things.
你只需要一个控制电视画面宽度的旋钮 You want one knob to just affect
the width of your TV image.
同理 如果你的算法不适合训练集的成本函数 So in a similar way, if your algorithmis not fitting the training set well on
你需要一个旋钮 是的 这是我画的一个旋钮 the cost function, you want one knob,
yes, that's my attempt to draw a knob.
或一组可以使用的旋钮 Or maybe one specific setof knobs that you can use,
以确保你可以优化算法 使其适用于训练组 to make sure you can tune your algorithmto make it fit well on the training set.
这些旋钮，可能用来训练一个更大的网络
或者，你可能需要换一种更好的优化算法
比如Adam优化算法等
其他选择，我们本周后半节和下周继续讨论
相反，如果你发现算法对dev集的拟合结果不太好
那么 就需要另一套独立的旋钮
是的，这就是我画的不太好的，另一套旋钮
你需要有一套不同的旋钮 来调整(开发集的拟合结果)
举例来说 如果你的算法在开发集上拟合效果不好
但是在训练集上效果很好 那么你需要一套旋钮
使用（这个旋钮）调整，使结果满足第二个条件
类比（电视的例子），既然你已经适配了电视的宽度
如果图像的高度不太对
那么你需要一个不同的旋钮去调整图像的高度
你希望这个调整，对图像的宽度，
不要产生太大的影响
用一个更大的训练集，是另一个可以用的旋钮
来帮助你的学习算法，在dev集上普及得更好
现在，我们已经调整了图像的宽度和高度
如果图像不满足第三个标准呢？
如果（算法）在dev集上表现很好 但是在测试集上不行呢？
如果这样，
你可能想要一个调整的旋钮 用来获取一个更大的dev集
因为如果（算法）在dev集上表现很好 但测试集上不行，这可能意味着，
你过度调节了dev集 你需要返回去，寻找一个更大的dev集
最后，如果（算法）在测试集上表现也很好 但是依然不能给你
愉快的用户体验，这意味着你需要返回去
调整你的dev集或者成本函数
因为如果在测试中成本函数上的优异表现
并不符合现实世界对算法的需求
就意味着你的验证数据或者测试数据有错误
或者你的成本函数所衡量的东西不正确
我知道我现在对这些例子讲得比较快，不过我们会
在这周和下周的课里细讲这些调整方式
因此，如果你感觉现在对具体操作还不清楚 不要担心
但我想给你一个正交过程的基本认识
就是你要清楚的知道大概这四个结果中
你可以调整的不同的东西，哪些在试图解决问题
当我训练神经网络的时候，我倾向于不提前停止训练
这不是一个不好的技巧， 有很多人都在用这个技巧
但我个人发现提前 停止训练很难想
因为这是一个同时会影响你拟合训练集的效果的操作
因为如果你提前停止训练，你就不是能很好的拟合训练集
它也同时经常被用来提高你的开发集的执行效果
所以这是一个更少正交化的旋钮
因为它同时影响了两件事情
它就像是一个旋钮，同时影响了你的电视图像的宽度和
电视图像的高度
这不意味着这不好，不能用，如果你想的话你可以用它
但你有更多正则化控制时
像是我写在这里的其他的这些（正则化控制）
然后, 它只是使调整您的网络的过程更容易。
所以我希望这部分的讲解可以给你一个正则化含义的大致认识
就像是你看电视图像时， 你可以说，我的电视图像
太宽，所以我要去调整这个旋钮，或者它太高，所以我要
调整那个旋钮，或者它太像梯形 所以我必须要去调整那个旋钮
在机器学习中，很不错的是如果你能看着你的系统然后
说，它的这部分错了
它在训练集上表现的不是很好， 在开发环境里也表现的不是很好
它在测试集上也表现的不太好， 或者说它在某些特定的测试集上表现的好一些
但是这些表现的好的特定测试集都不是在现实中的
但找出究竟是什么错了, 然后有完全一个旋钮, 或
一组特定的旋钮, 有助于解决这个问题
但这些解决问题的方法都限制着整个机器学期系统的性能
所以我们这周要做的和之后要学的就是如何诊断
诊断到底是遇到了什么瓶颈限制了系统的性能
同时也找到控着这些特定方面的旋钮， 以通过调节他们来调整个系统
从而提升这个方面的性能
于是让我们开始更细致地了解这个调参的过程吧。 GTC字幕组翻译
