Advertisement
English transcript
In this video, lets delve deeper and get even better intuition about what the cost
function is doing. This video assumes that you're familiar with contour plots. If you
are not familiar with contour plots or contour figures some of the illustrations
in this video may or may not make sense to you but is okay and if you end up skipping
this video or some of it does not quite make sense because you haven't seen
contour plots before. That's okay and you will still understand the rest of this course
without those parts of this. Here's our problem formulation as usual, with the
hypothesis parameters, cost function, and our optimization objective. Unlike
before, unlike the last video, I'm going to keep both of my parameters, theta
zero, and theta one, as we generate our visualizations for the cost function. So, same
as last time, we want to understand the hypothesis H and the cost function J. So,
here's my training set of housing prices and let's make some hypothesis. You know,
like that one, this is not a particularly good hypothesis. But, if I set theta
zero=50 and theta one=0.06, then I end up with this hypothesis down here and that
corresponds to that straight line. Now given these value of theta zero and theta one,
we want to plot the corresponding, you know, cost function on the right. What we
did last time was, right, when we only had theta one. In other words, drawing plots
that look like this as a function of theta one. But now we have two parameters, theta
zero, and theta one, and so the plot gets a little more complicated. It turns out
that when we have only one parameter, that the parts we drew had this sort of bow
shaped function. Now, when we have two parameters, it turns out the cost function
also has a similar sort of bow shape. And, in fact, depending on your training set,
you might get a cost function that maybe looks something like this. So, this is a
3-D surface plot, where the axes are labeled theta zero and theta one. So
as you vary theta zero and theta one, the two parameters, you get different values of the
cost function J (theta zero, theta one) and the height of this surface above a
particular point of theta zero, theta one. Right, that's, that's the vertical axis. The
height of the surface of the points indicates the value of J of theta zero, J
of theta one. And you can see it sort of has this bow like shape. Let me show you
the same plot in 3D. So here's the same figure in 3D, horizontal axis theta one and
vertical axis J(theta zero, theta one), and if I rotate this plot around. You kinda of a
get a sense, I hope, of this bowl shaped surface as that's what the cost
function J looks like. Now for the purpose of illustration in the rest of this video
I'm not actually going to use these sort of 3D surfaces to show you the cost
function J, instead I'm going to use contour plots. Or what I also call contour
figures. I guess they mean the same thing. To show you these surfaces. So here's an
example of a contour figure, shown on the right, where the axis are theta zero and
theta one. And what each of these ovals, what each of these ellipsis shows is a set
of points that takes on the same value for J(theta zero, theta one). So
concretely, for example this, you'll take that point and that point and that point.
All three of these points that I just drew in magenta, they have the same value
for J (theta zero, theta one). Okay. Where, right, these, this is the theta
zero, theta one axis but those three have the same Value for J (theta zero, theta one)
and if you haven't seen contour plots much before think of, imagine if you
will. A bow shaped function that's coming out of my screen. So that the minimum, so
the bottom of the bow is this point right there, right? This middle, the middle of
these concentric ellipses. And imagine a bow shape that sort of grows out of my
screen like this, so that each of these ellipses, you know, has the same height
above my screen. And the minimum with the bow, right, is right down there. And so
the contour figures is a, is way to, is maybe a more convenient way to
visualize my function J. [sound] So, let's look at some examples. Over here, I have a
particular point, right? And so this is, with, you know, theta zero equals maybe
about 800, and theta one equals maybe a -0.15 . And so this point, right, this
point in red corresponds to one set of pair values of theta zero, theta one
and the corresponding, in fact, to that hypothesis, right, theta zero is
about 800, that is, where it intersects the vertical axis is around 800, and this is
slope of about -0.15. Now this line is really not such a good fit to the
data, right. This hypothesis, h(x), with these values of theta zero,
theta one, it's really not such a good fit to the data. And so you find that, it's
cost. Is a value that's out here that's you know pretty far from the minimum right
it's pretty far this is a pretty high cost because this is just not that good a fit
to the data. Let's look at some more examples. Now here's a different
hypothesis that's you know still not a great fit for the data but may be slightly
better so here right that's my point that those are my parameters theta zero theta
one and so my theta zero value. Right? That's bout 360 and my value for theta
one. Is equal to zero. So, you know, let's break it out. Let's take theta zero equals
360 theta one equals zero. And this pair of parameters corresponds to that
hypothesis, corresponds to flat line, that is, h(x) equals 360 plus zero
times x. So that's the hypothesis. And this hypothesis again has some cost, and
that cost is, you know, plotted as the height of the J function at that point.
Let's look at just a couple of examples. Here's one more, you know, at this value
of theta zero, and at that value of theta one, we end up with this hypothesis, h(x)
and again, not a great fit to the data, and is actually further away from the minimum. Last example, this is
actually not quite at the minimum, but it's pretty close to the minimum. So this
is not such a bad fit to the, to the data, where, for a particular value, of, theta
zero. Which, one of them has value, as in for a particular value for theta one. We
get a particular h(x). And this is, this is not quite at the minimum, but it's
pretty close. And so the sum of squares errors is sum of squares distances between
my, training samples and my hypothesis. Really, that's a sum of square distances,
right? Of all of these errors. This is pretty close to the minimum even though
it's not quite the minimum. So with these figures I hope that gives you a better
understanding of what values of the cost function J, how they are and how that
corresponds to different hypothesis and so as how better hypotheses may corresponds to points
that are closer to the minimum of this cost function J. Now of course what we really
want is an efficient algorithm, right, a efficient piece of software for
automatically finding The value of theta zero and theta one, that minimizes the
cost function J, right? And what we, what we don't wanna do is to, you know, how to
write software, to plot out this point, and then try to manually read off the
numbers, that this is not a good way to do it. And, in fact, we'll see it later, that
when we look at more complicated examples, we'll have high dimensional figures with
more parameters, that, it turns out, we'll see in a few, we'll see later in
this course, examples where this figure, you know, cannot really be plotted, and
this becomes much harder to visualize. And so, what we want is to have software
to find the value of theta zero, theta one that minimizes this function and
in the next video we start to talk about an algorithm for automatically finding
that value of theta zero and theta one that minimizes the cost function J.
function is doing. This video assumes that you're familiar with contour plots. If you
are not familiar with contour plots or contour figures some of the illustrations
in this video may or may not make sense to you but is okay and if you end up skipping
this video or some of it does not quite make sense because you haven't seen
contour plots before. That's okay and you will still understand the rest of this course
without those parts of this. Here's our problem formulation as usual, with the
hypothesis parameters, cost function, and our optimization objective. Unlike
before, unlike the last video, I'm going to keep both of my parameters, theta
zero, and theta one, as we generate our visualizations for the cost function. So, same
as last time, we want to understand the hypothesis H and the cost function J. So,
here's my training set of housing prices and let's make some hypothesis. You know,
like that one, this is not a particularly good hypothesis. But, if I set theta
zero=50 and theta one=0.06, then I end up with this hypothesis down here and that
corresponds to that straight line. Now given these value of theta zero and theta one,
we want to plot the corresponding, you know, cost function on the right. What we
did last time was, right, when we only had theta one. In other words, drawing plots
that look like this as a function of theta one. But now we have two parameters, theta
zero, and theta one, and so the plot gets a little more complicated. It turns out
that when we have only one parameter, that the parts we drew had this sort of bow
shaped function. Now, when we have two parameters, it turns out the cost function
also has a similar sort of bow shape. And, in fact, depending on your training set,
you might get a cost function that maybe looks something like this. So, this is a
3-D surface plot, where the axes are labeled theta zero and theta one. So
as you vary theta zero and theta one, the two parameters, you get different values of the
cost function J (theta zero, theta one) and the height of this surface above a
particular point of theta zero, theta one. Right, that's, that's the vertical axis. The
height of the surface of the points indicates the value of J of theta zero, J
of theta one. And you can see it sort of has this bow like shape. Let me show you
the same plot in 3D. So here's the same figure in 3D, horizontal axis theta one and
vertical axis J(theta zero, theta one), and if I rotate this plot around. You kinda of a
get a sense, I hope, of this bowl shaped surface as that's what the cost
function J looks like. Now for the purpose of illustration in the rest of this video
I'm not actually going to use these sort of 3D surfaces to show you the cost
function J, instead I'm going to use contour plots. Or what I also call contour
figures. I guess they mean the same thing. To show you these surfaces. So here's an
example of a contour figure, shown on the right, where the axis are theta zero and
theta one. And what each of these ovals, what each of these ellipsis shows is a set
of points that takes on the same value for J(theta zero, theta one). So
concretely, for example this, you'll take that point and that point and that point.
All three of these points that I just drew in magenta, they have the same value
for J (theta zero, theta one). Okay. Where, right, these, this is the theta
zero, theta one axis but those three have the same Value for J (theta zero, theta one)
and if you haven't seen contour plots much before think of, imagine if you
will. A bow shaped function that's coming out of my screen. So that the minimum, so
the bottom of the bow is this point right there, right? This middle, the middle of
these concentric ellipses. And imagine a bow shape that sort of grows out of my
screen like this, so that each of these ellipses, you know, has the same height
above my screen. And the minimum with the bow, right, is right down there. And so
the contour figures is a, is way to, is maybe a more convenient way to
visualize my function J. [sound] So, let's look at some examples. Over here, I have a
particular point, right? And so this is, with, you know, theta zero equals maybe
about 800, and theta one equals maybe a -0.15 . And so this point, right, this
point in red corresponds to one set of pair values of theta zero, theta one
and the corresponding, in fact, to that hypothesis, right, theta zero is
about 800, that is, where it intersects the vertical axis is around 800, and this is
slope of about -0.15. Now this line is really not such a good fit to the
data, right. This hypothesis, h(x), with these values of theta zero,
theta one, it's really not such a good fit to the data. And so you find that, it's
cost. Is a value that's out here that's you know pretty far from the minimum right
it's pretty far this is a pretty high cost because this is just not that good a fit
to the data. Let's look at some more examples. Now here's a different
hypothesis that's you know still not a great fit for the data but may be slightly
better so here right that's my point that those are my parameters theta zero theta
one and so my theta zero value. Right? That's bout 360 and my value for theta
one. Is equal to zero. So, you know, let's break it out. Let's take theta zero equals
360 theta one equals zero. And this pair of parameters corresponds to that
hypothesis, corresponds to flat line, that is, h(x) equals 360 plus zero
times x. So that's the hypothesis. And this hypothesis again has some cost, and
that cost is, you know, plotted as the height of the J function at that point.
Let's look at just a couple of examples. Here's one more, you know, at this value
of theta zero, and at that value of theta one, we end up with this hypothesis, h(x)
and again, not a great fit to the data, and is actually further away from the minimum. Last example, this is
actually not quite at the minimum, but it's pretty close to the minimum. So this
is not such a bad fit to the, to the data, where, for a particular value, of, theta
zero. Which, one of them has value, as in for a particular value for theta one. We
get a particular h(x). And this is, this is not quite at the minimum, but it's
pretty close. And so the sum of squares errors is sum of squares distances between
my, training samples and my hypothesis. Really, that's a sum of square distances,
right? Of all of these errors. This is pretty close to the minimum even though
it's not quite the minimum. So with these figures I hope that gives you a better
understanding of what values of the cost function J, how they are and how that
corresponds to different hypothesis and so as how better hypotheses may corresponds to points
that are closer to the minimum of this cost function J. Now of course what we really
want is an efficient algorithm, right, a efficient piece of software for
automatically finding The value of theta zero and theta one, that minimizes the
cost function J, right? And what we, what we don't wanna do is to, you know, how to
write software, to plot out this point, and then try to manually read off the
numbers, that this is not a good way to do it. And, in fact, we'll see it later, that
when we look at more complicated examples, we'll have high dimensional figures with
more parameters, that, it turns out, we'll see in a few, we'll see later in
this course, examples where this figure, you know, cannot really be plotted, and
this becomes much harder to visualize. And so, what we want is to have software
to find the value of theta zero, theta one that minimizes this function and
in the next video we start to talk about an algorithm for automatically finding
that value of theta zero and theta one that minimizes the cost function J.