YouSub
Untitled

en English

Untitled English subtitles

0 downloads · 914 lines · added Nov 18, 2015

Report a problem
What's wrong?
Advertisement

English transcript

PATRICK WINSTON: You know, it's unfortunate that politics
has become so serious.
Back when you were little it was a lot more fun.
You could make fun of politicians.
Here's a politician some of you may recognize.
But it's convenient to be able to vary what this particular
politician looks like.
For example, we can go from a cookie baker to radical.
[LAUGHTER]
PATRICK WINSTON: We can go from superwoman to bimbo.
[LAUGHTER]
PATRICK WINSTON: Socialite--
I put socialite into this.
There she is.
Or we can move the slider over the other way to bag lady.
Alert, asleep, sad, happy.
How does that work?
I don't know.
But I bet by the end of this hour you'll
know how that works.
And not only that, you'll understand something about
what it takes to recognize faces.
It turns out to some theories of face recognition are based
on the same principles that this program is based on.
But you can kind of guess what's happening here.
There are many stored images and when I move those sliders
it's interpolating amongst them.
So that's how that works.
But the main subject of today is this matter
of recognizing objects.
Faces could be the objects, but they don't have to be.
This could be an object that you might want to recognize.
And I want to talk to you a little bit about the history
of this problem and where it stands today.
It's still not solved.
But it's an interesting exercise to see how the
attempts at solution have evolved slowly
over the past 30 years.
So slowly, in fact, that I think if someone told me how
long it would take to get to where we are 30 years ago I
think I would have hung myself.
But things do move slowly.
And it's important to see how slowly they move.
Because they will continue to move slowly in the future.
And you have to understand that that's the way things
work sometimes.
So to start this all off, we have to go back to the ideas
of the legendary David Marr, who dropped dead from leukemia
in about 1980.
I say, the gospel according to Marr, because he was such a
powerful and central figure that almost anything he said
was believed by a large collection of devotees.
But Marr articulated a set of ideas about how computer
vision would work that started off by suggesting that with
the input from the camera, you look for edges.
And you find edge fragments.
And normally they wouldn't be even as well-drawn as I've
done them now.
Or as badly drawn as I've done them now.
But the first step, then, in visual recognition would be to
form this edge-based description of what's out
there in the world.
And Marr called that the primal sketch.
And from the primal sketch, the next step was to decorate
the primal sketch with some vectors, some surface normals,
showing where the faces on the object were oriented.
He called that the two and a half D sketch.
Now why is it two and a half D?
Well, it's sort of 2D in the sense that it's still
camera-centric in its way of presenting information.
But at same time, it attempts to say something about the
three-dimensional arrangement of the faces.
So the speculation was that you couldn't get to where you
wanted to go in one step.
So you needed several steps to get from the image to
something you could recognize.
And the third step was to convert the two and a half D
sketch into generalized cylinders.
And the idea is this.
If you have a regular cylinder, you can think of it
as a circular area moving along an axis like so.
So that's the description of a cylinder.
A circular area moving along an axis.
You can get a different kind of cylinder if you go along
the same axis but you allow the size of the circle to
change as you go.
So for example, if you were to describe a wine bottle.
It would be a function of distance along the axis that
would shrink the circle appropriately to match the
dimensions of a wine bottle.
A fine burgundy, I perceive.
In any case, this one once converted into a generalized
cylinder, when matched against a library of such
descriptions, results in recognition.
Great theory, based on the idea that you start off by
looking at edges and you end up, in several steps of
transformation, producing something that you could look
up in a library of descriptions.
Great idea.
Trouble is, no one could make it work.
It was too hard to do this.
It was too hard to do that.
And the generalized cylinders produced, if
any, were too coarse.
You couldn't tell the difference between a Ford and
a Chevrolet or between a Volkswagen and a Cadillac.
Because they were just too coarse.
So although it was a great idea based on the idea that
you have to do recognition in several transformations of
representational apparatus, it just didn't work.
So much later, maybe 15 years later or so, we get to the
next part of our story.
Which is the alignment theories, most notably the one
produced by Shimon Ullman, one of Marr's students.
So the alignment theory of recognition is based on a very
strange and exotic idea.
It doesn't seem strange and exotic to mechanical engineers
for a while, because they're used to mechanical drawings.
But here's the strange and miraculous idea.
Imagine this object.
You take three pictures of it.
You can reconstruct any view of that object.
Now, I have to be a little bit careful about how I say that.
First of all, some of the vertexes are not visible in
the views that you have.
So, of course, you can't do anything with those.
So let's say that we have a transparent object where you
can see all the vertexes.
If you have three pictures of that, you can reconstruct any
view of that object.
Now I have to be a little careful about how I say that,
because it's not true.
What's true is, you can produce any view of that in
orthographic projection.
So if you're close enough to the object that you get
perspective, it doesn't work.
But for the most part, you can neglect perspective after you
get about two and a half times as far away as
the object is big.
And you can presume that you've got orthographic
projection.
So that's a strange and exotic idea.
But how can you make a recognition
theory out of that?
So let me show you.
Well, here's one drawing of the object, I need two more.
Let's see.
Let's have this one.
And maybe one that's tilted up a little bit.
It's important that these pictures not be just rotations
on one axis.
Because they wouldn't form what you might think of as a
kind of basis set.
So there are three pictures.
We'll call them a, b, and c.
And then we want a fourth picture.
Which will look like this.
It doesn't have to be too precise.
And we'll call that the unknown.
And what we really want to know is if the unknown is the
same object that these three pictures were made from.
So let me begin with an assertion.
I'll need four colors of chalk to make this assertion.
What I want to do is I want to pick a particular place on the
object, like this one.
And maybe the same place on this object over here.
Those are corresponding places, right?
So I can now write an equation that the x-coordinate of that
unknown object is equal to, oh, I don't know, alpha x sub
a plus beta x sub b plus gamma x sub c plus
some constant, tau.
Well, of course, that's obviously true.
Because I'm letting you take those alpha, beta, gamma, and
tau and make them anything you want.
So although that's conspicuously obviously true,
it's not interesting.
So let me take another point.
And of course, I can write the same equation down for this
purple point.
And now that I'm on a roll and having a great deal of fun
with this, I can take this point
and make a blue equation.
And you know I'm destined to do it, so I've
got one more color.
I might as well use it.
Let's just make sure I get something that works here.
That's this one, that's this one.
I hope I've got these correspondences right.
STUDENT: [INAUDIBLE].
PATRICK WINSTON: Have I got one off?
STUDENT: [INAUDIBLE].
PATRICK WINSTON: Which color?
STUDENT: Blue.
[INAUDIBLE].
PATRICK WINSTON: OK.
So this one goes with this one, goes with this one.
Is that one wrong?
STUDENTS: Yeah.
PATRICK WINSTON: Oh, oh, oh.
Of course this one, excuse me, goes down here.
Right?
And then this one is off as well.
I wouldn't get a very good recognition scheme if I can't
get those correspondences right.
Which is one of the lessons of today.
OK.
Now I've got them right.
And now that equation is correct.
I think I've got this one right already.
So now I can just write that down.
I'm on a roll, I'm just copying this.
So those are a bunch of equations.
And now the astonishing part is that I can choose alpha,
beta, gamma, and tau to be all the same.
That is, there's one set of alpha, beta, gamma, and tau
that works for everything, for all four points.
So you look at that puzzled.
And that's OK to be puzzled.
Because I certainly haven't proved it.
I'm asserting it.
But right away, there's something interesting about
this and that is that the relationship between the
points on the unknown object and the points in this stored
library of images are related linearly.
That's true because it's orthographic projection.
Linearly related.
So I can generate the points in some fourth object from the
points in three sample objects with linear operations.
Christopher?
STUDENT: Is that the x-coordinate of--
PATRICK WINSTON: It's the x-coordinate.
Christopher asked about the x-coordinates.
Each of these x-coordinates are meant to be color coded.
It gets a little complicated with notation and stuff.
So that's the reason I'm color coding the coordinates.
So the orange x sub u is the x-coordinate of that
particular point.
STUDENT: In 3D space?
PATRICK WINSTON: No.
Not in 3D space.
In the image.
STUDENT: So it's a 2D projection of it?
PATRICK WINSTON: It's a 2D projection of it, an
orthographic projection.
OK?
So we're looking at drawings.
And those coordinates over there are the two-dimensional
coordinates in the drawing.
Just as if it were on your retina.
STUDENT: [INAUDIBLE]
vertexes on the 3D projection or can curved surfaces also?
PATRICK WINSTON: So he asked about curved surfaces.
And the answer is that you have to find corresponding
points on the object.
So if you have a totally curved surface and you can't
identify any corresponding points, you lose.
But if you consider our faces, there are some obvious points,
even though our face are not by any means
flat like these objects.
We have the tip of our nose and the center of our
eyeballs and so on.
So if that's true, what does that mean about recovering
alpha, beta, gamma, and tau?
Can we find them?
[INAUDIBLE], what do you think?
How do we go about finding them?
You're nodding your head in the right direction.
[LAUGHTER]
STUDENT: It's four equations and--
PATRICK WINSTON: Splendid.
It's four equations and four unknowns.
Four linear equations and four unknowns.
So obviously, you can solve for alpha, beta, gamma, and
tau if you know that these equations are correct.
So how does that help us with recognition?
It helps us with recognition because we can take another
point, let me say this square point here and this
corresponding square point here and this corresponding
square point here, and what can we do with those three
points now?
We've got alpha, beta, gamma, and tau, so we can predict
where it's going to be in the fourth image.
So we can predict that that square point is going to be
right there.
And if it isn't, we're highly suspicious about whether this
object is the kind of object we think it is.
So you look at me in disbelief.
You'd like me to demonstrate this, I imagine.
STUDENT: Yeah.
PATRICK WINSTON: OK.
Let me see if I can demonstrate this.
So I'm going to do this in a slightly simplified version.
I'm only going to allow rotation around
the vertical axis.
And just so you know I'm not cheating, there's a little
slider here that rotates that third object.
Let's see, why are there just two known
objects and one unknown?
Well that's because I've restricted the motion to
rotation around the vertical axis and some translation.
So now that I've spun that around a little bit, let me
pick some corresponding points.
Oops.
What's happened?
Wow.
Let me run that by again.
OK.
So there's one point I've selected
from the model objects.
The corresponding point over here on the
unknown is right there.
I'm going to be a little off.
But that's OK.
So let me just pick that one and then that
corresponds to this one.
Krishna, would you like to specify a point so people know
I'm not cheating.
Pick a point.
Pick a point, Krishna.
STUDENT: Oh, the right?
PATRICK WINSTON: The right?
STUDENT: Yeah.
PATRICK WINSTON: This one?
STUDENT: Yep.
PATRICK WINSTON: Oops.
OK, let's pick it out on the model first.
Now pick it over here.
Boom.
So all the points are where they're supposed to be.
Isn't that cool?
Well, let's suppose that the unknown is something else.
This is a carefully selected object.
Because the points are all the correct positions vertically,
but they're not necessarily the correct positions in the
other two dimensions.
So let me pick this point, and this point, and this point,
and this point.
And Krishna had me pick this point.
So let me pick this point.
So if it thinks that the unknown is one of these
obelisk objects, then we would expect to see all of the
corresponding points correctly identified.
But boom.
All the points are off.
So it seems to work in this particular example.
I find the alpha and beta using two images.
And I predict the locations of the other points.
And I determine whether those positions are correct.
And if they are correct, then I have a pretty good idea that
I have in fact identified the object on the right as either
an obelisk or an organ, depending on which of the
model choices and the unknown choices I've selected.
So the only thing I have left to do is to demonstrate that
what I said about this is true.
So I'm going to actually demonstrate that what I said
about this is true using the configuration in this
demonstration.
Because it's much too hard for me to remember matrix
transformations for generalized rotation in three
dimensions.
So here's how it's going to work.
The z-axis is going up that way.
Or, it's going to be pointing toward you.
And what I'm going to do is I'm going to
rotate around this axis.
And what I want to do is I want to find out how the
x-coordinate in the image of the points move
as I do that rotation.
So here's the x-axis.
This is the coordinate that you can see.
Here is the y-axis.
That's in depth, so you can't tell how far away it is.
And the z-axis--
x, y, z-axis must be pointing out that way toward you.
So now I'm going to consider just a single point on the
object and see what happens to it.
So I'm going to say to myself, let's put the object in some
kind of standard position.
I don't care what it is.
It can be just random, just spin it around.
Some position, we'll call that the standard position, S. And
that means that the x-coordinate of the standard
position is x sub s.
And the y-coordinate of the standard position is y sub s.
And now I'm going to rotate the object three times.
Once to form the a picture, once to form the b picture,
and once to form the c picture.
And you can make those choices.
Those can be anything, right?
So let's say that the a picture is out here.
So that's the a picture.
The B picture is out here.
And the unknown is up that way.
And so what I want to know depends on these vectors.
We'll call that theta sub a, and this is theta sub b.
And this one is theta sub u.
So I would like to know how x sub a depends on x
sub s and y sub s.
And I can never remember how to do that, because I can
never remember the transformation
equations for rotation.
So I have to figure it out every time.
And this is no exception.
So what I'm going to say is that this vector that goes out
to S consists of two pieces.
There's the x part and the y part.
And I know that I can rotate this vector by alpha sub a by
rotating this vector and rotating that vector and
adding up the results.
So if I rotate this vector by alpha sub a, then the
contribution of that to the x-coordinate of a is going to
be given by the cosine of theta sub a
multiplied by x sub s.
So you can just exaggerate that motion, say, well if I
pitch it up that way then the projection down on the x-axis
is going to be this length of the vector times the
cosine of the angle.
Now there's also going to be a dependence on y sub s.
Let's figure out what that's going to be.
I've got this vector here.
And I'm going to rotate it by theta sub a as well.
If I rotate that by theta sub a and see what the projection
is on the x-axis, that's going to be given by
the sine of the angle.
But it's going the wrong way, so I have to subtract it off.
So that's how I don't have to remember what the signs are on
these equations.
Well, that was good.
And now that I'm off and running I can
do what I did before.
It makes it easy to give the lecture.
Because this is going to be x sub b is equal to x sub s
times the cosine of theta sub b minus y sub s times the
cosine of theta--
oh, you're letting me make mistakes.
Shame.
I can generally tell by all the troubled looks.
But there should be some shouting as well.
That's the sine and that's the sine.
And one more time.
x sub u is equal to x sub s times the cosine of theta sub
u minus y sub s times the sine of theta sub u.
And I forgot the b up there.
So there are some equations.
And we don't know what we're doing.
We're just going to stare at them awhile and see if they
sing us a song.
So let's see if they sing us a song.
What about x sub a and x sub b?
These are things that we see in the image.
These are things that we can measure.
What about all those cosines and sines of theta
a's and theta b's.
Well, we have no idea what they are.
But one thing is clear.
They're true for all of the points on the object.
Because when we rotate the object around by angle theta,
we're rotating all of the points through
the same angle, right?
So with respect to any
particular view of the object--
here we are in the standard position.
Here we are in position a.
The vectors to all of the points on the object are
rotated by the same angle when we go from the standard
position to the a position.
So that means that for all of the images in this particular
rendering, with a particular rotation by theta a, theta b,
and theta u, those are constants.
Now remember this is for a particular theta a, a
particular theta be, and a particular theta u.
As long as we're talking about all of the points for each of
those rotations, those angles and cosines are going to be
the same for all possible points on the object.
OK.
So now we go back to our high school algebra experts and we
say, look at these first two equations, We've got two
equations and what we can now construe to be two unknowns.
What are the unknowns that are left?
We can measure a and b.
Whatever the cosines are, they're the
same for all the pictures.
So if we treat those as constants, then we can solve
for x sub s and y sub s.
Right?
We can solve for x sub s and y sub s in terms of x sub a and
x sub b and a whole bunch of constants.
But, I don't know, a whole bunch of constants, let's see.
We can gather up all of those cosines and ratios of sines
and cosines and all that stuff and put them all together.
Because they're all constants.
And then we can do this.
We can say x sub u is equal to--
well, it's going to depend on x sub a and x sub b.
And by the time we wash or manipulate or screw around
with all those cosines, we can say that the multiplier for x
sub a is some constant alpha and the multiplier for x sub b
is some constant beta.
So that's not a slight of hand.
That's just linear
manipulation of those equations.
And that's what we wanted to show, that for orthographic
projection, which this is-- there is no perspective
involved here, we're just taking the projection along
the x-axis--
we can demonstrate for this simplified situation that that
equation must hold.
Now I want to give you a few puzzles.
Because this stuff is so simple.
Suppose I allow translation as well as rotation.
What's going t
…

Need subtitles from another video?