YouSub
Untitled

en English

Untitled English subtitles

0 downloads · 908 lines · added Nov 18, 2015

Report a problem
What's wrong?
Advertisement

English transcript

PATRICK WINSTON: So today we're gonna talk about a few
miracles of learning in the context of the theme that
we're developing here in the class.
We started off with a discussion
of some basic methods.
We talked about nearest neighbors.
And we talked about identification trees.
And those are kind of basic things that have been around
for a long time.
Still useful.
Still the right things to do when you're faced with a
learning problem and you're not sure what method to try.
Then we went on to talk about some naive biological mimicry.
We talked about neural nets.
And we talked about genetic algorithms.
And you look at those things and you think and reflect back
on what we talked about.
And you have to say to yourself, are
these nugatory ideas?
Perhaps pistareens?
Or are they supererogatory ideas that deserve to be
center stage?
Does anybody know what those words mean?
A pistareen?
Well, a pistareen is a Spanish coin.
It was so small.
It was of little worth.
These ideas like neural nets, genetic algorithms, I classify
them as pistareens because getting them to do something
is rather like getting a dog to walk on its hind legs.
You can make it happen, but they never do it very well.
And you have to think it took a lot of trickery and training
to make it happen.
So not too personally high on those ideas.
But we teach them to you anyway because, of course, we
only editorialize part of time and part of time we like to
cover what's in the field.
Today we're starting a couple of discussions of mechanisms
or ideas or things to know about
that are quite different.
Because now we're going to focus on the problem rather
than on the mechanism.
And then a later on we're going to talk about deep
theory, FIOS, for its own sake.
But this week I want to talk about
mechanisms that were devised.
I want to talk about research that was done.
Let me not say mechanisms.
Let me say research that was done to attempt an account of
some of the things that we humans do well.
Sometimes without even knowing that we do it.
Now Krishna here tells me his first language was Telugu.
Telugu.
I once had another student whose first
language was Telugu.
I said to him, that must be one of those
obscure Indian languages.
And he said, yes.
It's spoken by 56 million people.
French is spoken by 52.
[LAUGHTER]
PATRICK WINSTON: He's going to be our experimental subject.
Krishna, if I pluralize words-- you know what it means
to pluralize a word.
So if I say for example, horse, then if I ask you for
the plural you'll say horses.
So if I say dog, what's the plural?
STUDENT: Then dogs.
Or in my language?
PATRICK WINSTON: No, no, no.
In English.
STUDENT: Oh, dogs.
PATRICK WINSTON: Well, what about cat?
STUDENT: Cats.
PATRICK WINSTON: And he got it right.
Isn't that a miracle?
When did you start speaking English?
STUDENT: Second grade.
PATRICK WINSTON: Second grade.
But he still got it right.
But he never learned that he's actually pluralizing those
words differently.
But he is.
So when you pluralize dog, what's the
sound that comes after?
It's a z sound.
Zzzzzz.
Dogzzz.
If you stick your fingers up here you can probably feel
your vocal cords vibrating.
If you stick a piece of paper in front of your mouth you'll
see it vibrate.
But when you say cats, the pluralizing sound
is sss, like that.
No vocalizing.
No vibration of the vocal cords.
And old Krishna here learned that rule, as did all of you
other non-native speakers of English, effortlessly and
without noticing it.
You learned it.
Buy you always get it right.
How can that possibly be?
Well, be the end of hour you'll know how that might be.
And you'll experience a case study in how questions of that
sort can be approached with a sort of
engineering point of view.
You can say, what if God were an engineer?
Or alternatively, what if I were God and I am an engineer?
Think about how it might happen that way.
So we want to understand how it might be that the machine
could learn rules like that.
Phonological rules.
Not just that one, but all the phonological rules you'd
acquire in a course on phonology.
That part of speaking that deals with those syllabic and
sub-syllabic sounds.
The phones of the language.
So when Yip and Sussman undertook to solve this
engineering problem, both being dedicated engineers, the
first thing they did was learn the science.
So they went to sit at the foot of Morris Halle, who
would develop-- was largely responsible for the
development theories of
so-called distinctive features.
And here's how all that works.
You start off with a person who wants to say something.
And out that person's mouth comes some sort of acoustic
pressure wave.
And if I say, hello, George.
And you say hello, George.
Everybody will understand that we said the same thing.
But that acoustic waveform won't look anything alike.
It'll be very different for all of us.
So it's a miracle that words can be understood.
In any case, it goes into an ear.
And it's processed.
And out comes a sequence of distinctive feature.
Vectors.
A distinctive feature is a binary variable like is the
phone voices or not.
That is to say, are your vocal cords vibrating
when you say it?
If so, then that's plus voiced.
If not, it's minus voiced.
So according to the original distinctive feature theory and
consistent with most of the theories that have been
derived since the original one, there are on the order of
14 of these distinctive features that determine which
phone you're saying.
So if you say ah, that's one combination of
these binary features.
If you say tuh, that's another combination of
these binary features.
14 of them.
So how many sounds does that mean, in principle, there
could be in a language?
SEBASTIAN: 2 to the 14th.
PATRICK WINSTON: And what's 2 the 14th, Sebastian?
Well, it ought to be about 16,000, don't you think?
2 to the 10th is 1,000.
2 the fourth is 16.
So there are about 16,000 possible combination.
But no language on Earth has more than 100 phones.
That's strange, isn't it?
Because some of those choices are probably excluded on
physical ground.
But most of them are not.
So we could have a lot more phones in our language than we
actually do.
English is about 40.
So the sequence of distinctive features could be viewed as
then producing meaning after, perhaps, a long series of
operations.
But in the end, those operations feedback in here
because many of the distinctive features are
actually hallucinated.
We think we heard them, but they're not there.
Or they're not even in the acoustic waveform.
They're there for the convenience of the phonologist
who make rules out of them.
It's remarkable how much of this feedback there is, and
even injection from other modalities.
Many of you may have heard about the McGurk Effect.
Here's who the McGurk Effect works.
Look at me while I say ga, ga, ga, ga, ga, ga.
OK.
I said, g-a.
Now how about ba, ba, ba, ba.
OK.
I said ba like a sheep.
But if I take the sound I make when I say ba and play it
while you're taking video of me saying ga, what do you
think you hear?
You don't hear ba.
Some people report that they hear a d-a sound like da.
When I look at it, I can't make any sense out of it.
It looks like there's a disconnection between the
speech and the video.
But it does not sound like ba.
But if I shut my eyes and say ba, ba, it's absolutely clear
that it's b-a.
So what you see has a large influence on what you hear.
It's also interesting--
although a side issue-- it's also interesting to note that
it's very difficult pronounced things correctly if you don't
see the speaker.
So many people wonder when they learn foreign languages
why they can't speak like a native.
And the answer is, they're not watching the
mouth of the speaker.
I was talking to a German friend once and said, you
know, I just can't say the damned umlaut right.
And he said, oh, the trouble with you Americans is you
don't realize that American cows say moo but
German cows say muu.
[LAUGHTER]
PATRICK WINSTON: And, of course, I got instantly
because I could see that the umlaut sounds are produced
with protruding lips, which we don't have any sounds an
English that require that.
Ah, but back to what we know from the phonologists about
all this stuff.
If you talk to Morris Halle, he will tell
you that over here--
I like to think of it as a marionette.
There are five pieces of meat down here.
And the combination of distinctive features that
you're trying to utter are like the control of a
marionette on those five pieces of meat.
So if you want to say an a sound, the marionette control
goes into a position that produces that combination.
So let's see.
What does that distinctive feature sequence look like for
typical word?
Well, here's a word.
A-e-p-l.
Apples.
And we can talk about what distinctive features are
arrayed in that particular combination of phones.
So one of the features that they like to
talk about is syllabic.
Syllabic.
That roughly means, can that sound form the sort of core of
a syllable?
And the answer is a can, buy these can't.
So it's plus, minus, minus, minus.
Down here a little ways you'll run into the voiced feature.
And for the voiced feature, well, we can do
the experiment ourselves.
Ahh.
Sounds like it's voices to me.
Pa.
No.
That's not voiced.
Oo.
Yep.
Zzz.
We already said that was voiced.
So that's the combination you see when you utter apples for
the voiced feature.
Then another one is the continuent one.
That roughly says is your vocal apparatus open?
Is there no obstruction?
And so ahh plus pa is constricted.
Oo, open.
Zzz, open.
So that one happens to run right along with voiced in
that particular word.
Oh, and there are 14 altogether.
But let me just write down one more.
The strident one.
That says, do you use your tongue to form a
little jet of air?
So you don't on aa, pa, oo.
Buy you do on z.
So that gets a plus.
So that's a glimpse through a soda straw of what it would
like to represent the word apples as a set of distinctive
features all arranged in a sequence.
So it's a matrix of features.
Going down in the columns we have our distinctive features.
And going across we have time.
So as the first thing Sussman and Yip did in their effort to
understand how phonological rules could be learned is to
design a machine that would interpret words and sounds and
things that you see so as to produce the
sounds of the language.
So they imagined the following kind of machine.
The machine has some kind of mystery apparatus over here
that looks out into the world and sees what's there.
So I'm looking out in the world and I see two apples.
So what this machine might do then is, at some point, decide
that there are two apples out there.
Then, thinking in terms of these guys as computer
engineers, they think in terms of a set of registers that
hold values for concepts like noun and verb and plural.
And we've not done anything with the machine yet.
We've provided no input.
So those registers are all empty.
Then, up in here, we have a set of words.
And they're all kinds of words.
Apple is one of them.
And those words up there know about how the concept is
rendered as a sequence of a phones, that is to say a
sequence of distinct features.
Then, over here, most importantly, they have a set
of constraints.
So we'll talk about a particular constrain, the
plural constraint.
Plural constraint number one.
And it's going to reach around and connect itself to some
other parts of the machine.
Finally, there's a buffer of phones to be uttered.
And they're going to flow out this way to the speaker's
mouth and get translated into a acoustic wave form.
So those are the elements of the machine.
Now how are the elements connected together?
Well, the words are connected, of course, into the buffer
that is used to generate the sound over
here on the far left.
The plural register is connected to what
you see in the world.
What you see in the world is connected not only to plural
register, but to all of the objects in the word
repertoire.
This plural constraint here deserves extra attention
because it's going to be desirous of actuating itself
in the event but the thing observed in
the world is plural.
There are lots of them.
So it's going to be connected then to the plural port.
There's going to be a z sound port down here connecting to
that file element in the buffer.
And finally, over here is going to be a plussed voiced
port, which is going to be connected to the second
phoneme in the sequence.
That's how the machine is going to be arranged.
An of course, this is just one of many constraints.
But it's a constraint that has a very peculiar property.
Information can flow through it in multiple ways.
So we think of most programs as having an
input and an output.
But I try to be careful to draw circles
here instead of arrows.
Because these are ports and information can flow in any
direction along them.
What I want to do now is to show you how this machine
would react if I suddenly present it with a pair of
apples like so.
So the assumption is that the vision apparatus comes in and
produces the notion, the concept, of two apples.
So once that has happened--
that's operation number one--
then information flows from that meaning register up here
to the apple word.
So that's part of stage number two.
Another part of stage number two is information flows along
this wire and marks that as plus plural.
So operation number one is the activity of the vision system.
Activity number two is the flow of information from that
vision system into the word lexicon and
into this plural register.
So far so good.
Here's activity number three.
This word is also connected to the registers.
And information flows along those wires so as to indicate
that it's a noun but not a verb.
That's part of part number three.
At the same time, part number three, information flows down
this wire and writes a-p-l into those are
elements of the buffer.
Now this constraint up here, this box, says, well, I can
now see some stuff in that buffer that
wasn't there before.
So it says, do I see enough stuff on my ports to get
excited about expressing values on other ports?
Well, let's see.
What has it got?
It's got the elements in this buffer.
Also up here in step three flow the plural thing.
So it know that the word is plural.
So it says, is this voiced?
P is pa.
That's not voiced.
Is this a z sound.
No, that's not as z sound.
So it sees what it likes on only one of its three ports.
So it says, I'm not going to do anything.
I'm [INAUDIBLE].
I'm not in this particular combat.
So far so good.
What happens next?
What happens next is that some time passes.
And the elements of the buffer flow to the left toward the
speaker's mouth.
So we get an a, p, l.
Same as we had before, but shifted over.
Now what happens?
Now what happens is that the l is now in
the penultimate position.
So information flows up here.
Item number four-- oh, I guess that's item number five.
Item number four is the leftward flow of the word.
So in phase number five, the p is witnessed by this
constraint.
p is--
sorry, l is witnessed by this constraint.
We moved it over one.
L is lll.
L is voiced.
So we have some flow up here like that.
That's number five.
Now we have voiced and we have plural.
And we have nothing here.
So there's a great desire of this buffer to have something
written into it.
So now there's a flow down in there, of z,
as item number six.
So that's how the machine would work in expressing the
idea that there are apples in the field of view.
Mmm.
Real apples.
Not plastic imitations.
So that's how the machine works.
But all those connections are reversible.
So if I hear apples then I get the machine running backwards
and my visual apparatus can imagine that there
are apples out there.
That's how it works.
That's just by way of background the machine that
they could see it for using the phonological rules once
they're learned.
All the phonological rules are expressed in these
constraints.
But since these constraints are such that information can
flow in any direction, they deserve to be called
propagators.
And in the good old days when everyone took 6.001, they
learned about propagators as a kind of architecture for
building complex systems.
But in any event, there's the Sussman-Yip machine.
And now comes the big question.
How do you learn rule rules like that?
Well, what we need is we need some positive examples and
some negative examples.
And for the simple classroom example I've chosen the same
challenge that I presented to Krishna.
We're gonna have cats and dogs.
So we're gonna look at the distinctive features that are
associated with those words.
Syllabic.
Voiced.
Continuent.
And strident.
Just four of the 14 features that are associated with each
of the sounds on those words.
Could you close the laptop, please?
Just for the distinctive features that are arrayed in
those words by way of illustration.
So here we have k-a-t-z.
Phonetically spelled.
And if we work that out, let's see.
What is syllabic?
That's not.
That is.
That's not.
Voiced?
Ka.
Nope.
Ah.
Yep.
T. Nope.
Z. Yes.
That can't be right.
Cats.
I misspelled it.
Because cats.
Sss.
His a hissing sound but there's no voicing.
So that's not as z sound.
That's an s sound.
So that's not plus voiced.
It's minused voiced.
Continuent.
Let's see.
Is my mouth open when I say k?
No.
Ah?
Yes.
T?
No.
S?
Yes.
And strident.
Minus, minus, minus, plus.
It's only with the s sound that I have that kind of jet
forming with my tongue.
Now we can look at dogs.
And now we have the z sound as the pluralization.
We know that because when we say it, dogzz.
Yep.
There it comes out as a--
we're only gonna look at the last two columns because
they're the only ones that are going to matter to us.
So that's plus.
And that's minus.
Gu, gu, gu, gu.
That's plussed.
And that's plussed.
They're both voiced.
Is that right?
Dogu?
Gu.
Is g sound voiced?
Yeah, I didn't think so.
G sound is voiced?
Look-- oh.
Oh, it is voiced buy it's not a continuent.
Just like that.
Yeah.
Cat, dogu zz.
Yeah.
It is voiced.
And it has to be for my example to work out.
And that's minus, minus, minus, plus.
So what we're interested in is, how come one word gets an
s sound and how come the other words gets a z sound?
Well, it's a pretty sparse space out there.
We've already decided that there are 14,000 possible
phonemes and there are only 40 in the language.
So that's one thing we can consider.
The other thing that we can think is that, well, maybe
this is a logical problem.
Like the kind of problem you'd face if you
were designing a computer.
And so Sussman and Yip got stuck for three months
thinking about the problem that way.
Couldn't make any progress whatsoever.
And that happens a lot when you're doing a search.
You think you've got a way of approaching it.
Try to make it work.
You stay up all night.
Stay up all night again.
Still can't make it work.
Eventually, you abandon ship and try something else.
So then they began to say, well, let's see.
All we care about is the stuff before the two ending sounds.
We care about that part of the matrix.
And we care about that part of the matrix.
And we can ask, in what ways are those things different?
And they're different all over the place.
That's why they're different words.
We can ask the question a little bit differently.
And we can say, what can we not care about?
And still retain enough of an understanding of how the words
are different so as to put the proper plural ending on them.
And they worried about that for a long time.
Couldn't find a solution.
The search space was too big.
And then they said, maybe what we ought to do is we ought to
think about generalizing this guy here so that we
don't care about it.
So now we don't care about that guy.
And then he went down through here saying, well, let's see
when we have to stop generalizing.
Because we've screwed everything up and we can no
longer keep the z sound words separated
from the s sound words.
So that eventually distilled itself down to
the following algorithm.
First thing they did was to c
…

Need subtitles from another video?