Advertisement
English transcript
Hello everybody, we will continue our lecture in this optimal control course. And then,
this particular lecture, we will talk about linear quadratic regulator and then subsequent
two, three lectures also, we will keep on talking about these. There are several reasons
for that and as you see in the previous couple of lectures, these are computationally complex
problems in general, any optimal control formulation leads to these two point boundary value problems
and all that. So, that requires a lot of numerical intensive
procedure and all. So, then the, then at least in 50’s and 60’s people started thinking,
that is there any class of problems for which we would not really need all those kind of
numerical techniques? And we can get solutions in, in the other computational efficient way,
even though it is limited to a class of problems actually. So, that leads to this, this definition
of linear quadratic regulator problems and extensively studied and extensively utilized
as well. It is one of the very popular tricks, tricks of optimal controller or class of optimal
control problem that is found in usage in industries as well, actually.
So, we will see that and then proceed further, some, some of this extension proofs and things
like that in the subsequent lectures actually. So, let us see, understand, what is this LQR
problem and then, furthermore will be subsequent lectures, anyway.
So, generic optimal control problem, as I discussed so many times before, this is what
it is, J, J takes the form of some phi, this is terminal penalty plus some path penalty
sort of thing. And then, along with that, that, there is a path constraint, which is
a system dynamic equation, as well as boundary conditions, actually.
And we will consider, typically this X t f is free, X of t a, but some problems, it can
be and also basically, actually, alright, anyway. So, this, but this particular LQR
problem does not talk about generic formulation any more. Thus, it talks about a specific
form of cross function and a specific form of system analysis.
So, the, this problem can be defined this way. The objective is to drive the state of
a linear or many times, linear is nothing, but a linearized system, anyway we know that.
So, the objective is to drive the state X of a linear system, given by this equation,
to the origin, that means, X of T f has to be 0 0, by, so how do you, so that it can
be done by minimizing the following quadratic performance index actually.
So, this, the, this is, this is the terminal, terminal penalty X f transpose S f X f and
this is the path penalty, where we want to maintain the state deviation small, small
is possible throughout and we want to maintain the control as well, actually. The, we want
to use as much as less control as possible and in general this, X and del, X and U are
nothing, but delta X and delta U, remember that. When we, when you talk about linear
system, X is not really the true state, it is the deviation state and U is not really
the true control either, actually the deviation control.
So, what you want to minimize is deviation of the state should remain a close to 0 and
deviation of control should also remain close to 0, this is the whole problem actually.
There are certain necessities as well, and this has to be an ultimately a meaningful
performance index, has to be minimized. So, for that we need this S f and Q. These two
matrices has to be positive semi-definite and this R matrix has to be positive definite.
And not only that, there are, I mean, there are some one or two other conditions as well,
we will talk in a couple of few minutes later.
So, just remember, that S f and Q are positive semi-definite and R has to be positive definite,
so the, so that, I mean, how do you select this, this matrices now, actually. The question
is, we are talking about selection of a matrix because there are the turning parameters for
the LQR, I mean, I mean a design approach. So, how do you select this matrices, and one
of the, one of the way to select is just to select this as diagonal matrices and that
is what, most of the time it is used that way. So, and once you select the any, any
matrix as the, as diagonal matrix, the diagonal elements are nothing, but I mean, Eigen values.
So, if you just put positive elements in diagonal matrices, it will give you positive definite
matrix actually. So, that is the trick that people play and they put select diagonal matrices
with positive entries, actually. Now, even in positive entries how will you select entries
itself actually? So, that is what I am talking, this for example,
q, whatever q we are talking here, q is nothing, but something like a diagonal matrix with
q, with q i, everything else is 0. Now, how will you select this q i? So, the guideline
here is, recommended guideline turns out, that you expect a maximum deviation, maximum
expected or acceptable value of, this value of, this value actually 1 by x i square.
q i is equal to maximum expected value of 1 by x i square or similarly, r i is maximum
expected value or maximum acceptable value of 1 by u i square. In other words, if you
do that, remember these functions are nothing, but u 1 square r 1 times by, u 1 square r
1 plus u 2 times r 2 like that, actually. So, what it happens here is some sort of normalization
actually.
So, you are talking about something like u 1 square divided by u 1 max square plus u
2 square. This expression what we are looking at, this particular thing in that selection,
what I just talked to, this will turn out to be something like u 1 square divided by
u 1 max square plus u 2 square divided by u 2 max square, like that. So, that means,
this is some set of a normalization actually. Similarly, we are talking about doing similar
operation here, is similar operation here as well, actually. So, that is what I want
to tell actually by doing this.
There are some other kinds, I mean, some other facts to remember. I told you, that this pair
A, B, obviously needs to be controllable, the very, very idea of controllability is
always there. And if the pair A, B is not controllable, LQR control cannot be done.
In fact, any other control cannot be done either, actually. So, the very fundamental
requirement is A, B needs to be controllable and also it turns out, that A and square root
of Q also needs to be detected. So, these are all theoretical requirements,
as some of you want to know, you can pick up a LQR related book, linear quadratic books
are available and Anderson, Anderson and More is probably one of the standard books preferable,
actually. And see, some of the proofs are reasons for that, anything like that, that
way, anyway. So, this pair A, B needs to be controllable
and the pair A is n square root Q needs to be detectable and remember, as long as a matrix
is positive semi-definite, you can talk about square root of Q; this is computable actually.
Alright, so, this is the first thing and this we already talked and by default it is assumed,
that t f tends to infinity. In general, it can be finite time LQR, but, but, but if somebody
does not tells you, that is, it is a final time LQR is, simply tells this LQR problem,
then by default it is assumed, that t f goes to infinity actually. And also remember, that
constraint problems, that means, problems with state and control inequality constraints
are not considered here, we will consider them later, but not here right now. So, with
these things to, with these things in mind, let us proceed further actually.
So, what is our performance index? This is, this is going to be our performance index,
half of this plus integral of t naught to t f half of this quantity. So, sincerely,
in our generic framework this happens to be our phi and this happens to be our l actually.
So, path constrained is a linear equation X dot equal to X plus BU and boundary condition
is given by like this actually.
So, what is our terminal penalty? phi of X f is like this.
What is our Hamiltonian? Hamiltonian is nothing, but l plus lambda transpose f and remember,
this is, this is our f. This is our f of X, small u, here, big U, well, and I, ever noticed
any small thing is scalar and any big thing is a vector. So, just be careful about, that
we are talking about vector variables in general, actually.
So, this is our phi and this is our l and this is our f and these are the conditions
that are available to us actually. Alright, so this is phi, this is our Hamiltonian l,
l coming from here plus lambda transpose f, f is this X plus BU. So, this is what it is.
Once you know Hamiltonian and a phi, necessary conditions are all there with us.
The state equation is already there, X dot equal to X plus BU lambda dot equal to minus
del H by del X. So, it happens, T be minus of Q X plus A transpose lambda and an optimal
control equation is del H by del U equal to 0. So, that means, U equal to minus R inverse
B transpose lambda because you can, you can see this from here, let me do this.
Anyways, let me, so let me quickly do this. Hamiltonian is nothing, but well, this alright,
so Hamiltonian is nothing, but our the half of X transpose Q X plus U transpose RU l plus
lambda transpose f, f is nothing, but AX plus BU; again this problem. So, this cross-function,
sorry, this costate equation lambda dot is equal to minus del H by del X. So, if you
have this, this H coming here, this is minus of del H by del X, one term comes from here,
which is QX and other term from here, this is lambda transpose AX.
So, this lambda transpose AX, that I can talk about to derivative A transpose lambda basically.
And similarly, if I take del H by del U, del H by del U is nothing, but one term comes
from this half U transpose RU. So, that is, RU plus the other term is lambda transpose
BU. So, that means, if I take derivative, it becomes B transpose lambda. So, then, this
is to be equal to 0, that means, you can solve U equal to minus R inverse B transpose lambda.
So, that is how it comes, this optimal control equation and this is for your costate equation
comes actually. Alright, so this is what, what has been done here.
Alright, so X dot equal AX plus BU; lambda dot is equal to minus QX plus A transpose
lambda and this equation del H by del U equal to 0, gives us this U equal to minus R inverse
B transpose lambda, actually. The boundary conditions, this is, remember this is our
terminal penalty, del phi by, del phi by del X f will turn out to be S f into X f. So,
this is how it is. So, all this condition starting from here,
this condition has to be satisfied for our control solution actually.
Now, here is the trick in this, this particular problem. This lambda of t is, guessed is,
some time varying matrix P of t into X of t and there is, there are justifications for
that actually, one into two engineering justification comes from here, that lambda f is nothing,
but S f X f, that means, lambda f is a linear function of X f.
So, if, it final term t f, lambda f is a linear function of X f, then how about guessing,
that every point of time it is also a linear function of X, that the motivation comes from
there also. If you think a little bit, then it is LQR problems, such unique solutions.
And so, if you are able to assume some thing, like this lambda of t is equal to P t times
into X t, and finally get a solution, then that has to be the only solution, because,
because it has a, it has a uniqueness property actually. But even going a little more further,
it is, it has been formally proven from functional analysis theory of normed linear space and
all that, that lambda of t f in the dual space of X actually, and hence, it has to be the
linear functional of that. So, we would not talk about too much on that,
but somebody is interested, you can always see that classic book, very good book actually,
Optimizing by vector space methods. If you see that, that the some of the, I mean, I
mean proofs and all that will be available. But very engineering science, it, it happens
at the final time. So, we can assume, that it, it can happen in the, at every point of
time, from t naught to t f and because LQR admits the uniqueness of the theorem. So,
if, if at all we get a solution that has to be the only solution actually. The solution
form can differ, I mean, you can, somebody can write in different symbolic sense and
different math sense and all that. So, it is something like sine theta equal
to cosine of phi by 2 to minus theta. So, it, both are similar basically. You can, the
expression seems to, appears to be different, but the ultimately, they are same actually,
anyway. So, this is, this is, what it is. So, with that justification lambda is assumed
to be some time varying matrix P times X of t actually. So, this is what it is.
Then, with respect to that selection, let us analyze or let us try to use all these
equations, that we have each equally. So, now lambda of t is, P t, P t X t. So,
what is lambda dot? Lambda dot is P dot times X plus P times X dot and again, I emphasize,
do not exchange this, this, I mean, multiplication sequence; sequence is important here actually.
So, lambda dot is P dot time X plus P times X dot. So, let us keep this P dot time X,
we do not change it, thus keep it like that, but X dot we know, that it is nothing, but
state equations. So, we put AX plus BU, that is state equation
U, we just derive that. U is nothing, but R inverse B transpose lambda, so we will put
that, that here, R inverse B transpose lambda. And then we, lambda, again we know it is nothing,
but P times X. So, we will put P times X here. Now, what is lambda of t? What is lambda dot
of t? Lambda dot of t is nothing, but this costate equation minus Q X term, minus Q X
plus A transpose lambda. So, we will put that, minus Q X plus A transpose lambda and lambda
again, P times X that is put here actually. So, this side of the story is costate equation;
here we use state equation and here is optimal control equation actually. So, state equation
getting used here, costate equation getting used here, this side of the story and then,
optimal control equation is used here actually. So, all the three equations are embedded here
actually. Now, once you have it something like that,
it is all in the some equations like this, you can take everything into the left side
and make it something a big matrix times X equal to 0 and remember, X is not necessarily
equal to 0. Basically, X is, X is a trajectory, which evolves with time and X in general is
not, not 0, but this equation has to be true. So, in that sense and it has to be true for
all possible value of X, that is, that also requirement actually. It is valid for all
possible values of X, then the coefficient has to go to 0 and then, that is how will
get it, Riccati equation actually, is famously called as Riccati equation. Italian mathematician
who kind of developed this, I mean, came across this equation in a scalar variable sense for
the first time actually. So, that is how it is named as Riccati equation actually, anyway.
So, this is popularly known as differential Riccati equation and if you remember, this
is P dot here. So, obviously, we need a boundary condition for P also and this boundary condition
is obtained like this because we know lambda f equal to S f X f, alright.
This lambda f is nothing, but S f X f. But what is lambda f? Lambda is nothing, but P
of t f into X of t f from here. So, you put that P of t f into X of t f or X f equal to
S f X f. So, again X f is free, X f is not 0. So, hence, this two has to be equal actually.
So, we get a differential equation, ok, metric differential equation with corresponding boundary
condition at t f actually. So, now, what is the beauty? Here this problem
is, just see this equation and this boundary condition, they are independent of the problem
definition. So, we do not really bother about which initial condition the problem operates
and things like that. We simply start with this boundary condition and integrate this
differential equation backward and store this values of P of t from t naught to t f and
then start using, I mean, whenever the operation time starts at t naught.
So, we have available solution ready because P of t and X of t are available, your lambda
is available and hence, once lambda is available, your control is also available. So, that is
how the things proceed actually.
So, this is what is written here, you use the boundary conditions and integrate the
Riccati equations backwards, from t f to t naught, store the solution history of the
Riccati matrix from t naught to t f and compute the optimal control online, actually. When
online, you, to compute it, compute, compute it equal to minus R inverse B transpose lambda,
lambda is nothing, but P times X. So, this part of the things you can think
about something like a gain matrix, you can, then hence, you can write U equal to minus
K times X actually. So, that is how it is computed.
Now, the question is, do really need to do this? Because this is differential matrix
equations and all that, so may not be very good to do that always. So, is there a simplified
way of doing that? We do not have to keep on integrating, storing and things like that,
actually.
Now, it turns out, that this great theorem by Kalman again has simplified the quite a
lot and that is what is, that when t f goes to infinity and you have select a, if somebody
selects a constant Q and R matrix, they are time variable matrices, then P dot goes to
0 for all time, from beginning to end actually. That means, from P, P dot remains 0 throughout
actually, it does not change. Essentially, the P turns out to be a constant matrix actually.
If P is a constant matrix, then P dot is 0. Hence, if P dot is 0, suddenly the differential
equation turns out to be an algebraic equation actually. That is what is very popular in
everywhere, actually. So, that is the reason why, if nobody tells
t f, then by default we assume t f tends to infinity, actually. So, we can keep on using
this as long as we wish, anyway. So, t f goes to infinity, P dot goes to 0 and hence, this
differential Riccati equations turns out to be an algebraic Riccati equation, actually.
But remember, that ARE or algebraic Riccati equation is still a non-linear equation for
the Riccati matrix. And hence, it is not very straight forward to solve, it is not like,
by the way, somebody little bit clever can see, that if this non-linear term is not there,
this PA BR inverse B transpose P, that is not there, then this is nothing, but a equation,
linear equation, actually. So, that can be solved extremely easily.
But in general here, this is a non-linear equation and it is, it is not that straightforward
actually, but what happened is because this equation is so, I mean, it keeps on appearing
in a number of problems throughout, across the field, people have done lot of research
on that, how to, how to come up with efficient numerical methods, actually. So, it is all
available now and some of these has also gone into some routines of these control systems
toolbox actually. So, if you just use ARE, that is, algebraic
Riccati equation, there is a command called ARE. These, these are common for LQR, also
LQR, LQR 2, then discrete LQRD, LQR ARE is available for algebraic Riccati equation.
So, things like that. There are bunch of functions available in
metal also actually. So, ARE is still a non-linear equation for, for Riccati matrix, hence, but
the question, I mean, the point here is, it has drawn quite a bit of attention and hence
some good solution techniques are available for these actually. But also remember, that
a positive definite solution for Riccati matrix is needed to obtain a stabilizing controller.
So, you can, you can prove, that also, that with a positive definite P lot of good things
do happen actually. But it is, remember why, because these non-linear
equations, it can admin multiple solutions actually. So, you have to discard all other
solution and
…
this particular lecture, we will talk about linear quadratic regulator and then subsequent
two, three lectures also, we will keep on talking about these. There are several reasons
for that and as you see in the previous couple of lectures, these are computationally complex
problems in general, any optimal control formulation leads to these two point boundary value problems
and all that. So, that requires a lot of numerical intensive
procedure and all. So, then the, then at least in 50’s and 60’s people started thinking,
that is there any class of problems for which we would not really need all those kind of
numerical techniques? And we can get solutions in, in the other computational efficient way,
even though it is limited to a class of problems actually. So, that leads to this, this definition
of linear quadratic regulator problems and extensively studied and extensively utilized
as well. It is one of the very popular tricks, tricks of optimal controller or class of optimal
control problem that is found in usage in industries as well, actually.
So, we will see that and then proceed further, some, some of this extension proofs and things
like that in the subsequent lectures actually. So, let us see, understand, what is this LQR
problem and then, furthermore will be subsequent lectures, anyway.
So, generic optimal control problem, as I discussed so many times before, this is what
it is, J, J takes the form of some phi, this is terminal penalty plus some path penalty
sort of thing. And then, along with that, that, there is a path constraint, which is
a system dynamic equation, as well as boundary conditions, actually.
And we will consider, typically this X t f is free, X of t a, but some problems, it can
be and also basically, actually, alright, anyway. So, this, but this particular LQR
problem does not talk about generic formulation any more. Thus, it talks about a specific
form of cross function and a specific form of system analysis.
So, the, this problem can be defined this way. The objective is to drive the state of
a linear or many times, linear is nothing, but a linearized system, anyway we know that.
So, the objective is to drive the state X of a linear system, given by this equation,
to the origin, that means, X of T f has to be 0 0, by, so how do you, so that it can
be done by minimizing the following quadratic performance index actually.
So, this, the, this is, this is the terminal, terminal penalty X f transpose S f X f and
this is the path penalty, where we want to maintain the state deviation small, small
is possible throughout and we want to maintain the control as well, actually. The, we want
to use as much as less control as possible and in general this, X and del, X and U are
nothing, but delta X and delta U, remember that. When we, when you talk about linear
system, X is not really the true state, it is the deviation state and U is not really
the true control either, actually the deviation control.
So, what you want to minimize is deviation of the state should remain a close to 0 and
deviation of control should also remain close to 0, this is the whole problem actually.
There are certain necessities as well, and this has to be an ultimately a meaningful
performance index, has to be minimized. So, for that we need this S f and Q. These two
matrices has to be positive semi-definite and this R matrix has to be positive definite.
And not only that, there are, I mean, there are some one or two other conditions as well,
we will talk in a couple of few minutes later.
So, just remember, that S f and Q are positive semi-definite and R has to be positive definite,
so the, so that, I mean, how do you select this, this matrices now, actually. The question
is, we are talking about selection of a matrix because there are the turning parameters for
the LQR, I mean, I mean a design approach. So, how do you select this matrices, and one
of the, one of the way to select is just to select this as diagonal matrices and that
is what, most of the time it is used that way. So, and once you select the any, any
matrix as the, as diagonal matrix, the diagonal elements are nothing, but I mean, Eigen values.
So, if you just put positive elements in diagonal matrices, it will give you positive definite
matrix actually. So, that is the trick that people play and they put select diagonal matrices
with positive entries, actually. Now, even in positive entries how will you select entries
itself actually? So, that is what I am talking, this for example,
q, whatever q we are talking here, q is nothing, but something like a diagonal matrix with
q, with q i, everything else is 0. Now, how will you select this q i? So, the guideline
here is, recommended guideline turns out, that you expect a maximum deviation, maximum
expected or acceptable value of, this value of, this value actually 1 by x i square.
q i is equal to maximum expected value of 1 by x i square or similarly, r i is maximum
expected value or maximum acceptable value of 1 by u i square. In other words, if you
do that, remember these functions are nothing, but u 1 square r 1 times by, u 1 square r
1 plus u 2 times r 2 like that, actually. So, what it happens here is some sort of normalization
actually.
So, you are talking about something like u 1 square divided by u 1 max square plus u
2 square. This expression what we are looking at, this particular thing in that selection,
what I just talked to, this will turn out to be something like u 1 square divided by
u 1 max square plus u 2 square divided by u 2 max square, like that. So, that means,
this is some set of a normalization actually. Similarly, we are talking about doing similar
operation here, is similar operation here as well, actually. So, that is what I want
to tell actually by doing this.
There are some other kinds, I mean, some other facts to remember. I told you, that this pair
A, B, obviously needs to be controllable, the very, very idea of controllability is
always there. And if the pair A, B is not controllable, LQR control cannot be done.
In fact, any other control cannot be done either, actually. So, the very fundamental
requirement is A, B needs to be controllable and also it turns out, that A and square root
of Q also needs to be detected. So, these are all theoretical requirements,
as some of you want to know, you can pick up a LQR related book, linear quadratic books
are available and Anderson, Anderson and More is probably one of the standard books preferable,
actually. And see, some of the proofs are reasons for that, anything like that, that
way, anyway. So, this pair A, B needs to be controllable
and the pair A is n square root Q needs to be detectable and remember, as long as a matrix
is positive semi-definite, you can talk about square root of Q; this is computable actually.
Alright, so, this is the first thing and this we already talked and by default it is assumed,
that t f tends to infinity. In general, it can be finite time LQR, but, but, but if somebody
does not tells you, that is, it is a final time LQR is, simply tells this LQR problem,
then by default it is assumed, that t f goes to infinity actually. And also remember, that
constraint problems, that means, problems with state and control inequality constraints
are not considered here, we will consider them later, but not here right now. So, with
these things to, with these things in mind, let us proceed further actually.
So, what is our performance index? This is, this is going to be our performance index,
half of this plus integral of t naught to t f half of this quantity. So, sincerely,
in our generic framework this happens to be our phi and this happens to be our l actually.
So, path constrained is a linear equation X dot equal to X plus BU and boundary condition
is given by like this actually.
So, what is our terminal penalty? phi of X f is like this.
What is our Hamiltonian? Hamiltonian is nothing, but l plus lambda transpose f and remember,
this is, this is our f. This is our f of X, small u, here, big U, well, and I, ever noticed
any small thing is scalar and any big thing is a vector. So, just be careful about, that
we are talking about vector variables in general, actually.
So, this is our phi and this is our l and this is our f and these are the conditions
that are available to us actually. Alright, so this is phi, this is our Hamiltonian l,
l coming from here plus lambda transpose f, f is this X plus BU. So, this is what it is.
Once you know Hamiltonian and a phi, necessary conditions are all there with us.
The state equation is already there, X dot equal to X plus BU lambda dot equal to minus
del H by del X. So, it happens, T be minus of Q X plus A transpose lambda and an optimal
control equation is del H by del U equal to 0. So, that means, U equal to minus R inverse
B transpose lambda because you can, you can see this from here, let me do this.
Anyways, let me, so let me quickly do this. Hamiltonian is nothing, but well, this alright,
so Hamiltonian is nothing, but our the half of X transpose Q X plus U transpose RU l plus
lambda transpose f, f is nothing, but AX plus BU; again this problem. So, this cross-function,
sorry, this costate equation lambda dot is equal to minus del H by del X. So, if you
have this, this H coming here, this is minus of del H by del X, one term comes from here,
which is QX and other term from here, this is lambda transpose AX.
So, this lambda transpose AX, that I can talk about to derivative A transpose lambda basically.
And similarly, if I take del H by del U, del H by del U is nothing, but one term comes
from this half U transpose RU. So, that is, RU plus the other term is lambda transpose
BU. So, that means, if I take derivative, it becomes B transpose lambda. So, then, this
is to be equal to 0, that means, you can solve U equal to minus R inverse B transpose lambda.
So, that is how it comes, this optimal control equation and this is for your costate equation
comes actually. Alright, so this is what, what has been done here.
Alright, so X dot equal AX plus BU; lambda dot is equal to minus QX plus A transpose
lambda and this equation del H by del U equal to 0, gives us this U equal to minus R inverse
B transpose lambda, actually. The boundary conditions, this is, remember this is our
terminal penalty, del phi by, del phi by del X f will turn out to be S f into X f. So,
this is how it is. So, all this condition starting from here,
this condition has to be satisfied for our control solution actually.
Now, here is the trick in this, this particular problem. This lambda of t is, guessed is,
some time varying matrix P of t into X of t and there is, there are justifications for
that actually, one into two engineering justification comes from here, that lambda f is nothing,
but S f X f, that means, lambda f is a linear function of X f.
So, if, it final term t f, lambda f is a linear function of X f, then how about guessing,
that every point of time it is also a linear function of X, that the motivation comes from
there also. If you think a little bit, then it is LQR problems, such unique solutions.
And so, if you are able to assume some thing, like this lambda of t is equal to P t times
into X t, and finally get a solution, then that has to be the only solution, because,
because it has a, it has a uniqueness property actually. But even going a little more further,
it is, it has been formally proven from functional analysis theory of normed linear space and
all that, that lambda of t f in the dual space of X actually, and hence, it has to be the
linear functional of that. So, we would not talk about too much on that,
but somebody is interested, you can always see that classic book, very good book actually,
Optimizing by vector space methods. If you see that, that the some of the, I mean, I
mean proofs and all that will be available. But very engineering science, it, it happens
at the final time. So, we can assume, that it, it can happen in the, at every point of
time, from t naught to t f and because LQR admits the uniqueness of the theorem. So,
if, if at all we get a solution that has to be the only solution actually. The solution
form can differ, I mean, you can, somebody can write in different symbolic sense and
different math sense and all that. So, it is something like sine theta equal
to cosine of phi by 2 to minus theta. So, it, both are similar basically. You can, the
expression seems to, appears to be different, but the ultimately, they are same actually,
anyway. So, this is, this is, what it is. So, with that justification lambda is assumed
to be some time varying matrix P times X of t actually. So, this is what it is.
Then, with respect to that selection, let us analyze or let us try to use all these
equations, that we have each equally. So, now lambda of t is, P t, P t X t. So,
what is lambda dot? Lambda dot is P dot times X plus P times X dot and again, I emphasize,
do not exchange this, this, I mean, multiplication sequence; sequence is important here actually.
So, lambda dot is P dot time X plus P times X dot. So, let us keep this P dot time X,
we do not change it, thus keep it like that, but X dot we know, that it is nothing, but
state equations. So, we put AX plus BU, that is state equation
U, we just derive that. U is nothing, but R inverse B transpose lambda, so we will put
that, that here, R inverse B transpose lambda. And then we, lambda, again we know it is nothing,
but P times X. So, we will put P times X here. Now, what is lambda of t? What is lambda dot
of t? Lambda dot of t is nothing, but this costate equation minus Q X term, minus Q X
plus A transpose lambda. So, we will put that, minus Q X plus A transpose lambda and lambda
again, P times X that is put here actually. So, this side of the story is costate equation;
here we use state equation and here is optimal control equation actually. So, state equation
getting used here, costate equation getting used here, this side of the story and then,
optimal control equation is used here actually. So, all the three equations are embedded here
actually. Now, once you have it something like that,
it is all in the some equations like this, you can take everything into the left side
and make it something a big matrix times X equal to 0 and remember, X is not necessarily
equal to 0. Basically, X is, X is a trajectory, which evolves with time and X in general is
not, not 0, but this equation has to be true. So, in that sense and it has to be true for
all possible value of X, that is, that also requirement actually. It is valid for all
possible values of X, then the coefficient has to go to 0 and then, that is how will
get it, Riccati equation actually, is famously called as Riccati equation. Italian mathematician
who kind of developed this, I mean, came across this equation in a scalar variable sense for
the first time actually. So, that is how it is named as Riccati equation actually, anyway.
So, this is popularly known as differential Riccati equation and if you remember, this
is P dot here. So, obviously, we need a boundary condition for P also and this boundary condition
is obtained like this because we know lambda f equal to S f X f, alright.
This lambda f is nothing, but S f X f. But what is lambda f? Lambda is nothing, but P
of t f into X of t f from here. So, you put that P of t f into X of t f or X f equal to
S f X f. So, again X f is free, X f is not 0. So, hence, this two has to be equal actually.
So, we get a differential equation, ok, metric differential equation with corresponding boundary
condition at t f actually. So, now, what is the beauty? Here this problem
is, just see this equation and this boundary condition, they are independent of the problem
definition. So, we do not really bother about which initial condition the problem operates
and things like that. We simply start with this boundary condition and integrate this
differential equation backward and store this values of P of t from t naught to t f and
then start using, I mean, whenever the operation time starts at t naught.
So, we have available solution ready because P of t and X of t are available, your lambda
is available and hence, once lambda is available, your control is also available. So, that is
how the things proceed actually.
So, this is what is written here, you use the boundary conditions and integrate the
Riccati equations backwards, from t f to t naught, store the solution history of the
Riccati matrix from t naught to t f and compute the optimal control online, actually. When
online, you, to compute it, compute, compute it equal to minus R inverse B transpose lambda,
lambda is nothing, but P times X. So, this part of the things you can think
about something like a gain matrix, you can, then hence, you can write U equal to minus
K times X actually. So, that is how it is computed.
Now, the question is, do really need to do this? Because this is differential matrix
equations and all that, so may not be very good to do that always. So, is there a simplified
way of doing that? We do not have to keep on integrating, storing and things like that,
actually.
Now, it turns out, that this great theorem by Kalman again has simplified the quite a
lot and that is what is, that when t f goes to infinity and you have select a, if somebody
selects a constant Q and R matrix, they are time variable matrices, then P dot goes to
0 for all time, from beginning to end actually. That means, from P, P dot remains 0 throughout
actually, it does not change. Essentially, the P turns out to be a constant matrix actually.
If P is a constant matrix, then P dot is 0. Hence, if P dot is 0, suddenly the differential
equation turns out to be an algebraic equation actually. That is what is very popular in
everywhere, actually. So, that is the reason why, if nobody tells
t f, then by default we assume t f tends to infinity, actually. So, we can keep on using
this as long as we wish, anyway. So, t f goes to infinity, P dot goes to 0 and hence, this
differential Riccati equations turns out to be an algebraic Riccati equation, actually.
But remember, that ARE or algebraic Riccati equation is still a non-linear equation for
the Riccati matrix. And hence, it is not very straight forward to solve, it is not like,
by the way, somebody little bit clever can see, that if this non-linear term is not there,
this PA BR inverse B transpose P, that is not there, then this is nothing, but a equation,
linear equation, actually. So, that can be solved extremely easily.
But in general here, this is a non-linear equation and it is, it is not that straightforward
actually, but what happened is because this equation is so, I mean, it keeps on appearing
in a number of problems throughout, across the field, people have done lot of research
on that, how to, how to come up with efficient numerical methods, actually. So, it is all
available now and some of these has also gone into some routines of these control systems
toolbox actually. So, if you just use ARE, that is, algebraic
Riccati equation, there is a command called ARE. These, these are common for LQR, also
LQR, LQR 2, then discrete LQRD, LQR ARE is available for algebraic Riccati equation.
So, things like that. There are bunch of functions available in
metal also actually. So, ARE is still a non-linear equation for, for Riccati matrix, hence, but
the question, I mean, the point here is, it has drawn quite a bit of attention and hence
some good solution techniques are available for these actually. But also remember, that
a positive definite solution for Riccati matrix is needed to obtain a stabilizing controller.
So, you can, you can prove, that also, that with a positive definite P lot of good things
do happen actually. But it is, remember why, because these non-linear
equations, it can admin multiple solutions actually. So, you have to discard all other
solution and
…