"Paul Kerr-Delworth" wrote in message <k5ojec$sud$1@newscl01ah.mathworks.com>...
> "mave" wrote in message <k5mu84$s40$1@newscl01ah.mathworks.com>...
> > Alan_Weiss <aweiss@mathworks.com> wrote in message <k5mofq$525$1@newscl01ah.mathworks.com>...
> > > On 10/17/2012 12:10 PM, mave wrote:
> > > > Hello
> > > >
> > > > I am trying to use fmincon to solve an engineering problem, where the
> > > > optimization variables are not explicitly written in any of the
> > > > constraints nor the objective function. The problem is working well
> > > > because I can introduce the optimization results into a simulator and
> > > > double check that the system can actually operate in reality. Now, the
> > > > problem that I am having is that before using fmincon to solve a part
> > > > of the problem I was using lsqnonlin. lsqnonlin was working well, but
> > > > with there was no way to guarantee that the residual is less than a
> > > > required tolerance, so it will need several iterations before it
> > > > converged to the value I needed. To speed the problem up I decided to
> > > > switch to fmincon and use the same objective function that lsqnonlin
> > > > uses, but twitching it a little to make it converge to the tolerance
> > > > is less steps.
> > > >
> > > > All the changes that I was introducing to my code were working well,
> > > > until today. The last change (that is very similar to another one that
> > > > I had previously introduced) gave me this answer:
> > > >
> > > > Initial point is a local minimum that satisfies the constraints.
> > > >
> > > > Optimization completed because at the initial point, the objective
> > > > function is non-decreasing in feasible directions to within the
> > > > selected value of the function tolerance, and constraints are
> > > > satisfied to within the selected value of the constraint tolerance.
> > > >
> > > > Stopping criteria details:
> > > >
> > > > Optimization completed: The final point is the initial point.
> > > > The first-order optimality measure, 4.110543e-23, is less than
> > > > options.TolFun = 1.000000e+00, and the maximum constraint
> > > > violation, 1.000000e+00, is less than options.TolCon = 1.000000e+00.
> > > >
> > > > Optimization Metric Options
> > > > first-order optimality = 4.11e-23 TolFun = 1e+00
> > > > (selected)
> > > > max(constraint violation) = 1.00e+00 TolCon = 1e+00
> > > > (selected)
> > > >
> > > >
> > > > Which is OK, it seems as if my optimization if working well. Do not
> > > > worry about the fact that it finishes at the initial point, that is
> > > > because I am trying to solve it for the initial conditions of my
> > > > system, which are the same ones that I use for the initial guess.
> > > > However, if I read the maximum constraint violation from the output
> > > > structure (in my case I called it outp) of fmincon I get this:
> > > >
> > > > outp.constrviolation
> > > >
> > > > ans =
> > > >
> > > > 78.222
> > > >
> > > > Which is definitely larger than 1.00e+00. So there is a little problem
> > > > here: either I am not understanding the concept of the TolCon value or
> > > > the value of the maximum constraint violation of the output structure
> > > > and the message are not the same, and thus it's giving me the wrong
> > > > answer.
> > > >
> > > > I had bumped into a similar problem before when using fmincon but
> > > > always found another way out of it, but this time I really have no
> > > > option. Can somebody please help me with this? I would really
> > > > appreciate it!
> > > >
> > > > Thanks!
> > > >
> > > > Maria
> > >
> > > I believe the problem is that fmincon uses RELATIVE constraint
> > > tolerances, meaning the constraint value at the current point x DIVIDED
> > > BY the value at the initial point. You set TolCon to 1.0, so the
> > > relative constraint tolerance is always satisfied at the initial point.
> > >
> > > Solution: choose a smaller constraint tolerance, perhaps 1e-2.
> > >
> > > Alan Weiss
> > > MATLAB mathematical toolbox documentation
> >
> > Thanks for your reply Alan
> >
> > I will try that, but I still have one doubt. Normally, if the tolerance is relative, then the output will say so. Something like this:
> >
> > Optimization Metric Options
> > first-order optimality = 4.11e-23 TolFun = 1e+00 (selected)
> > relative max(constraint violation) = 1.00e+00 TolCon = 1e+00 (selected)
> >
> > And that relative word is nowhere to be found in this case. I have looked a lot at what TolCon refers to: if it is either absolute or relative; and the answers I have found are that is absolute (http://www.mathworks.co.uk/support/solutions/en/data/1-17SBI/index.html). So, how can it be relative and absolute at the same time?
> >
> > Maria
>
> Hi Maria,
>
> From what you have written, I think you are running the 'active-set' algorithm in fmincon. I believe that this algorithm uses TolCon as an absolute tolerance to test whether a point is feasible. Consequently when you run the problem you do not see the word "relative" in the exit message, e.g.
>
> >> fmincon(@(x)peaks(x(1), x(2)), [1 3], [], [], [], [], [-3 -3], [3 3])
>
> [snipped output]
>
> Optimization Metric Options
> abs(steplength*directional derivative) = 2.31e-07 TolFun = 1e-06 (default)
> max(constraint violation) = 0.00e+00 TolCon = 1e-06 (default)
>
> Now if you switch to use the 'sqp' algorithm in fmincon, this uses TolCon as a relative tolerance to test whether a point is feasible. As you can see now, you do see the word "relative" in the exit message, e.g.
>
> >> opts = optimset('Algorithm', 'sqp')
> >> fmincon(@(x)peaks(x(1), x(2)), [1 3], [], [], [], [], [-3 -3], [3 3], [], opts)
>
> [snipped output]
>
> Optimization Metric Options
> relative first-order optimality = 0.00e+00 TolFun = 1e-06 (default)
> relative max(constraint violation) = 0.00e+00 TolCon = 1e-06 (default)
>
> As you can see, individual algorithms within a solver can implement tolerances in different ways. However, as you can verify iin the above exit messages, you can always check that the metrics being used by the algorithm do meet the tolerances.
>
> Hope this helps.
>
> Best regards,
>
> Paul
Paul, thanks for your help. That definitely clears out some doubts about the tolerances. Actually, I am using the 'interior-point' algorithm as it is better for my problem than 'sqp'; but my guess is that TolCon works the same way for the 'interior-point' and the 'active-set' algorithms.
However, I am still puzzled with the reason of why it converges even though the tolerance is not met by the constraints and the TolCon value is supposed to be absolute for the 'interior-point' algorithm. Do you also suggest, as Alan did, to decrease the TolCon value? Because I am still having second thoughts about this option.
Thanks again,
Regards,
Maria![]()
> "mave" wrote in message <k5mu84$s40$1@newscl01ah.mathworks.com>...
> > Alan_Weiss <aweiss@mathworks.com> wrote in message <k5mofq$525$1@newscl01ah.mathworks.com>...
> > > On 10/17/2012 12:10 PM, mave wrote:
> > > > Hello
> > > >
> > > > I am trying to use fmincon to solve an engineering problem, where the
> > > > optimization variables are not explicitly written in any of the
> > > > constraints nor the objective function. The problem is working well
> > > > because I can introduce the optimization results into a simulator and
> > > > double check that the system can actually operate in reality. Now, the
> > > > problem that I am having is that before using fmincon to solve a part
> > > > of the problem I was using lsqnonlin. lsqnonlin was working well, but
> > > > with there was no way to guarantee that the residual is less than a
> > > > required tolerance, so it will need several iterations before it
> > > > converged to the value I needed. To speed the problem up I decided to
> > > > switch to fmincon and use the same objective function that lsqnonlin
> > > > uses, but twitching it a little to make it converge to the tolerance
> > > > is less steps.
> > > >
> > > > All the changes that I was introducing to my code were working well,
> > > > until today. The last change (that is very similar to another one that
> > > > I had previously introduced) gave me this answer:
> > > >
> > > > Initial point is a local minimum that satisfies the constraints.
> > > >
> > > > Optimization completed because at the initial point, the objective
> > > > function is non-decreasing in feasible directions to within the
> > > > selected value of the function tolerance, and constraints are
> > > > satisfied to within the selected value of the constraint tolerance.
> > > >
> > > > Stopping criteria details:
> > > >
> > > > Optimization completed: The final point is the initial point.
> > > > The first-order optimality measure, 4.110543e-23, is less than
> > > > options.TolFun = 1.000000e+00, and the maximum constraint
> > > > violation, 1.000000e+00, is less than options.TolCon = 1.000000e+00.
> > > >
> > > > Optimization Metric Options
> > > > first-order optimality = 4.11e-23 TolFun = 1e+00
> > > > (selected)
> > > > max(constraint violation) = 1.00e+00 TolCon = 1e+00
> > > > (selected)
> > > >
> > > >
> > > > Which is OK, it seems as if my optimization if working well. Do not
> > > > worry about the fact that it finishes at the initial point, that is
> > > > because I am trying to solve it for the initial conditions of my
> > > > system, which are the same ones that I use for the initial guess.
> > > > However, if I read the maximum constraint violation from the output
> > > > structure (in my case I called it outp) of fmincon I get this:
> > > >
> > > > outp.constrviolation
> > > >
> > > > ans =
> > > >
> > > > 78.222
> > > >
> > > > Which is definitely larger than 1.00e+00. So there is a little problem
> > > > here: either I am not understanding the concept of the TolCon value or
> > > > the value of the maximum constraint violation of the output structure
> > > > and the message are not the same, and thus it's giving me the wrong
> > > > answer.
> > > >
> > > > I had bumped into a similar problem before when using fmincon but
> > > > always found another way out of it, but this time I really have no
> > > > option. Can somebody please help me with this? I would really
> > > > appreciate it!
> > > >
> > > > Thanks!
> > > >
> > > > Maria
> > >
> > > I believe the problem is that fmincon uses RELATIVE constraint
> > > tolerances, meaning the constraint value at the current point x DIVIDED
> > > BY the value at the initial point. You set TolCon to 1.0, so the
> > > relative constraint tolerance is always satisfied at the initial point.
> > >
> > > Solution: choose a smaller constraint tolerance, perhaps 1e-2.
> > >
> > > Alan Weiss
> > > MATLAB mathematical toolbox documentation
> >
> > Thanks for your reply Alan
> >
> > I will try that, but I still have one doubt. Normally, if the tolerance is relative, then the output will say so. Something like this:
> >
> > Optimization Metric Options
> > first-order optimality = 4.11e-23 TolFun = 1e+00 (selected)
> > relative max(constraint violation) = 1.00e+00 TolCon = 1e+00 (selected)
> >
> > And that relative word is nowhere to be found in this case. I have looked a lot at what TolCon refers to: if it is either absolute or relative; and the answers I have found are that is absolute (http://www.mathworks.co.uk/support/solutions/en/data/1-17SBI/index.html). So, how can it be relative and absolute at the same time?
> >
> > Maria
>
> Hi Maria,
>
> From what you have written, I think you are running the 'active-set' algorithm in fmincon. I believe that this algorithm uses TolCon as an absolute tolerance to test whether a point is feasible. Consequently when you run the problem you do not see the word "relative" in the exit message, e.g.
>
> >> fmincon(@(x)peaks(x(1), x(2)), [1 3], [], [], [], [], [-3 -3], [3 3])
>
> [snipped output]
>
> Optimization Metric Options
> abs(steplength*directional derivative) = 2.31e-07 TolFun = 1e-06 (default)
> max(constraint violation) = 0.00e+00 TolCon = 1e-06 (default)
>
> Now if you switch to use the 'sqp' algorithm in fmincon, this uses TolCon as a relative tolerance to test whether a point is feasible. As you can see now, you do see the word "relative" in the exit message, e.g.
>
> >> opts = optimset('Algorithm', 'sqp')
> >> fmincon(@(x)peaks(x(1), x(2)), [1 3], [], [], [], [], [-3 -3], [3 3], [], opts)
>
> [snipped output]
>
> Optimization Metric Options
> relative first-order optimality = 0.00e+00 TolFun = 1e-06 (default)
> relative max(constraint violation) = 0.00e+00 TolCon = 1e-06 (default)
>
> As you can see, individual algorithms within a solver can implement tolerances in different ways. However, as you can verify iin the above exit messages, you can always check that the metrics being used by the algorithm do meet the tolerances.
>
> Hope this helps.
>
> Best regards,
>
> Paul
Paul, thanks for your help. That definitely clears out some doubts about the tolerances. Actually, I am using the 'interior-point' algorithm as it is better for my problem than 'sqp'; but my guess is that TolCon works the same way for the 'interior-point' and the 'active-set' algorithms.
However, I am still puzzled with the reason of why it converges even though the tolerance is not met by the constraints and the TolCon value is supposed to be absolute for the 'interior-point' algorithm. Do you also suggest, as Alan did, to decrease the TolCon value? Because I am still having second thoughts about this option.
Thanks again,
Regards,
Maria