Subject: Re: CCL:constrained multiple linear regression

From: Geoff Hutchison <[email-masked]>
Date: Mon, 2 Dec 2002 22:49:44 -0600

On Monday, December 2, 2002, at 07:08  PM, [email-masked] wrote:

>         I am looking for software (preferably free or inexpensive, code 
> is OK) that will allow me to perform
> constrained multiple linear regression.  For example, I have an 
> equation of the form:
>         Y = a*X1 + b*X2 + c*X3 + d

I'm not certain if constrained regression is part of the base package, 
but I'd assume that the R statistical package can do this (among many, 
many other types of regression and other things). R is free and has been 
able to handle lots of thorny statistical problems I've thrown at it.

http://www.r-project.org/

>         I must be able to constrain the range of values for a, b, c, 
> and d during the fitting.

You might also try contacting the R project mailing list:
[email-masked]

--
-Geoff Hutchison	<[email-masked]>
Ratner/Marks Groups	(847) 491-3295
Northwestern Chemistry	http://www.chem.northwestern.edu/

---

From: "William R. Smith" <[email-masked]>
Date: Tue, 3 Dec 2002 07:13:58 -0500

If you get stuck and can't find a stats package to do this, you are looking at 
a "nonlinear optimization problem" in general.  Regression can be viewed as a special 
case of nonlinear optimization, and you should be able to find something in the 
literature of that field.

If you use a least squares criterion, your problem is a "quadratic programming problem", 
and algorithms exist to solve it.  

For constrained problems like yours, a trick that sometimes works is to incorporating the 
constraints into a new expression, resulting in an unconstrained optimization problem 
(which many packages can handle).  For your example, introduce the new (nonlinear) 
variable a1 and set

a = a0 + (\epsilon_a)*sin(a1)

where a0 is the mid-point of the constrained interval for a and \epsilon_a is its half-
width.  a1 is then the new nonlinear parameter.  

Quoting [email-masked]:

> CCL,
> 
>         I am looking for software (preferably free or inexpensive, code is 
> OK) that will allow me to perform
> constrained multiple linear regression.  For example, I have an equation 
> of the form:
> 
>         where: Y is the dependent variable
>                   X1 - X3 are independent variables (more than 1)
>                 a, b, c are coefficients
>                         d is a constant
> 
>         I must be able to constrain the range of values for a, b, c, and d 
> during the fitting.
> 
>         Please note that programs such as JMP, KalediaGraph, GraphPad 
> Prism, and several other packages do 
> NOT allow the user to do this, especially for more than one independent 
> variable.
> 
>         Can someone suggest a program or code to do this?
> 
>         Thank you,
>         Jim Metz
> 
> 
> James T. Metz, Ph.D.
> Research Investigator Chemist
> 
> GPRD R46Y AP10-2
> Abbott Laboratories
> 100 Abbott Park Road
> Abbott Park, IL  60064-6100
> U.S.A.
> 
> Office (847) 936 - 0441
> FAX    (847) 935 - 0548
> 
> [email-masked]
> 

-- 
Best Regards,

W. R. Smith, Professor
Dept. of Mathematics and Statistics, and School of Engineering
Room 546 MacNaughton Building
University of Guelph
Stone Road and Gordon Streets
Guelph, Ontario, CANADA N1G 2W1
Tel: 519-824-4120, ext. 3038; FAX: 519-837-0221; 
http://www.mathstat.uoguelph.ca/faculty/smith/

---

From: Anan Wu <[email-masked]>
Date: Tue, 3 Dec 2002 15:37:47 +0100

If you can get access to Matlab, you will find a function called 
"fmincon". It fits into your problem very well. 

Anan Wu