Subject: neural networks problem

From: Adina Milac <[email-masked]>
Date: Thu, 4 Apr 2002 13:40:48 -0200 (GMT+2)

We are trying to develop a neural network that would be able to predict
the inhibition of some chemical compounds based on some physico-chemical
properties of these compounds.  From the mathematical point of view, the
problem is to associate training vectors with 4 fields with a scalar.

The problem we face is the following:   after we have tried different
variants of training sets, we have found that if "extreme" vectors
(vectors in which one of the fields has either maximum or minimum
values) are excluded from the training set, the prediction for that vector
is extremely poor.  In other words, the network is not able to extrapolate
the data it learns.

In my opinion, this could be due to one (or more) of the following
reasons (from least to most probable):

1.  network architecture is not appropriate for the task (although I have
tried many variants of network dimensions and transfer functions)

2.  the network is unable to determine a function based on which it could
make the extrapolation

3.  the physico-chemical properties we have used for building input
vectors are not adequate for predicting inhibition

What could be, in your opinion, the cause for this failure and what do you
suggest me to do?

Hoping that you will find the time to answer my message, I thank you and I
wish you all the best,

ADINA MILAC

Ph.D Student, Research Assistant
Institute of Biochemistry, Romanian Academy
Splaiul Independentei 296, 77700 Bucharest 17, Romania 
Phone: (+401).223.90.69; FAX: (+401).223.90.68
e-mail: [email-masked]

---

From: Szilveszter Juhos <[email-masked]>
Date: Thu, 4 Apr 2002 16:17:26 +0000 (GMT)

> reasons (from least to most probable):
[..]

Unfortunatelly neural nets are not that easy as some people think. In
most of the cases the problem is what you mentioned: if you want to
extrapolate to a region of parameter space that is not represeneted by
the training data than you can have incorrect predictions. If I can
recommend one book is that one that was written by C. Bishop (
http://www.ncrg.aston.ac.uk/People/bishopc/Welcome.html ) where most of
these problems are discussed. Also there are a loads of good papers on
these pages. 

My first run would be to play with the properties (3rd problem) checking
the weights. If one of the properties have a very small weight than it is
unlikely to affect the overall prediction so it can be replaced by an
other property (that can improve prediction if we are lucky). Still likely
you have to include extreme values into the training set.

Szilva

---

From: "Vijay Pargaonkar" <[email-masked]>
Date: Thu, 4 Apr 2002 19:34:22 +0530

Dear Adina Milac,
                 Your observations are right. I have also experienced the
same fact about different data sets.   As the degree of extrapolation
increases
the prediction capabilities of neural netwroks decreases. Neural Networks
are
 efficient interpolators. They are not meant for extrapolations.
Hence, Maxima and minima play crucial role in training set.

               There are reasons for poorer performance. I have a lot of
experience on Neural Networks. However, Accurate information about your
Neural
Network will enable me in explaining you how to tackle this situation.

Thanks.

Pargaonkar Vijay

----- Original Message -----
From: "Adina Milac" <[email-masked]>
To: "Computational Chemistry List" <[email-masked]>
Sent: Thursday, April 04, 2002 9:10 PM
Subject: CCL:neural networks problem

> -= This is automatically added to each message by mailing script =-
> [email-masked] -- To Everybody  | [email-masked] -- To
Admins
> [email-masked] -- HELP CHEMISTRY or HELP SEARCH
> [email-masked] -- archive search    |    Gopher: gopher.ccl.net
70
> Ftp: ftp.ccl.net  |  WWW: http://www.ccl.net/chemistry/   | Jan:
[email-masked]

*********************************************************
Disclaimer

This message (including any attachments) contains 
confidential information intended for a specific 
individual and purpose, and is protected by law. 
If you are not the intended recipient, you should 
delete this message and are hereby notified that 
any disclosure, copying, or distribution of this
message, or the taking of any action based on it, 
is strictly prohibited.

*********************************************************
Visit us at http://www.mahindrabt.com