Subject: Question about neural networks

From: [email-masked] (Ed Jaeger)
Date: Wed, 24 Nov 93 09:17:13 EST

Peter,

I posted a note to a stat mailing list see what answers they might be
able to provide.  The response was not overwhelming.  I provide my
mail and the two responses below.  I don't believe that they will be
much help.  It might be an area where bootstrapping/crossvalidation
would applicable assuming the cpu costs were not too overwhelming.

Ed Jaeger <[email-masked]> wrote:
   A colleague is using neural nets to build a nonlinear model of the
   relationship between chemical structure and toxicity.  He is
   looking for diagnostic methods to seek "outliers" in his training
   set.

   Can anyone suggest such diagnostic tools or point me to where I
   might find this information?

[email-masked] wrote:
   Hi Ed,

   I'm not too deep into diagnostics yet, but the two standard
   references,

   CHATTERJEE,S.,HADI,A.S.: Sensitivity Analysis in Linear Regression.
	Wiley 1988.
   BELSLEY,D.A.,KUH,E.,HELSCH,R.E.: Regression Diagnostics:
	Identifying Influencial Data And Sources of Collinearity.
	Wiley 1980.

   define outliers as observations with very high some kind of
   residual.  They recommend computing the externally studentized
   residual (residual divided by leave-one-out standard deviation
   times square root of 1 - diagonal element of hat-matrix), for its
   distribution is known in a linear model.

   Once You're in those leave-one-out diagnostics, look up DFFITS in
   ls.diag.

   Multivariate and nonlinear generalisations may move along the line
   of thought wheighting vector instead of deletion -> influence curve 
   (basis of robust
   statistics I read in HUBER: Robust Statistics. Wiley 198? and HAMPEL,et.al.:
   Robust Statistics: The Approach Based on Influence Functions. Wiley 1986.)
   -> maximal local change in influence -> likelihood displacement
   as in COOK,R.D.:Assesment of Local Influence. In: J.R.Statist.Soc.B 1986.

   In this way You get diagnostics not for outliers but for
   influencial points.  I wrote about it, because perhaps that is,
   what You need. Perhaps You better start as everyone did, by
   observing how some most interesting output changes, when one or
   several points are excluded. Afterwards You know wether it is
   rewarding to investigate in which observation one has to corrupt,
   in order to get results that differ from the real thing by a given
   margin.

   It would be nice to hear, what You implemented

   Reinhard Vonthein, WWZ/ISO, Uni-Basel, Petersgraben 51, CH - 4051
   Basel [email-masked]

[email-masked] (Woogon Chung) wrote:
   How about detecting outliers using domouSe from Statlib and train your
   neural net?

   Woogon.

ej
Ed Jaeger  Sterling Winthrop Inc., P.O. Box 5000, Collegeville PA 19426-0900 
               [email-masked] || ph:215-983-5509 || fax:215-983-5559}