Re: Means of histograms in the case of negative weights

Hi,

 As it was mentioned in the previous linked posts, the issue with negative weights can be tricky, because it depends on the Interpretation we give to it.
When, we fill an histogram they could be considered as a negative event, which must be subtracted from the sample or just as a scale factor, for example when plotting a quantity which can in some case be negative. In the ROOT histogram we support now only the second case and we use the sign of the weights only when calculating the bin content, but not for the averages and errors.

 The first case (negative weight as a negative count) is currently not supported. We could support in principle this case by adding for example an histogram option in the constructor and we need to maintain the previous case for backward compatibility. However, for negative weights we need to figure out what is the correct statistical error to assign to each bin. Using sqrt(sum(weight^2) ) is probably not correct. We need probably to keep track also of the number of entries per bin as it is done in the profile. In addition, we have to ensure that the total sum of weights in the histogram is always positive, otherwise non meaningful results could be obtained (for example averages outside the histogram range). Do we need also to ensure that the bin sum of weights is also always positive ? Also one has to be careful for the variance calculations. Eventually, we need to add the support also for the profile histograms.

I would appreciate if you can point to me a statistical paper or book or note on using negative weights. If a statistical reference exists, we could eventually implement something based on that reference, maybe as an option in the histogram class or, if needed, as a separate derived class.

   Best Regards

 Lorenzo

On Jun 7, 2010, at 1:58 PM, Amnon Harel wrote:

> Dear all,
>
> I'd like to add that this thread does not provide any motivation for the
> current treatment. In Tim's example that treatment looks silly, and it's
> a bit unfair to the developers to leave it at that.
> Some of the thinking behind the current treatment was presented in the
> discussions of ROOT bug #44811, where I thought one of the conclusion was a
> feature request for better support for negative weights, which the ROOT team
> was willing to think about. Has there been any progress on that?
>
> cheers,
> Amnon
>
>
> -----Original Message-----
> From: owner-roottalk_at_root.cern.ch on behalf of Tim Head
> Sent: Mon 07-Jun-10 3:51 AM
> To: Rene Brun
> Cc: roottalk_at_root.cern.ch
> Subject: Re: [ROOT] Means of histograms in the case of negative weights
>
> Hello,
>
> On 7 June 2010 01:27, Rene Brun <Rene.Brun_at_cern.ch> wrote:
> > Hi Tim,
> >
> > see discussion at:
> > http://root.cern.ch/phpBB3//viewtopic.php?t=7654&view=previous&sid=7617ae5a0b2fde2737fabaec119044ee
> >
>
> Thanks for the pointer Rene.
>
> The posts are fairly old and end without a good conclusion, does
> anyone know of more developments/recommendations made since then?
>
> Tim
>
> --
> http://tim.jottit.com/
>
>
>
Received on Tue Jun 08 2010 - 11:01:47 CEST