Re: histogram subtraction and mean

Hi Lorenzo

>  The TProfile is a different thing. It represents an average of y values. The Add (or subtraction) operation of profile is like
> merging or removing events. I will need check if the results (bin by bin) are correct when doing the subtraction operations

yes, that is exactly what I need, subtract background events, get correct mean, and do all that in an easy way instead of handling hundreds of histograms. I suppose it does not work because of a bug like abs(w) in TH1.

Margar

> Lorenzo
>
> On Mar 1, 2011, at 10:24 PM, Margar Simonyan wrote:
>
>> Hi Lorenzo
>>
>> thanks for checking. It is such a common operation that I could not
>> believe a bug can survive. I have one last question un-answered: can
>> one use TProfile and do "background subtraction" instead of dealing
>> with individual histograms?
>>
>> Margar
>>
>>
>>
>> On Tue, Mar 1, 2011 at 9:36 PM, Lorenzo Moneta <Lorenzo.Moneta_at_cern.ch> wrote:
>>> Hi Margar,
>>>
>>>  I have checked the code and it is true, the statistics is not correctly calculated. It uses the abs(w) and this is wrong if w is negative.
>>> As I said before, I will correct for negative w, BUT if there are negative bins it cannot be computed anymore !
>>> I will set artificial 0 values in this case
>>>
>>> A similar problem happens when you fill the histogram with negative weights
>>>
>>>  Cheers
>>>
>>>  Lorenzo
>>> On Mar 1, 2011, at 6:02 PM, Margar Simonyan wrote:
>>>
>>>> Hello Lorenzo
>>>>
>>>> thank you for the answer. Please have a look the attached figure, I
>>>> make difference between the two histograms:
>>>>
>>>> blue.Add(red, w)
>>>>
>>>> where w is between -1 and 0. Naively I expect after Add the mean of
>>>> blue histogram to increase, but it depends on w. I don't understand
>>>> this results. I verified that there are no bins with negative content
>>>> before and after Add.
>>>>
>>>> Thanks,
>>>>       Margar
>>>>
>>>>
>>>>
>>>> On Tue, Mar 1, 2011 at 4:14 PM, Lorenzo Moneta <Lorenzo.Moneta_at_cern.ch> wrote:
>>>>> Hello Margar,
>>>>>
>>>>>  when you are getting an histogram with negative bins content (for example from the subtraction of two histograms)
>>>>> the statistics (mean , s.d., etc..) is computed now in ROOT  using the absolute value of the bin content.
>>>>> In my opinion, if a bin has negative content, it does not make any sense to compute any statistics using the bin centers.
>>>>> You would need to compute it using the original entries from the histogram.
>>>>> It is my plan to set artificially a mean/s.d. to zero )or whatever not defined value) in this particular cases to avoid computing a
>>>>> totally wrong result and avoiding confusion
>>>>>
>>>>> Best Regards
>>>>>
>>>>>  Lorenzo
>>>>>
>>>>>
>>>>> On Mar 1, 2011, at 3:54 PM, Margar Simonyan wrote:
>>>>>
>>>>>> Dear Rene
>>>>>>
>>>>>> thanks, now I understand the observed differences. In a real example I
>>>>>> have another issue, signal+background distribution has empty bins, but
>>>>>> background distribution can have non-zero content for the same bins,
>>>>>> then the difference has bins with negative content. I tried to re-bin,
>>>>>> but the results were depend significantly on grouping. Is there a
>>>>>> better way of solving this issue?
>>>>>>
>>>>>> Can background subtraction from signal+background done with TProfile
>>>>>> (Add)? I attach updated version of my script. Certainly TProfile:Add
>>>>>> does something different.
>>>>>>
>>>>>> Best regards,
>>>>>>        Margar
>>>>>>
>>>>>> On Tue, Mar 1, 2011 at 12:56 PM, Rene Brun <Rene.Brun_at_cern.ch> wrote:
>>>>>>> What you get is perfectly normal.
>>>>>>> Following an operation on your histogram (Add, Substract, Rebin, etc) the
>>>>>>> statitics for moments (mean, sigma, etc)
>>>>>>> are recomputed from the bin contents, assuming the center of the bin.
>>>>>>>
>>>>>>> Rene Brun
>>>>>>>
>>>>>>>
>>>>>>> On 01/03/2011 12:37, Margar Simonyan wrote:
>>>>>>>>
>>>>>>>> Hello ROOTTalk
>>>>>>>>
>>>>>>>> I get strange results after histogram subtraction, the attached script
>>>>>>>> written in Python demonstrates the issue. My goal is to subtract
>>>>>>>> background from signal+background distribution and get meaningful
>>>>>>>> results for mean.
>>>>>>>> There are several unexpected (for me) results:
>>>>>>>> First, the mean changes after subtracting empty histogram, this is not
>>>>>>>> a big issue. Second, after subtracting background I don't get exactly
>>>>>>>> the signal value. Third, rebinning before subtracting changes the
>>>>>>>> results once more.
>>>>>>>> Can somebody explain this? I am using ROOT 5.26/00e complied on SLC5
>>>>>>>> with gcc43.
>>>>>>>>
>>>>>>>>  Thanks,
>>>>>>>>        Margar
>>>>>>>>  -------------------------------------------------------------------------
>>>>>>>>   Dr Margar Simonyan,  post-doctoral researcher
>>>>>>>>   Niels Bohr Institute, Copenhagen University
>>>>>>>>  -------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>> <histo.py>
>>>>>
>>>>>
>>>> <test.png>
>>>
>>>
>
>
Received on Wed Mar 02 2011 - 11:14:34 CET