Re: TFile memory leak?

Hello Philippe,

Thanks again for your advise. kCanDelete works fine in my actual longer script.

Regards,

--
OKUMURA, Akira oxon@{astro.isas.jaxa.jp,stanford.edu}
Institute of Space and Astronautical Science (ISAS/JAXA)
Now at KIPAC/SLAC/Stanford
Varian Physics #306
382 Via Pueblo Mall, MC 406 Stanford, CA94305
TEL 650-736-0971/FAX 650-724-5065
Skype : okumura.akira

On 2010/09/09, at 11:07, Philippe Canal wrote:

> Hi,
> 
> If you know that a file has a canvas which needs to own its content simply do:
> 
>   TFile f("tmp.root");
>   TCanvas* c = (TCanvas*)f.Get("can");
>   TIter next( c->GetListOfPrimitives() );
>   TObject *obj;
>   while( (obj = next() ) {
>     obj->SetBit(TObject::kCanDelete()); // Add missing ownership delegation
>   }
> 
> Cheers,
> Philippe.
> 
> 
> On 9/9/10 12:57 PM, OKUMURA, Akira wrote:
>> Hello Philippe,
>> 
>> Thank you for your detailed explanation. I now understand that it was my mistake and how I should save canvases in ROOT files.
>> 
>> In addition, I would like to know how I can delete the allocated memory with the bad files which have been already created in my HDD. I have many files which were created without caring about ownership delegation. Of course, I can recreate the files with small modification, but do you have any easy solution?
>> 
>> Regards,
>> 
>> --
>> OKUMURA, Akira oxon@{astro.isas.jaxa.jp,stanford.edu}
>> Institute of Space and Astronautical Science (ISAS/JAXA)
>> Now at KIPAC/SLAC/Stanford
>> Varian Physics #306
>> 382 Via Pueblo Mall, MC 406 Stanford, CA94305
>> TEL 650-736-0971/FAX 650-724-5065
>> Skype : okumura.akira
>> 
>> On 2010/09/09, at 10:32, Philippe Canal wrote:
>> 


>>> Hi Akira,

>>> 

>>> The issue is the ownership of the histogram object.  The simplest fix is to use:

>>> 

>>> TCanvas can("can", "can");

>>> TH2D h("h", "h", 100, -3, 3, 100, -3, 3);

>>> h.FillRandom("gaus", 100000);

>>> h.DrawCopy("col");

>>> can.SaveAs("tmp.root");

>>> 

>>> The issue is that when you do h.Draw, it does associate the histogram with

>>> the Canvas but it __correctly__ (especially in your case) does not delegate

>>> ownership of the histogram to the Canvas.   Since you did not delegate

>>> ownership of the histogram to the Canvas, the Canvas __must__ assume

>>> that you will store the histogram independently and thus upon reading

>>> the file, the Canvas does not take ownership of the histogram and do

>>> not delete it when you delete the Canvas.

>>> 

>>> By using DrawCopy you create a new histogram and delegate its

>>> ownership to the Canvas.

>>> 

>>> Note that you of course need to delete the Canvas in the reading

>>> loop in order to really get rid of the leak (this is because Canvas

>>> are __not__ owned by the TFile they are read from).

>>> 

>>> Cheers,

>>> Philippe.

>>> 

>>> PS.  A slightly more cpu efficient option is:

>>> 

>>> TCanvas can("can", "can");

>>> TH2D *h = new TH2D("h", "h", 100, -3, 3, 100, -3, 3);

>>> h->FillRandom("gaus", 100000);

>>> h->Draw("col");

>>> h->SetBit(TObject::kCanDelete); // Delegate ownership to the Canvas.

>>> can.SaveAs("tmp.root");

>>> 

>>> On 9/9/10 10:34 AM, OKUMURA, Akira wrote:

>>>> Hello all,
>>>> 
>>>> Here is a minimum example. You don't need to download http://slac.stanford.edu/~oxon/tmp.tgz
>>>> 
>>>> ==============================
>>>> #include<TFile.h>
>>>> #include<TCanvas.h>
>>>> #include<TH2D.h>
>>>> #include<iostream>
>>>> 
>>>> int main(int argc, char** argv)
>>>> {
>>>>  TCanvas can("can", "can");
>>>>  TH2D h("h", "h", 100, -3, 3, 100, -3, 3);
>>>>  h.FillRandom("gaus", 100000);
>>>>  h.Draw("col");
>>>>  can.SaveAs("tmp.root");
>>>> 
>>>>  const Int_t kN = 10000;
>>>>  for(Int_t i = 0; i<   kN; i++){
>>>>    TFile f("tmp.root");
>>>>    std::cout<<   "Reading"<<   f.GetName()<<   "...\n";
>>>>    TCanvas* c = (TCanvas*)f.Get("can");
>>>>    f.Close();
>>>>  } // i
>>>> 
>>>>  return 0;
>>>> }
>>>> ==============================
>>>> 
>>>> I would like to know how to delete the allocated memory to TCanvas* c. C++'s delete does not work.
>>>> 
>>>> Regards,
>>>> 
>>>> --
>>>> OKUMURA, Akira oxon@{astro.isas.jaxa.jp,stanford.edu}
>>>> Institute of Space and Astronautical Science (ISAS/JAXA)
>>>> Now at KIPAC/SLAC/Stanford
>>>> Varian Physics #306
>>>> 382 Via Pueblo Mall, MC 406 Stanford, CA94305
>>>> TEL 650-736-0971/FAX 650-724-5065
>>>> Skype : okumura.akira
>>>> 
>>>> On 2010/09/09, at 8:20, OKUMURA, Akira wrote:
>>>> 
>>>>> Hello John and all,
>>>>> 
>>>>> I uploaded the log file here.
>>>>> http://slac.stanford.edu/~oxon/valgrind.log
>>>>> 
>>>>> Regards,
>>>>> 
>>>>> --
>>>>> OKUMURA, Akira oxon@{astro.isas.jaxa.jp,stanford.edu}
>>>>> Institute of Space and Astronautical Science (ISAS/JAXA)
>>>>> Now at KIPAC/SLAC/Stanford
>>>>> Varian Physics #306
>>>>> 382 Via Pueblo Mall, MC 406 Stanford, CA94305
>>>>> TEL 650-736-0971/FAX 650-724-5065
>>>>> Skype : okumura.akira
>>>>> 
>>>>> On 2010/09/09, at 2:06, John Idarraga wrote:
>>>>> 
>>>>>> Hello Akira
>>>>>> 
>>>>>> Can you&>   that to a log file and send it ?
>>>>>> 
>>>>>> John
>>>>>> 
>>>>>> OKUMURA, Akira wrote:
>>>>>>> Hello Claudi and all,
>>>>>>> 
>>>>>>> Thank you for your advice. I attach a part of the valgrind output. Even though this is a first time to use valgrind for me, the output seems that there is memory leak.
>>>>>>> 
>>>>>>> ==17737== 5,189,819 (27,328 direct, 5,162,491 indirect) bytes in 61 blocks are definitely lost in loss record 45,809 of 45,811
>>>>>>> ==17737==    at 0x1000146CE: operator new(unsigned long) (vg_replace_malloc.c:261)
>>>>>>> ==17737==    by 0x1000386C9: TStorage::ObjectAlloc(unsigned long) (in /usr/local/root_svn/lib/libCore.so)
>>>>>>> ==17737==    by 0x100233BB4: TObject::operator new(unsigned long) (in /usr/local/root_svn/lib/libCore.so)
>>>>>>> ==17737==    by 0x102DE6955: ROOT::new_TPaletteAxis(void*) (in /usr/local/root_svn/lib/libHistPainter.so)
>>>>>>> ==17737==    by 0x10010AAA5: TClass::New(TClass::ENewType) const (in /usr/local/root_svn/lib/libCore.so)
>>>>>>> ==17737==    by 0x100FEB2BF: TBufferFile::ReadObjectAny(TClass const*) (in /usr/local/root_svn/lib/libRIO.so)
>>>>>>> ==17737==    by 0x1000EF390: TBuffer&   operator>><TObject>(TBuffer&, TObject*&) (in /usr/local/root_svn/lib/libCore.so)
>>>>>>> ==17737==    by 0x1000F2C22: TList::Streamer(TBuffer&) (in /usr/local/root_svn/lib/libCore.so)
>>>>>>> ==17737==    by 0x100FE8A18: TBufferFile::ReadFastArray(void**, TClass const*, int, bool, TMemberStreamer*, TClass const*) (in /usr/local/root_svn/lib/libRIO.so)
>>>>>>> ==17737==    by 0x1010A1864: int TStreamerInfo::ReadBuffer<char**>(TBuffer&, char** const&, int, int, int, int) (in /usr/local/root_svn/lib/libRIO.so)
>>>>>>> ==17737==    by 0x100FE9C15: TBufferFile::ReadClassBuffer(TClass const*, void*, int, unsigned int, unsigned int, TClass const*) (in /usr/local/root_svn/lib/libRIO.so)
>>>>>>> ==17737==    by 0x101363C23: TH1::Streamer(TBuffer&) (in /usr/local/root_svn/lib/libHist.so)
>>>>>>> ==17737== ==17737== 69,739,543 (823,176 direct, 68,916,367 indirect) bytes in 999 blocks are definitely lost in loss record 45,810 of 45,811
>>>>>>> ==17737==    at 0x1000146CE: operator new(unsigned long) (vg_replace_malloc.c:261)
>>>>>>> ==17737==    by 0x1000386C9: TStorage::ObjectAlloc(unsigned long) (in /usr/local/root_svn/lib/libCore.so)
>>>>>>> ==17737==    by 0x100233BB4: TObject::operator new(unsigned long) (in /usr/local/root_svn/lib/libCore.so)
>>>>>>> ==17737==    by 0x101A9EA7E: ROOT::new_TCanvas(void*) (in /usr/local/root_svn/lib/libGpad.so)
>>>>>>> ==17737==    by 0x10010AAA5: TClass::New(TClass::ENewType) const (in /usr/local/root_svn/lib/libCore.so)
>>>>>>> ==17737==    by 0x101013FBB: TKey::ReadObj() (in /usr/local/root_svn/lib/libRIO.so)
>>>>>>> ==17737==    by 0x100FF5AEE: TDirectoryFile::Get(char const*) (in /usr/local/root_svn/lib/libRIO.so)
>>>>>>> ==17737==    by 0x100000EFC: main (test.C:19)
>>>>>>> ==17737== ==17737== LEAK SUMMARY:
>>>>>>> ==17737==    definitely lost: 859,176 bytes in 1,222 blocks
>>>>>>> ==17737==    indirectly lost: 85,444,368 bytes in 14,771 blocks
>>>>>>> ==17737==      possibly lost: 387,824 bytes in 1,534 blocks
>>>>>>> ==17737==    still reachable: 9,549,367 bytes in 84,421 blocks
>>>>>>> ==17737==         suppressed: 852,204 bytes in 23,733 blocks
>>>>>>> ==17737== Reachable blocks (those to which a pointer was found) are not shown.
>>>>>>> ==17737== To see them, rerun with: --leak-check=full --show-reachable=yes
>>>>>>> ==17737== ==17737== For counts of detected and suppressed errors, rerun with: -v
>>>>>>> ==17737== Use --track-origins=yes to see where uninitialised values come from
>>>>>>> ==17737== ERROR SUMMARY: 705 errors from 705 contexts (suppressed: 1692 from 1692)
>>>>>>> 
>>>>>>> Regards,
>>>>>>> 
>>>>>>> --
>>>>>>> OKUMURA, Akira oxon@{astro.isas.jaxa.jp,stanford.edu}
>>>>>>> Institute of Space and Astronautical Science (ISAS/JAXA)
>>>>>>> Now at KIPAC/SLAC/Stanford
>>>>>>> Varian Physics #306
>>>>>>> 382 Via Pueblo Mall, MC 406 Stanford, CA94305
>>>>>>> TEL 650-736-0971/FAX 650-724-5065
>>>>>>> Skype : okumura.akira
>>>>>>> 
>>>>>>> 
>>>>>>> On 2010/09/08, at 14:31, Claudi Astres wrote:
>>>>>>> 
>>>>>>> 
>>>>>>>> Hi Akira,
>>>>>>>> You could use "valgrind" utility to check whether the problem is really a memory leak. Maybe you already knew it.
>>>>>>>> Valgrind has become crucial for me: http://valgrind.org/
>>>>>>>> 
>>>>>>>> Regards.
>>>>>>>> Claudi
>>>>>>>> 
>>>>>>>> 
>>>>>>>> El 08/09/2010, a las 22:14, OKUMURA, Akira escribió:
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> Hello ROOTers,
>>>>>>>>> 
>>>>>>>>> This question may be a very basic one but I could not find similar questions on the web.
>>>>>>>>> 
>>>>>>>>> http://slac.stanford.edu/~oxon/tmp.tgz
>>>>>>>>> 
>>>>>>>>> This tgz file includes a .root file and a very simple script which opens TFile 10000 times. When I execute the script,
>>>>>>>>> 
>>>>>>>>> root [0] .x test.C
>>>>>>>>> 
>>>>>>>>> the memory usage of the process becomes more than 1 GB while all TFile instances are closed.
>>>>>>>>> 
>>>>>>>>> The .root file was created via TPad::SaveAs, as follows.
>>>>>>>>> 
>>>>>>>>> root [0] TCanvas can("can", "can")
>>>>>>>>> root [1] TH2D* h = new TH2D("h", "h", 100, -3, 3, 100, -3, 3)
>>>>>>>>> root [2] h->FillRandom("gaus", 1000000)
>>>>>>>>> root [3] h->Draw()
>>>>>>>>> root [4] can.SaveAs("aho.root")
>>>>>>>>> 
>>>>>>>>> How can I suppress this large memory consumption? I am not sure if this is an intended behavior or memory leak.
>>>>>>>>> 
>>>>>>>>> I am using svn trunk on Snow Leopard and SL 5.
>>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> OKUMURA, Akira oxon@{astro.isas.jaxa.jp,stanford.edu}
>>>>>>>>> Institute of Space and Astronautical Science (ISAS/JAXA)
>>>>>>>>> Now at KIPAC/SLAC/Stanford
>>>>>>>>> Varian Physics #306
>>>>>>>>> 382 Via Pueblo Mall, MC 406 Stanford, CA94305
>>>>>>>>> TEL 650-736-0971/FAX 650-724-5065
>>>>>>>>> Skype : okumura.akira
>>>>>>>>> 
>>>>>>>>> 
>