RE: TEventList::Enter(), 2 issues

From: Rene Brun <Rene.Brun_at_cern.ch>
Date: Fri, 3 Feb 2006 18:30:51 +0100 (MET)


Kerry,

What is the point of having a TEventlist of 200 million entries?

Rene

On Fri, 3
Feb 2006, LEE, KERRY T. (JSC-SF) (UHCL) wrote:

> Thanks Philippe,
>
> Resizing works. I was trying to avoid writing to a new tree to increase
> speed. Now the problem is that when I try to write the very large
> TEventList to a TFile via TFile::Write(), I get the same pointer error. Is
> there a way to keep TFile from trying to allocate so much memory? I tried
> using TObject::kOverwrite, but that did not help. Also, did you see the
> second issue with TEventList::Enter() that I mentioned in my first mail?
>
> Thanks
> Kerry
>
> -----Original Message-----
> From: owner-roottalk_at_pcroot.cern.ch
> To: LEE, KERRY T. (JSC-SF) (UHCL); roottalk_at_pcroot.cern.ch
> Sent: 2/2/2006 12:25 PM
> Subject: RE: [ROOT] TEventList::Enter(), 2 issues
>
> Hi,
>
> The entries in the TEventList are stored as Long64_t. When reaching
> 105000000, TEventList will allocates to holf twice than many entry
> and requires more than 1.600Gb of memory.
>
> There are some technics will could use to mitigate the issue (aka allow
> for sligtly bigger event list) but given the your pattern of use,
> TEventList is no longer appropriate.
>
> Instead I recommend that you store the data in another TTree.
>
> If your limit is really 200 million, you can also make it work by
> pre-setting the size (this avoid the need to copy and to hold in
> memory 2 gigantic buffer):
>
> root [0] TEventList *E = new TEventList();
> root [2] E->Resize(200000000);
> root [1] {
> end with '}', '@':abort > for(int i=0;i<105000000;i++){
> end with '}', '@':abort > E->Enter(i);
> end with '}', '@':abort > if(i>104500000)cout<<"\r"<<i<<flush;
> end with '}', '@':abort > }
> end with '}', '@':abort > }
>
> Cheers,
> Philippe.
>
> -----Original Message-----
> From: owner-roottalk_at_pcroot.cern.ch
> [mailto:owner-roottalk_at_pcroot.cern.ch]
> On Behalf Of LEE, KERRY T. (JSC-SF) (UHCL)
> Sent: Thursday, February 02, 2006 10:38 AM
> To: 'roottalk_at_pcroot.cern.ch'
> Subject: [ROOT] TEventList::Enter(), 2 issues
>
> Dear ROOT team,
>
> I am using ROOT v5.08 on linux CentOS 4 compiled with gcc 3.4.4.
>
> First issue:
>
> I would like to create a very large TEventList with about 200 million
> entries. When it reaches just under 105 millon entries I get an error.
> I
> have plenty of physical memory (2GB + 2GB swap) to perform this task. A
> session below shows the problem. On the other hand, I have no problem
> creating an array of type Long64_t with 200 million entries.
>
> *******************************************
> * *
> * W E L C O M E to R O O T *
> * *
> * Version 5.08/00 13 December 2005 *
> * *
> * You are welcome to visit our Web site *
> * http://root.cern.ch *
> * *
> *******************************************
>
> FreeType Engine v2.1.9 used to render TrueType fonts.
> Compiled on 20 December 2005 for linux with thread support.
>
> CINT/ROOT C/C++ Interpreter version 5.16.5, November 30 2005
> Type ? for help. Commands must be C++ statements.
> Enclose multiple statements between { }.
> root [0] TEventList *E = new TEventList();
> root [1] {
> end with '}', '@':abort > for(int i=0;i<105000000;i++){
> end with '}', '@':abort > E->Enter(i);
> end with '}', '@':abort > if(i>104500000)cout<<"\r"<<i<<flush;
> end with '}', '@':abort > }
> end with '}', '@':abort > }
> 104857599Syntax Error: * (tmpfile):3:
> Error: Illegal pointer operation (tovalue) (tmpfile):3:
> *** Interpreter error recovered ***
>
>
> Second issue:
>
>
> I would like to add random integers to a TEventList, but it seems that
> only higher values than the previous one entered into the TEventList is
> allowed.
>
> root [11] TEventList *lll = new TEventList()
> root [12] lll->Enter(1)
> root [13] lll->GetEntry(0)
> (const Long64_t)1
> root [14] lll->Enter(5)
> root [16] lll->GetEntry(1)
> (const Long64_t)5
> root [17] lll->Enter(3)
> root [18] lll->GetEntry(2)
> (const Long64_t)5
>
> As you can see adding the number 3 to the list after adding 5 produces
> an entry of 5. Is this expected, or is this a bug? If for some reason
> this is not allowed an error message would be very useful. I can only
> assume, since there exists a TEventList::Sort() function this is not the
> desired behavior.
>
> The purpose for doing this is to randomly sample a ROOT tree without
> having to know anything about it except the total number of entries. I
> would like to save this TEventList to a file, then read it and the TTree
> back into an interacive ROOT session.
>
> Thanks
> Kerry
>
Received on Fri Feb 03 2006 - 18:31:01 MET

This archive was generated by hypermail 2.2.0 : Mon Jan 01 2007 - 16:31:57 MET