Re: [ROOT] looping over many events

From: Rene Brun (Rene.Brun@cern.ch)
Date: Fri Dec 14 2001 - 16:24:39 MET


Hi Andre,

Sorry to answer only now this old mail from you.

The TTree::Query function returns a TTreeResult object that holds the
result of your query (all rows and columns). This may be a very large object.
In the version 3.02/06 recently released, the Query function has been
drastically
improved in time (about a factor 8 in your example) and the space in memory
used to store the result divided by about two. This has been achieved
by rewriting the class TTreeRow.

You will notice that even in running the new version, the OS does not release
the space occupied by the previous object when you delete the object. This
happens
in case many tiny objects are allocated during the query. In this case, the OS
tries to be clever and anticipate more creations of these tiny objects
before reusing the space.
This is not a memory leak, but the way the OS is doing the memory management.
After a few queries, you will see that the space does not grow anymore.

Rene Brun


Andre Holzner wrote:
> 
> Hello,
> 
> I'd like to loop over many events of a TTree
> while taking advantage of Root's mathematical expression
> evaluation. (The application needs to calculate some
> sort of correlation matrices between several variables
> of events).
> 
> I therefore need to keep only one event at a time in memory.
> 
> I have the following test program:
> 
> #include <stdio.h>
> #include <iostream.h>
> #include <stdlib.h>
> #include <unistd.h>
> 
> #include <strstream>
> 
> #include <TChain.h>
> #include <TSQLResult.h>
> #include <TROOT.h>
> #include <TFile.h>
> 
> //--------------------
> 
> void mem_usage(const char *description)
> {
>   static pid_t pid = getpid();
> 
>   cout << "--------------------\n"
>        << description << endl;
> 
>   ostrstream buf;
>   buf << "ps u -p " << pid << ends;
>   system(buf.str());
> 
>   cout << "--------------------\n";
> }
> 
> //--------------------
> 
> void main()
> {
>   TChain *ch = new TChain("h20","h20");
>   ch->Add("~/qqqq/presel-ntp-205/ww-207.root/h20");
> 
>   mem_usage("point 1");
>   TSQLResult *query_res = ch->Query("thru:xov1:log34:xmj1:mijmin");
> 
>   mem_usage("point 2");
>   delete query_res;
>   mem_usage("point 3");
>   delete ch;
>   mem_usage("point 4");
> }
> 
> The memory consumption is however quite large, it looks like root
> is keeping all events in memory:
> --------------------
> point 1
> USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
> holzner   4709 41.0  4.1 18348 10956 pts/0   S    12:43   0:00
> ./test-sql
> --------------------
> --------------------
> point 2
> USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
> holzner   4709 94.3 31.4 91792 82244 pts/0   S    12:43   0:30
> ./test-sql
> --------------------
> --------------------
> point 3
> USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
> holzner   4709 96.6 30.3 89092 79544 pts/0   S    12:43   0:31
> ./test-sql
> --------------------
> --------------------
> point 4
> USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
> holzner   4709 93.7 30.3 89092 79544 pts/0   S    12:43   0:31
> ./test-sql
> --------------------
> 
> and increases with every variable added. In fact, adding more and more
> variables,
> the memory consumption becomes so big that the program can not run
> anymore.
> 
> In this simple case, I could assign
> a placeholder variable in the program to loop over the events, however,
> I would like to keep it general so that e.g. I can also use expressions
> like 'var1+var2' etc.
> 
> Is there currently any way to do that ? (I'm using root from CVS of
> yesterday on
> a Linux/i386 machine).
> 
> best regards & thanks for the help,
> 
> André



This archive was generated by hypermail 2b29 : Tue Jan 01 2002 - 17:51:12 MET