Re: [ROOT] Files sizes?

From: Creighton Hogg (wchogg@hep.wisc.edu)
Date: Fri Jul 23 2004 - 00:52:10 MEST


On Thu, 22 Jul 2004, Rene Brun wrote:

> This could be:
>   -a compression effect
>   -the fact that you do not reset some arrays
>
> Could you send
>   -the result of tree.Print() for 100, 1000 amd 10000 events
>   - for the file with 10000 events the result of
>      -tree.Show(50);
>      -tree.Show(500);
>      -tree.Show(5000);

This will be an awful lot of text, and I apologize for any inconvenience 
it might cause.

What's really confusing to me is the discrepancy between what ROOT lists 
as the file size under tree.Print() and what I get when I do ls -s in the 
directory, which is
total 1217188
  79052 edde_1000.root  1136371 edde_10000.root     1765 edde_100_1.root
So 80 megs for the 1000, 1.1 gigs for the 10000, and 2 megs for the 100.

I'd appreciate any help or pointers to a way to handle this.  I just can't 
believe that these files should be this big.

For the 100 event file tree.Print()
******************************************************************************
*Tree    :h1        : h1                                                     
*
*Entries :      100 : Total =          593483 bytes  File  Size =      
24778 *
*        :          : Tree compression factor =   1.00                       
*
******************************************************************************
*Br    0 :gemet     : gEMEt[4]/F                                             
*
*Entries :      100 : Total  Size=      14008 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    1 :gemphi    : gEMPhi[4]/F                                            
*
*Entries :      100 : Total  Size=      14014 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    2 :gemeta    : gEMEta[4]/F                                            
*
*Entries :      100 : Total  Size=      14014 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    3 :iemet     : iEMEt[4]/F                                             
*
*Entries :      100 : Total  Size=      14008 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    4 :iemphi    : iEMPhi[4]/F                                            
*
*Entries :      100 : Total  Size=      14014 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    5 :iemeta    : iEMEta[4]/F                                            
*
*Entries :      100 : Total  Size=      14014 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    6 :iembit    : iEMBit[4]/I                                            
*
*Entries :      100 : Total  Size=      14014 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    7 :nemet     : nEMEt[4]/F                                             
*
*Entries :      100 : Total  Size=      14008 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    8 :nemphi    : nEMPhi[4]/F                                            
*
*Entries :      100 : Total  Size=      14014 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    9 :nemeta    : nEMEta[4]/F                                            
*
*Entries :      100 : Total  Size=      14014 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br   10 :nembit    : nEMBit[4]/I                                            
*
*Entries :      100 : Total  Size=      14014 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br   11 :uemet     : uEMEt[4]/F                                             
*
*Entries :      100 : Total  Size=      14008 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*

For the 1000 event file tree.Print()
******************************************************************************
*Tree    :h1        : h1                                                     
*
*Entries :     1000 : Total =         1353748 bytes  File  Size =      
94933 *
*        :          : Tree compression factor =  21.01                       
*
******************************************************************************
*Br    0 :gemet     : gEMEt[4]/F                                             
*
*Entries :     1000 : Total  Size=      28408 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    1 :gemphi    : gEMPhi[4]/F                                            
*
*Entries :     1000 : Total  Size=      28414 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    2 :gemeta    : gEMEta[4]/F                                            
*
*Entries :     1000 : Total  Size=      28414 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    3 :iemet     : iEMEt[4]/F                                             
*
*Entries :     1000 : Total  Size=      28408 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    4 :iemphi    : iEMPhi[4]/F                                            
*
*Entries :     1000 : Total  Size=      28414 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    5 :iemeta    : iEMEta[4]/F                                            
*
*Entries :     1000 : Total  Size=      28414 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    6 :iembit    : iEMBit[4]/I                                            
*
*Entries :     1000 : Total  Size=      28414 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    7 :nemet     : nEMEt[4]/F                                             
*
*Entries :     1000 : Total  Size=      28408 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    8 :nemphi    : nEMPhi[4]/F                                            
*
*Entries :     1000 : Total  Size=      28414 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br    9 :nemeta    : nEMEta[4]/F                                            
*
*Entries :     1000 : Total  Size=      28414 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br   10 :nembit    : nEMBit[4]/I                                            
*
*Entries :     1000 : Total  Size=      28414 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*
*Br   11 :uemet     : uEMEt[4]/F                                             
*
*Entries :     1000 : Total  Size=      28408 bytes  One basket in memory    
*
*Baskets :        0 : Basket Size=      32000 bytes  Compression=   1.00     
*
*............................................................................*

And the 10000 event file

******************************************************************************
*Tree    :h1        : h1                                                     
*
*Entries :    10000 : Total =         8966618 bytes  File  Size =     
738928 *
*        :          : Tree compression factor =  11.77                       
*
******************************************************************************
*Br    0 :gemet     : gEMEt[4]/F                                             
*
*Entries :    10000 : Total  Size=     172738 bytes  File Size  =      
18944 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression=   8.45     
*
*............................................................................*
*Br    1 :gemphi    : gEMPhi[4]/F                                            
*
*Entries :    10000 : Total  Size=     172749 bytes  File Size  =      
19579 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression=   8.17     
*
*............................................................................*
*Br    2 :gemeta    : gEMEta[4]/F                                            
*
*Entries :    10000 : Total  Size=     172749 bytes  File Size  =      
19587 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression=   8.17     
*
*............................................................................*
*Br    3 :iemet     : iEMEt[4]/F                                             
*
*Entries :    10000 : Total  Size=     172738 bytes  File Size  =      
43928 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression=   3.64     
*
*............................................................................*
*Br    4 :iemphi    : iEMPhi[4]/F                                            
*
*Entries :    10000 : Total  Size=     172749 bytes  File Size  =      
45083 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression=   3.55     
*
*............................................................................*
*Br    5 :iemeta    : iEMEta[4]/F                                            
*
*Entries :    10000 : Total  Size=     172749 bytes  File Size  =      
41709 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression=   3.84     
*
*............................................................................*
*Br    6 :iembit    : iEMBit[4]/I                                            
*
*Entries :    10000 : Total  Size=     172749 bytes  File Size  =       
1140 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression= 140.35     
*
*............................................................................*
*Br    7 :nemet     : nEMEt[4]/F                                             
*
*Entries :    10000 : Total  Size=     172738 bytes  File Size  =      
28419 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression=   5.63     
*
*............................................................................*
*Br    8 :nemphi    : nEMPhi[4]/F                                            
*
*Entries :    10000 : Total  Size=     172749 bytes  File Size  =      
29694 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression=   5.39     
*
*............................................................................*
*Br    9 :nemeta    : nEMEta[4]/F                                            
*
*Entries :    10000 : Total  Size=     172749 bytes  File Size  =      
29200 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression=   5.48     
*
*............................................................................*
*Br   10 :nembit    : nEMBit[4]/I                                            
*
*Entries :    10000 : Total  Size=     172749 bytes  File Size  =      
16959 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression=   9.43     
*
*............................................................................*
*Br   11 :uemet     : uEMEt[4]/F                                             
*
*Entries :    10000 : Total  Size=     172738 bytes  File Size  =      
15749 *
*Baskets :        5 : Basket Size=      32000 bytes  Compression=  10.16     
*
*............................................................................*


And the tree.Show() for events 50, 500, and 5000 of the 10000 event file

======> EVENT:50
 gEMEt           = 10.007, 
                  5.72417, 3.7496, 2.27648
 gEMPhi          = -0.352982, 
                  -0.366715, -0.277842, -0.270547
 gEMEta          = -2.47447, 
                  -2.26015, -2.30888, -2.26326
 iEMEt           = 7.5, 
                  2.5, 2.5, 1.5
 iEMPhi          = -0.523599, 
                  2.26893, -0.174533, -1.5708
 iEMEta          = -2.586, 
                  -0.87, -2.586, -2.586
 iEMBit          = 0, 
                  0, 0, 0
 nEMEt           = 0, 
                  0, 0, 0
 nEMPhi          = 999, 
                  999, 999, 999
 nEMEta          = 999, 
                  999, 999, 999
 nEMBit          = 0, 
                  0, 0, 0
 uEMEt           = 0, 
                  0, 0, 0
 uEMPhi          = 999, 
                  999, 999, 999
 uEMEta          = 999, 
                  999, 999, 999
 uEMBit          = 0, 
                  0, 0, 0
 gJetEt          = 45.1232, 
                  3.61072, 0, 0, 0, 0, 
                  0, 0
 gJetPhi         = -0.315401, 
                  2.34274, 0, 0, 0, 0, 
                  0, 0
 gJetEta         = -2.38672, 
                  -2.1912, 0, 0, 0, 0, 
                  0, 0
 gFJetEt         = 39.7752, 
                  1.65573, 0, 0
 gFJetEta        = -3.39799, 
                 -3.14461, 0, 0
 gFJetPhi        = 2.85322, 
                  2.29809, 0, 0
 rJetEt          = 20, 
                  0, 0, 0
 rJetPhi         = -0.523599, 
                  0, 0, 0
 rJetEta         = -2.586, 
                  0, 0, 0
 fJetEt          = 7, 
                  0, 0, 0
 fJetEta         = -3.0835, 
                  0, 0, 0
 fJetPhi         = 2.96706, 
                  0, 0, 0
 gTauEt          = 0, 
                  0, 0, 0
 gTauEta         = 0, 
                  0, 0, 0
 gTauPhi         = 0, 
                  0, 0, 0
 rTauEt          = 0, 
                  0, 0, 0
 rTauEta         = 0, 
                  0, 0, 0
 rTauPhi         = 0, 
                  0, 0, 0
 gPUPt           = 0, 
                  0, 0, 0, 0, 0, 
                  0, 0, 0, 0, 0, 
                  0, 0, 0, 0, 0, 
                  0, 0, 0, 0
 gPt             = 9.11696e+11
 nPU             = 807425093
 gTotEt          = 59.2842
 gMisEt          = 18.2258
 rTotEt          = 41
 rMisEt          = 14
 rHT             = 20

======> EVENT:500
 gEMEt           = 5.65494, 
                  3.76494, 2.90497, 2.36828
 gEMPhi          = 0.673534, 
                  0.931451, -2.11552, -2.36309
 gEMEta          = -1.14793, 
                  -1.46625, -2.67335, -2.25104
 iEMEt           = 5.5, 
                  5, 4.5, 4
 iEMPhi          = 0.523599, 
                  -2.26893, 0.872665, 0.523599
 iEMEta          = -1.218, 
                  -2.586, -1.566, -0.87
 iEMBit          = 0, 
                  0, 0, 0
 nEMEt           = 0, 
                  0, 0, 0
 nEMPhi          = 2.61799, 
                  999, 2.61799, 2.96706
 nEMEta          = 1.956, 
                  999, 2.586, 2.586
 nEMBit          = 0, 
                  0, 0, 0
 uEMEt           = 0, 
                  0, 0, 0
 uEMPhi          = 999, 
                  999, 999, 2.96706
 uEMEta          = 999, 
                  999, 999, 2.586
 uEMBit          = 0, 
                  0, 0, 0
 gJetEt          = 45.7361, 
                  45.0293, 1.02085, 0, 0, 0, 
                  0, 0
 gJetPhi         = 0.710199, 
                  -2.44848, -1.77321, 0, 0, 0, 
                  0, 0
 gJetEta         = -1.20366, 
                  -2.46634, -1.89238, 0, 0, 0, 
                  0, 0
 gFJetEt         = 0, 
                  0, 0, 0
 gFJetEta        = 0, 
                  0, 0, 0
 gFJetPhi        = 0, 
                  0, 0, 0
 rJetEt          = 12, 
                  0, 0, 0
 rJetPhi         = -2.26893, 
                  0, 0, 0
 rJetEta         = -2.586, 
                  0, 0, 0
 fJetEt          = 0, 
                  0, 0, 0
 fJetEta         = 0, 
                  0, 0, 0
 fJetPhi         = 0, 
                  0, 0, 0
 gTauEt          = 0, 
                  0, 0, 0
 gTauEta         = 0, 
                  0, 0, 0
 gTauPhi         = 0, 
                  0, 0, 0
 rTauEt          = 11, 
                  0, 0, 0
 rTauEta         = -1.218, 
                  0, 0, 0
 rTauPhi         = 0.523599, 
                  0, 0, 0
 gPUPt           = 0, 
                  0, 0, 0, 0, 0, 
                  0, 0, 0, 0, 0, 
                  0, 0, 0, 0, 0, 
                  0, 0, 0, 0
 gPt             = 9.11696e+11
 nPU             = 807425093
 gTotEt          = 93.109
 gMisEt          = 0.307852
 rTotEt          = 23
 rMisEt          = 1
 rHT             = 23

======> EVENT:5000
 gEMEt           = 5.65494, 
                  3.76494, 2.90497, 2.36828
 gEMPhi          = 0.673534, 
                  0.931451, -2.11552, -2.36309
 gEMEta          = -1.14793, 
                  -1.46625, -2.67335, -2.25104
 iEMEt           = 3, 
                  2.5, 1.5, 1
 iEMPhi          = 0.872665, 
                  -2.26893, -2.61799, 0.872665
 iEMEta          = -1.566, 
                  -2.586, -2.586, -1.218
 iEMBit          = 0, 
                  0, 0, 0
 nEMEt           = 0, 
                  0, 0, 0
 nEMPhi          = 2.61799, 
                  2.96706, 2.61799, 2.96706
 nEMEta          = 1.956, 
                  1.956, 2.586, 2.586
 nEMBit          = 0, 
                  0, 0, 0
 uEMEt           = 0, 
                  0, 0, 0
 uEMPhi          = 2.61799, 
                  2.96706, 999, 2.96706
 uEMEta          = 1.956, 
                  1.956, 999, 2.586
 uEMBit          = 0, 
                  0, 0, 0
 gJetEt          = 45.7361, 
                  45.0293, 1.02085, 0, 0, 0, 
                  0, 0
 gJetPhi         = 0.710199, 
                  -2.44848, -1.77321, 0, 0, 0, 
                  0, 0
 gJetEta         = -1.20366, 
                  -2.46634, -1.89238, 0, 0, 0, 
                  0, 0
 gFJetEt         = 0, 
                  0, 0, 0
 gFJetEta        = 0, 
                  0, 0, 0
 gFJetPhi        = 0, 
                  0, 0, 0
 rJetEt          = 0, 
                  0, 0, 0
 rJetPhi         = 0, 
                  0, 0, 0
 rJetEta         = 0, 
                  0, 0, 0
 fJetEt          = 0, 
                  0, 0, 0
 fJetEta         = 0, 
                  0, 0, 0
 fJetPhi         = 0, 
                  0, 0, 0
 gTauEt          = 0, 
                  0, 0, 0
 gTauEta         = 0, 
                  0, 0, 0
 gTauPhi         = 0, 
                  0, 0, 0
 rTauEt          = 8, 
                  0, 0, 0
 rTauEta         = 0.522, 
                  0, 0, 0
 rTauPhi         = 1.22173, 
                  0, 0, 0
 gPUPt           = 0, 
                  0, 0, 0, 0, 0, 
                  0, 0, 0, 0, 0, 
                  0, 0, 0, 0, 0, 
                  0, 0, 0, 0
 gPt             = 9.11696e+11
 nPU             = 807425093
 gTotEt          = 93.109
 gMisEt          = 0.307852
 rTotEt          = 27
 rMisEt          = 7
 rHT             = 0




On Thu, 22 Jul 2004, Rene Brun wrote:

> This could be:
>   -a compression effect
>   -the fact that you do not reset some arrays
> 
> Could you send 
>   -the result of tree.Print() for 100, 1000 amd 10000 events
>   - for the file with 10000 events the result of 
>      -tree.Show(50);
>      -tree.Show(500);
>      -tree.Show(5000);
> 
> 
> Rene Brun
> 
> Creighton Hogg wrote:
> > 
> > Hello,
> > I'm having trouble with unusually large root files being produced by an
> > ORCA program I have written.  It writes 6 floats and 34 arrays of 4 floats each
> > per event and the file size seems to vary non-linearly with the number of
> > events.  With 100 events it is roughly 2Meg.  With 1000 events it is
> > roughly 70 Meg.  With 10000 events it is roughly 1Gig.
> > Does anyone have an idea why this would happen?
> > Thanks
> 
> 



This archive was generated by hypermail 2b29 : Sun Jan 02 2005 - 05:50:09 MET