This must be a very stupid problem somewhere!
Are you storing more than one Tree in your files?
>From your Tree::Print, I see that the space occupied by your Tree is very small.
Could you send me the result of
root > TFile f("myfilewith1000.root");
root > f.ls(); >f1.log
root > f.Map(); >f2.log
and send me the files f1.log and f2.log ?
Rene Brun
Creighton Hogg wrote:
>
> I'm afraid that I'm not confused, though I double checked just to be sure.
> Sorry, I forgot to say that our system is set up so that ls -s gives
> files size in 1k blocks. so
> erodium(wchogg): ls -s
> total 1217188
> 79052 edde_1000.root 1136371 edde_10000.root 1765 edde_100_1.root
> So the largest file indeed is 1.1 million k, or 1.1 Gigs.
>
> This is extremely strange because the sizes seem to have little relation
> with respect to what is described in tree.Print()
> Thanks again,
> Creighton
>
> On Fri, 23 Jul 2004, Rene Brun wrote:
>
> > It looks like you are confusing Gigabytes with Megabytes!
> > My original guess was correct. Your arrays contain a lot of zeros or
> > identical numbers that are compressed by ROOT.
> > The result of tree.Print may be somewhat misleading with small
> > statistics when most or all baskets are in memory because they have not
> > been compressed yet.
> >
> > Rene Brun
> >
> > On
> > Thu, 22
> > Jul
> > 2004, Creighton Hogg wrote:
> >
> > > On Thu, 22 Jul 2004, Rene Brun wrote:
> > >
> > > > This could be:
> > > > -a compression effect
> > > > -the fact that you do not reset some arrays
> > > >
> > > > Could you send
> > > > -the result of tree.Print() for 100, 1000 amd 10000 events
> > > > - for the file with 10000 events the result of
> > > > -tree.Show(50);
> > > > -tree.Show(500);
> > > > -tree.Show(5000);
> > >
> > > This will be an awful lot of text, and I apologize for any inconvenience
> > > it might cause.
> > >
> > > What's really confusing to me is the discrepancy between what ROOT lists
> > > as the file size under tree.Print() and what I get when I do ls -s in the
> > > directory, which is
> > > total 1217188
> > > 79052 edde_1000.root 1136371 edde_10000.root 1765 edde_100_1.root
> > > So 80 megs for the 1000, 1.1 gigs for the 10000, and 2 megs for the 100.
> > >
> > > I'd appreciate any help or pointers to a way to handle this. I just can't
> > > believe that these files should be this big.
> > >
> > > For the 100 event file tree.Print()
> > > ******************************************************************************
> > > *Tree :h1 : h1
> > > *
> > > *Entries : 100 : Total = 593483 bytes File Size =
> > > 24778 *
> > > * : : Tree compression factor = 1.00
> > > *
> > > ******************************************************************************
> > > *Br 0 :gemet : gEMEt[4]/F
> > > *
> > > *Entries : 100 : Total Size= 14008 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 1 :gemphi : gEMPhi[4]/F
> > > *
> > > *Entries : 100 : Total Size= 14014 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 2 :gemeta : gEMEta[4]/F
> > > *
> > > *Entries : 100 : Total Size= 14014 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 3 :iemet : iEMEt[4]/F
> > > *
> > > *Entries : 100 : Total Size= 14008 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 4 :iemphi : iEMPhi[4]/F
> > > *
> > > *Entries : 100 : Total Size= 14014 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 5 :iemeta : iEMEta[4]/F
> > > *
> > > *Entries : 100 : Total Size= 14014 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 6 :iembit : iEMBit[4]/I
> > > *
> > > *Entries : 100 : Total Size= 14014 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 7 :nemet : nEMEt[4]/F
> > > *
> > > *Entries : 100 : Total Size= 14008 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 8 :nemphi : nEMPhi[4]/F
> > > *
> > > *Entries : 100 : Total Size= 14014 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 9 :nemeta : nEMEta[4]/F
> > > *
> > > *Entries : 100 : Total Size= 14014 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 10 :nembit : nEMBit[4]/I
> > > *
> > > *Entries : 100 : Total Size= 14014 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 11 :uemet : uEMEt[4]/F
> > > *
> > > *Entries : 100 : Total Size= 14008 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > >
> > > For the 1000 event file tree.Print()
> > > ******************************************************************************
> > > *Tree :h1 : h1
> > > *
> > > *Entries : 1000 : Total = 1353748 bytes File Size =
> > > 94933 *
> > > * : : Tree compression factor = 21.01
> > > *
> > > ******************************************************************************
> > > *Br 0 :gemet : gEMEt[4]/F
> > > *
> > > *Entries : 1000 : Total Size= 28408 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 1 :gemphi : gEMPhi[4]/F
> > > *
> > > *Entries : 1000 : Total Size= 28414 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 2 :gemeta : gEMEta[4]/F
> > > *
> > > *Entries : 1000 : Total Size= 28414 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 3 :iemet : iEMEt[4]/F
> > > *
> > > *Entries : 1000 : Total Size= 28408 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 4 :iemphi : iEMPhi[4]/F
> > > *
> > > *Entries : 1000 : Total Size= 28414 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 5 :iemeta : iEMEta[4]/F
> > > *
> > > *Entries : 1000 : Total Size= 28414 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 6 :iembit : iEMBit[4]/I
> > > *
> > > *Entries : 1000 : Total Size= 28414 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 7 :nemet : nEMEt[4]/F
> > > *
> > > *Entries : 1000 : Total Size= 28408 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 8 :nemphi : nEMPhi[4]/F
> > > *
> > > *Entries : 1000 : Total Size= 28414 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 9 :nemeta : nEMEta[4]/F
> > > *
> > > *Entries : 1000 : Total Size= 28414 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 10 :nembit : nEMBit[4]/I
> > > *
> > > *Entries : 1000 : Total Size= 28414 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > > *Br 11 :uemet : uEMEt[4]/F
> > > *
> > > *Entries : 1000 : Total Size= 28408 bytes One basket in memory
> > > *
> > > *Baskets : 0 : Basket Size= 32000 bytes Compression= 1.00
> > > *
> > > *............................................................................*
> > >
> > > And the 10000 event file
> > >
> > > ******************************************************************************
> > > *Tree :h1 : h1
> > > *
> > > *Entries : 10000 : Total = 8966618 bytes File Size =
> > > 738928 *
> > > * : : Tree compression factor = 11.77
> > > *
> > > ******************************************************************************
> > > *Br 0 :gemet : gEMEt[4]/F
> > > *
> > > *Entries : 10000 : Total Size= 172738 bytes File Size =
> > > 18944 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 8.45
> > > *
> > > *............................................................................*
> > > *Br 1 :gemphi : gEMPhi[4]/F
> > > *
> > > *Entries : 10000 : Total Size= 172749 bytes File Size =
> > > 19579 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 8.17
> > > *
> > > *............................................................................*
> > > *Br 2 :gemeta : gEMEta[4]/F
> > > *
> > > *Entries : 10000 : Total Size= 172749 bytes File Size =
> > > 19587 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 8.17
> > > *
> > > *............................................................................*
> > > *Br 3 :iemet : iEMEt[4]/F
> > > *
> > > *Entries : 10000 : Total Size= 172738 bytes File Size =
> > > 43928 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 3.64
> > > *
> > > *............................................................................*
> > > *Br 4 :iemphi : iEMPhi[4]/F
> > > *
> > > *Entries : 10000 : Total Size= 172749 bytes File Size =
> > > 45083 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 3.55
> > > *
> > > *............................................................................*
> > > *Br 5 :iemeta : iEMEta[4]/F
> > > *
> > > *Entries : 10000 : Total Size= 172749 bytes File Size =
> > > 41709 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 3.84
> > > *
> > > *............................................................................*
> > > *Br 6 :iembit : iEMBit[4]/I
> > > *
> > > *Entries : 10000 : Total Size= 172749 bytes File Size =
> > > 1140 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 140.35
> > > *
> > > *............................................................................*
> > > *Br 7 :nemet : nEMEt[4]/F
> > > *
> > > *Entries : 10000 : Total Size= 172738 bytes File Size =
> > > 28419 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 5.63
> > > *
> > > *............................................................................*
> > > *Br 8 :nemphi : nEMPhi[4]/F
> > > *
> > > *Entries : 10000 : Total Size= 172749 bytes File Size =
> > > 29694 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 5.39
> > > *
> > > *............................................................................*
> > > *Br 9 :nemeta : nEMEta[4]/F
> > > *
> > > *Entries : 10000 : Total Size= 172749 bytes File Size =
> > > 29200 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 5.48
> > > *
> > > *............................................................................*
> > > *Br 10 :nembit : nEMBit[4]/I
> > > *
> > > *Entries : 10000 : Total Size= 172749 bytes File Size =
> > > 16959 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 9.43
> > > *
> > > *............................................................................*
> > > *Br 11 :uemet : uEMEt[4]/F
> > > *
> > > *Entries : 10000 : Total Size= 172738 bytes File Size =
> > > 15749 *
> > > *Baskets : 5 : Basket Size= 32000 bytes Compression= 10.16
> > > *
> > > *............................................................................*
> > >
> > >
> > > And the tree.Show() for events 50, 500, and 5000 of the 10000 event file
> > >
> > > ======> EVENT:50
> > > gEMEt = 10.007,
> > > 5.72417, 3.7496, 2.27648
> > > gEMPhi = -0.352982,
> > > -0.366715, -0.277842, -0.270547
> > > gEMEta = -2.47447,
> > > -2.26015, -2.30888, -2.26326
> > > iEMEt = 7.5,
> > > 2.5, 2.5, 1.5
> > > iEMPhi = -0.523599,
> > > 2.26893, -0.174533, -1.5708
> > > iEMEta = -2.586,
> > > -0.87, -2.586, -2.586
> > > iEMBit = 0,
> > > 0, 0, 0
> > > nEMEt = 0,
> > > 0, 0, 0
> > > nEMPhi = 999,
> > > 999, 999, 999
> > > nEMEta = 999,
> > > 999, 999, 999
> > > nEMBit = 0,
> > > 0, 0, 0
> > > uEMEt = 0,
> > > 0, 0, 0
> > > uEMPhi = 999,
> > > 999, 999, 999
> > > uEMEta = 999,
> > > 999, 999, 999
> > > uEMBit = 0,
> > > 0, 0, 0
> > > gJetEt = 45.1232,
> > > 3.61072, 0, 0, 0, 0,
> > > 0, 0
> > > gJetPhi = -0.315401,
> > > 2.34274, 0, 0, 0, 0,
> > > 0, 0
> > > gJetEta = -2.38672,
> > > -2.1912, 0, 0, 0, 0,
> > > 0, 0
> > > gFJetEt = 39.7752,
> > > 1.65573, 0, 0
> > > gFJetEta = -3.39799,
> > > -3.14461, 0, 0
> > > gFJetPhi = 2.85322,
> > > 2.29809, 0, 0
> > > rJetEt = 20,
> > > 0, 0, 0
> > > rJetPhi = -0.523599,
> > > 0, 0, 0
> > > rJetEta = -2.586,
> > > 0, 0, 0
> > > fJetEt = 7,
> > > 0, 0, 0
> > > fJetEta = -3.0835,
> > > 0, 0, 0
> > > fJetPhi = 2.96706,
> > > 0, 0, 0
> > > gTauEt = 0,
> > > 0, 0, 0
> > > gTauEta = 0,
> > > 0, 0, 0
> > > gTauPhi = 0,
> > > 0, 0, 0
> > > rTauEt = 0,
> > > 0, 0, 0
> > > rTauEta = 0,
> > > 0, 0, 0
> > > rTauPhi = 0,
> > > 0, 0, 0
> > > gPUPt = 0,
> > > 0, 0, 0, 0, 0,
> > > 0, 0, 0, 0, 0,
> > > 0, 0, 0, 0, 0,
> > > 0, 0, 0, 0
> > > gPt = 9.11696e+11
> > > nPU = 807425093
> > > gTotEt = 59.2842
> > > gMisEt = 18.2258
> > > rTotEt = 41
> > > rMisEt = 14
> > > rHT = 20
> > >
> > > ======> EVENT:500
> > > gEMEt = 5.65494,
> > > 3.76494, 2.90497, 2.36828
> > > gEMPhi = 0.673534,
> > > 0.931451, -2.11552, -2.36309
> > > gEMEta = -1.14793,
> > > -1.46625, -2.67335, -2.25104
> > > iEMEt = 5.5,
> > > 5, 4.5, 4
> > > iEMPhi = 0.523599,
> > > -2.26893, 0.872665, 0.523599
> > > iEMEta = -1.218,
> > > -2.586, -1.566, -0.87
> > > iEMBit = 0,
> > > 0, 0, 0
> > > nEMEt = 0,
> > > 0, 0, 0
> > > nEMPhi = 2.61799,
> > > 999, 2.61799, 2.96706
> > > nEMEta = 1.956,
> > > 999, 2.586, 2.586
> > > nEMBit = 0,
> > > 0, 0, 0
> > > uEMEt = 0,
> > > 0, 0, 0
> > > uEMPhi = 999,
> > > 999, 999, 2.96706
> > > uEMEta = 999,
> > > 999, 999, 2.586
> > > uEMBit = 0,
> > > 0, 0, 0
> > > gJetEt = 45.7361,
> > > 45.0293, 1.02085, 0, 0, 0,
> > > 0, 0
> > > gJetPhi = 0.710199,
> > > -2.44848, -1.77321, 0, 0, 0,
> > > 0, 0
> > > gJetEta = -1.20366,
> > > -2.46634, -1.89238, 0, 0, 0,
> > > 0, 0
> > > gFJetEt = 0,
> > > 0, 0, 0
> > > gFJetEta = 0,
> > > 0, 0, 0
> > > gFJetPhi = 0,
> > > 0, 0, 0
> > > rJetEt = 12,
> > > 0, 0, 0
> > > rJetPhi = -2.26893,
> > > 0, 0, 0
> > > rJetEta = -2.586,
> > > 0, 0, 0
> > > fJetEt = 0,
> > > 0, 0, 0
> > > fJetEta = 0,
> > > 0, 0, 0
> > > fJetPhi = 0,
> > > 0, 0, 0
> > > gTauEt = 0,
> > > 0, 0, 0
> > > gTauEta = 0,
> > > 0, 0, 0
> > > gTauPhi = 0,
> > > 0, 0, 0
> > > rTauEt = 11,
> > > 0, 0, 0
> > > rTauEta = -1.218,
> > > 0, 0, 0
> > > rTauPhi = 0.523599,
> > > 0, 0, 0
> > > gPUPt = 0,
> > > 0, 0, 0, 0, 0,
> > > 0, 0, 0, 0, 0,
> > > 0, 0, 0, 0, 0,
> > > 0, 0, 0, 0
> > > gPt = 9.11696e+11
> > > nPU = 807425093
> > > gTotEt = 93.109
> > > gMisEt = 0.307852
> > > rTotEt = 23
> > > rMisEt = 1
> > > rHT = 23
> > >
> > > ======> EVENT:5000
> > > gEMEt = 5.65494,
> > > 3.76494, 2.90497, 2.36828
> > > gEMPhi = 0.673534,
> > > 0.931451, -2.11552, -2.36309
> > > gEMEta = -1.14793,
> > > -1.46625, -2.67335, -2.25104
> > > iEMEt = 3,
> > > 2.5, 1.5, 1
> > > iEMPhi = 0.872665,
> > > -2.26893, -2.61799, 0.872665
> > > iEMEta = -1.566,
> > > -2.586, -2.586, -1.218
> > > iEMBit = 0,
> > > 0, 0, 0
> > > nEMEt = 0,
> > > 0, 0, 0
> > > nEMPhi = 2.61799,
> > > 2.96706, 2.61799, 2.96706
> > > nEMEta = 1.956,
> > > 1.956, 2.586, 2.586
> > > nEMBit = 0,
> > > 0, 0, 0
> > > uEMEt = 0,
> > > 0, 0, 0
> > > uEMPhi = 2.61799,
> > > 2.96706, 999, 2.96706
> > > uEMEta = 1.956,
> > > 1.956, 999, 2.586
> > > uEMBit = 0,
> > > 0, 0, 0
> > > gJetEt = 45.7361,
> > > 45.0293, 1.02085, 0, 0, 0,
> > > 0, 0
> > > gJetPhi = 0.710199,
> > > -2.44848, -1.77321, 0, 0, 0,
> > > 0, 0
> > > gJetEta = -1.20366,
> > > -2.46634, -1.89238, 0, 0, 0,
> > > 0, 0
> > > gFJetEt = 0,
> > > 0, 0, 0
> > > gFJetEta = 0,
> > > 0, 0, 0
> > > gFJetPhi = 0,
> > > 0, 0, 0
> > > rJetEt = 0,
> > > 0, 0, 0
> > > rJetPhi = 0,
> > > 0, 0, 0
> > > rJetEta = 0,
> > > 0, 0, 0
> > > fJetEt = 0,
> > > 0, 0, 0
> > > fJetEta = 0,
> > > 0, 0, 0
> > > fJetPhi = 0,
> > > 0, 0, 0
> > > gTauEt = 0,
> > > 0, 0, 0
> > > gTauEta = 0,
> > > 0, 0, 0
> > > gTauPhi = 0,
> > > 0, 0, 0
> > > rTauEt = 8,
> > > 0, 0, 0
> > > rTauEta = 0.522,
> > > 0, 0, 0
> > > rTauPhi = 1.22173,
> > > 0, 0, 0
> > > gPUPt = 0,
> > > 0, 0, 0, 0, 0,
> > > 0, 0, 0, 0, 0,
> > > 0, 0, 0, 0, 0,
> > > 0, 0, 0, 0
> > > gPt = 9.11696e+11
> > > nPU = 807425093
> > > gTotEt = 93.109
> > > gMisEt = 0.307852
> > > rTotEt = 27
> > > rMisEt = 7
> > > rHT = 0
> > >
> > >
> > >
> > >
> > > On Thu, 22 Jul 2004, Rene Brun wrote:
> > >
> > > > This could be:
> > > > -a compression effect
> > > > -the fact that you do not reset some arrays
> > > >
> > > > Could you send
> > > > -the result of tree.Print() for 100, 1000 amd 10000 events
> > > > - for the file with 10000 events the result of
> > > > -tree.Show(50);
> > > > -tree.Show(500);
> > > > -tree.Show(5000);
> > > >
> > > >
> > > > Rene Brun
> > > >
> > > > Creighton Hogg wrote:
> > > > >
> > > > > Hello,
> > > > > I'm having trouble with unusually large root files being produced by an
> > > > > ORCA program I have written. It writes 6 floats and 34 arrays of 4 floats each
> > > > > per event and the file size seems to vary non-linearly with the number of
> > > > > events. With 100 events it is roughly 2Meg. With 1000 events it is
> > > > > roughly 70 Meg. With 10000 events it is roughly 1Gig.
> > > > > Does anyone have an idea why this would happen?
> > > > > Thanks
> > > >
> > > >
> > >
> > >
> > >
> >
> >
This archive was generated by hypermail 2b29 : Sun Jan 02 2005 - 05:50:09 MET