RE: [ROOT] trouble reading a large root file

From: Ed Oltman (eoltman@imago.com)
Date: Thu Nov 14 2002 - 00:35:05 MET


Hello Rene,
 The application that generates the root file is an MFC application - I
could not call m_Ntuple->Print().  So, I built a console application and
ran that.   Strangly enough, the root code performed as it was expected -
I was able to add a m_Ntuple->Print() and got the following.  When I went
to read the ntuple in to root, everything worked fine.

Very puzzling..

Ed


 the current keyboard layout is 437
****************************************************************************
**
*Tree    :nt        : Raw
*
*Entries : 55798842 : Total =      2013592842 bytes  File  Size =  697021417
*
*        :          : Tree compression factor =   2.89
*
****************************************************************************
**
*Br    0 :x         :
*
*Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =  119308739
*
*Baskets :     6987 : Basket Size=      32000 bytes  Compression=   1.87
*
*...........................................................................
.*
*Br    1 :y         :
*
*Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =  120235511
*
*Baskets :     6987 : Basket Size=      32000 bytes  Compression=   1.86
*
*...........................................................................
.*
*Br    2 :tof       :
*
*Entries : 55798842 : Total  Size=  223734094 bytes  File Size  =  144517194
*
*Baskets :     6988 : Basket Size=      32000 bytes  Compression=   1.55
*
*...........................................................................
.*
*Br    3 :z         :
*
*Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =   22759189
*
*Baskets :     6987 : Basket Size=      32000 bytes  Compression=   9.82
*
*...........................................................................
.*
*Br    4 :v         :
*
*Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =   11260843
*
*Baskets :     6987 : Basket Size=      32000 bytes  Compression=  19.85
*
*...........................................................................
.*
*Br    5 :delta     :
*
*Entries : 55798842 : Total  Size=  223748082 bytes  File Size  =   57303282
*
*Baskets :     6988 : Basket Size=      32000 bytes  Compression=   3.90
*
*...........................................................................
.*
*Br    6 :pulse     :
*
*Entries : 55798842 : Total  Size=  223748082 bytes  File Size  =   33798419
*
*Baskets :     6988 : Basket Size=      32000 bytes  Compression=   6.62
*
*...........................................................................
.*
*Br    7 :vap       :
*
*Entries : 55798842 : Total  Size=  223734094 bytes  File Size  =     849142
*
*Baskets :     6988 : Basket Size=      32000 bytes  Compression= 263.31
*
*...........................................................................
.*
*Br    8 :erate     :
*
*Entries : 55798842 : Total  Size=  223748082 bytes  File Size  =  186360721
*
*Baskets :     6988 : Basket Size=      32000 bytes  Compression=   1.20
*
*...........................................................................
.*
Press any key to continue

> -----Original Message-----
> From: Rene Brun [mailto:brun@pcbrun.cern.ch]
> Sent: Wednesday, November 13, 2002 11:59 AM
> To: Ed Oltman
> Cc: Roottalk@Pcroot. Cern. Ch
> Subject: Re: [ROOT] trouble reading a large root file
>
>
> Hi Ed,
>
> It looks like this file has been overwritten at some point.
> To help understanding this problem, could you
>  -add the statement m_Ntuple->Print() at the end of the job
>   producing the ntuple, just before closing the file.
>   and send me the output.
>
> If this does not crash, add the following statement between
> m_Ntuple->Print() and closing the file:
>  m_Ntuple_Draw("pulse");
>
> Is it possible to have access to your input file (gzipped)
> and your small program reading it and converting it to Root?
>
> Rene Brun
>
> On Wed, 13 Nov 2002, Ed Oltman wrote:
>
> > Hello,
> >  I am having trouble reading a "large" root file I generated
> > with a stand-alone application (reads non-root data, produces
> > a root file that consists of a couple of small histos and
> > a TNtuple)  I've used this stand-alone program thousands of
> > times over the past year with no problem.
> >
> > This time, the data is much larger there are approximately
> > 56 million entries in the TNtuple, each with 9 variables.
> > I created the TFile with
> >
> > 	m_hFile = new TFile(pname,"RECREATE","ap data");
> >
> > I created the TNtuple with
> >
> > 	m_Ntuple = new
> TNtuple("nt","Raw","x:y:tof:z:v:delta:pulse:vap:erate");
> >
> > The program that generates the ntuple terminates normally..  The output
> > file is 696,229,888 Bytes - smaller than the 1900 MByte limit
> of a TTree.
> >
> > When I open with root, I get the errors shown below.  Any ideas what I
> > need to do to fix or diagnose the problem?  Thanks
> >
> > I use root v3.03/9a on win2k.
> >
> >
> > Ed Oltman
> >
> >
> > C:\root>root
> >  the current keyboard layout is 437
> >   *******************************************
> >   *                                         *
> >   *        W E L C O M E  to  R O O T       *
> >   *                                         *
> >   *   Version   3.03/09   29 October 2002   *
> >   *                                         *
> >   *  You are welcome to visit our Web site  *
> >   *          http://root.cern.ch            *
> >   *                                         *
> >   *******************************************
> >
> > Compiled for win32.
> >
> > CINT/ROOT C/C++ Interpreter version 5.15.61, Oct 6 2002
> > Type ? for help. Commands must be C++ statements.
> > Enclose multiple statements between { }.
> > root [0] TFile *f = new TFile("ap.root")
> > Error in <TObjArray::At>: index 64 out of bounds (size: 13, this:
> > 0x016aeb90)
> > Warning in <TExMap::Remove>: key 131136 not found at 356
> > Warning in <TBuffer::CheckObject>: reference to object of
> unavailable class
> > TObject, pointer will be
> >  0
> > Error in <TBuffer::CheckByteCount>: object of class
> TStreamerInfo read too
> > many bytes: 34 instead of
> >  2
> > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> not in sync
> > with data on file, fix S
> > treamer()
> > Warning in <TExMap::Remove>: key 131072 not found at 292
> > Warning in <TBuffer::CheckObject>: reference to object of
> unavailable class
> > TObject, pointer will be
> >  0
> > Error in <TBuffer::CheckByteCount>: object of class
> TStreamerInfo read too
> > many bytes: 34 instead of
> >  2
> > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> not in sync
> > with data on file, fix S
> > treamer()
> > Error in <TBuffer::CheckByteCount>: object of class
> TStreamerInfo read too
> > many bytes: 30 instead of
> >  2
> > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> not in sync
> > with data on file, fix S
> > treamer()
> > Warning in <TExMap::Remove>: key 50399232 not found at 141
> > Warning in <TBuffer::CheckObject>: reference to object of
> unavailable class
> > TObject, pointer will be
> >  0
> > Error in <TBuffer::CheckByteCount>: object of class
> TStreamerInfo read too
> > many bytes: 28 instead of
> >  2
> > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> not in sync
> > with data on file, fix S
> > treamer()
> > ERROR reading TStreamerInfo:  fClassVersion=-2147483557
> > ERROR reading TStreamerInfo:  fClassVersion=-2147483557
> > ERROR reading TStreamerInfo:  fClassVersion=385881856
> > root [1]ERROR reading TStreamerInfo:  fClassVersion=385881856
> >
> >
> > I tried
> >
> > f->Recover() and got the following:
> > root [5] f->Recover()
> > Info in <TFile::Recover>: recovered key TH1C:pHeader at address 192
> > Info in <TFile::Recover>: recovered key TList:StreamerInfo at address
> > 39341120
> > Info in <TFile::Recover>: recovered key TNtuple:nt at address 77927255
> > Info in <TFile::Recover>: recovered key TNtuple:nt at address 116052219
> > Info in <TFile::Recover>: recovered key TH1F:pHits at address 696225755
> > Info in <TFile::Recover>: recovered key TH1F:pDoubles at
> address 696225803
> > Info in <TFile::Recover>: recovered key TH1F:pFlag at address 696225861
> > Info in <TFile::Recover>: recovered key TFile:C:\root\ap.root at address
> > 696225909
> > Info in <TFile::Recover>: recovered key TFile:C:\root\ap.root at address
> > 696226249
> > (Int_t)9
> >
> > but was still unable to do anything with the file..
> >
> > I tried f->Dump():
> >
> > root [11] f->Dump()
> > ==>Dumping object at:989530, name=ap.root, class=TFile
> >
> > fSumBuffer                    0           Sum of buffer sizes of objects
> > written so far
> > fSum2Buffer                   0           Sum of squares of
> buffer sizes of
> > objects written so far
> > fBytesWrite                   0           Number of bytes
> written to this
> > file
> > fBytesRead                    6.4407e+007 Number of bytes read from this
> > file
> > fBEGIN                        64          First used byte in file
> > fEND                          696226329   Last used byte in file
> > fSeekFree                     696226249   Location on disk of
> free segments
> > structure
> > fSeekInfo                     39341120    Location on disk of
> StreamerInfo
> > record
> > fD                            7           File descriptor
> > fVersion                      30309       File format version
> > fCompress                     1           Compression level from 0(not
> > compressed) to 9 (max compres
> > sion)
> > fNbytesFree                   76          Number of bytes for
> free segments
> > structure
> > fNbytesInfo                   159         Number of bytes for
> StreamerInfo
> > record
> > fWritten                      0           Number of objects
> written so far
> > fNProcessIDs                  0           Number of TProcessID
> written to
> > this file
> > fOption                       ->9895ec    File options
> > fOption.*fData                READ
> > fUnits                        4           Number of bytes for
> file pointers
> > *fFree                        ->0         Free segments linked
> list table
> > *fClassIndex                  ->169ee68   !Index of
> TStreamerInfo classes
> > written to this file
> > *fCache                       ->0         !Page cache used to
> reduce number
> > of small I/O's
> > *fProcessIDs                  ->16adb20   !Array of pointers to
> TProcessIDs
> > fModified                     1           true if directory has been
> > modified
> > fWritable                     0           true if directory is writable
> > fDatimeC                      ->989550    Date and time when
> directory is
> > created
> > fDatimeC.fDatime              20021113/105303
> > fDatimeM                      ->989558    Date and time of last
> modification
> > fDatimeM.fDatime              20021113/111332
> > fNbytesKeys                   340         Number of bytes for the keys
> > fNbytesName                   80          Number of bytes in TNamed at
> > creation time
> > fSeekDir                      64          Location of directory on file
> > fSeekParent                   0           Location of parent
> directory on
> > file
> > fSeekKeys                     696225909   Location of Keys
> record on file
> > *fFile                        ->989530    pointer to current
> file in memory
> > *fMother                      ->0         pointer to mother of
> the directory
> > *fList                        ->167cab8   Pointer to objects
> list in memory
> > *fKeys                        ->167cb28   Pointer to keys list in memory
> > fUUID                         ->989584    Unique identifier
> > fUUID.fUUIDIndex              0           !index in the list of UUIDs in
> > TProcessUUID
> > fUUID.fTimeLow                1607350938  60 bit time, lower 32 bits
> > fUUID.fTimeMid                63272       middle 16 time bits
> > fUUID.fTimeHiAndVersion       4566        high 12 time bits + 4
> UUID version
> > bits
> > fUUID.fClockSeqHiAndReserved  171         high 6 clock bits + 2 bits
> > reserved
> > fUUID.fClockSeqLow            70          low 8 clock bits
> > fUUID.fNode[6]                95          6 node id bytes
> > fName                         ->98953c    object identifier
> > fName.*fData                  ap.root
> > fTitle                        ->989544    object title
> > fTitle.*fData                 ap data
> > fUniqueID                     0           object unique identifier
> > fBits                         50332705    bit field status word
> >
> >
>
>
>
>



This archive was generated by hypermail 2b29 : Sat Jan 04 2003 - 23:51:17 MET