RE: [ROOT] trouble reading a large root file

From: Faine, Valeri (fine@bnl.gov)
Date: Fri Nov 15 2002 - 20:11:07 MET


Hi Ed,
ROOT and all its DLL's are multithread binaries.
You have to compile your portion against of the same share C++ run-time
library also. This means you have to apply "multithread" option even
there is no thread in your piece of the software.
I am not sure whether this would cure the problem but definitely it
helps to detect the real one if any.

  Can you try that?
                  Best regards, Valeri

> -----Original Message-----
> From: owner-roottalk@pcroot.cern.ch
[mailto:owner-roottalk@pcroot.cern.ch]
> On Behalf Of Ed Oltman
> Sent: Friday, November 15, 2002 11:03 AM
> To: Rene Brun
> Cc: Roottalk@Pcroot. Cern. Ch
> Subject: RE: [ROOT] trouble reading a large root file
> 
> Rene,
>  I'm wondering of the problem I've observed is a
> threading problem: The difference between the mfc
> and console applications is that in mfc, I use
> multi-threaded dll runtime library and with the
> console app, I use the single threaded non-dll
> runtime library.  I have included a file which
> includes my root wrapper class:
> 
>   main.cpp - console program, that works just fine
>   rawdata.h - definitition of raw data file
>   wrapper.h - definition of root wrapper class
>   wrapper.cpp - root wrapper class
> 
> Thanks,
>  Ed
> 
> 
> > -----Original Message-----
> > From: Ed Oltman
> > Sent: Wednesday, November 13, 2002 5:26 PM
> > To: Rene Brun
> > Cc: Roottalk@Pcroot. Cern. Ch
> > Subject: RE: [ROOT] trouble reading a large root file
> >
> >
> > Hello Rene,
> >  The application that generates the root file is an MFC application
- I
> > could not call m_Ntuple->Print().  So, I built a console application
and
> > ran that.   Strangly enough, the root code performed as it was
expected
> -
> > I was able to add a m_Ntuple->Print() and got the following.  When I
> went
> > to read the ntuple in to root, everything worked fine.
> >
> > Very puzzling..
> >
> > Ed
> >
> >
> >  the current keyboard layout is 437
> > ******************************************************************
> > ************
> > *Tree    :nt        : Raw
> >            *
> > *Entries : 55798842 : Total =      2013592842 bytes  File  Size =
> >  697021417 *
> > *        :          : Tree compression factor =   2.89
> >            *
> > ******************************************************************
> > ************
> > *Br    0 :x         :
> >            *
> > *Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =
> >  119308739 *
> > *Baskets :     6987 : Basket Size=      32000 bytes  Compression=
> >   1.87     *
> > *.................................................................
> > ...........*
> > *Br    1 :y         :
> >            *
> > *Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =
> >  120235511 *
> > *Baskets :     6987 : Basket Size=      32000 bytes  Compression=
> >   1.86     *
> > *.................................................................
> > ...........*
> > *Br    2 :tof       :
> >            *
> > *Entries : 55798842 : Total  Size=  223734094 bytes  File Size  =
> >  144517194 *
> > *Baskets :     6988 : Basket Size=      32000 bytes  Compression=
> >   1.55     *
> > *.................................................................
> > ...........*
> > *Br    3 :z         :
> >            *
> > *Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =
> >   22759189 *
> > *Baskets :     6987 : Basket Size=      32000 bytes  Compression=
> >   9.82     *
> > *.................................................................
> > ...........*
> > *Br    4 :v         :
> >            *
> > *Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =
> >   11260843 *
> > *Baskets :     6987 : Basket Size=      32000 bytes  Compression=
> >  19.85     *
> > *.................................................................
> > ...........*
> > *Br    5 :delta     :
> >            *
> > *Entries : 55798842 : Total  Size=  223748082 bytes  File Size  =
> >   57303282 *
> > *Baskets :     6988 : Basket Size=      32000 bytes  Compression=
> >   3.90     *
> > *.................................................................
> > ...........*
> > *Br    6 :pulse     :
> >            *
> > *Entries : 55798842 : Total  Size=  223748082 bytes  File Size  =
> >   33798419 *
> > *Baskets :     6988 : Basket Size=      32000 bytes  Compression=
> >   6.62     *
> > *.................................................................
> > ...........*
> > *Br    7 :vap       :
> >            *
> > *Entries : 55798842 : Total  Size=  223734094 bytes  File Size  =
> >     849142 *
> > *Baskets :     6988 : Basket Size=      32000 bytes  Compression=
> > 263.31     *
> > *.................................................................
> > ...........*
> > *Br    8 :erate     :
> >            *
> > *Entries : 55798842 : Total  Size=  223748082 bytes  File Size  =
> >  186360721 *
> > *Baskets :     6988 : Basket Size=      32000 bytes  Compression=
> >   1.20     *
> > *.................................................................
> > ...........*
> > Press any key to continue
> >
> > > -----Original Message-----
> > > From: Rene Brun [mailto:brun@pcbrun.cern.ch]
> > > Sent: Wednesday, November 13, 2002 11:59 AM
> > > To: Ed Oltman
> > > Cc: Roottalk@Pcroot. Cern. Ch
> > > Subject: Re: [ROOT] trouble reading a large root file
> > >
> > >
> > > Hi Ed,
> > >
> > > It looks like this file has been overwritten at some point.
> > > To help understanding this problem, could you
> > >  -add the statement m_Ntuple->Print() at the end of the job
> > >   producing the ntuple, just before closing the file.
> > >   and send me the output.
> > >
> > > If this does not crash, add the following statement between
> > > m_Ntuple->Print() and closing the file:
> > >  m_Ntuple_Draw("pulse");
> > >
> > > Is it possible to have access to your input file (gzipped)
> > > and your small program reading it and converting it to Root?
> > >
> > > Rene Brun
> > >
> > > On Wed, 13 Nov 2002, Ed Oltman wrote:
> > >
> > > > Hello,
> > > >  I am having trouble reading a "large" root file I generated
> > > > with a stand-alone application (reads non-root data, produces
> > > > a root file that consists of a couple of small histos and
> > > > a TNtuple)  I've used this stand-alone program thousands of
> > > > times over the past year with no problem.
> > > >
> > > > This time, the data is much larger there are approximately
> > > > 56 million entries in the TNtuple, each with 9 variables.
> > > > I created the TFile with
> > > >
> > > > 	m_hFile = new TFile(pname,"RECREATE","ap data");
> > > >
> > > > I created the TNtuple with
> > > >
> > > > 	m_Ntuple = new
> > > TNtuple("nt","Raw","x:y:tof:z:v:delta:pulse:vap:erate");
> > > >
> > > > The program that generates the ntuple terminates normally..
> > The output
> > > > file is 696,229,888 Bytes - smaller than the 1900 MByte limit
> > > of a TTree.
> > > >
> > > > When I open with root, I get the errors shown below.  Any ideas
what
> I
> > > > need to do to fix or diagnose the problem?  Thanks
> > > >
> > > > I use root v3.03/9a on win2k.
> > > >
> > > >
> > > > Ed Oltman
> > > >
> > > >
> > > > C:\root>root
> > > >  the current keyboard layout is 437
> > > >   *******************************************
> > > >   *                                         *
> > > >   *        W E L C O M E  to  R O O T       *
> > > >   *                                         *
> > > >   *   Version   3.03/09   29 October 2002   *
> > > >   *                                         *
> > > >   *  You are welcome to visit our Web site  *
> > > >   *          http://root.cern.ch            *
> > > >   *                                         *
> > > >   *******************************************
> > > >
> > > > Compiled for win32.
> > > >
> > > > CINT/ROOT C/C++ Interpreter version 5.15.61, Oct 6 2002
> > > > Type ? for help. Commands must be C++ statements.
> > > > Enclose multiple statements between { }.
> > > > root [0] TFile *f = new TFile("ap.root")
> > > > Error in <TObjArray::At>: index 64 out of bounds (size: 13,
this:
> > > > 0x016aeb90)
> > > > Warning in <TExMap::Remove>: key 131136 not found at 356
> > > > Warning in <TBuffer::CheckObject>: reference to object of
> > > unavailable class
> > > > TObject, pointer will be
> > > >  0
> > > > Error in <TBuffer::CheckByteCount>: object of class
> > > TStreamerInfo read too
> > > > many bytes: 34 instead of
> > > >  2
> > > > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> > > not in sync
> > > > with data on file, fix S
> > > > treamer()
> > > > Warning in <TExMap::Remove>: key 131072 not found at 292
> > > > Warning in <TBuffer::CheckObject>: reference to object of
> > > unavailable class
> > > > TObject, pointer will be
> > > >  0
> > > > Error in <TBuffer::CheckByteCount>: object of class
> > > TStreamerInfo read too
> > > > many bytes: 34 instead of
> > > >  2
> > > > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> > > not in sync
> > > > with data on file, fix S
> > > > treamer()
> > > > Error in <TBuffer::CheckByteCount>: object of class
> > > TStreamerInfo read too
> > > > many bytes: 30 instead of
> > > >  2
> > > > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> > > not in sync
> > > > with data on file, fix S
> > > > treamer()
> > > > Warning in <TExMap::Remove>: key 50399232 not found at 141
> > > > Warning in <TBuffer::CheckObject>: reference to object of
> > > unavailable class
> > > > TObject, pointer will be
> > > >  0
> > > > Error in <TBuffer::CheckByteCount>: object of class
> > > TStreamerInfo read too
> > > > many bytes: 28 instead of
> > > >  2
> > > > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> > > not in sync
> > > > with data on file, fix S
> > > > treamer()
> > > > ERROR reading TStreamerInfo:  fClassVersion=-2147483557
> > > > ERROR reading TStreamerInfo:  fClassVersion=-2147483557
> > > > ERROR reading TStreamerInfo:  fClassVersion=385881856
> > > > root [1]ERROR reading TStreamerInfo:  fClassVersion=385881856
> > > >
> > > >
> > > > I tried
> > > >
> > > > f->Recover() and got the following:
> > > > root [5] f->Recover()
> > > > Info in <TFile::Recover>: recovered key TH1C:pHeader at address
192
> > > > Info in <TFile::Recover>: recovered key TList:StreamerInfo at
> address
> > > > 39341120
> > > > Info in <TFile::Recover>: recovered key TNtuple:nt at address
> 77927255
> > > > Info in <TFile::Recover>: recovered key TNtuple:nt at address
> > 116052219
> > > > Info in <TFile::Recover>: recovered key TH1F:pHits at address
> > 696225755
> > > > Info in <TFile::Recover>: recovered key TH1F:pDoubles at
> > > address 696225803
> > > > Info in <TFile::Recover>: recovered key TH1F:pFlag at address
> > 696225861
> > > > Info in <TFile::Recover>: recovered key TFile:C:\root\ap.root
> > at address
> > > > 696225909
> > > > Info in <TFile::Recover>: recovered key TFile:C:\root\ap.root
> > at address
> > > > 696226249
> > > > (Int_t)9
> > > >
> > > > but was still unable to do anything with the file..
> > > >
> > > > I tried f->Dump():
> > > >
> > > > root [11] f->Dump()
> > > > ==>Dumping object at:989530, name=ap.root, class=TFile
> > > >
> > > > fSumBuffer                    0           Sum of buffer sizes
> > of objects
> > > > written so far
> > > > fSum2Buffer                   0           Sum of squares of
> > > buffer sizes of
> > > > objects written so far
> > > > fBytesWrite                   0           Number of bytes
> > > written to this
> > > > file
> > > > fBytesRead                    6.4407e+007 Number of bytes
> > read from this
> > > > file
> > > > fBEGIN                        64          First used byte in
file
> > > > fEND                          696226329   Last used byte in file
> > > > fSeekFree                     696226249   Location on disk of
> > > free segments
> > > > structure
> > > > fSeekInfo                     39341120    Location on disk of
> > > StreamerInfo
> > > > record
> > > > fD                            7           File descriptor
> > > > fVersion                      30309       File format version
> > > > fCompress                     1           Compression level from
> 0(not
> > > > compressed) to 9 (max compres
> > > > sion)
> > > > fNbytesFree                   76          Number of bytes for
> > > free segments
> > > > structure
> > > > fNbytesInfo                   159         Number of bytes for
> > > StreamerInfo
> > > > record
> > > > fWritten                      0           Number of objects
> > > written so far
> > > > fNProcessIDs                  0           Number of TProcessID
> > > written to
> > > > this file
> > > > fOption                       ->9895ec    File options
> > > > fOption.*fData                READ
> > > > fUnits                        4           Number of bytes for
> > > file pointers
> > > > *fFree                        ->0         Free segments linked
> > > list table
> > > > *fClassIndex                  ->169ee68   !Index of
> > > TStreamerInfo classes
> > > > written to this file
> > > > *fCache                       ->0         !Page cache used to
> > > reduce number
> > > > of small I/O's
> > > > *fProcessIDs                  ->16adb20   !Array of pointers to
> > > TProcessIDs
> > > > fModified                     1           true if directory has
been
> > > > modified
> > > > fWritable                     0           true if directory
> > is writable
> > > > fDatimeC                      ->989550    Date and time when
> > > directory is
> > > > created
> > > > fDatimeC.fDatime              20021113/105303
> > > > fDatimeM                      ->989558    Date and time of last
> > > modification
> > > > fDatimeM.fDatime              20021113/111332
> > > > fNbytesKeys                   340         Number of bytes for
the
> keys
> > > > fNbytesName                   80          Number of bytes in
TNamed
> at
> > > > creation time
> > > > fSeekDir                      64          Location of
> > directory on file
> > > > fSeekParent                   0           Location of parent
> > > directory on
> > > > file
> > > > fSeekKeys                     696225909   Location of Keys
> > > record on file
> > > > *fFile                        ->989530    pointer to current
> > > file in memory
> > > > *fMother                      ->0         pointer to mother of
> > > the directory
> > > > *fList                        ->167cab8   Pointer to objects
> > > list in memory
> > > > *fKeys                        ->167cb28   Pointer to keys
> > list in memory
> > > > fUUID                         ->989584    Unique identifier
> > > > fUUID.fUUIDIndex              0           !index in the list
> > of UUIDs in
> > > > TProcessUUID
> > > > fUUID.fTimeLow                1607350938  60 bit time, lower 32
bits
> > > > fUUID.fTimeMid                63272       middle 16 time bits
> > > > fUUID.fTimeHiAndVersion       4566        high 12 time bits + 4
> > > UUID version
> > > > bits
> > > > fUUID.fClockSeqHiAndReserved  171         high 6 clock bits + 2
bits
> > > > reserved
> > > > fUUID.fClockSeqLow            70          low 8 clock bits
> > > > fUUID.fNode[6]                95          6 node id bytes
> > > > fName                         ->98953c    object identifier
> > > > fName.*fData                  ap.root
> > > > fTitle                        ->989544    object title
> > > > fTitle.*fData                 ap data
> > > > fUniqueID                     0           object unique
identifier
> > > > fBits                         50332705    bit field status word
> > > >
> > > >
> > >
> > >
> > >
> > >



This archive was generated by hypermail 2b29 : Sat Jan 04 2003 - 23:51:18 MET