RE: [ROOT] trouble reading a large root file

From: Ed Oltman (eoltman@imago.com)
Date: Fri Nov 15 2002 - 17:03:05 MET


Rene,
 I'm wondering of the problem I've observed is a
threading problem: The difference between the mfc
and console applications is that in mfc, I use
multi-threaded dll runtime library and with the
console app, I use the single threaded non-dll
runtime library.  I have included a file which
includes my root wrapper class:

  main.cpp - console program, that works just fine
  rawdata.h - definitition of raw data file
  wrapper.h - definition of root wrapper class
  wrapper.cpp - root wrapper class

Thanks,
 Ed


> -----Original Message-----
> From: Ed Oltman
> Sent: Wednesday, November 13, 2002 5:26 PM
> To: Rene Brun
> Cc: Roottalk@Pcroot. Cern. Ch
> Subject: RE: [ROOT] trouble reading a large root file
>
>
> Hello Rene,
>  The application that generates the root file is an MFC application - I
> could not call m_Ntuple->Print().  So, I built a console application and
> ran that.   Strangly enough, the root code performed as it was expected -
> I was able to add a m_Ntuple->Print() and got the following.  When I went
> to read the ntuple in to root, everything worked fine.
>
> Very puzzling..
>
> Ed
>
>
>  the current keyboard layout is 437
> ******************************************************************
> ************
> *Tree    :nt        : Raw
>            *
> *Entries : 55798842 : Total =      2013592842 bytes  File  Size =
>  697021417 *
> *        :          : Tree compression factor =   2.89
>            *
> ******************************************************************
> ************
> *Br    0 :x         :
>            *
> *Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =
>  119308739 *
> *Baskets :     6987 : Basket Size=      32000 bytes  Compression=
>   1.87     *
> *.................................................................
> ...........*
> *Br    1 :y         :
>            *
> *Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =
>  120235511 *
> *Baskets :     6987 : Basket Size=      32000 bytes  Compression=
>   1.86     *
> *.................................................................
> ...........*
> *Br    2 :tof       :
>            *
> *Entries : 55798842 : Total  Size=  223734094 bytes  File Size  =
>  144517194 *
> *Baskets :     6988 : Basket Size=      32000 bytes  Compression=
>   1.55     *
> *.................................................................
> ...........*
> *Br    3 :z         :
>            *
> *Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =
>   22759189 *
> *Baskets :     6987 : Basket Size=      32000 bytes  Compression=
>   9.82     *
> *.................................................................
> ...........*
> *Br    4 :v         :
>            *
> *Entries : 55798842 : Total  Size=  223720044 bytes  File Size  =
>   11260843 *
> *Baskets :     6987 : Basket Size=      32000 bytes  Compression=
>  19.85     *
> *.................................................................
> ...........*
> *Br    5 :delta     :
>            *
> *Entries : 55798842 : Total  Size=  223748082 bytes  File Size  =
>   57303282 *
> *Baskets :     6988 : Basket Size=      32000 bytes  Compression=
>   3.90     *
> *.................................................................
> ...........*
> *Br    6 :pulse     :
>            *
> *Entries : 55798842 : Total  Size=  223748082 bytes  File Size  =
>   33798419 *
> *Baskets :     6988 : Basket Size=      32000 bytes  Compression=
>   6.62     *
> *.................................................................
> ...........*
> *Br    7 :vap       :
>            *
> *Entries : 55798842 : Total  Size=  223734094 bytes  File Size  =
>     849142 *
> *Baskets :     6988 : Basket Size=      32000 bytes  Compression=
> 263.31     *
> *.................................................................
> ...........*
> *Br    8 :erate     :
>            *
> *Entries : 55798842 : Total  Size=  223748082 bytes  File Size  =
>  186360721 *
> *Baskets :     6988 : Basket Size=      32000 bytes  Compression=
>   1.20     *
> *.................................................................
> ...........*
> Press any key to continue
>
> > -----Original Message-----
> > From: Rene Brun [mailto:brun@pcbrun.cern.ch]
> > Sent: Wednesday, November 13, 2002 11:59 AM
> > To: Ed Oltman
> > Cc: Roottalk@Pcroot. Cern. Ch
> > Subject: Re: [ROOT] trouble reading a large root file
> >
> >
> > Hi Ed,
> >
> > It looks like this file has been overwritten at some point.
> > To help understanding this problem, could you
> >  -add the statement m_Ntuple->Print() at the end of the job
> >   producing the ntuple, just before closing the file.
> >   and send me the output.
> >
> > If this does not crash, add the following statement between
> > m_Ntuple->Print() and closing the file:
> >  m_Ntuple_Draw("pulse");
> >
> > Is it possible to have access to your input file (gzipped)
> > and your small program reading it and converting it to Root?
> >
> > Rene Brun
> >
> > On Wed, 13 Nov 2002, Ed Oltman wrote:
> >
> > > Hello,
> > >  I am having trouble reading a "large" root file I generated
> > > with a stand-alone application (reads non-root data, produces
> > > a root file that consists of a couple of small histos and
> > > a TNtuple)  I've used this stand-alone program thousands of
> > > times over the past year with no problem.
> > >
> > > This time, the data is much larger there are approximately
> > > 56 million entries in the TNtuple, each with 9 variables.
> > > I created the TFile with
> > >
> > > 	m_hFile = new TFile(pname,"RECREATE","ap data");
> > >
> > > I created the TNtuple with
> > >
> > > 	m_Ntuple = new
> > TNtuple("nt","Raw","x:y:tof:z:v:delta:pulse:vap:erate");
> > >
> > > The program that generates the ntuple terminates normally..
> The output
> > > file is 696,229,888 Bytes - smaller than the 1900 MByte limit
> > of a TTree.
> > >
> > > When I open with root, I get the errors shown below.  Any ideas what I
> > > need to do to fix or diagnose the problem?  Thanks
> > >
> > > I use root v3.03/9a on win2k.
> > >
> > >
> > > Ed Oltman
> > >
> > >
> > > C:\root>root
> > >  the current keyboard layout is 437
> > >   *******************************************
> > >   *                                         *
> > >   *        W E L C O M E  to  R O O T       *
> > >   *                                         *
> > >   *   Version   3.03/09   29 October 2002   *
> > >   *                                         *
> > >   *  You are welcome to visit our Web site  *
> > >   *          http://root.cern.ch            *
> > >   *                                         *
> > >   *******************************************
> > >
> > > Compiled for win32.
> > >
> > > CINT/ROOT C/C++ Interpreter version 5.15.61, Oct 6 2002
> > > Type ? for help. Commands must be C++ statements.
> > > Enclose multiple statements between { }.
> > > root [0] TFile *f = new TFile("ap.root")
> > > Error in <TObjArray::At>: index 64 out of bounds (size: 13, this:
> > > 0x016aeb90)
> > > Warning in <TExMap::Remove>: key 131136 not found at 356
> > > Warning in <TBuffer::CheckObject>: reference to object of
> > unavailable class
> > > TObject, pointer will be
> > >  0
> > > Error in <TBuffer::CheckByteCount>: object of class
> > TStreamerInfo read too
> > > many bytes: 34 instead of
> > >  2
> > > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> > not in sync
> > > with data on file, fix S
> > > treamer()
> > > Warning in <TExMap::Remove>: key 131072 not found at 292
> > > Warning in <TBuffer::CheckObject>: reference to object of
> > unavailable class
> > > TObject, pointer will be
> > >  0
> > > Error in <TBuffer::CheckByteCount>: object of class
> > TStreamerInfo read too
> > > many bytes: 34 instead of
> > >  2
> > > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> > not in sync
> > > with data on file, fix S
> > > treamer()
> > > Error in <TBuffer::CheckByteCount>: object of class
> > TStreamerInfo read too
> > > many bytes: 30 instead of
> > >  2
> > > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> > not in sync
> > > with data on file, fix S
> > > treamer()
> > > Warning in <TExMap::Remove>: key 50399232 not found at 141
> > > Warning in <TBuffer::CheckObject>: reference to object of
> > unavailable class
> > > TObject, pointer will be
> > >  0
> > > Error in <TBuffer::CheckByteCount>: object of class
> > TStreamerInfo read too
> > > many bytes: 28 instead of
> > >  2
> > > Warning in <TBuffer::CheckByteCount>: TStreamerInfo::Streamer()
> > not in sync
> > > with data on file, fix S
> > > treamer()
> > > ERROR reading TStreamerInfo:  fClassVersion=-2147483557
> > > ERROR reading TStreamerInfo:  fClassVersion=-2147483557
> > > ERROR reading TStreamerInfo:  fClassVersion=385881856
> > > root [1]ERROR reading TStreamerInfo:  fClassVersion=385881856
> > >
> > >
> > > I tried
> > >
> > > f->Recover() and got the following:
> > > root [5] f->Recover()
> > > Info in <TFile::Recover>: recovered key TH1C:pHeader at address 192
> > > Info in <TFile::Recover>: recovered key TList:StreamerInfo at address
> > > 39341120
> > > Info in <TFile::Recover>: recovered key TNtuple:nt at address 77927255
> > > Info in <TFile::Recover>: recovered key TNtuple:nt at address
> 116052219
> > > Info in <TFile::Recover>: recovered key TH1F:pHits at address
> 696225755
> > > Info in <TFile::Recover>: recovered key TH1F:pDoubles at
> > address 696225803
> > > Info in <TFile::Recover>: recovered key TH1F:pFlag at address
> 696225861
> > > Info in <TFile::Recover>: recovered key TFile:C:\root\ap.root
> at address
> > > 696225909
> > > Info in <TFile::Recover>: recovered key TFile:C:\root\ap.root
> at address
> > > 696226249
> > > (Int_t)9
> > >
> > > but was still unable to do anything with the file..
> > >
> > > I tried f->Dump():
> > >
> > > root [11] f->Dump()
> > > ==>Dumping object at:989530, name=ap.root, class=TFile
> > >
> > > fSumBuffer                    0           Sum of buffer sizes
> of objects
> > > written so far
> > > fSum2Buffer                   0           Sum of squares of
> > buffer sizes of
> > > objects written so far
> > > fBytesWrite                   0           Number of bytes
> > written to this
> > > file
> > > fBytesRead                    6.4407e+007 Number of bytes
> read from this
> > > file
> > > fBEGIN                        64          First used byte in file
> > > fEND                          696226329   Last used byte in file
> > > fSeekFree                     696226249   Location on disk of
> > free segments
> > > structure
> > > fSeekInfo                     39341120    Location on disk of
> > StreamerInfo
> > > record
> > > fD                            7           File descriptor
> > > fVersion                      30309       File format version
> > > fCompress                     1           Compression level from 0(not
> > > compressed) to 9 (max compres
> > > sion)
> > > fNbytesFree                   76          Number of bytes for
> > free segments
> > > structure
> > > fNbytesInfo                   159         Number of bytes for
> > StreamerInfo
> > > record
> > > fWritten                      0           Number of objects
> > written so far
> > > fNProcessIDs                  0           Number of TProcessID
> > written to
> > > this file
> > > fOption                       ->9895ec    File options
> > > fOption.*fData                READ
> > > fUnits                        4           Number of bytes for
> > file pointers
> > > *fFree                        ->0         Free segments linked
> > list table
> > > *fClassIndex                  ->169ee68   !Index of
> > TStreamerInfo classes
> > > written to this file
> > > *fCache                       ->0         !Page cache used to
> > reduce number
> > > of small I/O's
> > > *fProcessIDs                  ->16adb20   !Array of pointers to
> > TProcessIDs
> > > fModified                     1           true if directory has been
> > > modified
> > > fWritable                     0           true if directory
> is writable
> > > fDatimeC                      ->989550    Date and time when
> > directory is
> > > created
> > > fDatimeC.fDatime              20021113/105303
> > > fDatimeM                      ->989558    Date and time of last
> > modification
> > > fDatimeM.fDatime              20021113/111332
> > > fNbytesKeys                   340         Number of bytes for the keys
> > > fNbytesName                   80          Number of bytes in TNamed at
> > > creation time
> > > fSeekDir                      64          Location of
> directory on file
> > > fSeekParent                   0           Location of parent
> > directory on
> > > file
> > > fSeekKeys                     696225909   Location of Keys
> > record on file
> > > *fFile                        ->989530    pointer to current
> > file in memory
> > > *fMother                      ->0         pointer to mother of
> > the directory
> > > *fList                        ->167cab8   Pointer to objects
> > list in memory
> > > *fKeys                        ->167cb28   Pointer to keys
> list in memory
> > > fUUID                         ->989584    Unique identifier
> > > fUUID.fUUIDIndex              0           !index in the list
> of UUIDs in
> > > TProcessUUID
> > > fUUID.fTimeLow                1607350938  60 bit time, lower 32 bits
> > > fUUID.fTimeMid                63272       middle 16 time bits
> > > fUUID.fTimeHiAndVersion       4566        high 12 time bits + 4
> > UUID version
> > > bits
> > > fUUID.fClockSeqHiAndReserved  171         high 6 clock bits + 2 bits
> > > reserved
> > > fUUID.fClockSeqLow            70          low 8 clock bits
> > > fUUID.fNode[6]                95          6 node id bytes
> > > fName                         ->98953c    object identifier
> > > fName.*fData                  ap.root
> > > fTitle                        ->989544    object title
> > > fTitle.*fData                 ap data
> > > fUniqueID                     0           object unique identifier
> > > fBits                         50332705    bit field status word
> > >
> > >
> >
> >
> >
> >





This archive was generated by hypermail 2b29 : Sat Jan 04 2003 - 23:51:18 MET