Hi Ruben,
> But I did not realise that in order to
> profit from clones new/delete mechanism the branch must be split.
This is not quite that simple. In order to be able to take
advantage of the "clones new/delete mechanism", you __only__
need to insure that the TClonesArray is 'reused' and not
regenerated. One case where this should be true is the case
where the TClonesArray object has its own branch but this not
the only case.
> But I noticed that even in this case the TClonesArrays were recreated
> at every GetEntry()
This is surprising. Could you send me your current code?
Cheers,
Philippe.
-----Original Message-----
From: Ruben Shahoyan [mailto:Ruben.Shahoyan@cern.ch]
Sent: Sunday, March 28, 2004 4:17 PM
To: Philippe Canal
Cc: Ruben Shahoyan; Rene Brun; roottalk@cern.ch
Subject: RE: [ROOT] Problem with Automatic schema evolution?
Hi Philippe,
> You found a problem we have with schema evolution when the objects
> are stored non-split and member-wise in a TClonesArray and the base class
> of the object changes.
> A patch will be uploaded shortly to allow the reading of your test1.root
> file in a different session. You will still be unable to read it back
> in the same session where you read the file test0.root.
>
Thank you, reading in new version for me is enough.
> On the other hand, I am not sure that you used the feature you intented.
> You are currently using a TObjArray of TClonesArray, this effectively
> disable your ability to split the TClonesArray and remove their ability
> to 'save' on new and deletes (because the TObjArray can not be split and
> hence the TClonesArray are recreated when each entry is read).
>
The splitting by itself was not an issue in our application, since we
always read the digits of the entire Detector class (as a collection of
sensor's data of given detector). But I did not realise that in order to
profit from clones new/delete mechanism the branch must be split (now I
understand why is it so). Following to your suggestion I have changed
the branch initialization a la
TTree::Branch(Detector,32000,kSplitLev), with splitlev=1
But I noticed that even in this case the TClonesArrays were recreated
at every GetEntry(). I managed to force reusing of arrays only after
setting manually TBranch::SetAutoDelete(kFALSE) for each branch created
by by the command above. At the same time documentation claims that the
default mode is AutoDelete == kFALSE, but I see in the Branch streamer
code that in certain situations the streamer itself sets kAutoDelete to
True. Aparently this happens in my case. I was wondering if this is a
safe operation: to redefine the AutoDelete when it was set by streamer?
Regards,
Ruben
> Instead of using the line:
> OutTree->Branch("detBranch","TObjArray",&Detector,32000,kSplitLev);
> you may have intended to use
> OutTree->Branch(Detector,32000,kSplitLev);
> which would created one branch per TClonesArray.
> To make this syntax more usefull you can also rename the TClonesArray
> (i.e. call SetName on the TClonesArray object). This name will be
> used as the name of the branch.
>
> If your intention was to have just one branch, then you can work-around
> the problem completely by either replaceing the TClonesArray by another
> container or by calling clones->BypassStreamer(kFALSE) on each clones.
>
> Cheers,
> Philippe.
>
> -----Original Message-----
> From: owner-roottalk@pcroot.cern.ch
> [mailto:owner-roottalk@pcroot.cern.ch]On Behalf Of Ruben Shahoyan
> Sent: Thursday, March 25, 2004 9:00 AM
> To: Rene Brun
> Cc: roottalk@cern.ch
> Subject: Re: [ROOT] Problem with Automatic schema evolution?
>
>
> Hello Rene,
>
> On Thu, 25 Mar 2004, Rene Brun wrote:
> > Could you specify which version of ROOT?
> > In case you use an old version, could you try with 3.10/02 or better
> 4.00/03?
>
> I am using root3.05/02, but I checked, the same happens in 3.10.
>
> > Could you send a small tar file with teh strict minimum to reproduce this
> > problem?
>
> I attached the code to reproduce the problem.
>
> 1) First you should generate the file with digits of version 1, i.e.
> #define STOREFULLINFO in the TestDigit.h is commented:
>
> ~/NA60/test > root
> *******************************************
> * *
> * W E L C O M E to R O O T *
> * *
> * Version 3.05/02 11 February 2003 *
> * *
> * You are welcome to visit our Web site *
> * http://root.cern.ch *
> * *
> *******************************************
> ..
> root [0] .L TestDigit.cxx++
> Info in <TUnixSystem::ACLiC>: creating shared library
> /home/shahoian/NA60/test/./TestDigit_cxx.so
> root [1] TestGen("test0.root"); // THIS WILL GENERATE DIGITS OF VERSION 1
> Event 0
> Col# 0 Row# 18 | Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
> Col# 10 Row# 4 | Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> Col# 11 Row# 9 | Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> Col# 12 Row# 1 | Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> Event 1
> Col# 100 Row# 30 | Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
> Col# 110 Row# 18 | Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> Col# 111 Row# 30 | Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> Col# 112 Row# 7 | Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> Col# 113 Row# 26 | Digit# 3 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> root [2] .q
>
> 2) Uncomment #define STOREFULLINFO in the TestDigit.h, read just generated
> digits and store
> them in a new file, as a digits of version 2
>
> ~/NA60/test > root
> ...
> root [0] .L TestDigit.cxx++
> Info in <TUnixSystem::ACLiC>: creating shared library
> /home/shahoian/NA60/test/./TestDigit_cxx.so
> root [1] TestReadWrite("test0.root","test1.root"); // THIS WILL CONVERT THE
> FILE WITH DIGITS_V1 TO FILE OF DIGITS_V2
> Event 0
> Col# 0 Row# 18 | Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 10 Row# 4 | Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 11 Row# 9 | Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 12 Row# 1 | Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Event 1
> Col# 100 Row# 30 | Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 110 Row# 18 | Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 111 Row# 30 | Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 112 Row# 7 | Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 113 Row# 26 | Digit# 3 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> root [2]
> //
> // as you see, the reading of v1 digits with the code of v2 works fine, we
> recover old values. The problem
> // arises when we try to read just stored digits. If it is done in the same
> root session, we get:
> //
> root [2] TestRead("test1.root")
> Event 0
> Error in <TBuffer::CheckByteCount>: object of class TClonesArray read too
> few bytes: 71 instead of 87
> Warning in <TBuffer::CheckByteCount>: TClonesArray::Streamer() not in sync
> with data on file, fix Streamer()
> Error in <TBuffer::CheckByteCount>: object of class TClonesArray read too
> few bytes: 123 instead of 171
> Warning in <TBuffer::CheckByteCount>: TClonesArray::Streamer() not in sync
> with data on file, fix Streamer()
> Col# 0 Row# 18 | Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 0 Row# 4 | Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 0 Row# 9 | Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 0 Row# 1 | Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Event 1
> Error in <TBuffer::CheckByteCount>: object of class TClonesArray read too
> few bytes: 71 instead of 87
> Warning in <TBuffer::CheckByteCount>: TClonesArray::Streamer() not in sync
> with data on file, fix Streamer()
> Error in <TBuffer::CheckByteCount>: object of class TClonesArray read too
> few bytes: 149 instead of 213
> Warning in <TBuffer::CheckByteCount>: TClonesArray::Streamer() not in sync
> with data on file, fix Streamer()
> Col# 0 Row# 30 | Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 0 Row# 18 | Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 0 Row# 30 | Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 0 Row# 7 | Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 0 Row# 26 | Digit# 3 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> root [3] .q
>
> 3) if the just converted digits are read from new root session, we get:
> ~/NA60/test > root
> root [0] .L TestDigit.cxx++
> root [1] TestRead("test1.root")
> Event 0
> Col# 0 Row# 18 | Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 10 Row# 4 | Digit# 0 of Sensor 1 | In Clus# 9, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 11 Row# 1 | Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 12 Row# -1 | Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Event 1
> Col# 100 Row# 30 | Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 110 Row# 18 | Digit# 0 of Sensor 1 | In Clus# 30, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 111 Row# 7 | Digit# 1 of Sensor 1 | In Clus# 26, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 112 Row# -1 | Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
> Col# 113 Row# -1 | Digit# 3 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
> X: 0.0 Y: 0.0 Z: 0.0 E: 0.0
>
> The data member fI of NaDigit1D class is corrupted: it looks like due to the
> wrong offsets it mixes
> the fCluster member (which in this example should be always -1) and fI.
>
> Best regards,
> Ruben
>
> --
>
>
> > Ruben Shahoyan wrote:
> > >
> > > Hello,
> > > we have a following problem with I/O of the objects having streamer
> > > versions:
> > > the base object for na60 digits is
> > >
> > > class NaDigit1D :
> > > public TObject {
> > > ...
> > > protected:
> > > Int_t fI; // Value of the digit
> > > Int_t fCluster; // Number of the Cluster to which it is attached
> > > TArrayI* fTracks; // Optional array of references on track
> > > //
> > > #ifdef STOREFULLINFO
> > > Float_t fXYZE[4];
> > > ClassDef(NaDigit1D,2) // Base class for 1Dim digits with FULL TRACKING
> INFO
> > > #else
> > > ClassDef(NaDigit1D,1) // Base class for 1Dim digits
> > > #endif
> > > };
> > >
> > > the #define STOREFULLINFO in the NaDigit1D.h allows us to switch from
> > > version 1 to more complete version 2.
> > >
> > > The default constructor is:
> > > NaDigit1D::NaDigit1D()
> > > {
> > > fI = -1;
> > > fCluster = -1;
> > > fTracks = 0;
> > > #ifdef STOREFULLINFO
> > > fXYZE[0]=fXYZE[1]=fXYZE[2]=fXYZE[3]=0.0;
> > > #endif
> > > //
> > > }
> > >
> > > The I/O works fine when I read/rewrite the digits of some version with
> the
> > > code compiled with the corresponding version of the class.
> > >
> > > But we have large amount of data written with the version 1 (i.e. w/o
> > > fXYZE member) which we need to read and store after some processing as a
> > > digits of version 2 (with fXYZE member). The code compiled with
> > > #define STOREFULLINFO reads w/o problem the old digits of version 1,
> > > but when I sotre them again, and try to read them back as a digits of
> > > version 2, they appear to be completely corrupted.
> > >
> > > Here is what I get with gDebug = 1 for reading of the class NaPixDigit,
> > > which derives from NaDigit1D by adding a new member fCol.
> > >
> > > (1) Reading the NaDigit1D of version 1 with the class compiled for
> version 2
> > > ...
> > > ====>Rebuilding TStreamerInfo for class: NaPixDigit, version: 1
> > > ====>Rebuilding TStreamerInfo for class: NaDigit1D, version: 1
> > > StreamerInfo for class: NaDigit1D, version=1
> > > TObject BASE offset= 0 type=66 Basic ROOT object
> > > Int_t fI offset= 12 type= 3 Value of the digit
> > > Int_t fCluster offset= 16 type= 3 Number of the Cluster
> to which it is attached
> > > TArrayI* fTracks offset= 20 type=69 Optional array of
> references on track
> > > i= 0, TObject type= 66, offset= 0, len=1, method=0
> > > i= 1, fI type= 3, offset= 12, len=1, method=0
> > > i= 2, fCluster type= 3, offset= 16, len=1, method=0
> > > i= 3, fTracks type= 69, offset= 20, len=1, method=1074526452
> > >
> > > StreamerInfo for class: NaPixDigit, version=1
> > > NaDigit1D BASE offset= 0 type= 0 Base class for 1Dim
> digits
> > > Int_t fCol offset= 40 type= 3 Column of the pixel
> (NaDigit1D::fI is row)
> > > i= 0, NaDigit1D type= 0, offset= 0, len=1, method=161037840
> > > i= 1, fCol type= 3, offset= 40, len=1, method=0
> > >
> > > ....
> > > root [10] gVerTel->GetSensor(0)->GetDigit(2)->Dump()
> > > ==>Dumping object at:a273f80, name=NaPixDigit, class=NaPixDigit
> > > fCol 0 Column of the pixel
> (NaDigit1D::fI is row)
> > > fI 147 Value of the digit
> > > fCluster 2 Number of the Cluster to which
> it is attached
> > > *fTracks ->0 Optional array of references
> on track
> > > fXYZE[4] 0
> > > fUniqueID 131185 object unique identifier
> > > fBits 50331648 bit field status word
> > >
> > > Here everything is correct. Now I store these digits and try to
> > > (2) read them back with the same code of NaDigit1D version 2:
> > >
> > > ====>Rebuilding TStreamerInfo for class: NaPixDigit, version: 1
> > > Creating StreamerInfo for class: NaDigit1D, version: 2
> > >
> > > StreamerInfo for class: NaDigit1D, version=2
> > > TObject BASE offset= 0 type=66 Basic ROOT object
> > > Int_t fI offset= 12 type= 3 Value of the digit
> > > Int_t fCluster offset= 16 type= 3 Number of the Cluster
> to which it is attached
> > > TArrayI* fTracks offset= 20 type=69 Optional array of
> references on track
> > > Float_t fXYZE[4] offset= 24 type=25
> > > i= 0, TObject type= 66, offset= 0, len=1, method=0
> > > i= 1, fI type= 23, offset= 12, len=2, method=0
> > > i= 2, fTracks type= 69, offset= 20, len=1, method=1074526452
> > > i= 3, fXYZE type= 25, offset= 24, len=4, method=0
> > >
> > > StreamerInfo for class: NaPixDigit, version=1
> > > NaDigit1D BASE offset= 0 type= 0 Base class for 1Dim
> digits
> > > Int_t fCol offset= 40 type= 3 Column of the pixel
> (NaDigit1D::fI is row)
> > > i= 0, NaDigit1D type= 0, offset= 0, len=1, method=160981416
> > > i= 1, fCol type= 3, offset= 40, len=1, method=0
> > >
> > > ...
> > > root [9] gVerTel->GetSensor(0)->GetDigit(2)->Dump()
> > > ==>Dumping object at:a5c2f38, name=NaPixDigit, class=NaPixDigit
> > >
> > > fCol 0 Column of the pixel
> (NaDigit1D::fI is row)
> > > fI 178 Value of the digit
> > > fCluster 185 Number of the Cluster to which
> it is attached
> > > *fTracks ->0 Optional array of references
> on track
> > > fXYZE[4] 0
> > > fUniqueID 131185 object unique identifier
> > > fBits 50331648 bit field status word
> > >
> > > The same digit now has completely different values, and the StreamerInfo
> > > also looks different: in the output (1) it had a table entry for
> > > i= 2, fCluster type= 3, offset= 16, len=1, method=0
> > >
> > > while in the output (2) it has disappeared, which apparently leads to
> > > misalignment during the reading.
> > >
> > > Have I done something wrong or there is a problem in the schema
> evolution?
> > >
> > > Ruben Shahoyan
> >
>
>
This archive was generated by hypermail 2b29 : Sun Jan 02 2005 - 05:50:07 MET