Re: [ROOT] Problem with Automatic schema evolution?

From: Ruben Shahoyan (Ruben.Shahoyan@cern.ch)
Date: Thu Mar 25 2004 - 16:00:23 MET


Hello Rene,

On Thu, 25 Mar 2004, Rene Brun wrote:
> Could you specify which version of ROOT?
> In case you use an old version, could you try with 3.10/02 or better 4.00/03?

I am using root3.05/02, but I checked, the same happens in 3.10.

> Could you send a small tar file with teh strict minimum to reproduce this
> problem?

I attached the code to reproduce the problem. 

1) First you should generate the file with digits of version 1, i.e. 
#define STOREFULLINFO in the TestDigit.h is commented:

~/NA60/test > root
  *******************************************
  *                                         *
  *        W E L C O M E  to  R O O T       *
  *                                         *
  *   Version   3.05/02  11 February 2003   *
  *                                         *
  *  You are welcome to visit our Web site  *
  *          http://root.cern.ch            *
  *                                         *
  *******************************************
..
root [0] .L TestDigit.cxx++
Info in <TUnixSystem::ACLiC>: creating shared library /home/shahoian/NA60/test/./TestDigit_cxx.so
root [1] TestGen("test0.root"); // THIS WILL GENERATE DIGITS OF VERSION 1
Event 0
Col#  0 Row#  18 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col# 10 Row#   4 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Col# 11 Row#   9 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Col# 12 Row#   1 |  Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Event 1
Col# 100 Row#  30 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col# 110 Row#  18 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Col# 111 Row#  30 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Col# 112 Row#   7 |  Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Col# 113 Row#  26 |  Digit# 3 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
root [2] .q

2) Uncomment #define STOREFULLINFO in the TestDigit.h, read just generated digits and store
them in a new file, as a digits of version 2

~/NA60/test > root
...
root [0] .L TestDigit.cxx++                      
Info in <TUnixSystem::ACLiC>: creating shared library /home/shahoian/NA60/test/./TestDigit_cxx.so
root [1] TestReadWrite("test0.root","test1.root"); // THIS WILL CONVERT THE FILE WITH DIGITS_V1 TO FILE OF DIGITS_V2
Event 0
Col#  0 Row#  18 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 10 Row#   4 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 11 Row#   9 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 12 Row#   1 |  Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Event 1
Col# 100 Row#  30 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 110 Row#  18 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 111 Row#  30 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 112 Row#   7 |  Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 113 Row#  26 |  Digit# 3 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
root [2]                                         
//
// as you see, the reading of v1 digits with the code of v2 works fine, we recover old values. The problem
// arises when we try to read just stored digits. If it is done in the same root session, we get:
//
root [2] TestRead("test1.root")                  
Event 0
Error in <TBuffer::CheckByteCount>: object of class TClonesArray read too few bytes: 71 instead of 87
Warning in <TBuffer::CheckByteCount>: TClonesArray::Streamer() not in sync with data on file, fix Streamer()
Error in <TBuffer::CheckByteCount>: object of class TClonesArray read too few bytes: 123 instead of 171
Warning in <TBuffer::CheckByteCount>: TClonesArray::Streamer() not in sync with data on file, fix Streamer()
Col#  0 Row#  18 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col#  0 Row#   4 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col#  0 Row#   9 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col#  0 Row#   1 |  Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Event 1
Error in <TBuffer::CheckByteCount>: object of class TClonesArray read too few bytes: 71 instead of 87
Warning in <TBuffer::CheckByteCount>: TClonesArray::Streamer() not in sync with data on file, fix Streamer()
Error in <TBuffer::CheckByteCount>: object of class TClonesArray read too few bytes: 149 instead of 213
Warning in <TBuffer::CheckByteCount>: TClonesArray::Streamer() not in sync with data on file, fix Streamer()
Col#  0 Row#  30 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col#  0 Row#  18 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col#  0 Row#  30 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col#  0 Row#   7 |  Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col#  0 Row#  26 |  Digit# 3 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
root [3] .q

3) if the just converted digits are read from new root session, we get:
~/NA60/test > root
root [0] .L TestDigit.cxx++                     
root [1] TestRead("test1.root")
Event 0
Col#  0 Row#  18 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 10 Row#   4 |  Digit# 0 of Sensor 1 | In Clus#  9, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 11 Row#   1 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 12 Row#  -1 |  Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Event 1
Col# 100 Row#  30 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 110 Row#  18 |  Digit# 0 of Sensor 1 | In Clus# 30, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 111 Row#   7 |  Digit# 1 of Sensor 1 | In Clus# 26, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 112 Row#  -1 |  Digit# 2 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 113 Row#  -1 |  Digit# 3 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0

The data member fI of NaDigit1D class is corrupted: it looks like due to the wrong offsets it mixes
the fCluster member (which in this example should be always -1) and fI.

Best regards,
     Ruben

-- 


> Ruben Shahoyan wrote:
> > 
> > Hello,
> > we have a following problem with I/O of the objects having streamer
> > versions:
> > the base object for na60 digits is
> > 
> > class NaDigit1D :
> > public TObject {
> > ...
> >  protected:
> >   Int_t fI;             // Value of the digit
> >   Int_t fCluster;       // Number of the Cluster to which it is attached
> >   TArrayI* fTracks;     // Optional array of references on track
> >   //
> > #ifdef STOREFULLINFO
> >   Float_t fXYZE[4];
> >   ClassDef(NaDigit1D,2) // Base class for 1Dim digits with FULL TRACKING INFO
> > #else
> >   ClassDef(NaDigit1D,1) // Base class for 1Dim digits
> > #endif
> > };
> > 
> > the #define STOREFULLINFO in the NaDigit1D.h allows us to switch from
> > version 1 to more complete version 2.
> > 
> > The default constructor is:
> > NaDigit1D::NaDigit1D()
> > {
> >   fI = -1;
> >   fCluster = -1;
> >   fTracks = 0;
> > #ifdef STOREFULLINFO
> >   fXYZE[0]=fXYZE[1]=fXYZE[2]=fXYZE[3]=0.0;
> > #endif
> >   //
> > }
> > 
> > The I/O works fine when I read/rewrite the digits of some version with the
> > code compiled with the corresponding version of the class.
> > 
> > But we have large amount of data written with the version 1 (i.e. w/o
> > fXYZE member) which we need to read and store after some processing as a
> > digits of version 2 (with fXYZE member). The code compiled with
> > #define STOREFULLINFO reads w/o problem the old digits of version 1,
> > but when I sotre them again, and try to read them back as a digits of
> > version 2, they appear to be completely corrupted.
> > 
> > Here is what I get with gDebug = 1 for reading of the class NaPixDigit,
> > which derives from NaDigit1D by adding a new member fCol.
> > 
> > (1) Reading the NaDigit1D of version 1 with the class compiled for version 2
> > ...
> > ====>Rebuilding TStreamerInfo for class: NaPixDigit, version: 1
> > ====>Rebuilding TStreamerInfo for class: NaDigit1D, version: 1
> > StreamerInfo for class: NaDigit1D, version=1
> >   TObject       BASE            offset=  0 type=66 Basic ROOT object
> >   Int_t         fI              offset= 12 type= 3 Value of the digit
> >   Int_t         fCluster        offset= 16 type= 3 Number of the Cluster to which it is attached
> >   TArrayI*      fTracks         offset= 20 type=69 Optional array of references on track
> >    i= 0, TObject         type= 66, offset=  0, len=1, method=0
> >    i= 1, fI              type=  3, offset= 12, len=1, method=0
> >    i= 2, fCluster        type=  3, offset= 16, len=1, method=0
> >    i= 3, fTracks         type= 69, offset= 20, len=1, method=1074526452
> > 
> > StreamerInfo for class: NaPixDigit, version=1
> >   NaDigit1D     BASE            offset=  0 type= 0 Base class for 1Dim digits
> >   Int_t         fCol            offset= 40 type= 3 Column of the pixel (NaDigit1D::fI is row)
> >    i= 0, NaDigit1D       type=  0, offset=  0, len=1, method=161037840
> >    i= 1, fCol            type=  3, offset= 40, len=1, method=0
> > 
> > ....
> > root [10] gVerTel->GetSensor(0)->GetDigit(2)->Dump()
> > ==>Dumping object at:a273f80, name=NaPixDigit, class=NaPixDigit
> > fCol                          0           Column of the pixel (NaDigit1D::fI is row)
> > fI                            147         Value of the digit
> > fCluster                      2           Number of the Cluster to which it is attached
> > *fTracks                      ->0         Optional array of references on track
> > fXYZE[4]                      0
> > fUniqueID                     131185      object unique identifier
> > fBits                         50331648    bit field status word
> > 
> > Here everything is correct. Now I store these digits and try to
> > (2) read them back with the same code of NaDigit1D version 2:
> > 
> > ====>Rebuilding TStreamerInfo for class: NaPixDigit, version: 1
> > Creating StreamerInfo for class: NaDigit1D, version: 2
> > 
> > StreamerInfo for class: NaDigit1D, version=2
> >   TObject       BASE            offset=  0 type=66 Basic ROOT object
> >   Int_t         fI              offset= 12 type= 3 Value of the digit
> >   Int_t         fCluster        offset= 16 type= 3 Number of the Cluster to which it is attached
> >   TArrayI*      fTracks         offset= 20 type=69 Optional array of references on track
> >   Float_t       fXYZE[4]        offset= 24 type=25
> >    i= 0, TObject         type= 66, offset=  0, len=1, method=0
> >    i= 1, fI              type= 23, offset= 12, len=2, method=0
> >    i= 2, fTracks         type= 69, offset= 20, len=1, method=1074526452
> >    i= 3, fXYZE           type= 25, offset= 24, len=4, method=0
> > 
> > StreamerInfo for class: NaPixDigit, version=1
> >   NaDigit1D     BASE            offset=  0 type= 0 Base class for 1Dim digits
> >   Int_t         fCol            offset= 40 type= 3 Column of the pixel (NaDigit1D::fI is row)
> >    i= 0, NaDigit1D       type=  0, offset=  0, len=1, method=160981416
> >    i= 1, fCol            type=  3, offset= 40, len=1, method=0
> > 
> > ...
> > root [9] gVerTel->GetSensor(0)->GetDigit(2)->Dump()
> > ==>Dumping object at:a5c2f38, name=NaPixDigit, class=NaPixDigit
> > 
> > fCol                          0           Column of the pixel (NaDigit1D::fI is row)
> > fI                            178         Value of the digit
> > fCluster                      185         Number of the Cluster to which it is attached
> > *fTracks                      ->0         Optional array of references on track
> > fXYZE[4]                      0
> > fUniqueID                     131185      object unique identifier
> > fBits                         50331648    bit field status word
> > 
> > The same digit now has completely different values, and the StreamerInfo
> > also looks different: in the output (1) it had a table entry for
> > i= 2, fCluster        type=  3, offset= 16, len=1, method=0
> > 
> > while in the output (2) it has disappeared, which apparently leads to
> > misalignment during the reading.
> > 
> > Have I done something wrong or there is a problem in the schema evolution?
> > 
> > Ruben Shahoyan
> 





This archive was generated by hypermail 2b29 : Sun Jan 02 2005 - 05:50:06 MET