RE: [ROOT] Problem with Automatic schema evolution?

From: Ruben Shahoyan (Ruben.Shahoyan@cern.ch)
Date: Tue Mar 30 2004 - 03:12:53 MEST


Hello Philippe,
sorry for long message, but very soon na60 shall start large data 
processing, and I want to be sure in my code...

On Mon, 29 Mar 2004, Philippe Canal wrote:
> > But I noticed that even in this case the TClonesArrays were recreated
> > at every GetEntry()
> 
> This is surprising.  Could you send me your current code?
> 
I attach the toy version of na60's digit's storage: when TestGen and
TestReadWrite are called with first non-zero parameter, one branch
per TClonesArray is created, otherwise the data is stored in old mode:
single TObjArray of TClonesArrays...

My aim is to convert our old trees to new format:

1) I generate the old tree:

root [0] .L TestDigit1.cxx+
root [1] TestGen(0,"test0.root")
Write in OLD format:  Single Branch/Detector
Event 0
Col#  0 Row#   3 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col#  1 Row#  18 |  Digit# 1 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col# 10 Row#  10 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Col# 11 Row#   4 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Event 1
Col# 100 Row#   9 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col# 110 Row#   1 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |

Input: -------------------------------------------
N/A

Output: -------------------------------------------
Old Detector branch: detBranch, Address: 0x414c101c  AutoDelete: 1

State of sensor arrays: 
Name: sens0, Size: 10  Last: 0
Name: sens1, Size: 10  Last: 0
root [2] .q

2) I read this tree and rewrite it in a new format

root [0] .L TestDigit1.cxx+     
root [1] TestReadWrite(1,"test0.root","test01.root")  
Write in NEW format: Detector is split in 1 branch/sensor
Read OLD format: Single Branch/Detector
Event 0
Read 255 bytes
Col#  0 Row#   3 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col#  1 Row#  18 |  Digit# 1 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col# 10 Row#  10 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Col# 11 Row#   4 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Wrote 218 bytes
Event 1
Read 203 bytes
Col# 100 Row#   9 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col# 110 Row#   1 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Wrote 166 bytes

Input: -------------------------------------------
Old Detector branch: detBranch, Address: 0x414c101c  AutoDelete: 1

Output: -------------------------------------------
New Sensor branch: sens0, Address: 0x414c3ce4  AutoDelete: 1
New Sensor branch: sens1, Address: 0x414c3ce8  AutoDelete: 1

State of sensor arrays: 
Name: sens0, Size: 1  Last: 0
Name: sens1, Size: 1  Last: 0
root [2] .q

3) Now I can read the tree in a new format:
root [0] .L TestDigit1.cxx+
root [1] TestRead("test01.root")
Read NEW format: Detector is split in 1 branch/sensor
Event 0
Read 218 bytes
Col#  0 Row#   3 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col#  1 Row#  18 |  Digit# 1 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col# 10 Row#  10 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Col# 11 Row#   4 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Event 1
Read 166 bytes
Col# 100 Row#   9 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col# 110 Row#   1 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |

Input: -------------------------------------------
New Sensor branch: sens0, Address: 0x414c3ce4  AutoDelete: 1
New Sensor branch: sens1, Address: 0x414c3ce8  AutoDelete: 1

Output: -------------------------------------------
N/A

State of sensor arrays: 
Name: sens0, Size: 1  Last: 0
Name: sens1, Size: 1  Last: 0
root [2] .q

As you can see, the 2 input branches for tclonesarrays have AutoDelete set on, and the arrays
themselves were regenarated: after the last reading their size is equal to number of digits in
the last event. But it startes to reuse the initial arrays provided I set AutoDelete off:


root [0] .L TestDigit1.cxx+
root [1] NoAuto = 1;       
root [2] TestRead("test01.root")
Read NEW format: Detector is split in 1 branch/sensor
Event 0
Read 218 bytes
Col#  0 Row#   3 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col#  1 Row#  18 |  Digit# 1 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col# 10 Row#  10 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Col# 11 Row#   4 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |
Event 1
Read 166 bytes
Col# 100 Row#   9 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) |
Col# 110 Row#   1 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) |

Input: -------------------------------------------
New Sensor branch: sens0, Address: 0x414c3ce4  AutoDelete: 0
New Sensor branch: sens1, Address: 0x414c3ce8  AutoDelete: 0

Output: -------------------------------------------
N/A

State of sensor arrays: 
Name: sens0, Size: 10  Last: 0
Name: sens1, Size: 10  Last: 0

Now it looks fine, the sizes of arrays are equal to initial ones.

---------------
Now concerning the bug in TStreamerInfo: the previous examples were with version 1 of the NaDigit1D
class. If I now uncomment STOREFULLINFO in TestDigit1.h, and try to rewrite a tree with OLD branch with 
NaDigit1D verison 1 as a tree with NEW branches and NaDigit1D version 2, I am able to write it:

root [0] .L TestDigit1.cxx+     
root [1] TestReadWrite(1,"test0.root","test01c2.root") 
Write in NEW format: Detector is split in 1 branch/sensor
Read OLD format: Single Branch/Detector
Event 0
Read 255 bytes
Col#  0 Row#   3 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col#  1 Row#  18 |  Digit# 1 of Sensor 0 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 10 Row#  10 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 11 Row#   4 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Wrote 282 bytes
Event 1
Read 203 bytes
Col# 100 Row#   9 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 110 Row#   1 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Wrote 198 bytes

Input: -------------------------------------------
Old Detector branch: detBranch, Address: 0x414c101c  AutoDelete: 1

Output: -------------------------------------------
New Sensor branch: sens0, Address: 0x414c3e44  AutoDelete: 1
New Sensor branch: sens1, Address: 0x414c3e48  AutoDelete: 1

State of sensor arrays: 
Name: sens0, Size: 1  Last: 0
Name: sens1, Size: 1  Last: 0
root [2] .q

but when I am reading it back it still misses the offsets:


root [0] .L TestDigit1.cxx+                           
root [1] NoAuto = 1;                                  
root [2] TestRead("test01c2.root")                    
Read NEW format: Detector is split in 1 branch/sensor
Event 0
Read 282 bytes
Col#  0 Row#   3 |  Digit# 0 of Sensor 0 | In Clus# 18, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col#  1 Row#  -1 |  Digit# 1 of Sensor 0 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 10 Row#  10 |  Digit# 0 of Sensor 1 | In Clus#  4, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 11 Row#  -1 |  Digit# 1 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Event 1
Read 198 bytes
Col# 100 Row#   9 |  Digit# 0 of Sensor 0 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0
Col# 110 Row#   1 |  Digit# 0 of Sensor 1 | In Clus# -1, Track Ptr: (nil) | X:  0.0 Y:  0.0 Z:  0.0 E:  0.0

Input: -------------------------------------------
New Sensor branch: sens0, Address: 0x414c3e44  AutoDelete: 0
New Sensor branch: sens1, Address: 0x414c3e48  AutoDelete: 0

Output: -------------------------------------------
N/A

State of sensor arrays: 
Name: sens0, Size: 10  Last: 0
Name: sens1, Size: 10  Last: 0
root [3] .q

Cheers,
	Ruben





This archive was generated by hypermail 2b29 : Sun Jan 02 2005 - 05:50:07 MET