Change the default for streaming the content of STL containers from object-wise
to member-wise.
We evaluated the impact of moving to MemberWise streaming using 5 different CMS data files
cms1.root an older split using level 99 Reco file
cms2.root a more recent non split raw data file
cms3.root a more recent non split Reco file
cms4.root is an example of the lepton plus jet analysis format known as a user PAT-tuple (split)
cms5.root is an example AOD (analysis object dataset) format file. It is not split because the objects here are a strict subset of the RECO objects.
We rewrote all the files using the v5.26/00 new basket clustering algorithm using both memberwise streaming and objectwise streaming.
In the table below the read time is the CPU time to completion including loading the libraries.
When reading the file where always in the os cache (since we are focusing evaluating cpu time).
The number of event actually read varies from file set to file set but was calibrated to result
in about 10s of cpu time. The files are read using a library generated with TFile::MakeProject.
The object-wise files are between 2% and 10% larger compared to their member-wise coutnerpart.
The CPU time for reading object-wise files is 12% higher for split files
and 30% higher for non-split files.
So the improvement is significant enough to warrant switch the default from objectwise to memberwise.
Split files:
Filename Memberwise Size Cpu Time To read
cms1.root N 17.5 Gb 10.55s ± 0.15 (2200 entries)
cms1.root Y 16.8 Gb 9.12s ± 0.08
cms4.root N 1.47 Gb 10.18s ± 0.19 (2500 entries)
cms4.root Y 1.43 Gb 9.24s ± 0.06
Non Split files:
Filename Memberwise Size
cms2.root N 1.65 Gb 10.95s ± 0.05 (1000 entries)
cms2.root Y 1.53 Gb 8.20s ± 0.05
cms3.root N 0.780 Gb 10.59s ± 0.05 (700 entries)
cms3.root Y 0.717 Gb 8.29s ± 0.08
cms5.root N 1.55 Gb 10.20s ± 0.17 (700 entries)
cms5.root Y 1.40 Gb 8.09s ± 0.08
|
trunk/core/meta/src/TVirtualStreamerInfo.cxx