[root] Log of /trunk/tree/tree/src/TBranchRef.cxx

Revision 40872 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Sep 13 21:33:33 2011 UTC (3 years, 4 months ago) by pcanal
File length: 7970 byte(s)
Diff to previous 39611

From Chris Jones:

Put in place the infrastructure to optimize the I/O writes in the same way we optimized the I/O reads.

Rename TBuffer::ReadSequence to TBuffer::ApplySequence as they can be used both for reading and writing.  The 3 new signatures:

1. virtual Int_t ApplySequence(const TStreamerInfoActions::TActionSequence &sequence, void *object);      
2. virtual Int_t ApplySequenceVecPtr(const TStreamerInfoActions::TActionSequence &sequence, void *start_collection, void *end_collection);      
3. virtual Int_t ApplySequence(const TStreamerInfoActions::TActionSequence &sequence, void *start_collection, void *end_collection);

The 1st version is optimized to read a single object. The 2nd version is optimized to read the content of TClonesArrays and vectors of pointers to objects.
The 3rd version is used to streamed any collections.

In TBranchElement, introduce a set FillLeaves member functions to precalculate many of the (existing) conditional.  Introduction support for the StreamerInfo write actions and sequences.

Revision 39611 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Jun 8 19:35:17 2011 UTC (3 years, 7 months ago) by pcanal
File length: 7845 byte(s)
Diff to previous 38854

Introduce TMemFile and update TFileMerger to support incremental merges.

Add new tutorials ( net/treeClient.C + net/fastMergeServer.C )
demonstrating how a TMemFile can be used to do parallel merge
from many client. ( TMemFile still needs to be better integrated
with TMessage and TSocket).

The new TMemFile class support the TFile interface but only store
the information in memory.   This version is limited to 32MB.

   TMessage mess;
   ... 
   mess->ReadFastArray(scratch,length);
   transient = new TMemFile("hsimple.memroot",scratch,length);

will copy the content of 'scratch' into the in-memory buffer
created by/for the TMemFile.

   TMemFile *file = new TMemFile("hsimple.memroot","RECREATE");

Will create an empty in-memory of (currently fixed) size 32MB.

   file->ResetAfterMerge(0);

Will reset the objects in the TDirectory list of objects
so that they are ready for more data accumulations (i.e.
returns the data to 0 but keep the customizations).

Introduce the new function TFileMerger::IncrementalMerge will
will Merge the list of file _with_ the content of the output
file (if any).   This allows make several successive Merge
into the same TFile object.

Introduce non-static version of TFile::Cp allows the copy of
an existing TFile object.

Introduce new explicit interface for providing reseting 
capability after a merge.  If a class has a method with 
the name and signature:

   void ResetAfterMerge(TFileMergeInfo*);

it will be used by a TMemFile to reset its objects after
a merge operation has been done.

If this method does not exist, the TClass will use
a method with the name and signature:
  
   void Reset(Optiont_t *);

TClass now provides a quick access to these merging 
function via TClass::GetResetAfterMerge.   The wrapper function
is automatically created by rootcint and can be installed
via TClass::SetResetAfterMerge.   The wrapper function should have
the signature/type ROOT::ResetAfterMergeFunc_t:

   void (*)(void *thisobj, TFileMergeInfo*);

ResetAfterMerge functions were added to the following classes:
TDirectoryFile, TMemFile, TTree, TChain, TBranch, TBranhcElement, 
TBranchClones, TBranchObject and TBranchRef.

Revision 38854 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Apr 14 18:03:34 2011 UTC (3 years, 9 months ago) by pcanal
File length: 7477 byte(s)
Diff to previous 37275

In TBranchRef distinguish between the entry we need (now called RequestedEntry) and the
entry we have read (fReadEntry) so that we can avoid re-reading the same entry too many
times.

Revision 34969 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Aug 24 19:01:19 2010 UTC (4 years, 5 months ago) by pcanal
File length: 7236 byte(s)
Diff to previous 24454

Replace the ReadLeaves virtual function by a fReadLeaves pointer to member function,
this allows the customization of the ReadLeaves function at run-time depending on the
underlying user class layout in TBranchElement.  This removes many if statements whose
'answer' is known at initialization time.  [ReadLeavesMakeClass and ReadLeavesCollection
still need further decomposition]

Introduce TBranch::Set/GetMakeClass to independently set each branch in MakeClass mode 
and to have a good place to switch the ReadLeaves function appropriately (to and from
the MakeClass mode (also known as the decomposed object mode)).   This can also be
used to reset the mode of some branch with a MakeClass/MakeSelector file.

Revision 24454 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sat Jun 21 03:13:04 2008 UTC (6 years, 7 months ago) by pcanal
File length: 7113 byte(s)
Diff to previous 22902

Disk and Memory Space Gain

In ROOT older than v5.20/00, the branches' last basket, also known as the write basket, was always saved in the same "key" as the TTree object and was always present in memory when reading or writing. When reading this write basket was always present in memory even if the branch was never accessed.

Starting in v5.20/00, TTree::Write closes out, compresses (when requested) and writes to disk in their own file record the write baskets of all the branches. (This is implemented via the new function TTree::FlushBaskets, TBranch::FlushBaskets, TBranch::FlushOneBaskets)

TTree::AutoSave supports a new option "FlushBaskets" which will call FlushBaskets before saving the TTree object.

Benefits

- Flushing the write baskets has several advantages:
- Reduce the file size of the TTree object (it not longer contains the last basket), improving read time of the TTree object
- Reduce memory footprint of the TTree object.
- In a TTree which "flushed" buffer, there is now usually only zero or one buffer in memory.
- Previously each branch always had at least one basket in memory and usually 2 (the write basket and one read basket).
- Now only the basket of the branches actually read are loaded in memory.
- allow for the basket to be compressed and stored separated, increasing the compression factor.

Note: Calling FlushBaskets too often (either directly of via AutoSave("FlushBaskets")) can lead to unnecessary fragmentation of the ROOT file, since it write the baskets to disk (and a new basket will be started at the next fill) whether or not the content was close to filling the basket or not.

The fast tree cloning (TTreeCloner) was enhanced to support copying in-memory TTrees (that have been save as a single key on file). This issue was preventing hadd to fast clone files containing any 'in-memory' tree.

Revision 20882 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Nov 19 11:31:26 2007 UTC (7 years, 2 months ago) by rdm
Original Path: trunk/tree/src/TBranchRef.cxx
File length: 7237 byte(s)
Diff to previous 19826

Set property svn:eol-style LF on all source and Makefiles. This should avoid
problems with Win32 line endings ending up in the repository. All MS tools
support LF eols fine.

Revision 18209 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Mar 8 23:29:01 2007 UTC (7 years, 10 months ago) by pcanal
Original Path: trunk/tree/src/TBranchRef.cxx
File length: 7278 byte(s)
Diff to previous 14367

From Paul:
Modify TRefTable, TBranchRef, and TBranchElement so that the small
integer which is assigned to identify a TBranchElement by TRefTable
is cached so that we need to find it only once, instead of at every
call to TBranchElement::GetEntry() or TBranchElement::Fill().

Revision 14054 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Feb 17 05:16:38 2006 UTC (8 years, 11 months ago) by pcanal
Original Path: trunk/tree/src/TBranchRef.cxx
File length: 7020 byte(s)
Diff to previous 13207

From Axel:
---
Add support for auto-loading of TRefs from multiple TProcessIDs:
TRefTable now stores 2d arrays of branch numbers to load [PID][UID] (via
FillBuffer). The PID index <-> TProcessID-GUID mapping is stored via
TRefTable::Streamer (i.e. once per TTree). Lookup is cached. This change
is both backward and forward compatible, both concerning the interface
and .root files. Only exception is TRefTable::fParentIDs, which is now
only streamed via FillBuffer / ReadBuffer, not as a member of TRefTable.

Minor side-notes:
* Speed-up in TRef::GetObject de-referencing (only once call to
TProcessID::GetObjectWithID).
* No re-reading of a branch if it was already read for an event (if
TBranchRef's fReadEntry == branch-to-autoload's fReadEntry then don't
call GetEntry on it).
* Added getter for TProcessID::fgPIDs->Size()
* updated doc for TRef, TRefTable, TBranchRef, to reflect the current
status of auto-loading.
---

The only change that's breaking backward compatibility (as mentioned
above) is the clear split between per-entry and global streaming of the
TRefTable: there is no use in storing the fParentIDs[][] array (and
fN[]) as part of the TTree members - it should only be stored as part of
the TTree entry. So I've marked them as //!; they're made persistent in
TRefTable::FillBuffer.

The member TRefTable::fSize is only around to suppress a warning in the
schema evolution algorithm (it was a counter before, and the schema
evolution algo is unhappy about fSize missing in the new class
definition, nor does it appreciate fSize being marked as //!). This is
ugly, but I prefer it over a custom streamer.

The patch should fix the following issues:
* autoloading doesn't work because TRefs were created by different
processes,
* changed objects magically overwritten in memory due to re-loading of
referenced objects

Revision 13014 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Oct 25 22:11:58 2005 UTC (9 years, 3 months ago) by pcanal
Original Path: trunk/tree/src/TBranchRef.cxx
File length: 5305 byte(s)
Diff to previous 10760

When cloning a TTree and removing some of the branches, if we have
a TBranchRef, we need to completely reset the TRefTable (which caches
the address of the branches that contains referenced objects).
Thanks to Reiner Hauser for discovering and solving his issue.

Revision 10760 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Dec 13 12:12:35 2004 UTC (10 years, 1 month ago) by brun
Original Path: trunk/tree/src/TBranchRef.cxx
File length: 5305 byte(s)
Diff to previous 9831

In TBranchRef::Notify implement support for friend Trees.
With this extension, when dereferencing a TRef (via TRef::GetObject),
the branch containing the referenced object will be automatically loaded
if the object is in another branch of the same Tree or a branch
of a friend Tree.

Revision 9831 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Aug 24 10:41:58 2004 UTC (10 years, 5 months ago) by brun
Original Path: trunk/tree/src/TBranchRef.cxx
File length: 4686 byte(s)
Diff to previous 9820

Improve the functionality of the TRefTable.
The new algorithm
  -does not require a pointer to the TRefTable in TFile.
  -can automatically load the branch of a Tree when calling TRef::GetObject
  -works when writing/reading multiple Trees in parallel

Revision 9801 - (view) (download) (as text) (annotate) - [select for diffs]
Added Fri Aug 20 14:54:53 2004 UTC (10 years, 5 months ago) by brun
Original Path: trunk/tree/src/TBranchRef.cxx
File length: 4338 byte(s)

New class TBranchRef deriving from TBranch.
This is a special branch keeping the information about the
branch numbers having referenced objects.
The TBranchRef object is created by the new function
   TTree::BranchRef

This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, enter a numeric revision.

Links to HEAD:	(view) (download) (as text) (annotate)
Sticky Revision: