[root] Log of /trunk/io/xml/inc/TBufferXML.h

Revision 49019 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sun Mar 31 17:18:33 2013 UTC (21 months, 3 weeks ago) by pcanal
File length: 17142 byte(s)
Diff to previous 40872

Import revision 48931 from the v5-34-00 patch branch:
Add the concept of implicit rules to (centrally) support automatic translation of
STL collection from numeric type to other numeric type or for when the contained
class has some rules.  Currently enabled only when the target is an std::vector.

Implement the transformation from STL container of numeric to any other numeric
type (both in TGenCollectionStreamer and TStreamerInfo[Actions]).

Migrate more of the conversions to TStreamerInfoActions.

Fix the Conversion of a Double32 or Float16 inside a collection of object to another
type (the previous implemetation was not supporting the customization of the factor
nor the number of bits).

Add TBuffer::ReadFastArrayWithNbits and WithFactor for use in the StreamerInfo Actions.

TStreamerInfoActions:

Introduce WithFactorMarker and NoFactorMarker to allow for less code duplicaton when
supporting Float16_t and Double32_t.
Replace the collection-memberwise action from loop over simple operation to using
ReadFastArray (to be compatible with TBufferXML).

Revision 40872 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Sep 13 21:33:33 2011 UTC (3 years, 4 months ago) by pcanal
File length: 16770 byte(s)
Diff to previous 39696

From Chris Jones:

Put in place the infrastructure to optimize the I/O writes in the same way we optimized the I/O reads.

Rename TBuffer::ReadSequence to TBuffer::ApplySequence as they can be used both for reading and writing.  The 3 new signatures:

1. virtual Int_t ApplySequence(const TStreamerInfoActions::TActionSequence &sequence, void *object);      
2. virtual Int_t ApplySequenceVecPtr(const TStreamerInfoActions::TActionSequence &sequence, void *start_collection, void *end_collection);      
3. virtual Int_t ApplySequence(const TStreamerInfoActions::TActionSequence &sequence, void *start_collection, void *end_collection);

The 1st version is optimized to read a single object. The 2nd version is optimized to read the content of TClonesArrays and vectors of pointers to objects.
The 3rd version is used to streamed any collections.

In TBranchElement, introduce a set FillLeaves member functions to precalculate many of the (existing) conditional.  Introduction support for the StreamerInfo write actions and sequences.

Revision 39696 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Jun 13 21:35:34 2011 UTC (3 years, 7 months ago) by pcanal
File length: 16767 byte(s)
Diff to previous 36061

From David Dagenhart:

Implement an option to allow ROOT to use the LZMA compression
algorithm to compress data instead of the ZLIB compression algorithm.
LZMA compression typically results in smaller files, but takes more
CPU time to compress data. To use the new feature, the external XZ
package must be installed when ROOT is configured and built:

Download 5.0.3 from here http://tukaani.org/xz/
and make sure to configure with fPIC:
   ./configure CFLAGS='-fPIC'

Then the client C++ code must call routines to explicitly request LZMA
compression.

ZLIB compression is still the default.

Setting the Compression Level and Algorithm:

There are three equivalent ways to set the compression level and 
algorithm. For example, to set the compression to the LZMA algorithm
and compression level 5.

1. TFile f(filename, option, title);
   f.SetCompressionSettings(ROOT::CompressionSettings(ROOT::kLZMA, 5));

OR

2. TFile f(filename, option, title, ROOT::CompressionSettings(ROOT::kLZMA, 5));

OR

3. TFile f(filename, option, title);
   f.SetCompressionAlgorithm(ROOT::kLZMA);
   f.SetCompressionLevel(5);

These methods work for TFile, TBranch, TMessage, TSocket, and TBufferXML. 
The compression algorithm and level settings only affect compression of
data after they have been set. TFile passes its settings to its branches 
only at the time the branches are created. This can be overidden by 
explicitly setting the level and algorithm for the branch. These classes 
also have the following methods to access the algorithm and level for 
compression.

   Int_t GetCompressionAlgorithm() const;
   Int_t GetCompressionLevel() const;
   Int_t GetCompressionSettings() const;

If the compression level is set to 0, then no compression will be
done. All of the currently supported algorithms allow the level to be
set to any value from 1 to 9. The higher the level, the larger the
compression factors will be (smaller compressed data size). The
tradeoff is that for higher levels more CPU time is used for
compression and possibly more memory. The ZLIB algorithm takes less
CPU time during compression than the LZMA algorithm, but the LZMA
algorithm usually delivers higher compression factors.

The header file core/zip/inc/Compression.h declares the function
"CompressionSettings" and the enumeration for the algorithms.
Currently the following selections can be made for the algorithm:
kZLIB (1), kLZMA (2), kOldCompressionAlgo (3), and kUseGlobalSetting
(0). The last option refers to an older interface used to control the
algorithm that is maintained for backward compatibility. The following
function is defined in core/zip/inc/Bits.h and it set the global
variable.

   R__SetZipMode(int algorithm)

If the algorithm is set to kUseGlobalSetting (0), the global variable
controls the algorithm for compression operations.  This is the
default and the default value for the global variable is kZLIB.

Note that the LZMA algorithm will only be available if the LZMA
libraries from the XZ package were available when the ROOT executable
being used was configured and built. If you are building ROOT and want
LZMA then you must do something similar to the following if XZ is not
already installed on your system.

  Download XZ version 5.0.3 from http://tukaani.org/xz/
  unwind the tarball
  cd xz-5.0.3
  ./configure CFLAGS='-fPIC'
  make

ROOT was tested with version 5.0.3 of XZ.  It might work with earlier
versions and will probably work with later versions of XZ.

Then either use "make install" to put the library and headers into
/usr/local/* so the scripts building ROOT can find them or use
arguments to the ROOT configure script to point to the proper
directories containing the XZ library and headers. These are the
library and the included header (lzma.h includes other headers in the
lzma subdirectory).

  src/liblzma/.libs/liblzma.a
  src/liblzma/api/lzma.h

WARNING: Data compressed with the LZMA algorithm cannot be read by
ROOT executables that were not built with LZMA support.

Revision 36061 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Oct 4 16:05:51 2010 UTC (4 years, 3 months ago) by pcanal
File length: 15926 byte(s)
Diff to previous 29170

Introduce an optimized infrastructure for reading objects using a StreamerInfo.

Rather than driving the streaming using a switch statement inside TStreamerInfo::ReadBuffer,
the streaming is now driven using a simple loop over a sequence of configured StreamerInfo actions.

This improves run-time performance by allowing a dramatic reduction in function calls and code
branches at the expense of some code duplication.


There are 3 versions of this loop implemented in TBufferFile and overloaded in TBufferXML and TBufferSQL:

1. virtual Int_t ReadSequence(const TStreamerInfoActions::TActionSequence &sequence, void *object);      
2. virtual Int_t ReadSequenceVecPtr(const TStreamerInfoActions::TActionSequence &sequence, void *start_collection, void *end_collection);      
3. virtual Int_t ReadSequence(const TStreamerInfoActions::TActionSequence &sequence, void *start_collection, void *end_collection);

The 1st version is optimized to read a single object. The 2nd version is optimized to read the content of TClonesArrays and vectors of pointers to objects.
The 3rd version is used to streamed any collections.

TBufferXML and TBufferSQL overload the loops to introduce extra code to help the buffer keep track of which streamer
element is being streamed (this functionality is not used by TBufferFile.)

A TStreamerInfoActions::TActionSequence is an ordered sequence of configured actions.   

A configured action has both an action which is a free standing function and a configuration object deriving 
from TStreamerInfoActions::TConfiguration. The configuration contains information that is specific to the action
but varies from use to use, including the offset from the beginning of the object that needs to be updated.   
Other examples of configuration include the number of bits requested for storing a Double32_t or its factor and minimum.   


When the sequence is intended for a collection, the sequence also has a configuration object deriving
from TStreamerInfoActions::TLoopConfiguration which contains for example the size of the element of
a vector or the pointers to the iterators functions (see below).


Each TStreamerInfo has 2 reading sequences, one for object-wise reading (GetReadObjectWiseActions)
and one for member-wise reading (GetReadMemberWiseActions) which is used when streaming a TClonesArray
of a vector of pointer to the type of objects described by the TClass.


Each collection proxy has at least one reading sequences, one for the reading each version of the
contained class layout.


Each case of the TStreamerInfo::ReadBuffer switch statement is replaced by 4 new action functions,
one for the object wise reading, one for the member wise reading for TClonesArray and vector of pointers,
one for the member wise reading for a vector of object and one for all other collections.


Each collection (proxy) needs to provide 5 new free standing functions:

   // Set of functions to iterate easily throught the collection
   static const Int_t fgIteratorArenaSize = 16; // greater than sizeof(void*) + sizeof(UInt_t)

   typedef void (*CreateIterators_t)(void *collection, void **begin_arena, void **end_arena);
   virtual CreateIterators_t GetFunctionCreateIterators(Bool_t read = kTRUE) = 0; 
   // begin_arena and end_arena should contain the location of a memory arena of size fgIteratorSize. 
   // If the collection iterator are of that size or less, the iterators will be constructed in place in those location (new with placement)
   // Otherwise the iterators will be allocated via a regular new and their address returned by modifying the value of begin_arena and end_arena.
   
   typedef void* (*CopyIterator_t)(void *dest, const void *source);
   virtual CopyIterator_t GetFunctionCopyIterator(Bool_t read = kTRUE) = 0;
   // Copy the iterator source, into dest.   dest should contain the location of a memory arena of size fgIteratorSize.
   // If the collection iterator is of that size or less, the iterator will be constructed in place in this location (new with placement)
   // Otherwise the iterator will be allocated via a regular new and its address returned by modifying the value of dest.
   
   typedef void* (*Next_t)(void *iter, const void *end);
   virtual Next_t GetFunctionNext(Bool_t read = kTRUE) = 0;
   // iter and end should be pointers to respectively an iterator to be incremented and the result of collection.end()
   // If the iterator has not reached the end of the collection, 'Next' increment the iterator 'iter' and return 0 if 
   // the iterator reached the end.
   // If the end was not reached, 'Next' returns the address of the content pointed to by the iterator before the 
   // incrementation ; if the collection contains pointers, 'Next' will return the value of the pointer.
   
   typedef void (*DeleteIterator_t)(void *iter);
   typedef void (*DeleteTwoIterators_t)(void *begin, void *end);

   virtual DeleteIterator_t GetFunctionDeleteIterator(Bool_t read = kTRUE) = 0;
   virtual DeleteTwoIterators_t GetFunctionDeleteTwoIterators(Bool_t read = kTRUE) = 0;
   // If the size of the iterator is greater than fgIteratorArenaSize, call delete on the addresses,
   // Otherwise just call the iterator's destructor.
 
This functions are currently only 'emulated' using the old infrastructure for the collections other than vector ; they
will later one be provided by the dictionaries.

Revision 25450 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Sep 18 21:13:42 2008 UTC (6 years, 4 months ago) by pcanal
File length: 14946 byte(s)
Diff to previous 25411

Import the code from the branch dev/datamodelevolution revision 25436

This implements the infrastructure for the new Data Model Evolution Scheme.

This Data Model Evolution is brought to your courtesy of BNL/STAR/ATLAS/Fermi/Cern

Current Capabilities:
  Assign values to transient data members
  Rename classes
  Rename data members
  Change the shape of the data structures or convert one class structure to another
  Change the meaning of data members
  Ability to access the TBuffer directly when needed
  Ensure that the objects in collections are handled in the same way as the ones stored separately
  Supported in object-wise, member-wise and split modes.

Coming soon:
  Make things operational also in bare ROOT mode
  Ability to transform data before writing
  Support for changing the class type of nested object in a split branch
  Support for access to onfile version of nested objects from within the parent rule

LinkDef rule syntax:
  Setting a transient member:

#pragma read sourceClass="ACache" targetClass="ACache" source="" version="[1-]" target="zcalc" \
   code="{ zcalc = false; }"

  Setting a new member from 2 removed members

#pragma read sourceClass="ACache" targetClass="ACache" source="int x; int y; char c"  version="[8]" target="z" \
   code="{ z = onfile.x*1000 + onfile.y*10; }"

   Renaming a class

#pragma read sourceClass="ACache" version="[8]" targetClass="Axis" \
   source="int x; int y;"  target="z" \
   code="{ z = onfile.x*1000 + onfile.y*
#pragma read sourceClass="ACache" version="[9]" targetClass="Axis";

Selection XML syntax

  <read  sourceClass="ACache" targetClass="ACache" source="" version="[1-]" target="zcalc">
  <![CDATA[
   { zcalc = false; }
  ]]></read>

Revision 17402 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Jan 19 16:48:00 2007 UTC (8 years ago) by brun
Original Path: trunk/xml/inc/TBufferXML.h
File length: 14226 byte(s)
Diff to previous 13927

CVs been changed into a pure abstract interface.
The concrete implementation is in the new class TBufferFile.
All classes previously deriving from TBuffer derive now from TBufferFile, ie

  TBuffer  <- TBufferFile  <- TMessage
                           <- TBufferXML
                           <- TBufferSQL
                           <- TBufferSQL2

Because there are several problems with C++ operators overloading,
The I/O operators are defined in TBuffer. These are inline functions
calling C++ virtual functions defined in TBuffer and overloaded
by TBufferFile and all other derived classes when necessary.

The previous implementation of TBuffer.h included <vector> and Bytes.h.
The two include statements have been moved to TBufferFile.h. As a result the
compilation of the ROOT system is now slightly faster and a big bonus
is that changes in TBufferFile or Bytes.h will affect only TBufferFile
and will not force the recompilation of the entire system.
This change has some side-effects. If you assumed that include <vector>
was done by TBuffer.h, you may have to specify this include directly
in your class. This was the case for a few ROOT classes.


 : ----------------------------------------------------------------------

Revision 13871 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Jan 20 01:12:13 2006 UTC (9 years ago) by pcanal
Original Path: trunk/xml/inc/TBufferXML.h
File length: 14031 byte(s)
Diff to previous 13471

From Sergei Linev:

1. Better support of TClonesArray.
   Now TBufferXML all set bit, that member-wise streaming is not supported.
   Therefore, objects in TClonesArray are stored in XML file in "normal" form
   and you can see correctly them in xml.

2. Object conversion to/from XML
   Some times ago I get a request to convert object directly to XML form without
   file I/O. I implement four static methods in TBufferXML:

   static TString   ConvertToXML(TObject* obj, Bool_t GenericLayout = kFALSE, Bool_t UseNamespaces = kFALSE);
   static TString   ConvertToXML(void* obj, TClass* cl, Bool_t GenericLayout = kFALSE, Bool_t UseNamespaces = kFALSE);
   static TObject*  ConvertFromXML(const char* str, Bool_t GenericLayout = kFALSE, Bool_t UseNamespaces = kFALSE);
   static void*     ConvertFromXMLAny(const char* str, TClass** cl = 0, Bool_t GenericLayout = kFALSE, Bool_t UseNamespaces = kFALSE);

   They convert object to/from string with XML nodes. Simplest code is look like:
     TH1* h = new TH1I("name","title", 1000, 0, 1000);
     TString s = TBufferXML::ConvertToXML(h);
     cout << s << endl;
     TObject* obj = TBufferXML::ConvertFromXML(s);
     if (obj) obj->Draw();

3. More information stored in XML file.
    I now store file creation/modification time, file UUID, file title, key title and creation time.
    I also put TXMLFile version into file itself to support old format of XML file for case of TClonesArray.

4. Preparation for custom streamers support
    I already put implementation of methods
         virtual void     ClassBegin(const TClass*, Version_t = -1);
         virtual void     ClassEnd(const TClass*);
         virtual void     ClassMember(const char* name, const char* typeName = 0, Int_t arrsize1 = -1, Int_t arrsize2 = -1);
    into TBufferXML class to enable support for custom streamers.
    They work the same way as I did for TBufferSQL2 class,
    but to enable them, I need declaration of that functions in TBuffer class.
    For TBuffer these functions does not required any implementation.

This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, enter a numeric revision.

Links to HEAD:	(view) (download) (as text) (annotate)
Sticky Revision: