Log of /trunk/io/sql/inc/TBufferSQL2.h
Parent Directory
Revision
49019 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Mar 31 17:18:33 2013 UTC (21 months, 3 weeks ago) by
pcanal
File length: 15750 byte(s)
Diff to
previous 40872
Import revision 48931 from the v5-34-00 patch branch:
Add the concept of implicit rules to (centrally) support automatic translation of
STL collection from numeric type to other numeric type or for when the contained
class has some rules. Currently enabled only when the target is an std::vector.
Implement the transformation from STL container of numeric to any other numeric
type (both in TGenCollectionStreamer and TStreamerInfo[Actions]).
Migrate more of the conversions to TStreamerInfoActions.
Fix the Conversion of a Double32 or Float16 inside a collection of object to another
type (the previous implemetation was not supporting the customization of the factor
nor the number of bits).
Add TBuffer::ReadFastArrayWithNbits and WithFactor for use in the StreamerInfo Actions.
TStreamerInfoActions:
Introduce WithFactorMarker and NoFactorMarker to allow for less code duplicaton when
supporting Float16_t and Double32_t.
Replace the collection-memberwise action from loop over simple operation to using
ReadFastArray (to be compatible with TBufferXML).
Revision
40872 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Sep 13 21:33:33 2011 UTC (3 years, 4 months ago) by
pcanal
File length: 15378 byte(s)
Diff to
previous 36061
From Chris Jones:
Put in place the infrastructure to optimize the I/O writes in the same way we optimized the I/O reads.
Rename TBuffer::ReadSequence to TBuffer::ApplySequence as they can be used both for reading and writing. The 3 new signatures:
1. virtual Int_t ApplySequence(const TStreamerInfoActions::TActionSequence &sequence, void *object);
2. virtual Int_t ApplySequenceVecPtr(const TStreamerInfoActions::TActionSequence &sequence, void *start_collection, void *end_collection);
3. virtual Int_t ApplySequence(const TStreamerInfoActions::TActionSequence &sequence, void *start_collection, void *end_collection);
The 1st version is optimized to read a single object. The 2nd version is optimized to read the content of TClonesArrays and vectors of pointers to objects.
The 3rd version is used to streamed any collections.
In TBranchElement, introduce a set FillLeaves member functions to precalculate many of the (existing) conditional. Introduction support for the StreamerInfo write actions and sequences.
Revision
36061 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Oct 4 16:05:51 2010 UTC (4 years, 3 months ago) by
pcanal
File length: 15375 byte(s)
Diff to
previous 29321
Introduce an optimized infrastructure for reading objects using a StreamerInfo.
Rather than driving the streaming using a switch statement inside TStreamerInfo::ReadBuffer,
the streaming is now driven using a simple loop over a sequence of configured StreamerInfo actions.
This improves run-time performance by allowing a dramatic reduction in function calls and code
branches at the expense of some code duplication.
There are 3 versions of this loop implemented in TBufferFile and overloaded in TBufferXML and TBufferSQL:
1. virtual Int_t ReadSequence(const TStreamerInfoActions::TActionSequence &sequence, void *object);
2. virtual Int_t ReadSequenceVecPtr(const TStreamerInfoActions::TActionSequence &sequence, void *start_collection, void *end_collection);
3. virtual Int_t ReadSequence(const TStreamerInfoActions::TActionSequence &sequence, void *start_collection, void *end_collection);
The 1st version is optimized to read a single object. The 2nd version is optimized to read the content of TClonesArrays and vectors of pointers to objects.
The 3rd version is used to streamed any collections.
TBufferXML and TBufferSQL overload the loops to introduce extra code to help the buffer keep track of which streamer
element is being streamed (this functionality is not used by TBufferFile.)
A TStreamerInfoActions::TActionSequence is an ordered sequence of configured actions.
A configured action has both an action which is a free standing function and a configuration object deriving
from TStreamerInfoActions::TConfiguration. The configuration contains information that is specific to the action
but varies from use to use, including the offset from the beginning of the object that needs to be updated.
Other examples of configuration include the number of bits requested for storing a Double32_t or its factor and minimum.
When the sequence is intended for a collection, the sequence also has a configuration object deriving
from TStreamerInfoActions::TLoopConfiguration which contains for example the size of the element of
a vector or the pointers to the iterators functions (see below).
Each TStreamerInfo has 2 reading sequences, one for object-wise reading (GetReadObjectWiseActions)
and one for member-wise reading (GetReadMemberWiseActions) which is used when streaming a TClonesArray
of a vector of pointer to the type of objects described by the TClass.
Each collection proxy has at least one reading sequences, one for the reading each version of the
contained class layout.
Each case of the TStreamerInfo::ReadBuffer switch statement is replaced by 4 new action functions,
one for the object wise reading, one for the member wise reading for TClonesArray and vector of pointers,
one for the member wise reading for a vector of object and one for all other collections.
Each collection (proxy) needs to provide 5 new free standing functions:
// Set of functions to iterate easily throught the collection
static const Int_t fgIteratorArenaSize = 16; // greater than sizeof(void*) + sizeof(UInt_t)
typedef void (*CreateIterators_t)(void *collection, void **begin_arena, void **end_arena);
virtual CreateIterators_t GetFunctionCreateIterators(Bool_t read = kTRUE) = 0;
// begin_arena and end_arena should contain the location of a memory arena of size fgIteratorSize.
// If the collection iterator are of that size or less, the iterators will be constructed in place in those location (new with placement)
// Otherwise the iterators will be allocated via a regular new and their address returned by modifying the value of begin_arena and end_arena.
typedef void* (*CopyIterator_t)(void *dest, const void *source);
virtual CopyIterator_t GetFunctionCopyIterator(Bool_t read = kTRUE) = 0;
// Copy the iterator source, into dest. dest should contain the location of a memory arena of size fgIteratorSize.
// If the collection iterator is of that size or less, the iterator will be constructed in place in this location (new with placement)
// Otherwise the iterator will be allocated via a regular new and its address returned by modifying the value of dest.
typedef void* (*Next_t)(void *iter, const void *end);
virtual Next_t GetFunctionNext(Bool_t read = kTRUE) = 0;
// iter and end should be pointers to respectively an iterator to be incremented and the result of collection.end()
// If the iterator has not reached the end of the collection, 'Next' increment the iterator 'iter' and return 0 if
// the iterator reached the end.
// If the end was not reached, 'Next' returns the address of the content pointed to by the iterator before the
// incrementation ; if the collection contains pointers, 'Next' will return the value of the pointer.
typedef void (*DeleteIterator_t)(void *iter);
typedef void (*DeleteTwoIterators_t)(void *begin, void *end);
virtual DeleteIterator_t GetFunctionDeleteIterator(Bool_t read = kTRUE) = 0;
virtual DeleteTwoIterators_t GetFunctionDeleteTwoIterators(Bool_t read = kTRUE) = 0;
// If the size of the iterator is greater than fgIteratorArenaSize, call delete on the addresses,
// Otherwise just call the iterator's destructor.
This functions are currently only 'emulated' using the old infrastructure for the collections other than vector ; they
will later one be provided by the dictionaries.
Revision
29321 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jul 3 10:42:10 2009 UTC (5 years, 6 months ago) by
brun
File length: 14646 byte(s)
Diff to
previous 29170
From Sergei Linev:
1. New static methods to set/get time formatting in TOracleServer
2. Move float format constant for TBufferSQL2 to TSQLServer
3. Use that format in several other appropriate places like TODBCStatement, TOracleRow and so on.
4. Replace sprintf to snprintf calls in several places.
Revision
29170 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jun 23 14:09:56 2009 UTC (5 years, 7 months ago) by
brun
File length: 14775 byte(s)
Diff to
previous 25450
From Sergey:
There was a request to modify XML classes such that formatting of float/double variables should be changed to "%e".
It sounds reasonable - nobody, seems to be, was using small floats with XML. Therefore in patch following changes are implemented:
1. Conversion from float/double to string per default performed with "%e" (exponential) format.
2. Format can be configured with SetFloatFormat methods that one can specify precision, width arguments of printf call
3. sscanf works as before - "%f" accpet both exponential and decimal format
4. Similar changes done for TBufferXML and TBufferSQL2 classes.
Revision
25450 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Sep 18 21:13:42 2008 UTC (6 years, 4 months ago) by
pcanal
File length: 14544 byte(s)
Diff to
previous 25411
Import the code from the branch dev/datamodelevolution revision 25436
This implements the infrastructure for the new Data Model Evolution Scheme.
This Data Model Evolution is brought to your courtesy of BNL/STAR/ATLAS/Fermi/Cern
Current Capabilities:
Assign values to transient data members
Rename classes
Rename data members
Change the shape of the data structures or convert one class structure to another
Change the meaning of data members
Ability to access the TBuffer directly when needed
Ensure that the objects in collections are handled in the same way as the ones stored separately
Supported in object-wise, member-wise and split modes.
Coming soon:
Make things operational also in bare ROOT mode
Ability to transform data before writing
Support for changing the class type of nested object in a split branch
Support for access to onfile version of nested objects from within the parent rule
LinkDef rule syntax:
Setting a transient member:
#pragma read sourceClass="ACache" targetClass="ACache" source="" version="[1-]" target="zcalc" \
code="{ zcalc = false; }"
Setting a new member from 2 removed members
#pragma read sourceClass="ACache" targetClass="ACache" source="int x; int y; char c" version="[8]" target="z" \
code="{ z = onfile.x*1000 + onfile.y*10; }"
Renaming a class
#pragma read sourceClass="ACache" version="[8]" targetClass="Axis" \
source="int x; int y;" target="z" \
code="{ z = onfile.x*1000 + onfile.y*
#pragma read sourceClass="ACache" version="[9]" targetClass="Axis";
Selection XML syntax
<read sourceClass="ACache" targetClass="ACache" source="" version="[1-]" target="zcalc">
<![CDATA[
{ zcalc = false; }
]]></read>
Revision
17402 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jan 19 16:48:00 2007 UTC (8 years ago) by
brun
Original Path:
trunk/sql/inc/TBufferSQL2.h
File length: 13757 byte(s)
Diff to
previous 13977
CVs been changed into a pure abstract interface.
The concrete implementation is in the new class TBufferFile.
All classes previously deriving from TBuffer derive now from TBufferFile, ie
TBuffer <- TBufferFile <- TMessage
<- TBufferXML
<- TBufferSQL
<- TBufferSQL2
Because there are several problems with C++ operators overloading,
The I/O operators are defined in TBuffer. These are inline functions
calling C++ virtual functions defined in TBuffer and overloaded
by TBufferFile and all other derived classes when necessary.
The previous implementation of TBuffer.h included <vector> and Bytes.h.
The two include statements have been moved to TBufferFile.h. As a result the
compilation of the ROOT system is now slightly faster and a big bonus
is that changes in TBufferFile or Bytes.h will affect only TBufferFile
and will not force the recompilation of the entire system.
This change has some side-effects. If you assumed that include <vector>
was done by TBuffer.h, you may have to specify this include directly
in your class. This was the case for a few ROOT classes.
: ----------------------------------------------------------------------
Revision
13977 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Feb 1 18:57:41 2006 UTC (8 years, 11 months ago) by
pcanal
Original Path:
trunk/sql/inc/TBufferSQL2.h
File length: 13572 byte(s)
Diff to
previous 13927
From Sergei Linev:
Move CreateKey from TDirectory to TFile
Here is also optimisation of SQL statements and adjustement for Oracle.
Implementation of ClassBegin()/ClassMember()/ClassEnd() methdos for TBufferSQL2 and TBufferXML.
I also implementation for ClassMemeber() method for case of "raw:data" for SQL and XML cases.
Revision
13435 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Dec 1 16:30:43 2005 UTC (9 years, 1 month ago) by
pcanal
Original Path:
trunk/sql/inc/TBufferSQL2.h
File length: 13490 byte(s)
Diff to
previous 13383
From Sergei Linev:
This patch allows more clear representation of TClonesArray data in TSQLFile.
It also solves problem of correct saving of array of objects in TSQLFile.
I redefine in TBufferSQL2 following functions:
void WriteFastArray(void* start, const TClass* cl, Int_t n, TMemberStreamer* s)
Int_t WriteFastArray(void** startp, const TClass* cl, Int_t n, Bool_t isPreAlloc, TMemberStreamer* s)
void ReadFastArray(void* start, const TClass* cl, Int_t n = 1, TMemberStreamer* s = 0)
void ReadFastArray(void** startp, const TClass* cl, Int_t n = 1, Bool_t isPreAlloc = kFALSE, TMemberStreamer* s = 0)
In these functions I make replacement which I propose before - instead of cl->Streamer(buf, obj),
I use buf->StreamObject(obj, cl) ("parenthesis" arround cl->Streamer(buf, obj) call)
In case of TBufferSQL2 I treat StreamObject() as WriteObject()/ReadObject()
calls. Therefore, I can solve problem with objects array. Now it is easy to produce one column for
each array element.
Revision
13383 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 28 23:22:31 2005 UTC (9 years, 1 month ago) by
pcanal
Original Path:
trunk/sql/inc/TBufferSQL2.h
File length: 13740 byte(s)
Diff to
previous 13317
From Sergei Linev:
I made following modification:
1. Usage of table indexes. By default, they created only for basic tables: KeysTable and ObjectsTable.
It is possible to configure TSQLFile to create indexes also for other type of tables or disable indexes at all.
2. Use of transaction. Now by default this option on and protect storage of object data to datatables.
For some types of MySQL tables transaction are not supported, therefore I include in configurations
possibility to setup type of table which should be used.
3. Configurations. For all kind of configuration I create small table.
Configurations can be changed only when TSQLFile is created with option "CREATE" or "RECREATE".
In that case configurations can be changed until first write operation.
4. Locking. I did not use "native" database locking mechnism while they are very different in MySQL and Oracle and
may differ also in other SQL database. I put in Configuration table one flag, which says if database is already
opened for writing by other TSQLFile instance and prevent other TSQLFile to have write access.
For emergency cases one can use "BREAKLOCK" option in TSQLFile constructor to ignore that locking.
This is not real locking and any other user with normal sql queries can disturb tables data, but I do not see
now other solution, which may work for different databases. May be you know better solutions?
Revision
13317 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 22 20:42:37 2005 UTC (9 years, 2 months ago) by
pcanal
Original Path:
trunk/sql/inc/TBufferSQL2.h
File length: 13664 byte(s)
Diff to
previous 13312
From Sergei:
Fix white spaces, add more comment to functions, add CVS tag lines and copyright.
It should now work for array of objects and most stl classes.
I put limitation for array size. If array with fixed size exceed limit (default 20),
array will be converted to raw data, otherwise each element of array will be presented as
single column.
Revision
13312 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Added
Mon Nov 21 17:38:24 2005 UTC (9 years, 2 months ago) by
pcanal
Original Path:
trunk/sql/inc/TBufferSQL2.h
File length: 13055 byte(s)
From Sergey:
Introduce a "transparent" access to SQL data base via standard TFile interface.
The main approach that each class (but not each object) has one or two tables
with names like $(CLASSNAME)_ver$(VERSION) and $(CLASSNAME)_streamer_ver$(VERSION)
For example: TAxis_ver8 or TList_streamer_ver5
Second kind of tables appears, when some of class members can not be converted to
normalized form or when class has custom streamer.
For instance, for TH1 class two tables are required: TH1_ver4 and TH1_streamer_ver4
Most of memebers are stored in TH1_ver4 table columnwise, and only memeber:
Double_t* fBuffer; //[fBufferSize]
can not be represented as column while size of array is not known apriory.
Therefore, fBuffer will be written as list of values in TH1_streamer_ver4 table.
All objects, stored in the DB, will be registered in table "ObjectsTable".
In this there are following columns:
"key:id" - key identifier to which belong object
"obj:id" - object identifier
"Class" - object class name
"Version" - object class version
Data in each "ObjectsTable" row uniqly identify, in which table
and which column object is stored.
In normal situation all class data should be sorted columnwise.
Up to now following member are supported:
1) Basic data types
Here is everything clear. Column SQL type will be as much as possible
close to the original type of value.
2) Fixed array of basic data types
In this case n columns like fArr[0], fArr[1] and so on will be created.
If there is multidimensional array, names will be fArr2[1][2][1] and so on
3) Parent class
In this case version of parent class is stored and
data of parent class will be stored with the same obj:id in corrspondent table.
There is a special case, when parent store nothing (this is for instance TQObject).
In that case just -1 is written to avoid any extra checks if table exist or not.
4) Object as data member.
In that case object is saved in normal way to data base and column
will contain id of this object.
5) Pointer on object
Same as before. In case if object was already stored, just its id
will be placed in the column. For NULL pointer 0 is used.
6) TString
Now column with limited width like VARCAHR(255) in MySQL is used.
Later this will be improved to support maximum possible strings
7) Anything else.
Data will be converted to raw format and saved in _streamer_ table.
Each row supplied with obj:id and row:id, where row:id indicates
data, corresponding to this particular data member, and column
will contain this raw:id
See the TSQLFile documentation for more details.
example of a session saving data to a SQL data base
=====================================================
const char* dbname = "mysql://host.domain:3306/dbname";
const char* username = "username";
const char* userpass = "userpass";
// Clean data base and create primary tables
TSQLFile* f = new TSQLFile(dbname, "recreate", username, userpass);
// Write with standard I/O functions
arr->Write("arr",TObject::kSingleKey);
h1->Write("histo");
// Close connection to DB
delete f;
example of a session read data from SQL data base
=====================================================
// Open database again in read-only mode
TSQLFile* f = new TSQLFile(dbname, "open", username, userpass);
// Show list of keys
f->ls();
// Read stored object, again standard ROOT I/O
TH1* h1 = (TH1*) f->Get("histo");
if (h1!=0) { h1->SetDirectory(0); h1->Draw(); }
TObject* obj = f->Get("arr");
if (obj!=0) obj->Print("*");
// close connection to DB
delete f;
Known problems and open questions.
1) TTree is not supported by TSQLFile. There is independent development
of TTreeSQL, which allows to store trees directly in SQL database
2) TClonesArray is not tested, will be adjusted soon.
3) TDirectory cannot work. Hopefully, will (changes in ROOT basic I/O is required)
4) Streamer infos are not written to file, therefore schema evolution
is not yet supported. All eforts are done to enable this feature in
the near future
This form allows you to request diffs between any two revisions of this file.
For each of the two "sides" of the diff,
enter a numeric revision.
