Logo ROOT  
Reference Guide
ROOT::Experimental::TBufferMerger Class Reference

TBufferMerger is a class to facilitate writing data in parallel from multiple threads, while writing to a single output file.

A TBufferMergerFile is similar to a TMemFile, but when data is written to it, it is appended to the TBufferMerger queue.

Its purpose is similar to TParallelMergingFile, but instead of using processes that connect to a network socket, TBufferMerger uses threads that each write to a TBufferMergerFile, which in turn push data into a queue managed by the TBufferMerger.

The TBufferMerger merges all data into the output file on disk.

Definition at line 41 of file TBufferMerger.hxx.

Public Member Functions

 TBufferMerger (const char *name, Option_t *option="RECREATE", Int_t compress=ROOT::RCompressionSetting::EDefaults::kUseCompiledDefault)
 Constructor. More...
 
 TBufferMerger (std::unique_ptr< TFile > output)
 Constructor. More...
 
virtual ~TBufferMerger ()
 Destructor. More...
 
size_t GetAutoSave () const
 Returns the current value of the auto save setting in bytes (default = 0). More...
 
size_t GetBuffered () const
 Returns the number of bytes currently buffered (i.e. More...
 
Bool_t GetCompressTemporaryKeys () const
 Returns whether to compressed the TKey in the TMemFile for the object held by the TDirectories. More...
 
std::shared_ptr< TBufferMergerFileGetFile ()
 Returns a TBufferMergerFile to which data can be written. More...
 
const char * GetMergeOptions ()
 Returns the current merge options. More...
 
Bool_t GetNotrees () const
 Returns whether the the file has been marked as not containing any TTree objects and thus that steps that are specific to TTree can be skipped. More...
 
size_t GetQueueSize () const
 Returns the number of buffers currently in the queue. More...
 
void SetAutoSave (size_t size)
 By default, TBufferMerger will call TFileMerger::PartialMerge() for each buffer pushed onto its merge queue. More...
 
void SetCompressTemporaryKeys (Bool_t request_compression=true)
 Indicates that the temporary keys (corresponding to the object held by the directories of the TMemFile) should be compressed or not. More...
 
void SetMergeOptions (const TString &options)
 Sets the merge options. More...
 
void SetNotrees (Bool_t notrees=kFALSE)
 Indicates that any TTree objects in the file should be skipped and thus that steps that are specific to TTree can be skipped. More...
 

Private Member Functions

 TBufferMerger ()
 TBufferMerger has no default constructor. More...
 
 TBufferMerger (const TBufferMerger &)
 TBufferMerger has no copy constructor. More...
 
void Init (std::unique_ptr< TFile >)
 
void Merge ()
 
void MergeImpl ()
 
TBufferMergeroperator= (const TBufferMerger &)
 TBufferMerger has no copy operator. More...
 
void Push (TBufferFile *buffer)
 
bool TryMerge (TBufferMergerFile *memfile)
 

Private Attributes

std::vector< std::weak_ptr< TBufferMergerFile > > fAttachedFiles
 
size_t fAutoSave {0}
 
std::atomic< size_t > fBuffered {0}
 
bool fCompressTemporaryKeys {false}
 
std::mutex fMergeMutex
 
TFileMerger fMerger {false, false}
 
std::queue< TBufferFile * > fQueue
 
std::mutex fQueueMutex
 

Friends

class TBufferMergerFile
 

#include <ROOT/TBufferMerger.hxx>

Constructor & Destructor Documentation

◆ TBufferMerger() [1/4]

ROOT::Experimental::TBufferMerger::TBufferMerger ( const char *  name,
Option_t option = "RECREATE",
Int_t  compress = ROOT::RCompressionSetting::EDefaults::kUseCompiledDefault 
)

Constructor.

Parameters
nameOutput file name
optionOutput file creation options
compressionOutput file compression level

Definition at line 24 of file TBufferMerger.cxx.

◆ TBufferMerger() [2/4]

ROOT::Experimental::TBufferMerger::TBufferMerger ( std::unique_ptr< TFile output)

Constructor.

Parameters
outputOutput TFile

Definition at line 32 of file TBufferMerger.cxx.

◆ ~TBufferMerger()

ROOT::Experimental::TBufferMerger::~TBufferMerger ( )
virtual

Destructor.

Definition at line 45 of file TBufferMerger.cxx.

◆ TBufferMerger() [3/4]

ROOT::Experimental::TBufferMerger::TBufferMerger ( )
private

TBufferMerger has no default constructor.

◆ TBufferMerger() [4/4]

ROOT::Experimental::TBufferMerger::TBufferMerger ( const TBufferMerger )
private

TBufferMerger has no copy constructor.

Member Function Documentation

◆ GetAutoSave()

size_t ROOT::Experimental::TBufferMerger::GetAutoSave ( ) const

Returns the current value of the auto save setting in bytes (default = 0).

Definition at line 87 of file TBufferMerger.cxx.

◆ GetBuffered()

size_t ROOT::Experimental::TBufferMerger::GetBuffered ( ) const
inline

Returns the number of bytes currently buffered (i.e.

in the queue).

Definition at line 74 of file TBufferMerger.hxx.

◆ GetCompressTemporaryKeys()

Bool_t ROOT::Experimental::TBufferMerger::GetCompressTemporaryKeys ( ) const
inline

Returns whether to compressed the TKey in the TMemFile for the object held by the TDirectories.

See TBufferMerger::SetCompressTemporaryKeys for more details.

Definition at line 135 of file TBufferMerger.hxx.

◆ GetFile()

std::shared_ptr< TBufferMergerFile > ROOT::Experimental::TBufferMerger::GetFile ( )

Returns a TBufferMergerFile to which data can be written.

At the end, all TBufferMergerFiles get merged into the output file. The user is responsible to "cd" into the file to associate objects such as histograms or trees to it.

After the creation of this file, the user must reset the kMustCleanup bit on any objects attached to it and take care of their deletion, as there is a possibility that a race condition will happen that causes a crash if ROOT manages these objects.

Definition at line 60 of file TBufferMerger.cxx.

◆ GetMergeOptions()

const char * ROOT::Experimental::TBufferMerger::GetMergeOptions ( )

Returns the current merge options.

Definition at line 92 of file TBufferMerger.cxx.

◆ GetNotrees()

Bool_t ROOT::Experimental::TBufferMerger::GetNotrees ( ) const
inline

Returns whether the the file has been marked as not containing any TTree objects and thus that steps that are specific to TTree can be skipped.

Definition at line 111 of file TBufferMerger.hxx.

◆ GetQueueSize()

size_t ROOT::Experimental::TBufferMerger::GetQueueSize ( ) const

Returns the number of buffers currently in the queue.

Definition at line 69 of file TBufferMerger.cxx.

◆ Init()

void ROOT::Experimental::TBufferMerger::Init ( std::unique_ptr< TFile output)
private

Definition at line 37 of file TBufferMerger.cxx.

◆ Merge()

void ROOT::Experimental::TBufferMerger::Merge ( )
private

Definition at line 108 of file TBufferMerger.cxx.

◆ MergeImpl()

void ROOT::Experimental::TBufferMerger::MergeImpl ( )
private

Definition at line 116 of file TBufferMerger.cxx.

◆ operator=()

TBufferMerger & ROOT::Experimental::TBufferMerger::operator= ( const TBufferMerger )
private

TBufferMerger has no copy operator.

◆ Push()

void ROOT::Experimental::TBufferMerger::Push ( TBufferFile buffer)
private

Definition at line 75 of file TBufferMerger.cxx.

◆ SetAutoSave()

void ROOT::Experimental::TBufferMerger::SetAutoSave ( size_t  size)

By default, TBufferMerger will call TFileMerger::PartialMerge() for each buffer pushed onto its merge queue.

This function lets the user change this behaviour by telling TBufferMerger to accumulate at least size bytes in memory before performing a partial merge and flushing to disk. This can be useful to avoid an excessive amount of work to happen in the output thread, as the number of TTree headers (which require compression) written to disk can be reduced.

Definition at line 98 of file TBufferMerger.cxx.

◆ SetCompressTemporaryKeys()

void ROOT::Experimental::TBufferMerger::SetCompressTemporaryKeys ( Bool_t  request_compression = true)
inline

Indicates that the temporary keys (corresponding to the object held by the directories of the TMemFile) should be compressed or not.

Those object are stored in the TMemFile (and thus possibly compressed) when a thread push its data forward (by calling TBufferMergerFile::Write) and the queue is being processed by another. Once the TMemFile is picked (by any thread to be merged), after taking the TBufferMerger::fMergeMutex, those object are read back (and thus possibly uncompressed) and then used by merging. In order word, the compression of those objects/keys is only usefull to reduce the size in memory (of the TMemFile) and does not affect (at all) the compression factor of the end result.

Definition at line 127 of file TBufferMerger.hxx.

◆ SetMergeOptions()

void ROOT::Experimental::TBufferMerger::SetMergeOptions ( const TString options)

Sets the merge options.

SetMergeOptions("fast") will disable recompression of input data into the output if they have different compression settings.

Parameters
optionsTFileMerger/TFileMergeInfo merge options

Definition at line 103 of file TBufferMerger.cxx.

◆ SetNotrees()

void ROOT::Experimental::TBufferMerger::SetNotrees ( Bool_t  notrees = kFALSE)
inline

Indicates that any TTree objects in the file should be skipped and thus that steps that are specific to TTree can be skipped.

Definition at line 104 of file TBufferMerger.hxx.

◆ TryMerge()

bool ROOT::Experimental::TBufferMerger::TryMerge ( TBufferMergerFile memfile)
private

Definition at line 135 of file TBufferMerger.cxx.

Friends And Related Function Documentation

◆ TBufferMergerFile

friend class TBufferMergerFile
friend

Definition at line 140 of file TBufferMerger.hxx.

Member Data Documentation

◆ fAttachedFiles

std::vector<std::weak_ptr<TBufferMergerFile> > ROOT::Experimental::TBufferMerger::fAttachedFiles
private

Definition at line 167 of file TBufferMerger.hxx.

◆ fAutoSave

size_t ROOT::Experimental::TBufferMerger::fAutoSave {0}
private

Definition at line 161 of file TBufferMerger.hxx.

◆ fBuffered

std::atomic<size_t> ROOT::Experimental::TBufferMerger::fBuffered {0}
private

Definition at line 162 of file TBufferMerger.hxx.

◆ fCompressTemporaryKeys

bool ROOT::Experimental::TBufferMerger::fCompressTemporaryKeys {false}
private

Definition at line 160 of file TBufferMerger.hxx.

◆ fMergeMutex

std::mutex ROOT::Experimental::TBufferMerger::fMergeMutex
private

Definition at line 164 of file TBufferMerger.hxx.

◆ fMerger

TFileMerger ROOT::Experimental::TBufferMerger::fMerger {false, false}
private

Definition at line 163 of file TBufferMerger.hxx.

◆ fQueue

std::queue<TBufferFile *> ROOT::Experimental::TBufferMerger::fQueue
private

Definition at line 166 of file TBufferMerger.hxx.

◆ fQueueMutex

std::mutex ROOT::Experimental::TBufferMerger::fQueueMutex
mutableprivate

Definition at line 165 of file TBufferMerger.hxx.

Libraries for ROOT::Experimental::TBufferMerger:
[legend]

The documentation for this class was generated from the following files: