Logo ROOT   6.14/05
Reference Guide
List of all members | Public Member Functions | Private Member Functions | Private Attributes | Friends | List of all members
ROOT::Experimental::TBufferMerger Class Reference

TBufferMerger is a class to facilitate writing data in parallel from multiple threads, while writing to a single output file.

A TBufferMergerFile is similar to a TMemFile, but when data is written to it, it is appended to the TBufferMerger queue.

Its purpose is similar to TParallelMergingFile, but instead of using processes that connect to a network socket, TBufferMerger uses threads that each write to a TBufferMergerFile, which in turn push data into a queue managed by the TBufferMerger.

The TBufferMerger merges all data into the output file on disk.

Definition at line 43 of file TBufferMerger.hxx.

Public Member Functions

 TBufferMerger (const char *name, Option_t *option="RECREATE", Int_t compress=1)
 Constructor. More...
 TBufferMerger (std::unique_ptr< TFile > output)
 Constructor. More...
virtual ~TBufferMerger ()
 Destructor. More...
size_t GetAutoSave () const
 Returns the current value of the auto save setting in bytes (default = 0). More...
std::shared_ptr< TBufferMergerFileGetFile ()
 Returns a TBufferMergerFile to which data can be written. More...
size_t GetQueueSize () const
 Returns the number of buffers currently in the queue. More...
void RegisterCallback (const std::function< void(void)> &f)
 Register a user callback function to be called after a buffer has been removed from the merging queue and finished being processed. More...
void SetAutoSave (size_t size)
 By default, TBufferMerger will call TFileMerger::PartialMerge() for each buffer pushed onto its merge queue. More...

Private Member Functions

 TBufferMerger ()
 TBufferMerger has no default constructor. More...
 TBufferMerger (const TBufferMerger &)
 TBufferMerger has no copy constructor. More...
void Init (std::unique_ptr< TFile >)
void Merge ()
TBufferMergeroperator= (const TBufferMerger &)
 TBufferMerger has no copy operator. More...
void Push (TBufferFile *buffer)
void WriteOutputFile ()

Private Attributes

std::vector< std::weak_ptr< TBufferMergerFile > > fAttachedFiles
size_t fAutoSave {0}
size_t fBuffered {0}
std::function< void(void)> fCallback
std::condition_variable fDataAvailable
TFileMerger fMerger {false, false}
std::unique_ptr< std::thread > fMergingThread
std::queue< TBufferFile * > fQueue
std::mutex fQueueMutex


class TBufferMergerFile

#include <ROOT/TBufferMerger.hxx>

Constructor & Destructor Documentation

◆ TBufferMerger() [1/4]

ROOT::Experimental::TBufferMerger::TBufferMerger ( const char *  name,
Option_t option = "RECREATE",
Int_t  compress = 1 


nameOutput file name
optionOutput file creation options
compressionOutput file compression level

Definition at line 24 of file TBufferMerger.cxx.

◆ TBufferMerger() [2/4]

ROOT::Experimental::TBufferMerger::TBufferMerger ( std::unique_ptr< TFile output)


outputOutput TFile

Definition at line 32 of file TBufferMerger.cxx.

◆ ~TBufferMerger()

ROOT::Experimental::TBufferMerger::~TBufferMerger ( )


Definition at line 46 of file TBufferMerger.cxx.

◆ TBufferMerger() [3/4]

ROOT::Experimental::TBufferMerger::TBufferMerger ( )

TBufferMerger has no default constructor.

◆ TBufferMerger() [4/4]

ROOT::Experimental::TBufferMerger::TBufferMerger ( const TBufferMerger )

TBufferMerger has no copy constructor.

Member Function Documentation

◆ GetAutoSave()

size_t ROOT::Experimental::TBufferMerger::GetAutoSave ( ) const

Returns the current value of the auto save setting in bytes (default = 0).

Definition at line 83 of file TBufferMerger.cxx.

◆ GetFile()

std::shared_ptr< TBufferMergerFile > ROOT::Experimental::TBufferMerger::GetFile ( )

Returns a TBufferMergerFile to which data can be written.

At the end, all TBufferMergerFiles get merged into the output file. The user is responsible to "cd" into the file to associate objects such as histograms or trees to it.

After the creation of this file, the user must reset the kMustCleanup bit on any objects attached to it and take care of their deletion, as there is a possibility that a race condition will happen that causes a crash if ROOT manages these objects.

Definition at line 55 of file TBufferMerger.cxx.

◆ GetQueueSize()

size_t ROOT::Experimental::TBufferMerger::GetQueueSize ( ) const

Returns the number of buffers currently in the queue.

Definition at line 64 of file TBufferMerger.cxx.

◆ Init()

void ROOT::Experimental::TBufferMerger::Init ( std::unique_ptr< TFile output)

Definition at line 37 of file TBufferMerger.cxx.

◆ Merge()

void ROOT::Experimental::TBufferMerger::Merge ( )

Definition at line 93 of file TBufferMerger.cxx.

◆ operator=()

TBufferMerger& ROOT::Experimental::TBufferMerger::operator= ( const TBufferMerger )

TBufferMerger has no copy operator.

◆ Push()

void ROOT::Experimental::TBufferMerger::Push ( TBufferFile buffer)

Definition at line 74 of file TBufferMerger.cxx.

◆ RegisterCallback()

void ROOT::Experimental::TBufferMerger::RegisterCallback ( const std::function< void(void)> &  f)

Register a user callback function to be called after a buffer has been removed from the merging queue and finished being processed.

This function can be useful to allow asynchronous launching of new tasks to push more data into the queue once its size satisfies user requirements.

Definition at line 69 of file TBufferMerger.cxx.

◆ SetAutoSave()

void ROOT::Experimental::TBufferMerger::SetAutoSave ( size_t  size)

By default, TBufferMerger will call TFileMerger::PartialMerge() for each buffer pushed onto its merge queue.

This function lets the user change this behaviour by telling TBufferMerger to accumulate at least

sizebytes in memory before performing a partial merge and flushing to disk. This can be useful to avoid an excessive amount of work to happen in the output thread, as the number of TTree headers (which require compression) written to disk can be reduced.

Definition at line 88 of file TBufferMerger.cxx.

◆ WriteOutputFile()

void ROOT::Experimental::TBufferMerger::WriteOutputFile ( )

Definition at line 103 of file TBufferMerger.cxx.

Friends And Related Function Documentation

◆ TBufferMergerFile

friend class TBufferMergerFile

Definition at line 95 of file TBufferMerger.hxx.

Member Data Documentation

◆ fAttachedFiles

std::vector<std::weak_ptr<TBufferMergerFile> > ROOT::Experimental::TBufferMerger::fAttachedFiles

Definition at line 120 of file TBufferMerger.hxx.

◆ fAutoSave

size_t ROOT::Experimental::TBufferMerger::fAutoSave {0}

Definition at line 113 of file TBufferMerger.hxx.

◆ fBuffered

size_t ROOT::Experimental::TBufferMerger::fBuffered {0}

Definition at line 114 of file TBufferMerger.hxx.

◆ fCallback

std::function<void(void)> ROOT::Experimental::TBufferMerger::fCallback

Definition at line 121 of file TBufferMerger.hxx.

◆ fDataAvailable

std::condition_variable ROOT::Experimental::TBufferMerger::fDataAvailable

Definition at line 117 of file TBufferMerger.hxx.

◆ fMerger

TFileMerger ROOT::Experimental::TBufferMerger::fMerger {false, false}

Definition at line 115 of file TBufferMerger.hxx.

◆ fMergingThread

std::unique_ptr<std::thread> ROOT::Experimental::TBufferMerger::fMergingThread

Definition at line 119 of file TBufferMerger.hxx.

◆ fQueue

std::queue<TBufferFile *> ROOT::Experimental::TBufferMerger::fQueue

Definition at line 118 of file TBufferMerger.hxx.

◆ fQueueMutex

std::mutex ROOT::Experimental::TBufferMerger::fQueueMutex

Definition at line 116 of file TBufferMerger.hxx.

Libraries for ROOT::Experimental::TBufferMerger:

The documentation for this class was generated from the following files: