Logo ROOT  
Reference Guide
ROOT::Experimental::Detail::RPageSinkBuf Class Reference

Wrapper sink that coalesces cluster column page writes.

TODO(jblomer): The interplay of derived class and RPageSink is not yet optimally designed for page storage wrapper classes like this one. Header and footer serialization, e.g., are done twice. To be revised.

Definition at line 42 of file RPageSinkBuf.hxx.

Classes

class  RColumnBuf
 A buffered column. More...
 
struct  RCounters
 I/O performance counters that get registered in fMetrics. More...
 

Public Member Functions

 RPageSinkBuf (const RPageSinkBuf &)=delete
 
 RPageSinkBuf (RPageSinkBuf &&)=default
 
 RPageSinkBuf (std::unique_ptr< RPageSink > inner)
 
virtual ~RPageSinkBuf ()=default
 
RNTupleMetricsGetMetrics () final
 Returns the default metrics object. Subclasses might alternatively provide their own metrics object by overriding this. More...
 
RPageSinkBufoperator= (const RPageSinkBuf &)=delete
 
RPageSinkBufoperator= (RPageSinkBuf &&)=default
 
void ReleasePage (RPage &page) final
 Every page store needs to be able to free pages it handed out. More...
 
RPage ReservePage (ColumnHandle_t columnHandle, std::size_t nElements) final
 Get a new, empty page for the given column that can be filled with up to nElements. More...
 
- Public Member Functions inherited from ROOT::Experimental::Detail::RPageSink
 RPageSink (const RPageSink &)=delete
 
 RPageSink (RPageSink &&)=default
 
 RPageSink (std::string_view ntupleName, const RNTupleWriteOptions &options)
 
virtual ~RPageSink ()
 
ColumnHandle_t AddColumn (DescriptorId_t fieldId, const RColumn &column) final
 Register a new column. More...
 
std::uint64_t CommitCluster (NTupleSize_t nEntries)
 Finalize the current cluster and create a new one for the following data. More...
 
void CommitClusterGroup ()
 Write out the page locations (page list envelope) for all the committed clusters since the last call of CommitClusterGroup (or the beginning of writing). More...
 
void CommitDataset ()
 Finalize the current cluster and the entrire data set. More...
 
void CommitPage (ColumnHandle_t columnHandle, const RPage &page)
 Write a page to the storage. The column must have been added before. More...
 
void CommitSealedPage (DescriptorId_t columnId, const RPageStorage::RSealedPage &sealedPage)
 Write a preprocessed page to storage. The column must have been added before. More...
 
void CommitSealedPageV (std::span< RPageStorage::RSealedPageGroup > ranges)
 Write a vector of preprocessed pages to storage. The corresponding columns must have been added before. More...
 
void Create (RNTupleModel &model)
 Physically creates the storage container to hold the ntuple (e.g., a keys a TFile or an S3 bucket) To do so, Create() calls CreateImpl() after updating the descriptor. More...
 
void DropColumn (ColumnHandle_t) final
 Unregisters a column. More...
 
virtual RNTupleMetricsGetMetrics () override
 Returns the default metrics object. Subclasses might alternatively provide their own metrics object by overriding this. More...
 
EPageStorageType GetType () final
 Whether the concrete implementation is a sink or a source. More...
 
const RNTupleWriteOptionsGetWriteOptions () const
 Returns the sink's write options. More...
 
RPageSinkoperator= (const RPageSink &)=delete
 
RPageSinkoperator= (RPageSink &&)=default
 
virtual RPage ReservePage (ColumnHandle_t columnHandle, std::size_t nElements)=0
 Get a new, empty page for the given column that can be filled with up to nElements. More...
 
- Public Member Functions inherited from ROOT::Experimental::Detail::RPageStorage
 RPageStorage (const RPageStorage &other)=delete
 
 RPageStorage (RPageStorage &&other)=default
 
 RPageStorage (std::string_view name)
 
virtual ~RPageStorage ()
 
virtual ColumnHandle_t AddColumn (DescriptorId_t fieldId, const RColumn &column)=0
 Register a new column. More...
 
virtual void DropColumn (ColumnHandle_t columnHandle)=0
 Unregisters a column. More...
 
virtual RNTupleMetricsGetMetrics ()=0
 Page storage implementations have their own metrics. More...
 
const std::string & GetNTupleName () const
 Returns the NTuple name. More...
 
virtual EPageStorageType GetType ()=0
 Whether the concrete implementation is a sink or a source. More...
 
RPageStorageoperator= (const RPageStorage &other)=delete
 
RPageStorageoperator= (RPageStorage &&other)=default
 
virtual void ReleasePage (RPage &page)=0
 Every page store needs to be able to free pages it handed out. More...
 
void SetTaskScheduler (RTaskScheduler *taskScheduler)
 

Protected Member Functions

RNTupleLocator CommitClusterGroupImpl (unsigned char *serializedPageList, std::uint32_t length) final
 Returns the locator of the page list envelope of the given buffer that contains the serialized page list. More...
 
std::uint64_t CommitClusterImpl (NTupleSize_t nEntries) final
 Returns the number of bytes written to storage (excluding metadata) More...
 
void CommitDatasetImpl (unsigned char *serializedFooter, std::uint32_t length) final
 
RNTupleLocator CommitPageImpl (ColumnHandle_t columnHandle, const RPage &page) final
 
RNTupleLocator CommitSealedPageImpl (DescriptorId_t columnId, const RSealedPage &sealedPage) final
 
void CreateImpl (const RNTupleModel &model, unsigned char *serializedHeader, std::uint32_t length) final
 
- Protected Member Functions inherited from ROOT::Experimental::Detail::RPageSink
virtual RNTupleLocator CommitClusterGroupImpl (unsigned char *serializedPageList, std::uint32_t length)=0
 Returns the locator of the page list envelope of the given buffer that contains the serialized page list. More...
 
virtual std::uint64_t CommitClusterImpl (NTupleSize_t nEntries)=0
 Returns the number of bytes written to storage (excluding metadata) More...
 
virtual void CommitDatasetImpl (unsigned char *serializedFooter, std::uint32_t length)=0
 
virtual RNTupleLocator CommitPageImpl (ColumnHandle_t columnHandle, const RPage &page)=0
 
virtual RNTupleLocator CommitSealedPageImpl (DescriptorId_t columnId, const RPageStorage::RSealedPage &sealedPage)=0
 
virtual std::vector< RNTupleLocatorCommitSealedPageVImpl (std::span< RPageStorage::RSealedPageGroup > ranges)
 Vector commit of preprocessed pages. More...
 
virtual void CreateImpl (const RNTupleModel &model, unsigned char *serializedHeader, std::uint32_t length)=0
 
void EnableDefaultMetrics (const std::string &prefix)
 Enables the default set of metrics provided by RPageSink. More...
 
RSealedPage SealPage (const RPage &page, const RColumnElementBase &element, int compressionSetting)
 Helper for streaming a page. More...
 

Private Attributes

std::vector< RColumnBuffBufferedColumns
 Vector of buffered column pages. Indexed by column id. More...
 
std::unique_ptr< RCountersfCounters
 
std::unique_ptr< RNTupleModelfInnerModel
 The buffered page sink maintains a copy of the RNTupleModel for the inner sink. More...
 
std::unique_ptr< RPageSinkfInnerSink
 The inner sink, responsible for actually performing I/O. More...
 
RNTupleMetrics fMetrics
 

Additional Inherited Members

- Public Types inherited from ROOT::Experimental::Detail::RPageStorage
using ColumnHandle_t = RColumnHandle
 The column handle identifies a column with the current open page storage. More...
 
using SealedPageSequence_t = std::deque< RSealedPage >
 
- Static Public Member Functions inherited from ROOT::Experimental::Detail::RPageSink
static std::unique_ptr< RPageSinkCreate (std::string_view ntupleName, std::string_view location, const RNTupleWriteOptions &options=RNTupleWriteOptions())
 Guess the concrete derived page source from the file name (location) More...
 
- Static Protected Member Functions inherited from ROOT::Experimental::Detail::RPageSink
static RSealedPage SealPage (const RPage &page, const RColumnElementBase &element, int compressionSetting, void *buf)
 Seal a page using the provided buffer. More...
 
- Protected Attributes inherited from ROOT::Experimental::Detail::RPageSink
std::unique_ptr< RNTupleCompressorfCompressor
 Helper to zip pages and header/footer; includes a 16MB (kMAXZIPBUF) zip buffer. More...
 
std::unique_ptr< RCountersfCounters
 
RNTupleDescriptorBuilder fDescriptorBuilder
 
RNTupleMetrics fMetrics
 
std::uint64_t fNextClusterInGroup = 0
 Remembers the starting cluster id for the next cluster group. More...
 
std::vector< RClusterDescriptor::RColumnRangefOpenColumnRanges
 Keeps track of the number of elements in the currently open cluster. Indexed by column id. More...
 
std::vector< RClusterDescriptor::RPageRangefOpenPageRanges
 Keeps track of the written pages in the currently open cluster. Indexed by column id. More...
 
std::unique_ptr< RNTupleWriteOptionsfOptions
 
NTupleSize_t fPrevClusterNEntries = 0
 Used to calculate the number of entries in the current cluster. More...
 
- Protected Attributes inherited from ROOT::Experimental::Detail::RPageStorage
std::string fNTupleName
 
RTaskSchedulerfTaskScheduler = nullptr
 

#include <ROOT/RPageSinkBuf.hxx>

Inheritance diagram for ROOT::Experimental::Detail::RPageSinkBuf:
[legend]

Constructor & Destructor Documentation

◆ RPageSinkBuf() [1/3]

ROOT::Experimental::Detail::RPageSinkBuf::RPageSinkBuf ( std::unique_ptr< RPageSink inner)
explicit

Definition at line 24 of file RPageSinkBuf.cxx.

◆ RPageSinkBuf() [2/3]

ROOT::Experimental::Detail::RPageSinkBuf::RPageSinkBuf ( const RPageSinkBuf )
delete

◆ RPageSinkBuf() [3/3]

ROOT::Experimental::Detail::RPageSinkBuf::RPageSinkBuf ( RPageSinkBuf &&  )
default

◆ ~RPageSinkBuf()

virtual ROOT::Experimental::Detail::RPageSinkBuf::~RPageSinkBuf ( )
virtualdefault

Member Function Documentation

◆ CommitClusterGroupImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Detail::RPageSinkBuf::CommitClusterGroupImpl ( unsigned char *  serializedPageList,
std::uint32_t  length 
)
finalprotectedvirtual

Returns the locator of the page list envelope of the given buffer that contains the serialized page list.

Typically, the implementation takes care of compressing and writing the provided buffer.

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 138 of file RPageSinkBuf.cxx.

◆ CommitClusterImpl()

std::uint64_t ROOT::Experimental::Detail::RPageSinkBuf::CommitClusterImpl ( NTupleSize_t  nEntries)
finalprotectedvirtual

Returns the number of bytes written to storage (excluding metadata)

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 90 of file RPageSinkBuf.cxx.

◆ CommitDatasetImpl()

void ROOT::Experimental::Detail::RPageSinkBuf::CommitDatasetImpl ( unsigned char *  serializedFooter,
std::uint32_t  length 
)
finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 146 of file RPageSinkBuf.cxx.

◆ CommitPageImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Detail::RPageSinkBuf::CommitPageImpl ( ColumnHandle_t  columnHandle,
const RPage page 
)
finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 46 of file RPageSinkBuf.cxx.

◆ CommitSealedPageImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Detail::RPageSinkBuf::CommitSealedPageImpl ( DescriptorId_t  columnId,
const RSealedPage sealedPage 
)
finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 80 of file RPageSinkBuf.cxx.

◆ CreateImpl()

void ROOT::Experimental::Detail::RPageSinkBuf::CreateImpl ( const RNTupleModel model,
unsigned char *  serializedHeader,
std::uint32_t  length 
)
finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 36 of file RPageSinkBuf.cxx.

◆ GetMetrics()

RNTupleMetrics & ROOT::Experimental::Detail::RPageSinkBuf::GetMetrics ( )
inlinefinalvirtual

Returns the default metrics object. Subclasses might alternatively provide their own metrics object by overriding this.

Reimplemented from ROOT::Experimental::Detail::RPageSink.

Definition at line 148 of file RPageSinkBuf.hxx.

◆ operator=() [1/2]

RPageSinkBuf & ROOT::Experimental::Detail::RPageSinkBuf::operator= ( const RPageSinkBuf )
delete

◆ operator=() [2/2]

RPageSinkBuf & ROOT::Experimental::Detail::RPageSinkBuf::operator= ( RPageSinkBuf &&  )
default

◆ ReleasePage()

void ROOT::Experimental::Detail::RPageSinkBuf::ReleasePage ( RPage page)
finalvirtual

Every page store needs to be able to free pages it handed out.

But Sinks and sources have different means of allocating pages.

Implements ROOT::Experimental::Detail::RPageStorage.

Definition at line 158 of file RPageSinkBuf.cxx.

◆ ReservePage()

ROOT::Experimental::Detail::RPage ROOT::Experimental::Detail::RPageSinkBuf::ReservePage ( ColumnHandle_t  columnHandle,
std::size_t  nElements 
)
finalvirtual

Get a new, empty page for the given column that can be filled with up to nElements.

If nElements is zero, the page sink picks an appropriate size.

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 153 of file RPageSinkBuf.cxx.

Member Data Documentation

◆ fBufferedColumns

std::vector<RColumnBuf> ROOT::Experimental::Detail::RPageSinkBuf::fBufferedColumns
private

Vector of buffered column pages. Indexed by column id.

Definition at line 127 of file RPageSinkBuf.hxx.

◆ fCounters

std::unique_ptr<RCounters> ROOT::Experimental::Detail::RPageSinkBuf::fCounters
private

Definition at line 119 of file RPageSinkBuf.hxx.

◆ fInnerModel

std::unique_ptr<RNTupleModel> ROOT::Experimental::Detail::RPageSinkBuf::fInnerModel
private

The buffered page sink maintains a copy of the RNTupleModel for the inner sink.

For the unbuffered case, the RNTupleModel is instead managed by a RNTupleWriter.

Definition at line 125 of file RPageSinkBuf.hxx.

◆ fInnerSink

std::unique_ptr<RPageSink> ROOT::Experimental::Detail::RPageSinkBuf::fInnerSink
private

The inner sink, responsible for actually performing I/O.

Definition at line 122 of file RPageSinkBuf.hxx.

◆ fMetrics

RNTupleMetrics ROOT::Experimental::Detail::RPageSinkBuf::fMetrics
private

Definition at line 120 of file RPageSinkBuf.hxx.

Libraries for ROOT::Experimental::Detail::RPageSinkBuf:
[legend]

The documentation for this class was generated from the following files: