Logo ROOT  
Reference Guide
ROOT::Experimental::Detail::RPageSinkDaos Class Reference

Storage provider that writes ntuple pages to into a DAOS container.

Currently, an object is allocated for ntuple metadata (anchor/header/footer). Objects can correspond to pages or clusters of pages depending on the RNTuple-DAOS mapping strategy.

Definition at line 92 of file RPageStorageDaos.hxx.

Public Member Functions

 RPageSinkDaos (std::string_view ntupleName, std::string_view uri, const RNTupleWriteOptions &options)
 
virtual ~RPageSinkDaos ()
 
void ReleasePage (RPage &page) final
 Every page store needs to be able to free pages it handed out. More...
 
RPage ReservePage (ColumnHandle_t columnHandle, std::size_t nElements) final
 Get a new, empty page for the given column that can be filled with up to nElements. More...
 
- Public Member Functions inherited from ROOT::Experimental::Detail::RPageSink
 RPageSink (const RPageSink &)=delete
 
 RPageSink (RPageSink &&)=default
 
 RPageSink (std::string_view ntupleName, const RNTupleWriteOptions &options)
 
virtual ~RPageSink ()
 
ColumnHandle_t AddColumn (DescriptorId_t fieldId, const RColumn &column) final
 Register a new column. More...
 
std::uint64_t CommitCluster (NTupleSize_t nEntries)
 Finalize the current cluster and create a new one for the following data. More...
 
void CommitClusterGroup ()
 Write out the page locations (page list envelope) for all the committed clusters since the last call of CommitClusterGroup (or the beginning of writing). More...
 
void CommitDataset ()
 Finalize the current cluster and the entrire data set. More...
 
void CommitPage (ColumnHandle_t columnHandle, const RPage &page)
 Write a page to the storage. The column must have been added before. More...
 
void CommitSealedPage (DescriptorId_t columnId, const RPageStorage::RSealedPage &sealedPage)
 Write a preprocessed page to storage. The column must have been added before. More...
 
void CommitSealedPageV (std::span< RPageStorage::RSealedPageGroup > ranges)
 Write a vector of preprocessed pages to storage. The corresponding columns must have been added before. More...
 
void Create (RNTupleModel &model)
 Physically creates the storage container to hold the ntuple (e.g., a keys a TFile or an S3 bucket) To do so, Create() calls CreateImpl() after updating the descriptor. More...
 
void DropColumn (ColumnHandle_t) final
 Unregisters a column. More...
 
virtual RNTupleMetricsGetMetrics () override
 Returns the default metrics object. Subclasses might alternatively provide their own metrics object by overriding this. More...
 
EPageStorageType GetType () final
 Whether the concrete implementation is a sink or a source. More...
 
const RNTupleWriteOptionsGetWriteOptions () const
 Returns the sink's write options. More...
 
RPageSinkoperator= (const RPageSink &)=delete
 
RPageSinkoperator= (RPageSink &&)=default
 
virtual RPage ReservePage (ColumnHandle_t columnHandle, std::size_t nElements)=0
 Get a new, empty page for the given column that can be filled with up to nElements. More...
 
- Public Member Functions inherited from ROOT::Experimental::Detail::RPageStorage
 RPageStorage (const RPageStorage &other)=delete
 
 RPageStorage (RPageStorage &&other)=default
 
 RPageStorage (std::string_view name)
 
virtual ~RPageStorage ()
 
virtual ColumnHandle_t AddColumn (DescriptorId_t fieldId, const RColumn &column)=0
 Register a new column. More...
 
virtual void DropColumn (ColumnHandle_t columnHandle)=0
 Unregisters a column. More...
 
virtual RNTupleMetricsGetMetrics ()=0
 Page storage implementations have their own metrics. More...
 
const std::string & GetNTupleName () const
 Returns the NTuple name. More...
 
virtual EPageStorageType GetType ()=0
 Whether the concrete implementation is a sink or a source. More...
 
RPageStorageoperator= (const RPageStorage &other)=delete
 
RPageStorageoperator= (RPageStorage &&other)=default
 
virtual void ReleasePage (RPage &page)=0
 Every page store needs to be able to free pages it handed out. More...
 
void SetTaskScheduler (RTaskScheduler *taskScheduler)
 

Protected Member Functions

RNTupleLocator CommitClusterGroupImpl (unsigned char *serializedPageList, std::uint32_t length) final
 Returns the locator of the page list envelope of the given buffer that contains the serialized page list. More...
 
std::uint64_t CommitClusterImpl (NTupleSize_t nEntries) final
 Returns the number of bytes written to storage (excluding metadata) More...
 
void CommitDatasetImpl (unsigned char *serializedFooter, std::uint32_t length) final
 
RNTupleLocator CommitPageImpl (ColumnHandle_t columnHandle, const RPage &page) final
 
RNTupleLocator CommitSealedPageImpl (DescriptorId_t columnId, const RPageStorage::RSealedPage &sealedPage) final
 
std::vector< RNTupleLocatorCommitSealedPageVImpl (std::span< RPageStorage::RSealedPageGroup > ranges) final
 Vector commit of preprocessed pages. More...
 
void CreateImpl (const RNTupleModel &model, unsigned char *serializedHeader, std::uint32_t length) final
 
void WriteNTupleAnchor ()
 
void WriteNTupleFooter (const void *data, size_t nbytes, size_t lenFooter)
 
void WriteNTupleHeader (const void *data, size_t nbytes, size_t lenHeader)
 
- Protected Member Functions inherited from ROOT::Experimental::Detail::RPageSink
virtual RNTupleLocator CommitClusterGroupImpl (unsigned char *serializedPageList, std::uint32_t length)=0
 Returns the locator of the page list envelope of the given buffer that contains the serialized page list. More...
 
virtual std::uint64_t CommitClusterImpl (NTupleSize_t nEntries)=0
 Returns the number of bytes written to storage (excluding metadata) More...
 
virtual void CommitDatasetImpl (unsigned char *serializedFooter, std::uint32_t length)=0
 
virtual RNTupleLocator CommitPageImpl (ColumnHandle_t columnHandle, const RPage &page)=0
 
virtual RNTupleLocator CommitSealedPageImpl (DescriptorId_t columnId, const RPageStorage::RSealedPage &sealedPage)=0
 
virtual std::vector< RNTupleLocatorCommitSealedPageVImpl (std::span< RPageStorage::RSealedPageGroup > ranges)
 Vector commit of preprocessed pages. More...
 
virtual void CreateImpl (const RNTupleModel &model, unsigned char *serializedHeader, std::uint32_t length)=0
 
void EnableDefaultMetrics (const std::string &prefix)
 Enables the default set of metrics provided by RPageSink. More...
 
RSealedPage SealPage (const RPage &page, const RColumnElementBase &element, int compressionSetting)
 Helper for streaming a page. More...
 

Private Attributes

std::atomic< std::uint64_t > fClusterGroupId {0}
 Cluster group counter for the next committed cluster pagelist; incremented in CommitClusterGroupImpl() More...
 
std::unique_ptr< RDaosContainerfDaosContainer
 Underlying DAOS container. More...
 
std::uint64_t fNBytesCurrentCluster {0}
 Tracks the number of bytes committed to the current cluster. More...
 
RDaosNTupleAnchor fNTupleAnchor
 
std::unique_ptr< RPageAllocatorHeapfPageAllocator
 
std::atomic< std::uint64_t > fPageId {0}
 Page identifier for the next committed page; it is automatically incremented in CommitSealedPageImpl() More...
 
std::string fURI
 A URI to a DAOS pool of the form 'daos://pool-label/container-label'. More...
 

Additional Inherited Members

- Public Types inherited from ROOT::Experimental::Detail::RPageStorage
using ColumnHandle_t = RColumnHandle
 The column handle identifies a column with the current open page storage. More...
 
using SealedPageSequence_t = std::deque< RSealedPage >
 
- Static Public Member Functions inherited from ROOT::Experimental::Detail::RPageSink
static std::unique_ptr< RPageSinkCreate (std::string_view ntupleName, std::string_view location, const RNTupleWriteOptions &options=RNTupleWriteOptions())
 Guess the concrete derived page source from the file name (location) More...
 
- Static Protected Member Functions inherited from ROOT::Experimental::Detail::RPageSink
static RSealedPage SealPage (const RPage &page, const RColumnElementBase &element, int compressionSetting, void *buf)
 Seal a page using the provided buffer. More...
 
- Protected Attributes inherited from ROOT::Experimental::Detail::RPageSink
std::unique_ptr< RNTupleCompressorfCompressor
 Helper to zip pages and header/footer; includes a 16MB (kMAXZIPBUF) zip buffer. More...
 
std::unique_ptr< RCountersfCounters
 
RNTupleDescriptorBuilder fDescriptorBuilder
 
RNTupleMetrics fMetrics
 
std::uint64_t fNextClusterInGroup = 0
 Remembers the starting cluster id for the next cluster group. More...
 
std::vector< RClusterDescriptor::RColumnRangefOpenColumnRanges
 Keeps track of the number of elements in the currently open cluster. Indexed by column id. More...
 
std::vector< RClusterDescriptor::RPageRangefOpenPageRanges
 Keeps track of the written pages in the currently open cluster. Indexed by column id. More...
 
std::unique_ptr< RNTupleWriteOptionsfOptions
 
NTupleSize_t fPrevClusterNEntries = 0
 Used to calculate the number of entries in the current cluster. More...
 
- Protected Attributes inherited from ROOT::Experimental::Detail::RPageStorage
std::string fNTupleName
 
RTaskSchedulerfTaskScheduler = nullptr
 

#include <ROOT/RPageStorageDaos.hxx>

Inheritance diagram for ROOT::Experimental::Detail::RPageSinkDaos:
[legend]

Constructor & Destructor Documentation

◆ RPageSinkDaos()

ROOT::Experimental::Detail::RPageSinkDaos::RPageSinkDaos ( std::string_view  ntupleName,
std::string_view  uri,
const RNTupleWriteOptions options 
)

Definition at line 154 of file RPageStorageDaos.cxx.

◆ ~RPageSinkDaos()

ROOT::Experimental::Detail::RPageSinkDaos::~RPageSinkDaos ( )
virtualdefault

Member Function Documentation

◆ CommitClusterGroupImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Detail::RPageSinkDaos::CommitClusterGroupImpl ( unsigned char *  serializedPageList,
std::uint32_t  length 
)
finalprotectedvirtual

Returns the locator of the page list envelope of the given buffer that contains the serialized page list.

Typically, the implementation takes care of compressing and writing the provided buffer.

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 282 of file RPageStorageDaos.cxx.

◆ CommitClusterImpl()

std::uint64_t ROOT::Experimental::Detail::RPageSinkDaos::CommitClusterImpl ( NTupleSize_t  nEntries)
finalprotectedvirtual

Returns the number of bytes written to storage (excluding metadata)

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 276 of file RPageStorageDaos.cxx.

◆ CommitDatasetImpl()

void ROOT::Experimental::Detail::RPageSinkDaos::CommitDatasetImpl ( unsigned char *  serializedFooter,
std::uint32_t  length 
)
finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 299 of file RPageStorageDaos.cxx.

◆ CommitPageImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Detail::RPageSinkDaos::CommitPageImpl ( ColumnHandle_t  columnHandle,
const RPage page 
)
finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 191 of file RPageStorageDaos.cxx.

◆ CommitSealedPageImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Detail::RPageSinkDaos::CommitSealedPageImpl ( DescriptorId_t  columnId,
const RPageStorage::RSealedPage sealedPage 
)
finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 205 of file RPageStorageDaos.cxx.

◆ CommitSealedPageVImpl()

std::vector< ROOT::Experimental::RNTupleLocator > ROOT::Experimental::Detail::RPageSinkDaos::CommitSealedPageVImpl ( std::span< RPageStorage::RSealedPageGroup ranges)
finalprotectedvirtual

Vector commit of preprocessed pages.

The ranges array specifies a range of sealed pages to be committed for each column. The returned vector contains, in order, the RNTupleLocator for each page on each range in ranges, i.e. the first N entries refer to the N pages in ranges[0], followed by M entries that refer to the M pages in ranges[1], etc. The default is to call CommitSealedPageImpl for each page; derived classes may provide an optimized implementation though.

Reimplemented from ROOT::Experimental::Detail::RPageSink.

Definition at line 227 of file RPageStorageDaos.cxx.

◆ CreateImpl()

void ROOT::Experimental::Detail::RPageSinkDaos::CreateImpl ( const RNTupleModel model,
unsigned char *  serializedHeader,
std::uint32_t  length 
)
finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 169 of file RPageStorageDaos.cxx.

◆ ReleasePage()

void ROOT::Experimental::Detail::RPageSinkDaos::ReleasePage ( RPage page)
finalvirtual

Every page store needs to be able to free pages it handed out.

But Sinks and sources have different means of allocating pages.

Implements ROOT::Experimental::Detail::RPageStorage.

Definition at line 343 of file RPageStorageDaos.cxx.

◆ ReservePage()

ROOT::Experimental::Detail::RPage ROOT::Experimental::Detail::RPageSinkDaos::ReservePage ( ColumnHandle_t  columnHandle,
std::size_t  nElements 
)
finalvirtual

Get a new, empty page for the given column that can be filled with up to nElements.

If nElements is zero, the page sink picks an appropriate size.

Implements ROOT::Experimental::Detail::RPageSink.

Definition at line 335 of file RPageStorageDaos.cxx.

◆ WriteNTupleAnchor()

void ROOT::Experimental::Detail::RPageSinkDaos::WriteNTupleAnchor ( )
protected

Definition at line 326 of file RPageStorageDaos.cxx.

◆ WriteNTupleFooter()

void ROOT::Experimental::Detail::RPageSinkDaos::WriteNTupleFooter ( const void *  data,
size_t  nbytes,
size_t  lenFooter 
)
protected

Definition at line 317 of file RPageStorageDaos.cxx.

◆ WriteNTupleHeader()

void ROOT::Experimental::Detail::RPageSinkDaos::WriteNTupleHeader ( const void *  data,
size_t  nbytes,
size_t  lenHeader 
)
protected

Definition at line 308 of file RPageStorageDaos.cxx.

Member Data Documentation

◆ fClusterGroupId

std::atomic<std::uint64_t> ROOT::Experimental::Detail::RPageSinkDaos::fClusterGroupId {0}
private

Cluster group counter for the next committed cluster pagelist; incremented in CommitClusterGroupImpl()

Definition at line 104 of file RPageStorageDaos.hxx.

◆ fDaosContainer

std::unique_ptr<RDaosContainer> ROOT::Experimental::Detail::RPageSinkDaos::fDaosContainer
private

Underlying DAOS container.

An internal std::shared_ptr keep the pool connection alive. ISO C++ ensures the correct destruction order, i.e., ~RDaosContainer is invoked first (which calls daos_cont_close(); the destructor for the std::shared_ptr<RDaosPool> is invoked after (which calls daos_pool_disconect()).

Definition at line 100 of file RPageStorageDaos.hxx.

◆ fNBytesCurrentCluster

std::uint64_t ROOT::Experimental::Detail::RPageSinkDaos::fNBytesCurrentCluster {0}
private

Tracks the number of bytes committed to the current cluster.

Definition at line 108 of file RPageStorageDaos.hxx.

◆ fNTupleAnchor

RDaosNTupleAnchor ROOT::Experimental::Detail::RPageSinkDaos::fNTupleAnchor
private

Definition at line 110 of file RPageStorageDaos.hxx.

◆ fPageAllocator

std::unique_ptr<RPageAllocatorHeap> ROOT::Experimental::Detail::RPageSinkDaos::fPageAllocator
private

Definition at line 94 of file RPageStorageDaos.hxx.

◆ fPageId

std::atomic<std::uint64_t> ROOT::Experimental::Detail::RPageSinkDaos::fPageId {0}
private

Page identifier for the next committed page; it is automatically incremented in CommitSealedPageImpl()

Definition at line 102 of file RPageStorageDaos.hxx.

◆ fURI

std::string ROOT::Experimental::Detail::RPageSinkDaos::fURI
private

A URI to a DAOS pool of the form 'daos://pool-label/container-label'.

Definition at line 106 of file RPageStorageDaos.hxx.


The documentation for this class was generated from the following files: