Logo ROOT  
Reference Guide
 
Loading...
Searching...
No Matches
ROOT::Experimental::Internal::RPageSinkFile Class Reference

Storage provider that write ntuple pages into a file.

The written file can be either in ROOT format or in RNTuple bare format.

Definition at line 56 of file RPageStorageFile.hxx.

Public Member Functions

 RPageSinkFile (const RPageSinkFile &)=delete
 
 RPageSinkFile (RPageSinkFile &&)=default
 
 RPageSinkFile (std::string_view ntupleName, std::string_view path, const RNTupleWriteOptions &options)
 
 RPageSinkFile (std::string_view ntupleName, TFile &file, const RNTupleWriteOptions &options)
 
 ~RPageSinkFile () override
 
RPageSinkFileoperator= (const RPageSinkFile &)=delete
 
RPageSinkFileoperator= (RPageSinkFile &&)=default
 
void ReleasePage (RPage &page) final
 Every page store needs to be able to free pages it handed out.
 
RPage ReservePage (ColumnHandle_t columnHandle, std::size_t nElements) final
 Get a new, empty page for the given column that can be filled with up to nElements.
 
- Public Member Functions inherited from ROOT::Experimental::Internal::RPagePersistentSink
 RPagePersistentSink (const RPagePersistentSink &)=delete
 
 RPagePersistentSink (RPagePersistentSink &&)=default
 
 RPagePersistentSink (std::string_view ntupleName, const RNTupleWriteOptions &options)
 
 ~RPagePersistentSink () override
 
ColumnHandle_t AddColumn (DescriptorId_t fieldId, const RColumn &column) final
 Register a new column.
 
std::uint64_t CommitCluster (NTupleSize_t nEntries) final
 Finalize the current cluster and create a new one for the following data.
 
void CommitClusterGroup () final
 Write out the page locations (page list envelope) for all the committed clusters since the last call of CommitClusterGroup (or the beginning of writing).
 
void CommitPage (ColumnHandle_t columnHandle, const RPage &page) final
 Write a page to the storage. The column must have been added before.
 
void CommitSealedPage (DescriptorId_t physicalColumnId, const RPageStorage::RSealedPage &sealedPage) final
 Write a preprocessed page to storage. The column must have been added before.
 
void CommitSealedPageV (std::span< RPageStorage::RSealedPageGroup > ranges) final
 Write a vector of preprocessed pages to storage. The corresponding columns must have been added before.
 
const RNTupleDescriptorGetDescriptor () const final
 Return the RNTupleDescriptor being constructed.
 
void InitFromDescriptor (const RNTupleDescriptor &descriptor)
 Initialize sink based on an existing descriptor and fill into the descriptor builder.
 
RPagePersistentSinkoperator= (const RPagePersistentSink &)=delete
 
RPagePersistentSinkoperator= (RPagePersistentSink &&)=default
 
void UpdateExtraTypeInfo (const RExtraTypeInfoDescriptor &extraTypeInfo) final
 Adds an extra type information record to schema.
 
void UpdateSchema (const RNTupleModelChangeset &changeset, NTupleSize_t firstEntry) final
 Incorporate incremental changes to the model into the ntuple descriptor.
 
- Public Member Functions inherited from ROOT::Experimental::Internal::RPageSink
 RPageSink (const RPageSink &)=delete
 
 RPageSink (RPageSink &&)=default
 
 RPageSink (std::string_view ntupleName, const RNTupleWriteOptions &options)
 
 ~RPageSink () override
 
void CommitDataset ()
 Run the registered callbacks and finalize the current cluster and the entrire data set.
 
void DropColumn (ColumnHandle_t) final
 Unregisters a column.
 
virtual RSinkGuard GetSinkGuard ()
 
EPageStorageType GetType () final
 Whether the concrete implementation is a sink or a source.
 
const RNTupleWriteOptionsGetWriteOptions () const
 Returns the sink's write options.
 
void Init (RNTupleModel &model)
 Physically creates the storage container to hold the ntuple (e.g., a keys a TFile or an S3 bucket) Init() associates column handles to the columns referenced by the model.
 
bool IsInitialized () const
 
RPageSinkoperator= (const RPageSink &)=delete
 
RPageSinkoperator= (RPageSink &&)=default
 
void RegisterOnCommitDatasetCallback (Callback_t callback)
 The registered callback is executed at the beginning of CommitDataset();.
 
- Public Member Functions inherited from ROOT::Experimental::Internal::RPageStorage
 RPageStorage (const RPageStorage &other)=delete
 
 RPageStorage (RPageStorage &&other)=default
 
 RPageStorage (std::string_view name)
 
virtual ~RPageStorage ()
 
virtual Detail::RNTupleMetricsGetMetrics ()
 Returns the default metrics object.
 
const std::string & GetNTupleName () const
 Returns the NTuple name.
 
RPageStorageoperator= (const RPageStorage &other)=delete
 
RPageStorageoperator= (RPageStorage &&other)=default
 
void SetTaskScheduler (RTaskScheduler *taskScheduler)
 

Protected Member Functions

RNTupleLocator CommitClusterGroupImpl (unsigned char *serializedPageList, std::uint32_t length) final
 Returns the locator of the page list envelope of the given buffer that contains the serialized page list.
 
std::uint64_t CommitClusterImpl () final
 Returns the number of bytes written to storage (excluding metadata)
 
void CommitDatasetImpl () final
 
void CommitDatasetImpl (unsigned char *serializedFooter, std::uint32_t length) final
 
virtual void CommitDatasetImpl (unsigned char *serializedFooter, std::uint32_t length)=0
 
RNTupleLocator CommitPageImpl (ColumnHandle_t columnHandle, const RPage &page) final
 
RNTupleLocator CommitSealedPageImpl (DescriptorId_t physicalColumnId, const RPageStorage::RSealedPage &sealedPage) final
 
std::vector< RNTupleLocatorCommitSealedPageVImpl (std::span< RPageStorage::RSealedPageGroup > ranges) final
 Vector commit of preprocessed pages.
 
void InitImpl (RNTupleModel &model) final
 Updates the descriptor and calls InitImpl() that handles the backend-specific details (file, DAOS, etc.)
 
void InitImpl (unsigned char *serializedHeader, std::uint32_t length) final
 
virtual void InitImpl (unsigned char *serializedHeader, std::uint32_t length)=0
 
- Protected Member Functions inherited from ROOT::Experimental::Internal::RPagePersistentSink
void EnableDefaultMetrics (const std::string &prefix)
 Enables the default set of metrics provided by RPageSink.
 
- Protected Member Functions inherited from ROOT::Experimental::Internal::RPageSink
RSealedPage SealPage (const RPage &page, const RColumnElementBase &element)
 Helper for streaming a page.
 
- Protected Member Functions inherited from ROOT::Experimental::Internal::RPageStorage
void WaitForAllTasks ()
 

Private Member Functions

 RPageSinkFile (std::string_view ntupleName, const RNTupleWriteOptions &options)
 
RNTupleLocator WriteSealedPage (const RPageStorage::RSealedPage &sealedPage, std::size_t bytesPacked)
 

Private Attributes

std::uint64_t fNBytesCurrentCluster = 0
 Number of bytes committed to storage in the current cluster.
 
std::unique_ptr< RPageAllocatorHeapfPageAllocator
 
std::unique_ptr< RNTupleFileWriterfWriter
 

Additional Inherited Members

- Public Types inherited from ROOT::Experimental::Internal::RPageSink
using Callback_t = std::function< void(RPageSink &)>
 
- Public Types inherited from ROOT::Experimental::Internal::RPageStorage
using ColumnHandle_t = RColumnHandle
 The column handle identifies a column with the current open page storage.
 
using SealedPageSequence_t = std::deque< RSealedPage >
 
- Static Public Member Functions inherited from ROOT::Experimental::Internal::RPagePersistentSink
static std::unique_ptr< RPageSinkCreate (std::string_view ntupleName, std::string_view location, const RNTupleWriteOptions &options=RNTupleWriteOptions())
 Guess the concrete derived page source from the location.
 
- Static Protected Member Functions inherited from ROOT::Experimental::Internal::RPageSink
static RSealedPage SealPage (const RPage &page, const RColumnElementBase &element, int compressionSetting, void *buf, bool allowAlias=true)
 Seal a page using the provided buffer.
 
- Protected Attributes inherited from ROOT::Experimental::Internal::RPagePersistentSink
std::unique_ptr< RCountersfCounters
 
Internal::RNTupleDescriptorBuilder fDescriptorBuilder
 
- Protected Attributes inherited from ROOT::Experimental::Internal::RPageSink
std::unique_ptr< RNTupleCompressorfCompressor
 Helper to zip pages and header/footer; includes a 16MB (kMAXZIPBUF) zip buffer.
 
std::unique_ptr< RNTupleWriteOptionsfOptions
 
- Protected Attributes inherited from ROOT::Experimental::Internal::RPageStorage
Detail::RNTupleMetrics fMetrics
 
std::string fNTupleName
 
RTaskSchedulerfTaskScheduler = nullptr
 

#include <ROOT/RPageStorageFile.hxx>

Inheritance diagram for ROOT::Experimental::Internal::RPageSinkFile:
[legend]

Constructor & Destructor Documentation

◆ RPageSinkFile() [1/5]

ROOT::Experimental::Internal::RPageSinkFile::RPageSinkFile ( std::string_view  ntupleName,
const RNTupleWriteOptions options 
)
private

Definition at line 45 of file RPageStorageFile.cxx.

◆ RPageSinkFile() [2/5]

ROOT::Experimental::Internal::RPageSinkFile::RPageSinkFile ( std::string_view  ntupleName,
std::string_view  path,
const RNTupleWriteOptions options 
)

Definition at line 58 of file RPageStorageFile.cxx.

◆ RPageSinkFile() [3/5]

ROOT::Experimental::Internal::RPageSinkFile::RPageSinkFile ( std::string_view  ntupleName,
TFile file,
const RNTupleWriteOptions options 
)

Definition at line 66 of file RPageStorageFile.cxx.

◆ RPageSinkFile() [4/5]

ROOT::Experimental::Internal::RPageSinkFile::RPageSinkFile ( const RPageSinkFile )
delete

◆ RPageSinkFile() [5/5]

ROOT::Experimental::Internal::RPageSinkFile::RPageSinkFile ( RPageSinkFile &&  )
default

◆ ~RPageSinkFile()

ROOT::Experimental::Internal::RPageSinkFile::~RPageSinkFile ( )
override

Definition at line 73 of file RPageStorageFile.cxx.

Member Function Documentation

◆ CommitClusterGroupImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Internal::RPageSinkFile::CommitClusterGroupImpl ( unsigned char *  serializedPageList,
std::uint32_t  length 
)
finalprotectedvirtual

Returns the locator of the page list envelope of the given buffer that contains the serialized page list.

Typically, the implementation takes care of compressing and writing the provided buffer.

Implements ROOT::Experimental::Internal::RPagePersistentSink.

Definition at line 182 of file RPageStorageFile.cxx.

◆ CommitClusterImpl()

std::uint64_t ROOT::Experimental::Internal::RPageSinkFile::CommitClusterImpl ( )
finalprotectedvirtual

Returns the number of bytes written to storage (excluding metadata)

Implements ROOT::Experimental::Internal::RPagePersistentSink.

Definition at line 174 of file RPageStorageFile.cxx.

◆ CommitDatasetImpl() [1/3]

void ROOT::Experimental::Internal::RPagePersistentSink::CommitDatasetImpl ( )
finalprotectedvirtual

Reimplemented from ROOT::Experimental::Internal::RPagePersistentSink.

Definition at line 415 of file RPageStorage.cxx.

◆ CommitDatasetImpl() [2/3]

void ROOT::Experimental::Internal::RPageSinkFile::CommitDatasetImpl ( unsigned char *  serializedFooter,
std::uint32_t  length 
)
finalprotectedvirtual

◆ CommitDatasetImpl() [3/3]

virtual void ROOT::Experimental::Internal::RPagePersistentSink::CommitDatasetImpl ( unsigned char *  serializedFooter,
std::uint32_t  length 
)
protectedvirtual

◆ CommitPageImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Internal::RPageSinkFile::CommitPageImpl ( ColumnHandle_t  columnHandle,
const RPage page 
)
finalprotectedvirtual

◆ CommitSealedPageImpl()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Internal::RPageSinkFile::CommitSealedPageImpl ( DescriptorId_t  physicalColumnId,
const RPageStorage::RSealedPage sealedPage 
)
finalprotectedvirtual

◆ CommitSealedPageVImpl()

std::vector< ROOT::Experimental::RNTupleLocator > ROOT::Experimental::Internal::RPageSinkFile::CommitSealedPageVImpl ( std::span< RPageStorage::RSealedPageGroup ranges)
finalprotectedvirtual

Vector commit of preprocessed pages.

The ranges array specifies a range of sealed pages to be committed for each column. The returned vector contains, in order, the RNTupleLocator for each page on each range in ranges, i.e. the first N entries refer to the N pages in ranges[0], followed by M entries that refer to the M pages in ranges[1], etc. The default is to call CommitSealedPageImpl for each page; derived classes may provide an optimized implementation though.

Reimplemented from ROOT::Experimental::Internal::RPagePersistentSink.

Definition at line 128 of file RPageStorageFile.cxx.

◆ InitImpl() [1/3]

void ROOT::Experimental::Internal::RPagePersistentSink::InitImpl ( RNTupleModel model)
finalprotectedvirtual

Updates the descriptor and calls InitImpl() that handles the backend-specific details (file, DAOS, etc.)

Reimplemented from ROOT::Experimental::Internal::RPagePersistentSink.

Definition at line 403 of file RPageStorage.cxx.

◆ InitImpl() [2/3]

void ROOT::Experimental::Internal::RPageSinkFile::InitImpl ( unsigned char *  serializedHeader,
std::uint32_t  length 
)
finalprotectedvirtual

◆ InitImpl() [3/3]

virtual void ROOT::Experimental::Internal::RPagePersistentSink::InitImpl ( unsigned char *  serializedHeader,
std::uint32_t  length 
)
protectedvirtual

◆ operator=() [1/2]

RPageSinkFile & ROOT::Experimental::Internal::RPageSinkFile::operator= ( const RPageSinkFile )
delete

◆ operator=() [2/2]

RPageSinkFile & ROOT::Experimental::Internal::RPageSinkFile::operator= ( RPageSinkFile &&  )
default

◆ ReleasePage()

void ROOT::Experimental::Internal::RPageSinkFile::ReleasePage ( RPage page)
finalvirtual

Every page store needs to be able to free pages it handed out.

But Sinks and sources have different means of allocating pages.

Implements ROOT::Experimental::Internal::RPageStorage.

Definition at line 214 of file RPageStorageFile.cxx.

◆ ReservePage()

ROOT::Experimental::Internal::RPage ROOT::Experimental::Internal::RPageSinkFile::ReservePage ( ColumnHandle_t  columnHandle,
std::size_t  nElements 
)
finalvirtual

Get a new, empty page for the given column that can be filled with up to nElements.

If nElements is zero, the page sink picks an appropriate size.

Implements ROOT::Experimental::Internal::RPageSink.

Definition at line 206 of file RPageStorageFile.cxx.

◆ WriteSealedPage()

ROOT::Experimental::RNTupleLocator ROOT::Experimental::Internal::RPageSinkFile::WriteSealedPage ( const RPageStorage::RSealedPage sealedPage,
std::size_t  bytesPacked 
)
inlineprivate

Definition at line 84 of file RPageStorageFile.cxx.

Member Data Documentation

◆ fNBytesCurrentCluster

std::uint64_t ROOT::Experimental::Internal::RPageSinkFile::fNBytesCurrentCluster = 0
private

Number of bytes committed to storage in the current cluster.

Definition at line 62 of file RPageStorageFile.hxx.

◆ fPageAllocator

std::unique_ptr<RPageAllocatorHeap> ROOT::Experimental::Internal::RPageSinkFile::fPageAllocator
private

Definition at line 58 of file RPageStorageFile.hxx.

◆ fWriter

std::unique_ptr<RNTupleFileWriter> ROOT::Experimental::Internal::RPageSinkFile::fWriter
private

Definition at line 60 of file RPageStorageFile.hxx.

Libraries for ROOT::Experimental::Internal::RPageSinkFile:

The documentation for this class was generated from the following files: