Wrapper sink that coalesces cluster column page writes.
Definition at line 40 of file RPageSinkBuf.hxx.
Classes | |
class | RColumnBuf |
A buffered column. More... | |
struct | RCounters |
I/O performance counters that get registered in fMetrics. More... | |
Public Member Functions | |
RPageSinkBuf (const RPageSinkBuf &)=delete | |
RPageSinkBuf (RPageSinkBuf &&)=default | |
RPageSinkBuf (std::unique_ptr< RPageSink > inner) | |
~RPageSinkBuf () override | |
ColumnHandle_t | AddColumn (DescriptorId_t fieldId, const RColumn &column) final |
Register a new column. | |
std::uint64_t | CommitCluster (NTupleSize_t nNewEntries) final |
Finalize the current cluster and create a new one for the following data. | |
void | CommitClusterGroup () final |
Write out the page locations (page list envelope) for all the committed clusters since the last call of CommitClusterGroup (or the beginning of writing). | |
void | CommitDatasetImpl () final |
void | CommitPage (ColumnHandle_t columnHandle, const RPage &page) final |
Write a page to the storage. The column must have been added before. | |
void | CommitSealedPage (DescriptorId_t physicalColumnId, const RSealedPage &sealedPage) final |
Write a preprocessed page to storage. The column must have been added before. | |
void | CommitSealedPageV (std::span< RPageStorage::RSealedPageGroup > ranges) final |
Write a vector of preprocessed pages to storage. The corresponding columns must have been added before. | |
const RNTupleDescriptor & | GetDescriptor () const final |
Return the RNTupleDescriptor being constructed. | |
void | InitImpl (RNTupleModel &model) final |
RPageSinkBuf & | operator= (const RPageSinkBuf &)=delete |
RPageSinkBuf & | operator= (RPageSinkBuf &&)=default |
void | ReleasePage (RPage &page) final |
Every page store needs to be able to free pages it handed out. | |
RPage | ReservePage (ColumnHandle_t columnHandle, std::size_t nElements) final |
Get a new, empty page for the given column that can be filled with up to nElements. | |
void | UpdateExtraTypeInfo (const RExtraTypeInfoDescriptor &extraTypeInfo) final |
Adds an extra type information record to schema. | |
void | UpdateSchema (const RNTupleModelChangeset &changeset, NTupleSize_t firstEntry) final |
Incorporate incremental changes to the model into the ntuple descriptor. | |
![]() | |
RPageSink (const RPageSink &)=delete | |
RPageSink (RPageSink &&)=default | |
RPageSink (std::string_view ntupleName, const RNTupleWriteOptions &options) | |
~RPageSink () override | |
void | CommitDataset () |
Run the registered callbacks and finalize the current cluster and the entrire data set. | |
void | DropColumn (ColumnHandle_t) final |
Unregisters a column. | |
virtual RSinkGuard | GetSinkGuard () |
EPageStorageType | GetType () final |
Whether the concrete implementation is a sink or a source. | |
const RNTupleWriteOptions & | GetWriteOptions () const |
Returns the sink's write options. | |
void | Init (RNTupleModel &model) |
Physically creates the storage container to hold the ntuple (e.g., a keys a TFile or an S3 bucket) Init() associates column handles to the columns referenced by the model. | |
bool | IsInitialized () const |
RPageSink & | operator= (const RPageSink &)=delete |
RPageSink & | operator= (RPageSink &&)=default |
void | RegisterOnCommitDatasetCallback (Callback_t callback) |
The registered callback is executed at the beginning of CommitDataset();. | |
![]() | |
RPageStorage (const RPageStorage &other)=delete | |
RPageStorage (RPageStorage &&other)=default | |
RPageStorage (std::string_view name) | |
virtual | ~RPageStorage () |
virtual Detail::RNTupleMetrics & | GetMetrics () |
Returns the default metrics object. | |
const std::string & | GetNTupleName () const |
Returns the NTuple name. | |
RPageStorage & | operator= (const RPageStorage &other)=delete |
RPageStorage & | operator= (RPageStorage &&other)=default |
void | SetTaskScheduler (RTaskScheduler *taskScheduler) |
Private Member Functions | |
void | ConnectFields (const std::vector< RFieldBase * > &fields, NTupleSize_t firstEntry) |
Private Attributes | |
std::vector< RColumnBuf > | fBufferedColumns |
Vector of buffered column pages. Indexed by column id. | |
std::unique_ptr< RCounters > | fCounters |
std::unique_ptr< RNTupleModel > | fInnerModel |
The buffered page sink maintains a copy of the RNTupleModel for the inner sink. | |
std::unique_ptr< RPageSink > | fInnerSink |
The inner sink, responsible for actually performing I/O. | |
DescriptorId_t | fNColumns = 0 |
DescriptorId_t | fNFields = 0 |
Additional Inherited Members | |
![]() | |
using | Callback_t = std::function< void(RPageSink &)> |
![]() | |
using | ColumnHandle_t = RColumnHandle |
The column handle identifies a column with the current open page storage. | |
using | SealedPageSequence_t = std::deque< RSealedPage > |
![]() | |
RSealedPage | SealPage (const RPage &page, const RColumnElementBase &element) |
Helper for streaming a page. | |
![]() | |
void | WaitForAllTasks () |
![]() | |
static RSealedPage | SealPage (const RPage &page, const RColumnElementBase &element, int compressionSetting, void *buf, bool allowAlias=true) |
Seal a page using the provided buffer. | |
![]() | |
std::unique_ptr< RNTupleCompressor > | fCompressor |
Helper to zip pages and header/footer; includes a 16MB (kMAXZIPBUF) zip buffer. | |
std::unique_ptr< RNTupleWriteOptions > | fOptions |
![]() | |
Detail::RNTupleMetrics | fMetrics |
std::string | fNTupleName |
RTaskScheduler * | fTaskScheduler = nullptr |
#include <ROOT/RPageSinkBuf.hxx>
|
explicit |
Definition at line 39 of file RPageSinkBuf.cxx.
|
delete |
|
default |
|
override |
Definition at line 52 of file RPageSinkBuf.cxx.
|
finalvirtual |
Register a new column.
When reading, the column must exist in the ntuple on disk corresponding to the meta-data. When writing, every column can only be attached once.
Implements ROOT::Experimental::Internal::RPageStorage.
Definition at line 61 of file RPageSinkBuf.cxx.
|
finalvirtual |
Finalize the current cluster and create a new one for the following data.
Returns the number of bytes written to storage (excluding meta-data).
Implements ROOT::Experimental::Internal::RPageSink.
Definition at line 187 of file RPageSinkBuf.cxx.
|
finalvirtual |
Write out the page locations (page list envelope) for all the committed clusters since the last call of CommitClusterGroup (or the beginning of writing).
Implements ROOT::Experimental::Internal::RPageSink.
Definition at line 213 of file RPageSinkBuf.cxx.
|
finalvirtual |
Implements ROOT::Experimental::Internal::RPageSink.
Definition at line 220 of file RPageSinkBuf.cxx.
|
finalvirtual |
Write a page to the storage. The column must have been added before.
Implements ROOT::Experimental::Internal::RPageSink.
Definition at line 139 of file RPageSinkBuf.cxx.
|
finalvirtual |
Write a preprocessed page to storage. The column must have been added before.
Implements ROOT::Experimental::Internal::RPageSink.
Definition at line 176 of file RPageSinkBuf.cxx.
|
finalvirtual |
Write a vector of preprocessed pages to storage. The corresponding columns must have been added before.
Implements ROOT::Experimental::Internal::RPageSink.
Definition at line 182 of file RPageSinkBuf.cxx.
|
private |
Definition at line 66 of file RPageSinkBuf.cxx.
|
finalvirtual |
Return the RNTupleDescriptor being constructed.
Implements ROOT::Experimental::Internal::RPageSink.
Definition at line 84 of file RPageSinkBuf.cxx.
|
finalvirtual |
Implements ROOT::Experimental::Internal::RPageSink.
Definition at line 89 of file RPageSinkBuf.cxx.
|
delete |
|
default |
|
finalvirtual |
Every page store needs to be able to free pages it handed out.
But Sinks and sources have different means of allocating pages.
Implements ROOT::Experimental::Internal::RPageStorage.
Definition at line 233 of file RPageSinkBuf.cxx.
|
finalvirtual |
Get a new, empty page for the given column that can be filled with up to nElements.
If nElements is zero, the page sink picks an appropriate size.
Implements ROOT::Experimental::Internal::RPageSink.
Definition at line 228 of file RPageSinkBuf.cxx.
|
finalvirtual |
Adds an extra type information record to schema.
The extra type information will be written to the extension header. The information in the record will be merged with the existing information, e.g. duplicate streamer info records will be removed. This method is called by the "on commit dataset" callback registered by specific fields (e.g., unsplit field) and during merging.
Implements ROOT::Experimental::Internal::RPageSink.
Definition at line 132 of file RPageSinkBuf.cxx.
|
finalvirtual |
Incorporate incremental changes to the model into the ntuple descriptor.
This happens, e.g. if new fields were added after the initial call to RPageSink::Init(RNTupleModel &)
. firstEntry
specifies the global index for the first stored element in the added columns.
Implements ROOT::Experimental::Internal::RPageSink.
Definition at line 97 of file RPageSinkBuf.cxx.
|
private |
Vector of buffered column pages. Indexed by column id.
Definition at line 128 of file RPageSinkBuf.hxx.
|
private |
Definition at line 121 of file RPageSinkBuf.hxx.
|
private |
The buffered page sink maintains a copy of the RNTupleModel for the inner sink.
For the unbuffered case, the RNTupleModel is instead managed by a RNTupleWriter.
Definition at line 126 of file RPageSinkBuf.hxx.
|
private |
The inner sink, responsible for actually performing I/O.
Definition at line 123 of file RPageSinkBuf.hxx.
|
private |
Definition at line 130 of file RPageSinkBuf.hxx.
|
private |
Definition at line 129 of file RPageSinkBuf.hxx.