Logo ROOT  
Reference Guide
ROOT::Experimental::Detail::RPageSourceFile Class Reference

Storage provider that reads ntuple pages from a file.

Definition at line 117 of file RPageStorageFile.hxx.

Public Member Functions

 RPageSourceFile (const RPageSourceFile &)=delete
 
 RPageSourceFile (RPageSourceFile &&)=default
 
 RPageSourceFile (std::string_view ntupleName, std::string_view path, const RNTupleReadOptions &options)
 
virtual ~RPageSourceFile ()
 
std::unique_ptr< RPageSourceClone () const final
 The cloned page source creates a new raw file and reader and opens its own file descriptor to the data. More...
 
std::vector< std::unique_ptr< RCluster > > LoadClusters (std::span< RCluster::RKey > clusterKeys) final
 Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages. More...
 
void LoadSealedPage (DescriptorId_t columnId, const RClusterIndex &clusterIndex, RSealedPage &sealedPage) final
 Read the packed and compressed bytes of a page into the memory buffer provided by selaedPage. More...
 
RPageSourceFileoperator= (const RPageSourceFile &)=delete
 
RPageSourceFileoperator= (RPageSourceFile &&)=default
 
RPage PopulatePage (ColumnHandle_t columnHandle, const RClusterIndex &clusterIndex) final
 Another version of PopulatePage that allows to specify cluster-relative indexes. More...
 
RPage PopulatePage (ColumnHandle_t columnHandle, NTupleSize_t globalIndex) final
 Allocates and fills a page that contains the index-th element. More...
 
void ReleasePage (RPage &page) final
 Every page store needs to be able to free pages it handed out. More...
 
- Public Member Functions inherited from ROOT::Experimental::Detail::RPageSource
 RPageSource (const RPageSource &)=delete
 
 RPageSource (RPageSource &&)=default
 
 RPageSource (std::string_view ntupleName, const RNTupleReadOptions &fOptions)
 
virtual ~RPageSource ()
 
ColumnHandle_t AddColumn (DescriptorId_t fieldId, const RColumn &column) override
 Register a new column. More...
 
void Attach ()
 Open the physical storage container for the tree. More...
 
virtual std::unique_ptr< RPageSourceClone () const =0
 Open the same storage multiple time, e.g. for reading in multiple threads. More...
 
void DropColumn (ColumnHandle_t columnHandle) override
 Unregisters a column. More...
 
ColumnId_t GetColumnId (ColumnHandle_t columnHandle)
 
const RNTupleDescriptorGetDescriptor () const
 
virtual RNTupleMetricsGetMetrics () override
 Returns the default metrics object. Subclasses might alternatively override the method and provide their own metrics object. More...
 
NTupleSize_t GetNElements (ColumnHandle_t columnHandle)
 
NTupleSize_t GetNEntries ()
 
const RNTupleReadOptionsGetReadOptions () const
 
EPageStorageType GetType () final
 Whether the concrete implementation is a sink or a source. More...
 
virtual std::vector< std::unique_ptr< RCluster > > LoadClusters (std::span< RCluster::RKey > clusterKeys)=0
 Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages. More...
 
virtual void LoadSealedPage (DescriptorId_t columnId, const RClusterIndex &clusterIndex, RSealedPage &sealedPage)=0
 Read the packed and compressed bytes of a page into the memory buffer provided by selaedPage. More...
 
RPageSourceoperator= (const RPageSource &)=delete
 
RPageSourceoperator= (RPageSource &&)=default
 
virtual RPage PopulatePage (ColumnHandle_t columnHandle, const RClusterIndex &clusterIndex)=0
 Another version of PopulatePage that allows to specify cluster-relative indexes. More...
 
virtual RPage PopulatePage (ColumnHandle_t columnHandle, NTupleSize_t globalIndex)=0
 Allocates and fills a page that contains the index-th element. More...
 
void UnzipCluster (RCluster *cluster)
 Parallel decompression and unpacking of the pages in the given cluster. More...
 
- Public Member Functions inherited from ROOT::Experimental::Detail::RPageStorage
 RPageStorage (const RPageStorage &other)=delete
 
 RPageStorage (RPageStorage &&other)=default
 
 RPageStorage (std::string_view name)
 
virtual ~RPageStorage ()
 
virtual ColumnHandle_t AddColumn (DescriptorId_t fieldId, const RColumn &column)=0
 Register a new column. More...
 
virtual void DropColumn (ColumnHandle_t columnHandle)=0
 Unregisters a column. More...
 
virtual RNTupleMetricsGetMetrics ()=0
 Page storage implementations have their own metrics. More...
 
const std::string & GetNTupleName () const
 Returns the NTuple name. More...
 
virtual EPageStorageType GetType ()=0
 Whether the concrete implementation is a sink or a source. More...
 
RPageStorageoperator= (const RPageStorage &other)=delete
 
RPageStorageoperator= (RPageStorage &&other)=default
 
virtual void ReleasePage (RPage &page)=0
 Every page store needs to be able to free pages it handed out. More...
 
void SetTaskScheduler (RTaskScheduler *taskScheduler)
 

Static Public Attributes

static constexpr std::size_t kMaxPageSize = 1024 * 1024
 Cannot process pages larger than 1MB. More...
 

Protected Member Functions

RNTupleDescriptor AttachImpl () final
 
void UnzipClusterImpl (RCluster *cluster) final
 
- Protected Member Functions inherited from ROOT::Experimental::Detail::RPageSource
virtual RNTupleDescriptor AttachImpl ()=0
 
void EnableDefaultMetrics (const std::string &prefix)
 Enables the default set of metrics provided by RPageSource. More...
 
std::unique_ptr< unsigned char[]> UnsealPage (const RSealedPage &sealedPage, const RColumnElementBase &element)
 Helper for unstreaming a page. More...
 
virtual void UnzipClusterImpl (RCluster *)
 

Private Member Functions

 RPageSourceFile (std::string_view ntupleName, const RNTupleReadOptions &options)
 
RPage PopulatePageFromCluster (ColumnHandle_t columnHandle, const RClusterDescriptor &clusterDescriptor, ClusterSize_t::ValueType idxInCluster)
 
std::unique_ptr< RClusterPrepareSingleCluster (const RCluster::RKey &clusterKey, std::vector< ROOT::Internal::RRawFile::RIOVec > &readRequests)
 Helper function for LoadClusters: it prepares the memory buffer (page map) and the read requests for a given cluster and columns. More...
 

Private Attributes

std::unique_ptr< RClusterPoolfClusterPool
 The cluster pool asynchronously preloads the next few clusters. More...
 
RClusterfCurrentCluster = nullptr
 The last cluster from which a page got populated. Points into fClusterPool->fPool. More...
 
std::unique_ptr< ROOT::Internal::RRawFilefFile
 An RRawFile is used to request the necessary byte ranges from a local or a remote file. More...
 
std::unique_ptr< RPageAllocatorFilefPageAllocator
 Populated pages might be shared; there memory buffer is managed by the RPageAllocatorFile. More...
 
std::shared_ptr< RPagePoolfPagePool
 The page pool might, at some point, be used by multiple page sources. More...
 
Internal::RMiniFileReader fReader
 Takes the fFile to read ntuple blobs from it. More...
 

Additional Inherited Members

- Public Types inherited from ROOT::Experimental::Detail::RPageStorage
using ColumnHandle_t = RColumnHandle
 The column handle identifies a column with the current open page storage. More...
 
- Static Public Member Functions inherited from ROOT::Experimental::Detail::RPageSource
static std::unique_ptr< RPageSourceCreate (std::string_view ntupleName, std::string_view location, const RNTupleReadOptions &options=RNTupleReadOptions())
 Guess the concrete derived page source from the file name (location) More...
 
- Protected Attributes inherited from ROOT::Experimental::Detail::RPageSource
RCluster::ColumnSet_t fActiveColumns
 The active columns are implicitly defined by the model fields or views. More...
 
std::unique_ptr< RCountersfCounters
 
std::unique_ptr< RNTupleDecompressorfDecompressor
 Helper to unzip pages and header/footer; comprises a 16MB (kMAXZIPBUF) unzip buffer. More...
 
RNTupleDescriptor fDescriptor
 
RNTupleMetrics fMetrics
 Wraps the I/O counters and is observed by the RNTupleReader metrics. More...
 
RNTupleReadOptions fOptions
 
- Protected Attributes inherited from ROOT::Experimental::Detail::RPageStorage
std::string fNTupleName
 
RTaskSchedulerfTaskScheduler = nullptr
 

#include <ROOT/RPageStorageFile.hxx>

Inheritance diagram for ROOT::Experimental::Detail::RPageSourceFile:
[legend]

Constructor & Destructor Documentation

◆ RPageSourceFile() [1/4]

ROOT::Experimental::Detail::RPageSourceFile::RPageSourceFile ( std::string_view  ntupleName,
const RNTupleReadOptions options 
)
private

Definition at line 212 of file RPageStorageFile.cxx.

◆ RPageSourceFile() [2/4]

ROOT::Experimental::Detail::RPageSourceFile::RPageSourceFile ( std::string_view  ntupleName,
std::string_view  path,
const RNTupleReadOptions options 
)

Definition at line 224 of file RPageStorageFile.cxx.

◆ RPageSourceFile() [3/4]

ROOT::Experimental::Detail::RPageSourceFile::RPageSourceFile ( const RPageSourceFile )
delete

◆ RPageSourceFile() [4/4]

ROOT::Experimental::Detail::RPageSourceFile::RPageSourceFile ( RPageSourceFile &&  )
default

◆ ~RPageSourceFile()

ROOT::Experimental::Detail::RPageSourceFile::~RPageSourceFile ( )
virtualdefault

Member Function Documentation

◆ AttachImpl()

ROOT::Experimental::RNTupleDescriptor ROOT::Experimental::Detail::RPageSourceFile::AttachImpl ( )
finalprotectedvirtual

Implements ROOT::Experimental::Detail::RPageSource.

Definition at line 237 of file RPageStorageFile.cxx.

◆ Clone()

std::unique_ptr< ROOT::Experimental::Detail::RPageSource > ROOT::Experimental::Detail::RPageSourceFile::Clone ( ) const
finalvirtual

The cloned page source creates a new raw file and reader and opens its own file descriptor to the data.

The meta-data (header and footer) is reread and parsed by the clone.

Implements ROOT::Experimental::Detail::RPageSource.

Definition at line 367 of file RPageStorageFile.cxx.

◆ LoadClusters()

std::vector< std::unique_ptr< ROOT::Experimental::Detail::RCluster > > ROOT::Experimental::Detail::RPageSourceFile::LoadClusters ( std::span< RCluster::RKey clusterKeys)
finalvirtual

Populates all the pages of the given cluster ids and columns; it is possible that some columns do not contain any pages.

The page source may load more columns than the minimal necessary set from columns. To indicate which columns have been loaded, LoadClusters() must mark them with SetColumnAvailable(). That includes the ones from the columns that don't have pages; otherwise subsequent requests for the cluster would assume an incomplete cluster and trigger loading again. LoadClusters() is typically called from the I/O thread of a cluster pool, i.e. the method runs concurrently to other methods of the page source.

Implements ROOT::Experimental::Detail::RPageSource.

Definition at line 491 of file RPageStorageFile.cxx.

◆ LoadSealedPage()

void ROOT::Experimental::Detail::RPageSourceFile::LoadSealedPage ( DescriptorId_t  columnId,
const RClusterIndex clusterIndex,
RSealedPage sealedPage 
)
finalvirtual

Read the packed and compressed bytes of a page into the memory buffer provided by selaedPage.

The sealed page can be used subsequently in a call to RPageSink::CommitSealedPage. The fSize and fNElements member of the sealedPage parameters are always set. If sealedPage.fBuffer is nullptr, no data will be copied but the returned size information can be used by the caller to allocate a large enough buffer and call LoadSealedPage again.

Implements ROOT::Experimental::Detail::RPageSource.

Definition at line 258 of file RPageStorageFile.cxx.

◆ operator=() [1/2]

RPageSourceFile & ROOT::Experimental::Detail::RPageSourceFile::operator= ( const RPageSourceFile )
delete

◆ operator=() [2/2]

RPageSourceFile & ROOT::Experimental::Detail::RPageSourceFile::operator= ( RPageSourceFile &&  )
default

◆ PopulatePage() [1/2]

ROOT::Experimental::Detail::RPage ROOT::Experimental::Detail::RPageSourceFile::PopulatePage ( ColumnHandle_t  columnHandle,
const RClusterIndex clusterIndex 
)
finalvirtual

Another version of PopulatePage that allows to specify cluster-relative indexes.

Implements ROOT::Experimental::Detail::RPageSource.

Definition at line 347 of file RPageStorageFile.cxx.

◆ PopulatePage() [2/2]

ROOT::Experimental::Detail::RPage ROOT::Experimental::Detail::RPageSourceFile::PopulatePage ( ColumnHandle_t  columnHandle,
NTupleSize_t  globalIndex 
)
finalvirtual

Allocates and fills a page that contains the index-th element.

Implements ROOT::Experimental::Detail::RPageSource.

Definition at line 330 of file RPageStorageFile.cxx.

◆ PopulatePageFromCluster()

ROOT::Experimental::Detail::RPage ROOT::Experimental::Detail::RPageSourceFile::PopulatePageFromCluster ( ColumnHandle_t  columnHandle,
const RClusterDescriptor clusterDescriptor,
ClusterSize_t::ValueType  idxInCluster 
)
private

Definition at line 273 of file RPageStorageFile.cxx.

◆ PrepareSingleCluster()

std::unique_ptr< ROOT::Experimental::Detail::RCluster > ROOT::Experimental::Detail::RPageSourceFile::PrepareSingleCluster ( const RCluster::RKey clusterKey,
std::vector< ROOT::Internal::RRawFile::RIOVec > &  readRequests 
)
private

Helper function for LoadClusters: it prepares the memory buffer (page map) and the read requests for a given cluster and columns.

The reead requests are appended to the provided vector. This way, requests can be collected for multiple clusters before sending them to RRawFile::ReadV().

Definition at line 376 of file RPageStorageFile.cxx.

◆ ReleasePage()

void ROOT::Experimental::Detail::RPageSourceFile::ReleasePage ( RPage page)
finalvirtual

Every page store needs to be able to free pages it handed out.

But Sinks and sources have different means of allocating pages.

Implements ROOT::Experimental::Detail::RPageStorage.

Definition at line 362 of file RPageStorageFile.cxx.

◆ UnzipClusterImpl()

void ROOT::Experimental::Detail::RPageSourceFile::UnzipClusterImpl ( RCluster cluster)
finalprotectedvirtual

Reimplemented from ROOT::Experimental::Detail::RPageSource.

Definition at line 514 of file RPageStorageFile.cxx.

Member Data Documentation

◆ fClusterPool

std::unique_ptr<RClusterPool> ROOT::Experimental::Detail::RPageSourceFile::fClusterPool
private

The cluster pool asynchronously preloads the next few clusters.

Definition at line 134 of file RPageStorageFile.hxx.

◆ fCurrentCluster

RCluster* ROOT::Experimental::Detail::RPageSourceFile::fCurrentCluster = nullptr
private

The last cluster from which a page got populated. Points into fClusterPool->fPool.

Definition at line 128 of file RPageStorageFile.hxx.

◆ fFile

std::unique_ptr<ROOT::Internal::RRawFile> ROOT::Experimental::Detail::RPageSourceFile::fFile
private

An RRawFile is used to request the necessary byte ranges from a local or a remote file.

Definition at line 130 of file RPageStorageFile.hxx.

◆ fPageAllocator

std::unique_ptr<RPageAllocatorFile> ROOT::Experimental::Detail::RPageSourceFile::fPageAllocator
private

Populated pages might be shared; there memory buffer is managed by the RPageAllocatorFile.

Definition at line 124 of file RPageStorageFile.hxx.

◆ fPagePool

std::shared_ptr<RPagePool> ROOT::Experimental::Detail::RPageSourceFile::fPagePool
private

The page pool might, at some point, be used by multiple page sources.

Definition at line 126 of file RPageStorageFile.hxx.

◆ fReader

Internal::RMiniFileReader ROOT::Experimental::Detail::RPageSourceFile::fReader
private

Takes the fFile to read ntuple blobs from it.

Definition at line 132 of file RPageStorageFile.hxx.

◆ kMaxPageSize

constexpr std::size_t ROOT::Experimental::Detail::RPageSourceFile::kMaxPageSize = 1024 * 1024
staticconstexpr

Cannot process pages larger than 1MB.

Definition at line 120 of file RPageStorageFile.hxx.


The documentation for this class was generated from the following files: