Common user-tunable settings for storing RNTuples.

All page sink classes need to support the common options.

Option name	Type	Default	Description
`Compression`	`std::uint32_t`	RCompressionSetting::EDefaults::kUseGeneralPurpose	The compression settings for this RNTuple
`ApproxZippedClusterSize`	`std::size_t`	128 MiB	Approximation of the target compressed cluster size
`MaxUnzippedClusterSize`	`std::size_t`	1280 MiB	Memory limit for committing a cluster: with very high compression ratio, we need a limit on how large the I/O buffer can grow during writing.
`InitialUnzippedPageSize`	`std::size_t`	256	Initially, columns start with a page of this size. The default value is chosen to accomodate at least 32 elements of 64 bits, or 64 elements of 32 bits. If more elements are needed, pages are increased up until the byte limit given by the option `MaxUnzippedPageSize` or until the total page buffer limit is reached (as a sum of all page buffers). The total write buffer limit needs to be large enough to hold the initial pages of all columns.
`MaxUnzippedPageSize`	`std::size_t`	1 MiB	Pages can grow only to the given limit in bytes.
`PageBufferBudget`	`std::size_t`	0 / auto	The maximum size that the sum of all page buffers used for writing into a persistent sink are allowed to use. If set to zero, RNTuple will auto-adjust the budget based on the value of `ApproxZippedClusterSize`. If set manually, the size needs to be large enough to hold all initial page buffers. The total amount of memory for writing is larger, e.g. for the additional compressed buffers etc. Use RNTupleModel::EstimateWriteMemoryUsage() for the total estimated memory use for writing. The default values are tuned for a total write memory of around 400 MiB per fill context.
`UseBufferedWrite`	`bool`	`true`	Whether to use buffered writing (with RPageSinkBuf). This buffers compressed pages in memory, reorders them to keep pages of the same column adjacent, and coalesces the writes when committing a cluster.
`UseDirectIO`	`bool`	`false`	Whether to use Direct I/O for writing. Note that this introduces alignment requirements that may very between filesystems and platforms.
`WriteBufferSize`	`std::size_t`	4 MiB	Buffer size to use for writing to files, must be a multiple of 4096 bytes. Testing suggests that 4MiB gives best performance (with Direct I/O) at a reasonable memory consumption.
`UseImplicitMT`	EImplicitMT	EImplicitMT::kDefault	Whether to use implicit multi-threading to compress pages. Only has an effect if buffered writing is turned on. The meaning of EImplicitMT::kDefault depends on the used writer: For the (sequential) RNTupleWriter, it translates to EImplicitMT::kOn and the user has to manually disable the use of implicit multi-threading if it is not wanted. For the RNTupleParalellWriter, the implementation defaults to EImplicitMT::kOff in order to avoid interference with explicit parallelism that might create one RNTupleFillContext per thread. If implicit multi-threading is wanted on top of this, the user has to explicitly request EImplicitMT::kOn.
`EnablePageChecksums`	`bool`	`true`	If set, checksums will be calculated and written for every page. If turned off, will also turn off `EnableSamePageMerging`.
`EnableSamePageMerging`	`bool`	`true`	If set, identical pages are deduplicated and aliased on disk. Requires `EnablePageChecksums` and will throw if previously disabled.

Definition at line 182 of file RNTupleWriteOptions.hxx.

Public Types
enum class	EImplicitMT { kOff , kOn , kDefault }

Public Member Functions
virtual	~RNTupleWriteOptions ()=default

virtual std::unique_ptr< RNTupleWriteOptions >	Clone () const

std::size_t	GetApproxZippedClusterSize () const

std::uint32_t	GetCompression () const

bool	GetEnablePageChecksums () const

bool	GetEnableSamePageMerging () const

std::size_t	GetInitialUnzippedPageSize () const

std::uint64_t	GetMaxKeySize () const

std::size_t	GetMaxUnzippedClusterSize () const

std::size_t	GetMaxUnzippedPageSize () const

std::size_t	GetPageBufferBudget () const

bool	GetUseBufferedWrite () const

bool	GetUseDirectIO () const

EImplicitMT	GetUseImplicitMT () const

std::size_t	GetWriteBufferSize () const

void	SetApproxZippedClusterSize (std::size_t val)

void	SetCompression (RCompressionSetting::EAlgorithm::EValues algorithm, int compressionLevel)

void	SetCompression (std::uint32_t val)

void	SetEnablePageChecksums (bool val)
	Note that turning off page checksums will also turn off the same page merging optimization (see tuning.md)

void	SetEnableSamePageMerging (bool val)

void	SetInitialUnzippedPageSize (std::size_t val)

void	SetMaxUnzippedClusterSize (std::size_t val)

void	SetMaxUnzippedPageSize (std::size_t val)

void	SetPageBufferBudget (std::size_t val)

void	SetUseBufferedWrite (bool val)

void	SetUseDirectIO (bool val)

void	SetUseImplicitMT (EImplicitMT val)

void	SetWriteBufferSize (std::size_t val)

Static Public Attributes
static constexpr std::uint64_t	kDefaultMaxKeySize = 0x4000'0000

Protected Attributes
std::size_t	fApproxZippedClusterSize = 128 * 1024 * 1024

std::uint32_t	fCompression {RCompressionSetting::EDefaults::kUseGeneralPurpose}

bool	fEnablePageChecksums = true

bool	fEnableSamePageMerging = true

std::size_t	fInitialUnzippedPageSize = 256

std::uint64_t	fMaxKeySize = kDefaultMaxKeySize
	Specifies the max size of a payload storeable into a single TKey.

std::size_t	fMaxUnzippedClusterSize = 10 * fApproxZippedClusterSize

std::size_t	fMaxUnzippedPageSize = 1024 * 1024

std::size_t	fPageBufferBudget = 0

bool	fUseBufferedWrite = true

bool	fUseDirectIO = false

EImplicitMT	fUseImplicitMT = EImplicitMT::kDefault

std::size_t	fWriteBufferSize = 4 * 1024 * 1024

#include <ROOT/RNTupleWriteOptions.hxx>

Inheritance diagram for ROOT::RNTupleWriteOptions:

[legend]

Member Enumeration Documentation

◆ EImplicitMT

enum class ROOT::RNTupleWriteOptions::EImplicitMT

strong

Enumerator
kOff
kOn
kDefault

Definition at line 184 of file RNTupleWriteOptions.hxx.

Constructor & Destructor Documentation

◆ ~RNTupleWriteOptions()

virtual ROOT::RNTupleWriteOptions::~RNTupleWriteOptions ( )

virtualdefault

Member Function Documentation

◆ Clone()

std::unique_ptr< ROOT::RNTupleWriteOptions > ROOT::RNTupleWriteOptions::Clone ( ) const

virtual

Reimplemented in ROOT::Experimental::RNTupleWriteOptionsDaos.

Definition at line 48 of file RNTupleWriteOptions.cxx.

◆ GetApproxZippedClusterSize()

std::size_t ROOT::RNTupleWriteOptions::GetApproxZippedClusterSize ( ) const

inline

Definition at line 225 of file RNTupleWriteOptions.hxx.

◆ GetCompression()

std::uint32_t ROOT::RNTupleWriteOptions::GetCompression ( ) const

inline

Definition at line 218 of file RNTupleWriteOptions.hxx.

◆ GetEnablePageChecksums()

bool ROOT::RNTupleWriteOptions::GetEnablePageChecksums ( ) const

inline

Definition at line 252 of file RNTupleWriteOptions.hxx.

◆ GetEnableSamePageMerging()

bool ROOT::RNTupleWriteOptions::GetEnableSamePageMerging ( ) const

inline

Definition at line 262 of file RNTupleWriteOptions.hxx.

◆ GetInitialUnzippedPageSize()

std::size_t ROOT::RNTupleWriteOptions::GetInitialUnzippedPageSize ( ) const

inline

Definition at line 231 of file RNTupleWriteOptions.hxx.

◆ GetMaxKeySize()

std::uint64_t ROOT::RNTupleWriteOptions::GetMaxKeySize ( ) const

inline

Definition at line 265 of file RNTupleWriteOptions.hxx.

◆ GetMaxUnzippedClusterSize()

std::size_t ROOT::RNTupleWriteOptions::GetMaxUnzippedClusterSize ( ) const

inline

Definition at line 228 of file RNTupleWriteOptions.hxx.

◆ GetMaxUnzippedPageSize()

std::size_t ROOT::RNTupleWriteOptions::GetMaxUnzippedPageSize ( ) const

inline

Definition at line 234 of file RNTupleWriteOptions.hxx.

◆ GetPageBufferBudget()

std::size_t ROOT::RNTupleWriteOptions::GetPageBufferBudget ( ) const

Definition at line 85 of file RNTupleWriteOptions.cxx.

◆ GetUseBufferedWrite()

bool ROOT::RNTupleWriteOptions::GetUseBufferedWrite ( ) const

inline

Definition at line 240 of file RNTupleWriteOptions.hxx.

◆ GetUseDirectIO()

bool ROOT::RNTupleWriteOptions::GetUseDirectIO ( ) const

inline

Definition at line 243 of file RNTupleWriteOptions.hxx.

◆ GetUseImplicitMT()

EImplicitMT ROOT::RNTupleWriteOptions::GetUseImplicitMT ( ) const

inline

Definition at line 249 of file RNTupleWriteOptions.hxx.

◆ GetWriteBufferSize()

std::size_t ROOT::RNTupleWriteOptions::GetWriteBufferSize ( ) const

inline

Definition at line 246 of file RNTupleWriteOptions.hxx.

◆ SetApproxZippedClusterSize()

void ROOT::RNTupleWriteOptions::SetApproxZippedClusterSize ( std::size_t val )

Definition at line 53 of file RNTupleWriteOptions.cxx.

◆ SetCompression() [1/2]

void ROOT::RNTupleWriteOptions::SetCompression	(	RCompressionSetting::EAlgorithm::EValues	algorithm,
		int	compressionLevel )

inline

Definition at line 220 of file RNTupleWriteOptions.hxx.

◆ SetCompression() [2/2]

void ROOT::RNTupleWriteOptions::SetCompression ( std::uint32_t val )

inline

Definition at line 219 of file RNTupleWriteOptions.hxx.

◆ SetEnablePageChecksums()

void ROOT::RNTupleWriteOptions::SetEnablePageChecksums ( bool val )

inline

Note that turning off page checksums will also turn off the same page merging optimization (see tuning.md)

Definition at line 254 of file RNTupleWriteOptions.hxx.

◆ SetEnableSamePageMerging()

void ROOT::RNTupleWriteOptions::SetEnableSamePageMerging ( bool val )

Definition at line 77 of file RNTupleWriteOptions.cxx.

◆ SetInitialUnzippedPageSize()

void ROOT::RNTupleWriteOptions::SetInitialUnzippedPageSize ( std::size_t val )

Definition at line 65 of file RNTupleWriteOptions.cxx.

◆ SetMaxUnzippedClusterSize()

void ROOT::RNTupleWriteOptions::SetMaxUnzippedClusterSize ( std::size_t val )

Definition at line 59 of file RNTupleWriteOptions.cxx.

◆ SetMaxUnzippedPageSize()

void ROOT::RNTupleWriteOptions::SetMaxUnzippedPageSize ( std::size_t val )

Definition at line 71 of file RNTupleWriteOptions.cxx.

◆ SetPageBufferBudget()

void ROOT::RNTupleWriteOptions::SetPageBufferBudget ( std::size_t val )

inline

Definition at line 238 of file RNTupleWriteOptions.hxx.

◆ SetUseBufferedWrite()

void ROOT::RNTupleWriteOptions::SetUseBufferedWrite ( bool val )

inline

Definition at line 241 of file RNTupleWriteOptions.hxx.

◆ SetUseDirectIO()

void ROOT::RNTupleWriteOptions::SetUseDirectIO ( bool val )

inline

Definition at line 244 of file RNTupleWriteOptions.hxx.

◆ SetUseImplicitMT()

void ROOT::RNTupleWriteOptions::SetUseImplicitMT ( EImplicitMT val )

inline

Definition at line 250 of file RNTupleWriteOptions.hxx.

◆ SetWriteBufferSize()

void ROOT::RNTupleWriteOptions::SetWriteBufferSize ( std::size_t val )

inline

Definition at line 247 of file RNTupleWriteOptions.hxx.

Member Data Documentation

◆ fApproxZippedClusterSize

std::size_t ROOT::RNTupleWriteOptions::fApproxZippedClusterSize = 128 * 1024 * 1024

protected

Definition at line 198 of file RNTupleWriteOptions.hxx.

◆ fCompression

std::uint32_t ROOT::RNTupleWriteOptions::fCompression {RCompressionSetting::EDefaults::kUseGeneralPurpose}

protected

Definition at line 197 of file RNTupleWriteOptions.hxx.

◆ fEnablePageChecksums

bool ROOT::RNTupleWriteOptions::fEnablePageChecksums = true

protected

Definition at line 207 of file RNTupleWriteOptions.hxx.

◆ fEnableSamePageMerging

bool ROOT::RNTupleWriteOptions::fEnableSamePageMerging = true

protected

Definition at line 208 of file RNTupleWriteOptions.hxx.

◆ fInitialUnzippedPageSize

std::size_t ROOT::RNTupleWriteOptions::fInitialUnzippedPageSize = 256

protected

Definition at line 200 of file RNTupleWriteOptions.hxx.

◆ fMaxKeySize

std::uint64_t ROOT::RNTupleWriteOptions::fMaxKeySize = kDefaultMaxKeySize

protected

Specifies the max size of a payload storeable into a single TKey.

When writing an RNTuple to a ROOT file, any payload whose size exceeds this will be split into multiple keys.

Definition at line 211 of file RNTupleWriteOptions.hxx.

◆ fMaxUnzippedClusterSize

std::size_t ROOT::RNTupleWriteOptions::fMaxUnzippedClusterSize = 10 * fApproxZippedClusterSize

protected

Definition at line 199 of file RNTupleWriteOptions.hxx.

◆ fMaxUnzippedPageSize

std::size_t ROOT::RNTupleWriteOptions::fMaxUnzippedPageSize = 1024 * 1024

protected

Definition at line 201 of file RNTupleWriteOptions.hxx.

◆ fPageBufferBudget

std::size_t ROOT::RNTupleWriteOptions::fPageBufferBudget = 0

protected

Definition at line 202 of file RNTupleWriteOptions.hxx.

◆ fUseBufferedWrite

bool ROOT::RNTupleWriteOptions::fUseBufferedWrite = true

protected

Definition at line 203 of file RNTupleWriteOptions.hxx.

◆ fUseDirectIO

bool ROOT::RNTupleWriteOptions::fUseDirectIO = false

protected

Definition at line 204 of file RNTupleWriteOptions.hxx.

◆ fUseImplicitMT

EImplicitMT ROOT::RNTupleWriteOptions::fUseImplicitMT = EImplicitMT::kDefault

protected

Definition at line 206 of file RNTupleWriteOptions.hxx.

◆ fWriteBufferSize

std::size_t ROOT::RNTupleWriteOptions::fWriteBufferSize = 4 * 1024 * 1024

protected

Definition at line 205 of file RNTupleWriteOptions.hxx.

◆ kDefaultMaxKeySize

constexpr std::uint64_t ROOT::RNTupleWriteOptions::kDefaultMaxKeySize = 0x4000'0000

staticconstexpr

Definition at line 191 of file RNTupleWriteOptions.hxx.

Libraries for ROOT::RNTupleWriteOptions:

[legend]

The documentation for this class was generated from the following files:

tree/ntuple/inc/ROOT/RNTupleWriteOptions.hxx
tree/ntuple/src/RNTupleWriteOptions.cxx

Public Types

Public Member Functions

Static Public Attributes

Protected Attributes

Member Enumeration Documentation

◆ EImplicitMT

Constructor & Destructor Documentation

◆ ~RNTupleWriteOptions()

Member Function Documentation

◆ Clone()

◆ GetApproxZippedClusterSize()

◆ GetCompression()

◆ GetEnablePageChecksums()

◆ GetEnableSamePageMerging()

◆ GetInitialUnzippedPageSize()

◆ GetMaxKeySize()

◆ GetMaxUnzippedClusterSize()

◆ GetMaxUnzippedPageSize()

◆ GetPageBufferBudget()

◆ GetUseBufferedWrite()

◆ GetUseDirectIO()

◆ GetUseImplicitMT()

◆ GetWriteBufferSize()

◆ SetApproxZippedClusterSize()

◆ SetCompression() [1/2]

◆ SetCompression() [2/2]

◆ SetEnablePageChecksums()

◆ SetEnableSamePageMerging()

◆ SetInitialUnzippedPageSize()

◆ SetMaxUnzippedClusterSize()

◆ SetMaxUnzippedPageSize()

◆ SetPageBufferBudget()

◆ SetUseBufferedWrite()

◆ SetUseDirectIO()

◆ SetUseImplicitMT()

◆ SetWriteBufferSize()

Member Data Documentation

◆ fApproxZippedClusterSize

◆ fCompression

◆ fEnablePageChecksums

◆ fEnableSamePageMerging

◆ fInitialUnzippedPageSize

◆ fMaxKeySize

◆ fMaxUnzippedClusterSize

◆ fMaxUnzippedPageSize

◆ fPageBufferBudget

◆ fUseBufferedWrite

◆ fUseDirectIO

◆ fUseImplicitMT

◆ fWriteBufferSize

◆ kDefaultMaxKeySize