ROOT   6.14/05 Reference Guide
TTreeCache Class Reference

A specialized TFileCacheRead object for a TTree.

This class acts as a file cache, registering automatically the baskets from the branches being processed (TTree::Draw or TTree::Process and TSelectors) when in the learning phase. The learning phase is by default 100 entries. It can be changed via TTreeCache::SetLearnEntries.

This cache speeds-up considerably the performance, in particular when the Tree is accessed remotely via a high latency network.

The default cache size (10 Mbytes) may be changed via the function TTree::SetCacheSize

Only the baskets for the requested entry range are put in the cache

For each Tree being processed a TTreeCache object is created. This object is automatically deleted when the Tree is deleted or when the file is deleted.

• Special case of a TChain Once the training is done on the first Tree, the list of branches in the cache is kept for the following files.
• Special case of a TEventlist if the Tree or TChain has a TEventlist, only the buffers referenced by the list are put in the cache.

The learning period is started or restarted when:

• TTree automatically creates a cache. This feature can be controlled with an env. variable or the TTreeCache.Size option.
• TTree::SetCacheSize is called with a non-zero size and a cache did not previously exist
• TTreeCache::StartLearningPhase is called.
• TTreeCache::SetEntryRange is called and the learning is not yet finished and has not been set to manual and the new minimun entry is different.

The learning period is stopped (and prefetching is actually started) when:

• TTreeCache::StopLearningPhase is called.
• An entry outside the 'learning' range is requested The 'learning range is from fEntryMin (default to 0) to fEntryMin + fgLearnEntries (default to 100).
• A 'cached' TChain switches over to a new file.

Further, the TreeCache can optimize its behavior on a cache miss. When miss optimization is enabled, it will track all branches utilized after the learning phase (those that cause a cache miss). When one cache miss occurs, then all the utilized branches will be prefetched for that event. This optimization utilizes the observation that infrequently accessed branches are often accessed together. For example, this will greatly speed up an analysis where the results of a trigger are read out for every branch, but the majority of event collections are read only when the trigger results pass a set of filters. NOTE - when this mode is enabled, the memory dedicated to the cache will up to double in the case of cache miss. Additionally, on the first miss of an event, we must iterate through all the "active branches" for the miss cache and find the correct basket. This can be potentially a CPU-expensive operation compared to, e.g., the latency of a SSD. This is why the miss cache is currently disabled by default.

WHY DO WE NEED the TreeCache when doing data analysis?

When writing a TTree, the branch buffers are kept in memory. A typical branch buffersize (before compression) is typically 32 KBytes. After compression, the zipped buffer may be just a few Kbytes. The branch buffers cannot be much larger in case of Trees with several hundred or thousand branches.

When writing, this does not generate a performance problem because branch buffers are always written sequentially and the OS is in general clever enough to flush the data to the output file when a few MBytes of data have to be written. When reading at the contrary, one may hit a performance problem when reading across a network (LAN or WAN) and the network latency is high. For example in a WAN with 10ms latency, reading 1000 buffers of 10 KBytes each with no cache will imply 10s penalty where a local read of the 10 MBytes would take about 1 second.

The TreeCache will try to prefetch all the buffers for the selected branches such that instead of transferring 1000 buffers of 10 Kbytes, it will be able to transfer one single large buffer of 10 Mbytes in one single transaction. Not only the TreeCache minimizes the number of transfers, but in addition it can sort the blocks to be read in increasing order such that the file is read sequentially.

Systems like xrootd, dCache or httpd take advantage of the TreeCache in reading ahead as much data as they can and return to the application the maximum data specified in the cache and have the next chunk of data ready when the next request comes.

HOW TO USE the TreeCache

A few use cases are discussed below. A cache may be created with automatic sizing when a TTree is used:

Caches are created and automatically sized for TTrees when TTreeCache.Size or the environment variable ROOT_TTREECACHE_SIZE is set to a sizing factor.

But there are many possible configurations where manual control may be wanted. In some applications you know a priori the list of branches to read. In other applications the analysis loop calls several layers of user functions where it is impossible to predict a priori which branches will be used. This is probably the most frequent case. In this case ROOT I/O will flag used branches automatically when a branch buffer is read during the learning phase. The TreeCache interface provides functions to instruct the cache about the used branches if they are known a priori. In the examples below, portions of analysis code are shown. The few statements involving the TreeCache are marked with //<<<

1. with TTree::Draw

the TreeCache is automatically used by TTree::Draw. The function knows which branches are used in the query and it puts automatically these branches in the cache. The entry range is also known automatically.

2. with TTree::Process and TSelectors

You must enable the cache and tell the system which branches to cache and also specify the entry range. It is important to specify the entry range in case you process only a subset of the events, otherwise you run the risk to store in the cache entries that you do not need.

#### example 2a

TTree *T = (TTree*)f->Get("mytree");
Long64_t nentries = T->GetEntries();
Int_t cachesize = 10000000; //10 MBytes
T->SetCacheSize(cachesize); //<<<
T->Process('myselector.C+");
//in the TSelector::Process function we read all branches
T->GetEntry(i);
... here you process your entry

example 2b

in the Process function we read a subset of the branches. Only the branches used in the first entry will be put in the cache

TTree *T = (TTree*)f->Get("mytree");
//we want to process only the 200 first entries
Long64_t nentries=200;
int efirst= 0;
int elast = efirst+nentries;
Int_t cachesize = 10000000; //10 MBytes
TTreeCache::SetLearnEntries(1); //<<< we can take the decision after 1 entry
T->SetCacheSize(cachesize); //<<<
T->SetCacheEntryRange(efirst,elast); //<<<
T->Process('myselector.C+","",nentries,efirst);
// in the TSelector::Process we read only 2 branches
TBranch *b1 = T->GetBranch("branch1");
b1->GetEntry(i);
if (somecondition) return;
TBranch *b2 = T->GetBranch("branch2");
b2->GetEntry(i);
... here you process your entry

3. with your own event loop

example 3a

in your analysis loop, you always use 2 branches. You want to prefetch the branch buffers for these 2 branches only.

TTree *T = (TTree*)f->Get("mytree");
TBranch *b1 = T->GetBranch("branch1");
TBranch *b2 = T->GetBranch("branch2");
Long64_t nentries = T->GetEntries();
Int_t cachesize = 10000000; //10 MBytes
T->SetCacheSize(cachesize); //<<<
for (Long64_t i=0;i<nentries;i++) {
T->LoadTree(i); //<<< important call when calling TBranch::GetEntry after
b1->GetEntry(i);
if (some condition not met) continue;
b2->GetEntry(i);
if (some condition not met) continue;
//here we read the full event only in some rare cases.
//there is no point in caching the other branches as it might be
//more economical to read only the branch buffers really used.
T->GetEntry(i);
.. process the rare but interesting cases.
... here you process your entry
}

example 3b

in your analysis loop, you always use 2 branches in the main loop. you also call some analysis functions where a few more branches will be read. but you do not know a priori which ones. There is no point in prefetching branches that will be used very rarely.

TTree *T = (TTree*)f->Get("mytree");
Long64_t nentries = T->GetEntries();
Int_t cachesize = 10000000; //10 MBytes
T->SetCacheSize(cachesize); //<<<
T->SetCacheLearnEntries(5); //<<< we can take the decision after 5 entries
TBranch *b1 = T->GetBranch("branch1");
TBranch *b2 = T->GetBranch("branch2");
for (Long64_t i=0;i<nentries;i++) {
b1->GetEntry(i);
if (some condition not met) continue;
b2->GetEntry(i);
//at this point we may call a user function where a few more branches
//will be read conditionally. These branches will be put in the cache
//if they have been used in the first 10 entries
if (some condition not met) continue;
//here we read the full event only in some rare cases.
//there is no point in caching the other branches as it might be
//more economical to read only the branch buffers really used.
T->GetEntry(i);
.. process the rare but interesting cases.
... here you process your entry
}

SPECIAL CASES WHERE TreeCache should not be activated

When reading only a small fraction of all entries such that not all branch buffers are read, it might be faster to run without a cache.

HOW TO VERIFY That the TreeCache has been used and check its performance

Once your analysis loop has terminated, you can access/print the number of effective system reads for a given file with a code like (where TFile* f is a pointer to your file)

Definition at line 35 of file TTreeCache.h.

Classes

struct  IOPos

struct  MissCache

Public Types

enum  EPrefillType { kNoPrefill, kAllBranches }

Public Types inherited from TObject
enum  {
kIsOnHeap = 0x01000000, kNotDeleted = 0x02000000, kZombie = 0x04000000, kInconsistent = 0x08000000,
}

enum  { kSingleKey = BIT(0), kOverwrite = BIT(1), kWriteDelete = BIT(2) }

enum  EDeprecatedStatusBits { kObjInCanvas = BIT(3) }

enum  EStatusBits {
kCanDelete = BIT(0), kMustCleanup = BIT(3), kIsReferenced = BIT(4), kHasUUID = BIT(5),
kCannotPick = BIT(6), kNoContextMenu = BIT(8), kInvalidObject = BIT(13)
}

Public Member Functions

TTreeCache ()
Default Constructor. More...

TTreeCache (TTree *tree, Int_t buffersize=0)
Constructor. More...

virtual ~TTreeCache ()
Destructor. (in general called by the TFile destructor) More...

virtual Int_t AddBranch (TBranch *b, Bool_t subgbranches=kFALSE)
Add a branch to the list of branches to be stored in the cache this function is called by TBranch::GetBasket Returns: More...

virtual Int_t AddBranch (const char *branch, Bool_t subbranches=kFALSE)
Add a branch to the list of branches to be stored in the cache this is to be used by user (thats why we pass the name of the branch). More...

virtual void Disable ()

virtual Int_t DropBranch (TBranch *b, Bool_t subbranches=kFALSE)
Remove a branch to the list of branches to be stored in the cache this function is called by TBranch::GetBasket. More...

virtual Int_t DropBranch (const char *branch, Bool_t subbranches=kFALSE)
Remove a branch to the list of branches to be stored in the cache this is to be used by user (thats why we pass the name of the branch). More...

virtual void Enable ()

virtual Bool_t FillBuffer ()
Fill the cache buffer with the branches in the cache. More...

const TObjArrayGetCachedBranches () const

EPrefillType GetConfiguredPrefillType () const
Return the desired prefill type from the environment or resource variable. More...

Double_t GetEfficiency () const
Give the total efficiency of the primary cache... More...

Double_t GetEfficiencyRel () const
This will indicate a sort of relative efficiency... More...

virtual Int_t GetEntryMax () const

virtual Int_t GetEntryMin () const

virtual EPrefillType GetLearnPrefill () const

Double_t GetMissEfficiency () const
The total efficiency of the 'miss cache' - defined as the ratio of blocks found in the cache versus the number of blocks prefetched. More...

Double_t GetMissEfficiencyRel () const
Relative efficiency of the 'miss cache' - ratio of the reads found in cache to the number of reads so far. More...

Bool_t GetOptimizeMisses () const

TTreeGetTree () const

Bool_t IsAutoCreated () const

virtual Bool_t IsEnabled () const

virtual Bool_t IsLearning () const

virtual void LearnPrefill ()
Perform an initial prefetch, attempting to read as much of the learning phase baskets for all branches at once. More...

virtual void Print (Option_t *option="") const
Print cache statistics. More...

virtual Int_t ReadBuffer (char *buf, Long64_t pos, Int_t len)
Read buffer at position pos if the request is in the list of prefetched blocks read from fBuffer. More...

virtual Int_t ReadBufferNormal (char *buf, Long64_t pos, Int_t len)

virtual Int_t ReadBufferPrefetch (char *buf, Long64_t pos, Int_t len)
Used to read a chunk from a block previously fetched. More...

virtual void ResetCache ()
This will simply clear the cache. More...

void ResetMissCache ()
Reset all the miss cache training. More...

void SetAutoCreated (Bool_t val)

virtual Int_t SetBufferSize (Int_t buffersize)
Change the underlying buffer size of the cache. More...

virtual void SetEntryRange (Long64_t emin, Long64_t emax)
Set the minimum and maximum entry number to be processed this information helps to optimize the number of baskets to read when prefetching the branch buffers. More...

virtual void SetFile (TFile *file, TFile::ECacheAction action=TFile::kDisconnect)
Overload to make sure that the object specific. More...

virtual void SetLearnPrefill (EPrefillType type=kNoPrefill)
Set whether the learning period is started with a prefilling of the cache and which type of prefilling is used. More...

void SetOptimizeMisses (Bool_t opt)
Start of methods for the miss cache. More...

void StartLearningPhase ()
The name should be enough to explain the method. More...

virtual void StopLearningPhase ()
This is the counterpart of StartLearningPhase() and can be used to stop the learning phase. More...

virtual void UpdateBranches (TTree *tree)
Update pointer to current Tree and recompute pointers to the branches in the cache. More...

Public Member Functions inherited from TFileCacheRead
Default Constructor. More...

TFileCacheRead (TFile *file, Int_t buffersize, TObject *tree=0)
Creates a TFileCacheRead data structure. More...

Destructor. More...

virtual void Close (Option_t *option="")
Close out any threads or asynchronous fetches used by the underlying implementation. More...

virtual Int_t GetBufferSize () const

TFileGetFile () const

Int_t GetNseek () const

Int_t GetNtot () const

Long64_t GetPrefetchedBlocks () const

virtual TFilePrefetchGetPrefetchObj ()

virtual Int_t GetUnzipBuffer (char **, Long64_t, Int_t, Bool_t *)

virtual Bool_t IsEnablePrefetching () const

virtual void Prefetch (Long64_t pos, Int_t len)
Add block of length len at position pos in the list of blocks to be prefetched. More...

virtual Int_t ReadBufferExt (char *buf, Long64_t pos, Int_t len, Int_t &loc)

virtual Int_t ReadBufferExtNormal (char *buf, Long64_t pos, Int_t len, Int_t &loc)

virtual Int_t ReadBufferExtPrefetch (char *buf, Long64_t pos, Int_t len, Int_t &loc)
prefetch the first block More...

virtual void SecondPrefetch (Long64_t, Int_t)

virtual void SecondSort ()
Sort buffers to be prefetched in increasing order of positions. More...

virtual void SetEnablePrefetching (Bool_t setPrefetching=kFALSE)
Set the prefetching mode of this file. More...

virtual void SetSkipZip (Bool_t=kTRUE)

virtual void Sort ()
Sort buffers to be prefetched in increasing order of positions. More...

virtual void WaitFinishPrefetch ()

Public Member Functions inherited from TObject
TObject ()
TObject constructor. More...

TObject (const TObject &object)
TObject copy ctor. More...

virtual ~TObject ()
TObject destructor. More...

void AbstractMethod (const char *method) const
Use this method to implement an "abstract" method that you don't want to leave purely abstract. More...

Append graphics object to current pad. More...

virtual void Browse (TBrowser *b)
Browse object. May be overridden for another default action. More...

ULong_t CheckedHash ()
Checked and record whether for this class has a consistent Hash/RecursiveRemove setup (*) and then return the regular Hash value for this object. More...

virtual const char * ClassName () const
Returns name of class to which the object belongs. More...

virtual void Clear (Option_t *="")

virtual TObjectClone (const char *newname="") const
Make a clone of an object using the Streamer facility. More...

virtual Int_t Compare (const TObject *obj) const
Compare abstract method. More...

virtual void Copy (TObject &object) const
Copy this to obj. More...

virtual void Delete (Option_t *option="")
Delete this object. More...

virtual Int_t DistancetoPrimitive (Int_t px, Int_t py)
Computes distance from point (px,py) to the object. More...

virtual void Draw (Option_t *option="")
Default Draw method for all objects. More...

virtual void DrawClass () const
Draw class inheritance tree of the class to which this object belongs. More...

virtual TObjectDrawClone (Option_t *option="") const
Draw a clone of this object in the current selected pad for instance with: gROOT->SetSelectedPad(gPad). More...

virtual void Dump () const
Dump contents of object on stdout. More...

virtual void Error (const char *method, const char *msgfmt,...) const
Issue error message. More...

virtual void Execute (const char *method, const char *params, Int_t *error=0)
Execute method on this object with the given parameter string, e.g. More...

virtual void Execute (TMethod *method, TObjArray *params, Int_t *error=0)
Execute method on this object with parameters stored in the TObjArray. More...

virtual void ExecuteEvent (Int_t event, Int_t px, Int_t py)
Execute action corresponding to an event at (px,py). More...

virtual void Fatal (const char *method, const char *msgfmt,...) const
Issue fatal error message. More...

virtual TObjectFindObject (const char *name) const
Must be redefined in derived classes. More...

virtual TObjectFindObject (const TObject *obj) const
Must be redefined in derived classes. More...

virtual Option_tGetDrawOption () const
Get option used by the graphics system to draw this object. More...

virtual const char * GetIconName () const
Returns mime type name of object. More...

virtual const char * GetName () const
Returns name of object. More...

virtual char * GetObjectInfo (Int_t px, Int_t py) const
Returns string containing info about the object at position (px,py). More...

virtual Option_tGetOption () const

virtual const char * GetTitle () const
Returns title of object. More...

virtual UInt_t GetUniqueID () const
Return the unique object id. More...

virtual Bool_t HandleTimer (TTimer *timer)
Execute action in response of a timer timing out. More...

virtual ULong_t Hash () const
Return hash value for this object. More...

Bool_t HasInconsistentHash () const
Return true is the type of this object is known to have an inconsistent setup for Hash and RecursiveRemove (i.e. More...

virtual void Info (const char *method, const char *msgfmt,...) const
Issue info message. More...

virtual Bool_t InheritsFrom (const char *classname) const
Returns kTRUE if object inherits from class "classname". More...

virtual Bool_t InheritsFrom (const TClass *cl) const
Returns kTRUE if object inherits from TClass cl. More...

virtual void Inspect () const
Dump contents of this object in a graphics canvas. More...

void InvertBit (UInt_t f)

virtual Bool_t IsEqual (const TObject *obj) const
Default equal comparison (objects are equal if they have the same address in memory). More...

virtual Bool_t IsFolder () const
Returns kTRUE in case object contains browsable objects (like containers or lists of other objects). More...

R__ALWAYS_INLINE Bool_t IsOnHeap () const

virtual Bool_t IsSortable () const

R__ALWAYS_INLINE Bool_t IsZombie () const

virtual void ls (Option_t *option="") const
The ls function lists the contents of a class on stdout. More...

void MayNotUse (const char *method) const
Use this method to signal that a method (defined in a base class) may not be called in a derived class (in principle against good design since a child class should not provide less functionality than its parent, however, sometimes it is necessary). More...

virtual Bool_t Notify ()
This method must be overridden to handle object notification. More...

void Obsolete (const char *method, const char *asOfVers, const char *removedFromVers) const
Use this method to declare a method obsolete. More...

void operator delete (void *ptr)
Operator delete. More...

void operator delete[] (void *ptr)
Operator delete []. More...

voidoperator new (size_t sz)

voidoperator new (size_t sz, void *vp)

voidoperator new[] (size_t sz)

voidoperator new[] (size_t sz, void *vp)

TObjectoperator= (const TObject &rhs)
TObject assignment operator. More...

virtual void Paint (Option_t *option="")
This method must be overridden if a class wants to paint itself. More...

virtual void Pop ()
Pop on object drawn in a pad to the top of the display list. More...

virtual Int_t Read (const char *name)
Read contents of object with specified name from the current directory. More...

virtual void RecursiveRemove (TObject *obj)
Recursively remove this object from a list. More...

void ResetBit (UInt_t f)

virtual void SaveAs (const char *filename="", Option_t *option="") const
Save this object in the file specified by filename. More...

virtual void SavePrimitive (std::ostream &out, Option_t *option="")
Save a primitive as a C++ statement(s) on output stream "out". More...

void SetBit (UInt_t f, Bool_t set)
Set or unset the user status bits as specified in f. More...

void SetBit (UInt_t f)

virtual void SetDrawOption (Option_t *option="")
Set drawing option for object. More...

virtual void SetUniqueID (UInt_t uid)
Set the unique object id. More...

virtual void SysError (const char *method, const char *msgfmt,...) const
Issue system error message. More...

R__ALWAYS_INLINE Bool_t TestBit (UInt_t f) const

Int_t TestBits (UInt_t f) const

virtual void UseCurrentStyle ()
Set current style settings in this object This function is called when either TCanvas::UseCurrentStyle or TROOT::ForceStyle have been invoked. More...

virtual void Warning (const char *method, const char *msgfmt,...) const
Issue warning message. More...

virtual Int_t Write (const char *name=0, Int_t option=0, Int_t bufsize=0)
Write this object to the current directory. More...

virtual Int_t Write (const char *name=0, Int_t option=0, Int_t bufsize=0) const
Write this object to the current directory. More...

Static Public Member Functions

static Int_t GetLearnEntries ()
Static function returning the number of entries used to train the cache see SetLearnEntries. More...

static void SetLearnEntries (Int_t n=10)
Static function to set the number of entries to be used in learning mode The default value for n is 10. More...

Static Public Member Functions inherited from TObject
static Long_t GetDtorOnly ()
Return destructor only flag. More...

static Bool_t GetObjectStat ()
Get status of object stat flag. More...

static void SetDtorOnly (void *obj)
Set destructor only flag. More...

static void SetObjectStat (Bool_t stat)
Turn on/off tracking of objects in the TObjectTable. More...

Protected Attributes

Bool_t fAutoCreated {kFALSE}
! true if cache was automatically created More...

TObjArrayfBranches {nullptr}
! List of branches to be stored in the cache More...

TListfBrNames {nullptr}
! list of branch names in the cache More...

Long64_t fCurrentClusterStart {-1}
! Start of the cluster(s) where the current content was picked out More...

Bool_t fEnabled {kTRUE}
! cache enabled for cached reading More...

Long64_t fEntryCurrent {-1}
! current lowest entry number in the cache More...

Long64_t fEntryMax {1}
! last entry in the cache More...

Long64_t fEntryMin {0}
! first entry in the cache More...

Long64_t fEntryNext {-1}
! next entry number where cache must be filled More...

Int_t fFillTimes {0}
! how many times we can fill the current buffer More...

Bool_t fFirstBuffer {kTRUE}
! true if first buffer is used for prefetching More...

Long64_t fFirstEntry {-1}
! save the value of the first entry More...

Long64_t fFirstMiss {-1}
! set to the event # of the first miss. More...

Bool_t fFirstTime {kTRUE}
! save the fact that we processes the first entry More...

Bool_t fIsLearning {kTRUE}
! true if cache is in learning mode More...

Bool_t fIsManual {kFALSE}
! true if cache is StopLearningPhase was used More...

Long64_t fLastMiss {-1}
! set to the event # of the last miss. More...

std::unique_ptr< MissCachefMissCache
! Cache contents for misses More...

Int_t fNbranches {0}
! Number of branches in the cache More...

Long64_t fNextClusterStart {-1}
! End+1 of the cluster(s) where the current content was picked out More...

Number of blocks read, not found in the primary cache, and found in the secondary cache. More...

Number of blocks read into the secondary ("miss") cache. More...

Number of blocks read and found in the cache. More...

Number of blocks that were prefetched. More...

Bool_t fOneTime {kFALSE}
! used in the learning phase More...

Bool_t fOptimizeMisses {kFALSE}
! true if we should optimize cache misses. More...

EPrefillType fPrefillType
Whether a pre-filling is enabled (and if applicable which type) More...

! reading in reverse mode More...

TTreefTree {nullptr}
! pointer to the current Tree More...

Bool_t fBIsSorted

Bool_t fBIsTransferred

Int_tfBLen
[fBNb] More...

Int_t fBNb

Int_t fBNseek

Int_t fBNtot

Long64_tfBPos
[fBNb] More...

Long64_tfBSeek
[fBNseek] More...

Int_tfBSeekIndex
[fBNseek] More...

Int_tfBSeekLen
[fBNseek] More...

Int_tfBSeekPos
[fBNseek] More...

Int_t fBSeekSize

Long64_tfBSeekSort
[fBNseek] More...

Int_tfBSeekSortLen
[fBNseek] More...

char * fBuffer
[fBufferSize] buffer of contiguous prefetched blocks More...

Int_t fBufferLen
Current buffer length (<= fBufferSize) More...

Int_t fBufferSize
Allocated size of fBuffer (at a given time) More...

Int_t fBufferSizeMin
Original size of fBuffer. More...

Number of bytes read for this cache. More...

Bool_t fEnablePrefetching

TFilefFile
Pointer to file. More...

Bool_t fIsSorted
True if fSeek array is sorted. More...

Bool_t fIsTransferred
True when fBuffer contains something valid. More...

Int_tfLen
[fNb] Length of long buffers More...

Int_t fNb
Number of long buffers. More...

Int_t fNseek
Number of blocks to be prefetched. More...

Int_t fNtot
Total size of prefetched blocks. More...

Long64_tfPos
[fNb] start of long buffers More...

TFilePrefetchfPrefetch

Long64_t fPrefetchedBlocks
Number of blocks prefetched. More...

Number of read calls for this cache. More...

Long64_tfSeek
[fNseek] Position on file of buffers to be prefetched More...

Int_tfSeekIndex
[fNseek] sorted index table of fSeek More...

Int_tfSeekLen
[fNseek] Length of buffers to be prefetched More...

Int_tfSeekPos
[fNseek] Position of sorted blocks in fBuffer More...

Int_t fSeekSize
Allocated size of fSeek. More...

Long64_tfSeekSort
[fNseek] Position on file of buffers to be prefetched (sorted) More...

Int_tfSeekSortLen
[fNseek] Length of buffers to be prefetched (sorted) More...

Static Protected Attributes

static Int_t fgLearnEntries = 100
number of entries used for learning mode More...

Private Member Functions

TTreeCache (const TTreeCache &)=delete
this class cannot be copied More...

Bool_t CalculateMissCache ()
Calculate the appropriate miss cache to fetch; helper function for FillMissCache. More...

TBranchCalculateMissEntries (Long64_t, int, bool)
Given an file read, try to determine the corresponding branch. More...

Bool_t CheckMissCache (char *buf, Long64_t pos, int len)
Check the miss cache for a particular buffer, fetching if deemed necessary. More...

Bool_t FillMissCache ()
Fill the miss cache from the current set of active branches. More...

IOPos FindBranchBasketPos (TBranch &, Long64_t entry)
Given a branch and an entry, determine the file location (offset / size) of the corresponding basket. More...

TTreeCacheoperator= (const TTreeCache &)=delete

Bool_t ProcessMiss (Long64_t pos, int len)

Protected Member Functions inherited from TFileCacheRead
void SetEnablePrefetchingImpl (Bool_t setPrefetching=kFALSE)

Protected Member Functions inherited from TObject
virtual void DoError (int level, const char *location, const char *fmt, va_list va) const
Interface to ErrorHandler (protected). More...

void MakeZombie ()

#include <TTreeCache.h>

Inheritance diagram for TTreeCache:
[legend]

◆ EPrefillType

Enumerator
kNoPrefill
kAllBranches

Definition at line 38 of file TTreeCache.h.

◆ TTreeCache() [1/3]

 TTreeCache::TTreeCache ( const TTreeCache & )
privatedelete

this class cannot be copied

◆ TTreeCache() [2/3]

 TTreeCache::TTreeCache ( )

Default Constructor.

Definition at line 271 of file TTreeCache.cxx.

◆ TTreeCache() [3/3]

 TTreeCache::TTreeCache ( TTree * tree, Int_t buffersize = 0 )

Constructor.

Definition at line 278 of file TTreeCache.cxx.

◆ ~TTreeCache()

 TTreeCache::~TTreeCache ( )
virtual

Destructor. (in general called by the TFile destructor)

Definition at line 290 of file TTreeCache.cxx.

Member Function Documentation

 Int_t TTreeCache::AddBranch ( TBranch * b, Bool_t subbranches = kFALSE )
virtual

Add a branch to the list of branches to be stored in the cache this function is called by TBranch::GetBasket Returns:

• -1 on error

Reimplemented in TTreeCacheUnzip.

Definition at line 307 of file TTreeCache.cxx.

 Int_t TTreeCache::AddBranch ( const char * bname, Bool_t subbranches = kFALSE )
virtual

Add a branch to the list of branches to be stored in the cache this is to be used by user (thats why we pass the name of the branch).

It works in exactly the same way as TTree::SetBranchStatus so you probably want to look over there for details about the use of bname with regular expressions. The branches are taken with respect to the Owner of this TTreeCache (i.e. the original Tree) NB: if bname="*" all branches are put in the cache and the learning phase stopped Returns:

• -1 on error

Reimplemented in TTreeCacheUnzip.

Definition at line 387 of file TTreeCache.cxx.

◆ CalculateMissCache()

 Bool_t TTreeCache::CalculateMissCache ( )
private

Calculate the appropriate miss cache to fetch; helper function for FillMissCache.

◆ CalculateMissEntries()

 TBranch * TTreeCache::CalculateMissEntries ( Long64_t pos, int len, bool all )
private

Given an file read, try to determine the corresponding branch.

Given a particular IO description (offset / length) representing a 'miss' of the TTreeCache's primary cache, calculate all the corresponding IO that should be performed.

all indicates that this function should search the set of all branches in this TTree. When set to false, we only search through branches that have previously incurred a miss.

Returns:

• TBranch pointer corresponding to the basket that will be retrieved by this IO operation.
• If no corresponding branch could be found (or an error occurs), this returns nullptr.

Definition at line 731 of file TTreeCache.cxx.

◆ CheckMissCache()

 Bool_t TTreeCache::CheckMissCache ( char * buf, Long64_t pos, int len )
private

Check the miss cache for a particular buffer, fetching if deemed necessary.

Given an IO operation (pos, len) that was a cache miss in the primary TTC, try the operation again with the miss cache.

Returns true if the IO operation was successful and the contents of buf were populated with the requested data.

Definition at line 862 of file TTreeCache.cxx.

◆ Disable()

 virtual void TTreeCache::Disable ( )
inlinevirtual

Definition at line 137 of file TTreeCache.h.

◆ DropBranch() [1/2]

 Int_t TTreeCache::DropBranch ( TBranch * b, Bool_t subbranches = kFALSE )
virtual

Remove a branch to the list of branches to be stored in the cache this function is called by TBranch::GetBasket.

Returns:

• 0 branch dropped or not in cache
• -1 on error

Definition at line 482 of file TTreeCache.cxx.

◆ DropBranch() [2/2]

 Int_t TTreeCache::DropBranch ( const char * bname, Bool_t subbranches = kFALSE )
virtual

Remove a branch to the list of branches to be stored in the cache this is to be used by user (thats why we pass the name of the branch).

It works in exactly the same way as TTree::SetBranchStatus so you probably want to look over there for details about the use of bname with regular expressions. The branches are taken with respect to the Owner of this TTreeCache (i.e. the original Tree) NB: if bname="*" all branches are put in the cache and the learning phase stopped Returns:

• 0 branch dropped or not in cache
• -1 on error

Definition at line 527 of file TTreeCache.cxx.

◆ Enable()

 virtual void TTreeCache::Enable ( )
inlinevirtual

Definition at line 138 of file TTreeCache.h.

◆ FillBuffer()

 Bool_t TTreeCache::FillBuffer ( )
virtual

Fill the cache buffer with the branches in the cache.

Reimplemented in TTreeCacheUnzip.

Definition at line 1055 of file TTreeCache.cxx.

◆ FillMissCache()

 Bool_t TTreeCache::FillMissCache ( )
private

Fill the miss cache from the current set of active branches.

 TTreeCache::IOPos TTreeCache::FindBranchBasketPos ( TBranch & b, Long64_t entry )
private

Given a branch and an entry, determine the file location (offset / size) of the corresponding basket.

For the event currently being fetched into the miss cache, find the IO (offset / length tuple) to pull in the current basket for a given branch.

Returns:

• IOPos describing the IO operation necessary for the basket on this branch
• On failure, IOPos.length will be set to 0.

Definition at line 657 of file TTreeCache.cxx.

◆ GetCachedBranches()

 const TObjArray* TTreeCache::GetCachedBranches ( ) const
inline

Definition at line 140 of file TTreeCache.h.

◆ GetConfiguredPrefillType()

 TTreeCache::EPrefillType TTreeCache::GetConfiguredPrefillType ( ) const

Return the desired prefill type from the environment or resource variable.

• 0 - No prefill
• 1 - All branches

Definition at line 1730 of file TTreeCache.cxx.

◆ GetEfficiency()

 Double_t TTreeCache::GetEfficiency ( ) const

Give the total efficiency of the primary cache...

defined as the ratio of blocks found in the cache vs. the number of blocks prefetched ( it could be more than 1 if we read the same block from the cache more than once )

Note: This should eb used at the end of the processing or we will get incomplete stats

Definition at line 1753 of file TTreeCache.cxx.

◆ GetEfficiencyRel()

 Double_t TTreeCache::GetEfficiencyRel ( ) const

This will indicate a sort of relative efficiency...

a ratio of the reads found in the cache to the number of reads so far

Definition at line 1777 of file TTreeCache.cxx.

◆ GetEntryMax()

 virtual Int_t TTreeCache::GetEntryMax ( ) const
inlinevirtual

Definition at line 145 of file TTreeCache.h.

◆ GetEntryMin()

 virtual Int_t TTreeCache::GetEntryMin ( ) const
inlinevirtual

Definition at line 144 of file TTreeCache.h.

◆ GetLearnEntries()

 Int_t TTreeCache::GetLearnEntries ( )
static

Static function returning the number of entries used to train the cache see SetLearnEntries.

Definition at line 1802 of file TTreeCache.cxx.

◆ GetLearnPrefill()

 virtual EPrefillType TTreeCache::GetLearnPrefill ( ) const
inlinevirtual

Definition at line 147 of file TTreeCache.h.

◆ GetMissEfficiency()

 Double_t TTreeCache::GetMissEfficiency ( ) const

The total efficiency of the 'miss cache' - defined as the ratio of blocks found in the cache versus the number of blocks prefetched.

Definition at line 1765 of file TTreeCache.cxx.

◆ GetMissEfficiencyRel()

 Double_t TTreeCache::GetMissEfficiencyRel ( ) const

Relative efficiency of the 'miss cache' - ratio of the reads found in cache to the number of reads so far.

Definition at line 1789 of file TTreeCache.cxx.

◆ GetOptimizeMisses()

 Bool_t TTreeCache::GetOptimizeMisses ( ) const
inline

Definition at line 139 of file TTreeCache.h.

◆ GetTree()

 TTree* TTreeCache::GetTree ( ) const
inline

Definition at line 150 of file TTreeCache.h.

◆ IsAutoCreated()

 Bool_t TTreeCache::IsAutoCreated ( ) const
inline

Definition at line 151 of file TTreeCache.h.

◆ IsEnabled()

 virtual Bool_t TTreeCache::IsEnabled ( ) const
inlinevirtual

Definition at line 152 of file TTreeCache.h.

◆ IsLearning()

 virtual Bool_t TTreeCache::IsLearning ( ) const
inlinevirtual

Definition at line 153 of file TTreeCache.h.

◆ LearnPrefill()

 void TTreeCache::LearnPrefill ( )
virtual

Perform an initial prefetch, attempting to read as much of the learning phase baskets for all branches at once.

Definition at line 2165 of file TTreeCache.cxx.

◆ operator=()

 TTreeCache& TTreeCache::operator= ( const TTreeCache & )
privatedelete

◆ Print()

 void TTreeCache::Print ( Option_t * option = "" ) const
virtual

Print cache statistics.

Like:

******TreeCache statistics for file: cms2.root ******
Number of branches in the cache ...: 1093
Cache Efficiency ..................: 0.997372
Cache Efficiency Rel...............: 1.000000
Learn entries......................: 100
Reading............................: 72761843 bytes in 7 transactions
Average transaction................: 10394.549000 Kbytes
Number of blocks in current cache..: 210, total size: 6280352
• if option = "a" the list of blocks in the cache is printed see also class TTreePerfStats.
• if option contains 'cachedbranches', the list of branches being cached is printed.

Reimplemented in TTreeCacheUnzip.

Definition at line 1827 of file TTreeCache.cxx.

◆ ProcessMiss()

 Bool_t TTreeCache::ProcessMiss ( Long64_t pos, int len )
private

Process a cache miss; (pos, len) isn't in the buffer.

The first time we have a miss, we buffer as many baskets we can (up to the maximum size of the TTreeCache) in memory from all branches that are not in the prefetch list.

Subsequent times, we fetch all the buffers corresponding to branches that had previously seen misses. If it turns out the (pos, len) isn't in the list of branches, we treat this as if it was the first miss.

Returns true if we were able to pull the data into the miss cache.

Definition at line 804 of file TTreeCache.cxx.

 Int_t TTreeCache::ReadBuffer ( char * buf, Long64_t pos, Int_t len )
virtual

Read buffer at position pos if the request is in the list of prefetched blocks read from fBuffer.

Otherwise try to fill the cache from the list of selected branches, and recheck if pos is now in the list. Returns:

• -1 in case of read failure,
• 0 in case not in cache,

Definition at line 1955 of file TTreeCache.cxx.

 Int_t TTreeCache::ReadBufferNormal ( char * buf, Long64_t pos, Int_t len )
virtual

Definition at line 1855 of file TTreeCache.cxx.

 Int_t TTreeCache::ReadBufferPrefetch ( char * buf, Long64_t pos, Int_t len )
virtual

Used to read a chunk from a block previously fetched.

It will call FillBuffer even if the cache lookup succeeds, because it will try to prefetch the next block as soon as we start reading from the current block.

Definition at line 1914 of file TTreeCache.cxx.

◆ ResetCache()

 void TTreeCache::ResetCache ( )
virtual

This will simply clear the cache.

Reimplemented in TTreeCacheUnzip.

Definition at line 1968 of file TTreeCache.cxx.

◆ ResetMissCache()

 void TTreeCache::ResetMissCache ( )

Reset all the miss cache training.

The contents of the miss cache will be emptied as well as the list of branches used.

Definition at line 638 of file TTreeCache.cxx.

◆ SetAutoCreated()

 void TTreeCache::SetAutoCreated ( Bool_t val )
inline

Definition at line 164 of file TTreeCache.h.

◆ SetBufferSize()

 Int_t TTreeCache::SetBufferSize ( Int_t buffersize )
virtual

Change the underlying buffer size of the cache.

If the change of size means some cache content is lost, or if the buffer is now larger, setup for a cache refill the next time there is a read Returns:

• 0 if the buffer content is still available
• 1 if some or all of the buffer content has been made unavailable
• -1 on error

Reimplemented in TTreeCacheUnzip.

Definition at line 1987 of file TTreeCache.cxx.

◆ SetEntryRange()

 void TTreeCache::SetEntryRange ( Long64_t emin, Long64_t emax )
virtual

Set the minimum and maximum entry number to be processed this information helps to optimize the number of baskets to read when prefetching the branch buffers.

Reimplemented in TTreeCacheUnzip.

Definition at line 2020 of file TTreeCache.cxx.

◆ SetFile()

 void TTreeCache::SetFile ( TFile * file, TFile::ECacheAction action = TFile::kDisconnect )
virtual

Overload to make sure that the object specific.

Definition at line 2042 of file TTreeCache.cxx.

◆ SetLearnEntries()

 void TTreeCache::SetLearnEntries ( Int_t n = 10 )
static

Static function to set the number of entries to be used in learning mode The default value for n is 10.

n must be >= 1

Definition at line 2059 of file TTreeCache.cxx.

◆ SetLearnPrefill()

 void TTreeCache::SetLearnPrefill ( TTreeCache::EPrefillType type = kNoPrefill )
virtual

Set whether the learning period is started with a prefilling of the cache and which type of prefilling is used.

The two value currently supported are:

• TTreeCache::kNoPrefill disable the prefilling
• TTreeCache::kAllBranches fill the cache with baskets from all branches. The default prefilling behavior can be controlled by setting TTreeCache.Prefill or the environment variable ROOT_TTREECACHE_PREFILL.

Definition at line 2074 of file TTreeCache.cxx.

◆ SetOptimizeMisses()

 void TTreeCache::SetOptimizeMisses ( Bool_t opt )

Start of methods for the miss cache.

Enable / disable the miss cache.

The first time this is called on a TTreeCache object, the corresponding data structures will be allocated. Subsequent enable / disables will simply turn the functionality on/off.

Definition at line 624 of file TTreeCache.cxx.

◆ StartLearningPhase()

 void TTreeCache::StartLearningPhase ( )

The name should be enough to explain the method.

The only additional comments is that the cache is cleaned before the new learning phase.

Definition at line 2084 of file TTreeCache.cxx.

◆ StopLearningPhase()

 void TTreeCache::StopLearningPhase ( )
virtual

This is the counterpart of StartLearningPhase() and can be used to stop the learning phase.

It's useful when the user knows exactly what branches they are going to use. For the moment it's just a call to FillBuffer() since that method will create the buffer lists from the specified branches.

Reimplemented in TTreeCacheUnzip.

Definition at line 2101 of file TTreeCache.cxx.

◆ UpdateBranches()

 void TTreeCache::UpdateBranches ( TTree * tree )
virtual

Update pointer to current Tree and recompute pointers to the branches in the cache.

Reimplemented in TTreeCacheUnzip.

Definition at line 2125 of file TTreeCache.cxx.

◆ fAutoCreated

 Bool_t TTreeCache::fAutoCreated {kFALSE}
protected

! true if cache was automatically created

Definition at line 69 of file TTreeCache.h.

◆ fBranches

 TObjArray* TTreeCache::fBranches {nullptr}
protected

! List of branches to be stored in the cache

Definition at line 54 of file TTreeCache.h.

◆ fBrNames

 TList* TTreeCache::fBrNames {nullptr}
protected

! list of branch names in the cache

Definition at line 55 of file TTreeCache.h.

◆ fCurrentClusterStart

 Long64_t TTreeCache::fCurrentClusterStart {-1}
protected

! Start of the cluster(s) where the current content was picked out

Definition at line 45 of file TTreeCache.h.

◆ fEnabled

 Bool_t TTreeCache::fEnabled {kTRUE}
protected

! cache enabled for cached reading

Definition at line 66 of file TTreeCache.h.

◆ fEntryCurrent

 Long64_t TTreeCache::fEntryCurrent {-1}
protected

! current lowest entry number in the cache

Definition at line 43 of file TTreeCache.h.

◆ fEntryMax

 Long64_t TTreeCache::fEntryMax {1}
protected

! last entry in the cache

Definition at line 42 of file TTreeCache.h.

◆ fEntryMin

 Long64_t TTreeCache::fEntryMin {0}
protected

! first entry in the cache

Definition at line 41 of file TTreeCache.h.

◆ fEntryNext

 Long64_t TTreeCache::fEntryNext {-1}
protected

! next entry number where cache must be filled

Definition at line 44 of file TTreeCache.h.

◆ fFillTimes

 Int_t TTreeCache::fFillTimes {0}
protected

! how many times we can fill the current buffer

Definition at line 62 of file TTreeCache.h.

◆ fFirstBuffer

 Bool_t TTreeCache::fFirstBuffer {kTRUE}
protected

! true if first buffer is used for prefetching

Definition at line 59 of file TTreeCache.h.

◆ fFirstEntry

 Long64_t TTreeCache::fFirstEntry {-1}
protected

! save the value of the first entry

Definition at line 64 of file TTreeCache.h.

◆ fFirstMiss

 Long64_t TTreeCache::fFirstMiss {-1}
protected

! set to the event # of the first miss.

Definition at line 74 of file TTreeCache.h.

◆ fFirstTime

 Bool_t TTreeCache::fFirstTime {kTRUE}
protected

! save the fact that we processes the first entry

Definition at line 63 of file TTreeCache.h.

◆ fgLearnEntries

 Int_t TTreeCache::fgLearnEntries = 100
staticprotected

number of entries used for learning mode

Definition at line 68 of file TTreeCache.h.

◆ fIsLearning

 Bool_t TTreeCache::fIsLearning {kTRUE}
protected

! true if cache is in learning mode

Definition at line 57 of file TTreeCache.h.

◆ fIsManual

 Bool_t TTreeCache::fIsManual {kFALSE}
protected

! true if cache is StopLearningPhase was used

Definition at line 58 of file TTreeCache.h.

◆ fLastMiss

 Long64_t TTreeCache::fLastMiss {-1}
protected

! set to the event # of the last miss.

Definition at line 75 of file TTreeCache.h.

◆ fMissCache

 std::unique_ptr TTreeCache::fMissCache
protected

! Cache contents for misses

Definition at line 106 of file TTreeCache.h.

◆ fNbranches

 Int_t TTreeCache::fNbranches {0}
protected

! Number of branches in the cache

Definition at line 47 of file TTreeCache.h.

◆ fNextClusterStart

 Long64_t TTreeCache::fNextClusterStart {-1}
protected

! End+1 of the cluster(s) where the current content was picked out

Definition at line 46 of file TTreeCache.h.

protected

Definition at line 51 of file TTreeCache.h.

protected

Definition at line 49 of file TTreeCache.h.

protected

Number of blocks read into the secondary ("miss") cache.

Definition at line 53 of file TTreeCache.h.

protected

Definition at line 50 of file TTreeCache.h.

protected

Number of blocks read and found in the cache.

Definition at line 48 of file TTreeCache.h.

protected

Number of blocks that were prefetched.

Definition at line 52 of file TTreeCache.h.

◆ fOneTime

 Bool_t TTreeCache::fOneTime {kFALSE}
protected

! used in the learning phase

Definition at line 60 of file TTreeCache.h.

◆ fOptimizeMisses

 Bool_t TTreeCache::fOptimizeMisses {kFALSE}
protected

! true if we should optimize cache misses.

Definition at line 73 of file TTreeCache.h.

◆ fPrefillType

 EPrefillType TTreeCache::fPrefillType
protected

Whether a pre-filling is enabled (and if applicable which type)

Definition at line 67 of file TTreeCache.h.

protected

Definition at line 65 of file TTreeCache.h.

protected

Definition at line 61 of file TTreeCache.h.

◆ fTree

 TTree* TTreeCache::fTree {nullptr}
protected

! pointer to the current Tree

Definition at line 56 of file TTreeCache.h.

Libraries for TTreeCache:
[legend]

The documentation for this class was generated from the following files: