Logo ROOT   6.08/07
Reference Guide
TTreeCache.cxx
Go to the documentation of this file.
1 // @(#)root/tree:$Id$
2 // Author: Rene Brun 04/06/2006
3 
4 /*************************************************************************
5  * Copyright (C) 1995-2000, Rene Brun and Fons Rademakers. *
6  * All rights reserved. *
7  * *
8  * For the licensing terms see $ROOTSYS/LICENSE. *
9  * For the list of contributors see $ROOTSYS/README/CREDITS. *
10  *************************************************************************/
11 
12 /** \class TTreeCache
13 \ingroup tree
14 
15 A specialized TFileCacheRead object for a TTree.
16 
17 This class acts as a file cache, registering automatically the baskets from
18 the branches being processed (TTree::Draw or TTree::Process and TSelectors)
19 when in the learning phase. The learning phase is by default 100 entries.
20 It can be changed via TTreeCache::SetLearnEntries.
21 
22 This cache speeds-up considerably the performance, in particular
23 when the Tree is accessed remotely via a high latency network.
24 
25 The default cache size (10 Mbytes) may be changed via the function
26 TTree::SetCacheSize
27 
28 Only the baskets for the requested entry range are put in the cache
29 
30 For each Tree being processed a TTreeCache object is created.
31 This object is automatically deleted when the Tree is deleted or
32 when the file is deleted.
33 
34 - Special case of a TChain
35  Once the training is done on the first Tree, the list of branches
36  in the cache is kept for the following files.
37 
38 - Special case of a TEventlist
39  if the Tree or TChain has a TEventlist, only the buffers
40  referenced by the list are put in the cache.
41 
42 The learning period is started or restarted when:
43  - TTree automatically creates a cache. This feature can be
44  controlled with an env. variable or the TTreeCache.Size option.
45  - TTree::SetCacheSize is called with a non-zero size and a cache
46  did not previously exist
47  - TTreeCache::StartLearningPhase is called.
48  - TTree[Cache]::SetEntryRange is called
49  * and the learning is not yet finished
50  * and has not been set to manual
51  * and the new minimun entry is different.
52 
53 The learning period is stopped (and prefetching is actually started) when:
54  - TTree[Cache]::StopLearningPhase is called.
55  - An entry outside the 'learning' range is requested
56  The 'learning range is from fEntryMin (default to 0) to
57  fEntryMin + fgLearnEntries (default to 100).
58  - A 'cached' TChain switches over to a new file.
59 
60 ## WHY DO WE NEED the TreeCache when doing data analysis?
61 
62 When writing a TTree, the branch buffers are kept in memory.
63 A typical branch buffersize (before compression) is typically 32 KBytes.
64 After compression, the zipped buffer may be just a few Kbytes.
65 The branch buffers cannot be much larger in case of Trees with several
66 hundred or thousand branches.
67 
68 When writing, this does not generate a performance problem because branch
69 buffers are always written sequentially and the OS is in general clever enough
70 to flush the data to the output file when a few MBytes of data have to be written.
71 When reading at the contrary, one may hit a performance problem when reading
72 across a network (LAN or WAN) and the network latency is high.
73 For example in a WAN with 10ms latency, reading 1000 buffers of 10 KBytes each
74 with no cache will imply 10s penalty where a local read of the 10 MBytes would
75 take about 1 second.
76 
77 The TreeCache will try to prefetch all the buffers for the selected branches
78 such that instead of transfering 1000 buffers of 10 Kbytes, it will be able
79 to transfer one single large buffer of 10 Mbytes in one single transaction.
80 Not only the TreeCache minimizes the number of transfers, but in addition
81 it can sort the blocks to be read in increasing order such that the file
82 is read sequentially.
83 
84 Systems like xrootd, dCache or httpd take advantage of the TreeCache in
85 reading ahead as much data as they can and return to the application
86 the maximum data specified in the cache and have the next chunk of data ready
87 when the next request comes.
88 
89 ## HOW TO USE the TreeCache
90 
91 A few use cases are discussed below. A cache may be created with automatic sizing
92 when a TTree is used:
93 
94 Caches are created and automatically sized for TTrees when TTreeCache.Size or
95 the environment variable ROOT_TTREECACHE_SIZE is set to a sizing factor.
96 
97 But there are many possible configurations where manual control may be wanted.
98 In some applications you know a priori the list of branches to read. In other
99 applications the analysis loop calls several layers of user functions where it
100 is impossible to predict a priori which branches will be used. This
101 is probably the most frequent case. In this case ROOT I/O will flag used
102 branches automatically when a branch buffer is read during the learning phase.
103 The TreeCache interface provides functions to instruct the cache about the used
104 branches if they are known a priori. In the examples below, portions of analysis
105 code are shown. The few statements involving the TreeCache are marked with `//<<<`
106 
107 ### 1. with TTree::Draw
108 
109 the TreeCache is automatically used by TTree::Draw. The function knows
110 which branches are used in the query and it puts automatically these branches
111 in the cache. The entry range is also known automatically.
112 
113 ### 2. with TTree::Process and TSelectors
114 
115 You must enable the cache and tell the system which branches to cache
116 and also specify the entry range. It is important to specify the entry range
117 in case you process only a subset of the events, otherwise you run the risk
118 to store in the cache entries that you do not need.
119 
120 #### example 2a
121 ~~~ {.cpp}
122  TTree *T = (TTree*)f->Get("mytree");
123  Long64_t nentries = T->GetEntries();
124  Int_t cachesize = 10000000; //10 MBytes
125  T->SetCacheSize(cachesize); //<<<
126  T->AddBranchToCache("*",kTRUE); //<<< add all branches to the cache
127  T->Process('myselector.C+");
128  //in the TSelector::Process function we read all branches
129  T->GetEntry(i);
130  ... here you process your entry
131 ~~~
132 #### example 2b
133 
134 in the Process function we read a subset of the branches.
135 Only the branches used in the first entry will be put in the cache
136 ~~~ {.cpp}
137  TTree *T = (TTree*)f->Get("mytree");
138  //we want to process only the 200 first entries
139  Long64_t nentries=200;
140  int efirst= 0;
141  int elast = efirst+nentries;
142  Int_t cachesize = 10000000; //10 MBytes
143  TTreeCache::SetLearnEntries(1); //<<< we can take the decision after 1 entry
144  T->SetCacheSize(cachesize); //<<<
145  T->SetCacheEntryRange(efirst,elast); //<<<
146  T->Process('myselector.C+","",nentries,efirst);
147  // in the TSelector::Process we read only 2 branches
148  TBranch *b1 = T->GetBranch("branch1");
149  b1->GetEntry(i);
150  if (somecondition) return;
151  TBranch *b2 = T->GetBranch("branch2");
152  b2->GetEntry(i);
153  ... here you process your entry
154 ~~~
155 ### 3. with your own event loop
156 
157 #### example 3a
158 
159 in your analysis loop, you always use 2 branches. You want to prefetch
160 the branch buffers for these 2 branches only.
161 ~~~ {.cpp}
162  TTree *T = (TTree*)f->Get("mytree");
163  TBranch *b1 = T->GetBranch("branch1");
164  TBranch *b2 = T->GetBranch("branch2");
165  Long64_t nentries = T->GetEntries();
166  Int_t cachesize = 10000000; //10 MBytes
167  T->SetCacheSize(cachesize); //<<<
168  T->AddBranchToCache(b1,kTRUE); //<<<add branch1 and branch2 to the cache
169  T->AddBranchToCache(b2,kTRUE); //<<<
170  T->StopCacheLearningPhase(); //<<<
171  for (Long64_t i=0;i<nentries;i++) {
172  T->LoadTree(i); //<<< important call when calling TBranch::GetEntry after
173  b1->GetEntry(i);
174  if (some condition not met) continue;
175  b2->GetEntry(i);
176  if (some condition not met) continue;
177  //here we read the full event only in some rare cases.
178  //there is no point in caching the other branches as it might be
179  //more economical to read only the branch buffers really used.
180  T->GetEntry(i);
181  .. process the rare but interesting cases.
182  ... here you process your entry
183  }
184 ~~~
185 #### example 3b
186 
187 in your analysis loop, you always use 2 branches in the main loop.
188 you also call some analysis functions where a few more branches will be read.
189 but you do not know a priori which ones. There is no point in prefetching
190 branches that will be used very rarely.
191 ~~~ {.cpp}
192  TTree *T = (TTree*)f->Get("mytree");
193  Long64_t nentries = T->GetEntries();
194  Int_t cachesize = 10000000; //10 MBytes
195  T->SetCacheSize(cachesize); //<<<
196  T->SetCacheLearnEntries(5); //<<< we can take the decision after 5 entries
197  TBranch *b1 = T->GetBranch("branch1");
198  TBranch *b2 = T->GetBranch("branch2");
199  for (Long64_t i=0;i<nentries;i++) {
200  T->LoadTree(i);
201  b1->GetEntry(i);
202  if (some condition not met) continue;
203  b2->GetEntry(i);
204  //at this point we may call a user function where a few more branches
205  //will be read conditionally. These branches will be put in the cache
206  //if they have been used in the first 10 entries
207  if (some condition not met) continue;
208  //here we read the full event only in some rare cases.
209  //there is no point in caching the other branches as it might be
210  //more economical to read only the branch buffers really used.
211  T->GetEntry(i);
212  .. process the rare but interesting cases.
213  ... here you process your entry
214  }
215 ~~~
216 ## SPECIAL CASES WHERE TreeCache should not be activated
217 
218 When reading only a small fraction of all entries such that not all branch
219 buffers are read, it might be faster to run without a cache.
220 
221 ## HOW TO VERIFY That the TreeCache has been used and check its performance
222 
223 Once your analysis loop has terminated, you can access/print the number
224 of effective system reads for a given file with a code like
225 (where TFile* f is a pointer to your file)
226 ~~~ {.cpp}
227  printf("Reading %lld bytes in %d transactions\n",f->GetBytesRead(), f->GetReadCalls());
228 ~~~
229 */
230 
231 #include "TSystem.h"
232 #include "TEnv.h"
233 #include "TTreeCache.h"
234 #include "TChain.h"
235 #include "TList.h"
236 #include "TBranch.h"
237 #include "TEventList.h"
238 #include "TObjString.h"
239 #include "TRegexp.h"
240 #include "TLeaf.h"
241 #include "TFriendElement.h"
242 #include "TFile.h"
243 #include <limits.h>
244 
246 
248 
249 ////////////////////////////////////////////////////////////////////////////////
250 /// Default Constructor.
251 
253  fEntryMin(0),
254  fEntryMax(1),
255  fEntryCurrent(-1),
256  fEntryNext(-1),
257  fNbranches(0),
258  fNReadOk(0),
259  fNReadMiss(0),
260  fNReadPref(0),
261  fBranches(0),
262  fBrNames(0),
263  fTree(0),
264  fIsLearning(kTRUE),
265  fIsManual(kFALSE),
266  fFirstBuffer(kTRUE),
267  fOneTime(kFALSE),
268  fReverseRead(0),
269  fFillTimes(0),
270  fFirstTime(kTRUE),
271  fFirstEntry(-1),
272  fReadDirectionSet(kFALSE),
273  fEnabled(kTRUE),
274  fPrefillType(GetConfiguredPrefillType()),
275  fAutoCreated(kFALSE)
276 {
277 }
278 
279 ////////////////////////////////////////////////////////////////////////////////
280 /// Constructor.
281 
282 TTreeCache::TTreeCache(TTree *tree, Int_t buffersize) : TFileCacheRead(tree->GetCurrentFile(),buffersize,tree),
283  fEntryMin(0),
284  fEntryMax(tree->GetEntriesFast()),
285  fEntryCurrent(-1),
286  fEntryNext(0),
287  fNbranches(0),
288  fNReadOk(0),
289  fNReadMiss(0),
290  fNReadPref(0),
291  fBranches(0),
292  fBrNames(new TList),
293  fTree(tree),
294  fIsLearning(kTRUE),
295  fIsManual(kFALSE),
296  fFirstBuffer(kTRUE),
297  fOneTime(kFALSE),
298  fReverseRead(0),
299  fFillTimes(0),
300  fFirstTime(kTRUE),
301  fFirstEntry(-1),
302  fReadDirectionSet(kFALSE),
303  fEnabled(kTRUE),
304  fPrefillType(GetConfiguredPrefillType()),
305  fAutoCreated(kFALSE)
306 {
308  Int_t nleaves = tree->GetListOfLeaves()->GetEntries();
309  fBranches = new TObjArray(nleaves);
310 }
311 
312 ////////////////////////////////////////////////////////////////////////////////
313 /// Destructor. (in general called by the TFile destructor)
314 
316 {
317  // Informe the TFile that we have been deleted (in case
318  // we are deleted explicitly by legacy user code).
319  if (fFile) fFile->SetCacheRead(0, fTree);
320 
321  delete fBranches;
322  if (fBrNames) {fBrNames->Delete(); delete fBrNames; fBrNames=0;}
323 }
324 
325 ////////////////////////////////////////////////////////////////////////////////
326 /// Add a branch to the list of branches to be stored in the cache
327 /// this function is called by TBranch::GetBasket
328 /// Returns:
329 /// - 0 branch added or already included
330 /// - -1 on error
331 
332 Int_t TTreeCache::AddBranch(TBranch *b, Bool_t subbranches /*= kFALSE*/)
333 {
334  if (!fIsLearning) {
335  return -1;
336  }
337 
338  // Reject branch that are not from the cached tree.
339  if (!b || fTree->GetTree() != b->GetTree()) return -1;
340 
341  // Is this the first addition of a branch (and we are learning and we are in
342  // the expected TTree), then prefill the cache. (We expect that in future
343  // release the Prefill-ing will be the default so we test for that inside the
344  // LearnPrefill call).
345  if (fNbranches == 0 && fEntryMin >= 0 && b->GetReadEntry() == fEntryMin) LearnPrefill();
346 
347  //Is branch already in the cache?
348  Bool_t isNew = kTRUE;
349  for (int i=0;i<fNbranches;i++) {
350  if (fBranches->UncheckedAt(i) == b) {isNew = kFALSE; break;}
351  }
352  if (isNew) {
353  fTree = b->GetTree();
354  fBranches->AddAtAndExpand(b, fNbranches);
355  fBrNames->Add(new TObjString(b->GetName()));
356  fNbranches++;
357  if (gDebug > 0) printf("Entry: %lld, registering branch: %s\n",b->GetTree()->GetReadEntry(),b->GetName());
358  }
359 
360  // process subbranches
361  Int_t res = 0;
362  if (subbranches) {
363  TObjArray *lb = b->GetListOfBranches();
364  Int_t nb = lb->GetEntriesFast();
365  for (Int_t j = 0; j < nb; j++) {
366  TBranch* branch = (TBranch*) lb->UncheckedAt(j);
367  if (!branch) continue;
368  if (AddBranch(branch, subbranches)<0) {
369  res = -1;
370  }
371  }
372  }
373  return res;
374 }
375 
376 ////////////////////////////////////////////////////////////////////////////////
377 /// Add a branch to the list of branches to be stored in the cache
378 /// this is to be used by user (thats why we pass the name of the branch).
379 /// It works in exactly the same way as TTree::SetBranchStatus so you
380 /// probably want to look over ther for details about the use of bname
381 /// with regular expressions.
382 /// The branches are taken with respect to the Owner of this TTreeCache
383 /// (i.e. the original Tree)
384 /// NB: if bname="*" all branches are put in the cache and the learning phase stopped
385 /// Returns:
386 /// - 0 branch added or already included
387 /// - -1 on error
388 
389 Int_t TTreeCache::AddBranch(const char *bname, Bool_t subbranches /*= kFALSE*/)
390 {
391  TBranch *branch, *bcount;
392  TLeaf *leaf, *leafcount;
393 
394  Int_t i;
395  Int_t nleaves = (fTree->GetListOfLeaves())->GetEntriesFast();
396  TRegexp re(bname,kTRUE);
397  Int_t nb = 0;
398  Int_t res = 0;
399 
400  // first pass, loop on all branches
401  // for leafcount branches activate/deactivate in function of status
402  Bool_t all = kFALSE;
403  if (!strcmp(bname,"*")) all = kTRUE;
404  for (i=0;i<nleaves;i++) {
405  leaf = (TLeaf*)(fTree->GetListOfLeaves())->UncheckedAt(i);
406  branch = (TBranch*)leaf->GetBranch();
407  TString s = branch->GetName();
408  if (!all) { //Regexp gives wrong result for [] in name
409  TString longname;
410  longname.Form("%s.%s",fTree->GetName(),branch->GetName());
411  if (strcmp(bname,branch->GetName())
412  && longname != bname
413  && s.Index(re) == kNPOS) continue;
414  }
415  nb++;
416  if (AddBranch(branch, subbranches)<0) {
417  res = -1;
418  }
419  leafcount = leaf->GetLeafCount();
420  if (leafcount && !all) {
421  bcount = leafcount->GetBranch();
422  if (AddBranch(bcount, subbranches)<0) {
423  res = -1;
424  }
425  }
426  }
427  if (nb==0 && strchr(bname,'*')==0) {
428  branch = fTree->GetBranch(bname);
429  if (branch) {
430  if (AddBranch(branch, subbranches)<0) {
431  res = -1;
432  }
433  ++nb;
434  }
435  }
436 
437  //search in list of friends
438  UInt_t foundInFriend = 0;
439  if (fTree->GetListOfFriends()) {
440  TIter nextf(fTree->GetListOfFriends());
441  TFriendElement *fe;
442  TString name;
443  while ((fe = (TFriendElement*)nextf())) {
444  TTree *t = fe->GetTree();
445  if (t==0) continue;
446 
447  // If the alias is present replace it with the real name.
448  char *subbranch = (char*)strstr(bname,fe->GetName());
449  if (subbranch!=bname) subbranch = 0;
450  if (subbranch) {
451  subbranch += strlen(fe->GetName());
452  if ( *subbranch != '.' ) subbranch = 0;
453  else subbranch ++;
454  }
455  if (subbranch) {
456  name.Form("%s.%s",t->GetName(),subbranch);
457  if (AddBranch(name, subbranches)<0) {
458  res = -1;
459  }
460  ++foundInFriend;
461  }
462  }
463  }
464  if (!nb && !foundInFriend) {
465  if (gDebug > 0) printf("AddBranch: unknown branch -> %s \n", bname);
466  Error("AddBranch", "unknown branch -> %s", bname);
467  return -1;
468  }
469  //if all branches are selected stop the learning phase
470  if (*bname == '*') {
471  fEntryNext = -1; // We are likely to have change the set of branches, so for the [re-]reading of the cluster.
473  }
474  return res;
475 }
476 
477 ////////////////////////////////////////////////////////////////////////////////
478 /// Remove a branch to the list of branches to be stored in the cache
479 /// this function is called by TBranch::GetBasket.
480 /// Returns:
481 /// - 0 branch dropped or not in cache
482 /// - -1 on error
483 
484 Int_t TTreeCache::DropBranch(TBranch *b, Bool_t subbranches /*= kFALSE*/)
485 {
486  if (!fIsLearning) {
487  return -1;
488  }
489 
490  // Reject branch that are not from the cached tree.
491  if (!b || fTree->GetTree() != b->GetTree()) return -1;
492 
493  //Is branch already in the cache?
494  if (fBranches->Remove(b)) {
495  --fNbranches;
496  if (gDebug > 0) printf("Entry: %lld, un-registering branch: %s\n",b->GetTree()->GetReadEntry(),b->GetName());
497  }
498  delete fBrNames->Remove(fBrNames->FindObject(b->GetName()));
499 
500  // process subbranches
501  Int_t res = 0;
502  if (subbranches) {
503  TObjArray *lb = b->GetListOfBranches();
504  Int_t nb = lb->GetEntriesFast();
505  for (Int_t j = 0; j < nb; j++) {
506  TBranch* branch = (TBranch*) lb->UncheckedAt(j);
507  if (!branch) continue;
508  if (DropBranch(branch, subbranches)<0) {
509  res = -1;
510  }
511  }
512  }
513  return res;
514 }
515 
516 ////////////////////////////////////////////////////////////////////////////////
517 /// Remove a branch to the list of branches to be stored in the cache
518 /// this is to be used by user (thats why we pass the name of the branch).
519 /// It works in exactly the same way as TTree::SetBranchStatus so you
520 /// probably want to look over ther for details about the use of bname
521 /// with regular expresions.
522 /// The branches are taken with respect to the Owner of this TTreeCache
523 /// (i.e. the original Tree)
524 /// NB: if bname="*" all branches are put in the cache and the learning phase stopped
525 /// Returns:
526 /// - 0 branch dropped or not in cache
527 /// - -1 on error
528 
529 Int_t TTreeCache::DropBranch(const char *bname, Bool_t subbranches /*= kFALSE*/)
530 {
531  TBranch *branch, *bcount;
532  TLeaf *leaf, *leafcount;
533 
534  Int_t i;
535  Int_t nleaves = (fTree->GetListOfLeaves())->GetEntriesFast();
536  TRegexp re(bname,kTRUE);
537  Int_t nb = 0;
538  Int_t res = 0;
539 
540  // first pass, loop on all branches
541  // for leafcount branches activate/deactivate in function of status
542  Bool_t all = kFALSE;
543  if (!strcmp(bname,"*")) all = kTRUE;
544  for (i=0;i<nleaves;i++) {
545  leaf = (TLeaf*)(fTree->GetListOfLeaves())->UncheckedAt(i);
546  branch = (TBranch*)leaf->GetBranch();
547  TString s = branch->GetName();
548  if (!all) { //Regexp gives wrong result for [] in name
549  TString longname;
550  longname.Form("%s.%s",fTree->GetName(),branch->GetName());
551  if (strcmp(bname,branch->GetName())
552  && longname != bname
553  && s.Index(re) == kNPOS) continue;
554  }
555  nb++;
556  if (DropBranch(branch, subbranches)<0) {
557  res = -1;
558  }
559  leafcount = leaf->GetLeafCount();
560  if (leafcount && !all) {
561  bcount = leafcount->GetBranch();
562  if (DropBranch(bcount, subbranches)<0) {
563  res = -1;
564  }
565  }
566  }
567  if (nb==0 && strchr(bname,'*')==0) {
568  branch = fTree->GetBranch(bname);
569  if (branch) {
570  if (DropBranch(branch, subbranches)<0) {
571  res = -1;
572  }
573  ++nb;
574  }
575  }
576 
577  //search in list of friends
578  UInt_t foundInFriend = 0;
579  if (fTree->GetListOfFriends()) {
580  TIter nextf(fTree->GetListOfFriends());
581  TFriendElement *fe;
582  TString name;
583  while ((fe = (TFriendElement*)nextf())) {
584  TTree *t = fe->GetTree();
585  if (t==0) continue;
586 
587  // If the alias is present replace it with the real name.
588  char *subbranch = (char*)strstr(bname,fe->GetName());
589  if (subbranch!=bname) subbranch = 0;
590  if (subbranch) {
591  subbranch += strlen(fe->GetName());
592  if ( *subbranch != '.' ) subbranch = 0;
593  else subbranch ++;
594  }
595  if (subbranch) {
596  name.Form("%s.%s",t->GetName(),subbranch);
597  if (DropBranch(name, subbranches)<0) {
598  res = -1;
599  }
600  ++foundInFriend;
601  }
602  }
603  }
604  if (!nb && !foundInFriend) {
605  if (gDebug > 0) printf("DropBranch: unknown branch -> %s \n", bname);
606  Error("DropBranch", "unknown branch -> %s", bname);
607  return -1;
608  }
609  //if all branches are selected stop the learning phase
610  if (*bname == '*') {
611  fEntryNext = -1; // We are likely to have change the set of branches, so for the [re-]reading of the cluster.
612  }
613  return res;
614 }
615 
616 ////////////////////////////////////////////////////////////////////////////////
617 /// Fill the cache buffer with the branches in the cache.
618 
620 {
621  if (fNbranches <= 0) return kFALSE;
623  Long64_t entry = tree->GetReadEntry();
624  Long64_t fEntryCurrentMax = 0;
625 
626  if (fEnablePrefetching) { // Prefetching mode
627  if (fIsLearning) { // Learning mode
628  if (fEntryNext >= 0 && entry >= fEntryNext) {
629  // entry is outside the learn range, need to stop the learning
630  // phase. Doing so may trigger a recursive call to FillBuffer in
631  // the process of filling both prefetching buffers
633  fIsManual = kFALSE;
634  }
635  }
636  if (fIsLearning) { // Learning mode
637  entry = 0;
638  }
639  if (fFirstTime) {
640  //try to detect if it is normal or reverse read
641  fFirstEntry = entry;
642  }
643  else {
644  if (fFirstEntry == entry) return kFALSE;
645  // Set the read direction
646  if (!fReadDirectionSet) {
647  if (entry < fFirstEntry) {
650  }
651  else if (entry > fFirstEntry) {
654  }
655  }
656 
657  if (fReverseRead) {
658  // Reverse reading with prefetching
659  if (fEntryCurrent >0 && entry < fEntryNext) {
660  // We can prefetch the next buffer
661  if (entry >= fEntryCurrent) {
662  entry = fEntryCurrent - tree->GetAutoFlush() * fFillTimes;
663  }
664  if (entry < 0) entry = 0;
665  }
666  else if (fEntryCurrent >= 0) {
667  // We are still reading from the oldest buffer, no need to prefetch a new one
668  return kFALSE;
669  }
670  if (entry < 0) return kFALSE;
672  }
673  else {
674  // Normal reading with prefetching
675  if (fEnablePrefetching) {
676  if (entry < 0 && fEntryNext > 0) {
677  entry = fEntryCurrent;
678  } else if (entry >= fEntryCurrent) {
679  if (entry < fEntryNext) {
680  entry = fEntryNext;
681  }
682  }
683  else {
684  // We are still reading from the oldest buffer,
685  // no need to prefetch a new one
686  return kFALSE;
687  }
689  }
690  }
691  }
692  }
693 
694  // If the entry is in the range we previously prefetched, there is
695  // no point in retrying. Note that this will also return false
696  // during the training phase (fEntryNext is then set intentional to
697  // the end of the training phase).
698  if (fEntryCurrent <= entry && entry < fEntryNext) return kFALSE;
699 
700  // Triggered by the user, not the learning phase
701  if (entry == -1) entry = 0;
702 
703  fEntryCurrentMax = fEntryCurrent;
704  TTree::TClusterIterator clusterIter = tree->GetClusterIterator(entry);
705  fEntryCurrent = clusterIter();
706  fEntryNext = clusterIter.GetNextEntry();
707  auto firstClusterEnd = fEntryNext;
708 
710  if (fEntryMax <= 0) fEntryMax = tree->GetEntries();
712 
713  if ( fEnablePrefetching ) {
714  if ( entry == fEntryMax ) {
715  // We are at the end, no need to do anything else
716  return kFALSE;
717  }
718  }
719 
720  // Check if owner has a TEventList set. If yes we optimize for this
721  // Special case reading only the baskets containing entries in the
722  // list.
723  TEventList *elist = fTree->GetEventList();
724  Long64_t chainOffset = 0;
725  if (elist) {
726  if (fTree->IsA() ==TChain::Class()) {
727  TChain *chain = (TChain*)fTree;
728  Int_t t = chain->GetTreeNumber();
729  chainOffset = chain->GetTreeOffset()[t];
730  }
731  }
732 
733  //clear cache buffer
734  Int_t fNtotCurrentBuf = 0;
735  if (fEnablePrefetching){ //prefetching mode
736  if (fFirstBuffer) {
738  fNtotCurrentBuf = fNtot;
739  }
740  else {
742  fNtotCurrentBuf = fBNtot;
743  }
744  }
745  else {
747  fNtotCurrentBuf = fNtot;
748  }
749 
750  //store baskets
751  Int_t clusterIterations = 0;
752  Long64_t minEntry = fEntryCurrent;
753  Int_t prevNtot;
754  Int_t minBasket = 0; // We will use this to avoid re-checking the first baskets in the 2nd (or more) run in the while loop.
755  Long64_t maxReadEntry = minEntry; // If we are stopped before the end of the 2nd pass, this marker will where we need to start next time.
756  do {
757  prevNtot = fNtotCurrentBuf;
758  Int_t nextMinBasket = INT_MAX;
759  UInt_t pass = 0;
760  while (pass < 2) {
761  // The first pass we add one basket per branches.
762  // then in the second pass we add the other baskets of the cluster.
763  // This is to support the case where the cache is too small to hold a full cluster.
764  ++pass;
765  for (Int_t i=0;i<fNbranches;i++) {
767  if (b->GetDirectory()==0) continue;
768  if (b->GetDirectory()->GetFile() != fFile) continue;
769  Int_t nb = b->GetMaxBaskets();
770  Int_t *lbaskets = b->GetBasketBytes();
771  Long64_t *entries = b->GetBasketEntry();
772  if (!lbaskets || !entries) continue;
773  //we have found the branch. We now register all its baskets
774  //from the requested offset to the basket below fEntrymax
775  Int_t blistsize = b->GetListOfBaskets()->GetSize();
776  Int_t j=minBasket; // We need this out of the loop so we can find out how far we went.
777  Bool_t firstBasketSeen = kFALSE;
778  for (;j<nb;j++) {
779  // This basket has already been read, skip it
780  if (j<blistsize && b->GetListOfBaskets()->UncheckedAt(j)) continue;
781 
782  Long64_t pos = b->GetBasketSeek(j);
783  Int_t len = lbaskets[j];
784  if (pos <= 0 || len <= 0) continue;
785  if (len > fBufferSizeMin) {
786  // Do not cache a basket if it is bigger than the cache size!
787  continue;
788  }
789  //important: do not try to read fEntryNext, otherwise you jump to the next autoflush
790  if (entries[j] >= fEntryNext) break; // break out of the for each branch loop.
791  if (entries[j] < minEntry && (j<nb-1 && entries[j+1] <= minEntry)) continue;
792  if (elist) {
793  Long64_t emax = fEntryMax;
794  if (j<nb-1) emax = entries[j+1]-1;
795  if (!elist->ContainsRange(entries[j]+chainOffset,emax+chainOffset)) continue;
796  }
797  if (pass==2 && !firstBasketSeen) {
798  // Okay, this has already been requested in the first pass.
799  firstBasketSeen = kTRUE;
800  continue;
801  }
802  fNReadPref++;
803 
804  if ( (fNtotCurrentBuf+len) > fBufferSizeMin ) {
805  // Humm ... we are going to go over the requested size.
806  if (clusterIterations > 0) {
807  // We already have a full cluster and now we would go over the requested
808  // size, let's stop caching (and make sure we start next time from the
809  // end of the previous cluster).
810  if (gDebug > 5) {
811  Info("FillBuffer","Breaking early because %d is greater than %d at cluster iteration %d will restart at %lld",(fNtotCurrentBuf+len), fBufferSizeMin, clusterIterations,minEntry);
812  }
813  fEntryNext = minEntry;
814  break;
815  } else {
816  if (pass == 1) {
817  if ( (fNtotCurrentBuf+len) > 4*fBufferSizeMin ) {
818  // Okay, so we have not even made one pass and we already have
819  // accumulated request for more than twice the memory size ...
820  // So stop for now, and will restart at the same point, hoping
821  // that the basket will still be in memory and not asked again ..
822  fEntryNext = maxReadEntry;
823  if (gDebug > 5) {
824  Info("FillBuffer","Breaking early because %d is greater than 2*%d at cluster iteration %d pass %d will restart at %lld",(fNtotCurrentBuf+len), fBufferSizeMin, clusterIterations,pass,fEntryNext);
825  }
826  break;
827  }
828  } else {
829  // We have made one pass through the branches and thus already
830  // requested one basket per branch, let's stop prefetching
831  // now.
832  if ( (fNtotCurrentBuf+len) > 2*fBufferSizeMin ) {
833  fEntryNext = maxReadEntry;
834  if (gDebug > 5) {
835  Info("FillBuffer","Breaking early because %d is greater than 2*%d at cluster iteration %d pass %d will restart at %lld",(fNtotCurrentBuf+len), fBufferSizeMin, clusterIterations,pass,fEntryNext);
836  }
837  break;
838  }
839  }
840  }
841  }
842  if (fEnablePrefetching){
843  if (fFirstBuffer) {
844  TFileCacheRead::Prefetch(pos,len);
845  fNtotCurrentBuf = fNtot;
846  }
847  else {
849  fNtotCurrentBuf = fBNtot;
850  }
851  }
852  else {
853  TFileCacheRead::Prefetch(pos,len);
854  fNtotCurrentBuf = fNtot;
855  }
856  if ( ( j < (nb-1) ) && entries[j+1] > maxReadEntry ) {
857  maxReadEntry = entries[j+1];
858  }
859  if (fNtotCurrentBuf > 4*fBufferSizeMin) {
860  // Humm something wrong happened.
861  Warning("FillBuffer","There is more data in this cluster (starting at entry %lld to %lld, current=%lld) than usual ... with %d %.3f%% of the branches we already have %d bytes (instead of %d)",
862  fEntryCurrent,fEntryNext, entries[j], i, (100.0*i) / ((float)fNbranches), fNtotCurrentBuf,fBufferSizeMin);
863 
864  }
865  if (pass==1) {
866  // In the first pass, we record one basket per branch and move on to the next branch.
867  break;
868  }
869  }
870 
871  if (j < nextMinBasket) nextMinBasket = j;
872  if (gDebug > 0) printf("Entry: %lld, registering baskets branch %s, fEntryNext=%lld, fNseek=%d, fNtotCurrentBuf=%d\n",minEntry,((TBranch*)fBranches->UncheckedAt(i))->GetName(),fEntryNext,fNseek,fNtotCurrentBuf);
873  }
874  } // loop for the 2 passes.
875  clusterIterations++;
876 
877  minEntry = clusterIter.Next();
878  if (fIsLearning) {
879  fFillTimes++;
880  }
881 
882  // Continue as long as we still make progress (prevNtot < fNtotCurrentBuf), that the next entry range to be looked at,
883  // which start at 'minEntry', is not past the end of the requested range (minEntry < fEntryMax)
884  // and we guess that we not going to go over the requested amount of memory by asking for another set
885  // of entries (fBufferSizeMin > ((Long64_t)fNtotCurrentBuf*(clusterIterations+1))/clusterIterations).
886  // fNtotCurrentBuf / clusterIterations is the average size we are accumulated so far at each loop.
887  // and thus (fNtotCurrentBuf / clusterIterations) * (clusterIterations+1) is a good guess at what the next total size
888  // would be if we run the loop one more time. fNtotCurrentBuf and clusterIterations are Int_t but can sometimes
889  // be 'large' (i.e. 30Mb * 300 intervals) and can overflow the numercial limit of Int_t (i.e. become
890  // artificially negative). To avoid this issue we promote fNtotCurrentBuf to a long long (64 bits rather than 32 bits)
891  if (!((fBufferSizeMin > ((Long64_t)fNtotCurrentBuf*(clusterIterations+1))/clusterIterations) && (prevNtot < fNtotCurrentBuf) && (minEntry < fEntryMax)))
892  break;
893 
894  //for the reverse reading case
895  if (!fIsLearning && fReverseRead){
896  if (clusterIterations >= fFillTimes)
897  break;
898  if (minEntry >= fEntryCurrentMax && fEntryCurrentMax >0)
899  break;
900  }
901  minBasket = nextMinBasket;
902  fEntryNext = clusterIter.GetNextEntry();
904  } while (kTRUE);
905 
906  if (fEntryCurrent > entry || entry >= fEntryNext) {
907  // Something went very wrong and even-though we searched for the baskets
908  // holding 'entry' we somehow ended up with a range of entries that does
909  // validate. So we must have been unable to find or fit the needed basket.
910  // And thus even-though, we know the corresponding baskets wont be in the cache,
911  // Let's make it official that 'entry' is within the range of this TTreeCache ('s search.)
912 
913  // Without this, the next read will be flagged as 'out-of-range' and then we start at
914  // the exact same point as this FillBuffer execution resulting in both the requested
915  // entry still not being part of the cache **and** the beginning of the cluster being
916  // read **again**.
917 
918  fEntryNext = firstClusterEnd;
919  }
920 
921  if (fEnablePrefetching) {
922  if (fIsLearning) {
924  }
925  if (!fIsLearning && fFirstTime){
926  // First time we add autoFlush entries , after fFillTimes * autoFlush
927  // only in reverse prefetching mode
928  fFirstTime = kFALSE;
929  }
930  }
932  return kTRUE;
933 }
934 
935 ////////////////////////////////////////////////////////////////////////////////
936 /// Return the desired prefill type from the environment or resource variable
937 /// - 0 - No prefill
938 /// - 1 - All branches
939 
941 {
942  const char *stcp;
943  Int_t s = 0;
944 
945  if (!(stcp = gSystem->Getenv("ROOT_TTREECACHE_PREFILL")) || !*stcp) {
946  s = gEnv->GetValue("TTreeCache.Prefill", 1);
947  } else {
948  s = TString(stcp).Atoi();
949  }
950 
951  return static_cast<TTreeCache::EPrefillType>(s);
952 }
953 
954 ////////////////////////////////////////////////////////////////////////////////
955 /// Give the total efficiency of the cache... defined as the ratio
956 /// of blocks found in the cache vs. the number of blocks prefetched
957 /// ( it could be more than 1 if we read the same block from the cache more
958 /// than once )
959 ///
960 /// Note: This should eb used at the end of the processing or we will
961 /// get uncomplete stats
962 
964 {
965  if ( !fNReadPref )
966  return 0;
967 
968  return ((Double_t)fNReadOk / (Double_t)fNReadPref);
969 }
970 
971 ////////////////////////////////////////////////////////////////////////////////
972 /// This will indicate a sort of relative efficiency... a ratio of the
973 /// reads found in the cache to the number of reads so far
974 
976 {
977  if ( !fNReadOk && !fNReadMiss )
978  return 0;
979 
980  return ((Double_t)fNReadOk / (Double_t)(fNReadOk + fNReadMiss));
981 }
982 
983 ////////////////////////////////////////////////////////////////////////////////
984 /// Static function returning the number of entries used to train the cache
985 /// see SetLearnEntries
986 
988 {
989  return fgLearnEntries;
990 }
991 
992 ////////////////////////////////////////////////////////////////////////////////
993 /// Print cache statistics. Like:
994 ///
995 /// ~~~ {.cpp}
996 /// ******TreeCache statistics for file: cms2.root ******
997 /// Number of branches in the cache ...: 1093
998 /// Cache Efficiency ..................: 0.997372
999 /// Cache Efficiency Rel...............: 1.000000
1000 /// Learn entries......................: 100
1001 /// Reading............................: 72761843 bytes in 7 transactions
1002 /// Readahead..........................: 256000 bytes with overhead = 0 bytes
1003 /// Average transaction................: 10394.549000 Kbytes
1004 /// Number of blocks in current cache..: 210, total size: 6280352
1005 /// ~~~
1006 ///
1007 /// - if option = "a" the list of blocks in the cache is printed
1008 /// see also class TTreePerfStats.
1009 /// - if option contains 'cachedbranches', the list of branches being
1010 /// cached is printed.
1011 
1012 void TTreeCache::Print(Option_t *option) const
1013 {
1014  TString opt = option;
1015  opt.ToLower();
1016  printf("******TreeCache statistics for tree: %s in file: %s ******\n",fTree ? fTree->GetName() : "no tree set",fFile ? fFile->GetName() : "no file set");
1017  if (fNbranches <= 0) return;
1018  printf("Number of branches in the cache ...: %d\n",fNbranches);
1019  printf("Cache Efficiency ..................: %f\n",GetEfficiency());
1020  printf("Cache Efficiency Rel...............: %f\n",GetEfficiencyRel());
1021  printf("Learn entries......................: %d\n",TTreeCache::GetLearnEntries());
1022  if ( opt.Contains("cachedbranches") ) {
1023  opt.ReplaceAll("cachedbranches","");
1024  printf("Cached branches....................:\n");
1025  const TObjArray *cachedBranches = this->GetCachedBranches();
1026  Int_t nbranches = cachedBranches->GetEntriesFast();
1027  for (Int_t i = 0; i < nbranches; ++i) {
1028  TBranch* branch = (TBranch*) cachedBranches->UncheckedAt(i);
1029  printf("Branch name........................: %s\n",branch->GetName());
1030  }
1031  }
1032  TFileCacheRead::Print(opt);
1033 }
1034 
1035 ////////////////////////////////////////////////////////////////////////////////
1036 /// Old method ReadBuffer before the addition of the prefetch mechanism.
1037 
1039  //Is request already in the cache?
1040  if (TFileCacheRead::ReadBuffer(buf,pos,len) == 1){
1041  fNReadOk++;
1042  return 1;
1043  }
1044 
1045  //not found in cache. Do we need to fill the cache?
1046  Bool_t bufferFilled = FillBuffer();
1047  if (bufferFilled) {
1048  Int_t res = TFileCacheRead::ReadBuffer(buf,pos,len);
1049 
1050  if (res == 1)
1051  fNReadOk++;
1052  else if (res == 0)
1053  fNReadMiss++;
1054 
1055  return res;
1056  }
1057  fNReadMiss++;
1058 
1059  return 0;
1060 }
1061 
1062 ////////////////////////////////////////////////////////////////////////////////
1063 /// Used to read a chunk from a block previously fetched. It will call FillBuffer
1064 /// even if the cache lookup succeeds, because it will try to prefetch the next block
1065 /// as soon as we start reading from the current block.
1066 
1068  if (TFileCacheRead::ReadBuffer(buf, pos, len) == 1){
1069  //call FillBuffer to prefetch next block if necessary
1070  //(if we are currently reading from the last block available)
1071  FillBuffer();
1072  fNReadOk++;
1073  return 1;
1074  }
1075 
1076  //keep on prefetching until request is satisfied
1077  // try to prefetch a couple of times and if request is still not satisfied then
1078  // fall back to normal reading without prefetching for the current request
1079  Int_t counter = 0;
1080  while (1) {
1081  if(TFileCacheRead::ReadBuffer(buf, pos, len)) {
1082  break;
1083  }
1084  FillBuffer();
1085  fNReadMiss++;
1086  counter++;
1087  if (counter>1) {
1088  return 0;
1089  }
1090  }
1091 
1092  fNReadOk++;
1093  return 1;
1094 }
1095 
1096 ////////////////////////////////////////////////////////////////////////////////
1097 /// Read buffer at position pos if the request is in the list of
1098 /// prefetched blocks read from fBuffer.
1099 /// Otherwise try to fill the cache from the list of selected branches,
1100 /// and recheck if pos is now in the list.
1101 /// Returns:
1102 /// - -1 in case of read failure,
1103 /// - 0 in case not in cache,
1104 /// - 1 in case read from cache.
1105 /// This function overloads TFileCacheRead::ReadBuffer.
1106 
1108 {
1109  if (!fEnabled) return 0;
1110 
1111  if (fEnablePrefetching)
1112  return TTreeCache::ReadBufferPrefetch(buf, pos, len);
1113  else
1114  return TTreeCache::ReadBufferNormal(buf, pos, len);
1115 }
1116 
1117 ////////////////////////////////////////////////////////////////////////////////
1118 /// This will simply clear the cache
1119 
1121 {
1123 
1124  if (fEnablePrefetching) {
1125  fFirstTime = kTRUE;
1127  }
1128 }
1129 
1130 ////////////////////////////////////////////////////////////////////////////////
1131 /// Change the underlying buffer size of the cache.
1132 /// If the change of size means some cache content is lost, or if the buffer
1133 /// is now larger, setup for a cache refill the next time there is a read
1134 /// Returns:
1135 /// - 0 if the buffer content is still available
1136 /// - 1 if some or all of the buffer content has been made unavailable
1137 /// - -1 on error
1138 
1140 {
1141  Int_t prevsize = GetBufferSize();
1142  Int_t res = TFileCacheRead::SetBufferSize(buffersize);
1143  if (res < 0) {
1144  return res;
1145  }
1146 
1147  if (res == 0 && buffersize <= prevsize) {
1148  return res;
1149  }
1150 
1151  // if content was removed from the buffer, or the buffer was enlarged then
1152  // empty the prefetch lists and prime to fill the cache again
1153 
1155  if (fEnablePrefetching) {
1157  }
1158 
1159  fEntryCurrent = -1;
1160  if (!fIsLearning) {
1161  fEntryNext = -1;
1162  }
1163 
1164  return 1;
1165 }
1166 
1167 ////////////////////////////////////////////////////////////////////////////////
1168 /// Set the minimum and maximum entry number to be processed
1169 /// this information helps to optimize the number of baskets to read
1170 /// when prefetching the branch buffers.
1171 
1173 {
1174  // This is called by TTreePlayer::Process in an automatic way...
1175  // don't restart it if the user has specified the branches.
1176  Bool_t needLearningStart = (fEntryMin != emin) && fIsLearning && !fIsManual;
1177 
1178  fEntryMin = emin;
1179  fEntryMax = emax;
1181  if (gDebug > 0)
1182  Info("SetEntryRange", "fEntryMin=%lld, fEntryMax=%lld, fEntryNext=%lld",
1184 
1185  if (needLearningStart) {
1186  // Restart learning
1188  }
1189 }
1190 
1191 ////////////////////////////////////////////////////////////////////////////////
1192 /// Overload to make sure that the object specific
1193 
1195 {
1196  // The infinite recursion is 'broken' by the fact that
1197  // TFile::SetCacheRead remove the entry from fCacheReadMap _before_
1198  // calling SetFile (and also by setting fFile to zero before the calling).
1199  if (fFile) {
1200  TFile *prevFile = fFile;
1201  fFile = 0;
1202  prevFile->SetCacheRead(0, fTree, action);
1203  }
1204  TFileCacheRead::SetFile(file, action);
1205 }
1206 
1207 ////////////////////////////////////////////////////////////////////////////////
1208 /// Static function to set the number of entries to be used in learning mode
1209 /// The default value for n is 10. n must be >= 1
1210 
1212 {
1213  if (n < 1) n = 1;
1214  fgLearnEntries = n;
1215 }
1216 
1217 ////////////////////////////////////////////////////////////////////////////////
1218 /// Set whether the learning period is started with a prefilling of the
1219 /// cache and which type of prefilling is used.
1220 /// The two value currently supported are:
1221 /// - TTreeCache::kNoPrefill disable the prefilling
1222 /// - TTreeCache::kAllBranches fill the cache with baskets from all branches.
1223 /// The default prefilling behavior can be controlled by setting
1224 /// TTreeCache.Prefill or the environment variable ROOT_TTREECACHE_PREFILL.
1225 
1227 {
1228  fPrefillType = type;
1229 }
1230 
1231 ////////////////////////////////////////////////////////////////////////////////
1232 /// The name should be enough to explain the method.
1233 /// The only additional comments is that the cache is cleaned before
1234 /// the new learning phase.
1235 
1237 {
1238  fIsLearning = kTRUE;
1239  fIsManual = kFALSE;
1240  fNbranches = 0;
1241  if (fBrNames) fBrNames->Delete();
1243  fEntryCurrent = -1;
1244 }
1245 
1246 ////////////////////////////////////////////////////////////////////////////////
1247 /// This is the counterpart of StartLearningPhase() and can be used to stop
1248 /// the learning phase. It's useful when the user knows exactly what branches
1249 /// they are going to use.
1250 /// For the moment it's just a call to FillBuffer() since that method
1251 /// will create the buffer lists from the specified branches.
1252 
1254 {
1255  if (fIsLearning) {
1256  // This will force FillBuffer to read the buffers.
1257  fEntryNext = -1;
1258  fIsLearning = kFALSE;
1259  }
1260  fIsManual = kTRUE;
1261 
1262  //fill the buffers only once during learning
1263  if (fEnablePrefetching && !fOneTime) {
1264  fIsLearning = kTRUE;
1265  FillBuffer();
1266  fOneTime = kTRUE;
1267  }
1268 }
1269 
1270 ////////////////////////////////////////////////////////////////////////////////
1271 /// Update pointer to current Tree and recompute pointers to the branches in the cache.
1272 
1274 {
1275 
1276  fTree = tree;
1277 
1278  fEntryMin = 0;
1279  fEntryMax = fTree->GetEntries();
1280 
1281  fEntryCurrent = -1;
1282 
1283  if (fBrNames->GetEntries() == 0 && fIsLearning) {
1284  // We still need to learn.
1286  } else {
1287  // We learnt from a previous file.
1288  fIsLearning = kFALSE;
1289  fEntryNext = -1;
1290  }
1291  fNbranches = 0;
1292 
1293  TIter next(fBrNames);
1294  TObjString *os;
1295  while ((os = (TObjString*)next())) {
1296  TBranch *b = fTree->GetBranch(os->GetName());
1297  if (!b) {
1298  continue;
1299  }
1300  fBranches->AddAt(b, fNbranches);
1301  fNbranches++;
1302  }
1303 }
1304 
1305 ////////////////////////////////////////////////////////////////////////////////
1306 /// Perform an initial prefetch, attempting to read as much of the learning
1307 /// phase baskets for all branches at once
1308 
1310 {
1311  // This is meant for the learning phase
1312  if (!fIsLearning) return;
1313 
1314  // This should be called before reading entries, otherwise we'll
1315  // always exit here, since TBranch adds itself before reading
1316  if (fNbranches > 0) return;
1317 
1318  // Is the LearnPrefill enabled (using an Int_t here to allow for future
1319  // extension to alternative Prefilling).
1320  if (fPrefillType == kNoPrefill) return;
1321 
1322  // Force only the learn entries to be cached by temporarily setting min/max
1323  // to the learning phase entry range
1324  // But save all the old values, so we can restore everything to how it was
1325  Long64_t eminOld = fEntryMin;
1326  Long64_t emaxOld = fEntryMax;
1327  Long64_t ecurrentOld = fEntryCurrent;
1328  Long64_t enextOld = fEntryNext;
1329 
1332 
1333  // Add all branches to be cached. This also sets fIsManual, stops learning,
1334  // and makes fEntryNext = -1 (which forces a cache fill, which is good)
1335  AddBranch("*");
1336  fIsManual = kFALSE; // AddBranch sets fIsManual, so we reset it
1337 
1338  // Now, fill the buffer with the learning phase entry range
1339  FillBuffer();
1340 
1341  // Leave everything the way we found it
1342  fIsLearning = kTRUE;
1343  DropBranch("*"); // This doesn't work unless we're already learning
1344 
1345  // Restore entry values
1346  fEntryMin = eminOld;
1347  fEntryMax = emaxOld;
1348  fEntryCurrent = ecurrentOld;
1349  fEntryNext = enextOld;
1350 }
virtual const char * GetName() const
Returns name of object.
Definition: TNamed.h:51
Int_t fNtot
Total size of prefetched blocks.
A TLeaf describes individual elements of a TBranch See TBranch structure in TTree.
Definition: TLeaf.h:37
Long64_t fEntryMax
! last entry in the cache
Definition: TTreeCache.h:41
An array of TObjects.
Definition: TObjArray.h:39
Long64_t * GetBasketEntry() const
Definition: TBranch.h:150
Long64_t GetNextEntry()
Definition: TTree.h:289
virtual void Delete(Option_t *option="")
Remove all objects from the list AND delete all heap based objects.
Definition: TList.cxx:405
Int_t fNReadOk
Number of blocks read and found in the cache.
Definition: TTreeCache.h:45
virtual void Info(const char *method, const char *msgfmt,...) const
Issue info message.
Definition: TObject.cxx:899
TFile * fFile
Pointer to file.
long long Long64_t
Definition: RtypesCore.h:69
const TObjArray * GetCachedBranches() const
Definition: TTreeCache.h:80
TObjArray * GetListOfBaskets()
Definition: TBranch.h:177
Collectable string class.
Definition: TObjString.h:32
TObjArray * fBranches
! List of branches to be stored in the cache
Definition: TTreeCache.h:48
A cache when reading files over the network.
const char Option_t
Definition: RtypesCore.h:62
Bool_t fOneTime
! used in the learning phase
Definition: TTreeCache.h:54
virtual void SetCacheRead(TFileCacheRead *cache, TObject *tree=0, ECacheAction action=kDisconnect)
Set a pointer to the read cache.
Definition: TFile.cxx:2197
TString & ReplaceAll(const TString &s1, const TString &s2)
Definition: TString.h:635
virtual void SetLearnPrefill(EPrefillType type=kNoPrefill)
Set whether the learning period is started with a prefilling of the cache and which type of prefillin...
A specialized TFileCacheRead object for a TTree.
Definition: TTreeCache.h:34
A ROOT file is a suite of consecutive data records (TKey instances) with a well defined format...
Definition: TFile.h:50
virtual Int_t GetEntries() const
Definition: TCollection.h:92
virtual TList * GetListOfFriends() const
Definition: TTree.h:419
virtual Long64_t GetAutoFlush() const
Definition: TTree.h:378
Regular expression class.
Definition: TRegexp.h:35
TDirectory * GetDirectory() const
Definition: TBranch.h:158
virtual TObject * Remove(TObject *obj)
Remove object from array.
Definition: TObjArray.cxx:653
Ssiz_t Index(const char *pat, Ssiz_t i=0, ECaseCompare cmp=kExact) const
Definition: TString.h:582
Long64_t fEntryMin
! first entry in the cache
Definition: TTreeCache.h:40
Basic string class.
Definition: TString.h:137
void ToLower()
Change string to lower-case.
Definition: TString.cxx:1089
virtual void Prefetch(Long64_t pos, Int_t len)
Add block of length len at position pos in the list of blocks to be prefetched.
int Int_t
Definition: RtypesCore.h:41
bool Bool_t
Definition: RtypesCore.h:59
const Bool_t kFALSE
Definition: Rtypes.h:92
virtual void StopLearningPhase()
This is the counterpart of StartLearningPhase() and can be used to stop the learning phase...
const char * Class
Definition: TXMLSetup.cxx:64
virtual void LearnPrefill()
Perform an initial prefetch, attempting to read as much of the learning phase baskets for all branche...
void StartLearningPhase()
The name should be enough to explain the method.
virtual Int_t AddBranch(TBranch *b, Bool_t subgbranches=kFALSE)
Add a branch to the list of branches to be stored in the cache this function is called by TBranch::Ge...
Definition: TTreeCache.cxx:332
Long64_t * GetTreeOffset() const
Definition: TChain.h:119
virtual TObject * FindObject(const char *name) const
Find an object in this list using its name.
Definition: TList.cxx:497
virtual Int_t ReadBufferNormal(char *buf, Long64_t pos, Int_t len)
Old method ReadBuffer before the addition of the prefetch mechanism.
Helper class to iterate over cluster of baskets.
Definition: TTree.h:256
Int_t * GetBasketBytes() const
Definition: TBranch.h:149
virtual void SecondPrefetch(Long64_t, Int_t)
virtual void Print(Option_t *option="") const
Print cache statistics.
virtual Long64_t GetReadEntry() const
Definition: TTree.h:436
Bool_t fIsTransferred
True when fBuffer contains something valid.
virtual Bool_t ContainsRange(Long64_t entrymin, Long64_t entrymax)
Return TRUE if list contains entries from entrymin to entrymax included.
Definition: TEventList.cxx:171
virtual const char * Getenv(const char *env)
Get environment variable.
Definition: TSystem.cxx:1628
virtual TClusterIterator GetClusterIterator(Long64_t firstentry)
Return an iterator over the cluster of baskets starting at firstentry.
Definition: TTree.cxx:5042
virtual Int_t ReadBuffer(char *buf, Long64_t pos, Int_t len)
Read buffer at position pos if the request is in the list of prefetched blocks read from fBuffer...
virtual void ResetCache()
This will simply clear the cache.
Bool_t fEnabled
! cache enabled for cached reading
Definition: TTreeCache.h:60
TObjArray * GetListOfBranches()
Definition: TBranch.h:178
virtual TFile * GetFile() const
Definition: TDirectory.h:155
Double_t GetEfficiencyRel() const
This will indicate a sort of relative efficiency...
Definition: TTreeCache.cxx:975
virtual TTree * GetTree() const
Definition: TTree.h:444
A doubly linked list.
Definition: TList.h:47
virtual TBranch * GetBranch(const char *name)
Return pointer to the branch with the given name in this tree or its friends.
Definition: TTree.cxx:4869
TTree * fTree
! pointer to the current Tree
Definition: TTreeCache.h:50
const char * GetName() const
Returns name of object.
Definition: TObjString.h:42
EPrefillType GetConfiguredPrefillType() const
Return the desired prefill type from the environment or resource variable.
Definition: TTreeCache.cxx:940
virtual void SetEntryRange(Long64_t emin, Long64_t emax)
Set the minimum and maximum entry number to be processed this information helps to optimize the numbe...
virtual void AddAtAndExpand(TObject *obj, Int_t idx)
Add object at position idx.
Definition: TObjArray.cxx:222
virtual Int_t DropBranch(TBranch *b, Bool_t subbranches=kFALSE)
Remove a branch to the list of branches to be stored in the cache this function is called by TBranch:...
Definition: TTreeCache.cxx:484
R__EXTERN TSystem * gSystem
Definition: TSystem.h:549
TTree * GetTree() const
Definition: TTreeCache.h:88
virtual Int_t GetValue(const char *name, Int_t dflt)
Returns the integer value for a resource.
Definition: TEnv.cxx:496
virtual TObject * Remove(TObject *obj)
Remove object from the list.
Definition: TList.cxx:675
virtual Int_t ReadBufferPrefetch(char *buf, Long64_t pos, Int_t len)
Used to read a chunk from a block previously fetched.
virtual ~TTreeCache()
Destructor. (in general called by the TFile destructor)
Definition: TTreeCache.cxx:315
void Form(const char *fmt,...)
Formats a string using a printf style format descriptor.
Definition: TString.cxx:2322
unsigned int UInt_t
Definition: RtypesCore.h:42
Int_t GetEntriesFast() const
Definition: TObjArray.h:66
virtual void Error(const char *method, const char *msgfmt,...) const
Issue error message.
Definition: TObject.cxx:925
virtual TLeaf * GetLeafCount() const
Definition: TLeaf.h:71
virtual void SetFile(TFile *file, TFile::ECacheAction action=TFile::kDisconnect)
Set the file using this cache and reset the current blocks (if any).
Int_t GetMaxBaskets() const
Definition: TBranch.h:180
A TEventList object is a list of selected events (entries) in a TTree.
Definition: TEventList.h:33
Bool_t fReadDirectionSet
! read direction established
Definition: TTreeCache.h:59
Int_t fNReadMiss
Number of blocks read and not found in the cache.
Definition: TTreeCache.h:46
static void SetLearnEntries(Int_t n=10)
Static function to set the number of entries to be used in learning mode The default value for n is 1...
virtual Int_t SetBufferSize(Int_t buffersize)
Sets the buffer size.
virtual Bool_t FillBuffer()
Fill the cache buffer with the branches in the cache.
Definition: TTreeCache.cxx:619
virtual void AddAt(TObject *obj, Int_t idx)
Add object at position ids.
Definition: TObjArray.cxx:239
Int_t fNReadPref
Number of blocks that were prefetched.
Definition: TTreeCache.h:47
virtual Long64_t GetBasketSeek(Int_t basket) const
Return address of basket in the file.
Definition: TBranch.cxx:1160
Bool_t fReverseRead
! reading in reverse mode
Definition: TTreeCache.h:55
Bool_t fFirstTime
! save the fact that we processes the first entry
Definition: TTreeCache.h:57
Long64_t fEntryCurrent
! current lowest entry number in the cache
Definition: TTreeCache.h:42
Int_t fBufferSizeMin
Original size of fBuffer.
Long64_t Next()
Move on to the next cluster and return the starting entry of this next cluster.
Definition: TTree.cxx:599
TObject * UncheckedAt(Int_t i) const
Definition: TObjArray.h:91
TTreeCache()
Default Constructor.
Definition: TTreeCache.cxx:252
virtual void Print(Option_t *option="") const
Print cache statistics.
#define ClassImp(name)
Definition: Rtypes.h:279
double Double_t
Definition: RtypesCore.h:55
Bool_t fIsLearning
! true if cache is in learning mode
Definition: TTreeCache.h:51
virtual Int_t GetTreeNumber() const
Definition: TChain.h:118
int type
Definition: TGX11.cxx:120
R__EXTERN TEnv * gEnv
Definition: TEnv.h:174
virtual Int_t GetBufferSize() const
Bool_t Contains(const char *pat, ECaseCompare cmp=kExact) const
Definition: TString.h:567
virtual void SetFile(TFile *file, TFile::ECacheAction action=TFile::kDisconnect)
Overload to make sure that the object specific.
EPrefillType fPrefillType
Whether a pre-filling is enabled (and if applicable which type)
Definition: TTreeCache.h:61
Int_t fFillTimes
! how many times we can fill the current buffer
Definition: TTreeCache.h:56
virtual Int_t SetBufferSize(Int_t buffersize)
Change the underlying buffer size of the cache.
virtual Long64_t GetEntries() const
Definition: TTree.h:393
Int_t fNbranches
! Number of branches in the cache
Definition: TTreeCache.h:44
Bool_t fEnablePrefetching
reading by prefetching asynchronously
virtual void Add(TObject *obj)
Definition: TList.h:81
const Ssiz_t kNPOS
Definition: Rtypes.h:115
Long64_t fFirstEntry
! save the value of the first entry
Definition: TTreeCache.h:58
Definition: file.py:1
A TFriendElement TF describes a TTree object TF in a file.
A chain is a collection of files containg TTree objects.
Definition: TChain.h:35
TList * fBrNames
! list of branch names in the cache
Definition: TTreeCache.h:49
Long64_t GetReadEntry() const
Definition: TBranch.h:170
you should not use this method at all Int_t Int_t Double_t Double_t Double_t Int_t Double_t Double_t Double_t Double_t b
Definition: TRolke.cxx:630
Bool_t fFirstBuffer
! true if first buffer is used for prefetching
Definition: TTreeCache.h:53
static Int_t fgLearnEntries
number of entries used for learning mode
Definition: TTreeCache.h:62
TTree * GetTree() const
Definition: TBranch.h:184
Int_t GetEntries() const
Return the number of objects in array (i.e.
Definition: TObjArray.cxx:494
R__EXTERN Int_t gDebug
Definition: Rtypes.h:128
Int_t Atoi() const
Return integer value of string.
Definition: TString.cxx:1965
virtual Int_t ReadBuffer(char *buf, Long64_t pos, Int_t len)
Read buffer at position pos.
Definition: tree.py:1
A TTree object has a header with a name and a title.
Definition: TTree.h:98
TEventList * GetEventList() const
Definition: TTree.h:403
TBranch * GetBranch() const
Definition: TLeaf.h:70
Bool_t fIsManual
! true if cache is StopLearningPhase was used
Definition: TTreeCache.h:52
virtual const char * GetName() const
Returns name of object.
Definition: TObject.cxx:416
virtual Int_t GetSize() const
Definition: TCollection.h:95
A TTree is a list of TBranches.
Definition: TBranch.h:58
Long64_t fEntryNext
! next entry number where cache must be filled
Definition: TTreeCache.h:43
const Bool_t kTRUE
Definition: Rtypes.h:91
virtual void UpdateBranches(TTree *tree)
Update pointer to current Tree and recompute pointers to the branches in the cache.
const Int_t n
Definition: legend1.C:16
Double_t GetEfficiency() const
Give the total efficiency of the cache...
Definition: TTreeCache.cxx:963
char name[80]
Definition: TGX11.cxx:109
static Int_t GetLearnEntries()
Static function returning the number of entries used to train the cache see SetLearnEntries.
Definition: TTreeCache.cxx:987
virtual void Warning(const char *method, const char *msgfmt,...) const
Issue warning message.
Definition: TObject.cxx:911
Int_t fNseek
Number of blocks to be prefetched.
virtual TObjArray * GetListOfLeaves()
Definition: TTree.h:418
ECacheAction
TTreeCache flushing semantics.
Definition: TFile.h:62