Log of /trunk/proof/proofplayer/inc/TEventIter.h
Parent Directory
Revision
45283 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 30 10:42:38 2012 UTC (2 years, 5 months ago) by
ganis
File length: 6994 byte(s)
Diff to
previous 44652
Patch automatizing the usage of the file-based technology to handle outputs.
After this patch PROOF users can define as an option to Process an output file for their results
and choose if the merging goes via file or in memory. User can also decide to save theit results
in a set of files identified by a name (dataset).
Administrators can force merging-via-file or can make it conditional to the resident memory exceeding
a given threshold.
The new options are documented at
http://root.cern.ch/drupal/content/handling-outputs
Since TProofOutputFile is now used by TProof, the class has been moved from proof/proofplayer
(libProofPlayer) to proof/proof (libProof).
The selector tutorials/proof/ProofSimple.h,.C has been modified to allo testing the new options,
and a new test has been added to stressProof.
Revision
44652 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 11 16:16:14 2012 UTC (2 years, 7 months ago) by
ganis
File length: 6867 byte(s)
Diff to
previous 44010
- In TEventInterTree::Load, make sure that the name of the tree to load is taken
from the TDSet object name, unless specified explicitely. Allows to handle
correctly multiple tree in the same dataset in all cases.
- In stressProof, add a test to process different trees in the same files using
three different descriptions of the dataset: by-name, TDSet, TChain .
For this test, tutorials/proof/ProofNtuple.h,.C has been modified to create a
second ntuple in the same file .
Revision
44010 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Apr 29 23:07:48 2012 UTC (2 years, 8 months ago) by
ganis
File length: 6842 byte(s)
Diff to
previous 38810
Several optimizations in the low level PROOF event loop (TProofPlayer::Process):
- Per-event calls to TEventIter::GetNextEvent are replaced by per-packet calls
to a new function TEventIter::GetNextPacket and a for loop over the returned
range;
- Progress information is filled in on per-packet pace just before asking for
a new packet, instead of being filled for each event;
- In TProofProgressStatus, the relevant setters in were inlined, and the (expensive)
calls to SetLastUpdate() reduced to the minimum required;
- Status of objects is checked using bits instead of functions calls;
- Check whether to dump memory info is done before calling CheckMemUsage.
These optimizations allow to reduce dramatically the overhead introduced by the
operations PROOF needs to perform during the event loop. A measurement of the
overhead can be obtained from a very light computational task, for example,
generating one random number and filling one histogram; executing this task
within a PROOF-Lite session with 1 worker now takes only 1.8 times the time
required by a straight loop in the parent ROOT session; the same number before
was about 13.
The patch also introduces a couple of small changes enabling the use of
TProofPlayerLocal to process selectors in the local session using the PROOF
iteration mechanism; this is very useful for optimizations studies.
Finally, a couple of missing protections were added in TProofPlayer.
Revision
38810 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Apr 12 16:22:59 2011 UTC (3 years, 9 months ago) by
ganis
File length: 6056 byte(s)
Diff to
previous 37987
Patch to correctly honour selector abort status settings in PROOF. Currently only
the TSelector::kAbortProcess was handled by stopping processing. In particular
TSelector::kAbortFile was ignored; this recently created some problems in ALICE
with corrupted files, with repeated attempts to read events eventually leading to
bad_alloc exceptions.
This patch also fixes other related issues, in particular with the reporting of the
non-processed {files, events} in the final 'MissingFiles' list. This list should
now account much more precisely of the number of events which could not be processed.
It also fixes a problem with the final update of the progress information affecting
occasionally cases with skipped events.
Revision
37987 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 7 07:10:15 2011 UTC (3 years, 11 months ago) by
ganis
File length: 6015 byte(s)
Diff to
previous 30859
Add switch to control caching of the files read on MacOsX.
A call to fcntl(fd, F_NOCACHE, 1) is done after opening the file.
To disable OS file caching the parameter PROOF_DontCacheFiles - or the ROOT-rc
variable ProofPlayer.DontCacheFiles - must be set to 1.
By default caching is enabled.
Revision
30859 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Oct 24 14:53:07 2009 UTC (5 years, 3 months ago) by
ganis
File length: 5931 byte(s)
Diff to
previous 29126
Patch for improved performance monitoring. The 'Rate Plot' button in the progress
dialog is renamed 'Performance plot' and shows up to 4 plots with the event/sec,
the average read chunck size, the number of active workers and the number of active
PROOF sessions on the cluster, all as a function of processing time.
The read chunck size plot allows to monitor the usage of the cache.
The istantaneous processing rate (event/sec) is now better estimate: a few issues
with the normalizing times have ben solved, removing the artificial structures that
were observed.
The possibility to set a max packet time length is introduced (default 30 s); this
can be changes with the parameter PROOF_MaxPacketTime.
The size of the cache is also taken into account to optimize the use of the cache.
The parameter PROOF_UseParallelUnzip has been introduced to toggle the use of the
parallel unzip (default off for now).
A page describing the new performance plots is under preparation at
http://root.cern.ch/drupal/content/progress-dialog
Revision
29126 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 22 09:05:46 2009 UTC (5 years, 7 months ago) by
brun
File length: 5477 byte(s)
Diff to
previous 25733
From Gerri:
Initialize the tree cache only for remote files; solves a long standing crash experienced by ALICE
when mixtures of local and remote files were assigned to workers
Revision
20862 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Nov 18 14:51:42 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proofplayer/inc/TEventIter.h
File length: 5147 byte(s)
Diff to
previous 19826
Synchronize with braches/dev/proof r20835
Summary:
TProofServ:
- read session.rootrc with level kEnvChange to be able to change existing
settings
- Avoid deleting a query result twice in some special cases
- Fix a problem with the initialization of fKeptQueries
rootrc.in:
- restore default settings the asynchronous reading
TFileCacheRead:
- add missing protection in ReadBuffer
TEventIter:
- Enable the usage of TTreeCache
XrdProofdProtocol:
- Additional check on the ownership of the unix socket
- Improve notification during Reset
- Reduce default timeout on admin requests and make it configurable
XrdProofConn:
- Use the configurable maxtry everywhere where relevant
TPacketizerAdaptive:
- Store info on all the processed packets in per-worker lists
TProofMgr, TXProofMgr, TXSocket, XProofProtocol, XrdProofdProtocol:
- Add possibility for the admin to broadcast a message to the connected users
getProof:
- Add missing protection
Revision
11281 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Mar 10 17:57:05 2005 UTC (9 years, 10 months ago) by
rdm
Original Path:
trunk/proofplayer/inc/TEventIter.h
File length: 4032 byte(s)
Diff to
previous 9182
From Marek Biskup:
major new PROOF developments:
- access to PROOF directly via the TChain interface (creating a TDSet
in the background).
- access to PROOF based trees via the TTreeViewer
- fully implemented the TChain/TDSet::Draw() interface.
- PROOF sessions can now be browsed via the TBrowser
Revision
9182 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Jun 13 16:26:36 2004 UTC (10 years, 7 months ago) by
rdm
Original Path:
trunk/proofplayer/inc/TEventIter.h
File length: 3882 byte(s)
Diff to
previous 6519
From Maarten and Kristjan Gulbrandsen:
PROOF and file I/O performance monitoring framework. The TVirtualPerfStats
abstract class allows adding monitoring events in base classes (TFile and
TNetfile). The performance monitoring info allows detailed study of PROOF
performance (which files accessed by which slaves, latency, read/write times,
packet size, etc.). The result is send back to the client in the form of a
memory based TTree.
Remove TProofStats.h and TProofStats.cxx, .d, .o from proof/inc and src.
Revision
6421 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Apr 11 11:48:11 2003 UTC (11 years, 9 months ago) by
rdm
Original Path:
trunk/proofplayer/inc/TEventIter.h
File length: 3656 byte(s)
Diff to
previous 4385
Big (unsigned) long long support patch. Following provided:
- basic typedefs in Rtypes.h: Long64_t and ULong64_t
- basic Long64 I/O support in TBuffer
- automatic Long64 I/O support in TStreamerInfo
- Long64 byteswap in Bytes.h
- Long64 type handling in classes like TDataMember, TDataType, TCint,
TROOT, etc
- Removal of obsolete Long64_t typedefs in many PROOF classes
No changes for non-Long64 data types (no backward incompatibilies).
I/O tested for Long64 basic type, and static and dynamic arrays using
handcoded streamers, rootcint generated streamers and automatic StreamerInfo
streamers.
Revision
3699 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Added
Fri Jan 18 14:24:10 2002 UTC (13 years ago) by
rdm
Original Path:
trunk/proofplayer/inc/TEventIter.h
File length: 1580 byte(s)
relocate some proof file to remove circular dependencies between the
treeplayer and proof shared libraries. Also rename TPlayer to TProofPlayer
et al. Also remove Begin() from TSelector, use Begin(0) instead. This to
be backward compatible with existing TSelector scripts that don't have
Begin() and therefor give warnings about Begin() being hidden in the base
class.
This form allows you to request diffs between any two revisions of this file.
For each of the two "sides" of the diff,
enter a numeric revision.
