Log of /trunk/proof/proofplayer/src/TProofPlayer.cxx
Parent Directory
Revision
48992 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Mar 28 15:26:26 2013 UTC (21 months, 3 weeks ago) by
rdm
File length: 163423 byte(s)
Diff to
previous 48766
From Lifeng Sun:
The attached patchset fixes a bunch of typo in the source:
0001-succes-success.patch
0002-preceed-preced.patch
0003-informations-information.patch
0004-childs-children.patch
0005-avaliable-available.patch
0006-writeable-writable.patch
0007-comand-command.patch
0008-unkown-unknown.patch
0009-wierd-weird.patch
0010-wheter-whether.patch
0011-unecessary-unnecessary.patch
0012-splitted-split.patch
0013-registerd-registered.patch
0014-recieve-receive.patch
0015-processsing-processing.patch
0016-ouput-output.patch
0017-mutiple-multiple.patch
0018-lenght-length.patch
0019-interupted-interrupted.patch
0020-independant-independent.patch
0021-inconsistant-inconsistent.patch
0022-expresion-expression.patch
0023-explicitely-explicitly.patch
0024-enviroment-environment.patch
0025-deafult-default.patch
0026-continous-continuous.patch
0027-completly-completely.patch
0028-commited-committed.patch
0029-choosen-chosen.patch
0030-backgroud-background.patch
0031-auxilliary-auxiliary.patch
0032-authentification-authentication.patch
0033-appropiate-appropriate.patch
0034-an-other-another.patch
0035-environement-environment.patch
0036-targetting-targeting.patch
0037-suppported-supported.patch
0038-paramater-parameter.patch
Revision
48766 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Feb 28 14:33:44 2013 UTC (22 months, 3 weeks ago) by
ganis
File length: 163423 byte(s)
Diff to
previous 48226
Make sure that a selector passed by object is never destroyed.
Fixes an issue when a automatic allocated selector is passed by reference
reported on root-dev.
Revision
46992 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Nov 2 09:50:08 2012 UTC (2 years, 2 months ago) by
ganis
File length: 162992 byte(s)
Diff to
previous 46864
Make sure that objects are not deleted when destroying the output lists in the selector and in the proof player; addresses possible crash after finalisation
Revision
45740 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Aug 29 16:17:09 2012 UTC (2 years, 4 months ago) by
ganis
File length: 162910 byte(s)
Diff to
previous 45696
- Add support for data dirs on external servers, fully honouring URL options; this allows to
have usable paths registered in the final TFileCollection objects. One of the addressed use-case
is when local directories need to be accessed via the local xrootd server to preserve the namespace
enforced by xrootd; in such a case something like 'root://locahost//datadir?remote=1' needs to be
specified; the option 'remote=1' ensures that the final registered URL is still in 'root://' form
with 'localhost' replaced by the FQDN of the machine.
- Move the 'tests' argument to the stressProof function in the second position, to make it
more user friendly to run a specific test in the ROOT shell; CLI behevior is unchanged.
- In ProofSimple, add function to plot the histograms also in the case a dataset is created with the
worker files; the file are opened and the histograms merged on the fly.
- Fix potential segv at xproofd startup due to a double fclose in XrdProofdManager::ValidateLocalDataSetSrc
- Remove some redundant printing in TProof when handling output files.
- Make sure that when running TSelVerifyDataSet the save-to-file functionality is switched-off .
Revision
45367 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Aug 3 09:22:00 2012 UTC (2 years, 5 months ago) by
ganis
File length: 159962 byte(s)
Diff to
previous 45348
In TProofOutputFile, add a status bit to indicate that the file is the result of the automatic
save-to-file output handling. This allows a better interplay between client-side and server-side
settings.
In TProof::HandleOutputOptions, fix issue with local object saving.
Revision
45348 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Aug 2 12:23:54 2012 UTC (2 years, 5 months ago) by
ganis
File length: 159840 byte(s)
Diff to
previous 45289
Consolidation of the automatic output saving functionality
- Fix an issue with saving output from memory on the master
- Remove some duplications and fix several small bugs
Revision
45283 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 30 10:42:38 2012 UTC (2 years, 5 months ago) by
ganis
File length: 159121 byte(s)
Diff to
previous 44683
Patch automatizing the usage of the file-based technology to handle outputs.
After this patch PROOF users can define as an option to Process an output file for their results
and choose if the merging goes via file or in memory. User can also decide to save theit results
in a set of files identified by a name (dataset).
Administrators can force merging-via-file or can make it conditional to the resident memory exceeding
a given threshold.
The new options are documented at
http://root.cern.ch/drupal/content/handling-outputs
Since TProofOutputFile is now used by TProof, the class has been moved from proof/proofplayer
(libProofPlayer) to proof/proof (libProof).
The selector tutorials/proof/ProofSimple.h,.C has been modified to allo testing the new options,
and a new test has been added to stressProof.
Revision
44683 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 14 09:27:40 2012 UTC (2 years, 7 months ago) by
ganis
File length: 147352 byte(s)
Diff to
previous 44512
First version of the final patch enabling multi-workers on single node using
PROOF-Lite. This is supposed to improved scalability for large number of workers
(>300-400) and is currently available only for non-data driven tasks.
The new setup configuration is enabled by the directive 'xpd.remoteplite 1' in
the xproofd configuration file.
Revision
44397 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu May 31 13:28:14 2012 UTC (2 years, 7 months ago) by
ganis
File length: 147243 byte(s)
Diff to
previous 44065
Use by default the official Merge methods for histograms which correctly treats
histograms with labelled axis.
Simple addition was performed for histograms with same axis to overcome a
perfomance penalty due to additional general consistency checks present in
the first version of TH1::Merge. This performance issue has been fixed starting
with 5.32.
Should the issue reported in Savannah #94926.
Revision
44046 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue May 1 11:34:04 2012 UTC (2 years, 8 months ago) by
ganis
File length: 148318 byte(s)
Diff to
previous 44010
Simplify enabling of basic feedback.
- In TDrawFeedback::Feedback, call method Draw() of objects not identified as TH1
derivation. This allows user-defined objects implementing Draw to be displayed
via this utility class (request on the forum).
- In TPerfStats, add new histogram TH1I with the packets being processed per worker;
currently this is 0 or 1 but may change in future; it shows which workers are
still active, which may be useful for debugging at the end of query.
- Add new class TStatsFeedback to display in one canvas the distributions for
the events processed per workers, the packets processed per workers and the
packets being processed per worker.
- In TProof::Process, add support for switches "fb=name1,name2,name3,... " or
"feedback=name1,name2,name3,... " in the option field. This enables feedback
for the specified objects, creating a TDrawFeedback object attached to the session.
Feedback for the specified objects is disabled at the end of the query and the
created TDrawFeedback is destroyed.
The special name 'stats' enables the three feedback histograms required by
TStatsFeedback, and uses a TStatsFeedback instead of TDrawFeedback.
The feedback specs are removed from the option field passed to TProofPlayer::Process.
Revision
44010 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Apr 29 23:07:48 2012 UTC (2 years, 8 months ago) by
ganis
File length: 147510 byte(s)
Diff to
previous 43251
Several optimizations in the low level PROOF event loop (TProofPlayer::Process):
- Per-event calls to TEventIter::GetNextEvent are replaced by per-packet calls
to a new function TEventIter::GetNextPacket and a for loop over the returned
range;
- Progress information is filled in on per-packet pace just before asking for
a new packet, instead of being filled for each event;
- In TProofProgressStatus, the relevant setters in were inlined, and the (expensive)
calls to SetLastUpdate() reduced to the minimum required;
- Status of objects is checked using bits instead of functions calls;
- Check whether to dump memory info is done before calling CheckMemUsage.
These optimizations allow to reduce dramatically the overhead introduced by the
operations PROOF needs to perform during the event loop. A measurement of the
overhead can be obtained from a very light computational task, for example,
generating one random number and filling one histogram; executing this task
within a PROOF-Lite session with 1 worker now takes only 1.8 times the time
required by a straight loop in the parent ROOT session; the same number before
was about 13.
The patch also introduces a couple of small changes enabling the use of
TProofPlayerLocal to process selectors in the local session using the PROOF
iteration mechanism; this is very useful for optimizations studies.
Finally, a couple of missing protections were added in TProofPlayer.
Revision
43107 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Feb 24 12:45:22 2012 UTC (2 years, 10 months ago) by
ganis
File length: 145088 byte(s)
Diff to
previous 43044
From B. Butler:
Add the possibility to force submerging at node level, i.e. one submerger per physical machine.
In this way the network traffic can be minimized, for example when merging large output files.
The new feature is enabled by setting the Int_t parameter 'PROOF_MergersByHost' (or the directive
'Proof.MergersByHost') to a non-null value.
Revision
43044 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Feb 17 14:34:52 2012 UTC (2 years, 11 months ago) by
ganis
File length: 143357 byte(s)
Diff to
previous 42399
From Taesoo Kim and me.
Add support for selector-by-object processing in PROOF.
The selector object, created and configured locally by the user, is added to the input
list and recuperated from there on the worker machines for processing. Any input list
setting in the selector itself is not streamed but temporarly moved to then standard
input list, so that user can use the selector input list as container of processing
information if they find convenient to do so.
Process(...) methods with the file name argument replaced by 'TSelector *' have
introduced where relevant (TProof, TProofPlayer and their derivatives, TDSet).
Two tests for the new functionality have been added to test/stressProof.cxx (tests 25
and 26).
Revision
40699 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Aug 25 10:36:29 2011 UTC (3 years, 5 months ago) by
rdm
File length: 138655 byte(s)
Diff to
previous 40359
From Gerri:
Patch adding support for backend-dependent record formatting of PROOF
monitoring. This is achieved by introducing a new layer, described by
the abstract interface TProofMonSender, with the record format defined
in the backend implemenattion (currently TProofMonSenderML, for MonaLisa,
and TProofMonSenderSQL, for SQL backends).
Currently three types of records are sent: 'summary' (derived from what
was currently posted), 'dataset', with entries per dataset processed
in the query, and 'files', with entries per file processed in the query.
In SQL terms, each of this records corresponds to a different table.
Sending of any of the three records can be toggled independently.
Record format have been versionned, so that previous versions of the
'summary' record can be still sent chosing the corresponding version.
The latest version of 'summary' has been augmented with information
about the number of missing files and the ROOT version.
The headers of the relevant methods in TProofMonSenderSQL and
TProofMonSenderML contain the details about the records.
The Web doc is up-to-date.
The patch also add to TSQMonitoring::SendParameters the possibility to
do a bulk insert. This feature is used by TProofMonSenderSQL when
sending dataset or files information.
Revision
40186 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 11 12:00:50 2011 UTC (3 years, 6 months ago) by
ganis
File length: 138635 byte(s)
Diff to
previous 40017
- Record in TStatus also the max memory usage on the master and printed via TStatus::Print;
this allow a quick visualisation of the overall memroy usage at the end of the query:
root [] TStatus *st = (TStatus *) p->GetOutputList()->FindObject("PROOF_Status");
root [] st->Print();
OBJ: TStatus PROOF_Status OK
Max worker virtual memory: 394.90 MB Max worker resident memory: 229.47 MB
Max master virtual memory: 1104.55 MB Max master resident memory: 940.73 MB
- Fix issue in TProofPlayer::HandleHistogram preventing proper histogram cleaning right
after merging when using TH1::Add; histogram were still destroyed at the end of the
query, but there was no memory advantage in TH1::Add wrt TH1::Merge.
Revision
40017 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 27 16:23:01 2011 UTC (3 years, 6 months ago) by
ganis
File length: 138139 byte(s)
Diff to
previous 39835
- In TPerfStats, add 'status' field to the monitor record, an integer with the query exit status
(see TVirtualProofPlayer::EExitStatus (0 OK; 1 stopped; 2 aborted).
- Add exist status also to TStatus, so that it is available in the output list (TPerfStats takes
it from here).
- Send the output list, if non empty, also when the query is aborted, so that the TStatus object
is available also in those cases.
Revision
39835 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 20 15:37:30 2011 UTC (3 years, 7 months ago) by
ganis
File length: 137656 byte(s)
Diff to
previous 39585
- In TPacketizer and TPacketizerAdaptive, fix an issue with counter updating when a number
of events to be processed is specified (it was working up to a certain number of files and
then it was getting screwed up).
- In TPacketizerAdaptive, fix an issue with the option 'ForceLocal' on 'file:///' URLs.
- In TProofPlayer, optimize two conditional scopes.
- In TProofPlayerLite, make sure that the Progress timer is stopped when issuing STOP.
Revision
39585 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jun 7 20:56:07 2011 UTC (3 years, 7 months ago) by
ganis
File length: 137656 byte(s)
Diff to
previous 39442
In TProofPlayer, make sure that TVirtualPAcketizer::HandleTimer is not called concurrently
during 'STOP', because this can deadlock the master. Fixes an issue with 'STOP' reported by
ALICE.
Revision
39442 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri May 27 09:13:25 2011 UTC (3 years, 7 months ago) by
ganis
File length: 136635 byte(s)
Diff to
previous 39173
In TProofPlayerRemote::HandleHistograms, add support for an option to call directly TH1::Add
if the incoming histogram has the same axis of the one in memory. This allows to avoid going
through the whole Merge machinery for the 99.9% of the cases, considerably speeding up
histogram merging and significantly reducing memory requirements on the master.
This new way is on by default and is controlled by the rootrc 'ProofPlayer.UseTH1Merge' and
by the input list parameter PROOF_UseTH1Merge .
Revision
38810 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Apr 12 16:22:59 2011 UTC (3 years, 9 months ago) by
ganis
File length: 129046 byte(s)
Diff to
previous 38709
Patch to correctly honour selector abort status settings in PROOF. Currently only
the TSelector::kAbortProcess was handled by stopping processing. In particular
TSelector::kAbortFile was ignored; this recently created some problems in ALICE
with corrupted files, with repeated attempts to read events eventually leading to
bad_alloc exceptions.
This patch also fixes other related issues, in particular with the reporting of the
non-processed {files, events} in the final 'MissingFiles' list. This list should
now account much more precisely of the number of events which could not be processed.
It also fixes a problem with the final update of the progress information affecting
occasionally cases with skipped events.
Revision
38618 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Mar 24 19:48:13 2011 UTC (3 years, 10 months ago) by
ganis
File length: 126901 byte(s)
Diff to
previous 37987
Make sure that the default output file whne using TProofOutputFile is set
coherently to the loacl settings. In particular, if 'rootd' data serving
option is set the file should be readible via the 'rootd://' protocol.
This should fix Savannah issue #79845 .
Revision
37987 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 7 07:10:15 2011 UTC (3 years, 11 months ago) by
ganis
File length: 126430 byte(s)
Diff to
previous 37969
Add switch to control caching of the files read on MacOsX.
A call to fcntl(fd, F_NOCACHE, 1) is done after opening the file.
To disable OS file caching the parameter PROOF_DontCacheFiles - or the ROOT-rc
variable ProofPlayer.DontCacheFiles - must be set to 1.
By default caching is enabled.
Revision
37969 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Feb 3 17:32:55 2011 UTC (3 years, 11 months ago) by
ganis
File length: 126167 byte(s)
Diff to
previous 37396
In TProofPlayer::AddQueryResult, fix a bug affecting the order in whihc query
results are registered when the start time is within 1 second.
Revision
37396 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 8 13:12:00 2010 UTC (4 years, 1 month ago) by
rdm
File length: 126164 byte(s)
Diff to
previous 36835
From Gerri:
- Fix a problem with the registration of missing files in the 'MissingFiles'
list. The files that could not be open during processing were not properly
registered in the list (only those found missing during validation or
giving problmes durign reading were correctly added).
- Add method TProof::ShowMissingFiles() to facilitate the display of the
list of missing files.
- Add method TProof::GetMissingFiles() to get a TFileCollection (dataset)
with the missing files for further processing.
Revision
36592 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Nov 11 10:43:17 2010 UTC (4 years, 2 months ago) by
ganis
File length: 125332 byte(s)
Diff to
previous 36537
- Terminate a proofserv after it has been stopped for large memory footprint and it has
sent the results back
- Notify information about which object was being merged if a memory exception happens
during merging on the master
- Allow to set the HWM and stop-process fractions via the environment
Revision
36537 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 8 16:16:41 2010 UTC (4 years, 2 months ago) by
ganis
File length: 124652 byte(s)
Diff to
previous 36526
Patch to improve the diagnostic in case of exceptions. Information about the event and file being
processed at the moment the exception was raised is sent to the client, e.g.
0.5: caught exception triggered by signal '1' while processing dset:'EventTree',
file:'http://root.cern.ch/files/data/event_3.root', event:1 - check logs for possible stacktrace
The patch also fixes a problem with submergers observed when a worker was stopped because above
the memory limits: this worker was established as merger but could not do the work, for obvious
reasons, freezing the session.
Revision
36526 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 8 07:39:11 2010 UTC (4 years, 2 months ago) by
ganis
File length: 122270 byte(s)
Diff to
previous 36475
Add possibility for the to disable the use of submergers when set by default by the
administrator. This is now achieved by setting the parameter PROOF_UseMergers to -1.
Revision
36475 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 2 13:23:08 2010 UTC (4 years, 2 months ago) by
rdm
File length: 122256 byte(s)
Diff to
previous 36440
From Gerri:
add the possibility to control the resident and virtual memory of a proofserv
using 'ulimit', which has less limitations and more flexibility than setrlimit.
Revision
36440 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Oct 28 11:23:46 2010 UTC (4 years, 2 months ago) by
ganis
File length: 122245 byte(s)
Diff to
previous 36116
Add missing protection causing a crash in submergers when the output list
contained TProofOutputFile objects (see forum).
Revision
35335 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Sep 16 14:08:58 2010 UTC (4 years, 4 months ago) by
ganis
File length: 121626 byte(s)
Diff to
previous 35313
Make sure that TProof::Load, TProofPlayer::SendSelector and TSelector::GetSelector
treat consistently the extensions of the implementation files. Fix issue on the forum.
Revision
35313 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Sep 15 21:57:26 2010 UTC (4 years, 4 months ago) by
ganis
File length: 122018 byte(s)
Diff to
previous 35195
The feedback canvas and the final canvas are the same, so do not delete the
feedback canvas at the end of processing (should fix issue #72679).
Revision
35086 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Aug 31 10:55:49 2010 UTC (4 years, 4 months ago) by
ganis
File length: 122111 byte(s)
Diff to
previous 34959
- Warn about high memory usage ( > 0.80 * upper limit) only once per worker; avoids flooding
xproofd multiplexing with the same message thousand of times.
- Add the ordinal number information in the messages about high memory usage.
Revision
34748 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Aug 9 10:18:05 2010 UTC (4 years, 5 months ago) by
ganis
File length: 121784 byte(s)
Diff to
previous 34388
Add the possibility to save the perfomance information shown by the dialog into a small
ntuple included in the output list. The ntuple contains 5 floats (processing time, number
of active workers, event rate, MBytes read, number of effective sessions on the cluster)
and it is filled each time the number of active workers changes or at max 100 regular
intervals at least 5 secs apart; in this way the ntuple has at most O(100 entries + number
of workers).
To enable the saving of the ntuple execute the following:
proof->SetParameter("PROOF_SaveProgressPerf", "yes");
before running the query. The ntuple is called 'PROOF_ProgressPerfNtuple'.
This patch also adds to the output list the parameters used by the active packetizer. Some
parameters of general interest (currently MinPacketTime and MaxPacketTime) have been moved
to TVirtualPacketizer and are always added to the list. Each packetizer is then responsible
of adding its relevant specific parameter to the dedicated list. The dedicated list is hosted
in TVirtualPacketizer and is transferred to the output list by TProofPlayer when finalising
the output list at the end of the query.
Revision
34388 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jul 13 09:20:42 2010 UTC (4 years, 6 months ago) by
ganis
File length: 121369 byte(s)
Diff to
previous 34380
The newly introduced variable to control the use of submergers should be called Proof.UseMergers consistently with the related parameter
Revision
34380 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 12 14:12:13 2010 UTC (4 years, 6 months ago) by
ganis
File length: 121372 byte(s)
Diff to
previous 34286
Add support for the ROOTrc variable 'Proof.SubMergers' to trigger the use of submergers for
merging. This has the same meaning of the parameter 'PROOF_UseMergers'. It is provided to
allow AAF administrators to force the use of submergers to reduce the memory footprint.
The setting can be always superseeded by the user setting of the 'PROOF_UseMergers' in the
input list (or of the local Proof.SubMergers setting).
Revision
33867 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 14 10:02:39 2010 UTC (4 years, 7 months ago) by
ganis
File length: 120695 byte(s)
Diff to
previous 33632
From Axel:
Identify which TSelector data members point to which output list object
on the workers, and if consistent set the client's data members to point
to the corresponding objects of the merged output list.
Revision
33485 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed May 12 13:27:47 2010 UTC (4 years, 8 months ago) by
ganis
File length: 119626 byte(s)
Diff to
previous 32930
- Add option to skip the dataset lookup; this can be useful when working with back-ends not
properly supporting "locate", like DPM/Xrootd .
To skip the lookup, one should call the following before Process(...):
proof->SetParameter("PROOF_lookupOpt", "none")
Revision
32930 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Apr 9 16:18:32 2010 UTC (4 years, 9 months ago) by
pcanal
File length: 119443 byte(s)
Diff to
previous 32237
Prefer the use of InheritsFrom(TClass*) over InheritsFrom(const char*) as long as
it does not increase the library dependencies.
Revision
32237 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Feb 6 13:01:45 2010 UTC (4 years, 11 months ago) by
ganis
File length: 119468 byte(s)
Diff to
previous 32204
Fix a few issues found by running stressProof with PROOF-Lite:
- Order query results using TQueryResult::fStartTime instead of TQueryResult::fEndTime
(which is undefined at the beginning)
- Fix the hostname for submergers in PROOF-Lite
- Fix behaviour when submergers cannot open the required sockets (output was not
always sent ot the master)
- Fix several typos / wrong method name / wrong protocol name
- Update info about test numbers in stressProof
Revision
32204 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Feb 3 19:17:40 2010 UTC (4 years, 11 months ago) by
ganis
File length: 119464 byte(s)
Diff to
previous 32003
Add support for processing many datasets in one go in TProof::Process(const char *dataset, ...).
Two options are provided:
- 'grand dataset': the datasets are added up and considered as a single dataset;
syntax: "dataset1|dataset2|..."
- 'keep separated': the datasets are processed one after the other; the user is
notified in the selector of the change of dataset so she/he
has the opportunity to separate the results. A new packetizer,
TPacketizerMulti, has been developed for this case: it basically
contains a list of standard packetizers (one for each dataset) and
loops over them.
Syntax: "dataset1,dataset2,..." or dataset1 dataset2 ..."
In both cases, entry-list can be applied using the syntax "dataset<<entrylist", e.g.
"dataset1<<el1|dataset2<<el2|".
See http://root.cern.ch/drupal/content/working-data-sets#currentelem for more details.
A test for the new functionality has been added to test/stressProof.cxx .
Revision
32003 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jan 13 18:00:12 2010 UTC (5 years ago) by
ganis
File length: 117524 byte(s)
Diff to
previous 31840
- Avoid a possible double deletion in TProofServ::HandleSubMerger
- Fix a small memory leak in TEventIterUnit and TEventIterObj (the
processed 'packet' was not deleted before asking the next one)
- Improve comments in the TProofPlayer
Revision
31830 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Dec 11 08:54:32 2009 UTC (5 years, 1 month ago) by
ganis
File length: 117474 byte(s)
Diff to
previous 31683
Fix an issue with the 'ntuple' tutorial in PROOF:
- Make sure that when only one file is produced this is correctly 'merged' into the output file
Revision
31683 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Dec 8 16:12:08 2009 UTC (5 years, 1 month ago) by
ganis
File length: 117068 byte(s)
Diff to
previous 31598
TProofPlayerRemote
- In SendSelector, add missing option kCpBin when sending the selector source; the binary files were
never retrieved, even if present and valid
TProofPlayerSlave
- In Process, fix a problem with cache directory locking while building the selector; the net effect
was that each worker process was re-buidling its own selector binary.
- Fix warning from the nightlies (variable possibly clobbered by `longjmp' or `vfork')
TProofServ
- Fix the order in which the log file is sent in asynchronous processing; the wrong order was screwing
up an immediate synchronous query submission after an asynchronous run; this case occured, for example,
in 'stressProof'
Revision
31598 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Dec 7 15:21:47 2009 UTC (5 years, 1 month ago) by
rdm
File length: 116817 byte(s)
Diff to
previous 31505
From Gerri:
Fix a few pending issues in PROOF-Lite:
- Source the env file before running 'proofserv'; this allows to run valgrind
or other wrapper scripts (e.g. experiment-specific environment initializers)
before proofserv, as in normal PROOF
- Fix a problem with Load()
- Add the possibility for the administrator to control the number of workers
for PROOF-Lite or eventually to forbid the use of it (request on the forum).
The rootrc variable ProofLite.MaxWorkers is read out of /etc/system.rootrc
and cannot be overwritten by users.
Improve handling of Ctrl-C:
- Fix a problem in TMonitor affecting the effectiveness of interrupts.
- Do not send the results from workers to masters if the query has been
aborted.
Miscellanea:
- Make sure that the size of the cache sent back to the packetizer by the
workers is meaningful in case the defaults are used (i.e. not -1).
- Make the current processed element available to selectors via the
input list; this allows to write a more simple and effective packetizers
for tasks where the entry is a file (e.g., dataset generation in the
benchmark suite).
Revision
31505 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 2 13:20:06 2009 UTC (5 years, 1 month ago) by
brun
File length: 116175 byte(s)
Diff to
previous 31437
From Katerina Opocenska:
Patch implementing the new submerger functionality to speed up the merging phase.
At the end of the query, a set of workers are promoted submergers and assigned a
sub-set of workers to merge. Once each sub-merger has merged its sub-set of workers,
it sends its result to the master, which merges the partial results into the final
set of results.
The determination of the sub-mergers is always done dynamically, based on the recent
performance of workers. An optimal (i.e. giving the highest speed-up) number can be
calculated analytically under simpleassumptions.
Merging via submergers is by default disabled. To enable it, with the optimal number
of sub-mergers, one should set the integer parameter 'PROOF_UseMergers' to 0, i.e.
proof->SetParameter("PROOF_UseMergers", 0)
To force S sub-mergers (regardless of the optimal number) do
proof->SetParameter("PROOF_UseMergers", S)
Revision
31437 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Nov 26 11:42:25 2009 UTC (5 years, 1 month ago) by
ganis
File length: 115779 byte(s)
Diff to
previous 30910
- Re-enable the use of the tree cache also for local files
- Adapt the default settings for the cache to the recent changes
Revision
30859 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Oct 24 14:53:07 2009 UTC (5 years, 3 months ago) by
ganis
File length: 112553 byte(s)
Diff to
previous 30562
Patch for improved performance monitoring. The 'Rate Plot' button in the progress
dialog is renamed 'Performance plot' and shows up to 4 plots with the event/sec,
the average read chunck size, the number of active workers and the number of active
PROOF sessions on the cluster, all as a function of processing time.
The read chunck size plot allows to monitor the usage of the cache.
The istantaneous processing rate (event/sec) is now better estimate: a few issues
with the normalizing times have ben solved, removing the artificial structures that
were observed.
The possibility to set a max packet time length is introduced (default 30 s); this
can be changes with the parameter PROOF_MaxPacketTime.
The size of the cache is also taken into account to optimize the use of the cache.
The parameter PROOF_UseParallelUnzip has been introduced to toggle the use of the
parallel unzip (default off for now).
A page describing the new performance plots is under preparation at
http://root.cern.ch/drupal/content/progress-dialog
Revision
30562 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Oct 5 15:02:21 2009 UTC (5 years, 3 months ago) by
ganis
File length: 109427 byte(s)
Diff to
previous 30362
- Add the possibility to enable/disable the tree cache and to change its size on per-query base;
two new parameters are available:
PROOF_UseTreeCache Int_t Enable (0) or Disable (1) the tree cache (default 1)
PROOF_CacheSize Long64_t Cache size in bytes (default 10000000)
Examples:
a) to disable the cache for the next run enter:
proof->SetParameter("PROOF_UseTreeCache", 0)
b) to set the cache size to 20M
proof->SetParameter("PROOF_CacheSize", 20000000)
- Fix a problem with changing the tree cache size: the latter was reset to the default value
after the first file.
Revision
30174 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Sep 15 14:24:56 2009 UTC (5 years, 4 months ago) by
ganis
File length: 108691 byte(s)
Diff to
previous 29491
Patch to support automatic creation of a dataset out of files created on the
worker nodes by worker processes. The implementation is an extension of the
functionality of the class TProofOutputFile used for merging via file.
TProofOutputFile:
- Add options to steer the creation of a dataset; the extemded meaning of the
constructor arguments and anew constructor are described in
http://root.cern.ch/drupal/content/handling-large-outputs-root-files
The dataset (i.e. TFileCollection) assembing happens on the master via the
Merge method. Information about the node location of the file is happended
to the relevant TFileInfo as a special "file:" url with option "node=<host-fqdn>:<port>"
This information is needed to facilitate the cleaning of the files once they
are not needed anymore.
The dataset can be just created and returned in the output list, or registered
and/or verified.
TProofPlayerRemote::MergeOutputFiles:
- Correctly transmit the dataset object into the output list
TProofServ:
- Create the new subdirectory 'data' either in the sandbox or followinf ProofServ.DataDir .
- Register and verify the created dataset if required
TProof:
- Implement two new methods:
- ShowData(), to show the content of the assigned 'data' directory
- ClearData(UInt_t what, const char *dsname), to clear all or a subset of the files
in the assigned 'data' directory
Additional changes:
- TProofPlayer:
- Correctly handle the feedback canvas
- Set the drawing attributes
- TProofServ:
- Replace all 'Form("...")' with TString::Format("...") .
- Use the new tag kCollect in debug statements where relevant
- TProof:
- Use the new tag kCollect in debug statements where relevant
Revision
29491 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jul 16 15:12:51 2009 UTC (5 years, 6 months ago) by
ganis
File length: 104981 byte(s)
Diff to
previous 28983
Fix several issues in TProofOutputFile:
- Remove the 'mode' parameter as the 'SEQUENTIAL' mode was not really an option
for the cases when this merging technique is needed
- Fix a problem with the determination of the fDir member; this problem affects
mostly PROOF-Lite and it appeared after the change #25949 .
- Make sure the the local server information is correctly included in AdoptFile()
- Fix a serious issue with the 'CENTRAL' mode whose net effect was to delete the
outputfile just after having open it
In TProofPlayer and XrdProofdProofServMgr
- Cope with the changes in TProofOutputFile
Revision
28981 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Jun 14 13:45:39 2009 UTC (5 years, 7 months ago) by
ganis
File length: 104723 byte(s)
Diff to
previous 28856
- Fix a problem introduced by the optimization #28856 causing some histograms
not to be correctly added to the final output list.
Revision
28856 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jun 9 11:49:43 2009 UTC (5 years, 7 months ago) by
ganis
File length: 104690 byte(s)
Diff to
previous 28740
A few optimizations in transferring and merging of the results:
- Output objects are added to the same TMessage until a HWM is reached
(default 1MB; controlled by 'ProofServ.MsgSizeHWM'); this limits the
number of transfers in the case of large numbers of small objects.
- Reasonably small histograms (GetSize() < MsgSizeHWM) are merged in
one-go at the end instead of one-by-one to exploit the better performance
of TH1::Merge on the full list of histos.
- Add possibility to compress the messages; this is controlled by
ProofServ.CompressMessage <compression_level>
The default is still 'no compression' but this will allow to study the
impact of compression.
In addition, a sort of 'progress' counter for merging is now shown on the
client:
root [n] p->Process(...)
...
Mst-0: merging output objects ... / (4 workers still sending)
This asserts socket activity and fixes the timeout problems during long
merging phases reported on the forum.
Revision
28370 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Apr 28 06:43:41 2009 UTC (5 years, 8 months ago) by
ganis
File length: 104134 byte(s)
Diff to
previous 27893
Add the possibility to switch to asynchronous mode while running synchronously.
A new button "Run in background" has been added to the dialog box. The behaviour
of Ctrl-C has also been modified: the user is prompted for a choice among continuing
asynchronously, stopping (terminating) or aborting the query.
Revision
27234 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jan 26 07:57:14 2009 UTC (5 years, 11 months ago) by
ganis
File length: 104055 byte(s)
Diff to
previous 27198
TProofDraw
- Do not try to get "htemp" from the input list; this removes a useless warning
TProofPlayer
- Improve the treatment of histogram merging in case of small statistics, i.e. when
the autobinning is not or only partially active. The largest stat histogram is
taken as base for the merging. This is implemented both for output and feedback
histograms, and avoids the large axis boundaries variations observed in such cases.
Revision
27198 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jan 21 09:29:10 2009 UTC (6 years ago) by
ganis
File length: 100601 byte(s)
Diff to
previous 27035
TProofDraw
- Fix a problem preventing drawing into an existing histogram, i.e. chain->Draw("var>>myhist"),
to work properly (chain->Draw("var>>hnew(nbin,xmin,xmax) was working fine).
- Add missing initialization (fTree in the basic constructor)
TProofPlayer
- Remove the relevant map in the list fFeedbackLists if the final object is available in the
output list; this prevents a duplication of the final feedback object
- Improve debug messages in StoreFeedback
TProofServ, TProof
- Fix a couple of memory leaks due to missing destruction of TProofProgressStatus objects
Revision
26678 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Dec 5 14:53:23 2008 UTC (6 years, 1 month ago) by
ganis
File length: 99892 byte(s)
Diff to
previous 26447
Patch to:
- Add the possibility to set upper limits on the virtual memory (changes in TProofPlayer,
TProofServ, TXProofServ and pmain (separate patch)).
If enebled, the session gets firts a warning when it reaches 80% of the limit, and then
processing is stopped whenit exceeds 95% of the limit, sending back the results.
Also, the memory footprint is notified when the session is terminated.
- Make sure the the active valuse in XrdProofWorker are always correct; this was not the
case for dynamic startup as the notification at the end of the query was not done.
This information is crucial or the scheduler.
The way the information is stored in XrdProofdProofServ had to be modified
and a new internal message type (kReleaseWorker) added.
(Changes in several proofd classes, TProof, TXsocket and TXProofServ)
- Fix problem with setting a static upper limit of the sessions that can be started,
and enable this functionality for the dynamic mode (changes in XrdProofSched)
- Remove a deleted worker from all the lists in TProof::MarkBad to avoid later attempts of use.
- Better control the use of the internal pipe for socket readiness notification in TXSocket.
Revision
26447 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 25 11:11:39 2008 UTC (6 years, 1 month ago) by
ganis
File length: 98617 byte(s)
Diff to
previous 26383
- Fixes on XrdProofd for Ubuntu 8.10 (from F. Furano)
- Fixes in async mode for the dynamic scheduling (from Jan)
- In Xrdproofd:
- Use per-user connections for Reset and Log retrieval
- Fully reset the client session vector upon Reset
- Add several missing protections
- Remove some debug statements in XrdSecProtocolgsi
Revision
26383 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Nov 22 17:58:39 2008 UTC (6 years, 2 months ago) by
ganis
File length: 98393 byte(s)
Diff to
previous 26381
- Fix the number of entries in the Draw histos for PROOF-Lite
- Set the "number of workers per node" in PROOF-Lite to the number of workers in the session
- Remove some debug messages left by mistake
Revision
26381 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Nov 22 17:15:24 2008 UTC (6 years, 2 months ago) by
ganis
File length: 98426 byte(s)
Diff to
previous 26268
- Fixes for the asynchronous mode:
- Fully localize the general handling of the input messages into two new methods
TProof::HandleInputMessage(TMessage *) and TProofServ::HandleSocketInput(TMessage *);
these methods are callable form any place that needs to intercept some messages and
do something specific on them (examples are: TProofServ::GetNextPacket, TXProofServ::Get,
TPacketizerAdaptive::ValidateFiles, ...); this allows to remove several duplications
and to make sure that no message is lost or wrongly dispatched.
- Simplify the recursive infrastructure for TProofServ::HandleSocketInput; in particular,
TProofServ::HandleSocketInputDuringProcess is removed as it is a special case of the
HandleSocketInput, saving duplications.
- Always use a kPROOF_CHECKFILE message in replies to check file operations; failures were
signaled with kPROOF_FATAL which may have some undesired side-effects, depending on the
timing
- Add support for one level of recursivity in TProof::Collect .
- Fix a problem with TProof::Finalize when called with default arguments (on the last query)
- In TProof::SendFile, send to unique workers only in the "cache" option is specified
- Remove the call to Finalize for DrawSelect queries, as it is done via the feedback
mechanisms
- Additions/fixes in test/stressProof.cxx:
- New test for the asynchronous mode;
- Fine-tuning of the progress display in batch mode;
Revision
26268 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 18 16:27:09 2008 UTC (6 years, 2 months ago) by
ganis
File length: 98259 byte(s)
Diff to
previous 26257
- Fix a few problems with the input data machinery (thanks to C. Cheshkov)
- Use the SingeKey option to write out containers
- Send the input data after the call to Begin() to catch up also the objects
possibly added in there
- Fully enable the input data machinery also for PROOF-Lite; this required to
move some calls from TProofServ to TProof to avoid duplications.
- Additions to test/stressProof.C
- Add a test for the input data machinery
- Add some progress info for the tests involving processing
Revision
25918 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Oct 22 15:00:04 2008 UTC (6 years, 3 months ago) by
ganis
File length: 96973 byte(s)
Diff to
previous 25896
First version of PROOFLite, a 2-tier realization of PROOF intended for multi-core
machines. In PROOFLite the client start directly the workers; no daemon is required.
Currently this works on Unix only. Some issues with the internal connection still need
to be sorted out on Windows.
To start a session just use
TProof::Open("") or TProof::Open("lite")
From there on everything should be as in normal PROOF. To start a standard PROOF
session (i.e. via daemons) on the localhost use TProof::Open("localhost").
(The default meaning of "" in TProof::Open() is controlled via the rc value
'Proof.LocalDefault').
To run stressProof with PROOFLite pas the argument 'lite':
./stressProof lite
By default, PROOFLite sets the number of workers equal to the number of cores.
The number of workers is controlled by the rc variable 'ProofLite.Workers'; it can
also be set on the fly with the option "workers=<n>", e.g. TProof::Open("lite?workers=2").
This patch also introduces the rc variable 'Proof.Sandbox' to control the local location
of the sandboxes. The default has been changed from "~/proof" to "~/.proof" to avoid
interferences with possible users' working areas. The sandbox contains the directory
'packages' (with the clients packages; used by standard PROOF and PROOF Lite),
and some directories used only by PROOFLite: 'cache' (binary cache) and 'datasets' with
the datasets. The location of these directories is controlled by the rc variables
'Proof.PackageDir', 'Proof.CacheDir' and 'Proof.DataSetDir'.
The working directory for PROOFLite is in the form workdir=<sandbox>/path-to-current-dir where
~/path/to/current/dir is the directory from where the session in launched. The directory
<workdir> contains the session working dirs (in the form 'session-<host>-<time>-<pid>'),
with the symlink 'last-lite-session' pointing to the last started session, and the 'queries'
directory, with the results of queries started from the current directory.
Revision
25896 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Oct 20 17:01:31 2008 UTC (6 years, 3 months ago) by
ganis
File length: 95955 byte(s)
Diff to
previous 25840
From Jan:
- Using consistently the recently introduced TProofProgressStatus in the kPROOF_GETPACKET
messages sent to TPacketizerUnit, TPacketizerAdaptive and TPacketizer; the message contains
the status of progress since the start of processing on a given node.
- Introduce TVirtualPacketizer::TVirtualSlaveStat as a base class of all the TSlaveStat
packetizer specific auxilliary classes.
- Full implemenation of GetProgressStatus() and AddProcessed(TProofProgressStatus *st) members
for TPacketizerUnit::TSlaveStat.
This patch should fix some consistency problems experienced after the patch introducing
TProofProgressStatus .
Revision
25827 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Oct 15 14:02:59 2008 UTC (6 years, 3 months ago) by
ganis
File length: 95978 byte(s)
Diff to
previous 25704
From Jan:
- Added the possibility to handle removed workers and partly processed packets. When a worker is stopped
while processing a packet it finishes, the current event and the rest of the packet is reassigned to another
worker. This is done via two interfaces:
- TVirtualPacketizer::AddProcessed(TSlave *sl, TProofProgressStatus *st, TList **)
- TVirtualPacketizer::ReassignPacket.
- New class TProofProgressStatus used to keep the query progress stauts in all the TProofPlayer objects and in
TPacketizerAdaptive::TSlaveStat. This class is also used to structure the relevant information send in
kPROOF_GETPACKET and kPROOF_STOPPROCESS messages.
- The class TPacketizerProgressive is removed completely.
- The PROOF protocol version is increased to 19: this is to handle the changes in the kPROOF_STOPPROCESS and
kPROOF_GETPACKET messages in Master - worker communication.
Revision
25704 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Oct 5 21:44:36 2008 UTC (6 years, 3 months ago) by
ganis
File length: 95442 byte(s)
Diff to
previous 25570
TProof:
- Add support for input data objects. These are objects which are available via the input list
but - being potentially large - are distributed to the cluster in optimized mode, i.e. via a
ROOT file copied only to unique nodes. These objects are *not* automatically saved into the
TQueryResult object.
Three new public methods have been added to TProof:
- AddInputData(TObject *, Bool_t = kFALSE) flags an input object as 'data input object'
- ClearInputData(TObject * = 0) clears the list of input data objects
- ClearInputData(const char *name) removes object 'name' from the list of input
data objects
- SetInputDataFile(const char *filepath) specifies the file from where to take the
input data objects; object in the input data
list, if any, are also send over.
New internal methods:
- BroadcastFile(const char *file, Int_t opt, const char *rfile, TList *wrks);
- BroadcastFile(const char *file, Int_t opt, const char *rfile = 0, ESlaves list = kAllUnique);
- void SendInputDataFile();
- Fixes:
- in Process(TFileCollection *, ...), remove the TFileCollection object from the input list
after processing, to avoid destruction problems in case of multiple query submission.
TProofServ:
- Add support for input data objects.
New methods:
- Int_t GetInputData(TList *input);
- Int_t SaveInputData(TQueryResult *qr);
- Int_t SendInputData(TQueryResult *qr);
- Optimize the number of input list instantiations
- Improve readibility of HandleProcess by separating out the code to prepare the dataset (get it
from input list, from dataset manager, etc ...) into a new method AssertDataSet(TDSet *, TList *).
TProofQueryResult:
- add missing protection
Revision
25559 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Sep 26 11:09:41 2008 UTC (6 years, 3 months ago) by
ganis
File length: 95485 byte(s)
Diff to
previous 25434
- Make sure that in case multiple TProofOutputFile are present, each get merged correctly
- Add missing protection on frequency for memory dumping (lower bound 1 not 0)
Revision
25434 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Sep 17 15:23:36 2008 UTC (6 years, 4 months ago) by
ganis
File length: 95041 byte(s)
Diff to
previous 25355
- Fix problem with the determination and transmission of the name of the object to be processed.
The problem appeared when processing files containing >1 trees in changing order.
- Fix problem with TProof::Load (Savannah bugs #41719; forum topic #7138)
- Fix problem with the logs from TProof::Load not being transmitted to the client
Revision
25092 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Aug 8 10:26:13 2008 UTC (6 years, 5 months ago) by
ganis
File length: 94965 byte(s)
Diff to
previous 24936
From Anna:
- Complete patch for memory monitoring
- Fine-tune the tip for the memory plot button
- Fix some typos in comments
Revision
23986 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri May 23 09:10:18 2008 UTC (6 years, 8 months ago) by
ganis
File length: 93588 byte(s)
Diff to
previous 23928
Import fixes / new functionality from branches/dev/proof:
- proof/proofplayer/src/TPacketizerAdaptive.cxx
Implement the classic strategy of the TPacketizer in TPacketizerAdaptive.
The strategy can be changed from adaptive (default) to TPacketizer with:
"PROOF_PacketizerStrategy" parameter to PROOF
- proof/proofplayer/src/TProofPlayer.cxx
Fixed error messages for 'MissingFiles' and 'FailedPackets' lists.
Improve fault detection by creating a list of failed packets upon a mismatch
in the expected and actual number of processed events; the list is added to
the output list.
- proof/proofplayer/src/TVirtualPacketizer.cxx
Make sure that something has been processed before setting kIsDone.
Improve fault detection by creating a list of failed packets upon a mismatch
in the expected and actual number of processed events; the list is added to
the output list.
- proof/proofplayer/inc/TVirtualPacketizer.h
Improve fault detection by creating a list of failed packets upon a mismatch
in the expected and actual number of processed events; the list is added to
the output list.
- proof/proofplayer/inc/TPacketizerAdaptive.h
Implement the classic strategy of the TPacketizer in TPacketizerAdaptive
- proof/proof/src/TProofServ.cxx
Fixes:
+ option string: "stageOnly" --> "stagedOnly".
+ add parenthesis to avoid a warning after the previous patch.
+ remove the objects added to the missingFiles in TDSet::Add from the 'dataset'
before deleting it
+ fixed an error HandleCheckFile ('kPROOF_WorkDir' instead of 'kPROOF_PackDir').
+ make fCacheDir and fPackageDir controllable via directive
+ in TProofServ::ErrorHandler: do not create the related additional buffer if
not logging to syslog
Added functionality:
- add possibility to flag an "Info" message as service message using
the prefix "|Svc" in the location field; e.g.
Info("SetupCommon|Svc", "Test of SvcMsg");
will produce something like
09:28:24 6892 Mst-0 | SvcMsg in <TXProofServ::SetupCommon>: Test of SvcMsg
This is needed to be able in the future to filter-out messages needed
by some services (e.g. the forthcoming memory checker) which should not be
displayed by default.
- proof/proofd/src/XrdProofSched.cxx
Improve the calculation of the number of workers to assign by using fMinForQuery
as a minimum.
Fix the length of method separators.
Fix signed/unsigned warning.
Revision
23075 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 9 09:10:25 2008 UTC (6 years, 9 months ago) by
ganis
File length: 92261 byte(s)
Diff to
previous 22669
From Jan-Fiete Grosse-Oetringhaus (ALICE) and me.
Patch introducing the improved dataset manager developed by ALICE.
The client side interface is basically unchanged, the biggest change is the renaming
of TProof::CreateDataSet to TProof::RegisterDataSet.
On the server side the relevant code has ben extracted from TProofServ and hidden
behind an inteface class TProofDataSetManager describing the functionality needed by
TProofServ, and an implementation class TProofDataSetManagerFile handling dataset
meta data saved into ROOT files. TProofDataSetManagerFile is instantiated via the
plugin manager and by default it handles the <sand_box>/datasets area.
This separation is needed to allow loading implementation based on different backends;
for example, ATLAS foresees to use a MySQL-based backend.
A TWiki page with information and examples in on preparation.
Revision
20758 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Nov 11 21:29:49 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 89207 byte(s)
Diff to
previous 20682
Several small fixes and few additions imported from 'branches/dev/proof':
- XrdProofdProtocol:
+ Fix a problem in CleanuProofServ sometimes preventing proper cleanup
+ In ReadBuffer, do not trigger an error on empty files
+ Add missing protection against an empty string while setting the ROOT version
- XrdProofdManager:
+ Define the list of unique nodes to avoid broadcasting twice to the same node
- XrdProofWorker: improve matching technique
- TProof: add setter and getter for Int_t parameter
- TProofServ:
+ do not flush the log file in SendAsynMessage: this recent change
made the logs "disappear" form the ProgressDialog
+ Activate control on the max number of query result kept
- TPacketizerUnit: fix a problem with filling the histof for PerfStats
- TProofPlayer: call Begin() also during selecter re-initialization
Revision
20682 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 6 15:51:59 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 89196 byte(s)
Diff to
previous 20307
Import branches/dev/proof r20654
Summary:
+ Improvements
- Add support for SSH SOCKS4 tunnelling; the local port for the tunnel can be specified
in the master URL, e.g. TProof::Open("master/?tunnel:8000")
- Add the possibility to plot the estimated instantaneous rate
- Add "PROOF_ForceLocal" parameter to the !TPacketizerAdaptive; if set to 1, all the data
are processed locally.
- Add support for remote grep functionality while retrieving logs (needed by the forthcoming
memory monitor)
+ Bug fixes
- Several small fixes to revive the multi-master mode.
- XrdProofdProtocol:
- add missing lock to the client instance in SendMsg to avoid screwing up requests from
workers on the same machine
- lock the mutex of the requester when setting prorities
- add notification during Reset
- fix problem with the detection of the 'allow' directive
- fix problem with the parsing of the return value from XrdProofServProxy::TerminateProofServ()
- always use the effective user to retrieve info fom another server
(XrdProofdManager::GetProofConn is now used)
- fix possible dead-locks from debug notifications done after hard-killing a session
- re-enable the garbage collector thread of the connection manager in XrdProofConn to
fix a problem with closing physical connections;
- fix a problem with !CleanupProofServ in the case of a non-privileged daemon running
in multi-user mode
- introduce a timeout when waiting for the startup of a 'proofserv'.
- XrdProofConn: init mutex in the ctor; lock in SendRecv
- XrdProofSched: add support for using the priorities defined in the group manager to define
the number of workers for sessions
- TProof:
- Broadcast priorities to unique nodes only
- timeout after 5 mins the initial Collect to avoid clients getting stuck at this stage
- add support for generic timeout in Collect (disabled by default)
- fix a problem with SendFile.
- TXProofServ: add a call to !TProof::InterruptCurrentMonitor() in Terminate() to stop
infinite loops in Collect
- TXSocket:
- Implement a flag to interrupt a TXSocket while waiting for messages
- Split the session creation timeout in 4 attempts: the total timeout is the same but
it may circumvent occasional forking problems.
- XROOTD:
- fix a potential (possibly accademic) memory leak in the client
- fix a access permission problem with Kerberos ticket forwarding
- fix bug preventing 'locate' to work properly
- re-enable optimized 'locate'
Revision
20307 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Oct 11 10:58:50 2007 UTC (7 years, 3 months ago) by
ganis
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 84889 byte(s)
Diff to
previous 20086
Import branches/dev/proof r20306
Summary (see branch logs for more details):
- Set of changes related to CPU quota control including
- broadcast of centrally determined priorities
- mechanism to renice processes in a quantitative way
- New packetizer for non-tree based analysis and related API in TProof (from L. Tran-Thanh)
- Support for merging output objects saved in files on the workers (from L. Tran-Thanh and me)
- Improve version binary compatibility checks using also the SVN revision number
(when available) to define the running version.
- Extend the version binary compatibility checks also to the cached selector binaries.
- Extend TDSet::Lookup so that in case of missing files, it can remove them from the
dataset (option removeMissing must be set).
- Move the data set lookup to the TProofPlayerRemote::Process.
- Handle properly the case of incomple datasets: if the file is not found in the lookup
don't try to validate it; add it, instead, to a 'missingFiles' list returned in the
output list (fixing bug #28800 in Savannah).
Revision
20086 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Sep 24 18:19:34 2007 UTC (7 years, 4 months ago) by
ganis
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 79614 byte(s)
Diff to
previous 19826
From Jan:
- Add a heuristic mechanism to avoid undervaluing the processing rate and to short packets.
- Add parameters: PROOF_MinPacketTime and PROOF_PacketAsAFraction to allow tuning performance.
- Change name TAdaptivePacketizer to TPacketizerAdaptive in order to follow the naming
convention for packetizers
- Slightly tuning a few parameters.
Revision
18941 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jun 5 05:47:25 2007 UTC (7 years, 7 months ago) by
ganis
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 75477 byte(s)
Diff to
previous 18829
TAdaptivePacketizer (from Jan)
- New version featuring better performance and robustness.
Changes include:
- simplified and better metric for finding the most loaded file node;
- adding a new static member for defining whether network bandwidth or
hard disk transfer is greater (TAdaptivePacketizer::fgNetworkFasterThanHD);
- it can be set via "proof->SetParameter("PROOF_NetworkFasterThanHD", (Long_t)0);";
- fixing the confusion with fProcTime, used also in the progress information;
- adding TAdaptivePacketizer::fCumProcTime;
- improvements in comments.
TProofPlayer.cxx, .h
- Remove a few unused includes
- Make TAdaptivePacketizer the default
- Improve comments
Revision
17585 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jan 30 16:34:54 2007 UTC (7 years, 11 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 73125 byte(s)
Diff to
previous 17573
From Gerri:
- call TSystem::DispatchOneEvent() via an asynchronous timer
(period 1s) instead of at the end of each processed event;
this reduces considerably the overhead when processing many
CPU-light events
- remove some left-over hardcoded settings of the feedback period
- cleanup the stop/abort timer infrastructure
Revision
17175 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Dec 12 11:20:49 2006 UTC (8 years, 1 month ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 71303 byte(s)
Diff to
previous 16955
From Jan Iwaszkiewicz:
A prototype for a new packetizer is added - TAdaptivePacketizer.
This packetizer is based on TPacketizer but uses different
load-balancing algorithms and data structures. Two main improvements in
the load-balancing strategy:
- First one was to change the order in which the files are assigned
to the computing nodes in such a way that network transfers are
evenly distributed in the query time. Transfer of the remote files
was often becoming a bottleneck at the end of a query.
- The other improvement is the use of time-based packet size. We
measure the processing rate of all the nodes and calculate the
packet size, so that it takes certain amount of time. In this way
packetizer prevents the situation where the query can't finish
because of one slow node.
The data structures: TFileStat, TFileNode and TSlaveStat are
enriched + changed and TFileNode::Compare method is changed.
Revision
16955 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 27 14:14:24 2006 UTC (8 years, 1 month ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 71270 byte(s)
Diff to
previous 16870
From Gerri:
This is the patch to move TDSet and TChainProof in 'proof' and remove
any dependence on PROOF in 'tree'. I have renamed TChainProof as TProofChain,
for consistency. TProofChain is now deriving from TChain and overrides only
the required methods. It is loaded via PluginManager by TChain in SetProof.
TQueryResult stores now all the input-related objects in fInputList. There
is a new method
TObject *TQueryResult::GetInputObject(const char *classname)
to retrieve the first instance of a class from the input list, so for the
TDSet:
TQueryResult *qr;
TDSet *dset = (TDSet *) qr->GetInputObject("TDSet")
The changes in TSessionViewer are needed to adapt to this new way of
accessing TDSets.
Revision
16775 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Nov 15 17:45:55 2006 UTC (8 years, 2 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 71428 byte(s)
Diff to
previous 16727
From Gerri and Bertrand:
Fixes and new features developed in preparation for the Intel quad-core
launch demo:
- Add instantaneous event and MB processing rates in the Progress signal
and make necessary modification to exploit it in the Progress Dialog.
- Add possibility to change the Progress and Feedback periods
- A number of small improvements and bug fixes
Revision
15956 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Aug 8 21:27:14 2006 UTC (8 years, 5 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 69159 byte(s)
Diff to
previous 15944
From Gerri:
the attached small patch adds the TNamed in question. After the patch, this
TNamed *nm = (TNamed *) fInput->FindObject("PROOF_QueryTag");
if (nm) {
Printf("Found query tag: %s", nm->GetTitle());
} else {
Printf("Query tag not found");
}
should return the query tag in SlaveBegin or Terminate.
The ID is created on the master so it cannot be available in Begin.
Revision
15935 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Aug 6 07:15:01 2006 UTC (8 years, 5 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 68893 byte(s)
Diff to
previous 15930
Handle several issues related to the TTree::Process() and TTree::Draw()
return values:
- TSelector: fStatus increased from Int_t to Long64_t. It is used to
return the number of selected events by TTree::Draw() which returns
a Long64_t.
- TVirtualProof, TProof, TDSet, TProofPlayer: Process() and Draw() were
returning an Int_t, now a Long64_t to be consistent with TTree/TChain.
Actually returning of TSelector::GetStatus() from the workers to be
done tomorrow.
Also added proper comments describing the return values for Draw()
(number of selected events), and Process() (value returned by
TSelector::GetStatus()).
- TTree, TChain, TSelectorDraw: added proper comments describing the
return values of Draw() and Process().
Revision
15930 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Aug 5 20:04:47 2006 UTC (8 years, 5 months ago) by
brun
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 68646 byte(s)
Diff to
previous 15928
From Gerri:
I have realized that the new tarball requires the mods to the XrdProofd plugin
that I wanted to keep separated. This explains the problem on Mac (and also
everywhere the plug-in is build).
It also contains the fix of the problem mention by Fons with selectors from a library.
Revision
15857 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 26 14:28:59 2006 UTC (8 years, 6 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 68270 byte(s)
Diff to
previous 15849
From Gerri:
TVirtualProof.h
- Add abstract methods methods to communicate the data set status
(signal + sender)
pmain.cxx
- Add call to Exit(0). T(X)ProofServ::Terminate() just interrupt the event
loop (insteadof exiting) to avoid problems when the terminate is run in
a separate thread (like in xproofd).
proof/inc/TProof.h
- Add methods methods to communicate the data set status (signal + sender)
- Increase protocol number to 11 (new merging technology)
proof/inc/TProofPlayer.h, .cxx
- Implement new merging technology for the output list to reduce memory
consumption: add methods to directly merge received objects in the
output list
- Polish cleaning of the various temporary lists and objects
TProofServ.h, .cxx
- Implement new merging technology for the output list to reduce memory
consumption: objects are sent one-by-one
- Do not scan for previous query results at Setup (do it when the client
requests for it first); thsi allows to speed up start-up when a large number
of queries is present and also allos to avoid warning due dictionaries
not yet loaded.
- Add method for a global cleanup of old query results.
- Keep in memory only meta information about processed queries (i.e. not the
output list) to avoid large memory consumption in case of large
output lists
- In Terminate, change the call to Exit(0) to ExitLoop() ; Exit(0) is
called in pmain.cxx .
TProof.cxx
- Implement new merging technology for the output list to reduce memory
consumption: objects are received one-by-one and directly merged in the
output list
- Add protection in parallel startup against invelid slaves.
- Remove check on directory existence in UploadDataSet.
- Add methods to handle notifications from data set verification
- Fix inconsistency with notification in activation/deactivation
of non-existing workers
TXProofServ.cxx
- In Terminate, change the call to Exit(0) to ExitLoop() ; Exit(0) is
called in pmain.cxx.
TQueryResult.cxx
- Make a owned copy of the input list (instead of just storing the pointer) to
allow proper cleaning.
Revision
15849 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 26 13:36:44 2006 UTC (8 years, 6 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 60216 byte(s)
Diff to
previous 15690
Instead of providing default implementations for copy ctor's and assignment
operators we better make them private (and not implemented) to avoid people
from accidentally using them. Especially the collections classes and the
TQObject derived classes. In these classes the default implementations would
cause havoc due to multiple deletions of the same objects and other
potential memory corruptions.
Revision
15690 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jul 4 23:35:37 2006 UTC (8 years, 6 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 60698 byte(s)
Diff to
previous 15649
Some new features in TSelector and derivatives:
- new method Abort() which allows Process() to be aborted by the user
from within the selector. After Abort(), either the Process() is aborted
or the current file is aborted and the next file in a chain will be
processed. Abort() can also be called from Begin(), SlaveBegin(),
Init() and Notify() to interrupt processing. In all cases SlaveTerminate()
and Terminate() are still called.
- new method GetEntry() which is a wrapper around:
fChain->GetTree()->GetEntry()
- GetStatus() now also implemented in TSelectorCint so the selector
fStatus is correctly reported back by TTree::Process() also for
interpreted selectors (also documented in the generated selector).
- documented in the generated selector code that the boolean return
values of Notify() and Process() are not used.
- fixed a bug in TProofPlayer() that caused Init() and Notify() to be
called for every packet and not once per file.
Revision
15649 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Jul 1 12:05:49 2006 UTC (8 years, 6 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 60730 byte(s)
Diff to
previous 15507
From Maarten:
- Rename TPacketizer2 to TPacketizer
- Introduce a new development version named TPacketizerDev
- Implement packetizer selection via the PROOF input list.
The packetizer is selected by adding a object like:
TNamed("PROOF_Packetizer","TMyPacketizer")
Revision
15507 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 21 16:18:27 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 59941 byte(s)
Diff to
previous 15348
From Gerri:
The patch implements the following new functionality:
- Shutdown of idle sessions when quitting ROOT; if not idle the session
is shutdown as soon as processing terminates; it is possible to delay
the automatic shutdown by a tunable amount of time via a new directive
in the xrootd config file 'xpd.shutdown' (see xpd.cf.example).
- New static method TProof::Reset(const char *url) to hard-cleanup an
user area (implemented for Xproofd only); this implied quite some
additions in XrdProofdProtocol.
- Possibility to temporary remove / re-insert a worker from the active list
TProof::DeactivateWorker(const char *ord)
TProof::ActivateWorker(const char *ord)
with 'ord' the ordinal numer, e.g. "0.3" .
- A default url for the disk pool
const char *TVirtualProof::GetDataPoolUrl()
It is set by default to a string send back by the master, but it can
be changed via
void TVirtualProof::SetDataPoolUrl(const char *url)
Jan should then adapt his code to use this as default.
- Posibility to timeout Collect; for now this is only used at forced
termination.
The patch includes many small changes and bug fixes. One of the most important
is in TXNetSystem (a bug preventing correct treatment of multiple request
to the same redirector).
Revision
15312 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 2 15:14:35 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 54424 byte(s)
Diff to
previous 15138
From Gerri:
large PROOF patch. Main issues addressed:
- Output redirection to log file at 'proofserv' startup (instead of
after Setup())
- Log messages activated at session creation
- Session sandbox restructuring in such a way that the unique session
tag, created by the top master, uniquely identifies the session on
all nodes, for example on top master:
<User_Sandbox>/session-<top_master_session_tag>/
master-0-<top_master_session_tag>/master-0-<top_master_session_tag>.log
On worker 0.0
<User_Sandbox>/session-<top_master_session_tag>/
worker-0.0-<worker_0.0_session_tag>/worker-0.0-<worker_0.0_session_tag>.log
- Automatic creation of the dataset dir
- Improved handling of error conditions
- Reorganization of TProofServ and XrdProofdProtocol
- Several bug fixes, in particular in XrdProofdProtocol
Revision
14745 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 19 08:22:26 2006 UTC (8 years, 9 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 53399 byte(s)
Diff to
previous 14337
Change the TError.h macros:
Assert -> R__ASSERT
Check -> R__CHECK
Change the TCollection.h macro:
ForEach -> R__FOR_EACH
This to avoid potential problems due too trivial macro names.
The old macros will be removed in the next release. Currently
they will print out warning messages with the advice to move
to the new macro names.
Revision
14330 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Mar 20 21:26:55 2006 UTC (8 years, 10 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 53373 byte(s)
Diff to
previous 13195
From Gerri:
patch to fix the problem of running Finalize on the results
of a query processed with a selector fetched from a shared lib.
It works fine if the lib is in the current dynamic lib path; otherwise
it prints an error message suggesting the user to load the appropriate library.
Revision
12829 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Sep 24 11:33:41 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 50705 byte(s)
Diff to
previous 12822
From Gerri:
This is the patch for Marek's problem.
Basically, ReinitSelector() does not need to be called if running in
synchronous mode (new argument to TProofPlayer::Finalize()), and the aclic
mode must be recorded and used when reinitialization is needed.
It remains the problem with ::GetSelector that I mentioned in the
previous mail, which affects, for example, cases when one runs once
with "<selec>.C+", modifies <selec>.C and runs again forgetting the
'+'.
Revision
12822 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Sep 22 23:29:30 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 50351 byte(s)
Diff to
previous 12733
from Gerri:
- fix several issues with Begin() not being called when finalizing a query
done via TChain::Draw(), call setup code now also in Terminate().
- fix error with query progress dialog not finishing when analysing
subset of data set.
- add master and slave startup feedback code.
- fixes also a few additional problems with enabling/disabling of
Stop/Abort buttons and cleans up the treament of fOutputList in TQueryResult.
It is also make sure that there are no multiple copies of a TDSet in the
output list.
Revision
12732 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Sep 17 14:57:46 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 49592 byte(s)
Diff to
previous 12728
From Gerri:
By trying with a completely new proof.conf I also found another thing not
very solid in TProof::StartSlaveThread (the check to distinguish between
slave and master was done on something not always set correctly).
Finally, "config.h" was not included in TProof.cxx, so if ROOTLIBDIR was
not standard, libThread was not found ...
Revision
12722 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Sep 16 08:48:39 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 49747 byte(s)
Diff to
previous 12564
From Gerri:
Big patch supporting query queueing on the PROOF master, interogation and
interaction with the queue. Retrieval, finalization and archival of query
results and multiple PROOF sessions. Some other fixes:
- TMacro.h, .cxx
add method Checksum to get the MD5 of the current context
(needed for optimized selector reinitialization)
- Changes needed to create and fill the list of data sets (TROOT,
TDSet, TChain, TChainProof)
- A new method TSelector::IsStandardDraw() to check if a selector
is for standard draw actions
- TEventIter: fix for the missing last update (see above)
- Few protections in TPacketizers destructors
- Fix for code conventions in TPerfStats (the other will follow)
- Occasional missing last update of the progress bar (systematically
reproduced with SetParallel(1)): all this part is quite involved
and I have found a solution which does not harm but perhaps is not
the most elegant.
- SetParallel(0): the progress bar was not working at all in such a
case, since no packetizer is involved. Still it is useful to see
that things are not stuck, so I have added some progress messages
there too. The problem is that I have not found a way to get the total
number of entries from a data set without doing all the machinery
done in the packetizer.
Revision
12561 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Aug 30 10:25:29 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 39620 byte(s)
Diff to
previous 12284
From Gerri:
This is the patch to add support for asynchronous queries.
The principal modifications are in TProof and TProofPlayer.
In TProof, I have separated the parsing of the received messages
from the selection of the socket (new method CollectInputFrom()):
the new method is called directly by TProofInputHandler::Notify()
in the case of asynchronous mode.
Also in TProof I have added a method to redirect the logs to a
temporary file, to avoid having them on the main window while doing
something else. The method is TProof::RedirectLog(), it uses freopen,
fdopen and dup2, and works well back and forth.
The content of the temp file can be displayed into a log window
(new GUI class TProofProgressLog), via a new button in the
progress dialog ("Show Logs").
In TProofPlayer I have created a new method Finalize() with the
final part of Process(): this is necessary to be able to finalize
the queries processed asychronously.
There are several many other small changes and fixes: a couple
of non-backward comaptibilities introduced with the previous
patch, many small issues in the GUI (the stop and abort buttons
were never disabled ...).
Also, in view of multi-sessions, it does not make much sense to
have one progress dialog per query: one per session is enough;
I have added a method ResetProgressDialog() to reset it when a
new query is started.
Finally, as anticipation of the next patch, this patch includes
support for a list of queries created by the master and described
by the new class TProofQuery.
The list is displayed by TProof::ShowQueries().
Revision
12123 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 22 20:18:12 2005 UTC (9 years, 7 months ago) by
brun
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 37439 byte(s)
Diff to
previous 11988
From Constantin Loizides
This patch implements:
- decentralized, automatic mutex initialization,
see R__LOCKGUARD2 in TVirtualMutex
- PROOF parallel startup fixes
- more thread protection (in base, cont, meta, rest to be done)
- cleanups
From Eddy Offermann:
TString::Atoi and Atof are made const.
Revision
11373 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Mar 18 22:41:27 2005 UTC (9 years, 10 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 35888 byte(s)
Diff to
previous 11361
From Marek:
- TH1::Merge() adding overflows even when merging histograms with the
same limits. If the limits are different the overflows are ignored.
The same in TH2::Merge, TH3::Merge, TProfile::Merge and TProfile2D::Merge.
- Cleaning fHistogram (and this kind of) variables, so that the drawn
plots aren't deleted in the TProofDrawXXX's destructor.
- Removed TProofNTuple from TProofDrawTGraph.
- Put the initialization code of fFeedback and fPlayer (=MakePlayer())
in TProof in to Init().
- Added TGraph::Merge(). Added TPolyMarker3D::GetPoint(Int_t, Float&,
Float&, Float&).
- Removed TProofNTuple from TProofDrawPolyMarker3D.
- Fixed cleaning output lists after processing (set option = "nodelete").
Now the objects are on the player outputlist and are removed (but not
deleted) from the selector's outputlist.
- Fixed TPolyMarker3D::Merge() - it seems that nobody has ever tried to
run this function.
- Added setting tree attributes (like MarkerStyle, MarkerSize, LineColor etc.)
to a drawn TGraph and TPolyMarker3D in TProofDrawGraph and
TProofDrawPolyMarker3D.
Revision
11281 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Mar 10 17:57:05 2005 UTC (9 years, 10 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 35968 byte(s)
Diff to
previous 11264
From Marek Biskup:
major new PROOF developments:
- access to PROOF directly via the TChain interface (creating a TDSet
in the background).
- access to PROOF based trees via the TTreeViewer
- fully implemented the TChain/TDSet::Draw() interface.
- PROOF sessions can now be browsed via the TBrowser
Revision
11047 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 7 18:02:37 2005 UTC (9 years, 11 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 32719 byte(s)
Diff to
previous 10850
From Maarten and Kristjan Gulbrandsen:
Here is the patch with the restructured PROOF code. It parameterizes
PROOF startup, currently implementing "Standard", Condor, MLM and PEAC.
It can easily be extended with other "TProof" classes, e.g. AliEn Proof.
Or what ever next Grid implementation or scheduler comes along.
The MLM, Multi Level Master, code uses a basic packetizer, based on a
MSD (massstorage domain) tag. In MLM mode we have multi-tier client
server setup, like:
client -> super master -> sub master 0 -> slave[0..N]
...
-> sub master M -> slave[0..N]
It also contains a large number of smaller fixes. Credit for most of
the work goes to Kris!
We have tested this on multipe platforms (RH7.3, RH9, SL3) and in
multiple configurations. Should be stable.
Revision
10608 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Nov 24 07:41:32 2004 UTC (10 years, 2 months ago) by
brun
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 23717 byte(s)
Diff to
previous 9615
From Maarten Ballintijn and Kristj�n Gulbrandsen
a patch and files that complete the benchmark package.
I strongly recommend using this for demos rather then the H1 example.
The README file should walk you through a complete demo, which should
not take more then maybe ten minutes or so. After you have some
data files generated, you can show significant parts of the sytem
in a minute or two.
I added a feedback request api to TProof, this simplifies things
a lot from a usability perspective. I also modified TDrawFeedback
to use gProof by default, again simplifying things.
Kris wrote the support scripts and fixed the benchmark
scripts to read the full Events.
Please try it out and have fun!
It is as simple as this (I use log level 2 to slow things
down a bit :-))
gROOT->Proof()
.L make_tdset.C
TDSet *d= make_tdset("/data1/tmp",2)
TDrawFeedback fb
gProof->AddFeedback("PROOF_ProcTimeHist")
gProof->AddFeedback("PROOF_LatencyHist")
gProof->AddFeedback("PROOF_EventsHist")
gProof->UploadPackage("event.par")
gProof->EnablePackage("event")
gEnv->SetValue("Proof.StatsHist",1);
gProof->SetLogLevel(2)
d->Process("EventTree_Proc.C")
Revision
9615 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jul 29 10:54:55 2004 UTC (10 years, 5 months ago) by
brun
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 23676 byte(s)
Diff to
previous 9459
*****************WARNING*****************
With this mega patch, we introduce support for TTree/TChain with more than
2 billion entries. Several class data members have been changed from
Int_t (or Stat_t) to Long64_t.
==>Trees written with this new version cannot be read with older versions
******************************************
TBranch:
========
- replace the members with type Int_t or Stat_t by Long64_t
Long64_t fEntryNumber; // Current entry number (last one filled in this branch)
Long64_t fReadEntry; //! Current entry number when reading
Long64_t fEntries; // Number of entries
Long64_t fTotBytes; // Total number of bytes in all leaves before compression
Long64_t fZipBytes; // Total number of bytes in all leaves after compression
Long64_t *fBasketEntry; //[fMaxBaskets] Table of first entry in eack basket
- corresponding changes in the member functions
virtual Int_t GetEntry(Long64_t entry=0, Int_t getall = 0);
virtual Int_t GetEntryExport(Long64_t entry, Int_t getall, TClonesArray *list, Int_t n);
Int_t GetEvent(Long64_t entry=0) {return GetEntry(entry);}
Long64_t *GetBasketEntry() const {return fBasketEntry;}
Long64_t GetReadEntry() const {return fReadEntry;}
Long64_t GetTotalSize() const;
Long64_t GetTotBytes() const {return fTotBytes;}
Long64_t GetZipBytes() const {return fZipBytes;}
Long64_t GetEntryNumber() const {return fEntryNumber;}
Long64_t GetEntries() const {return fEntries;}
TBranch::Streamer has been modified to read old files and automatically translate
the old types to the new types.
The new version of Streamer uses the TClass::ReadBuffer/WriteBuffer.
TBranch::Print has been modified to take into account the new data types.
ClassDef version increased to 10.
TBranchClones:
==============
virtual Int_t GetEntry(Long64_t entry=0, Int_t getall = 0);
TBranchElement:
===============
Int_t GetEntry(Long64_t entry=0, Int_t getall = 0);
TBranchObject:
==============
virtual Int_t GetEntry(Long64_t entry=0, Int_t getall = 0);
TChain:
=======
Long64_t *fTreeOffset; //[fTreeOffsetLen]Array of variables
virtual Int_t Add(const char *name, Long64_t nentries=kBigNumber);
virtual Int_t AddFile(const char *name, Long64_t nentries=kBigNumber);
virtual Long64_t Draw(const char *varexp, const TCut &selection, Option_t *option=""
,Long64_t nentries=kBigNumber, Long64_t firstentry=0);
virtual Long64_t Draw(const char *varexp, const char *selection, Option_t *option=""
,Long64_t nentries=kBigNumber, Long64_t firstentry=0); // *MENU*
virtual Long64_t GetChainEntryNumber(Long64_t entry) const;
virtual Long64_t GetEntries() const;
virtual Int_t GetEntry(Long64_t entry=0, Int_t getall=0);
Long64_t LoadTree(Long64_t entry);
virtual void Loop(Option_t *option="",Long64_t nentries=kBigNumber, Long64_t firstentry=0); // *MENU*
virtual Long64_t Merge(const char *name);
virtual Long64_t Merge(TCollection *list);
virtual Long64_t Merge(TFile *file, Int_t basketsize, Option_t *option="");
virtual Long64_t Process(const char *filename,Option_t *option="", Long64_t nentries=kBigNumber, Long64_t firstentry=0); // *MENU*
virtual Long64_t Process(TSelector *selector,Option_t *option="", Long64_t nentries=kBigNumber, Long64_t firstentry=0);
ClassDef version changed from 4 to 5
TSelector
=========
virtual Bool_t ProcessCut(Long64_t /*entry*/) { return kTRUE; }
virtual void ProcessFill(Long64_t /*entry*/) { }
virtual Bool_t Process(Long64_t /*entry*/) { return kFALSE; }
TSelectorCint
=============
virtual Bool_t ProcessCut(Long64_t entry);
virtual void ProcessFill(Long64_t entry);
virtual Bool_t Process(Long64_t entry);
TSelectorDraw
=============
Long64_t fDraw; //! Last entry loop number when object was drawn
Long64_t fSelectedRows; // Number of selected entries
Long64_t fOldEstimate; // value of Tree fEstimate when selector is called
Double_t *fV1; //![fSelectedRows]Local buffer for variable 1
Double_t *fV2; //![fSelectedRows]Local buffer for variable 2
Double_t *fV3; //![fSelectedRows]Local buffer for variable 3
Double_t *fV4; //![fSelectedRows]Local buffer for variable 4
Double_t *fW; //![fSelectedRows]Local buffer for weights
virtual Long64_t GetSelectedRows() const {return fSelectedRows;}
virtual Bool_t Process(Long64_t /*entry*/) { return kFALSE; }
virtual void ProcessFill(Long64_t entry);
virtual void ProcessFillMultiple(Long64_t entry);
virtual void ProcessFillObject(Long64_t entry);
virtual void SetEstimate(Long64_t n);
TTree
=====
Modified data types
Long64_t fEntries; // Number of entries
Long64_t fTotBytes; // Total number of bytes in all branches before compression
Long64_t fZipBytes; // Total number of bytes in all branches after compression
Long64_t fSavedBytes; // Number of autosaved bytes
Long64_t fMaxEntryLoop; // Maximum number of entries to process
Long64_t fMaxVirtualSize; // Maximum total size of buffers kept in memory
Long64_t fAutoSave; // Autosave tree when fAutoSave bytes produced
Long64_t fEstimate; // Number of entries to estimate histogram limits
Long64_t fChainOffset; //! Offset of 1st entry of this Tree in a TChain
Long64_t fReadEntry; //! Number of the entry being processed
Long64_t fTotalBuffers; //! Total number of bytes in branch buffers
Long64_t fDebugMin; //! First entry number to debug
Long64_t fDebugMax; //! Last entry number to debug
New function signatures
virtual Long64_t AutoSave(Option_t *option="");
virtual TTree *CloneTree(Long64_t nentries=-1, Option_t *option="");
virtual Long64_t CopyEntries(TTree *tree, Long64_t nentries=-1);
virtual TTree *CopyTree(const char *selection, Option_t *option=""
,Long64_t nentries=1000000000, Long64_t firstentry=0);
virtual Long64_t Draw(const char *varexp, const TCut &selection, Option_t *option=""
,Long64_t nentries=1000000000, Long64_t firstentry=0);
virtual Long64_t Draw(const char *varexp, const char *selection, Option_t *option=""
,Long64_t nentries=1000000000, Long64_t firstentry=0); // *MENU*
virtual Long64_t Fit(const char *funcname ,const char *varexp, const char *selection="",Option_t *option="" ,Option_t *goption=""
,Long64_t nentries=1000000000, Long64_t firstentry=0); // *MENU*
virtual Long64_t GetChainEntryNumber(Long64_t entry) const {return entry;}
virtual Long64_t GetChainOffset() const { return fChainOffset; }
Long64_t GetDebugMax() const {return fDebugMax;}
Long64_t GetDebugMin() const {return fDebugMin;}
virtual Long64_t GetEntries() const {return fEntries;}
virtual Long64_t GetEntriesFast() const {return fEntries;}
virtual Long64_t GetEntriesFriend() const;
virtual Long64_t GetEstimate() const { return fEstimate; }
virtual Int_t GetEntry(Long64_t entry=0, Int_t getall=0);
Int_t GetEvent(Long64_t entry=0, Int_t getall=0) {return GetEntry(entry,getall);}
virtual Long64_t GetEntryNumberWithBestIndex(Int_t major, Int_t minor=0) const;
virtual Long64_t GetEntryNumberWithIndex(Int_t major, Int_t minor=0) const;
virtual Long64_t GetEntryNumber(Long64_t entry) const;
virtual Long64_t GetMaxEntryLoop() const {return fMaxEntryLoop;}
static Long64_t GetMaxTreeSize();
virtual Long64_t GetMaxVirtualSize() const {return fMaxVirtualSize;}
virtual Long64_t GetReadEntry() const {return fReadEntry;}
virtual Long64_t GetReadEvent() const {return fReadEntry;}
virtual Long64_t GetSelectedRows() {return GetPlayer()->GetSelectedRows();}
virtual Long64_t GetTotBytes() const {return fTotBytes;}
virtual Long64_t GetZipBytes() const {return fZipBytes;}
virtual Long64_t LoadTree(Long64_t entry);
virtual Long64_t LoadTreeFriend(Long64_t entry, TTree *T);
virtual Long64_t Merge(TCollection *list);
TPrincipal *Principal(const char *varexp="", const char *selection="", Option_t *option="np"
,Long64_t nentries=1000000000, Long64_t firstentry=0);
virtual Long64_t Process(const char *filename,Option_t *option="", Long64_t nentries=1000000000, Long64_t firstentry=0); // *MENU*
virtual Long64_t Process(TSelector *selector, Option_t *option="", Long64_t nentries=1000000000, Long64_t firstentry=0);
virtual Long64_t Project(const char *hname, const char *varexp, const char *selection="", Option_t *option=""
,Long64_t nentries=1000000000, Long64_t firstentry=0);
virtual TSQLResult *Query(const char *varexp="", const char *selection="", Option_t *option=""
,Long64_t nentries=1000000000, Long64_t firstentry=0);
virtual Long64_t Scan(const char *varexp="", const char *selection="", Option_t *option=""
,Long64_t nentries=1000000000, Long64_t firstentry=0); // *MENU*
virtual void SetDebug(Int_t level=1, Long64_t min=0, Long64_t max=9999999); // *MENU*
virtual void SetEntries(Long64_t n);
virtual void SetEstimate(Long64_t nentries=10000);
virtual void SetMaxEntryLoop(Long64_t maxev=1000000000) {fMaxEntryLoop = maxev;} // *MENU*
static void SetMaxTreeSize(Long64_t maxsize=1900000000);
virtual void SetMaxVirtualSize(Long64_t size=0) {fMaxVirtualSize = size;} // *MENU*
virtual void Show(Long64_t entry=-1, Int_t lenmax=20);
virtual Long64_t UnbinnedFit(const char *funcname ,const char *varexp, const char *selection="",Option_t *option=""
,Long64_t nentries=1000000000, Long64_t firstentry=0);
TTree::Streamer has been modified to real old files.
TTree::Print has been modified to take into account the new data types.
ClassDef version number incremented to 13.
TVirtualTreePlayer
==================
virtual TTree *CopyTree(const char *selection, Option_t *option=""
,Long64_t nentries=1000000000, Long64_t firstentry=0) = 0;
virtual Long64_t DrawScript(const char* wrapperPrefix,
const char *macrofilename, const char *cutfilename,
Option_t *option, Long64_t nentries, Long64_t firstentry) = 0;
virtual Long64_t DrawSelect(const char *varexp, const char *selection, Option_t *option
,Long64_t nentries, Long64_t firstentry) = 0;
virtual Long64_t Fit(const char *formula ,const char *varexp, const char *selection,Option_t *option ,Option_t *goption
,Long64_t nentries, Long64_t firstentry) = 0;
virtual Long64_t GetSelectedRows() const = 0;
virtual TPrincipal *Principal(const char *varexp="", const char *selection="", Option_t *option="np"
,Long64_t nentries=1000000000, Long64_t firstentry=0) = 0;
virtual Long64_t Process(const char *filename,Option_t *option="", Long64_t nentries=1000000000, Long64_t firstentry=0) = 0;
virtual Long64_t Process(TSelector *selector,Option_t *option="", Long64_t nentries=1000000000, Long64_t firstentry=0) = 0;
virtual Long64_t Scan(const char *varexp, const char *selection, Option_t *option
,Long64_t nentries, Long64_t firstentry) = 0;
virtual TSQLResult *Query(const char *varexp, const char *selection, Option_t *option
,Long64_t nentries, Long64_t firstentry) = 0;
virtual void SetEstimate(Long64_t n) = 0;
virtual Long64_t UnbinnedFit(const char *formula ,const char *varexp, const char *selection,Option_t *option
,Long64_t nentries, Long64_t firstentry) = 0;
TTreeFormula
============
TLeaf* GetLeafWithDatamember(const char* topchoice, const char* nextchice, Long64_t readentry) const;
Bool_t BranchHasMethod(TLeaf* leaf, TBranch* branch,
const char* method,const char* params,
Long64_t readentry) const;
TTreePlayer
===========
Long64_t fSelectedRows; // Number of selected entries
virtual TTree *CopyTree(const char *selection, Option_t *option
,Long64_t nentries, Long64_t firstentry);
virtual Long64_t DrawScript(const char* wrapperPrefix,
const char *macrofilename, const char *cutfilename,
Option_t *option, Long64_t nentries, Long64_t firstentry);
virtual Long64_t DrawSelect(const char *varexp, const char *selection, Option_t *option
,Long64_t nentries, Long64_t firstentry);
virtual Long64_t Fit(const char *formula ,const char *varexp, const char *selection,Option_t *option ,
Option_t *goption ,Long64_t nentries, Long64_t firstentry);
virtual Long64_t GetSelectedRows() const {return fSelectedRows;}
TPrincipal *Principal(const char *varexp, const char *selection, Option_t *option
,Long64_t nentries, Long64_t firstentry);
virtual Long64_t Process(const char *filename,Option_t *option, Long64_t nentries, Long64_t firstentry);
virtual Long64_t Process(TSelector *selector,Option_t *option, Long64_t nentries, Long64_t firstentry);
virtual Long64_t Scan(const char *varexp, const char *selection, Option_t *option
,Long64_t nentries, Long64_t firstentry);
virtual TSQLResult *Query(const char *varexp, const char *selection, Option_t *option
,Long64_t nentries, Long64_t firstentry);
virtual void SetEstimate(Long64_t n);
virtual Long64_t UnbinnedFit(const char *formula ,const char *varexp, const char *selection,Option_t *option
,Long64_t nentries, Long64_t firstentry);
The generated code by TTreePlayer::MakeCode, MakeClass, MakeProxy
uses Long64_t instead of Int_t
TTreeViewer
===========
Long64_t Process(const char* filename, Option_t *option="", Long64_t nentries=1000000000, Long64_t firstentry=0); // *MENU*
void SetCurrentRecord(Long64_t entry);
THbookBranch
============
virtual Int_t GetEntry(Long64_t entry=0, Int_t getall=0);
virtual void SetEntries(Long64_t n) {fEntries=n;}
THbookTree
==========
virtual Int_t GetEntry(Long64_t entry=0, Int_t getall=0);
virtual void InitBranches(Long64_t entry);
virtual void SetEntries(Long64_t n);
TProofDraw
==========
virtual Bool_t Process(Long64_t /*entry*/);
TProofPlayer
============
virtual Long64_t Process(TDSet *set,
virtual Long64_t DrawSelect(TDSet *set, const char *varexp,
Long64_t Process(TDSet *set, const char *selector,
Long64_t DrawSelect(TDSet *set, const char *varexp,
Revision
9322 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 25 17:27:09 2004 UTC (10 years, 7 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 23388 byte(s)
Diff to
previous 9182
From Maarten:
- new class TParameter
- new class TDrawFeedback
- fixes to the feedback of performance histograms
- selection of the packetizer
- cleanups
Example commands:
gROOT->Proof()
... package setup ...
TDSet *d = make_tdset(2)
gEnv->SetValue("Proof.StatsTrace",1)
gEnv->SetValue("Proof.StatsHist",1)
TList *fb = new TList
fb->SetName("FeedbackList")
fb->Add(new TObjString("EventsHist"))
fb->Add(new TObjString("CpuTimeHist"))
TDrawFeedback *dfb = new TDrawFeedback((TProof*)gProof)
gProof->AddInput(fb)
d->Process("EventTree_Proc.C","")
Revision
9182 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Jun 13 16:26:36 2004 UTC (10 years, 7 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 22893 byte(s)
Diff to
previous 9063
From Maarten and Kristjan Gulbrandsen:
PROOF and file I/O performance monitoring framework. The TVirtualPerfStats
abstract class allows adding monitoring events in base classes (TFile and
TNetfile). The performance monitoring info allows detailed study of PROOF
performance (which files accessed by which slaves, latency, read/write times,
packet size, etc.). The result is send back to the client in the form of a
memory based TTree.
Remove TProofStats.h and TProofStats.cxx, .d, .o from proof/inc and src.
Revision
9063 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun May 30 23:14:18 2004 UTC (10 years, 7 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 23428 byte(s)
Diff to
previous 8962
From Maarten:
- Cleanup and refactor packetizer functionality into utility classes
- Fix bugs in round-robin distribution of work
- Fix bugs in processing sub-range of the TDSet
- Add "slaves per file node monitoring" histogram and trace event
- Rename TPacketizer2 into stable TPacketizer, in preparation
for further packetizer developement
Revision
7664 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Nov 26 21:48:27 2003 UTC (11 years, 2 months ago) by
brun
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 22095 byte(s)
Diff to
previous 7496
From Philippe:
This update makes any selector created by the TTreePlayer directly from a source file local to the current TTreePlayer
object (instead of being a static).
It also insures that TProofPlayer and TTreePlayer do not call delete for an object of
a class which library was unloaded (since the code is gone, this would lead to a core
dump).
Revision
5717 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Dec 2 18:50:12 2002 UTC (12 years, 1 month ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 18603 byte(s)
Diff to
previous 5692
mega patch to remove almost all compiler warnings on MacOS X where the
compiler is by default in pedantic mode (LHCb also like to use this option).
The following issues have been fixed:
- removal of unused arguments
- comparison between signed and unsigned integers
- not calling of base class copy ctor in copy ctor's
To be done, the TGeo classes where we get still many hundred warnings of
the above nature. List forwarded to Andrei.
Revision
4955 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 17 12:29:38 2002 UTC (12 years, 6 months ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 10841 byte(s)
Diff to
previous 4711
PROOF mods from Maarten. New TVirtualPacketizer that is the base class for
different packetizers. You want different packetizers for different
processing strategies (depending on files being available on disk or if
they need to be sequentially retrieved from mass storage, etc.).
Added global PROOF debug facility.
Revision
3699 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Added
Fri Jan 18 14:24:10 2002 UTC (13 years ago) by
rdm
Original Path:
trunk/proofplayer/src/TProofPlayer.cxx
File length: 5615 byte(s)
relocate some proof file to remove circular dependencies between the
treeplayer and proof shared libraries. Also rename TPlayer to TProofPlayer
et al. Also remove Begin() from TSelector, use Begin(0) instead. This to
be backward compatible with existing TSelector scripts that don't have
Begin() and therefor give warnings about Begin() being hidden in the base
class.
This form allows you to request diffs between any two revisions of this file.
For each of the two "sides" of the diff,
enter a numeric revision.
