Log of /trunk/proof/proof/src/TProof.cxx
Parent Directory
Revision
48992 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Mar 28 15:26:26 2013 UTC (21 months, 3 weeks ago) by
rdm
File length: 433434 byte(s)
Diff to
previous 48378
From Lifeng Sun:
The attached patchset fixes a bunch of typo in the source:
0001-succes-success.patch
0002-preceed-preced.patch
0003-informations-information.patch
0004-childs-children.patch
0005-avaliable-available.patch
0006-writeable-writable.patch
0007-comand-command.patch
0008-unkown-unknown.patch
0009-wierd-weird.patch
0010-wheter-whether.patch
0011-unecessary-unnecessary.patch
0012-splitted-split.patch
0013-registerd-registered.patch
0014-recieve-receive.patch
0015-processsing-processing.patch
0016-ouput-output.patch
0017-mutiple-multiple.patch
0018-lenght-length.patch
0019-interupted-interrupted.patch
0020-independant-independent.patch
0021-inconsistant-inconsistent.patch
0022-expresion-expression.patch
0023-explicitely-explicitly.patch
0024-enviroment-environment.patch
0025-deafult-default.patch
0026-continous-continuous.patch
0027-completly-completely.patch
0028-commited-committed.patch
0029-choosen-chosen.patch
0030-backgroud-background.patch
0031-auxilliary-auxiliary.patch
0032-authentification-authentication.patch
0033-appropiate-appropriate.patch
0034-an-other-another.patch
0035-environement-environment.patch
0036-targetting-targeting.patch
0037-suppported-supported.patch
0038-paramater-parameter.patch
Revision
48378 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jan 23 10:01:49 2013 UTC (2 years ago) by
ganis
File length: 433436 byte(s)
Diff to
previous 48106
From Dario, fixes for Coverity reports:
In TDataSetManagerAliEn/TAliEnFind: fixed Coverity issues 48630 to 58637:
- fixed a couple of minor memory leaks;
- fixed use of invalidated iterator;
- added copy ctor and op=
In TProof.cxx: fixed a misspelled method name in debug messages
Revision
48060 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Dec 15 18:53:16 2012 UTC (2 years, 1 month ago) by
ganis
File length: 433380 byte(s)
Diff to
previous 47832
From Dario: patch introducing an improved dataset management model where the PROOF (ROOT)
dataset manager is a light frontend to the experiment file catalogs; TDataSetManagerFile
is still used as local cache of the experiment information or to store the work-in-progress
status of the dataset manager daemon. This model is expected to solve the scalability issues
observed currently at AAFs.
The patch includes the new class TDataSetManagerAliEn with the first concrete implementation
of experiment catallog interface. The equivalent for ATLAS will follow.
Revision
47832 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Dec 4 19:37:56 2012 UTC (2 years, 1 month ago) by
ganis
File length: 430879 byte(s)
Diff to
previous 47573
Make sure that the fSlaves list contains always ALL workers, eventhose which did not initially startup; this way GetListOfSlaveInfos can be used to find out which workers had problems strating up
Revision
47528 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Nov 21 12:47:27 2012 UTC (2 years, 2 months ago) by
ganis
File length: 430662 byte(s)
Diff to
previous 47451
In TProof::EnablePackage, add possibility to choose the level of version compatibility check
to decide whether to re-build the package or not. Three options are possible: 'off', no check
with possible failure at TSystem::Load, 'on' check the ROOT version, 'svn' check the ROOT version
and the SVN revision (default is 'on'). The choice can be done via a special option to the
option string('chkv=off|on|svn' or 'checkversion=off|on|svn') or via the ROOTrc variable
'Proof.Package.CheckVersion' .
Revision
47451 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 19 15:30:43 2012 UTC (2 years, 2 months ago) by
ganis
File length: 427473 byte(s)
Diff to
previous 47444
- In TProofServ::ResolveKeywords, add new keywords <rver> for the ROOT version in integer for, e.g. 53403,
and <build> for {version, architecture, compiler} string in the form '53403_linuxx8664gcc_gcc46'.
- Make sure that TProofServ::ResolveKeywords is always called for global package directories; allows to
optimize package building when working with many versions.
Revision
47444 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 19 13:48:59 2012 UTC (2 years, 2 months ago) by
ganis
File length: 427426 byte(s)
Diff to
previous 47270
- In case of non-tty, do not print all progress messages during the merging phase
to avoid filling up the logs with useless info.
- Add member TProof::fTty and getter TProof::IsTty() to centrally control the tty-ness
of the session
Revision
47270 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Nov 14 15:29:51 2012 UTC (2 years, 2 months ago) by
ganis
File length: 427249 byte(s)
Diff to
previous 45849
In TProof::PrintProgress (used in batch mode) add notification of the estimated time left and, at the end, of the processing time; request from SFrame users
Revision
45849 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Sep 5 09:54:29 2012 UTC (2 years, 4 months ago) by
ganis
File length: 427050 byte(s)
Diff to
previous 45740
In VerifyDataSet, issue an error if Proof is in sequential mode and sequetial dataset verification has not been explicitely asked for
Revision
45740 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Aug 29 16:17:09 2012 UTC (2 years, 4 months ago) by
ganis
File length: 426706 byte(s)
Diff to
previous 45718
- Add support for data dirs on external servers, fully honouring URL options; this allows to
have usable paths registered in the final TFileCollection objects. One of the addressed use-case
is when local directories need to be accessed via the local xrootd server to preserve the namespace
enforced by xrootd; in such a case something like 'root://locahost//datadir?remote=1' needs to be
specified; the option 'remote=1' ensures that the final registered URL is still in 'root://' form
with 'localhost' replaced by the FQDN of the machine.
- Move the 'tests' argument to the stressProof function in the second position, to make it
more user friendly to run a specific test in the ROOT shell; CLI behevior is unchanged.
- In ProofSimple, add function to plot the histograms also in the case a dataset is created with the
worker files; the file are opened and the histograms merged on the fly.
- Fix potential segv at xproofd startup due to a double fclose in XrdProofdManager::ValidateLocalDataSetSrc
- Remove some redundant printing in TProof when handling output files.
- Make sure that when running TSelVerifyDataSet the save-to-file functionality is switched-off .
Revision
45718 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Aug 28 10:06:59 2012 UTC (2 years, 4 months ago) by
ganis
File length: 426631 byte(s)
Diff to
previous 45632
- Extend parallel dataset verification to PROOF-Lite
- Remove some duplication between TProofServ and TProofLite
Should solve problem observed with stressProof in CDash.
Revision
45367 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Aug 3 09:22:00 2012 UTC (2 years, 5 months ago) by
ganis
File length: 423673 byte(s)
Diff to
previous 45348
In TProofOutputFile, add a status bit to indicate that the file is the result of the automatic
save-to-file output handling. This allows a better interplay between client-side and server-side
settings.
In TProof::HandleOutputOptions, fix issue with local object saving.
Revision
45348 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Aug 2 12:23:54 2012 UTC (2 years, 5 months ago) by
ganis
File length: 422896 byte(s)
Diff to
previous 45283
Consolidation of the automatic output saving functionality
- Fix an issue with saving output from memory on the master
- Remove some duplications and fix several small bugs
Revision
45283 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 30 10:42:38 2012 UTC (2 years, 5 months ago) by
ganis
File length: 423053 byte(s)
Diff to
previous 45093
Patch automatizing the usage of the file-based technology to handle outputs.
After this patch PROOF users can define as an option to Process an output file for their results
and choose if the merging goes via file or in memory. User can also decide to save theit results
in a set of files identified by a name (dataset).
Administrators can force merging-via-file or can make it conditional to the resident memory exceeding
a given threshold.
The new options are documented at
http://root.cern.ch/drupal/content/handling-outputs
Since TProofOutputFile is now used by TProof, the class has been moved from proof/proofplayer
(libProofPlayer) to proof/proof (libProof).
The selector tutorials/proof/ProofSimple.h,.C has been modified to allo testing the new options,
and a new test has been added to stressProof.
Revision
45093 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jul 17 10:11:09 2012 UTC (2 years, 6 months ago) by
ganis
File length: 409937 byte(s)
Diff to
previous 44699
Add auxilliary functions to retrieve environment information from the nodes,
typically from the master (datadir or some env settings).
This is done by temporarly activating only the target actor by ordinal number, e.g. "0"
for master or "0.4" for the fifth worker.
Internally, saving/restore capabilities are added to activate/deactivate worker technology.
Also, a new Exec(cmd, ordinal) has been added to execute a command on a given actor only.
The following TProof functions have been added to get environment information:
TString Getenv(const char *env, const char *ord = "0");
Int_t GetRC(const char *RCenv, Int_t &env, const char *ord = "0");
Int_t GetRC(const char *RCenv, Double_t &env, const char *ord = "0");
Int_t GetRC(const char *RCenv, TString &env, const char *ord = "0");
Revision
44683 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 14 09:27:40 2012 UTC (2 years, 7 months ago) by
ganis
File length: 403299 byte(s)
Diff to
previous 44507
First version of the final patch enabling multi-workers on single node using
PROOF-Lite. This is supposed to improved scalability for large number of workers
(>300-400) and is currently available only for non-data driven tasks.
The new setup configuration is enabled by the directive 'xpd.remoteplite 1' in
the xproofd configuration file.
Revision
44507 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 4 12:30:41 2012 UTC (2 years, 7 months ago) by
axel
File length: 401223 byte(s)
Diff to
previous 44065
Remove
using namespace std;
from Riostream.h, which has huge consequences for all of ROOT.
Riostream.h is now a simple wrapper for fstream, iostream, iomanip for backward compatibility; Riosfwd.h simply wraps iosfwd.
Because of templates and their inline functions, Riostream.h needed to be included in headers, too (e.g. TParameter.h), which violated the assumption that Riostream.h is not exposing its using namespace std to headers.
ROOT now requires R__ANSISTREAM, R__SSTREAM, which does not change the set of supported compilers.
Without "using namespace std", several identifiers are now prefixed by std::; e.g. roofit/* source files now have a using namespace std to keep their coding style.
TFile::MakeProject() now generates "using namespace std" to convert the CINT-style class names into C++ ones.
Revision
44049 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue May 1 15:27:13 2012 UTC (2 years, 8 months ago) by
ganis
File length: 401182 byte(s)
Diff to
previous 44046
In TProof::LoadPackageOnClient, do not create a symlink 'pack_name' to the package dir,
but add directly the package dir to the include path. This solves the longstanding
annoying problem of failure when a directory or file with the name of the package did
already exist in the local working directory.
Revision
44046 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue May 1 11:34:04 2012 UTC (2 years, 8 months ago) by
ganis
File length: 401967 byte(s)
Diff to
previous 44024
Simplify enabling of basic feedback.
- In TDrawFeedback::Feedback, call method Draw() of objects not identified as TH1
derivation. This allows user-defined objects implementing Draw to be displayed
via this utility class (request on the forum).
- In TPerfStats, add new histogram TH1I with the packets being processed per worker;
currently this is 0 or 1 but may change in future; it shows which workers are
still active, which may be useful for debugging at the end of query.
- Add new class TStatsFeedback to display in one canvas the distributions for
the events processed per workers, the packets processed per workers and the
packets being processed per worker.
- In TProof::Process, add support for switches "fb=name1,name2,name3,... " or
"feedback=name1,name2,name3,... " in the option field. This enables feedback
for the specified objects, creating a TDrawFeedback object attached to the session.
Feedback for the specified objects is disabled at the end of the query and the
created TDrawFeedback is destroyed.
The special name 'stats' enables the three feedback histograms required by
TStatsFeedback, and uses a TStatsFeedback instead of TDrawFeedback.
The feedback specs are removed from the option field passed to TProofPlayer::Process.
Revision
43107 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Feb 24 12:45:22 2012 UTC (2 years, 10 months ago) by
ganis
File length: 398626 byte(s)
Diff to
previous 43074
From B. Butler:
Add the possibility to force submerging at node level, i.e. one submerger per physical machine.
In this way the network traffic can be minimized, for example when merging large output files.
The new feature is enabled by setting the Int_t parameter 'PROOF_MergersByHost' (or the directive
'Proof.MergersByHost') to a non-null value.
Revision
43074 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Feb 21 17:26:56 2012 UTC (2 years, 11 months ago) by
ganis
File length: 395609 byte(s)
Diff to
previous 43044
- Properly handle the error status code in TProof::SendFile.
- Make sure that errors from TProofServ::ReceiveFile are properly transmitted back.
Revision
43044 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Feb 17 14:34:52 2012 UTC (2 years, 11 months ago) by
ganis
File length: 395510 byte(s)
Diff to
previous 42882
From Taesoo Kim and me.
Add support for selector-by-object processing in PROOF.
The selector object, created and configured locally by the user, is added to the input
list and recuperated from there on the worker machines for processing. Any input list
setting in the selector itself is not streamed but temporarly moved to then standard
input list, so that user can use the selector input list as container of processing
information if they find convenient to do so.
Process(...) methods with the file name argument replaced by 'TSelector *' have
introduced where relevant (TProof, TProofPlayer and their derivatives, TDSet).
Two tests for the new functionality have been added to test/stressProof.cxx (tests 25
and 26).
Revision
42882 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 6 11:10:21 2012 UTC (2 years, 11 months ago) by
ganis
File length: 390613 byte(s)
Diff to
previous 42858
From Sangsu Ryu and me.
Add support for paralell dataset verification. This is implemented via a dedicated
TSelector (TSelVerifyDataSet) which is run over the list of files in the dataset via
TPacketizerFile. The file order is preserved using the recently introduced index in
TFileInfo.
Parallel verification is the now the default. For test and comparison purposes it is
still possible to verify datasets serially by specifying 'S' in the option field of
VerifyDataSet and RegisterDataSet .
To avoid/reduce code duplication some of the static functions in TDataSetManager have
refactorized. The overall behavior should be unchanged.
Revision
42858 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Feb 3 11:34:05 2012 UTC (2 years, 11 months ago) by
ganis
File length: 386200 byte(s)
Diff to
previous 42762
This patch adds in TProofMgr two static functions to upload files.
These functionare called UploadFiles and differ in the first argument, which is used
to pass the files to be uploaded. These can be given as a list (of TFileInfo or TObjString),
a directory or specified in a text file.
The functions are documented at
http://root.cern.ch/drupal/content/uploading-data-files-proof-cluster
The patch also adds TProofMgr::GetMssUrl() and TProofMgr::SetMssUrl(...) to manage the mass
storage URL attached to a given cluster. By the default the information is retrieved by the
server daemon, but can be modified by the user. The functions TProof::GetDataPoolUrl() and
TProof::SetDataPoolUrl(...) now redirect to TProofMgr::GetMssUrl() and TProofMgr::SetMssUrl(...).
The obsolete functions TProof::UploadDataset have now been removed.
This patch addresses some issues raised on Savannah.
Revision
42761 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jan 23 12:55:12 2012 UTC (3 years ago) by
ganis
File length: 395160 byte(s)
Diff to
previous 42382
- Fix problem with TProof::Load reported by Gordon Watts in the PROOF forum.
The additional files were not copied in the master sandbox but left in the
cache. A workaround for backward compatibility has also been implemented.
Revision
42382 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Dec 2 16:50:21 2011 UTC (3 years, 1 month ago) by
ganis
File length: 394604 byte(s)
Diff to
previous 41788
In TProof, add interface to simplify the creation of the performance tree:
- proof->SetPerfTree("<file>")
Set parameters to create the performance tree and saves it to '<file>'
(default <file> is 'perftree.root')
- proof->SavePerfTree("<file>", "<queryref>")
Saves the performance tree of query '<queryref>' (default the last one)
to '<file>' (default 'perftree.root').
The perfomance tree creation is deisble after each query, so it needs to be
enabled each time.
Revision
41788 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Nov 4 17:55:59 2011 UTC (3 years, 2 months ago) by
ganis
File length: 391198 byte(s)
Diff to
previous 41635
The file with input data must stay open during the whole processing,
otherwise retrieval of certain objects will fail (and crash).
Revision
40659 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Aug 22 16:45:04 2011 UTC (3 years, 5 months ago) by
ganis
File length: 390528 byte(s)
Diff to
previous 40192
- Decouple from registered TChains in already TProof::Close; allows to avoid possible crash
at exit ('.q') occuring after the recent revision of the socket cleanup policy
- Make sure that the dataset name is registered in the TFileInfo objects being processed, so
that it can be used for monitoring.
Revision
40017 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 27 16:23:01 2011 UTC (3 years, 6 months ago) by
ganis
File length: 389880 byte(s)
Diff to
previous 39967
- In TPerfStats, add 'status' field to the monitor record, an integer with the query exit status
(see TVirtualProofPlayer::EExitStatus (0 OK; 1 stopped; 2 aborted).
- Add exist status also to TStatus, so that it is available in the output list (TPerfStats takes
it from here).
- Send the output list, if non empty, also when the query is aborted, so that the TStatus object
is available also in those cases.
Revision
39967 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Jun 26 10:11:19 2011 UTC (3 years, 6 months ago) by
ganis
File length: 389710 byte(s)
Diff to
previous 39580
From Anar Manafov and me.
Patch introducing support for the protocol "pod://" in TProof::Open to automatically
address the entry point of the local PoD installation.
Tha patch also fixes an issue with the string used to start a PROOF-Lite session. So far
both "" and "lite" could be used; however this prevented the possibility to have a machine
name "lite" on the local domain as PROOF master. With this patch we require users to
specify "lite://" to start a PROOF-Lite session. A reminder is printed if "lite" is used
and connection fails.
The empty "" string uses the cluster specified by Proof.LocalDefault; by default the latter
is "lite://", so the deafult behaviour of "" is unchanged.
Revision
39580 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jun 7 17:17:57 2011 UTC (3 years, 7 months ago) by
ganis
File length: 388162 byte(s)
Diff to
previous 39402
Refine the part patch #37942 mapping 'localhost' to gSystem->HostName(): this is only needed
for master-to-worker. Doing it for client-to-master breaks the use of SSH tunnels.
Revision
39402 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu May 26 12:56:31 2011 UTC (3 years, 8 months ago) by
axel
File length: 388320 byte(s)
Diff to
previous 39115
Replace legacy clib/getline and editline by a new library, textinput,
located at core/textinput/src/textinput. That new library is written
in C++, standalone (no external dependencies), cross platform
including Windows, and contains only what we need - unlike
e.g. editline. It implements almost all bash-like editing commands,
color highlighting, history etc. It allows for multiple input objects
and multiple display objects, e.g. GUI and terminal in parallel. It
can be used as a stand-alone library, too, and will become part of
cling - which explains the different license and file names (.cpp).
core/textinputs/src/Getline* contains the adapters to ROOT; the
previous core/editline/enhance.cxx is transformed to
Getline_color.cxx. This new adapter interfaces with TTabCom directly,
without going via an extern "C" function pointer, to allow for an
ostream being passed that will contain all completion options, one per
line. TApplication now defines a virtual function to be called for tab
completion; TRint implements it using TTabCom.
One major change is that the user input string is accessible as const
char* instead of being a char* - the buffer length is unknown
anyway. This caused changes in callers of Getline(); the history and
the getline function types needed to change for CINT.
--enable-editline has been deprecated and is inoperable.
Revision
39115 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu May 5 15:54:41 2011 UTC (3 years, 8 months ago) by
ganis
File length: 388308 byte(s)
Diff to
previous 39074
- Fix bug in TXSocket::Create whose effect was to make the three additional attempts
following the first to send a wrong and meaningless creation info (ordinal number, etc).
The bug was introduced while fixing the log path trasmission problem.
- Improve the way information about the logs files is saved when failure happens
during creation. The log paths for these failing now should be now correctly saved
and accessible via TProofLog.
- Improve TSlave::Print when the connection is invalid (print all the valid information,
available, not only the header one)
Revision
39074 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue May 3 09:30:02 2011 UTC (3 years, 8 months ago) by
ganis
File length: 387752 byte(s)
Diff to
previous 38819
From S. Ryu:
- Fix issue affecting the result of subsequent worker activation/deactivation requests when worker
ordinal numbers were not in increasing order
From me:
- Add support for block requests for worker activation/deactivation (much more efficient, because
FindUniqueSlaves is only called once); the list of workers to have their statis changed is comma-
separated.
- Make DeactivateWorker and ActivateWorker to return != 0 when something went wrong, with -2 indicating
that at least one of the worker ordinal numbers in the list is invalid (e.g. '0.134' for a cluster
with 20 workers only).
Revision
38819 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 13 10:17:21 2011 UTC (3 years, 9 months ago) by
ganis
File length: 384832 byte(s)
Diff to
previous 38810
Introduce method GetSandbox to parse in a common way the directives defining
the sandbox location in TProof and TProofLite. Add expansion of "." and ".."
(it was not working with input data files).
Revision
38810 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Apr 12 16:22:59 2011 UTC (3 years, 9 months ago) by
ganis
File length: 383980 byte(s)
Diff to
previous 38760
Patch to correctly honour selector abort status settings in PROOF. Currently only
the TSelector::kAbortProcess was handled by stopping processing. In particular
TSelector::kAbortFile was ignored; this recently created some problems in ALICE
with corrupted files, with repeated attempts to read events eventually leading to
bad_alloc exceptions.
This patch also fixes other related issues, in particular with the reporting of the
non-processed {files, events} in the final 'MissingFiles' list. This list should
now account much more precisely of the number of events which could not be processed.
It also fixes a problem with the final update of the progress information affecting
occasionally cases with skipped events.
Revision
38572 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Mar 23 13:02:41 2011 UTC (3 years, 10 months ago) by
ganis
File length: 383136 byte(s)
Diff to
previous 38525
Add new derivation of TList (TProofOutputList) to be used on the PROOF client to filter out
PROOF internal objects when displaying or printing the list. By default objects was names start
with 'PROOF_' are not shown. The presence of a non empty missing file list is notified.
To list or print the full list the string "ALL" must be passed before any other ls() or print()
options, e.g.
root [] gProof->GetOutputList()->ls("<opt>")
show only user objects (i.e. apply name filtering); and
root [] gProof->GetOutputList()->ls("ALL<opt>")
show all objects according to '<opt>' (i.e. no additional filter applied).
Revision
38102 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Feb 16 18:26:13 2011 UTC (3 years, 11 months ago) by
ganis
File length: 383136 byte(s)
Diff to
previous 37976
Fix problem with enabling packages with option 'notOnClient' in PROOF-Lite (see forum):
- Implement dedicated TProofLite::FindUniqueSlaves, filling the unique worker list with
the first worker
- In TProof::BuildPackage, always issue the build request if 'notOnClient' is set.
Revision
37976 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Feb 4 11:43:16 2011 UTC (3 years, 11 months ago) by
ganis
File length: 383098 byte(s)
Diff to
previous 37942
In TProofLite::Load
- Extend to this overload support for a comma-separated list of macros (cfr #37942)
In TSlave and related classes
- Add member for the unique session tag
- Fix worker name in TSlaveLite
Revision
37942 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Feb 2 14:32:01 2011 UTC (3 years, 11 months ago) by
ganis
File length: 383025 byte(s)
Diff to
previous 37894
- Make sure that the machine names are consistent when working on a local machine
('localhost' or 'localhost.localdomain' are mapped to gSystem->HostName()); solves
possible matching problems in the packetizer.
- In TProof::Load, add the possibility to define a list of files to be sent because
needed by the main macro file. The list is comma-separated and the first file is
considered as the main one. For example
proof->Load("<macropath>/mymacro.C+,<thispath>/thisheader.h,<thatpath>/thatheader.h")
will make sure that the files 'thisheader.h' and 'thatheader.h', needed by 'mymacro.C'
are available in the sandbox on the worker machines. Note that 'thisheader.h' and
'thatheader.h' will be available remotely in the sandbox, as 'mymacro.C'; so they should
be included directly by 'mymacro.C', e.g. '#include "thisheader.h"' .
- In TProofServ, fill the "grand total" message with more blanks, so that no remnants
of the previous message are visible on the screen.
Revision
37894 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jan 27 14:15:03 2011 UTC (3 years, 11 months ago) by
ganis
File length: 380733 byte(s)
Diff to
previous 37664
Add support for option ':lite:' to TProof::GetDataSets . This allows to fill the map
with only the summary information about the datasets (the header of TFileCollections),
significantly increasing the speed and the memory footprint when the number of datasets
is very large. Should solve issue #77303 .
Revision
37663 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Dec 20 18:43:06 2010 UTC (4 years, 1 month ago) by
ganis
File length: 380117 byte(s)
Diff to
previous 37432
In TProof::CLearPackages, make an attempt to go via the manager to execute the command
on all known worker machines. Should improve the consistency when re-istalling packages.
Revision
37396 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 8 13:12:00 2010 UTC (4 years, 1 month ago) by
rdm
File length: 379800 byte(s)
Diff to
previous 36576
From Gerri:
- Fix a problem with the registration of missing files in the 'MissingFiles'
list. The files that could not be open during processing were not properly
registered in the list (only those found missing during validation or
giving problmes durign reading were correctly added).
- Add method TProof::ShowMissingFiles() to facilitate the display of the
list of missing files.
- Add method TProof::GetMissingFiles() to get a TFileCollection (dataset)
with the missing files for further processing.
Revision
36537 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 8 16:16:41 2010 UTC (4 years, 2 months ago) by
ganis
File length: 376285 byte(s)
Diff to
previous 36504
Patch to improve the diagnostic in case of exceptions. Information about the event and file being
processed at the moment the exception was raised is sent to the client, e.g.
0.5: caught exception triggered by signal '1' while processing dset:'EventTree',
file:'http://root.cern.ch/files/data/event_3.root', event:1 - check logs for possible stacktrace
The patch also fixes a problem with submergers observed when a worker was stopped because above
the memory limits: this worker was established as merger but could not do the work, for obvious
reasons, freezing the session.
Revision
36504 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Nov 4 18:00:14 2010 UTC (4 years, 2 months ago) by
ganis
File length: 376193 byte(s)
Diff to
previous 36480
In TProof::HandleSubmerger, use the number of Active workers (instead of the number of
workers) to calculate the number of workers to merge. Also, in TProof::MarkBad, make
sure that the all worker lists and the active monitor lists are updated after a worker
turns bad.
Should fix the issue with submergers reported by ALICE in Savannah.
Revision
36480 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 2 14:32:37 2010 UTC (4 years, 2 months ago) by
ganis
File length: 375929 byte(s)
Diff to
previous 36437
Add possibility to deactivate a worker (i.e. remove it from the active list) during Collect
in the case a failure status (!= 0) is received. This cannot be the default because an action
can have failure among its possible return states. The new feature is used by EnablePackage
to deactivate a worker where the requested packages could not be enabled properly.
Should fix an issue reported by ALICE.
Revision
36437 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Oct 28 10:10:51 2010 UTC (4 years, 2 months ago) by
ganis
File length: 375229 byte(s)
Diff to
previous 36191
In the PrintProgress function used to display a text progress bar, show also the average
reading rate in [k,M,G}bytes/s in addition to the event processing rate.
This is useful to have a feeling of the rate when running of a remote machine in batch mode.
Revision
34282 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jul 1 15:37:22 2010 UTC (4 years, 6 months ago) by
ganis
File length: 373704 byte(s)
Diff to
previous 34254
Package manager: fix a problem with enabling multiple packages having the same signature for
SETUP: the first loaded SETUP was used for all the packages. This was introduced by patch r33749.
The macro is now renamed on the fly to <package>_SETUP so to avoid the clash.
Revision
34101 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 24 12:12:59 2010 UTC (4 years, 7 months ago) by
ganis
File length: 372721 byte(s)
Diff to
previous 33784
Following a request from ALICE, implement a timer to terminate idle sessions. The timeout
value is controlled by the variable ProofServ.IdleTimeout (value in seconds). For example,
to set a timeout of 20 mins, enter the following in the xproofd config file:
### Setup an idle timeout
xpd.putrc ProofServ.IdleTimeout 1200
The timer is disabled by default.
Revision
33784 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jun 8 15:03:31 2010 UTC (4 years, 7 months ago) by
ganis
File length: 372414 byte(s)
Diff to
previous 33749
- Add warning if server does nto support the recently added functionality in EnablePackage
- Add missing protection in EnablePackage
Revision
33749 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 4 13:56:55 2010 UTC (4 years, 7 months ago) by
ganis
File length: 372044 byte(s)
Diff to
previous 33714
Patch implementing a few changes/fixes in the package management
- Add support for arguments in the SETUP function: it is possible now to pass a string
Int_t SETUP(const char *opt)
or a list of objects
Int_t SETUP(TList *optls);
Two interface methods have been added to support this:
Int_t TProof::EnablePackage(const char *package, const char *opt, Bool_t notOnClient);
Int_t TProof::EnablePackage(const char *package, TList *optls, Bool_t notOnClient);
- Make sure the return code from BUILD.sh is properly checked and execution flow stoped on
failure; this required the introduction of a check on the return code of TSystem::ClosePipe().
- Allow building packages in the global package directories if 'write' permissions are available;
this facilitates management of global packages from an authorized account.
Revision
33562 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue May 18 16:28:50 2010 UTC (4 years, 8 months ago) by
ganis
File length: 365871 byte(s)
Diff to
previous 33547
Add method TProof::SetProgressDialog(Bool_t on = kTRUE) to disable the progress dialog
window while preserving graphics from the final actions in Terminate. This may be useful
when X over network is slow. To disable the dialog the call
proof->SetProgressDialog(0)
has to be placed before Process(...).
Revision
33396 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu May 6 07:18:38 2010 UTC (4 years, 8 months ago) by
ganis
File length: 365282 byte(s)
Diff to
previous 33297
In TProof::Process(const char *datasetname, ...) add support for specifying the a list
of dataset to be processed in one go via a textfile, where the dataset names are specified
in one or multiple lines, e.g.
/group1/user1/dsn1|/group1/user1/dsn2
/group2/user2/dsn1
Revision
33209 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Apr 26 10:27:40 2010 UTC (4 years, 8 months ago) by
ganis
File length: 362553 byte(s)
Diff to
previous 33187
- TDataSetManager, TDataSetManagerFile
- Add options to get a list of all dataset registered names from ScanDataSets
(option kList; the result is a TMap of {TObjString, TObjString} with the second
TObjString empty.
- Support wild cards in ScanDataSet .
- TProof, TProofServ
- Add interface to TDataSetManager::ShowCache/ClearCache (TProof::ShowDataSetCache,
TProof::ClearDataSetCache)
- If a non-default dataset directory was specified and could not be validated, do not
initialize the default dataset, as this could be misleading. An warning is issued
on the client shell.
Revision
33086 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Apr 18 18:15:56 2010 UTC (4 years, 9 months ago) by
ganis
File length: 361245 byte(s)
Diff to
previous 33045
- Some changes towards a parallel version of VerifyDataSet:
- Make the lower level of ScanDataSet a static in TDataSetManager and separate out
the part extracting the file meta-information into a dedicated method ScanFile
- In TDSetElement, rename the list of associated files to associated objects, so that
it can be more generally used (it was not yet used for associated files and it
was anyhow supposed to be a list of TObjString's).
- Move the parsing of the recently introduced 'DataSet.SrvMaps' env in TDataSetManager
(instead of TProof). This is more logical and it makes easier to apply the enable
the server mappings during staging requests via VerifyDataSet.
- Add a missing protection in TProof::AssertDataSet potentially causing crashes in the
case a dataset is not found.
Revision
33045 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Apr 15 14:57:32 2010 UTC (4 years, 9 months ago) by
ganis
File length: 363383 byte(s)
Diff to
previous 33010
Add the possibility to remap the server for the files in a dataset. This allows,
for example, to reuse the dataset information for the same files stored in a different
cluster.
The mapping can be defined globally by the cluster administrator using the ROOT env
variable
DataSet.SrvMaps: srv1|map1 srv2|map2 ...
where srv1 are the {protocol,host,port} of the server to be matched, and map1 are the
{protocol,host,port} to be used for those files mapping srv1, and so on.
The matching server string can contain the wildcard '*' in the host field or it can
be empty, in which case the replacement is done for all the dataset files.
If the '|' character is missing the whole string is assigned to the map part; i.e.
'map1' alone is equivalent to '*|map1' or '|map1' .
For example, the following can be used to relocate the files via 'redir.dom.ain'
DataSet.SrvMaps: root://dserv-*.dom.ain/|root://redir.dom.ain/
The user can change the default mapping(s) on the fly with the parameter PROOF_DataSetSrvMaps:
the above is equivalent to setting
p->SetParameter("PROOF_DataSetSrvMaps", "root://dserv-*.dom.ain/|root://redir.dom.ain/")
before starting a query.
Revision
33010 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 14 13:45:20 2010 UTC (4 years, 9 months ago) by
ganis
File length: 360084 byte(s)
Diff to
previous 32930
Make sure that TProof::LogViewer("") starts the viewer for a Lite session, in
parallel to whats happen for TProof::Open("").
Revision
32930 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Apr 9 16:18:32 2010 UTC (4 years, 9 months ago) by
pcanal
File length: 359836 byte(s)
Diff to
previous 32833
Prefer the use of InheritsFrom(TClass*) over InheritsFrom(const char*) as long as
it does not increase the library dependencies.
Revision
32833 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Mar 30 10:48:12 2010 UTC (4 years, 9 months ago) by
ganis
File length: 359822 byte(s)
Diff to
previous 32820
Fix a problem counting valid nodes in sequential or 'masteronly' mode,
generating the fake error message "GoParallel: attaching to candidate!"
Revision
32818 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Mar 29 15:35:57 2010 UTC (4 years, 9 months ago) by
ganis
File length: 359643 byte(s)
Diff to
previous 32816
- Rename the package download sub-directory 'downloaded'
- Make sure that it is cleaned when issuing ClearPackage or ClearPackages.
Revision
32816 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Mar 29 14:48:56 2010 UTC (4 years, 9 months ago) by
ganis
File length: 359290 byte(s)
Diff to
previous 32714
- Add support for automatic download of a package when available on the master but not locally.
The downloaded packages are store under <sandbox>/packages/.download and automatically checked
for updates against the master repository. If a local version of the same package is created
(using the UploadPackage) the entry in .download is cleared, so that the behaviour is unchanged.
- Add method TProof::DownloadPackage top retrieve a package from the master; this method is used
by the automatic package download mechanism but it can also be used to create local copies of
packages.
The new functionality is described in http://root.cern.ch/drupal/content/working-packages-par-files .
Revision
32713 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Mar 23 12:45:47 2010 UTC (4 years, 10 months ago) by
ganis
File length: 349509 byte(s)
Diff to
previous 32712
- In tutorial/proof/getProof.C
- Change default port to 80000 (11093 is often used for a second daemon)
- Add function killXrootdAt to stop a running daemon
- Wait 5 secs after a daemon stop request to free all ports
- Fix some typos in notification statements and comments
- In test/stressProof.C
- Refer external code to ROOTSYS instead of "../" so that it can be run from
anywhere (e.g. inside roottest)
- Add rough estimation of ROOTMARKS (to be fine-tuned and probably elaborated)
- Use 80000 as default port
- Stop the daemon after the test to avoid/reduce problems with port occupancy
when running next test
- Make the output of TProof::GetStatistics more compact
Revision
32712 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Mar 23 10:35:15 2010 UTC (4 years, 10 months ago) by
ganis
File length: 349548 byte(s)
Diff to
previous 32276
- Add methof TProof::GetStatistics to allow the client to retrieve the correct values of
fBytesRead, fRealTime and fCpuTime at any moment; this will be used to setup a sort of
ROOTmarks in stressProof .
- Fix an issue with TProof::AskStatistics (fBytesRead, fRealTime and fCpuTime were
not correctly filled on the client; the values on the master, displayed by TProof::Print
were correct).
- Add internal method GetLastLog to facilitate the parsing of the text output
of any command (this is used to extend support for new functionality where possible).
Revision
32237 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Feb 6 13:01:45 2010 UTC (4 years, 11 months ago) by
ganis
File length: 345911 byte(s)
Diff to
previous 32217
Fix a few issues found by running stressProof with PROOF-Lite:
- Order query results using TQueryResult::fStartTime instead of TQueryResult::fEndTime
(which is undefined at the beginning)
- Fix the hostname for submergers in PROOF-Lite
- Fix behaviour when submergers cannot open the required sockets (output was not
always sent ot the master)
- Fix several typos / wrong method name / wrong protocol name
- Update info about test numbers in stressProof
Revision
32217 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Feb 4 17:39:16 2010 UTC (4 years, 11 months ago) by
ganis
File length: 345303 byte(s)
Diff to
previous 32204
Add the possibility to add files to a dataset. This is achieved with a new option
'U' (for update) to RegisterDataSet. Feature requested by ATLAS.
Revision
32204 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Feb 3 19:17:40 2010 UTC (4 years, 11 months ago) by
ganis
File length: 344939 byte(s)
Diff to
previous 31961
Add support for processing many datasets in one go in TProof::Process(const char *dataset, ...).
Two options are provided:
- 'grand dataset': the datasets are added up and considered as a single dataset;
syntax: "dataset1|dataset2|..."
- 'keep separated': the datasets are processed one after the other; the user is
notified in the selector of the change of dataset so she/he
has the opportunity to separate the results. A new packetizer,
TPacketizerMulti, has been developed for this case: it basically
contains a list of standard packetizers (one for each dataset) and
loops over them.
Syntax: "dataset1,dataset2,..." or dataset1 dataset2 ..."
In both cases, entry-list can be applied using the syntax "dataset<<entrylist", e.g.
"dataset1<<el1|dataset2<<el2|".
See http://root.cern.ch/drupal/content/working-data-sets#currentelem for more details.
A test for the new functionality has been added to test/stressProof.cxx .
Revision
31961 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jan 5 09:52:13 2010 UTC (5 years ago) by
ganis
File length: 334424 byte(s)
Diff to
previous 31744
A few optimizations/fixes in interpreting the valgrind config string in PROOF-Lite:
- Interpret "valgrind" as "valgrind=workers#2" (instead of "valgrind=master").
- Make sure that the '#<nworkers>' specification is not ignored.
- Print a warning and ignore "valgrind=master", which does not make much sense.
Revision
31706 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 9 09:16:13 2009 UTC (5 years, 1 month ago) by
ganis
File length: 333753 byte(s)
Diff to
previous 31598
Fix issues found by Coverity:
- #8632, #8631: possible use of 'sl' after delete in SendFile, GoParallel
- #7373,#7372,#7371,#6503,#6502: use snprintf instead of sprintf in
SetLogLevel, SendFile, SendDataSetStatus, HandleInputMessage, SendGroupView
- #4618: negative return check in ShowLog
- #411-#419: check the result of dynamic_cast in ValidateDSet
- #408: use of wrong pointer in TProof::UploadPackage
- #173: consistency pointer check in HandleInputMessage (kPROOF_GETPACKET)
- #75, #87, #138: unreachable code
Revision
31598 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Dec 7 15:21:47 2009 UTC (5 years, 1 month ago) by
rdm
File length: 333203 byte(s)
Diff to
previous 31505
From Gerri:
Fix a few pending issues in PROOF-Lite:
- Source the env file before running 'proofserv'; this allows to run valgrind
or other wrapper scripts (e.g. experiment-specific environment initializers)
before proofserv, as in normal PROOF
- Fix a problem with Load()
- Add the possibility for the administrator to control the number of workers
for PROOF-Lite or eventually to forbid the use of it (request on the forum).
The rootrc variable ProofLite.MaxWorkers is read out of /etc/system.rootrc
and cannot be overwritten by users.
Improve handling of Ctrl-C:
- Fix a problem in TMonitor affecting the effectiveness of interrupts.
- Do not send the results from workers to masters if the query has been
aborted.
Miscellanea:
- Make sure that the size of the cache sent back to the packetizer by the
workers is meaningful in case the defaults are used (i.e. not -1).
- Make the current processed element available to selectors via the
input list; this allows to write a more simple and effective packetizers
for tasks where the entry is a file (e.g., dataset generation in the
benchmark suite).
Revision
31505 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 2 13:20:06 2009 UTC (5 years, 1 month ago) by
brun
File length: 332994 byte(s)
Diff to
previous 31441
From Katerina Opocenska:
Patch implementing the new submerger functionality to speed up the merging phase.
At the end of the query, a set of workers are promoted submergers and assigned a
sub-set of workers to merge. Once each sub-merger has merged its sub-set of workers,
it sends its result to the master, which merges the partial results into the final
set of results.
The determination of the sub-mergers is always done dynamically, based on the recent
performance of workers. An optimal (i.e. giving the highest speed-up) number can be
calculated analytically under simpleassumptions.
Merging via submergers is by default disabled. To enable it, with the optimal number
of sub-mergers, one should set the integer parameter 'PROOF_UseMergers' to 0, i.e.
proof->SetParameter("PROOF_UseMergers", 0)
To force S sub-mergers (regardless of the optimal number) do
proof->SetParameter("PROOF_UseMergers", S)
Revision
31441 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Nov 27 07:29:29 2009 UTC (5 years, 1 month ago) by
ganis
File length: 318434 byte(s)
Diff to
previous 31006
- Fix a backward compatibility issue with the recently introduced status files
- Improve synchronization between parent and child during fork
- Optimize loops over directory entries
- Improve error and notification messages
Revision
30995 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Nov 5 17:41:03 2009 UTC (5 years, 2 months ago) by
ganis
File length: 318339 byte(s)
Diff to
previous 30900
A couple of quick additions mostly for the new bench suite under development:
- Add the possibility to give indications about the number of workers at startup.
E.g.
1. To start max 5 workers
TProof::Open("<master>","workers=5")
2. To start max 2 workers per physical machine
TProof::Open("<master>","workers=2x")
This is useful in general when running tests (equivalent but quicker then full startup
followed by TProof::SetParallel(n) or TProof::DeactivateWorker(...)).
- Add support for the worker SysInfo_t in TSlaveInfo (obtained via TProof::GetListOfSlaveInfos())
Revision
30859 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Oct 24 14:53:07 2009 UTC (5 years, 3 months ago) by
ganis
File length: 315912 byte(s)
Diff to
previous 30806
Patch for improved performance monitoring. The 'Rate Plot' button in the progress
dialog is renamed 'Performance plot' and shows up to 4 plots with the event/sec,
the average read chunck size, the number of active workers and the number of active
PROOF sessions on the cluster, all as a function of processing time.
The read chunck size plot allows to monitor the usage of the cache.
The istantaneous processing rate (event/sec) is now better estimate: a few issues
with the normalizing times have ben solved, removing the artificial structures that
were observed.
The possibility to set a max packet time length is introduced (default 30 s); this
can be changes with the parameter PROOF_MaxPacketTime.
The size of the cache is also taken into account to optimize the use of the cache.
The parameter PROOF_UseParallelUnzip has been introduced to toggle the use of the
parallel unzip (default off for now).
A page describing the new performance plots is under preparation at
http://root.cern.ch/drupal/content/progress-dialog
Revision
30174 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Sep 15 14:24:56 2009 UTC (5 years, 4 months ago) by
ganis
File length: 313848 byte(s)
Diff to
previous 29171
Patch to support automatic creation of a dataset out of files created on the
worker nodes by worker processes. The implementation is an extension of the
functionality of the class TProofOutputFile used for merging via file.
TProofOutputFile:
- Add options to steer the creation of a dataset; the extemded meaning of the
constructor arguments and anew constructor are described in
http://root.cern.ch/drupal/content/handling-large-outputs-root-files
The dataset (i.e. TFileCollection) assembing happens on the master via the
Merge method. Information about the node location of the file is happended
to the relevant TFileInfo as a special "file:" url with option "node=<host-fqdn>:<port>"
This information is needed to facilitate the cleaning of the files once they
are not needed anymore.
The dataset can be just created and returned in the output list, or registered
and/or verified.
TProofPlayerRemote::MergeOutputFiles:
- Correctly transmit the dataset object into the output list
TProofServ:
- Create the new subdirectory 'data' either in the sandbox or followinf ProofServ.DataDir .
- Register and verify the created dataset if required
TProof:
- Implement two new methods:
- ShowData(), to show the content of the assigned 'data' directory
- ClearData(UInt_t what, const char *dsname), to clear all or a subset of the files
in the assigned 'data' directory
Additional changes:
- TProofPlayer:
- Correctly handle the feedback canvas
- Set the drawing attributes
- TProofServ:
- Replace all 'Form("...")' with TString::Format("...") .
- Use the new tag kCollect in debug statements where relevant
- TProof:
- Use the new tag kCollect in debug statements where relevant
Revision
29171 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jun 23 14:13:06 2009 UTC (5 years, 7 months ago) by
brun
File length: 302525 byte(s)
Diff to
previous 29133
From Gerri:
- Fix a few problems found running in valgrind:
- In XrdROOTMgr::Validate, add missing protection against inproper use of a
temporary file when in non debug mode
- Fix a problem with inproper deletion of XrdProofWorker objects when working
with a dynamically varying 'proof.conf'
- In TProof::HandleInputMessage, fix possible double delete after kPROOF_STOPPROCESS
- Fix a couple of issues on reconnection to a running session:
- make sure that the dialog buttons are in the correct state
- make sure that logs are redirected
- Fix an issue with the displaying speed of very large log files (bufferize instead of
doing line-by-line)
- In TProof::Draw, change the type of a notification message from 'error' to 'debug'
- Fix coding conventions in TDataSetManagerFile.cxx
Revision
29133 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 22 12:28:50 2009 UTC (5 years, 7 months ago) by
brun
File length: 302356 byte(s)
Diff to
previous 29059
From Gerri:
Rename TProofDataSetManager and TProofDataSetManagerFile to TDataSetManager and TDataSetManagerFile.
These classes are managers of TFileCollection which do not have anything strictly dependent on PROOF.
So far they were only used internally by PROOF, but they are going to be exposed to more public for generic
file collection handling. It is the right time to give them a more proper name.
Revision
29059 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 17 15:43:11 2009 UTC (5 years, 7 months ago) by
ganis
File length: 302366 byte(s)
Diff to
previous 28856
Automatize running of PROOF sessions in valgrind.
The second argument of TProof::Open is used to trigger the relevant settings.
To valgrind the master session start PROOF with
TProof::Open("<master>","valgrind")
or
TProof::Open("<master>","valgrind=master")
To valgrind two workers sessions use
TProof::Open("<master>","valgrind=workers")
To valgrind <n> workers sessions use
TProof::Open("<master>","valgrind=workers#<n>")
To valgrind master and 2 workers (for <n> workers add '#<n>')
TProof::Open("<master>","valgrind=master+workers")
The valgrind logs are available with the tag '<ordinal>-valgrind' in the log dialog
or form TProofMgr::GetSessionLogs() .
To add options to valgrind execute
TProof::AddEnvVar("PROOF_WRAPPERCMD", "valgrind_opts:<options>")
before starting the session.
Revision
28856 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jun 9 11:49:43 2009 UTC (5 years, 7 months ago) by
ganis
File length: 297701 byte(s)
Diff to
previous 28813
A few optimizations in transferring and merging of the results:
- Output objects are added to the same TMessage until a HWM is reached
(default 1MB; controlled by 'ProofServ.MsgSizeHWM'); this limits the
number of transfers in the case of large numbers of small objects.
- Reasonably small histograms (GetSize() < MsgSizeHWM) are merged in
one-go at the end instead of one-by-one to exploit the better performance
of TH1::Merge on the full list of histos.
- Add possibility to compress the messages; this is controlled by
ProofServ.CompressMessage <compression_level>
The default is still 'no compression' but this will allow to study the
impact of compression.
In addition, a sort of 'progress' counter for merging is now shown on the
client:
root [n] p->Process(...)
...
Mst-0: merging output objects ... / (4 workers still sending)
This asserts socket activity and fixes the timeout problems during long
merging phases reported on the forum.
Revision
28813 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 4 15:10:49 2009 UTC (5 years, 7 months ago) by
rdm
File length: 296629 byte(s)
Diff to
previous 28746
From Gerri:
Add support for standalone usage of TProofProgressLog. This is useful to
graphically browse session logs while not in process state. A new
constructor has been added taking a Url and a session index as arguments
(same first arguments of TProofMgr::GetSessionLogs).
The possibility to change Url and/or session index has been added to the
log window, and the title gives information about the session being browsed
and its starting time.
To start the log window in standalone, do
root[] TProof::LogViewer()
and enter the master Url on the window, or
root[] TProof::LogViewer("<master>")
to directly browse the last session on <master>.
Calling again TProof::LogViewer() opens a new window so that one can
easily compare different logs.
Revision
28746 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu May 28 13:03:29 2009 UTC (5 years, 7 months ago) by
ganis
File length: 295454 byte(s)
Diff to
previous 28743
Cleanup of the TProofMgr fucntions DetachSession and ShutdownSession, and better
handling of the internal list registration, to fix potential segvs when reopening
a PROOF session inside the same ROOT session.
Now, for both normal PROOF and PROOF-Lite, it should be possible to close a
session handled by the TProof object 'p' with any of the following:
root[] delete p
or
root[] TProof::Mgr("<server>")->ShutdownSession(p)
or
root[] TProof::Mgr("<server>")->ShutdownSession(id_p)
(where id_p is the local session id given by TProof::Mgr("<server>")->QuerySessions())
and re-open a session on the same server right after.
Other fixes / changes:
- New auxilliary methods:
- TProofMgr::DiscardSession(TProof *) to remove a session from the internal
list of a manager;
- TProofDesc *TProofMgr::GetProofDesc(TProof *) to quickly get the TProofDesc
container associated with a session by object
- Make sure that the default port is always 1093 (in a couple of places it was 1094!)
- Improve some comments
Revision
28395 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Apr 30 09:27:53 2009 UTC (5 years, 8 months ago) by
ganis
File length: 295187 byte(s)
Diff to
previous 28370
Implement two new functions:
1. TProof::SetDataSetTreeName(<dataset>,<treename>)
Sets/changes the default tree name in the dataset (TFileCollection)
2. TProof::ExistsDataSet(<dataset>)
Returns kTRUE if 'dataset' exists.
Fix a problem with attaching to an existing session in TProof::Open .
Revision
28370 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Apr 28 06:43:41 2009 UTC (5 years, 8 months ago) by
ganis
File length: 293051 byte(s)
Diff to
previous 27976
Add the possibility to switch to asynchronous mode while running synchronously.
A new button "Run in background" has been added to the dialog box. The behaviour
of Ctrl-C has also been modified: the user is prompted for a choice among continuing
asynchronously, stopping (terminating) or aborting the query.
Revision
27976 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Mar 27 17:10:45 2009 UTC (5 years, 9 months ago) by
ganis
File length: 291391 byte(s)
Diff to
previous 27933
Add support for 'MasterOnly' mode in starting a PROOF session. This avoids starting
the workers when one wants just to browse the datasets or retrieve results. To start
a session in 'MasterOnly' mode enter "masteronly" as second argument to TProof::Open,
e.g.
root[] TProof *p = TProof::Open("<masterurl>", "masteronly")
Revision
27576 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 23 12:15:26 2009 UTC (5 years, 11 months ago) by
ganis
File length: 290146 byte(s)
Diff to
previous 27263
- Fix a problem with dataset validation in multi-level master setups
- Make a TSlave::GetMsd() to return a "const char *" as all other string getters
Revision
27233 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Jan 25 20:59:39 2009 UTC (5 years, 11 months ago) by
ganis
File length: 290203 byte(s)
Diff to
previous 27231
- Increase the PROOF protocol number and fix backward compatibility issue
- Make sure that the ordinal numbers are synchronized between XrdProofd and
TProof
- Make sure that the default sandbox is under $HOME/.proof
- Enable regular scheduler checks
Revision
27231 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Jan 25 09:42:21 2009 UTC (5 years, 11 months ago) by
ganis
File length: 289941 byte(s)
Diff to
previous 27198
Patch adding support for session queuing in the scheduler (from Jan and me).
This allows to control the number of sessions allowed to process queries concurrently.
The feature is enabled by a new parameter 'queue:fifo' in the 'xpd.schedparam'
directive.
In case of static worker assigment (default, random, round-robin) the max number of
running sessions can be limited by another new parameter 'mxrun'; for example
'xpd.schedparam default mxrun:3 queue:fifo' will run concurrently only 3 sessions.
Additional requests are queued and run as soon as one of the running sessions goes idle.
The current policy is FIFO, so that there is a rotation among queued sessions.
In the case of load-based worker assignement, the max number of running queries is
determined dynamically.
This option requires dynamic, per-job scheduling.
Revision
27198 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jan 21 09:29:10 2009 UTC (6 years ago) by
ganis
File length: 288388 byte(s)
Diff to
previous 27171
TProofDraw
- Fix a problem preventing drawing into an existing histogram, i.e. chain->Draw("var>>myhist"),
to work properly (chain->Draw("var>>hnew(nbin,xmin,xmax) was working fine).
- Add missing initialization (fTree in the basic constructor)
TProofPlayer
- Remove the relevant map in the list fFeedbackLists if the final object is available in the
output list; this prevents a duplication of the final feedback object
- Improve debug messages in StoreFeedback
TProofServ, TProof
- Fix a couple of memory leaks due to missing destruction of TProofProgressStatus objects
Revision
27171 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jan 16 16:04:00 2009 UTC (6 years ago) by
ganis
File length: 288356 byte(s)
Diff to
previous 27042
From Jan and me:
- Remove a duplication by using AddWorkers also in StartSlaves
- Remove the obsolete code for threaded parallel startup.
Revision
26993 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 17 16:01:04 2008 UTC (6 years, 1 month ago) by
rdm
File length: 300012 byte(s)
Diff to
previous 26951
From Gerri:
- Fix a problem with filtering the SvcMsg in TProofMgrLite::GetSessionLogs
- Fix a weird backward-compatibility problem in TProof::SendFile (found
and needed by ALICE)
- Update the release notes
Revision
26951 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Dec 16 12:04:38 2008 UTC (6 years, 1 month ago) by
ganis
File length: 299941 byte(s)
Diff to
previous 26737
- Fix an issue with uploading packages in rapid sequence
- Fix an issue with TProof::Load() in forced compilation mode ('++')
- Fix an issue with grep in log retrieval
- Fix an issue with ownership of the xrootd log file
- Make sure the XrdProofdProofServMgr::fCurrentSessions is always up-to-date
Revision
26737 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Dec 8 18:00:09 2008 UTC (6 years, 1 month ago) by
ganis
File length: 298297 byte(s)
Diff to
previous 26678
TProof, TProofServ
- Make sure that SendFile has finished before sending new messages; this solve some issues
observed sometimes in fast submission of multiple queries
TProofserv
- Make sure that for all actions not allowed during processes a message meaningful to the
requester is sent back
TProof
- Protect all calls to fPlayer to avoid problems during asynchronous actions
Revision
26678 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Dec 5 14:53:23 2008 UTC (6 years, 1 month ago) by
ganis
File length: 295789 byte(s)
Diff to
previous 26624
Patch to:
- Add the possibility to set upper limits on the virtual memory (changes in TProofPlayer,
TProofServ, TXProofServ and pmain (separate patch)).
If enebled, the session gets firts a warning when it reaches 80% of the limit, and then
processing is stopped whenit exceeds 95% of the limit, sending back the results.
Also, the memory footprint is notified when the session is terminated.
- Make sure the the active valuse in XrdProofWorker are always correct; this was not the
case for dynamic startup as the notification at the end of the query was not done.
This information is crucial or the scheduler.
The way the information is stored in XrdProofdProofServ had to be modified
and a new internal message type (kReleaseWorker) added.
(Changes in several proofd classes, TProof, TXsocket and TXProofServ)
- Fix problem with setting a static upper limit of the sessions that can be started,
and enable this functionality for the dynamic mode (changes in XrdProofSched)
- Remove a deleted worker from all the lists in TProof::MarkBad to avoid later attempts of use.
- Better control the use of the internal pipe for socket readiness notification in TXSocket.
Revision
26624 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 3 12:22:00 2008 UTC (6 years, 1 month ago) by
ganis
File length: 294922 byte(s)
Diff to
previous 26577
TProof:
- Fix a problem with Collect in asynchronous mode causing timeouts in some cases
- Optimize the sending of files when loading macros via TProof::Load(..)
- Where possible, replace printouts via fprintf(...) with Printf(...);
TProofServ:
- Fix a problem with SetParallel(0) breaking Exec("<cmd>", kTRUE)
- Fix a problem with retrieving the source files from cache when a full compilation
was required ("++") and a binary already existed.
Revision
26577 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Dec 2 08:28:46 2008 UTC (6 years, 1 month ago) by
ganis
File length: 294884 byte(s)
Diff to
previous 26514
- Add protection in packetizer in MarkBad (from Jan)
- Add protection on the player in procesing kPROOF_GETPACKET
- Improve notification if GetWorkers fails
Revision
26514 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Nov 28 15:42:43 2008 UTC (6 years, 1 month ago) by
ganis
File length: 294735 byte(s)
Diff to
previous 26497
- Synchronize TProof::Close and TProof::MarkBad with a dedicated mutex. Upon errors on the master,
these can be called concurrently is different threads (reader thread and main thread) and may
lead to segvs in MarkBad.
Revision
26450 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 25 13:52:50 2008 UTC (6 years, 1 month ago) by
ganis
File length: 293723 byte(s)
Diff to
previous 26447
- Further fixes on XrdProofd for Ubuntu 8.10 (from F. Furano)
- Add an option to test the dynamic scheduling in test/stressProof (from Jan)
Revision
26447 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 25 11:11:39 2008 UTC (6 years, 1 month ago) by
ganis
File length: 293422 byte(s)
Diff to
previous 26383
- Fixes on XrdProofd for Ubuntu 8.10 (from F. Furano)
- Fixes in async mode for the dynamic scheduling (from Jan)
- In Xrdproofd:
- Use per-user connections for Reset and Log retrieval
- Fully reset the client session vector upon Reset
- Add several missing protections
- Remove some debug statements in XrdSecProtocolgsi
Revision
26383 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Nov 22 17:58:39 2008 UTC (6 years, 2 months ago) by
ganis
File length: 292556 byte(s)
Diff to
previous 26381
- Fix the number of entries in the Draw histos for PROOF-Lite
- Set the "number of workers per node" in PROOF-Lite to the number of workers in the session
- Remove some debug messages left by mistake
Revision
26381 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Nov 22 17:15:24 2008 UTC (6 years, 2 months ago) by
ganis
File length: 292634 byte(s)
Diff to
previous 26268
- Fixes for the asynchronous mode:
- Fully localize the general handling of the input messages into two new methods
TProof::HandleInputMessage(TMessage *) and TProofServ::HandleSocketInput(TMessage *);
these methods are callable form any place that needs to intercept some messages and
do something specific on them (examples are: TProofServ::GetNextPacket, TXProofServ::Get,
TPacketizerAdaptive::ValidateFiles, ...); this allows to remove several duplications
and to make sure that no message is lost or wrongly dispatched.
- Simplify the recursive infrastructure for TProofServ::HandleSocketInput; in particular,
TProofServ::HandleSocketInputDuringProcess is removed as it is a special case of the
HandleSocketInput, saving duplications.
- Always use a kPROOF_CHECKFILE message in replies to check file operations; failures were
signaled with kPROOF_FATAL which may have some undesired side-effects, depending on the
timing
- Add support for one level of recursivity in TProof::Collect .
- Fix a problem with TProof::Finalize when called with default arguments (on the last query)
- In TProof::SendFile, send to unique workers only in the "cache" option is specified
- Remove the call to Finalize for DrawSelect queries, as it is done via the feedback
mechanisms
- Additions/fixes in test/stressProof.cxx:
- New test for the asynchronous mode;
- Fine-tuning of the progress display in batch mode;
Revision
26268 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 18 16:27:09 2008 UTC (6 years, 2 months ago) by
ganis
File length: 289283 byte(s)
Diff to
previous 26258
- Fix a few problems with the input data machinery (thanks to C. Cheshkov)
- Use the SingeKey option to write out containers
- Send the input data after the call to Begin() to catch up also the objects
possibly added in there
- Fully enable the input data machinery also for PROOF-Lite; this required to
move some calls from TProofServ to TProof to avoid duplications.
- Additions to test/stressProof.C
- Add a test for the input data machinery
- Add some progress info for the tests involving processing
Revision
25918 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Oct 22 15:00:04 2008 UTC (6 years, 3 months ago) by
ganis
File length: 281220 byte(s)
Diff to
previous 25896
First version of PROOFLite, a 2-tier realization of PROOF intended for multi-core
machines. In PROOFLite the client start directly the workers; no daemon is required.
Currently this works on Unix only. Some issues with the internal connection still need
to be sorted out on Windows.
To start a session just use
TProof::Open("") or TProof::Open("lite")
From there on everything should be as in normal PROOF. To start a standard PROOF
session (i.e. via daemons) on the localhost use TProof::Open("localhost").
(The default meaning of "" in TProof::Open() is controlled via the rc value
'Proof.LocalDefault').
To run stressProof with PROOFLite pas the argument 'lite':
./stressProof lite
By default, PROOFLite sets the number of workers equal to the number of cores.
The number of workers is controlled by the rc variable 'ProofLite.Workers'; it can
also be set on the fly with the option "workers=<n>", e.g. TProof::Open("lite?workers=2").
This patch also introduces the rc variable 'Proof.Sandbox' to control the local location
of the sandboxes. The default has been changed from "~/proof" to "~/.proof" to avoid
interferences with possible users' working areas. The sandbox contains the directory
'packages' (with the clients packages; used by standard PROOF and PROOF Lite),
and some directories used only by PROOFLite: 'cache' (binary cache) and 'datasets' with
the datasets. The location of these directories is controlled by the rc variables
'Proof.PackageDir', 'Proof.CacheDir' and 'Proof.DataSetDir'.
The working directory for PROOFLite is in the form workdir=<sandbox>/path-to-current-dir where
~/path/to/current/dir is the directory from where the session in launched. The directory
<workdir> contains the session working dirs (in the form 'session-<host>-<time>-<pid>'),
with the symlink 'last-lite-session' pointing to the last started session, and the 'queries'
directory, with the results of queries started from the current directory.
Revision
25896 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Oct 20 17:01:31 2008 UTC (6 years, 3 months ago) by
ganis
File length: 273592 byte(s)
Diff to
previous 25841
From Jan:
- Using consistently the recently introduced TProofProgressStatus in the kPROOF_GETPACKET
messages sent to TPacketizerUnit, TPacketizerAdaptive and TPacketizer; the message contains
the status of progress since the start of processing on a given node.
- Introduce TVirtualPacketizer::TVirtualSlaveStat as a base class of all the TSlaveStat
packetizer specific auxilliary classes.
- Full implemenation of GetProgressStatus() and AddProcessed(TProofProgressStatus *st) members
for TPacketizerUnit::TSlaveStat.
This patch should fix some consistency problems experienced after the patch introducing
TProofProgressStatus .
Revision
25827 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Oct 15 14:02:59 2008 UTC (6 years, 3 months ago) by
ganis
File length: 269130 byte(s)
Diff to
previous 25704
From Jan:
- Added the possibility to handle removed workers and partly processed packets. When a worker is stopped
while processing a packet it finishes, the current event and the rest of the packet is reassigned to another
worker. This is done via two interfaces:
- TVirtualPacketizer::AddProcessed(TSlave *sl, TProofProgressStatus *st, TList **)
- TVirtualPacketizer::ReassignPacket.
- New class TProofProgressStatus used to keep the query progress stauts in all the TProofPlayer objects and in
TPacketizerAdaptive::TSlaveStat. This class is also used to structure the relevant information send in
kPROOF_GETPACKET and kPROOF_STOPPROCESS messages.
- The class TPacketizerProgressive is removed completely.
- The PROOF protocol version is increased to 19: this is to handle the changes in the kPROOF_STOPPROCESS and
kPROOF_GETPACKET messages in Master - worker communication.
Revision
25704 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Oct 5 21:44:36 2008 UTC (6 years, 3 months ago) by
ganis
File length: 268874 byte(s)
Diff to
previous 25434
TProof:
- Add support for input data objects. These are objects which are available via the input list
but - being potentially large - are distributed to the cluster in optimized mode, i.e. via a
ROOT file copied only to unique nodes. These objects are *not* automatically saved into the
TQueryResult object.
Three new public methods have been added to TProof:
- AddInputData(TObject *, Bool_t = kFALSE) flags an input object as 'data input object'
- ClearInputData(TObject * = 0) clears the list of input data objects
- ClearInputData(const char *name) removes object 'name' from the list of input
data objects
- SetInputDataFile(const char *filepath) specifies the file from where to take the
input data objects; object in the input data
list, if any, are also send over.
New internal methods:
- BroadcastFile(const char *file, Int_t opt, const char *rfile, TList *wrks);
- BroadcastFile(const char *file, Int_t opt, const char *rfile = 0, ESlaves list = kAllUnique);
- void SendInputDataFile();
- Fixes:
- in Process(TFileCollection *, ...), remove the TFileCollection object from the input list
after processing, to avoid destruction problems in case of multiple query submission.
TProofServ:
- Add support for input data objects.
New methods:
- Int_t GetInputData(TList *input);
- Int_t SaveInputData(TQueryResult *qr);
- Int_t SendInputData(TQueryResult *qr);
- Optimize the number of input list instantiations
- Improve readibility of HandleProcess by separating out the code to prepare the dataset (get it
from input list, from dataset manager, etc ...) into a new method AssertDataSet(TDSet *, TList *).
TProofQueryResult:
- add missing protection
Revision
25434 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Sep 17 15:23:36 2008 UTC (6 years, 4 months ago) by
ganis
File length: 259929 byte(s)
Diff to
previous 25273
- Fix problem with the determination and transmission of the name of the object to be processed.
The problem appeared when processing files containing >1 trees in changing order.
- Fix problem with TProof::Load (Savannah bugs #41719; forum topic #7138)
- Fix problem with the logs from TProof::Load not being transmitted to the client
Revision
25273 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Aug 27 08:56:06 2008 UTC (6 years, 4 months ago) by
rdm
File length: 259811 byte(s)
Diff to
previous 24848
From Jan:
- A new optional version of PROOF with dynamic worker startup.
It can be enabled by the admin or a user with 'Proof.DynamicStartup'.
A session starts only on the master. When a query processing starts
at the master TXProofServ::GetWorkers() is called. It receives a
list of machines from the scheduler and the workers on the
machines are started. The environment is copied from the master
to the workers. It includes: the include and dynamic library paths,
the set of enabled packages as well as the macros loaded by the user.
- A new method TProof::AddWorkers(TList *workers) was added. It adds
the workers just before the query.
- A packet resubmitting mechanism. When a worker dies all the packets
that it processed are resubmitted.
- In TPacketizerAdaptive: fixing initialization of fgMaxSlaveCnt. By
default it was initialized twice.
Revision
24766 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jul 10 12:35:24 2008 UTC (6 years, 6 months ago) by
ganis
File length: 249922 byte(s)
Diff to
previous 24719
TProofServ:
Terminate the master session if, as the result of the query, all workers are gone.
The session is anyhow unusable and in this way we avoid attaching again to it.
TProof:
Disable the buttons on the progress dialog after a kPROOF_FATAL
Revision
24719 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 9 07:07:25 2008 UTC (6 years, 6 months ago) by
ganis
File length: 249780 byte(s)
Diff to
previous 24676
Patch refactorizing the XrdProofd plugin.
The class XrdProofdProtocol is now in charge only of the operations strctly related
to the XProofd protocol. All auxilliary services have been moved to dedicated service
classes controlled by XrdProofdManager. In particular:
- XrdProofdClientMgr handles the clients (represented by instances of XrdProofdClient)
including login, authentication and access control
- XrdProofdProofServMgr handles the PROOF sessions (represented by instances of
XrdProofdProofServ) including creation, attachment, detachment, destruction and
environment setting
- XrdProofdNetMgr handles connections between instances of XProofd running on different
nodes
- XrdProofdPriorityMgr handles the session priorities
A special effort has been done to get rid of all possible internal dead-lock situations.
Internal actions on clients and sessions are now all asynchronous, governed by internal
pipes.
The new plugin also offers new functionality, among which:
- a XProofd admin area, located under <xrd.admin>/.xproof.<port>, keeps information about
active and terminated sessions, and the minimal state for active clients. This is used
to reguraly check the client and session activity, to cleanup orphalin sessions and
to shutdown inactive client connections.
In particular this allows a more solid implementation of Reset, which now exists in two
flavours: 'soft', TProof::Reset(<masterurl>), (default) asks the sessions to terminate gently;
'hard', TProof::Reset(<masterurl>,kTRUE) schedules all sessions for forced termination.
- support for automatic attempts for reconnection in the case the daemon restarts.
This allows to reconfigure the plugin by xrootd restart w/o affecting the running
sessions.
- support the definition of workers via config file directive, getting de facto rid of
the proof.conf file.
- domain+level control of printout message; the format has been improved: in particular
all information messages contain now the tag 'xpd-I' and all error messages the tag
'xpd-E', so that they can easily be grepped out from the log file.
The Wiki 'XrdProofdDirectives Directives' page has been updated with the new directives.
Revision
24588 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 27 13:52:23 2008 UTC (6 years, 6 months ago) by
ganis
File length: 248804 byte(s)
Diff to
previous 24573
Few additions in preparation of the new XrdProofd:
- New signature for TProofMgr::AttachSession to be used internally in case of
reconnections
- Accomodate support for hard/soft Reset operations.
Revision
24570 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 26 13:45:13 2008 UTC (6 years, 6 months ago) by
ganis
File length: 248337 byte(s)
Diff to
previous 24568
- Add the possibility to cleanly terminate an active worker at any moment.
This is needed by the dynamic worker configuration.
Revision
24568 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 26 11:57:42 2008 UTC (6 years, 6 months ago) by
ganis
File length: 246612 byte(s)
Diff to
previous 24490
- Fix a race condition possibly affecting the handling of workers death
- Improve diagnostic from MarkBad: clients will now receive something like this
root [1] Worker 'localhost-0.1' has been removed from the active list
+++ Message from top master at aleph025.cern.ch:1093 : marking localhost:1093 (0.1) as bad
+++ Reason: problems receiving a message in TProof::CollectInputFrom(...)
+++ Most likely your code crashed on worker 0.1 at localhost:1093.
+++ Please check the session logs for error messages either using
+++ the 'Show logs' button or executing
+++
+++ root [] TProof::Mgr("aleph025.cern.ch:1093")->GetSessionLogs()->Display("0.1",0)
Revision
23075 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 9 09:10:25 2008 UTC (6 years, 9 months ago) by
ganis
File length: 243820 byte(s)
Diff to
previous 22635
From Jan-Fiete Grosse-Oetringhaus (ALICE) and me.
Patch introducing the improved dataset manager developed by ALICE.
The client side interface is basically unchanged, the biggest change is the renaming
of TProof::CreateDataSet to TProof::RegisterDataSet.
On the server side the relevant code has ben extracted from TProofServ and hidden
behind an inteface class TProofDataSetManager describing the functionality needed by
TProofServ, and an implementation class TProofDataSetManagerFile handling dataset
meta data saved into ROOT files. TProofDataSetManagerFile is instantiated via the
plugin manager and by default it handles the <sand_box>/datasets area.
This separation is needed to allow loading implementation based on different backends;
for example, ATLAS foresees to use a MySQL-based backend.
A TWiki page with information and examples in on preparation.
Revision
22419 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Mar 3 00:25:01 2008 UTC (6 years, 10 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 245704 byte(s)
Diff to
previous 21467
From Andrew Savchenko:
ROOT can not be compiled with gcc-4.3.
Some ROOT source files doesn't contain required #include directives,
for example, they use strlen(), but #include <string.h> is missed or
malloc() is used and #include <stdlib.h> is missed.
Earlier versions of gcc allowed some headers to be included implicitly,
but issued a warning (-Wimplicit-function-declaration). Newer one,
gcc-4.3 denies such silly behaviour: all required headers must be explicitly
included.
Attached patch fixes this. Also it fixes another issue, which disallows
ROOT to compile under gcc-4.3: C functions don't belong to namespace std,
so expressions like std::memcpy() are no longer valid and plain memcpy()
should be used instead.
Revision
20999 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Nov 22 18:25:04 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proof/src/TProof.cxx
File length: 245370 byte(s)
Diff to
previous 20961
- First version of 'stressProof.cxx' and related changes in the test/Makefile
- Add missing protection in TProof to avoid starting the dialog when in batch mode
- Add proper default initialization and destruction in tutorials/proof/ProofSimple.C,.h
Revision
20862 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Nov 18 14:51:42 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proof/src/TProof.cxx
File length: 245424 byte(s)
Diff to
previous 20758
Synchronize with braches/dev/proof r20835
Summary:
TProofServ:
- read session.rootrc with level kEnvChange to be able to change existing
settings
- Avoid deleting a query result twice in some special cases
- Fix a problem with the initialization of fKeptQueries
rootrc.in:
- restore default settings the asynchronous reading
TFileCacheRead:
- add missing protection in ReadBuffer
TEventIter:
- Enable the usage of TTreeCache
XrdProofdProtocol:
- Additional check on the ownership of the unix socket
- Improve notification during Reset
- Reduce default timeout on admin requests and make it configurable
XrdProofConn:
- Use the configurable maxtry everywhere where relevant
TPacketizerAdaptive:
- Store info on all the processed packets in per-worker lists
TProofMgr, TXProofMgr, TXSocket, XProofProtocol, XrdProofdProtocol:
- Add possibility for the admin to broadcast a message to the connected users
getProof:
- Add missing protection
Revision
20758 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Nov 11 21:29:49 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proof/src/TProof.cxx
File length: 245424 byte(s)
Diff to
previous 20716
Several small fixes and few additions imported from 'branches/dev/proof':
- XrdProofdProtocol:
+ Fix a problem in CleanuProofServ sometimes preventing proper cleanup
+ In ReadBuffer, do not trigger an error on empty files
+ Add missing protection against an empty string while setting the ROOT version
- XrdProofdManager:
+ Define the list of unique nodes to avoid broadcasting twice to the same node
- XrdProofWorker: improve matching technique
- TProof: add setter and getter for Int_t parameter
- TProofServ:
+ do not flush the log file in SendAsynMessage: this recent change
made the logs "disappear" form the ProgressDialog
+ Activate control on the max number of query result kept
- TPacketizerUnit: fix a problem with filling the histof for PerfStats
- TProofPlayer: call Begin() also during selecter re-initialization
Revision
20682 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 6 15:51:59 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proof/src/TProof.cxx
File length: 244364 byte(s)
Diff to
previous 20411
Import branches/dev/proof r20654
Summary:
+ Improvements
- Add support for SSH SOCKS4 tunnelling; the local port for the tunnel can be specified
in the master URL, e.g. TProof::Open("master/?tunnel:8000")
- Add the possibility to plot the estimated instantaneous rate
- Add "PROOF_ForceLocal" parameter to the !TPacketizerAdaptive; if set to 1, all the data
are processed locally.
- Add support for remote grep functionality while retrieving logs (needed by the forthcoming
memory monitor)
+ Bug fixes
- Several small fixes to revive the multi-master mode.
- XrdProofdProtocol:
- add missing lock to the client instance in SendMsg to avoid screwing up requests from
workers on the same machine
- lock the mutex of the requester when setting prorities
- add notification during Reset
- fix problem with the detection of the 'allow' directive
- fix problem with the parsing of the return value from XrdProofServProxy::TerminateProofServ()
- always use the effective user to retrieve info fom another server
(XrdProofdManager::GetProofConn is now used)
- fix possible dead-locks from debug notifications done after hard-killing a session
- re-enable the garbage collector thread of the connection manager in XrdProofConn to
fix a problem with closing physical connections;
- fix a problem with !CleanupProofServ in the case of a non-privileged daemon running
in multi-user mode
- introduce a timeout when waiting for the startup of a 'proofserv'.
- XrdProofConn: init mutex in the ctor; lock in SendRecv
- XrdProofSched: add support for using the priorities defined in the group manager to define
the number of workers for sessions
- TProof:
- Broadcast priorities to unique nodes only
- timeout after 5 mins the initial Collect to avoid clients getting stuck at this stage
- add support for generic timeout in Collect (disabled by default)
- fix a problem with SendFile.
- TXProofServ: add a call to !TProof::InterruptCurrentMonitor() in Terminate() to stop
infinite loops in Collect
- TXSocket:
- Implement a flag to interrupt a TXSocket while waiting for messages
- Split the session creation timeout in 4 attempts: the total timeout is the same but
it may circumvent occasional forking problems.
- XROOTD:
- fix a potential (possibly accademic) memory leak in the client
- fix a access permission problem with Kerberos ticket forwarding
- fix bug preventing 'locate' to work properly
- re-enable optimized 'locate'
Revision
20411 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Oct 19 12:37:58 2007 UTC (7 years, 3 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 239875 byte(s)
Diff to
previous 20333
remove conditional around include of RConfigure.h:
#ifdef R__HAVE_CONFIG
#include "RConfigure.h"
#endif
Having this conditional caused RConfigure.h to be missing from the .d
files and hence these files would not be recompiled after re-configuring
using --prefix.
Revision
20307 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Oct 11 10:58:50 2007 UTC (7 years, 3 months ago) by
ganis
Original Path:
trunk/proof/src/TProof.cxx
File length: 239753 byte(s)
Diff to
previous 19826
Import branches/dev/proof r20306
Summary (see branch logs for more details):
- Set of changes related to CPU quota control including
- broadcast of centrally determined priorities
- mechanism to renice processes in a quantitative way
- New packetizer for non-tree based analysis and related API in TProof (from L. Tran-Thanh)
- Support for merging output objects saved in files on the workers (from L. Tran-Thanh and me)
- Improve version binary compatibility checks using also the SVN revision number
(when available) to define the running version.
- Extend the version binary compatibility checks also to the cached selector binaries.
- Extend TDSet::Lookup so that in case of missing files, it can remove them from the
dataset (option removeMissing must be set).
- Move the data set lookup to the TProofPlayerRemote::Process.
- Handle properly the case of incomple datasets: if the file is not found in the lookup
don't try to validate it; add it, instead, to a 'missingFiles' list returned in the
output list (fixing bug #28800 in Savannah).
Revision
19565 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Aug 8 01:08:43 2007 UTC (7 years, 5 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 236789 byte(s)
Diff to
previous 19492
in Init() avoid calling TSystem::GetHostByName() on non-existing hostnames,
like __master__, as some nameservers are configured to redirect all unknown
(or misspelled) hosts to one central machine (like ISP web server).
Revision
19145 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 22 17:16:35 2007 UTC (7 years, 7 months ago) by
ganis
Original Path:
trunk/proof/src/TProof.cxx
File length: 230492 byte(s)
Diff to
previous 19098
Add support for global package directories. The colon-separated list of directories
to be searched for is defined by the directive
Proof.GlobalPackageDirs globaldir1[:globaldir2[:globaldir3[:...]]]
which can be included in the xrootd config file via the xpd.putrc directive
xpd.putrc Proof.GlobalPackageDirs globaldir1[:globaldir2[:globaldir3[:...]]]
The directories must be readable by the user (a check is made in TProofServ::Setup).
Packages are searched first in the user local cache and then in the global
directories (in the given order, so that the first occurence is used).
The content of the global package directories is listed by TProof::ShowPackages().
The functionality is available also locally on the client. The Proof.GlobalPackageDirs
has to be specified in the users $HOME/.rootrc .
Global packages can be directly enabled via TProof::EnablePackage(<package_name>).
Revision
19003 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jun 12 10:06:00 2007 UTC (7 years, 7 months ago) by
ganis
Original Path:
trunk/proof/src/TProof.cxx
File length: 226520 byte(s)
Diff to
previous 18983
TSlave.h, .cxx
- Add new data member fGroup specifying the group the user runs under.
- If available, display the group in Print()
TProof.cxx
- If available, add the group information while creating the worker session
- If available, display the group in Print()
Revision
18888 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri May 25 13:53:59 2007 UTC (7 years, 8 months ago) by
ganis
Original Path:
trunk/proof/src/TProof.cxx
File length: 224813 byte(s)
Diff to
previous 18861
From Jan:
Add support for processing datasets "by name", i.e. by just sending the name
of a dataset known by the master.
TProof.h, .cxx
- New method
Long64_t TProof::Process(const char *dsetname, const char *selector,
Option_t *option = "", Long64_t nentries = -1,
Long64_t firstentry = 0, TEventList *evl = 0);
- Fix typos in comments
TProofServ.cxx
- In HandleProcess, retrieve the relevant dataset info "by name" from local
dirs in the case only the data set name is specified.
- In HandleRetrieve, remove a debug printout left from a previous patch.
Revision
18828 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon May 21 00:21:44 2007 UTC (7 years, 8 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 222585 byte(s)
Diff to
previous 18635
new static utility functions for retrieving parameters from input lists:
static Int_t GetParameter(TCollection *c, const char *par, TString &value);
static Int_t GetParameter(TCollection *c, const char *par, Long_t &value);
static Int_t GetParameter(TCollection *c, const char *par, Long64_t &value);
static Int_t GetParameter(TCollection *c, const char *par, Double_t &value);
simplifies code from:
TParameter<Int_t> *par = 0;
TObject *obj = input->FindObject("PROOF_ProgressPeriod");
Int_t period = 500;
if (obj && (par = dynamic_cast<TParameter<Int_t>*>(obj)))
period = par->GetVal();
to
Long_t period = 500;
TProof::GetParameter(input, "PROOF_ProgressPeriod", period);
Revision
18503 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Apr 17 09:05:24 2007 UTC (7 years, 9 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 219720 byte(s)
Diff to
previous 18384
From Gerri:
- Fix float to int convertion for the text progress bar (bug #25101)
- Make the ".par" extension not mandatory for the first argument to
TProof::UploadPackage; now the following is done:
1. The ".par" extension is added if missing
2. The resulting path is expanded and searched for
3. If the file does not exist or it is not readable, a PAR file
of the same name is searched for in the package directory
~/proof/packages
Revision
18349 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Mar 19 14:43:26 2007 UTC (7 years, 10 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 219276 byte(s)
Diff to
previous 18347
From Gerri:
- TProof::Load(const char *macro, Bool_t notOnClient) where 'macro' is the
name of a macro or the implementation file of a class (in which case the
corresponding header should have .h or .hh extension). If notOnClient
is kTRUE (default is kFALSE) the class/macro is loaded only
on master and workers.
- Re-usage of valid binaries from macros or selectors (they are cached under
<SandBox>/cache), this allows to save a few seconds while running a query
with an unchanged and compiled selector.
- Exports ROOTINCLUDEPATH with the current content of gSystem->GetIncludePath()
just before executing BUILD.sh so that the path can be used for build
purposes (bug #18079)
Revision
17750 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Feb 7 09:07:15 2007 UTC (7 years, 11 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 215872 byte(s)
Diff to
previous 17719
From Gerri:
This adds a text-oriented progress bar while processing in batch mode,
showing the percentage of events processed and the average rate.
The format is the same as in TFileMerger::Cp.
This fixes bug/request #20643.
Revision
17697 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 5 14:20:25 2007 UTC (7 years, 11 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 211169 byte(s)
Diff to
previous 17562
introduce macro R__PRIVATE to mark private methods as hidden so they won't
be exported from the shared library (reduces load time). Put R__PRIVATE
in front of some private methods, like:
R__PRIVATE Int_t TProof::Exec(const char *cmd, ESlaves list, Bool_t plusMaster)
or
Int_t fileGlobal R__PRIVATE;
R__PRIVATE expands only for GCC >= 3.3. In all other cases it is a noop.
Revision
17562 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jan 29 15:11:11 2007 UTC (7 years, 11 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 210909 byte(s)
Diff to
previous 17486
Frrom Gerri:
There is a new classes TProofServLogHandler allowing to redirect
the output to the client in real-time. It has two main constructors,
one taking an external command (like BUILD.sh) and opening a pipe,
the other taking a pointer to a open file. TProofServ uses the class
via the guard class TProofServLogHandlerGuard to better localize the
usage. This redirection is not always activated. For the time being it
is active for Process and Cache actions,and it can be deactivated via
TProof::SetRealTimeLog(kFALSE)
(for BUILD.sh - executed externally - is always ON).
In particular, with the patch applied, the outputs from BUILD.shi
will scroll down on the screen, and clicking on "Show logs" in the
dialog box during process shows the messages and allows to see what
the session is doing: this is quite nice, because getting
an empty window from "Show Logs" is frustrating, in my opinion.
So, all this fixes the bug/request #16845.
The patch also fixes bug #22555 (hanging while enabling a package
without BUILD.sh) and several other small bugs (a couple of unitialized
variables) or typos.
It also cleanups the API a bit (we had SetQueryType and SetQueryMode
doing exactly the same thing ...).
Revision
17209 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 13 11:21:55 2006 UTC (8 years, 1 month ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 206715 byte(s)
Diff to
previous 17181
Added new methods:
void SetParameter(const char *par, const char *value);
void SetParameter(const char *par, Long_t value);
void SetParameter(const char *par, Long64_t value);
void SetParameter(const char *par, Double_t value);
TObject *GetParameter(const char *par) const;
void DeleteParameters(const char *wildcard);
void ShowParameters(const char *wildcard = "PROOF_*") const;
they allow the easy setting and inspection of the special PROOF control
parameters (and other input parameters) like PROOF_MaxSlaverPerNode, etc.,
instead of:
proof->AddInput(new TParameter<Long_t>("PROOF_MaxSlavesPerNode", 2));
we can now do:
proof->SetParameter("PROOF_MaxSlavesPerNode", 2);
Revision
17149 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Dec 11 13:24:49 2006 UTC (8 years, 1 month ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 202924 byte(s)
Diff to
previous 17035
Two new features:
- TProof::EnablePackage() has now a bool argument notOnClient which when
true does not build and enable the package on the client. The default
is false.
- Package building now keeps track of which PROOF version has been used
to build a package. In case the version has changed the system will
first call "PROOF-INF/BUILD.sh clean" and then "PROOF-INF/BUILD.sh".
The user has to provide in BUILD.sh the code to handle the "clean"
argument to call e.g. "make clean" or whatever is needed to clean
the package and force a rebuild.
Revision
17035 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Dec 3 23:34:04 2006 UTC (8 years, 1 month ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 201190 byte(s)
Diff to
previous 16976
From Gerri:
0. Introduction of the static:
TProof::Mgr(const char *url)
to get the manager to the cluster; this gives access to all the manager
functionality, included Reset(). TProof::Reset() is removed.
1. Two new classes: TProofLog and TProofLogElem.
TProofLog contains a list of TProofLogElem, one per node. TProofLogElem
basically contains a TMacro plus methods to play with it.
2. The TProofLog object can be retrieved via the manager:
TProofLog *pl = TProof::Mgr("cluster")->GetSessionLogs()
by default the last session is retrieved: the signature for GetSessionLogs is
TProofLog *TProofMgr::GetSessionLogs(Int_t ridx, const char *stag)
the first argument is a reversed relative index (-1 gets the next to last
session), the second is the session unique tag; if the second is
specified the first is ignored.
3. TProofMgr has a new method ReadBuffer to quickly read a buffer from
a file via XrdProofd: this allows to avoid the second xrootd on the
redirector and in general is useful when a quick reading is needed.
Also, the buffer is always retrieved via the master, so we should not
have firewall problems in accessing directly the xrootd of the workers.
4. In the sandbox I have kept the .sessions file with the active sessions
and I have added .terminated in each session working dir which have
been terminated. On the master, each session working dir has the file
.workers with the relevant information about the workers.
5. The max number of old session workdirs is set to 10 by default
(configurable in xpd.cf): the sandbox is trimmed when closing a session.
6. Some unrelated changes/fixes contained in the patch:
- in XrdProofdProtocol:
- use of /proc (linux, solaris) or sysctl (mac) to verify/search
for running of a process. For other platforms I have left the
parsing of the output of 'ps'.
- I have introduced a 'cron' thread: it runs every minute and for
the moment it just cleans a vector; this is the place where to do
client independent actions, like checking if the config files have
changed, for example.
- I have added the possibility to control which user can access
the cluster; requested by Jan Fiete.
7. The patch contains also several fixes here and there I have found by
testing.
Revision
16955 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 27 14:14:24 2006 UTC (8 years, 1 month ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 195140 byte(s)
Diff to
previous 16870
From Gerri:
This is the patch to move TDSet and TChainProof in 'proof' and remove
any dependence on PROOF in 'tree'. I have renamed TChainProof as TProofChain,
for consistency. TProofChain is now deriving from TChain and overrides only
the required methods. It is loaded via PluginManager by TChain in SetProof.
TQueryResult stores now all the input-related objects in fInputList. There
is a new method
TObject *TQueryResult::GetInputObject(const char *classname)
to retrieve the first instance of a class from the input list, so for the
TDSet:
TQueryResult *qr;
TDSet *dset = (TDSet *) qr->GetInputObject("TDSet")
The changes in TSessionViewer are needed to adapt to this new way of
accessing TDSets.
Revision
16839 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 20 15:56:36 2006 UTC (8 years, 2 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 194517 byte(s)
Diff to
previous 16796
From Maarten + Gerri:
Patch to allow run time configuration of proofserv environment. The client
can pass a set of shell variables to PROOF. These variables can be used
to startup different versions of the proofserv, or they can instruct the
system to run proofserv under valgrind or tcheck, etc. Main changes:
- default wrapper script for proofserv
- modifies the build system for the wrapper script
- adds a list of environment variables to TVirtualProof, which are
passed to the wrapper by proofd/xrootd.
- adds code to TProofServ to pickup these variables and pass them
along to the slaves/submasters
Revision
16796 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Nov 16 17:18:32 2006 UTC (8 years, 2 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 194118 byte(s)
Diff to
previous 16775
Changes to get rid of the config.h and HAVE_CONFIG define. These were way
too trivial (just do a locate config.h to see how many there are):
- replace occurances of config.h by RConfigure.h
- add #warning in config.h that is should not be used
- change HAVE_CONFIG to R__HAVE_CONFIG
Revision
16775 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Nov 15 17:45:55 2006 UTC (8 years, 2 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 194085 byte(s)
Diff to
previous 16755
From Gerri and Bertrand:
Fixes and new features developed in preparation for the Intel quad-core
launch demo:
- Add instantaneous event and MB processing rates in the Progress signal
and make necessary modification to exploit it in the Progress Dialog.
- Add possibility to change the Progress and Feedback periods
- A number of small improvements and bug fixes
Revision
16603 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Oct 23 14:44:40 2006 UTC (8 years, 3 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 191722 byte(s)
Diff to
previous 16579
From Gerri:
The patch fixes the problem of xrootd not terminating in case the xpd plugin
loading fails. In addition is also fixes:
- SetUserEnvironment:
chdir to sandbox instead of HOME (which may not be accessible, e.g. on AFS
and no tokens)
- Add a new trace level, HDBG, for printings in loops or any other low
level details.
- Some cosmetics (tabs, misalignments, ...)
Revision
16557 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Oct 19 12:38:07 2006 UTC (8 years, 3 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 191131 byte(s)
Diff to
previous 16410
From Gerri:
Major set of robustness fixes, improving error reporting and reducing the
startup failures especially observed on AMD64. Summary of changes:
- XrdProofdProtocol.cxx, .h
- Remove zombie treatment: done by XrdScheduler::Reaper()
- Check the modification date of the PROOF config file so that changes
can be picked up on the fly
- Fix problem with the definition of the image
- Improved tracing (see etc/proof/xpd.cf.sample)
- Several bug fixes
- XrdProofConn
- fix problem with marshalling in case of retries.
- TProofServ, TXProofServ
- Add return codes to CreateServer() and Setup() for proper termination
in case of errors
- TProof
- Make sure that TXProofMgr is the last one in the list of sockets for proper
cleaning
Revision
15956 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Aug 8 21:27:14 2006 UTC (8 years, 5 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 181949 byte(s)
Diff to
previous 15935
From Gerri:
the attached small patch adds the TNamed in question. After the patch, this
TNamed *nm = (TNamed *) fInput->FindObject("PROOF_QueryTag");
if (nm) {
Printf("Found query tag: %s", nm->GetTitle());
} else {
Printf("Query tag not found");
}
should return the query tag in SlaveBegin or Terminate.
The ID is created on the master so it cannot be available in Begin.
Revision
15935 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Aug 6 07:15:01 2006 UTC (8 years, 5 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 181644 byte(s)
Diff to
previous 15928
Handle several issues related to the TTree::Process() and TTree::Draw()
return values:
- TSelector: fStatus increased from Int_t to Long64_t. It is used to
return the number of selected events by TTree::Draw() which returns
a Long64_t.
- TVirtualProof, TProof, TDSet, TProofPlayer: Process() and Draw() were
returning an Int_t, now a Long64_t to be consistent with TTree/TChain.
Actually returning of TSelector::GetStatus() from the workers to be
done tomorrow.
Also added proper comments describing the return values for Draw()
(number of selected events), and Process() (value returned by
TSelector::GetStatus()).
- TTree, TChain, TSelectorDraw: added proper comments describing the
return values of Draw() and Process().
Revision
15857 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 26 14:28:59 2006 UTC (8 years, 6 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 180351 byte(s)
Diff to
previous 15675
From Gerri:
TVirtualProof.h
- Add abstract methods methods to communicate the data set status
(signal + sender)
pmain.cxx
- Add call to Exit(0). T(X)ProofServ::Terminate() just interrupt the event
loop (insteadof exiting) to avoid problems when the terminate is run in
a separate thread (like in xproofd).
proof/inc/TProof.h
- Add methods methods to communicate the data set status (signal + sender)
- Increase protocol number to 11 (new merging technology)
proof/inc/TProofPlayer.h, .cxx
- Implement new merging technology for the output list to reduce memory
consumption: add methods to directly merge received objects in the
output list
- Polish cleaning of the various temporary lists and objects
TProofServ.h, .cxx
- Implement new merging technology for the output list to reduce memory
consumption: objects are sent one-by-one
- Do not scan for previous query results at Setup (do it when the client
requests for it first); thsi allows to speed up start-up when a large number
of queries is present and also allos to avoid warning due dictionaries
not yet loaded.
- Add method for a global cleanup of old query results.
- Keep in memory only meta information about processed queries (i.e. not the
output list) to avoid large memory consumption in case of large
output lists
- In Terminate, change the call to Exit(0) to ExitLoop() ; Exit(0) is
called in pmain.cxx .
TProof.cxx
- Implement new merging technology for the output list to reduce memory
consumption: objects are received one-by-one and directly merged in the
output list
- Add protection in parallel startup against invelid slaves.
- Remove check on directory existence in UploadDataSet.
- Add methods to handle notifications from data set verification
- Fix inconsistency with notification in activation/deactivation
of non-existing workers
TXProofServ.cxx
- In Terminate, change the call to Exit(0) to ExitLoop() ; Exit(0) is
called in pmain.cxx.
TQueryResult.cxx
- Make a owned copy of the input list (instead of just storing the pointer) to
allow proper cleaning.
Revision
15656 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 3 09:33:50 2006 UTC (8 years, 6 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 176375 byte(s)
Diff to
previous 15507
From Jan:
This patch adds following issues:
- public dataset management
- separation of directories for different users
- adding TProofServ::HandleDataSets method
- spliting communication in UploadDataSet in two parts (checking name
and saving)
- I also added these methods:
virtual Int_t UploadDataSet(const char *dataset,
TList *files,
const char *dest = 0,
Int_t opt = kAskUser,
TList *skippedFiles = 0) = 0;
virtual Int_t CreateDataSet(const char *dataset,
TList *files,
Int_t opt = kAskUser) = 0;
- and added arguments to
TList *GetDataSets(const char *dir = 0);
void ShowDataSets(const char *dir = 0);
Revision
15507 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 21 16:18:27 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 167127 byte(s)
Diff to
previous 15338
From Gerri:
The patch implements the following new functionality:
- Shutdown of idle sessions when quitting ROOT; if not idle the session
is shutdown as soon as processing terminates; it is possible to delay
the automatic shutdown by a tunable amount of time via a new directive
in the xrootd config file 'xpd.shutdown' (see xpd.cf.example).
- New static method TProof::Reset(const char *url) to hard-cleanup an
user area (implemented for Xproofd only); this implied quite some
additions in XrdProofdProtocol.
- Possibility to temporary remove / re-insert a worker from the active list
TProof::DeactivateWorker(const char *ord)
TProof::ActivateWorker(const char *ord)
with 'ord' the ordinal numer, e.g. "0.3" .
- A default url for the disk pool
const char *TVirtualProof::GetDataPoolUrl()
It is set by default to a string send back by the master, but it can
be changed via
void TVirtualProof::SetDataPoolUrl(const char *url)
Jan should then adapt his code to use this as default.
- Posibility to timeout Collect; for now this is only used at forced
termination.
The patch includes many small changes and bug fixes. One of the most important
is in TXNetSystem (a bug preventing correct treatment of multiple request
to the same redirector).
Revision
15338 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 5 22:51:14 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 161426 byte(s)
Diff to
previous 15321
From Gerri:
base/inc/TVirtualProof.h, proof/inc/TProof.h, proof/src/TProof.cxx
proof/src/TProofProgressDialog.cxx:
- fix a problem with the progress dialog when detaching a session:
a new signal CloseProgressDialog() is introduced, as well as the
possibility to empty the list of submitted queries at once via
TProof::Remove().
proofx/inc/TXHandler.h, .cxx:
- add generic argument to HandleInput() and HandleError().
proofx/inc/TXProofMgr.h, .cxx:
- move to new signature for HandleInput(), HandleError().
proof/inc/TSlave.h:
- add method to flush underlying socket.
proofx/inc/TXSlave.h, .cxx:
- move to new signature for HandleInput(), HandleError().
- implement new method to flush underlying socket.
proofx/inc/TXSocket.h, .cxx:
- Add method to flush socket.
- Handle interrupt messages in the reader thread.
- Remove interrupt semaphore (not needed any more).
- Remove timeout argument from GetInterrupt (not needed any more).
proofx/inc/TXProofServ.h, .cxx:
- move to new signature for HandleInput(), HandleError().
- implement interrupt handling in HandleInput().
- implement dedicated version of LockSession to deal with new format
of session tag (this is a bug fix).
proofd/inc/XrdProofConn.h, .cxx:
- Add mutex protection in the SendRecv method (it can be called from both
the reader and main threads).
proofd/src/XrdProofdProtocol.cxx:
- use an asynchronous message (instead of a signal) for interrupts.
proofx/inc/LinkDef.h, TXSocketHandler.h, .cxx, TXUnixSocket.cxx:
- add TXSocketHandler in the dictionary.
- cosmetics.
Revision
15312 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 2 15:14:35 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 159715 byte(s)
Diff to
previous 15181
From Gerri:
large PROOF patch. Main issues addressed:
- Output redirection to log file at 'proofserv' startup (instead of
after Setup())
- Log messages activated at session creation
- Session sandbox restructuring in such a way that the unique session
tag, created by the top master, uniquely identifies the session on
all nodes, for example on top master:
<User_Sandbox>/session-<top_master_session_tag>/
master-0-<top_master_session_tag>/master-0-<top_master_session_tag>.log
On worker 0.0
<User_Sandbox>/session-<top_master_session_tag>/
worker-0.0-<worker_0.0_session_tag>/worker-0.0-<worker_0.0_session_tag>.log
- Automatic creation of the dataset dir
- Improved handling of error conditions
- Reorganization of TProofServ and XrdProofdProtocol
- Several bug fixes, in particular in XrdProofdProtocol
Revision
14752 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 19 10:57:44 2006 UTC (8 years, 9 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 153428 byte(s)
Diff to
previous 14745
From Gerri:
This is the big patch.
Comments:
- Client side:
- Remove use of plugin manager to avoid unwanted locks
- Improved treatment of error conditions
- Server side:
- Several new functionalities described in the release notes
Revision
14745 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 19 08:22:26 2006 UTC (8 years, 9 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 152277 byte(s)
Diff to
previous 14714
Change the TError.h macros:
Assert -> R__ASSERT
Check -> R__CHECK
Change the TCollection.h macro:
ForEach -> R__FOR_EACH
This to avoid potential problems due too trivial macro names.
The old macros will be removed in the next release. Currently
they will print out warning messages with the advice to move
to the new macro names.
Revision
14687 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Apr 13 10:27:14 2006 UTC (8 years, 9 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 152320 byte(s)
Diff to
previous 14508
From Jan:
Changes to enable running from GUI instead of interaction on prompt
including adding skippedFiles return argument to UploadDataSet(),
TProof::GetDataSets() and changing ShowDataSet() methods as well as
enum EUploadDataSetAnswer to TVirtualProof.
VerifyDataSet returns now the number of missing files.
Changing all TSocket::Send on client side to Broadcast.
Correcting some synchronization problems and adding comments.
Using TUrl::SetPort to avoid default port 80.
When comparing user answer with "n", "none" was treated as "n".
Now we first compare with "none".
Revision
14355 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Mar 21 15:07:22 2006 UTC (8 years, 10 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 149065 byte(s)
Diff to
previous 14164
From Jan:
Adding EUploadDataSetOpt enum for options of UploadDataSet() and
UploadDataSetFromFile().
In TProof.cxx: adding static helper function
CollapseSlashesInPath(const char *path)
Adding few TProof methods for dataset upload and operations:
Int_t UploadDataSet(const char *files,
const char *dest,
const char *dataset,
Int_t opt = kAskUser);
Int_t UploadDataSetFromFile(const char *file,
const char *dest,
const char *dataset,
Int_t opt = kAskUser);
void ShowDataSets();
void ShowDataSet(const char *dataset);
Int_t RemoveDataSet(const char *dateset);
Int_t VerifyDataSet(const char *dataset);
TList* GetDataSet(const char *dataset);
Revision
14164 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Mar 3 15:42:37 2006 UTC (8 years, 10 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 135627 byte(s)
Diff to
previous 13949
From Gerri:
- TXSocket.cxx, TXProofMgr.cxx:
Fix problem causing in some cases a new unwanted session being created
with the wrongs IDs.
- TProof.cxx, TVirtualProof.cxx
Make sure to store always the FQDN (if available) as master host name.
- TProofMgr.cxx
Remove redundant debug printout.
Revision
13949 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jan 30 09:01:12 2006 UTC (8 years, 11 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 135286 byte(s)
Diff to
previous 13833
TStreamerInfo was including TFile.h, TROOT.h and TClonesArray.h iso of
forward declaring them. This caused massif recompilation if e.g.
TFile.h was changed. Correct some sources that did not include
TFile.h because it was coming via TStreamerInfo.h.
Revision
13616 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Dec 10 16:51:57 2005 UTC (9 years, 1 month ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 134687 byte(s)
Diff to
previous 13565
From Gerri:
introduction of major new PROOF features. Main features:
- PROOF session manager which allows disconnect/reconnect from running
sessions
- use xrootd/xproofd as connection daemon
Much more extensive description in release notes.
Revision
13565 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Dec 9 01:12:17 2005 UTC (9 years, 1 month ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 128771 byte(s)
Diff to
previous 13236
From Paul Nilsson:
new TProofResources base class and TProofResourcesStatic concrete class that
gets the PROOF config info from the static proof.conf file. Future PROOF
resources classes will get resource info from schedulers like LSF, Condor and
xrootd/olbd.
Revision
13236 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 14 21:36:03 2005 UTC (9 years, 2 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 131814 byte(s)
Diff to
previous 13227
added kUsingSessionGui status bit in TVirtualProof. This bit is set
when a query is issued via the SessionGui and prevents the standard
progress dialog from appearing. When a query is executed from the
command line the standard progress is shown.
Revision
12822 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Sep 22 23:29:30 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 131820 byte(s)
Diff to
previous 12810
from Gerri:
- fix several issues with Begin() not being called when finalizing a query
done via TChain::Draw(), call setup code now also in Terminate().
- fix error with query progress dialog not finishing when analysing
subset of data set.
- add master and slave startup feedback code.
- fixes also a few additional problems with enabling/disabling of
Stop/Abort buttons and cleans up the treament of fOutputList in TQueryResult.
It is also make sure that there are no multiple copies of a TDSet in the
output list.
Revision
12772 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Sep 19 14:49:09 2005 UTC (9 years, 4 months ago) by
brun
Original Path:
trunk/proof/src/TProof.cxx
File length: 128537 byte(s)
Diff to
previous 12736
From Gerri:
TQueryResult.cxx:
Make debug printout conditional
TProof.cxx:
- DrawSelect does not (yet) work in asynchronous mode: protection added.
- Use "ASYN" and "SYNC" as command line options for
asynchronous/synchronous mode instead of 'A'/'S' (to avoid interferences
with options to Draw).
- Fix small problem with log redirection in case of Draw actions.
- Fix problem with conditional in TProof::Finalize
Revision
12732 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Sep 17 14:57:46 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 126279 byte(s)
Diff to
previous 12727
From Gerri:
By trying with a completely new proof.conf I also found another thing not
very solid in TProof::StartSlaveThread (the check to distinguish between
slave and master was done on something not always set correctly).
Finally, "config.h" was not included in TProof.cxx, so if ROOTLIBDIR was
not standard, libThread was not found ...
Revision
12727 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Sep 17 13:52:55 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 126175 byte(s)
Diff to
previous 12722
From Gerri:
New startup for slaves splitting TSlave::Init() in two parts:
TSlave::Init()
opens the connection and waits the "Okay" from the proofd
TSlave::SetupServ()
finalizes the setup of the server (TProofServ::Setup)
Coding convention fixes.
Revision
12722 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Sep 16 08:48:39 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 125617 byte(s)
Diff to
previous 12707
From Gerri:
Big patch supporting query queueing on the PROOF master, interogation and
interaction with the queue. Retrieval, finalization and archival of query
results and multiple PROOF sessions. Some other fixes:
- TMacro.h, .cxx
add method Checksum to get the MD5 of the current context
(needed for optimized selector reinitialization)
- Changes needed to create and fill the list of data sets (TROOT,
TDSet, TChain, TChainProof)
- A new method TSelector::IsStandardDraw() to check if a selector
is for standard draw actions
- TEventIter: fix for the missing last update (see above)
- Few protections in TPacketizers destructors
- Fix for code conventions in TPerfStats (the other will follow)
- Occasional missing last update of the progress bar (systematically
reproduced with SetParallel(1)): all this part is quite involved
and I have found a solution which does not harm but perhaps is not
the most elegant.
- SetParallel(0): the progress bar was not working at all in such a
case, since no packetizer is involved. Still it is useful to see
that things are not stuck, so I have added some progress messages
there too. The problem is that I have not found a way to get the total
number of entries from a data set without doing all the machinery
done in the packetizer.
Revision
12578 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Aug 31 11:11:06 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 111906 byte(s)
Diff to
previous 12564
From Gerri:
proof/src/TProof.cxx:
- add missing protection in Close() preventing proper cleaning
of proofservs
- fix compilation problem on Windows, Solaris, Mac
proof/inc/TProofProgressDialog.h, proof/inc/TProofProgressLog.h:
- drop unused methods whose implementation was missing (compilation
failure on Windows)
Revision
12561 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Aug 30 10:25:29 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 111780 byte(s)
Diff to
previous 12477
From Gerri:
This is the patch to add support for asynchronous queries.
The principal modifications are in TProof and TProofPlayer.
In TProof, I have separated the parsing of the received messages
from the selection of the socket (new method CollectInputFrom()):
the new method is called directly by TProofInputHandler::Notify()
in the case of asynchronous mode.
Also in TProof I have added a method to redirect the logs to a
temporary file, to avoid having them on the main window while doing
something else. The method is TProof::RedirectLog(), it uses freopen,
fdopen and dup2, and works well back and forth.
The content of the temp file can be displayed into a log window
(new GUI class TProofProgressLog), via a new button in the
progress dialog ("Show Logs").
In TProofPlayer I have created a new method Finalize() with the
final part of Process(): this is necessary to be able to finalize
the queries processed asychronously.
There are several many other small changes and fixes: a couple
of non-backward comaptibilities introduced with the previous
patch, many small issues in the GUI (the stop and abort buttons
were never disabled ...).
Also, in view of multi-sessions, it does not make much sense to
have one progress dialog per query: one per session is enough;
I have added a method ResetProgressDialog() to reset it when a
new query is started.
Finally, as anticipation of the next patch, this patch includes
support for a list of queries created by the master and described
by the new class TProofQuery.
The list is displayed by TProof::ShowQueries().
Revision
12477 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Aug 15 15:57:18 2005 UTC (9 years, 5 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 98983 byte(s)
Diff to
previous 12337
From Gerri + some mods by me:
- This patch gets rid of TFTP, slightly modifying SendFile and CheckFile.
- The factorization of the technology dependent parts of TProof, TSlave
and TProofServ. In particular, I have moved the technicalities of Ping
and Interrupt from TProof to two new virtual methods in TSlave, which
will be overwritten in TXSlave.
In TSlave I have introduced the virtual method Init which will be
overwritten by TXSlave and is able to act on an open connection for
optimal fallback a la TXNetFile.
Slaves are created by TProof via the static method TSlave::Create
which uses the plugin manager to detect the presence of TXSlave.
In TProofServ I have made the technology-dependent methods
virtual (Setup, HandleSocketInput, HandleUrgentData, HandleSigPipe,
HandleSocketInputDuringProcess).
Revision
12337 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 18 16:20:53 2005 UTC (9 years, 6 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 101658 byte(s)
Diff to
previous 12315
From Gerri:
Big patch restructuring the authentication code so libCore does not depend
on libssl etc. anymore. Isolate the current authentication code behind a
plugin library to be loaded on demand.
Setup of a framework able to manage both sets of authentication modules
(the current one and the one based on xrdsec, coming soon).
Cleanup all direct reference to authentication in TSlave and TProofServ,
allowing for significant simplification for the forthcoming changes
in PROOF for XPD.
New module: auth
New files: base/inc/TVirtualAuth.h
auth/Module.Mk
auth/inc/LinkDefRoot.h
auth/inc/TRootAuth.h auth/inc/TRootSecContext.h
auth/src/TRootAuth.cxx auth/src/TRootSecContext.cxx
Moved files:
net/inc -> auth/inc : TAuthenticate.h THostAuth.h DaemonUtils.h AuthConst.h
net/src -> auth/src : TAuthenticate.cxx THostAuth.cxx DaemonUtils.cxx
Revision
12123 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 22 20:18:12 2005 UTC (9 years, 7 months ago) by
brun
Original Path:
trunk/proof/src/TProof.cxx
File length: 101629 byte(s)
Diff to
previous 11988
From Constantin Loizides
This patch implements:
- decentralized, automatic mutex initialization,
see R__LOCKGUARD2 in TVirtualMutex
- PROOF parallel startup fixes
- more thread protection (in base, cont, meta, rest to be done)
- cleanups
From Eddy Offermann:
TString::Atoi and Atof are made const.
Revision
11813 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed May 18 12:31:10 2005 UTC (9 years, 8 months ago) by
brun
Original Path:
trunk/proof/src/TProof.cxx
File length: 128729 byte(s)
Diff to
previous 11730
From Valeriy Onuchin:
this patch fixes all compilation problems for
ROOT under windows with VC6 compiler.
- The main problem is that VC6 compiler does not recognize
"long long" type in rootcint generated dictionaries,
To fix this, wherever "long long" type was writen in dictionary file it was
replaced with native win32 "long long" type_string "__int64".
Modified files are: cint/src/val2a.c, cint/src/newlink.c, cint/src/Class.cxx
Other changes:
- many classes contained "list" variables.
That produced name_clash with std::list class name.
- many classes contained
for (int i=0; ..) {..}
for (int i=0; ..) {..}
which caused "duplication of initialisation" bug.
- gl/src/CsgOps.cxx had outside declared templated parameter used inside
internal function.
- TString Atoi, Atof - std namespace was removed.
_ TASimage - custom Streamer was changed. Reading/writing class version
was added. That allows to keep "future compatibility"/"schema evolution"
Demo files galaxy.root, gallery.root were rewritten.
- I rolled back changes in TGPicture class which are related to reading
of XPM files because of problems under win32.
- TASImage::GetMask method was reimplemented.
- libAfterImage.tar.gz - fix all compilation problem under win32 (vc6)
Revision
11687 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Apr 28 16:14:28 2005 UTC (9 years, 8 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 127015 byte(s)
Diff to
previous 11475
From Gerri:
Patch introducing parallel startup of the slave servers. This new feature
will considerably reduce the startup time in PROOF setups with many 10's
to 100's of slaves. By default this feature is off, turn it on in the
[system].rootrc file by uncommenting the line:
Proof.ParallelStartup: yes
Summary of changes:
Index: Makefile
- Add the thread lib to PROOFLIBS
Index: base/inc/TVirtualMutex.h, base/src/TVirtualMutex.cxx
- Add global mutexes to protect printing and authentication
Index: base/src/TError.cxx
- Protect static buffer in DebugPrint with specific global mutex
Index: config/Makefile.depend
- Add dependence on thread lib for PROOF
Index: config/rootrc.in
- Add description of new envs
Proof.ParallelStartup, Auth.Timeout, ProofServ.UseSSH
Index: net/inc/TAuthenticate.h
- Add new member fTimeout to indicate timeout situations
and new method HasTimedOut() to retrieve it.
- Add new static member with the timeout value in sec
and method SetTimeOut() to set it
- Add new method CatchTimeOut() called at timeout expiration
Index: net/inc/THostAuth.h , net/src/THostAuth.cxx
- Add new method SetLast() to lower pririty of an attempted
protocol that has timed-out
Index: net/src/TAuthenticate.cxx
- Implementation of the new timeout setup using TTimer
- Add protection using specific global mutex
- change to start with lower case some local variables
in Authenticate
- re-introduce call to THostAuth::CountFailure() (which got
lost in a previous patch)
- Disable by default SSH for PROOF servers
- fix a couple of small memory leaks
Index: net/src/TPSocket.cxx
- Fix logic in Close() (if called too early (e.g. timeout)
the underlying socket can still be open and needs to be
closed).
Index: net/src/TSocket.cxx
- initialize member fTcpWindowSize in all constructors
- modify error logging after an unsuccessful authentication
attempt, checking for timeouts.
- disable SSH for PROOF servers
Index: proof/inc/TProof.h
- patch for parallel startup using threads:
- new methods SlaveStartupThread(void *) and
SubmasterStartupThread(void *) executed in threads
- add Bool_t argument to all implementations of StartSlaves
Index: proof/src/TProof.cxx
- patch for parallel startup using threads:
- add internal classes ProofThread and ProofThreadArg to keep
track of started threads and to pass arguments to threads
- cleanup fPlayer and fFeedBack in destructor
Index: rpdutils/src/rpdutils.cxx
- fix small memory leak
Index: thread/src/TThread.cxx
- initialize new global mutexes
Revision
11373 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Mar 18 22:41:27 2005 UTC (9 years, 10 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 110538 byte(s)
Diff to
previous 11363
From Marek:
- TH1::Merge() adding overflows even when merging histograms with the
same limits. If the limits are different the overflows are ignored.
The same in TH2::Merge, TH3::Merge, TProfile::Merge and TProfile2D::Merge.
- Cleaning fHistogram (and this kind of) variables, so that the drawn
plots aren't deleted in the TProofDrawXXX's destructor.
- Removed TProofNTuple from TProofDrawTGraph.
- Put the initialization code of fFeedback and fPlayer (=MakePlayer())
in TProof in to Init().
- Added TGraph::Merge(). Added TPolyMarker3D::GetPoint(Int_t, Float&,
Float&, Float&).
- Removed TProofNTuple from TProofDrawPolyMarker3D.
- Fixed cleaning output lists after processing (set option = "nodelete").
Now the objects are on the player outputlist and are removed (but not
deleted) from the selector's outputlist.
- Fixed TPolyMarker3D::Merge() - it seems that nobody has ever tried to
run this function.
- Added setting tree attributes (like MarkerStyle, MarkerSize, LineColor etc.)
to a drawn TGraph and TPolyMarker3D in TProofDrawGraph and
TProofDrawPolyMarker3D.
Revision
11281 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Mar 10 17:57:05 2005 UTC (9 years, 10 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 111508 byte(s)
Diff to
previous 11264
From Marek Biskup:
major new PROOF developments:
- access to PROOF directly via the TChain interface (creating a TDSet
in the background).
- access to PROOF based trees via the TTreeViewer
- fully implemented the TChain/TDSet::Draw() interface.
- PROOF sessions can now be browsed via the TBrowser
Revision
11047 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 7 18:02:37 2005 UTC (9 years, 11 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 105536 byte(s)
Diff to
previous 10608
From Maarten and Kristjan Gulbrandsen:
Here is the patch with the restructured PROOF code. It parameterizes
PROOF startup, currently implementing "Standard", Condor, MLM and PEAC.
It can easily be extended with other "TProof" classes, e.g. AliEn Proof.
Or what ever next Grid implementation or scheduler comes along.
The MLM, Multi Level Master, code uses a basic packetizer, based on a
MSD (massstorage domain) tag. In MLM mode we have multi-tier client
server setup, like:
client -> super master -> sub master 0 -> slave[0..N]
...
-> sub master M -> slave[0..N]
It also contains a large number of smaller fixes. Credit for most of
the work goes to Kris!
We have tested this on multipe platforms (RH7.3, RH9, SL3) and in
multiple configurations. Should be stable.
Revision
10608 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Nov 24 07:41:32 2004 UTC (10 years, 2 months ago) by
brun
Original Path:
trunk/proof/src/TProof.cxx
File length: 68670 byte(s)
Diff to
previous 10328
From Maarten Ballintijn and Kristj�n Gulbrandsen
a patch and files that complete the benchmark package.
I strongly recommend using this for demos rather then the H1 example.
The README file should walk you through a complete demo, which should
not take more then maybe ten minutes or so. After you have some
data files generated, you can show significant parts of the sytem
in a minute or two.
I added a feedback request api to TProof, this simplifies things
a lot from a usability perspective. I also modified TDrawFeedback
to use gProof by default, again simplifying things.
Kris wrote the support scripts and fixed the benchmark
scripts to read the full Events.
Please try it out and have fun!
It is as simple as this (I use log level 2 to slow things
down a bit :-))
gROOT->Proof()
.L make_tdset.C
TDSet *d= make_tdset("/data1/tmp",2)
TDrawFeedback fb
gProof->AddFeedback("PROOF_ProcTimeHist")
gProof->AddFeedback("PROOF_LatencyHist")
gProof->AddFeedback("PROOF_EventsHist")
gProof->UploadPackage("event.par")
gProof->EnablePackage("event")
gEnv->SetValue("Proof.StatsHist",1);
gProof->SetLogLevel(2)
d->Process("EventTree_Proc.C")
Revision
10305 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Oct 13 15:34:18 2004 UTC (10 years, 3 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 67169 byte(s)
Diff to
previous 9562
add status word to TSlaveInfo to keep track if a slave is active,
not-active or bad. TSlaveInfo::Print() supports now the "active",
"notactive" and "bad" options, e.g.:
gProof->GetSlaveInfo()->Print("", "notactive")
or
gProof->GetSlaveInfo()->Print("opla*", "bad")
are now supported.
Revision
9322 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 25 17:27:09 2004 UTC (10 years, 7 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 66023 byte(s)
Diff to
previous 9302
From Maarten:
- new class TParameter
- new class TDrawFeedback
- fixes to the feedback of performance histograms
- selection of the packetizer
- cleanups
Example commands:
gROOT->Proof()
... package setup ...
TDSet *d = make_tdset(2)
gEnv->SetValue("Proof.StatsTrace",1)
gEnv->SetValue("Proof.StatsHist",1)
TList *fb = new TList
fb->SetName("FeedbackList")
fb->Add(new TObjString("EventsHist"))
fb->Add(new TObjString("CpuTimeHist"))
TDrawFeedback *dfb = new TDrawFeedback((TProof*)gProof)
gProof->AddInput(fb)
d->Process("EventTree_Proc.C","")
Revision
9302 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 24 14:54:26 2004 UTC (10 years, 7 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 64424 byte(s)
Diff to
previous 9280
Following changes:
- change "node" line to "master" in PROOF config file (of course "node"
remains supported too). In addition to "slave" also "worker" is recongnized
to describe slave servers.
- first look for a "master" line in the config file, if the current node is
not a valid master return. Previously slaves were directly created while
reading of the file and then the session could still be terminated in case
the master line was not correct (after possibly having started many slaves).
- in Collect(), mess can be 0 in case the remote server died, this was
not protected.
Revision
8746 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Apr 29 23:43:38 2004 UTC (10 years, 8 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 63106 byte(s)
Diff to
previous 8620
From Maarten:
Can now use 1 slave server:
- 1 or more slaves -> parallel (i.e. run on slave servers)
- 0 slaves -> sequential (execute on/in master)
before the option of 1 slave was not possible, because we always
reverted in that case to running on the master.
Revision
8239 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Feb 19 00:11:19 2004 UTC (10 years, 11 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 62079 byte(s)
Diff to
previous 7810
Big authentication patch by Gerri:
General purpose:
Add support for 'authenticated' sockets on client side
and prepare for support of server authenticated sockets.
Main changes:
o TSocket, TPsocket:
Added support of 'authenticated' sockets: authentication
is delegated to TSocket by calling a static method
CreateAuthSocket() returning a TSocket*.
TSocket is now the only class calling TAuthenticate.
o TFTP, TNetFile, TSlave:
modified to cope with new socket authentication scheme.
o TAuthDetails replaced by TSecContext containing:
- local authentication info
- expiration time
- pointer to a security context for the method (if any)
- information for cleanup (via TSecContextCleanup)
o A list of valid TSecContext has been added to TROOT,
accessible via gROOT->GetListOfSecContexts().
o THostAuth rewritten using arrays to avoid multiple use
of 'new'; functionality extended by the addition of
several new methods.
o TAuthenticate:
- Loading of information from .rootauthrc has been optimized;
this allowed to simplify method signatures and drop several
methods. Also collection and transmission of authentication
information from TSlave to TProofServ gets simplified by the
new scheme.
- Added support for server dependent directives in .rootauthrc
- Added check for changes in .rootauthrc each time a new
TAuthenticate is instantiated, so that changes in the file
are picked-up interactively.
- Checking for an existing TSecContext (previously TAuthDetails)
has been improved
- Improved diagnostics for SshAuth
- Enlarged support for authentication method indication in protocol:
UsrPwd: rootup/proofup
SRP: roots/proofs (already supported)
Krb5: rootk/proofk (already supported)
Globus: rootg/proofg
SSH: rootsh/proofsh
UigGid: rootug/proofug
o GlobusAuth.cxx, Krb5Auth.cxx, SRPAuth.cxx
- Adapted to new class TSecContext
- Added support for improved search for reusable TSecContext
- Drop globals used to store established security contexts
(saved in TSecContext) and related methods
o Automatic creation of <RootDir>/etc/system.rootauthrc from
compilation flags; this has priority on old envs
'Rootd.Authentication' and 'Proofd.Authentication' and is
superseded by $HOME/.rootauthrc
o Automatic creation of <RootDir>/etc/system.authdaemonrc from
compilation flags; this is superseded by $HOME/.rootauthrc
o rpdutils:
- several almost identical methods moved in from rootd/proofd
- several new methods to simplify manipulation of the
authentication tab file
- moved (and improved) check for running of sshd in RpdSshAuth,
performed only in case of failure (to avoid spurious messages
printed by sshd in /var/log/messages)
o proofd.cxx, rootd.cxx
- increased protocol version
- all initialization stuff (included authentication and
login) done via RpdInitSession
- postpone opening of parallels sockets after authentication
(to solve problem of Kerberos/Globus authentication with
parallel sockets)(rootd)
- Fixed problem with kerberos and globus authentication in rootd
with parallel sockets.
Revision
7810 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Dec 30 13:16:51 2003 UTC (11 years ago) by
brun
Original Path:
trunk/proof/src/TProof.cxx
File length: 61987 byte(s)
Diff to
previous 7682
Mega patch to add support for large files (bigger than 2 GBytes)
----------------------------------------------------------------
==========>NEW VERSION 4.00/00 <===========
Because this new version has a considerable number of changes,
and new important additions by Philippe are scheduled for the coming days
we are starting a new major version 4.
-Support for large files
-Automatic schema evolution for foreign classes
-New data type Double32_t
Large files are currently tested only under Linux with gcc3.2.
Support for other systems will be gradually added in the coming days.
By default under Linux, files are created with the option LARGEFILE.
Note that when creating a Tree, the default maximum size for a Tree
is set to 1.9 GBytes. You can change this default value by calling
TTree::SetMaxTreeSize(Long64_t maxsize)
The implementation of this new feature has required changes in many places.
The files produced by this new version can still be read by old ROOT versions
if their size if smaller than 2 GBytes. And obviously, the new version
can digest old ROOT files.
WARNING: note the following backward incompatibility:
-TSystem::GetPathInfo has a new signature: (also TUnixSystem, TWinNTSystem, etc)
old: GetPathInfo(const char *path, Long_t *id, Long_t *size, Long_t *flags, Long_t *modtime);
new: GetPathInfo(const char *path, Long_t *id, Long64_t *size, Long_t *flags, Long_t *modtime);
-TFile::SysSeek and TFile::SysStat have a new signature (also TNetFile, TWebFile, etc)
old: Int_t SysSeek(Int_t fd, Long_t offset, Int_t whence);
Int_t SysStat(Int_t fd, Long_t *id, Long_t *size, Long_t *flags, Long_t *modtime);
new: Long64_t SysSeek(Int_t fd, Long64_t offset, Int_t whence);
Int_t SysStat(Int_t fd, Long_t *id, Long64_t *size, Long_t *flags, Long_t *modtime);
-TTree::SetMaxTreeSize has a new signature:
old: void TTree::SetMaxTreeSize(Int_t maxsize)
new: void TTree::SetMaxTreeSize(Long64_t maxsize)
All references to Seek_t have been replaced by Long64_t.
Revision
7406 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Oct 7 14:03:03 2003 UTC (11 years, 3 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 61913 byte(s)
Diff to
previous 7373
From Gerri:
clib/inc/rsalib.h, clib/src/rsalib.c
o Random seed initialization moved from rsalib.c to in rootd.cxx,
proofd.cxx and TAuthenticate.
o Introduced counter to avoid infinite loops during key generation
etc/rpdauth.allow
o Introduced default with all authentication methods allowed in
rpdauth.allow
globusauth/src/GlobusAuth.cxx
o Secure RSA public transmission to servers (receiving a server
key)
o Added test on Argc() before use of the Argv() arguments
o Use isatty to check if interactive session
krb5auth/src/Krb5Auth.cxx
o Secure RSA public transmission to servers (receiving a server
key)
o Added test on Argc() before use of the Argv() arguments
o Use isatty to check if interactive session
net/inc/TAuthenticate.h, net/src/TAuthenticate.cxx
o Secure RSA public transmission to servers (receiving a server
key)
o Added possibility for a "default" THostAuth instantiation valid
for all hosts if anything more specific is not found.
Used by TProofServ/TAuthenticate for host without specific
authentication directives in proof.conf .
o Random seed initialization moved from rsalib.c to in rootd.cxx,
proofd.cxx and TAuthenticate.
o Introduced counter to avoid infinite loops during key generation
o Fixed potential problem with SSH master-to-slave authentication;
force failure if, for some reason, the slave is not accepting
the standard key files (depending on the user ssh config
directives, this could cause interactive prompt for password
on the master, therefore hanging the system).
Specific directives to avoid prompting are passed via command
line with the '-o' ssh option.
o Fixed problem with forming the message used to notify an SSH
failure
o Use mkstemp instead of tmpnam
o Use isatty to check if interactive session
o Avoid creating duplicates of TAuthDetails instantiation for
offset = -1
o Fixed a few inconsistency of debug levels error messages after
failed authentications
o Fixed a few inconsistency between authentication method names
and ROOT environment variable names
o Fixed problem with null passwd sending in ClearAuth
o Fixed problem with method attempting order during negotiation
o Add possibility to avoid check on localhost for testing purposes
(in method GetAuthMeth)
net/src/THostAuth.cxx
o Added possibility for a "default" THostAuth instantiation valid
for all hosts if anything more specific is not found.
Used by TProofServ/TAuthenticate for host without specific
authentication directives in proof.conf .
proof/inc/TProof.h, proof/src/TProof.cxx
o Moved collection of Authentication information for slaves from
TProof to TProofServ
proof/inc/TProofServ.h, proof/src/TProofServ.cxx
o Improved transmission of RSA public key between TSlave and
TProofServ
o Moved collection of Authentication information for slaves
from TProof to TProofServ
o Added test on Argc() before use of the Argv() arguments
proof/src/TSlave.cxx
o Improved transmission of RSA public key between TSlave and
TProofServ.
Added possibility to securely forward the SRP password to
the master (ROOT environment variable: Proof.SendSRPPwd,
default 0 - off).
o Removed checks on Proofd.Authentication in TSlave; security
stuff is setup in TProofServ and used in TAuthenticate
proofd/src/proofd.cxx
o Random seed initialization moved from rsalib.c to in rootd.cxx,
proofd.cxx and TAuthenticate.
rootd/src/rootd.cxx
o Random seed initialization moved from rsalib.c to in rootd.cxx,
proofd.cxx and TAuthenticate.
rpdutils/inc/rpdp.h, rpdutils/src/rpdutils.cxx
o Secure RSA public transmission from clients (sending a server
key)
o Fixed ambiguity with method index resolving from name due to
the fact that 'usrpwd' contains the string 'srp'
o Add file existence check before trying open
o Add a few debug messages
o Fixed problem with domain checking in daemon.access
('cern.ch uidgid' or '.cern.ch uidgid' were not accepted)
o Fixed problem with null passwd receiveing in RpdUser
rpdutils/src/ssh.cxx
o Fixed function name in debug message in ssh.cxx
o Use mkstemp instead of mktemp
srputils/src/SRPAuth.cxx
o Secure RSA public transmission to servers (receiving a server
key)
Revision
7320 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Sep 23 08:54:50 2003 UTC (11 years, 4 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 77785 byte(s)
Diff to
previous 7275
From Maarten:
- TCondor
logging cleanup
fixed proofd port allocation
dynamic COD jobad
- Tempfile stuff (could be extended for other uses as required)
TSystem::TempDirectory()
TSystem::TempFilename()
using these methods removes the use mkstemp warning on linux
- minor cleanups
Revision
7275 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Sep 12 17:36:35 2003 UTC (11 years, 4 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 77755 byte(s)
Diff to
previous 7262
Few additional patches from Gerri:
- TSlave.cxx, TProofServ.cxx
send over the RSA key only when needed to decode the passwd
(ie UsrPwd and SRP)
- TAuthenticate.cxx
allow short host name in .netrc family files
- TProof.cxx
force using UidGid when the slave is on the same node as the master
- SRPAuth.cxx
Drop a left-over debug message
Revision
7262 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Sep 11 23:12:19 2003 UTC (11 years, 4 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 77076 byte(s)
Diff to
previous 7226
From Gerri:
This patch contains Maarten's suggestions, and the fixes I have done
since then.
There was problem in the transmission of the password to master/slaves
that I believe was causing the hanging ...
Revision
7226 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Sep 7 18:25:47 2003 UTC (11 years, 4 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 73876 byte(s)
Diff to
previous 7198
From Gerri:
- configure Maarten's comments + fix for some SRP libraries
not being found
- globusauth/Module.mk
Maarten's comments
- net/inc/TAuthenticate.h
net/src/TAuthenticate.cxx
net/src/TAuthDetails.cxx
net/src/THostAuth.cxx
net/src/TNetFile.cxx
proof/src/TProof.cxx
proof/src/TSlave.cxx
Unified AuthMeth[kMAXSEC] (your comments ...)
+ small consistency fixes (Host named checked
as FQDN everywhere ...)
- proofd/src/proofd.cxx
rootd/src/rootd.cxx
rpdutils/inc/rpdp.h
rpdutils/src/rpdutils.cxx
Unified AuthMeth[kMAXSEC]
+ gShmIdCred fix
Fons:
- TAuthenticate: simplified way to find index of auth method.
Revision
6243 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Mar 4 17:29:29 2003 UTC (11 years, 10 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 55813 byte(s)
Diff to
previous 5717
implementation of package manager. Package manager allows the distribution,
installation and setting up of user code in the PROOF sandbox environment.
The package manager distributes so called PAR files, which are standard
tarred and gziped files, but with extension .par. A file "mylibs.par" is
required to untar into a directory "mylibs". A PAR file contains in addition
to the user's code and libraries one special directory PROOF-INF that contains
two scripts: BUILD.sh and SETUP.C. BUILD.sh is a shell script that is
used to build the user's libraries on the PROOF cluster (which might be
of a different architecture/compiler then the user's machine). SETUP.C is
a ROOT macro that is used to load the user's libraries, built by BUILD.sh,
and to do any other kind of setup needed to run with the user libraries.
These scripts are executed in the package directory (not the PROOF-INF
directory). An example of a BUILD.sh is:
#! /bin/sh
make libEvent.so
An example of SETUP.C is:
void SETUP()
{
gSystem->Load("libEvent");
}
The package manager API is:
gProof->UploadPackage()
gProof->EnablePackage()
gProof->ClearPackage()
Revision
5717 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Dec 2 18:50:12 2002 UTC (12 years, 1 month ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 59159 byte(s)
Diff to
previous 5692
mega patch to remove almost all compiler warnings on MacOS X where the
compiler is by default in pedantic mode (LHCb also like to use this option).
The following issues have been fixed:
- removal of unused arguments
- comparison between signed and unsigned integers
- not calling of base class copy ctor in copy ctor's
To be done, the TGeo classes where we get still many hundred warnings of
the above nature. List forwarded to Andrei.
Revision
4955 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 17 12:29:38 2002 UTC (12 years, 6 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 57297 byte(s)
Diff to
previous 4720
PROOF mods from Maarten. New TVirtualPacketizer that is the base class for
different packetizers. You want different packetizers for different
processing strategies (depending on files being available on disk or if
they need to be sequentially retrieved from mass storage, etc.).
Added global PROOF debug facility.
Revision
4210 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Mar 20 18:54:57 2002 UTC (12 years, 10 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 51075 byte(s)
Diff to
previous 4193
add cases for Kerberos5 authentication. In particular the default authentication
method for rootd and proofd can now be set in system.rootrc. The default is
clear text passwd, but SRP and Krb5 can be specified. The default is used when
"root:" or "proof:" appears in the url. The default can be overridden by
specifying explicitely "roots:" or "rootk:", idem for proof.
Revision
4182 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Mar 15 17:23:40 2002 UTC (12 years, 10 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 50541 byte(s)
Diff to
previous 4167
implemented TProof::CheckFile() which is called by SendFile() to make sure
files are not send when not needed. They are only send when the md5 is
changed. It also makes a check on the remote nodes with unique file system
to see if the file is already there (by sending the md5 over in a message
of type kPROOF_CHECKFILE).
Revision
3699 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jan 18 14:24:10 2002 UTC (13 years ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 44980 byte(s)
Diff to
previous 3657
relocate some proof file to remove circular dependencies between the
treeplayer and proof shared libraries. Also rename TPlayer to TProofPlayer
et al. Also remove Begin() from TSelector, use Begin(0) instead. This to
be backward compatible with existing TSelector scripts that don't have
Begin() and therefor give warnings about Begin() being hidden in the base
class.
Revision
1750 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Mar 5 15:34:32 2001 UTC (13 years, 10 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 43412 byte(s)
Diff to
previous 1281
add PROOF specific interrupt handler class for handling ctlr-c when PROOF
is active. Is possible due to a recent fix in TSystem which now correctly
calls the Notify()'s of all handlers registered for a signal (before
only the Notify() of the first handler for a trapped signal was called).
Revision
1205 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 13 15:13:57 2000 UTC (14 years, 1 month ago) by
brun
Original Path:
trunk/proof/src/TProof.cxx
File length: 43613 byte(s)
Diff to
previous 1203
W A R N I N G !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================================================
A very long list of changes in this pre-release of version 3.00.
We have modified the signature of many functions (in particular TObject)
to introduce more constness in the system.
You must change your code if your class derives from TObject and uses
one of the modified functions such as ls, Print, Compare, Hash, etc.
The modified functions in TObject have the following signature:
virtual TObject *Clone() const;
virtual Int_t Compare(const TObject *obj) const;
virtual void Delete(Option_t *option=""); // *MENU*
virtual void DrawClass() const; // *MENU*
virtual void DrawClone(Option_t *option="") const; // *MENU*
virtual void Dump() const; // *MENU*
virtual TObject *FindObject(const TObject *obj) const;
virtual char *GetObjectInfo(Int_t px, Int_t py) const;
virtual ULong_t Hash() const;
virtual void Inspect() const; // *MENU*
virtual Bool_t IsEqual(const TObject *obj) const;
virtual void ls(Option_t *option="") const;
virtual void Print(Option_t *option="") const;
A similar operation has been done with classes such as TH1, TVirtualPad,
TTree, etc.
Revision
106 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Jun 11 12:25:48 2000 UTC (14 years, 7 months ago) by
rdm
Original Path:
trunk/proof/src/TProof.cxx
File length: 26924 byte(s)
Diff to
previous 3
now use 1093 as port number for proofd service. This port number has been
assigned by IANA for proofd usage. In the old scheme we used two ports
for PROOF: one for the master server (proofserv) and another for the client
servers (proofslave). This has now been changed to a single port (proofd)
and the server type is communicated by the client to the server.
Update the /etc/services file from:
proofserv 349/tcp
proofslave 359/tcp
rootd 432/tcp
to
proofd 1093/tcp
rootd 1094/tcp
And /etc/inetd.conf from:
proofserv stream tcp nowait root /usr/proof/bin/proofd proofserv /usr/proof
proofslave stream tcp nowait root /usr/proof/bin/proofd proofslave /usr/proof
to
proofd stream tcp nowait root /usr/proof/bin/proofd /usr/proof
This form allows you to request diffs between any two revisions of this file.
For each of the two "sides" of the diff,
enter a numeric revision.
