Log of /trunk/proof/proof/inc/TProofMgr.h
Parent Directory
Revision
42858 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Feb 3 11:34:05 2012 UTC (2 years, 11 months ago) by
ganis
File length: 8684 byte(s)
Diff to
previous 41904
This patch adds in TProofMgr two static functions to upload files.
These functionare called UploadFiles and differ in the first argument, which is used
to pass the files to be uploaded. These can be given as a list (of TFileInfo or TObjString),
a directory or specified in a text file.
The functions are documented at
http://root.cern.ch/drupal/content/uploading-data-files-proof-cluster
The patch also adds TProofMgr::GetMssUrl() and TProofMgr::SetMssUrl(...) to manage the mass
storage URL attached to a given cluster. By the default the information is retrieved by the
server daemon, but can be modified by the user. The functions TProof::GetDataPoolUrl() and
TProof::SetDataPoolUrl(...) now redirect to TProofMgr::GetMssUrl() and TProofMgr::SetMssUrl(...).
The obsolete functions TProof::UploadDataset have now been removed.
This patch addresses some issues raised on Savannah.
Revision
41904 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Nov 11 11:17:00 2011 UTC (3 years, 2 months ago) by
ganis
File length: 8088 byte(s)
Diff to
previous 40193
Extend TProofMgr::Ping functionality to 'xrootd' too. This is useful to test if
an xrootd server is available at a given address. There is a new second argument,
a boolean to switch to that:
TProofMgr::Ping("<host>[:<port>]", checkxrd)
The default for checkxrd is kFALSE, so that the old behaviour is kept.
The patch also fixes the default port values, so that specifying 1093 is not needed.
Revision
40193 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 11 16:07:08 2011 UTC (3 years, 6 months ago) by
ganis
File length: 8062 byte(s)
Diff to
previous 38789
- In TProofMgr, add 'ping' functionality to test in non-blocking way if a PROOF
service is listening at a given port of a given host (adapted from the macro
function CheckXproofdAt in getProof.C) .
Practical in many occasions.
Revision
38789 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Apr 9 18:03:24 2011 UTC (3 years, 9 months ago) by
pcanal
File length: 8019 byte(s)
Diff to
previous 38580
TProofMgr: Anything that is added to the list of Sockets must have a Close method.
TROOT: properly lookp over fSockets (knowning Close remove the object from the list)
and properly look up Close for non-TSockets.
Revision
38580 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Mar 23 15:24:54 2011 UTC (3 years, 10 months ago) by
ganis
File length: 7970 byte(s)
Diff to
previous 30791
Patch fixing a problem with log path transmission when the node dies early or not even starts.
The log path was empty and wrong was filled in when retrieving the log buffers, disorienting
debugging. This was experienced frequently by ALICE.
The log file is now trasmitted as part of the responde to the session 'Create' request, so it
is always registered by the master in the relevant field in TSlave. Also, the last errors, if
any, typically logged in the xproofd log file, are now copied to the relevant worker or master
log file so that they are also accessible to client.
The patch also factorizes some calls in XrdProofdProofServMgr in preparation to the improved
session lauch strategy.
Revision
28746 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu May 28 13:03:29 2009 UTC (5 years, 7 months ago) by
ganis
File length: 6771 byte(s)
Diff to
previous 27442
Cleanup of the TProofMgr fucntions DetachSession and ShutdownSession, and better
handling of the internal list registration, to fix potential segvs when reopening
a PROOF session inside the same ROOT session.
Now, for both normal PROOF and PROOF-Lite, it should be possible to close a
session handled by the TProof object 'p' with any of the following:
root[] delete p
or
root[] TProof::Mgr("<server>")->ShutdownSession(p)
or
root[] TProof::Mgr("<server>")->ShutdownSession(id_p)
(where id_p is the local session id given by TProof::Mgr("<server>")->QuerySessions())
and re-open a session on the same server right after.
Other fixes / changes:
- New auxilliary methods:
- TProofMgr::DiscardSession(TProof *) to remove a session from the internal
list of a manager;
- TProofDesc *TProofMgr::GetProofDesc(TProof *) to quickly get the TProofDesc
container associated with a session by object
- Make sure that the default port is always 1093 (in a couple of places it was 1094!)
- Improve some comments
Revision
26951 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Dec 16 12:04:38 2008 UTC (6 years, 1 month ago) by
ganis
File length: 6569 byte(s)
Diff to
previous 26823
- Fix an issue with uploading packages in rapid sequence
- Fix an issue with TProof::Load() in forced compilation mode ('++')
- Fix an issue with grep in log retrieval
- Fix an issue with ownership of the xrootd log file
- Make sure the XrdProofdProofServMgr::fCurrentSessions is always up-to-date
Revision
26823 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Dec 11 09:26:32 2008 UTC (6 years, 1 month ago) by
ganis
File length: 6565 byte(s)
Diff to
previous 25918
- In logs. filter-out by default the service messages used for memory monitoring;
a new check box has been added to the log dialog box to force display of those
messages; this reduces considerably the number of 'noise' messages and also the
default soze of logs.
Revision
25918 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Oct 22 15:00:04 2008 UTC (6 years, 3 months ago) by
ganis
File length: 6553 byte(s)
Diff to
previous 24588
First version of PROOFLite, a 2-tier realization of PROOF intended for multi-core
machines. In PROOFLite the client start directly the workers; no daemon is required.
Currently this works on Unix only. Some issues with the internal connection still need
to be sorted out on Windows.
To start a session just use
TProof::Open("") or TProof::Open("lite")
From there on everything should be as in normal PROOF. To start a standard PROOF
session (i.e. via daemons) on the localhost use TProof::Open("localhost").
(The default meaning of "" in TProof::Open() is controlled via the rc value
'Proof.LocalDefault').
To run stressProof with PROOFLite pas the argument 'lite':
./stressProof lite
By default, PROOFLite sets the number of workers equal to the number of cores.
The number of workers is controlled by the rc variable 'ProofLite.Workers'; it can
also be set on the fly with the option "workers=<n>", e.g. TProof::Open("lite?workers=2").
This patch also introduces the rc variable 'Proof.Sandbox' to control the local location
of the sandboxes. The default has been changed from "~/proof" to "~/.proof" to avoid
interferences with possible users' working areas. The sandbox contains the directory
'packages' (with the clients packages; used by standard PROOF and PROOF Lite),
and some directories used only by PROOFLite: 'cache' (binary cache) and 'datasets' with
the datasets. The location of these directories is controlled by the rc variables
'Proof.PackageDir', 'Proof.CacheDir' and 'Proof.DataSetDir'.
The working directory for PROOFLite is in the form workdir=<sandbox>/path-to-current-dir where
~/path/to/current/dir is the directory from where the session in launched. The directory
<workdir> contains the session working dirs (in the form 'session-<host>-<time>-<pid>'),
with the symlink 'last-lite-session' pointing to the last started session, and the 'queries'
directory, with the results of queries started from the current directory.
Revision
24588 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 27 13:52:23 2008 UTC (6 years, 6 months ago) by
ganis
File length: 6461 byte(s)
Diff to
previous 24540
Few additions in preparation of the new XrdProofd:
- New signature for TProofMgr::AttachSession to be used internally in case of
reconnections
- Accomodate support for hard/soft Reset operations.
Revision
20862 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Nov 18 14:51:42 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proof/inc/TProofMgr.h
File length: 6104 byte(s)
Diff to
previous 20682
Synchronize with braches/dev/proof r20835
Summary:
TProofServ:
- read session.rootrc with level kEnvChange to be able to change existing
settings
- Avoid deleting a query result twice in some special cases
- Fix a problem with the initialization of fKeptQueries
rootrc.in:
- restore default settings the asynchronous reading
TFileCacheRead:
- add missing protection in ReadBuffer
TEventIter:
- Enable the usage of TTreeCache
XrdProofdProtocol:
- Additional check on the ownership of the unix socket
- Improve notification during Reset
- Reduce default timeout on admin requests and make it configurable
XrdProofConn:
- Use the configurable maxtry everywhere where relevant
TPacketizerAdaptive:
- Store info on all the processed packets in per-worker lists
TProofMgr, TXProofMgr, TXSocket, XProofProtocol, XrdProofdProtocol:
- Add possibility for the admin to broadcast a message to the connected users
getProof:
- Add missing protection
Revision
20682 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 6 15:51:59 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proof/inc/TProofMgr.h
File length: 6033 byte(s)
Diff to
previous 19826
Import branches/dev/proof r20654
Summary:
+ Improvements
- Add support for SSH SOCKS4 tunnelling; the local port for the tunnel can be specified
in the master URL, e.g. TProof::Open("master/?tunnel:8000")
- Add the possibility to plot the estimated instantaneous rate
- Add "PROOF_ForceLocal" parameter to the !TPacketizerAdaptive; if set to 1, all the data
are processed locally.
- Add support for remote grep functionality while retrieving logs (needed by the forthcoming
memory monitor)
+ Bug fixes
- Several small fixes to revive the multi-master mode.
- XrdProofdProtocol:
- add missing lock to the client instance in SendMsg to avoid screwing up requests from
workers on the same machine
- lock the mutex of the requester when setting prorities
- add notification during Reset
- fix problem with the detection of the 'allow' directive
- fix problem with the parsing of the return value from XrdProofServProxy::TerminateProofServ()
- always use the effective user to retrieve info fom another server
(XrdProofdManager::GetProofConn is now used)
- fix possible dead-locks from debug notifications done after hard-killing a session
- re-enable the garbage collector thread of the connection manager in XrdProofConn to
fix a problem with closing physical connections;
- fix a problem with !CleanupProofServ in the case of a non-privileged daemon running
in multi-user mode
- introduce a timeout when waiting for the startup of a 'proofserv'.
- XrdProofConn: init mutex in the ctor; lock in SendRecv
- XrdProofSched: add support for using the priorities defined in the group manager to define
the number of workers for sessions
- TProof:
- Broadcast priorities to unique nodes only
- timeout after 5 mins the initial Collect to avoid clients getting stuck at this stage
- add support for generic timeout in Collect (disabled by default)
- fix a problem with SendFile.
- TXProofServ: add a call to !TProof::InterruptCurrentMonitor() in Terminate() to stop
infinite loops in Collect
- TXSocket:
- Implement a flag to interrupt a TXSocket while waiting for messages
- Split the session creation timeout in 4 attempts: the total timeout is the same but
it may circumvent occasional forking problems.
- XROOTD:
- fix a potential (possibly accademic) memory leak in the client
- fix a access permission problem with Kerberos ticket forwarding
- fix bug preventing 'locate' to work properly
- re-enable optimized 'locate'
Revision
18353 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Mar 19 15:14:10 2007 UTC (7 years, 10 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofMgr.h
File length: 5944 byte(s)
Diff to
previous 17035
From Gerri:
- Support for multiple ROOT versions on a PROOF cluster.
TProofMgr::ShowROOTVersions() giving the following output:
root [0] TProof::Mgr("")->ShowROOTVersions()
----------------------------------------------------------
Available versions (tag ROOT-vers remote-path PROOF-version):
* cvs 5.15/03 /home/ganis/local/root/head/head 12
dev 5.15/03 /home/ganis/local/root/head/root 12
----------------------------------------------------------
- TProofMgr::SetROOTVersion(const char *version_tag)
to change the default for the user, eg
root [1] TProof::Mgr("")->SetROOTVersion("dev")
root [2] TProof::Mgr("")->ShowROOTVersions()
----------------------------------------------------------
Available versions (tag ROOT-vers remote-path PROOF-version):
cvs 5.15/03 /home/ganis/local/root/head/head 12
* dev 5.15/03 /home/ganis/local/root/head/root 12
----------------------------------------------------------
Available version are defined by the administrator via the
"xpd.rootsys" directive:
xpd.rootsys /home/ganis/local/root/head/head cvs
xpd.rootsys /home/ganis/local/root/head/root dev
in the xrootd config file.
- Improve error reporting in TXSocket
- Fix a problem with resetting a label in TProofProgressDialog
Revision
17035 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Dec 3 23:34:04 2006 UTC (8 years, 1 month ago) by
rdm
Original Path:
trunk/proof/inc/TProofMgr.h
File length: 5842 byte(s)
Diff to
previous 16969
From Gerri:
0. Introduction of the static:
TProof::Mgr(const char *url)
to get the manager to the cluster; this gives access to all the manager
functionality, included Reset(). TProof::Reset() is removed.
1. Two new classes: TProofLog and TProofLogElem.
TProofLog contains a list of TProofLogElem, one per node. TProofLogElem
basically contains a TMacro plus methods to play with it.
2. The TProofLog object can be retrieved via the manager:
TProofLog *pl = TProof::Mgr("cluster")->GetSessionLogs()
by default the last session is retrieved: the signature for GetSessionLogs is
TProofLog *TProofMgr::GetSessionLogs(Int_t ridx, const char *stag)
the first argument is a reversed relative index (-1 gets the next to last
session), the second is the session unique tag; if the second is
specified the first is ignored.
3. TProofMgr has a new method ReadBuffer to quickly read a buffer from
a file via XrdProofd: this allows to avoid the second xrootd on the
redirector and in general is useful when a quick reading is needed.
Also, the buffer is always retrieved via the master, so we should not
have firewall problems in accessing directly the xrootd of the workers.
4. In the sandbox I have kept the .sessions file with the active sessions
and I have added .terminated in each session working dir which have
been terminated. On the master, each session working dir has the file
.workers with the relevant information about the workers.
5. The max number of old session workdirs is set to 10 by default
(configurable in xpd.cf): the sandbox is trimmed when closing a session.
6. Some unrelated changes/fixes contained in the patch:
- in XrdProofdProtocol:
- use of /proc (linux, solaris) or sysctl (mac) to verify/search
for running of a process. For other platforms I have left the
parsing of the output of 'ps'.
- I have introduced a 'cron' thread: it runs every minute and for
the moment it just cleans a vector; this is the place where to do
client independent actions, like checking if the config files have
changed, for example.
- I have added the possibility to control which user can access
the cluster; requested by Jan Fiete.
7. The patch contains also several fixes here and there I have found by
testing.
Revision
15507 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 21 16:18:27 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofMgr.h
File length: 2041 byte(s)
Diff to
previous 13616
From Gerri:
The patch implements the following new functionality:
- Shutdown of idle sessions when quitting ROOT; if not idle the session
is shutdown as soon as processing terminates; it is possible to delay
the automatic shutdown by a tunable amount of time via a new directive
in the xrootd config file 'xpd.shutdown' (see xpd.cf.example).
- New static method TProof::Reset(const char *url) to hard-cleanup an
user area (implemented for Xproofd only); this implied quite some
additions in XrdProofdProtocol.
- Possibility to temporary remove / re-insert a worker from the active list
TProof::DeactivateWorker(const char *ord)
TProof::ActivateWorker(const char *ord)
with 'ord' the ordinal numer, e.g. "0.3" .
- A default url for the disk pool
const char *TVirtualProof::GetDataPoolUrl()
It is set by default to a string send back by the master, but it can
be changed via
void TVirtualProof::SetDataPoolUrl(const char *url)
Jan should then adapt his code to use this as default.
- Posibility to timeout Collect; for now this is only used at forced
termination.
The patch includes many small changes and bug fixes. One of the most important
is in TXNetSystem (a bug preventing correct treatment of multiple request
to the same redirector).
Revision
13616 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Added
Sat Dec 10 16:51:57 2005 UTC (9 years, 1 month ago) by
rdm
Original Path:
trunk/proof/inc/TProofMgr.h
File length: 1993 byte(s)
From Gerri:
introduction of major new PROOF features. Main features:
- PROOF session manager which allows disconnect/reconnect from running
sessions
- use xrootd/xproofd as connection daemon
Much more extensive description in release notes.
This form allows you to request diffs between any two revisions of this file.
For each of the two "sides" of the diff,
enter a numeric revision.
