Log of /trunk/proof/proofx/inc/TXSlave.h
Parent Directory
Revision
44683 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 14 09:27:40 2012 UTC (2 years, 7 months ago) by
ganis
File length: 2800 byte(s)
Diff to
previous 38580
First version of the final patch enabling multi-workers on single node using
PROOF-Lite. This is supposed to improved scalability for large number of workers
(>300-400) and is currently available only for non-data driven tasks.
The new setup configuration is enabled by the directive 'xpd.remoteplite 1' in
the xproofd configuration file.
Revision
38580 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Mar 23 15:24:54 2011 UTC (3 years, 10 months ago) by
ganis
File length: 2709 byte(s)
Diff to
previous 25734
Patch fixing a problem with log path transmission when the node dies early or not even starts.
The log path was empty and wrong was filled in when retrieving the log buffers, disorienting
debugging. This was experienced frequently by ALICE.
The log file is now trasmitted as part of the responde to the session 'Create' request, so it
is always registered by the master in the relevant field in TSlave. Also, the last errors, if
any, typically logged in the xproofd log file, are now copied to the relevant worker or master
log file so that they are also accessible to client.
The patch also factorizes some calls in XrdProofdProofServMgr in preparation to the improved
session lauch strategy.
Revision
24719 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 9 07:07:25 2008 UTC (6 years, 6 months ago) by
ganis
File length: 2706 byte(s)
Diff to
previous 22635
Patch refactorizing the XrdProofd plugin.
The class XrdProofdProtocol is now in charge only of the operations strctly related
to the XProofd protocol. All auxilliary services have been moved to dedicated service
classes controlled by XrdProofdManager. In particular:
- XrdProofdClientMgr handles the clients (represented by instances of XrdProofdClient)
including login, authentication and access control
- XrdProofdProofServMgr handles the PROOF sessions (represented by instances of
XrdProofdProofServ) including creation, attachment, detachment, destruction and
environment setting
- XrdProofdNetMgr handles connections between instances of XProofd running on different
nodes
- XrdProofdPriorityMgr handles the session priorities
A special effort has been done to get rid of all possible internal dead-lock situations.
Internal actions on clients and sessions are now all asynchronous, governed by internal
pipes.
The new plugin also offers new functionality, among which:
- a XProofd admin area, located under <xrd.admin>/.xproof.<port>, keeps information about
active and terminated sessions, and the minimal state for active clients. This is used
to reguraly check the client and session activity, to cleanup orphalin sessions and
to shutdown inactive client connections.
In particular this allows a more solid implementation of Reset, which now exists in two
flavours: 'soft', TProof::Reset(<masterurl>), (default) asks the sessions to terminate gently;
'hard', TProof::Reset(<masterurl>,kTRUE) schedules all sessions for forced termination.
- support for automatic attempts for reconnection in the case the daemon restarts.
This allows to reconfigure the plugin by xrootd restart w/o affecting the running
sessions.
- support the definition of workers via config file directive, getting de facto rid of
the proof.conf file.
- domain+level control of printout message; the format has been improved: in particular
all information messages contain now the tag 'xpd-I' and all error messages the tag
'xpd-E', so that they can easily be grepped out from the log file.
The Wiki 'XrdProofdDirectives Directives' page has been updated with the new directives.
Revision
17035 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Dec 3 23:34:04 2006 UTC (8 years, 1 month ago) by
rdm
Original Path:
trunk/proofx/inc/TXSlave.h
File length: 2678 byte(s)
Diff to
previous 15507
From Gerri:
0. Introduction of the static:
TProof::Mgr(const char *url)
to get the manager to the cluster; this gives access to all the manager
functionality, included Reset(). TProof::Reset() is removed.
1. Two new classes: TProofLog and TProofLogElem.
TProofLog contains a list of TProofLogElem, one per node. TProofLogElem
basically contains a TMacro plus methods to play with it.
2. The TProofLog object can be retrieved via the manager:
TProofLog *pl = TProof::Mgr("cluster")->GetSessionLogs()
by default the last session is retrieved: the signature for GetSessionLogs is
TProofLog *TProofMgr::GetSessionLogs(Int_t ridx, const char *stag)
the first argument is a reversed relative index (-1 gets the next to last
session), the second is the session unique tag; if the second is
specified the first is ignored.
3. TProofMgr has a new method ReadBuffer to quickly read a buffer from
a file via XrdProofd: this allows to avoid the second xrootd on the
redirector and in general is useful when a quick reading is needed.
Also, the buffer is always retrieved via the master, so we should not
have firewall problems in accessing directly the xrootd of the workers.
4. In the sandbox I have kept the .sessions file with the active sessions
and I have added .terminated in each session working dir which have
been terminated. On the master, each session working dir has the file
.workers with the relevant information about the workers.
5. The max number of old session workdirs is set to 10 by default
(configurable in xpd.cf): the sandbox is trimmed when closing a session.
6. Some unrelated changes/fixes contained in the patch:
- in XrdProofdProtocol:
- use of /proc (linux, solaris) or sysctl (mac) to verify/search
for running of a process. For other platforms I have left the
parsing of the output of 'ps'.
- I have introduced a 'cron' thread: it runs every minute and for
the moment it just cleans a vector; this is the place where to do
client independent actions, like checking if the config files have
changed, for example.
- I have added the possibility to control which user can access
the cluster; requested by Jan Fiete.
7. The patch contains also several fixes here and there I have found by
testing.
Revision
15507 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 21 16:18:27 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proofx/inc/TXSlave.h
File length: 2662 byte(s)
Diff to
previous 15338
From Gerri:
The patch implements the following new functionality:
- Shutdown of idle sessions when quitting ROOT; if not idle the session
is shutdown as soon as processing terminates; it is possible to delay
the automatic shutdown by a tunable amount of time via a new directive
in the xrootd config file 'xpd.shutdown' (see xpd.cf.example).
- New static method TProof::Reset(const char *url) to hard-cleanup an
user area (implemented for Xproofd only); this implied quite some
additions in XrdProofdProtocol.
- Possibility to temporary remove / re-insert a worker from the active list
TProof::DeactivateWorker(const char *ord)
TProof::ActivateWorker(const char *ord)
with 'ord' the ordinal numer, e.g. "0.3" .
- A default url for the disk pool
const char *TVirtualProof::GetDataPoolUrl()
It is set by default to a string send back by the master, but it can
be changed via
void TVirtualProof::SetDataPoolUrl(const char *url)
Jan should then adapt his code to use this as default.
- Posibility to timeout Collect; for now this is only used at forced
termination.
The patch includes many small changes and bug fixes. One of the most important
is in TXNetSystem (a bug preventing correct treatment of multiple request
to the same redirector).
Revision
15338 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 5 22:51:14 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proofx/inc/TXSlave.h
File length: 2608 byte(s)
Diff to
previous 15312
From Gerri:
base/inc/TVirtualProof.h, proof/inc/TProof.h, proof/src/TProof.cxx
proof/src/TProofProgressDialog.cxx:
- fix a problem with the progress dialog when detaching a session:
a new signal CloseProgressDialog() is introduced, as well as the
possibility to empty the list of submitted queries at once via
TProof::Remove().
proofx/inc/TXHandler.h, .cxx:
- add generic argument to HandleInput() and HandleError().
proofx/inc/TXProofMgr.h, .cxx:
- move to new signature for HandleInput(), HandleError().
proof/inc/TSlave.h:
- add method to flush underlying socket.
proofx/inc/TXSlave.h, .cxx:
- move to new signature for HandleInput(), HandleError().
- implement new method to flush underlying socket.
proofx/inc/TXSocket.h, .cxx:
- Add method to flush socket.
- Handle interrupt messages in the reader thread.
- Remove interrupt semaphore (not needed any more).
- Remove timeout argument from GetInterrupt (not needed any more).
proofx/inc/TXProofServ.h, .cxx:
- move to new signature for HandleInput(), HandleError().
- implement interrupt handling in HandleInput().
- implement dedicated version of LockSession to deal with new format
of session tag (this is a bug fix).
proofd/inc/XrdProofConn.h, .cxx:
- Add mutex protection in the SendRecv method (it can be called from both
the reader and main threads).
proofd/src/XrdProofdProtocol.cxx:
- use an asynchronous message (instead of a signal) for interrupts.
proofx/inc/LinkDef.h, TXSocketHandler.h, .cxx, TXUnixSocket.cxx:
- add TXSocketHandler in the dictionary.
- cosmetics.
Revision
15312 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 2 15:14:35 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proofx/inc/TXSlave.h
File length: 2545 byte(s)
Diff to
previous 14752
From Gerri:
large PROOF patch. Main issues addressed:
- Output redirection to log file at 'proofserv' startup (instead of
after Setup())
- Log messages activated at session creation
- Session sandbox restructuring in such a way that the unique session
tag, created by the top master, uniquely identifies the session on
all nodes, for example on top master:
<User_Sandbox>/session-<top_master_session_tag>/
master-0-<top_master_session_tag>/master-0-<top_master_session_tag>.log
On worker 0.0
<User_Sandbox>/session-<top_master_session_tag>/
worker-0.0-<worker_0.0_session_tag>/worker-0.0-<worker_0.0_session_tag>.log
- Automatic creation of the dataset dir
- Improved handling of error conditions
- Reorganization of TProofServ and XrdProofdProtocol
- Several bug fixes, in particular in XrdProofdProtocol
Revision
14752 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 19 10:57:44 2006 UTC (8 years, 9 months ago) by
rdm
Original Path:
trunk/proofx/inc/TXSlave.h
File length: 2379 byte(s)
Diff to
previous 14110
From Gerri:
This is the big patch.
Comments:
- Client side:
- Remove use of plugin manager to avoid unwanted locks
- Improved treatment of error conditions
- Server side:
- Several new functionalities described in the release notes
Revision
13648 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Added
Mon Dec 12 12:54:27 2005 UTC (9 years, 1 month ago) by
rdm
Original Path:
trunk/proofx/inc/TXSlave.h
File length: 1988 byte(s)
From Gerri:
- last part of the major PROOF upgrade to support disconnect/reconnect
features using the libXrdProofd.so xrootd plugin.
More on how to configure and run with xrootd/xproofd coming very soon.
This form allows you to request diffs between any two revisions of this file.
For each of the two "sides" of the diff,
enter a numeric revision.
