Log of /trunk/proof/proof/inc/TSlave.h
Parent Directory
Revision
44683 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 14 09:27:40 2012 UTC (2 years, 7 months ago) by
ganis
File length: 7229 byte(s)
Diff to
previous 37976
First version of the final patch enabling multi-workers on single node using
PROOF-Lite. This is supposed to improved scalability for large number of workers
(>300-400) and is currently available only for non-data driven tasks.
The new setup configuration is enabled by the directive 'xpd.remoteplite 1' in
the xproofd configuration file.
Revision
37976 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Feb 4 11:43:16 2011 UTC (3 years, 11 months ago) by
ganis
File length: 7192 byte(s)
Diff to
previous 27576
In TProofLite::Load
- Extend to this overload support for a comma-separated list of macros (cfr #37942)
In TSlave and related classes
- Add member for the unique session tag
- Fix worker name in TSlaveLite
Revision
27576 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 23 12:15:26 2009 UTC (5 years, 11 months ago) by
ganis
File length: 6992 byte(s)
Diff to
previous 25918
- Fix a problem with dataset validation in multi-level master setups
- Make a TSlave::GetMsd() to return a "const char *" as all other string getters
Revision
25918 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Oct 22 15:00:04 2008 UTC (6 years, 3 months ago) by
ganis
File length: 6992 byte(s)
Diff to
previous 24719
First version of PROOFLite, a 2-tier realization of PROOF intended for multi-core
machines. In PROOFLite the client start directly the workers; no daemon is required.
Currently this works on Unix only. Some issues with the internal connection still need
to be sorted out on Windows.
To start a session just use
TProof::Open("") or TProof::Open("lite")
From there on everything should be as in normal PROOF. To start a standard PROOF
session (i.e. via daemons) on the localhost use TProof::Open("localhost").
(The default meaning of "" in TProof::Open() is controlled via the rc value
'Proof.LocalDefault').
To run stressProof with PROOFLite pas the argument 'lite':
./stressProof lite
By default, PROOFLite sets the number of workers equal to the number of cores.
The number of workers is controlled by the rc variable 'ProofLite.Workers'; it can
also be set on the fly with the option "workers=<n>", e.g. TProof::Open("lite?workers=2").
This patch also introduces the rc variable 'Proof.Sandbox' to control the local location
of the sandboxes. The default has been changed from "~/proof" to "~/.proof" to avoid
interferences with possible users' working areas. The sandbox contains the directory
'packages' (with the clients packages; used by standard PROOF and PROOF Lite),
and some directories used only by PROOFLite: 'cache' (binary cache) and 'datasets' with
the datasets. The location of these directories is controlled by the rc variables
'Proof.PackageDir', 'Proof.CacheDir' and 'Proof.DataSetDir'.
The working directory for PROOFLite is in the form workdir=<sandbox>/path-to-current-dir where
~/path/to/current/dir is the directory from where the session in launched. The directory
<workdir> contains the session working dirs (in the form 'session-<host>-<time>-<pid>'),
with the symlink 'last-lite-session' pointing to the last started session, and the 'queries'
directory, with the results of queries started from the current directory.
Revision
24719 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 9 07:07:25 2008 UTC (6 years, 6 months ago) by
ganis
File length: 6886 byte(s)
Diff to
previous 22635
Patch refactorizing the XrdProofd plugin.
The class XrdProofdProtocol is now in charge only of the operations strctly related
to the XProofd protocol. All auxilliary services have been moved to dedicated service
classes controlled by XrdProofdManager. In particular:
- XrdProofdClientMgr handles the clients (represented by instances of XrdProofdClient)
including login, authentication and access control
- XrdProofdProofServMgr handles the PROOF sessions (represented by instances of
XrdProofdProofServ) including creation, attachment, detachment, destruction and
environment setting
- XrdProofdNetMgr handles connections between instances of XProofd running on different
nodes
- XrdProofdPriorityMgr handles the session priorities
A special effort has been done to get rid of all possible internal dead-lock situations.
Internal actions on clients and sessions are now all asynchronous, governed by internal
pipes.
The new plugin also offers new functionality, among which:
- a XProofd admin area, located under <xrd.admin>/.xproof.<port>, keeps information about
active and terminated sessions, and the minimal state for active clients. This is used
to reguraly check the client and session activity, to cleanup orphalin sessions and
to shutdown inactive client connections.
In particular this allows a more solid implementation of Reset, which now exists in two
flavours: 'soft', TProof::Reset(<masterurl>), (default) asks the sessions to terminate gently;
'hard', TProof::Reset(<masterurl>,kTRUE) schedules all sessions for forced termination.
- support for automatic attempts for reconnection in the case the daemon restarts.
This allows to reconfigure the plugin by xrootd restart w/o affecting the running
sessions.
- support the definition of workers via config file directive, getting de facto rid of
the proof.conf file.
- domain+level control of printout message; the format has been improved: in particular
all information messages contain now the tag 'xpd-I' and all error messages the tag
'xpd-E', so that they can easily be grepped out from the log file.
The Wiki 'XrdProofdDirectives Directives' page has been updated with the new directives.
Revision
19003 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jun 12 10:06:00 2007 UTC (7 years, 7 months ago) by
ganis
Original Path:
trunk/proof/inc/TSlave.h
File length: 6820 byte(s)
Diff to
previous 18833
TSlave.h, .cxx
- Add new data member fGroup specifying the group the user runs under.
- If available, display the group in Print()
TProof.cxx
- If available, add the group information while creating the worker session
- If available, display the group in Print()
Revision
17035 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Dec 3 23:34:04 2006 UTC (8 years, 1 month ago) by
rdm
Original Path:
trunk/proof/inc/TSlave.h
File length: 6335 byte(s)
Diff to
previous 15507
From Gerri:
0. Introduction of the static:
TProof::Mgr(const char *url)
to get the manager to the cluster; this gives access to all the manager
functionality, included Reset(). TProof::Reset() is removed.
1. Two new classes: TProofLog and TProofLogElem.
TProofLog contains a list of TProofLogElem, one per node. TProofLogElem
basically contains a TMacro plus methods to play with it.
2. The TProofLog object can be retrieved via the manager:
TProofLog *pl = TProof::Mgr("cluster")->GetSessionLogs()
by default the last session is retrieved: the signature for GetSessionLogs is
TProofLog *TProofMgr::GetSessionLogs(Int_t ridx, const char *stag)
the first argument is a reversed relative index (-1 gets the next to last
session), the second is the session unique tag; if the second is
specified the first is ignored.
3. TProofMgr has a new method ReadBuffer to quickly read a buffer from
a file via XrdProofd: this allows to avoid the second xrootd on the
redirector and in general is useful when a quick reading is needed.
Also, the buffer is always retrieved via the master, so we should not
have firewall problems in accessing directly the xrootd of the workers.
4. In the sandbox I have kept the .sessions file with the active sessions
and I have added .terminated in each session working dir which have
been terminated. On the master, each session working dir has the file
.workers with the relevant information about the workers.
5. The max number of old session workdirs is set to 10 by default
(configurable in xpd.cf): the sandbox is trimmed when closing a session.
6. Some unrelated changes/fixes contained in the patch:
- in XrdProofdProtocol:
- use of /proc (linux, solaris) or sysctl (mac) to verify/search
for running of a process. For other platforms I have left the
parsing of the output of 'ps'.
- I have introduced a 'cron' thread: it runs every minute and for
the moment it just cleans a vector; this is the place where to do
client independent actions, like checking if the config files have
changed, for example.
- I have added the possibility to control which user can access
the cluster; requested by Jan Fiete.
7. The patch contains also several fixes here and there I have found by
testing.
Revision
15507 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 21 16:18:27 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proof/inc/TSlave.h
File length: 6319 byte(s)
Diff to
previous 15338
From Gerri:
The patch implements the following new functionality:
- Shutdown of idle sessions when quitting ROOT; if not idle the session
is shutdown as soon as processing terminates; it is possible to delay
the automatic shutdown by a tunable amount of time via a new directive
in the xrootd config file 'xpd.shutdown' (see xpd.cf.example).
- New static method TProof::Reset(const char *url) to hard-cleanup an
user area (implemented for Xproofd only); this implied quite some
additions in XrdProofdProtocol.
- Possibility to temporary remove / re-insert a worker from the active list
TProof::DeactivateWorker(const char *ord)
TProof::ActivateWorker(const char *ord)
with 'ord' the ordinal numer, e.g. "0.3" .
- A default url for the disk pool
const char *TVirtualProof::GetDataPoolUrl()
It is set by default to a string send back by the master, but it can
be changed via
void TVirtualProof::SetDataPoolUrl(const char *url)
Jan should then adapt his code to use this as default.
- Posibility to timeout Collect; for now this is only used at forced
termination.
The patch includes many small changes and bug fixes. One of the most important
is in TXNetSystem (a bug preventing correct treatment of multiple request
to the same redirector).
Revision
15338 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 5 22:51:14 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proof/inc/TSlave.h
File length: 6150 byte(s)
Diff to
previous 15312
From Gerri:
base/inc/TVirtualProof.h, proof/inc/TProof.h, proof/src/TProof.cxx
proof/src/TProofProgressDialog.cxx:
- fix a problem with the progress dialog when detaching a session:
a new signal CloseProgressDialog() is introduced, as well as the
possibility to empty the list of submitted queries at once via
TProof::Remove().
proofx/inc/TXHandler.h, .cxx:
- add generic argument to HandleInput() and HandleError().
proofx/inc/TXProofMgr.h, .cxx:
- move to new signature for HandleInput(), HandleError().
proof/inc/TSlave.h:
- add method to flush underlying socket.
proofx/inc/TXSlave.h, .cxx:
- move to new signature for HandleInput(), HandleError().
- implement new method to flush underlying socket.
proofx/inc/TXSocket.h, .cxx:
- Add method to flush socket.
- Handle interrupt messages in the reader thread.
- Remove interrupt semaphore (not needed any more).
- Remove timeout argument from GetInterrupt (not needed any more).
proofx/inc/TXProofServ.h, .cxx:
- move to new signature for HandleInput(), HandleError().
- implement interrupt handling in HandleInput().
- implement dedicated version of LockSession to deal with new format
of session tag (this is a bug fix).
proofd/inc/XrdProofConn.h, .cxx:
- Add mutex protection in the SendRecv method (it can be called from both
the reader and main threads).
proofd/src/XrdProofdProtocol.cxx:
- use an asynchronous message (instead of a signal) for interrupts.
proofx/inc/LinkDef.h, TXSocketHandler.h, .cxx, TXUnixSocket.cxx:
- add TXSocketHandler in the dictionary.
- cosmetics.
Revision
15312 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 2 15:14:35 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proof/inc/TSlave.h
File length: 6115 byte(s)
Diff to
previous 14752
From Gerri:
large PROOF patch. Main issues addressed:
- Output redirection to log file at 'proofserv' startup (instead of
after Setup())
- Log messages activated at session creation
- Session sandbox restructuring in such a way that the unique session
tag, created by the top master, uniquely identifies the session on
all nodes, for example on top master:
<User_Sandbox>/session-<top_master_session_tag>/
master-0-<top_master_session_tag>/master-0-<top_master_session_tag>.log
On worker 0.0
<User_Sandbox>/session-<top_master_session_tag>/
worker-0.0-<worker_0.0_session_tag>/worker-0.0-<worker_0.0_session_tag>.log
- Automatic creation of the dataset dir
- Improved handling of error conditions
- Reorganization of TProofServ and XrdProofdProtocol
- Several bug fixes, in particular in XrdProofdProtocol
Revision
14752 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 19 10:57:44 2006 UTC (8 years, 9 months ago) by
rdm
Original Path:
trunk/proof/inc/TSlave.h
File length: 6055 byte(s)
Diff to
previous 13616
From Gerri:
This is the big patch.
Comments:
- Client side:
- Remove use of plugin manager to avoid unwanted locks
- Improved treatment of error conditions
- Server side:
- Several new functionalities described in the release notes
Revision
13616 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Dec 10 16:51:57 2005 UTC (9 years, 1 month ago) by
rdm
Original Path:
trunk/proof/inc/TSlave.h
File length: 5612 byte(s)
Diff to
previous 12727
From Gerri:
introduction of major new PROOF features. Main features:
- PROOF session manager which allows disconnect/reconnect from running
sessions
- use xrootd/xproofd as connection daemon
Much more extensive description in release notes.
Revision
12727 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Sep 17 13:52:55 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proof/inc/TSlave.h
File length: 5507 byte(s)
Diff to
previous 12477
From Gerri:
New startup for slaves splitting TSlave::Init() in two parts:
TSlave::Init()
opens the connection and waits the "Okay" from the proofd
TSlave::SetupServ()
finalizes the setup of the server (TProofServ::Setup)
Coding convention fixes.
Revision
12477 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Aug 15 15:57:18 2005 UTC (9 years, 5 months ago) by
rdm
Original Path:
trunk/proof/inc/TSlave.h
File length: 5539 byte(s)
Diff to
previous 12337
From Gerri + some mods by me:
- This patch gets rid of TFTP, slightly modifying SendFile and CheckFile.
- The factorization of the technology dependent parts of TProof, TSlave
and TProofServ. In particular, I have moved the technicalities of Ping
and Interrupt from TProof to two new virtual methods in TSlave, which
will be overwritten in TXSlave.
In TSlave I have introduced the virtual method Init which will be
overwritten by TXSlave and is able to act on an open connection for
optimal fallback a la TXNetFile.
Slaves are created by TProof via the static method TSlave::Create
which uses the plugin manager to detect the presence of TXSlave.
In TProofServ I have made the technology-dependent methods
virtual (Setup, HandleSocketInput, HandleUrgentData, HandleSigPipe,
HandleSocketInputDuringProcess).
Revision
12337 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 18 16:20:53 2005 UTC (9 years, 6 months ago) by
rdm
Original Path:
trunk/proof/inc/TSlave.h
File length: 5087 byte(s)
Diff to
previous 12123
From Gerri:
Big patch restructuring the authentication code so libCore does not depend
on libssl etc. anymore. Isolate the current authentication code behind a
plugin library to be loaded on demand.
Setup of a framework able to manage both sets of authentication modules
(the current one and the one based on xrdsec, coming soon).
Cleanup all direct reference to authentication in TSlave and TProofServ,
allowing for significant simplification for the forthcoming changes
in PROOF for XPD.
New module: auth
New files: base/inc/TVirtualAuth.h
auth/Module.Mk
auth/inc/LinkDefRoot.h
auth/inc/TRootAuth.h auth/inc/TRootSecContext.h
auth/src/TRootAuth.cxx auth/src/TRootSecContext.cxx
Moved files:
net/inc -> auth/inc : TAuthenticate.h THostAuth.h DaemonUtils.h AuthConst.h
net/src -> auth/src : TAuthenticate.cxx THostAuth.cxx DaemonUtils.cxx
Revision
12123 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 22 20:18:12 2005 UTC (9 years, 7 months ago) by
brun
Original Path:
trunk/proof/inc/TSlave.h
File length: 5005 byte(s)
Diff to
previous 11068
From Constantin Loizides
This patch implements:
- decentralized, automatic mutex initialization,
see R__LOCKGUARD2 in TVirtualMutex
- PROOF parallel startup fixes
- more thread protection (in base, cont, meta, rest to be done)
- cleanups
From Eddy Offermann:
TString::Atoi and Atof are made const.
Revision
11047 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 7 18:02:37 2005 UTC (9 years, 11 months ago) by
rdm
Original Path:
trunk/proof/inc/TSlave.h
File length: 5016 byte(s)
Diff to
previous 10321
From Maarten and Kristjan Gulbrandsen:
Here is the patch with the restructured PROOF code. It parameterizes
PROOF startup, currently implementing "Standard", Condor, MLM and PEAC.
It can easily be extended with other "TProof" classes, e.g. AliEn Proof.
Or what ever next Grid implementation or scheduler comes along.
The MLM, Multi Level Master, code uses a basic packetizer, based on a
MSD (massstorage domain) tag. In MLM mode we have multi-tier client
server setup, like:
client -> super master -> sub master 0 -> slave[0..N]
...
-> sub master M -> slave[0..N]
It also contains a large number of smaller fixes. Credit for most of
the work goes to Kris!
We have tested this on multipe platforms (RH7.3, RH9, SL3) and in
multiple configurations. Should be stable.
Revision
8239 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Feb 19 00:11:19 2004 UTC (10 years, 11 months ago) by
rdm
Original Path:
trunk/proof/inc/TSlave.h
File length: 4201 byte(s)
Diff to
previous 7175
Big authentication patch by Gerri:
General purpose:
Add support for 'authenticated' sockets on client side
and prepare for support of server authenticated sockets.
Main changes:
o TSocket, TPsocket:
Added support of 'authenticated' sockets: authentication
is delegated to TSocket by calling a static method
CreateAuthSocket() returning a TSocket*.
TSocket is now the only class calling TAuthenticate.
o TFTP, TNetFile, TSlave:
modified to cope with new socket authentication scheme.
o TAuthDetails replaced by TSecContext containing:
- local authentication info
- expiration time
- pointer to a security context for the method (if any)
- information for cleanup (via TSecContextCleanup)
o A list of valid TSecContext has been added to TROOT,
accessible via gROOT->GetListOfSecContexts().
o THostAuth rewritten using arrays to avoid multiple use
of 'new'; functionality extended by the addition of
several new methods.
o TAuthenticate:
- Loading of information from .rootauthrc has been optimized;
this allowed to simplify method signatures and drop several
methods. Also collection and transmission of authentication
information from TSlave to TProofServ gets simplified by the
new scheme.
- Added support for server dependent directives in .rootauthrc
- Added check for changes in .rootauthrc each time a new
TAuthenticate is instantiated, so that changes in the file
are picked-up interactively.
- Checking for an existing TSecContext (previously TAuthDetails)
has been improved
- Improved diagnostics for SshAuth
- Enlarged support for authentication method indication in protocol:
UsrPwd: rootup/proofup
SRP: roots/proofs (already supported)
Krb5: rootk/proofk (already supported)
Globus: rootg/proofg
SSH: rootsh/proofsh
UigGid: rootug/proofug
o GlobusAuth.cxx, Krb5Auth.cxx, SRPAuth.cxx
- Adapted to new class TSecContext
- Added support for improved search for reusable TSecContext
- Drop globals used to store established security contexts
(saved in TSecContext) and related methods
o Automatic creation of <RootDir>/etc/system.rootauthrc from
compilation flags; this has priority on old envs
'Rootd.Authentication' and 'Proofd.Authentication' and is
superseded by $HOME/.rootauthrc
o Automatic creation of <RootDir>/etc/system.authdaemonrc from
compilation flags; this is superseded by $HOME/.rootauthrc
o rpdutils:
- several almost identical methods moved in from rootd/proofd
- several new methods to simplify manipulation of the
authentication tab file
- moved (and improved) check for running of sshd in RpdSshAuth,
performed only in case of failure (to avoid spurious messages
printed by sshd in /var/log/messages)
o proofd.cxx, rootd.cxx
- increased protocol version
- all initialization stuff (included authentication and
login) done via RpdInitSession
- postpone opening of parallels sockets after authentication
(to solve problem of Kerberos/Globus authentication with
parallel sockets)(rootd)
- Fixed problem with kerberos and globus authentication in rootd
with parallel sockets.
Revision
1205 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 13 15:13:57 2000 UTC (14 years, 1 month ago) by
brun
Original Path:
trunk/proof/inc/TSlave.h
File length: 4000 byte(s)
Diff to
previous 1203
W A R N I N G !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================================================
A very long list of changes in this pre-release of version 3.00.
We have modified the signature of many functions (in particular TObject)
to introduce more constness in the system.
You must change your code if your class derives from TObject and uses
one of the modified functions such as ls, Print, Compare, Hash, etc.
The modified functions in TObject have the following signature:
virtual TObject *Clone() const;
virtual Int_t Compare(const TObject *obj) const;
virtual void Delete(Option_t *option=""); // *MENU*
virtual void DrawClass() const; // *MENU*
virtual void DrawClone(Option_t *option="") const; // *MENU*
virtual void Dump() const; // *MENU*
virtual TObject *FindObject(const TObject *obj) const;
virtual char *GetObjectInfo(Int_t px, Int_t py) const;
virtual ULong_t Hash() const;
virtual void Inspect() const; // *MENU*
virtual Bool_t IsEqual(const TObject *obj) const;
virtual void ls(Option_t *option="") const;
virtual void Print(Option_t *option="") const;
A similar operation has been done with classes such as TH1, TVirtualPad,
TTree, etc.
This form allows you to request diffs between any two revisions of this file.
For each of the two "sides" of the diff,
enter a numeric revision.
