[root] Log of /trunk/proof/proofx/inc/TXSocket.h

Revision 48992 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Mar 28 15:26:26 2013 UTC (21 months, 3 weeks ago) by rdm
File length: 11807 byte(s)
Diff to previous 44103

From Lifeng Sun:
The attached patchset fixes a bunch of typo in the source:
0001-succes-success.patch
0002-preceed-preced.patch
0003-informations-information.patch
0004-childs-children.patch
0005-avaliable-available.patch
0006-writeable-writable.patch
0007-comand-command.patch
0008-unkown-unknown.patch
0009-wierd-weird.patch
0010-wheter-whether.patch
0011-unecessary-unnecessary.patch
0012-splitted-split.patch
0013-registerd-registered.patch
0014-recieve-receive.patch
0015-processsing-processing.patch
0016-ouput-output.patch
0017-mutiple-multiple.patch
0018-lenght-length.patch
0019-interupted-interrupted.patch
0020-independant-independent.patch
0021-inconsistant-inconsistent.patch
0022-expresion-expression.patch
0023-explicitely-explicitly.patch
0024-enviroment-environment.patch
0025-deafult-default.patch
0026-continous-continuous.patch
0027-completly-completely.patch
0028-commited-committed.patch
0029-choosen-chosen.patch
0030-backgroud-background.patch
0031-auxilliary-auxiliary.patch
0032-authentification-authentication.patch
0033-appropiate-appropriate.patch
0034-an-other-another.patch
0035-environement-environment.patch
0036-targetting-targeting.patch
0037-suppported-supported.patch
0038-paramater-parameter.patch

Revision 38538 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Mar 21 14:13:29 2011 UTC (3 years, 10 months ago) by ganis
File length: 11793 byte(s)
Diff to previous 34428

  In TXSocket and TXSlave:
  - Fix problem with the way collection over a socket just marked as 'bad' was interrupted;
    the interrupt was de facto ineffective, so that collection stayed always until the
    timeout expired (default: 5 minutes). Should solve some of the cases were slow response
    was experienced.

Revision 34101 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Jun 24 12:12:59 2010 UTC (4 years, 7 months ago) by ganis
File length: 10907 byte(s)
Diff to previous 33868

  Following a request from ALICE, implement a timer to terminate idle sessions. The timeout
  value is controlled by the variable ProofServ.IdleTimeout (value in seconds). For example,
  to set a timeout of 20 mins, enter the following in the xproofd config file:

       ### Setup an idle timeout
       xpd.putrc ProofServ.IdleTimeout 1200

  The timer is disabled by default.

Revision 27034 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Dec 19 15:30:06 2008 UTC (6 years, 1 month ago) by ganis
File length: 11282 byte(s)
Diff to previous 26993

   From Jan: prepare for query-enqueing support:
   - add new protocol message to resume an enqueued session
   - add framework to process enqueued queries after a resume: the processing part on masters
     has been moved to a new method ProcessNext which is called also after a resume request.
   - remove duplication in the definition of message types for SendCoordinator
     (same codes defined in XProofProtocol.h and TXSocket.h)

Revision 26951 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Dec 16 12:04:38 2008 UTC (6 years, 1 month ago) by ganis
File length: 11919 byte(s)
Diff to previous 26678

   - Fix an issue with uploading packages in rapid sequence
   - Fix an issue with TProof::Load() in forced compilation mode ('++')
   - Fix an issue with grep in log retrieval
   - Fix an issue with ownership of the xrootd log file
   - Make sure the XrdProofdProofServMgr::fCurrentSessions is always up-to-date

Revision 26678 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Dec 5 14:53:23 2008 UTC (6 years, 1 month ago) by ganis
File length: 11350 byte(s)
Diff to previous 24734


   Patch to:
   - Add the possibility to set upper limits on the virtual memory (changes in TProofPlayer,
     TProofServ, TXProofServ and pmain (separate patch)).
     If enebled, the session gets firts a warning when it reaches 80% of the limit, and then
     processing is stopped whenit exceeds 95% of the limit, sending back the results.
     Also, the memory footprint is notified when the session is terminated.
   - Make sure the the active valuse in XrdProofWorker are always correct; this was not the
     case for dynamic startup as the notification at the end of the query was not done.
     This information is crucial or the scheduler.
     The way the information is stored in XrdProofdProofServ had to be modified
     and a new internal message type (kReleaseWorker) added.
     (Changes in several proofd classes, TProof, TXsocket and TXProofServ)
   - Fix problem with setting a static upper limit of the sessions that can be started,
     and enable this functionality for the dynamic mode (changes in XrdProofSched)
   - Remove a deleted worker from all the lists in TProof::MarkBad to avoid later attempts of use.
   - Better control the use of the internal pipe for socket readiness notification in TXSocket.

Revision 24719 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Jul 9 07:07:25 2008 UTC (6 years, 6 months ago) by ganis
File length: 10755 byte(s)
Diff to previous 24568

Patch refactorizing the XrdProofd plugin.
The class XrdProofdProtocol is now in charge only of the operations strctly related
to the XProofd protocol. All auxilliary services have been moved to dedicated service
classes controlled by XrdProofdManager. In particular:
- XrdProofdClientMgr handles the clients (represented by instances of XrdProofdClient)
including login, authentication and access control
- XrdProofdProofServMgr handles the PROOF sessions (represented by instances of
XrdProofdProofServ) including creation, attachment, detachment, destruction and
environment setting
- XrdProofdNetMgr handles connections between instances of XProofd running on different
nodes
- XrdProofdPriorityMgr handles the session priorities

A special effort has been done to get rid of all possible internal dead-lock situations.
Internal actions on clients and sessions are now all asynchronous, governed by internal
pipes.

The new plugin also offers new functionality, among which:
- a XProofd admin area, located under <xrd.admin>/.xproof.<port>, keeps information about
active and terminated sessions, and the minimal state for active clients. This is used
to reguraly check the client and session activity, to cleanup orphalin sessions and
to shutdown inactive client connections.
In particular this allows a more solid implementation of Reset, which now exists in two
flavours: 'soft', TProof::Reset(<masterurl>), (default) asks the sessions to terminate gently;
'hard', TProof::Reset(<masterurl>,kTRUE) schedules all sessions for forced termination.
- support for automatic attempts for reconnection in the case the daemon restarts.
This allows to reconfigure the plugin by xrootd restart w/o affecting the running
sessions.
- support the definition of workers via config file directive, getting de facto rid of
the proof.conf file.
- domain+level control of printout message; the format has been improved: in particular
all information messages contain now the tag 'xpd-I' and all error messages the tag
'xpd-E', so that they can easily be grepped out from the log file.

The Wiki 'XrdProofdDirectives Directives' page has been updated with the new directives.

Revision 24568 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Jun 26 11:57:42 2008 UTC (6 years, 6 months ago) by ganis
File length: 10351 byte(s)
Diff to previous 23068

   - Fix a race condition possibly affecting the handling of workers death
   - Improve diagnostic from MarkBad: clients will now receive something like this

root [1] Worker 'localhost-0.1' has been removed from the active list

 +++ Message from top master at aleph025.cern.ch:1093 : marking localhost:1093 (0.1) as bad
 +++ Reason: problems receiving a message in TProof::CollectInputFrom(...)

 +++ Most likely your code crashed on worker 0.1 at localhost:1093.
 +++ Please check the session logs for error messages either using
 +++ the 'Show logs' button or executing
 +++
 +++ root [] TProof::Mgr("aleph025.cern.ch:1093")->GetSessionLogs()->Display("0.1",0)

Revision 20862 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sun Nov 18 14:51:42 2007 UTC (7 years, 2 months ago) by ganis
Original Path: trunk/proofx/inc/TXSocket.h
File length: 10619 byte(s)
Diff to previous 20682

   Synchronize with braches/dev/proof r20835

   Summary:

   TProofServ:
     - read session.rootrc with level kEnvChange to be able to change existing
       settings
     - Avoid deleting a query result twice in some special cases
     - Fix a problem with the initialization of fKeptQueries
   rootrc.in:
     - restore default settings the asynchronous reading
   TFileCacheRead:
     - add missing protection in ReadBuffer
   TEventIter:
     - Enable the usage of TTreeCache
   XrdProofdProtocol:
     - Additional check on the ownership of the unix socket
     - Improve notification during Reset
     - Reduce default timeout on admin requests and make it configurable
   XrdProofConn:
     - Use the configurable maxtry everywhere where relevant
   TPacketizerAdaptive:
     - Store info on all the processed packets in per-worker lists

   TProofMgr, TXProofMgr, TXSocket, XProofProtocol, XrdProofdProtocol:
     - Add possibility for the admin to broadcast a message to the connected users

   getProof:
     - Add missing protection

Revision 20682 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Nov 6 15:51:59 2007 UTC (7 years, 2 months ago) by ganis
Original Path: trunk/proofx/inc/TXSocket.h
File length: 10579 byte(s)
Diff to previous 19826

  Import branches/dev/proof r20654

  Summary:

  + Improvements
    - Add support for SSH SOCKS4 tunnelling; the local port for the tunnel can be specified
      in the master URL, e.g. TProof::Open("master/?tunnel:8000")
    - Add the possibility to plot the estimated instantaneous rate
    - Add "PROOF_ForceLocal" parameter to the !TPacketizerAdaptive; if set to 1, all the data
      are processed locally.
    - Add support for remote grep functionality while retrieving logs (needed by the forthcoming
      memory monitor)
  + Bug fixes
    - Several small fixes to revive the multi-master mode.
    - XrdProofdProtocol:
      - add missing lock to the client instance in SendMsg to avoid screwing up requests from
        workers on the same machine
         - lock the mutex of the requester when setting prorities
         - add notification during Reset
         - fix problem with the detection of the 'allow' directive
         - fix problem with the parsing of the return value from XrdProofServProxy::TerminateProofServ()
         - always use the effective user to retrieve info fom another server
           (XrdProofdManager::GetProofConn is now used)
         - fix possible dead-locks from debug notifications done after hard-killing a session
         - re-enable the garbage collector thread of the connection manager in XrdProofConn to
           fix a problem with closing physical connections;
         - fix a problem with !CleanupProofServ in the case of a non-privileged daemon running
           in multi-user mode
         - introduce a timeout when waiting for the startup of a 'proofserv'.
    - XrdProofConn: init mutex in the ctor; lock in SendRecv
    - XrdProofSched: add support for using the priorities defined in the group manager to define
      the number of workers for sessions
      - TProof:
         - Broadcast priorities to unique nodes only
         - timeout after 5 mins the initial Collect to avoid clients getting stuck at this stage
         - add support for generic timeout in Collect (disabled by default)
         - fix a problem with SendFile.
      - TXProofServ: add a call to !TProof::InterruptCurrentMonitor() in Terminate() to stop 
        infinite loops in Collect
      - TXSocket:
         - Implement a flag to interrupt a TXSocket while waiting for messages
         - Split the session creation timeout in 4 attempts: the total timeout is the same but
           it may circumvent occasional forking problems.
      - XROOTD: 
         - fix a potential (possibly accademic) memory leak in the client
         - fix a access permission problem with Kerberos ticket forwarding
         - fix bug preventing 'locate' to work properly
         - re-enable optimized 'locate'

Revision 18353 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Mar 19 15:14:10 2007 UTC (7 years, 10 months ago) by rdm
Original Path: trunk/proofx/inc/TXSocket.h
File length: 10426 byte(s)
Diff to previous 17035

From Gerri:
- Support for multiple ROOT versions on a PROOF cluster.
  TProofMgr::ShowROOTVersions() giving the following output:

root [0] TProof::Mgr("")->ShowROOTVersions()
----------------------------------------------------------

Available versions (tag ROOT-vers remote-path PROOF-version):

 * cvs 5.15/03 /home/ganis/local/root/head/head 12
   dev 5.15/03 /home/ganis/local/root/head/root 12

----------------------------------------------------------

- TProofMgr::SetROOTVersion(const char *version_tag)

    to change the default for the user, eg

root [1] TProof::Mgr("")->SetROOTVersion("dev")
root [2] TProof::Mgr("")->ShowROOTVersions()
----------------------------------------------------------

Available versions (tag ROOT-vers remote-path PROOF-version):

   cvs 5.15/03 /home/ganis/local/root/head/head 12
 * dev 5.15/03 /home/ganis/local/root/head/root 12

----------------------------------------------------------

Available version are defined by the administrator via the
"xpd.rootsys" directive:

xpd.rootsys /home/ganis/local/root/head/head cvs
xpd.rootsys /home/ganis/local/root/head/root dev

in the xrootd config file.

- Improve error reporting in TXSocket
- Fix a problem with resetting a label in TProofProgressDialog

Revision 17035 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sun Dec 3 23:34:04 2006 UTC (8 years, 1 month ago) by rdm
Original Path: trunk/proofx/inc/TXSocket.h
File length: 10344 byte(s)
Diff to previous 16839

From Gerri:
0. Introduction of the static:
      TProof::Mgr(const char *url)
   to get the manager to the cluster; this gives access to all the manager
   functionality, included Reset(). TProof::Reset() is removed.

1. Two new classes: TProofLog and TProofLogElem.
   TProofLog contains a list of TProofLogElem, one per node. TProofLogElem
   basically contains a TMacro plus methods to play  with it.

2. The TProofLog object can be retrieved via the manager:

      TProofLog *pl = TProof::Mgr("cluster")->GetSessionLogs()

   by default the last session is retrieved: the signature for GetSessionLogs is

      TProofLog *TProofMgr::GetSessionLogs(Int_t ridx, const char *stag)

   the first argument is a reversed relative index (-1 gets the next to last
   session), the second is the session unique tag; if the second is
   specified the first is ignored.

3. TProofMgr has a new method ReadBuffer to quickly read a buffer from
   a file via XrdProofd: this allows to avoid the second xrootd on the
   redirector and in general is useful when a quick reading is needed.
   Also, the buffer is always retrieved via the master, so we should not
   have firewall problems in accessing directly the xrootd of the workers.

4. In the sandbox I have kept the .sessions file with the active sessions
   and I have added .terminated in each session working dir which have
   been terminated. On the master, each session working dir has the file
   .workers with the relevant information about the workers.

5. The max number of old session workdirs is set to 10 by default
   (configurable in xpd.cf): the sandbox is trimmed when closing a session.

6. Some unrelated changes/fixes contained in the patch:
   - in XrdProofdProtocol:
      - use of /proc (linux, solaris) or sysctl (mac) to verify/search
        for running of a process. For other platforms I have left the
        parsing of the output of 'ps'.
      - I have introduced a 'cron'  thread: it runs every minute and for
        the moment it just cleans a vector; this is the place where to do
        client independent actions, like checking if the config files have
        changed, for example.
      - I have added the possibility to control which user can access
        the cluster; requested by Jan Fiete.

7. The patch contains also several fixes here and there I have found by
   testing.

Revision 16839 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Nov 20 15:56:36 2006 UTC (8 years, 2 months ago) by rdm
Original Path: trunk/proofx/inc/TXSocket.h
File length: 10193 byte(s)
Diff to previous 16775

From Maarten + Gerri:
Patch to allow run time configuration of proofserv environment. The client
can pass a set of shell variables to PROOF. These variables can be used
to startup different versions of the proofserv, or they can instruct the
system to run proofserv under valgrind or tcheck, etc. Main changes:
- default wrapper script for proofserv
- modifies the build system for the wrapper script
- adds a list of environment variables to TVirtualProof, which are
  passed to the wrapper by proofd/xrootd.
- adds code to TProofServ to pickup these variables and pass them
  along to the slaves/submasters

Revision 16775 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Nov 15 17:45:55 2006 UTC (8 years, 2 months ago) by rdm
Original Path: trunk/proofx/inc/TXSocket.h
File length: 10022 byte(s)
Diff to previous 15860

From Gerri and Bertrand:
Fixes and new features developed in preparation for the Intel quad-core
launch demo:
- Add instantaneous event and MB processing rates in the Progress signal
  and make necessary modification to exploit it in the Progress Dialog.
- Add possibility to change the Progress and Feedback periods
- A number of small improvements and bug fixes

Revision 15829 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Jul 20 01:24:48 2006 UTC (8 years, 6 months ago) by rdm
Original Path: trunk/proofx/inc/TXSocket.h
File length: 9879 byte(s)
Diff to previous 15507

From Gerri:
- Fix a couple of bugs affecting allocation and release of auxilliary buffers
- Make the list of spare buffers and its mutex static
- Implement an upper bound on the average amount of memory allocated
  in spare buffers: default 10 MB, can be changed via the env XProof.MaxBuffMem

Revision 15507 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Jun 21 16:18:27 2006 UTC (8 years, 7 months ago) by rdm
Original Path: trunk/proofx/inc/TXSocket.h
File length: 9701 byte(s)
Diff to previous 15338

From Gerri:
The patch implements the following new functionality:

  - Shutdown of idle sessions when quitting ROOT; if not idle the session
    is shutdown as soon as processing terminates; it is possible to delay
    the automatic shutdown by a tunable amount of time via a new directive
    in the xrootd config file 'xpd.shutdown' (see xpd.cf.example).

  - New static method TProof::Reset(const char *url) to hard-cleanup an
    user area (implemented for Xproofd only); this implied quite some
    additions in XrdProofdProtocol.

  - Possibility to temporary remove / re-insert a worker from the active list
       TProof::DeactivateWorker(const char *ord)
       TProof::ActivateWorker(const char *ord)
    with 'ord' the ordinal numer, e.g. "0.3" .

  - A default url for the disk pool
       const char *TVirtualProof::GetDataPoolUrl()
    It is set by default to a string send back by the master, but it can
    be changed via
       void  TVirtualProof::SetDataPoolUrl(const char *url)

    Jan should then adapt his code to use this as default.

  - Posibility to timeout Collect; for now this is only used at forced
    termination.

The patch includes many small changes and bug fixes. One of the most important
is in TXNetSystem (a bug preventing correct treatment of multiple request
to the same redirector).

Revision 15338 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Jun 5 22:51:14 2006 UTC (8 years, 7 months ago) by rdm
Original Path: trunk/proofx/inc/TXSocket.h
File length: 9293 byte(s)
Diff to previous 15312

From Gerri:
base/inc/TVirtualProof.h, proof/inc/TProof.h, proof/src/TProof.cxx
proof/src/TProofProgressDialog.cxx:
 - fix a problem with the progress dialog when detaching a session:
   a new signal CloseProgressDialog() is introduced, as well as the
   possibility to empty the list of submitted queries at once via
   TProof::Remove().

proofx/inc/TXHandler.h, .cxx:
 - add generic argument to HandleInput() and HandleError().

proofx/inc/TXProofMgr.h, .cxx:
 - move to new signature for HandleInput(), HandleError().

proof/inc/TSlave.h:
 - add method to flush underlying socket.

proofx/inc/TXSlave.h, .cxx:
 - move to new signature for HandleInput(), HandleError().
 - implement new method to flush underlying socket.

proofx/inc/TXSocket.h, .cxx:
 - Add method to flush socket.
 - Handle interrupt messages in the reader thread.
 - Remove interrupt semaphore (not needed any more).
 - Remove timeout argument from GetInterrupt (not needed any more).

proofx/inc/TXProofServ.h, .cxx:
 - move to new signature for HandleInput(), HandleError().
 - implement interrupt handling in HandleInput().
 - implement dedicated version of LockSession to deal with new format
   of session tag (this is a bug fix).

proofd/inc/XrdProofConn.h, .cxx:
 - Add mutex protection in the SendRecv method (it can be called from both
   the reader and main threads).

proofd/src/XrdProofdProtocol.cxx:
 - use an asynchronous message (instead of a signal) for interrupts.

proofx/inc/LinkDef.h, TXSocketHandler.h, .cxx, TXUnixSocket.cxx:
 - add TXSocketHandler in the dictionary.
 - cosmetics.

Revision 15312 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Jun 2 15:14:35 2006 UTC (8 years, 7 months ago) by rdm
Original Path: trunk/proofx/inc/TXSocket.h
File length: 9295 byte(s)
Diff to previous 15138

From Gerri:
large PROOF patch. Main issues addressed:

- Output redirection to log file at 'proofserv' startup (instead of
  after Setup())
- Log messages activated at session creation
- Session sandbox restructuring in such a way that the unique session
  tag, created by the top master, uniquely identifies the session on
  all nodes, for example on top master:

   <User_Sandbox>/session-<top_master_session_tag>/
    master-0-<top_master_session_tag>/master-0-<top_master_session_tag>.log

  On worker 0.0

   <User_Sandbox>/session-<top_master_session_tag>/
    worker-0.0-<worker_0.0_session_tag>/worker-0.0-<worker_0.0_session_tag>.log

- Automatic creation of the dataset dir
- Improved handling of error conditions
- Reorganization of TProofServ and XrdProofdProtocol
- Several bug fixes, in particular in XrdProofdProtocol

Revision 14752 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Apr 19 10:57:44 2006 UTC (8 years, 9 months ago) by rdm
Original Path: trunk/proofx/inc/TXSocket.h
File length: 8713 byte(s)
Diff to previous 14110

From Gerri:
This is the big patch.

Comments:

- Client side:
  - Remove use of plugin manager to avoid unwanted locks
  - Improved treatment of error conditions

- Server side:
  - Several new functionalities described in the release notes

This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, enter a numeric revision.

Links to HEAD:	(view) (download) (as text) (annotate)
Sticky Revision: