Log of /trunk/proof/proofx/src/TXSocket.cxx
Parent Directory
Revision
48992 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Mar 28 15:26:26 2013 UTC (21 months, 3 weeks ago) by
rdm
File length: 73858 byte(s)
Diff to
previous 46964
From Lifeng Sun:
The attached patchset fixes a bunch of typo in the source:
0001-succes-success.patch
0002-preceed-preced.patch
0003-informations-information.patch
0004-childs-children.patch
0005-avaliable-available.patch
0006-writeable-writable.patch
0007-comand-command.patch
0008-unkown-unknown.patch
0009-wierd-weird.patch
0010-wheter-whether.patch
0011-unecessary-unnecessary.patch
0012-splitted-split.patch
0013-registerd-registered.patch
0014-recieve-receive.patch
0015-processsing-processing.patch
0016-ouput-output.patch
0017-mutiple-multiple.patch
0018-lenght-length.patch
0019-interupted-interrupted.patch
0020-independant-independent.patch
0021-inconsistant-inconsistent.patch
0022-expresion-expression.patch
0023-explicitely-explicitly.patch
0024-enviroment-environment.patch
0025-deafult-default.patch
0026-continous-continuous.patch
0027-completly-completely.patch
0028-commited-committed.patch
0029-choosen-chosen.patch
0030-backgroud-background.patch
0031-auxilliary-auxiliary.patch
0032-authentification-authentication.patch
0033-appropiate-appropriate.patch
0034-an-other-another.patch
0035-environement-environment.patch
0036-targetting-targeting.patch
0037-suppported-supported.patch
0038-paramater-parameter.patch
Revision
44507 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 4 12:30:41 2012 UTC (2 years, 7 months ago) by
axel
File length: 73694 byte(s)
Diff to
previous 44405
Remove
using namespace std;
from Riostream.h, which has huge consequences for all of ROOT.
Riostream.h is now a simple wrapper for fstream, iostream, iomanip for backward compatibility; Riosfwd.h simply wraps iosfwd.
Because of templates and their inline functions, Riostream.h needed to be included in headers, too (e.g. TParameter.h), which violated the assumption that Riostream.h is not exposing its using namespace std to headers.
ROOT now requires R__ANSISTREAM, R__SSTREAM, which does not change the set of supported compilers.
Without "using namespace std", several identifiers are now prefixed by std::; e.g. roofit/* source files now have a using namespace std to keep their coding style.
TFile::MakeProject() now generates "using namespace std" to convert the CINT-style class names into C++ ones.
Revision
42858 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Feb 3 11:34:05 2012 UTC (2 years, 11 months ago) by
ganis
File length: 73536 byte(s)
Diff to
previous 41550
This patch adds in TProofMgr two static functions to upload files.
These functionare called UploadFiles and differ in the first argument, which is used
to pass the files to be uploaded. These can be given as a list (of TFileInfo or TObjString),
a directory or specified in a text file.
The functions are documented at
http://root.cern.ch/drupal/content/uploading-data-files-proof-cluster
The patch also adds TProofMgr::GetMssUrl() and TProofMgr::SetMssUrl(...) to manage the mass
storage URL attached to a given cluster. By the default the information is retrieved by the
server daemon, but can be modified by the user. The functions TProof::GetDataPoolUrl() and
TProof::SetDataPoolUrl(...) now redirect to TProofMgr::GetMssUrl() and TProofMgr::SetMssUrl(...).
The obsolete functions TProof::UploadDataset have now been removed.
This patch addresses some issues raised on Savannah.
Revision
41336 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Oct 11 14:29:29 2011 UTC (3 years, 3 months ago) by
ganis
File length: 73657 byte(s)
Diff to
previous 39696
- Move the check for the username after authentication. This is because authentication
may run some credentials-to-user mapping which can modify the requested username.
This way we really check the final username and not the one requested by the client,
which may even not exist on the machines.
Side modification: when the mapping function returns more usernames, the username
specified by the client is used to help choosing the effective username among the
available choices; if not match is found the handshake does any longer fail, the first
mapped username is chosen instead.
- In TXSocket::ProcessUnsolicitedMsg, fix an issue preventig server messages to be
displayed during setup, i.e. when the XrdClientConn instance is not yet defined.
Revision
39696 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 13 21:35:34 2011 UTC (3 years, 7 months ago) by
pcanal
File length: 73551 byte(s)
Diff to
previous 39115
From David Dagenhart:
Implement an option to allow ROOT to use the LZMA compression
algorithm to compress data instead of the ZLIB compression algorithm.
LZMA compression typically results in smaller files, but takes more
CPU time to compress data. To use the new feature, the external XZ
package must be installed when ROOT is configured and built:
Download 5.0.3 from here http://tukaani.org/xz/
and make sure to configure with fPIC:
./configure CFLAGS='-fPIC'
Then the client C++ code must call routines to explicitly request LZMA
compression.
ZLIB compression is still the default.
Setting the Compression Level and Algorithm:
There are three equivalent ways to set the compression level and
algorithm. For example, to set the compression to the LZMA algorithm
and compression level 5.
1. TFile f(filename, option, title);
f.SetCompressionSettings(ROOT::CompressionSettings(ROOT::kLZMA, 5));
OR
2. TFile f(filename, option, title, ROOT::CompressionSettings(ROOT::kLZMA, 5));
OR
3. TFile f(filename, option, title);
f.SetCompressionAlgorithm(ROOT::kLZMA);
f.SetCompressionLevel(5);
These methods work for TFile, TBranch, TMessage, TSocket, and TBufferXML.
The compression algorithm and level settings only affect compression of
data after they have been set. TFile passes its settings to its branches
only at the time the branches are created. This can be overidden by
explicitly setting the level and algorithm for the branch. These classes
also have the following methods to access the algorithm and level for
compression.
Int_t GetCompressionAlgorithm() const;
Int_t GetCompressionLevel() const;
Int_t GetCompressionSettings() const;
If the compression level is set to 0, then no compression will be
done. All of the currently supported algorithms allow the level to be
set to any value from 1 to 9. The higher the level, the larger the
compression factors will be (smaller compressed data size). The
tradeoff is that for higher levels more CPU time is used for
compression and possibly more memory. The ZLIB algorithm takes less
CPU time during compression than the LZMA algorithm, but the LZMA
algorithm usually delivers higher compression factors.
The header file core/zip/inc/Compression.h declares the function
"CompressionSettings" and the enumeration for the algorithms.
Currently the following selections can be made for the algorithm:
kZLIB (1), kLZMA (2), kOldCompressionAlgo (3), and kUseGlobalSetting
(0). The last option refers to an older interface used to control the
algorithm that is maintained for backward compatibility. The following
function is defined in core/zip/inc/Bits.h and it set the global
variable.
R__SetZipMode(int algorithm)
If the algorithm is set to kUseGlobalSetting (0), the global variable
controls the algorithm for compression operations. This is the
default and the default value for the global variable is kZLIB.
Note that the LZMA algorithm will only be available if the LZMA
libraries from the XZ package were available when the ROOT executable
being used was configured and built. If you are building ROOT and want
LZMA then you must do something similar to the following if XZ is not
already installed on your system.
Download XZ version 5.0.3 from http://tukaani.org/xz/
unwind the tarball
cd xz-5.0.3
./configure CFLAGS='-fPIC'
make
ROOT was tested with version 5.0.3 of XZ. It might work with earlier
versions and will probably work with later versions of XZ.
Then either use "make install" to put the library and headers into
/usr/local/* so the scripts building ROOT can find them or use
arguments to the ROOT configure script to point to the proper
directories containing the XZ library and headers. These are the
library and the included header (lzma.h includes other headers in the
lzma subdirectory).
src/liblzma/.libs/liblzma.a
src/liblzma/api/lzma.h
WARNING: Data compressed with the LZMA algorithm cannot be read by
ROOT executables that were not built with LZMA support.
Revision
39115 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu May 5 15:54:41 2011 UTC (3 years, 8 months ago) by
ganis
File length: 73536 byte(s)
Diff to
previous 38922
- Fix bug in TXSocket::Create whose effect was to make the three additional attempts
following the first to send a wrong and meaningless creation info (ordinal number, etc).
The bug was introduced while fixing the log path trasmission problem.
- Improve the way information about the logs files is saved when failure happens
during creation. The log paths for these failing now should be now correctly saved
and accessible via TProofLog.
- Improve TSlave::Print when the connection is invalid (print all the valid information,
available, not only the header one)
Revision
38922 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Apr 19 13:15:05 2011 UTC (3 years, 9 months ago) by
ganis
File length: 73196 byte(s)
Diff to
previous 38580
Adapt to new XrdClient constructors allowing to setup a connection using an open socket.
Also extend the protocol handshake and Send/Recv calls to use a generic physical connection.
Revision
38580 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Mar 23 15:24:54 2011 UTC (3 years, 10 months ago) by
ganis
File length: 72980 byte(s)
Diff to
previous 38538
Patch fixing a problem with log path transmission when the node dies early or not even starts.
The log path was empty and wrong was filled in when retrieving the log buffers, disorienting
debugging. This was experienced frequently by ALICE.
The log file is now trasmitted as part of the responde to the session 'Create' request, so it
is always registered by the master in the relevant field in TSlave. Also, the last errors, if
any, typically logged in the xproofd log file, are now copied to the relevant worker or master
log file so that they are also accessible to client.
The patch also factorizes some calls in XrdProofdProofServMgr in preparation to the improved
session lauch strategy.
Revision
38538 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Mar 21 14:13:29 2011 UTC (3 years, 10 months ago) by
ganis
File length: 72623 byte(s)
Diff to
previous 34428
In TXSocket and TXSlave:
- Fix problem with the way collection over a socket just marked as 'bad' was interrupted;
the interrupt was de facto ineffective, so that collection stayed always until the
timeout expired (default: 5 minutes). Should solve some of the cases were slow response
was experienced.
Revision
34428 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jul 15 12:35:34 2010 UTC (4 years, 6 months ago) by
ganis
File length: 72330 byte(s)
Diff to
previous 34254
Fix an issue with a semaphore closing actions and asynchronous processing.
This should remove a problem sometimes happening at the end of stressProof
where the main application goes into timeout when closing the PROOF session.
Revision
34101 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 24 12:12:59 2010 UTC (4 years, 7 months ago) by
ganis
File length: 72420 byte(s)
Diff to
previous 33868
Following a request from ALICE, implement a timer to terminate idle sessions. The timeout
value is controlled by the variable ProofServ.IdleTimeout (value in seconds). For example,
to set a timeout of 20 mins, enter the following in the xproofd config file:
### Setup an idle timeout
xpd.putrc ProofServ.IdleTimeout 1200
The timer is disabled by default.
Revision
31441 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Nov 27 07:29:29 2009 UTC (5 years, 1 month ago) by
ganis
File length: 71288 byte(s)
Diff to
previous 30949
- Fix a backward compatibility issue with the recently introduced status files
- Improve synchronization between parent and child during fork
- Optimize loops over directory entries
- Improve error and notification messages
Revision
30949 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 2 16:37:58 2009 UTC (5 years, 2 months ago) by
ganis
File length: 71118 byte(s)
Diff to
previous 30859
Import from the vendor branch new version of 'xrootd' (20091102-0415-root).
This version contains some important fix in the client 'readv' needed by ALICE
and new cache policies on the client side. It also contains the new listing
features implemented recently in the 'cns' module.
Revision
30859 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Oct 24 14:53:07 2009 UTC (5 years, 3 months ago) by
ganis
File length: 71203 byte(s)
Diff to
previous 29632
Patch for improved performance monitoring. The 'Rate Plot' button in the progress
dialog is renamed 'Performance plot' and shows up to 4 plots with the event/sec,
the average read chunck size, the number of active workers and the number of active
PROOF sessions on the cluster, all as a function of processing time.
The read chunck size plot allows to monitor the usage of the cache.
The istantaneous processing rate (event/sec) is now better estimate: a few issues
with the normalizing times have ben solved, removing the artificial structures that
were observed.
The possibility to set a max packet time length is introduced (default 30 s); this
can be changes with the parameter PROOF_MaxPacketTime.
The size of the cache is also taken into account to optimize the use of the cache.
The parameter PROOF_UseParallelUnzip has been introduced to toggle the use of the
parallel unzip (default off for now).
A page describing the new performance plots is under preparation at
http://root.cern.ch/drupal/content/progress-dialog
Revision
29613 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 29 11:21:27 2009 UTC (5 years, 5 months ago) by
ganis
File length: 69744 byte(s)
Diff to
previous 29579
In TXProofMgr::Cp and related:
- fix a problem with the locality determination
- improve the notification messages during transfer
Revision
28843 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 8 12:49:50 2009 UTC (5 years, 7 months ago) by
ganis
File length: 68020 byte(s)
Diff to
previous 28655
In TXSocket::Recv, make sure that kMESS_ACK is always filtered out from fWhat,
so that checks on TMessage::What() are not screwed-up.
Revision
27034 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Dec 19 15:30:06 2008 UTC (6 years, 1 month ago) by
ganis
File length: 67380 byte(s)
Diff to
previous 26993
From Jan: prepare for query-enqueing support:
- add new protocol message to resume an enqueued session
- add framework to process enqueued queries after a resume: the processing part on masters
has been moved to a new method ProcessNext which is called also after a resume request.
- remove duplication in the definition of message types for SendCoordinator
(same codes defined in XProofProtocol.h and TXSocket.h)
Revision
26993 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 17 16:01:04 2008 UTC (6 years, 1 month ago) by
rdm
File length: 67156 byte(s)
Diff to
previous 26951
From Gerri:
- Fix a problem with filtering the SvcMsg in TProofMgrLite::GetSessionLogs
- Fix a weird backward-compatibility problem in TProof::SendFile (found
and needed by ALICE)
- Update the release notes
Revision
26951 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Dec 16 12:04:38 2008 UTC (6 years, 1 month ago) by
ganis
File length: 67750 byte(s)
Diff to
previous 26738
- Fix an issue with uploading packages in rapid sequence
- Fix an issue with TProof::Load() in forced compilation mode ('++')
- Fix an issue with grep in log retrieval
- Fix an issue with ownership of the xrootd log file
- Make sure the XrdProofdProofServMgr::fCurrentSessions is always up-to-date
Revision
26678 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Dec 5 14:53:23 2008 UTC (6 years, 1 month ago) by
ganis
File length: 66925 byte(s)
Diff to
previous 26629
Patch to:
- Add the possibility to set upper limits on the virtual memory (changes in TProofPlayer,
TProofServ, TXProofServ and pmain (separate patch)).
If enebled, the session gets firts a warning when it reaches 80% of the limit, and then
processing is stopped whenit exceeds 95% of the limit, sending back the results.
Also, the memory footprint is notified when the session is terminated.
- Make sure the the active valuse in XrdProofWorker are always correct; this was not the
case for dynamic startup as the notification at the end of the query was not done.
This information is crucial or the scheduler.
The way the information is stored in XrdProofdProofServ had to be modified
and a new internal message type (kReleaseWorker) added.
(Changes in several proofd classes, TProof, TXsocket and TXProofServ)
- Fix problem with setting a static upper limit of the sessions that can be started,
and enable this functionality for the dynamic mode (changes in XrdProofSched)
- Remove a deleted worker from all the lists in TProof::MarkBad to avoid later attempts of use.
- Better control the use of the internal pipe for socket readiness notification in TXSocket.
Revision
26629 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 3 15:21:41 2008 UTC (6 years, 1 month ago) by
ganis
File length: 66291 byte(s)
Diff to
previous 26515
Condolidation fixes for the XrdProofd plugin:
- Better synchronize normal operations and regular checks in XrdProofdProofServMgr
- In XrdProofdAdmin, reduce the waiting time for Reset by optimizing the checks
- In XrdProofdClient
- Do not delete existing XrdProofdProofServ objs (just reset them for reuse)
- Eliminate some redundant methods
- In XrdProofdProfServ
- Optimize usage of local variables to remove some locks
- remove from IsValid some checks which are better placed elsewhere
- Add several protections
Revision
24719 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 9 07:07:25 2008 UTC (6 years, 6 months ago) by
ganis
File length: 65783 byte(s)
Diff to
previous 24677
Patch refactorizing the XrdProofd plugin.
The class XrdProofdProtocol is now in charge only of the operations strctly related
to the XProofd protocol. All auxilliary services have been moved to dedicated service
classes controlled by XrdProofdManager. In particular:
- XrdProofdClientMgr handles the clients (represented by instances of XrdProofdClient)
including login, authentication and access control
- XrdProofdProofServMgr handles the PROOF sessions (represented by instances of
XrdProofdProofServ) including creation, attachment, detachment, destruction and
environment setting
- XrdProofdNetMgr handles connections between instances of XProofd running on different
nodes
- XrdProofdPriorityMgr handles the session priorities
A special effort has been done to get rid of all possible internal dead-lock situations.
Internal actions on clients and sessions are now all asynchronous, governed by internal
pipes.
The new plugin also offers new functionality, among which:
- a XProofd admin area, located under <xrd.admin>/.xproof.<port>, keeps information about
active and terminated sessions, and the minimal state for active clients. This is used
to reguraly check the client and session activity, to cleanup orphalin sessions and
to shutdown inactive client connections.
In particular this allows a more solid implementation of Reset, which now exists in two
flavours: 'soft', TProof::Reset(<masterurl>), (default) asks the sessions to terminate gently;
'hard', TProof::Reset(<masterurl>,kTRUE) schedules all sessions for forced termination.
- support for automatic attempts for reconnection in the case the daemon restarts.
This allows to reconfigure the plugin by xrootd restart w/o affecting the running
sessions.
- support the definition of workers via config file directive, getting de facto rid of
the proof.conf file.
- domain+level control of printout message; the format has been improved: in particular
all information messages contain now the tag 'xpd-I' and all error messages the tag
'xpd-E', so that they can easily be grepped out from the log file.
The Wiki 'XrdProofdDirectives Directives' page has been updated with the new directives.
Revision
24568 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 26 11:57:42 2008 UTC (6 years, 6 months ago) by
ganis
File length: 59436 byte(s)
Diff to
previous 24480
- Fix a race condition possibly affecting the handling of workers death
- Improve diagnostic from MarkBad: clients will now receive something like this
root [1] Worker 'localhost-0.1' has been removed from the active list
+++ Message from top master at aleph025.cern.ch:1093 : marking localhost:1093 (0.1) as bad
+++ Reason: problems receiving a message in TProof::CollectInputFrom(...)
+++ Most likely your code crashed on worker 0.1 at localhost:1093.
+++ Please check the session logs for error messages either using
+++ the 'Show logs' button or executing
+++
+++ root [] TProof::Mgr("aleph025.cern.ch:1093")->GetSessionLogs()->Display("0.1",0)
Revision
24480 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 23 13:29:34 2008 UTC (6 years, 7 months ago) by
rdm
File length: 58323 byte(s)
Diff to
previous 23632
move the send streamer info and send process id code into:
SendStreamerInfos()
SendProcessIDs()
move the recv streamer info and recv process id code into:
RecvStreamerInfos()
RecvProcessIDs()
Add support for schema evolution and process ids also to TPSocket and
TXSocket.
Revision
20862 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Nov 18 14:51:42 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 56660 byte(s)
Diff to
previous 20682
Synchronize with braches/dev/proof r20835
Summary:
TProofServ:
- read session.rootrc with level kEnvChange to be able to change existing
settings
- Avoid deleting a query result twice in some special cases
- Fix a problem with the initialization of fKeptQueries
rootrc.in:
- restore default settings the asynchronous reading
TFileCacheRead:
- add missing protection in ReadBuffer
TEventIter:
- Enable the usage of TTreeCache
XrdProofdProtocol:
- Additional check on the ownership of the unix socket
- Improve notification during Reset
- Reduce default timeout on admin requests and make it configurable
XrdProofConn:
- Use the configurable maxtry everywhere where relevant
TPacketizerAdaptive:
- Store info on all the processed packets in per-worker lists
TProofMgr, TXProofMgr, TXSocket, XProofProtocol, XrdProofdProtocol:
- Add possibility for the admin to broadcast a message to the connected users
getProof:
- Add missing protection
Revision
20682 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 6 15:51:59 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 55956 byte(s)
Diff to
previous 20307
Import branches/dev/proof r20654
Summary:
+ Improvements
- Add support for SSH SOCKS4 tunnelling; the local port for the tunnel can be specified
in the master URL, e.g. TProof::Open("master/?tunnel:8000")
- Add the possibility to plot the estimated instantaneous rate
- Add "PROOF_ForceLocal" parameter to the !TPacketizerAdaptive; if set to 1, all the data
are processed locally.
- Add support for remote grep functionality while retrieving logs (needed by the forthcoming
memory monitor)
+ Bug fixes
- Several small fixes to revive the multi-master mode.
- XrdProofdProtocol:
- add missing lock to the client instance in SendMsg to avoid screwing up requests from
workers on the same machine
- lock the mutex of the requester when setting prorities
- add notification during Reset
- fix problem with the detection of the 'allow' directive
- fix problem with the parsing of the return value from XrdProofServProxy::TerminateProofServ()
- always use the effective user to retrieve info fom another server
(XrdProofdManager::GetProofConn is now used)
- fix possible dead-locks from debug notifications done after hard-killing a session
- re-enable the garbage collector thread of the connection manager in XrdProofConn to
fix a problem with closing physical connections;
- fix a problem with !CleanupProofServ in the case of a non-privileged daemon running
in multi-user mode
- introduce a timeout when waiting for the startup of a 'proofserv'.
- XrdProofConn: init mutex in the ctor; lock in SendRecv
- XrdProofSched: add support for using the priorities defined in the group manager to define
the number of workers for sessions
- TProof:
- Broadcast priorities to unique nodes only
- timeout after 5 mins the initial Collect to avoid clients getting stuck at this stage
- add support for generic timeout in Collect (disabled by default)
- fix a problem with SendFile.
- TXProofServ: add a call to !TProof::InterruptCurrentMonitor() in Terminate() to stop
infinite loops in Collect
- TXSocket:
- Implement a flag to interrupt a TXSocket while waiting for messages
- Split the session creation timeout in 4 attempts: the total timeout is the same but
it may circumvent occasional forking problems.
- XROOTD:
- fix a potential (possibly accademic) memory leak in the client
- fix a access permission problem with Kerberos ticket forwarding
- fix bug preventing 'locate' to work properly
- re-enable optimized 'locate'
Revision
20307 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Oct 11 10:58:50 2007 UTC (7 years, 3 months ago) by
ganis
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 54359 byte(s)
Diff to
previous 20094
Import branches/dev/proof r20306
Summary (see branch logs for more details):
- Set of changes related to CPU quota control including
- broadcast of centrally determined priorities
- mechanism to renice processes in a quantitative way
- New packetizer for non-tree based analysis and related API in TProof (from L. Tran-Thanh)
- Support for merging output objects saved in files on the workers (from L. Tran-Thanh and me)
- Improve version binary compatibility checks using also the SVN revision number
(when available) to define the running version.
- Extend the version binary compatibility checks also to the cached selector binaries.
- Extend TDSet::Lookup so that in case of missing files, it can remove them from the
dataset (option removeMissing must be set).
- Move the data set lookup to the TProofPlayerRemote::Process.
- Handle properly the case of incomple datasets: if the file is not found in the lookup
don't try to validate it; add it, instead, to a 'missingFiles' list returned in the
output list (fixing bug #28800 in Savannah).
Revision
18353 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Mar 19 15:14:10 2007 UTC (7 years, 10 months ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 52375 byte(s)
Diff to
previous 17035
From Gerri:
- Support for multiple ROOT versions on a PROOF cluster.
TProofMgr::ShowROOTVersions() giving the following output:
root [0] TProof::Mgr("")->ShowROOTVersions()
----------------------------------------------------------
Available versions (tag ROOT-vers remote-path PROOF-version):
* cvs 5.15/03 /home/ganis/local/root/head/head 12
dev 5.15/03 /home/ganis/local/root/head/root 12
----------------------------------------------------------
- TProofMgr::SetROOTVersion(const char *version_tag)
to change the default for the user, eg
root [1] TProof::Mgr("")->SetROOTVersion("dev")
root [2] TProof::Mgr("")->ShowROOTVersions()
----------------------------------------------------------
Available versions (tag ROOT-vers remote-path PROOF-version):
cvs 5.15/03 /home/ganis/local/root/head/head 12
* dev 5.15/03 /home/ganis/local/root/head/root 12
----------------------------------------------------------
Available version are defined by the administrator via the
"xpd.rootsys" directive:
xpd.rootsys /home/ganis/local/root/head/head cvs
xpd.rootsys /home/ganis/local/root/head/root dev
in the xrootd config file.
- Improve error reporting in TXSocket
- Fix a problem with resetting a label in TProofProgressDialog
Revision
17035 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Dec 3 23:34:04 2006 UTC (8 years, 1 month ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 51232 byte(s)
Diff to
previous 16858
From Gerri:
0. Introduction of the static:
TProof::Mgr(const char *url)
to get the manager to the cluster; this gives access to all the manager
functionality, included Reset(). TProof::Reset() is removed.
1. Two new classes: TProofLog and TProofLogElem.
TProofLog contains a list of TProofLogElem, one per node. TProofLogElem
basically contains a TMacro plus methods to play with it.
2. The TProofLog object can be retrieved via the manager:
TProofLog *pl = TProof::Mgr("cluster")->GetSessionLogs()
by default the last session is retrieved: the signature for GetSessionLogs is
TProofLog *TProofMgr::GetSessionLogs(Int_t ridx, const char *stag)
the first argument is a reversed relative index (-1 gets the next to last
session), the second is the session unique tag; if the second is
specified the first is ignored.
3. TProofMgr has a new method ReadBuffer to quickly read a buffer from
a file via XrdProofd: this allows to avoid the second xrootd on the
redirector and in general is useful when a quick reading is needed.
Also, the buffer is always retrieved via the master, so we should not
have firewall problems in accessing directly the xrootd of the workers.
4. In the sandbox I have kept the .sessions file with the active sessions
and I have added .terminated in each session working dir which have
been terminated. On the master, each session working dir has the file
.workers with the relevant information about the workers.
5. The max number of old session workdirs is set to 10 by default
(configurable in xpd.cf): the sandbox is trimmed when closing a session.
6. Some unrelated changes/fixes contained in the patch:
- in XrdProofdProtocol:
- use of /proc (linux, solaris) or sysctl (mac) to verify/search
for running of a process. For other platforms I have left the
parsing of the output of 'ps'.
- I have introduced a 'cron' thread: it runs every minute and for
the moment it just cleans a vector; this is the place where to do
client independent actions, like checking if the config files have
changed, for example.
- I have added the possibility to control which user can access
the cluster; requested by Jan Fiete.
7. The patch contains also several fixes here and there I have found by
testing.
Revision
16839 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 20 15:56:36 2006 UTC (8 years, 2 months ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 49615 byte(s)
Diff to
previous 16775
From Maarten + Gerri:
Patch to allow run time configuration of proofserv environment. The client
can pass a set of shell variables to PROOF. These variables can be used
to startup different versions of the proofserv, or they can instruct the
system to run proofserv under valgrind or tcheck, etc. Main changes:
- default wrapper script for proofserv
- modifies the build system for the wrapper script
- adds a list of environment variables to TVirtualProof, which are
passed to the wrapper by proofd/xrootd.
- adds code to TProofServ to pickup these variables and pass them
along to the slaves/submasters
Revision
16775 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Nov 15 17:45:55 2006 UTC (8 years, 2 months ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 49070 byte(s)
Diff to
previous 16557
From Gerri and Bertrand:
Fixes and new features developed in preparation for the Intel quad-core
launch demo:
- Add instantaneous event and MB processing rates in the Progress signal
and make necessary modification to exploit it in the Progress Dialog.
- Add possibility to change the Progress and Feedback periods
- A number of small improvements and bug fixes
Revision
16557 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Oct 19 12:38:07 2006 UTC (8 years, 3 months ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 49044 byte(s)
Diff to
previous 16460
From Gerri:
Major set of robustness fixes, improving error reporting and reducing the
startup failures especially observed on AMD64. Summary of changes:
- XrdProofdProtocol.cxx, .h
- Remove zombie treatment: done by XrdScheduler::Reaper()
- Check the modification date of the PROOF config file so that changes
can be picked up on the fly
- Fix problem with the definition of the image
- Improved tracing (see etc/proof/xpd.cf.sample)
- Several bug fixes
- XrdProofConn
- fix problem with marshalling in case of retries.
- TProofServ, TXProofServ
- Add return codes to CreateServer() and Setup() for proper termination
in case of errors
- TProof
- Make sure that TXProofMgr is the last one in the list of sockets for proper
cleaning
Revision
15930 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Aug 5 20:04:47 2006 UTC (8 years, 5 months ago) by
brun
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 49142 byte(s)
Diff to
previous 15860
From Gerri:
I have realized that the new tarball requires the mods to the XrdProofd plugin
that I wanted to keep separated. This explains the problem on Mac (and also
everywhere the plug-in is build).
It also contains the fix of the problem mention by Fons with selectors from a library.
Revision
15829 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jul 20 01:24:48 2006 UTC (8 years, 6 months ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 47916 byte(s)
Diff to
previous 15507
From Gerri:
- Fix a couple of bugs affecting allocation and release of auxilliary buffers
- Make the list of spare buffers and its mutex static
- Implement an upper bound on the average amount of memory allocated
in spare buffers: default 10 MB, can be changed via the env XProof.MaxBuffMem
Revision
15507 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 21 16:18:27 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 46845 byte(s)
Diff to
previous 15391
From Gerri:
The patch implements the following new functionality:
- Shutdown of idle sessions when quitting ROOT; if not idle the session
is shutdown as soon as processing terminates; it is possible to delay
the automatic shutdown by a tunable amount of time via a new directive
in the xrootd config file 'xpd.shutdown' (see xpd.cf.example).
- New static method TProof::Reset(const char *url) to hard-cleanup an
user area (implemented for Xproofd only); this implied quite some
additions in XrdProofdProtocol.
- Possibility to temporary remove / re-insert a worker from the active list
TProof::DeactivateWorker(const char *ord)
TProof::ActivateWorker(const char *ord)
with 'ord' the ordinal numer, e.g. "0.3" .
- A default url for the disk pool
const char *TVirtualProof::GetDataPoolUrl()
It is set by default to a string send back by the master, but it can
be changed via
void TVirtualProof::SetDataPoolUrl(const char *url)
Jan should then adapt his code to use this as default.
- Posibility to timeout Collect; for now this is only used at forced
termination.
The patch includes many small changes and bug fixes. One of the most important
is in TXNetSystem (a bug preventing correct treatment of multiple request
to the same redirector).
Revision
15338 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 5 22:51:14 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 41698 byte(s)
Diff to
previous 15321
From Gerri:
base/inc/TVirtualProof.h, proof/inc/TProof.h, proof/src/TProof.cxx
proof/src/TProofProgressDialog.cxx:
- fix a problem with the progress dialog when detaching a session:
a new signal CloseProgressDialog() is introduced, as well as the
possibility to empty the list of submitted queries at once via
TProof::Remove().
proofx/inc/TXHandler.h, .cxx:
- add generic argument to HandleInput() and HandleError().
proofx/inc/TXProofMgr.h, .cxx:
- move to new signature for HandleInput(), HandleError().
proof/inc/TSlave.h:
- add method to flush underlying socket.
proofx/inc/TXSlave.h, .cxx:
- move to new signature for HandleInput(), HandleError().
- implement new method to flush underlying socket.
proofx/inc/TXSocket.h, .cxx:
- Add method to flush socket.
- Handle interrupt messages in the reader thread.
- Remove interrupt semaphore (not needed any more).
- Remove timeout argument from GetInterrupt (not needed any more).
proofx/inc/TXProofServ.h, .cxx:
- move to new signature for HandleInput(), HandleError().
- implement interrupt handling in HandleInput().
- implement dedicated version of LockSession to deal with new format
of session tag (this is a bug fix).
proofd/inc/XrdProofConn.h, .cxx:
- Add mutex protection in the SendRecv method (it can be called from both
the reader and main threads).
proofd/src/XrdProofdProtocol.cxx:
- use an asynchronous message (instead of a signal) for interrupts.
proofx/inc/LinkDef.h, TXSocketHandler.h, .cxx, TXUnixSocket.cxx:
- add TXSocketHandler in the dictionary.
- cosmetics.
Revision
15312 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 2 15:14:35 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 41477 byte(s)
Diff to
previous 15138
From Gerri:
large PROOF patch. Main issues addressed:
- Output redirection to log file at 'proofserv' startup (instead of
after Setup())
- Log messages activated at session creation
- Session sandbox restructuring in such a way that the unique session
tag, created by the top master, uniquely identifies the session on
all nodes, for example on top master:
<User_Sandbox>/session-<top_master_session_tag>/
master-0-<top_master_session_tag>/master-0-<top_master_session_tag>.log
On worker 0.0
<User_Sandbox>/session-<top_master_session_tag>/
worker-0.0-<worker_0.0_session_tag>/worker-0.0-<worker_0.0_session_tag>.log
- Automatic creation of the dataset dir
- Improved handling of error conditions
- Reorganization of TProofServ and XrdProofdProtocol
- Several bug fixes, in particular in XrdProofdProtocol
Revision
14752 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 19 10:57:44 2006 UTC (8 years, 9 months ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 39189 byte(s)
Diff to
previous 14732
From Gerri:
This is the big patch.
Comments:
- Client side:
- Remove use of plugin manager to avoid unwanted locks
- Improved treatment of error conditions
- Server side:
- Several new functionalities described in the release notes
Revision
14717 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Apr 17 21:04:17 2006 UTC (8 years, 9 months ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 38452 byte(s)
Diff to
previous 14585
From Gerri:
This is the new XROOTD tarball including:
- support for Windows
- support pluggable OLB-like protocols (needed by Jan)
- several bug fixes relevant for asynchronous operations and PROOF.
Bertrand has run successfully a few minimal tests with TFile::Open and
TProof::Open from home to a server running on lxplus.
The patch contains the minimal set of mods to keep CVS compiling. It is
possible that you need some additional patch from Bertrand for the
Windows part. I guess it will be easy for him to send the remaining parts
once this is in.
Revision
14164 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Mar 3 15:42:37 2006 UTC (8 years, 10 months ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 37817 byte(s)
Diff to
previous 14109
From Gerri:
- TXSocket.cxx, TXProofMgr.cxx:
Fix problem causing in some cases a new unwanted session being created
with the wrongs IDs.
- TProof.cxx, TVirtualProof.cxx
Make sure to store always the FQDN (if available) as master host name.
- TProofMgr.cxx
Remove redundant debug printout.
Revision
13681 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Dec 13 10:12:02 2005 UTC (9 years, 1 month ago) by
brun
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 37254 byte(s)
Diff to
previous 13662
From Gerri:
Writing the notes I realized that the attach functionality of gROOT->Proof(...)
could be automatized, such that it attaches to the first session on the master,
if any:
gROOT->Proof("masterhost") // attach to first session, if any, or create one
gROOT->Proof("masterhost/?N") // force creation of new session
gROOT->Proof("masterhost/?2") // attach to session 2
In the standard case in which there is only a session this gives a very convenient
way of reconnecting.
I have fixed also a small problem with setting of fUser and fRemoteProtocol
for the second slave on a node.
Revision
13648 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Added
Mon Dec 12 12:54:27 2005 UTC (9 years, 1 month ago) by
rdm
Original Path:
trunk/proofx/src/TXSocket.cxx
File length: 37985 byte(s)
From Gerri:
- last part of the major PROOF upgrade to support disconnect/reconnect
features using the libXrdProofd.so xrootd plugin.
More on how to configure and run with xrootd/xproofd coming very soon.
This form allows you to request diffs between any two revisions of this file.
For each of the two "sides" of the diff,
enter a numeric revision.
