Log of /trunk/proof/proof/inc/TProofServ.h
Parent Directory
Revision
48060 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Dec 15 18:53:16 2012 UTC (2 years, 1 month ago) by
ganis
File length: 20166 byte(s)
Diff to
previous 45740
From Dario: patch introducing an improved dataset management model where the PROOF (ROOT)
dataset manager is a light frontend to the experiment file catalogs; TDataSetManagerFile
is still used as local cache of the experiment information or to store the work-in-progress
status of the dataset manager daemon. This model is expected to solve the scalability issues
observed currently at AAFs.
The patch includes the new class TDataSetManagerAliEn with the first concrete implementation
of experiment catallog interface. The equivalent for ATLAS will follow.
Revision
45740 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Aug 29 16:17:09 2012 UTC (2 years, 4 months ago) by
ganis
File length: 20062 byte(s)
Diff to
previous 45718
- Add support for data dirs on external servers, fully honouring URL options; this allows to
have usable paths registered in the final TFileCollection objects. One of the addressed use-case
is when local directories need to be accessed via the local xrootd server to preserve the namespace
enforced by xrootd; in such a case something like 'root://locahost//datadir?remote=1' needs to be
specified; the option 'remote=1' ensures that the final registered URL is still in 'root://' form
with 'localhost' replaced by the FQDN of the machine.
- Move the 'tests' argument to the stressProof function in the second position, to make it
more user friendly to run a specific test in the ROOT shell; CLI behevior is unchanged.
- In ProofSimple, add function to plot the histograms also in the case a dataset is created with the
worker files; the file are opened and the histograms merged on the fly.
- Fix potential segv at xproofd startup due to a double fclose in XrdProofdManager::ValidateLocalDataSetSrc
- Remove some redundant printing in TProof when handling output files.
- Make sure that when running TSelVerifyDataSet the save-to-file functionality is switched-off .
Revision
45718 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Aug 28 10:06:59 2012 UTC (2 years, 4 months ago) by
ganis
File length: 19928 byte(s)
Diff to
previous 45348
- Extend parallel dataset verification to PROOF-Lite
- Remove some duplication between TProofServ and TProofLite
Should solve problem observed with stressProof in CDash.
Revision
45348 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Aug 2 12:23:54 2012 UTC (2 years, 5 months ago) by
ganis
File length: 19845 byte(s)
Diff to
previous 45283
Consolidation of the automatic output saving functionality
- Fix an issue with saving output from memory on the master
- Remove some duplications and fix several small bugs
Revision
45283 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 30 10:42:38 2012 UTC (2 years, 5 months ago) by
ganis
File length: 19664 byte(s)
Diff to
previous 44655
Patch automatizing the usage of the file-based technology to handle outputs.
After this patch PROOF users can define as an option to Process an output file for their results
and choose if the merging goes via file or in memory. User can also decide to save theit results
in a set of files identified by a name (dataset).
Administrators can force merging-via-file or can make it conditional to the resident memory exceeding
a given threshold.
The new options are documented at
http://root.cern.ch/drupal/content/handling-outputs
Since TProofOutputFile is now used by TProof, the class has been moved from proof/proofplayer
(libProofPlayer) to proof/proof (libProof).
The selector tutorials/proof/ProofSimple.h,.C has been modified to allo testing the new options,
and a new test has been added to stressProof.
Revision
44010 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Apr 29 23:07:48 2012 UTC (2 years, 8 months ago) by
ganis
File length: 19566 byte(s)
Diff to
previous 42882
Several optimizations in the low level PROOF event loop (TProofPlayer::Process):
- Per-event calls to TEventIter::GetNextEvent are replaced by per-packet calls
to a new function TEventIter::GetNextPacket and a for loop over the returned
range;
- Progress information is filled in on per-packet pace just before asking for
a new packet, instead of being filled for each event;
- In TProofProgressStatus, the relevant setters in were inlined, and the (expensive)
calls to SetLastUpdate() reduced to the minimum required;
- Status of objects is checked using bits instead of functions calls;
- Check whether to dump memory info is done before calling CheckMemUsage.
These optimizations allow to reduce dramatically the overhead introduced by the
operations PROOF needs to perform during the event loop. A measurement of the
overhead can be obtained from a very light computational task, for example,
generating one random number and filling one histogram; executing this task
within a PROOF-Lite session with 1 worker now takes only 1.8 times the time
required by a straight loop in the parent ROOT session; the same number before
was about 13.
The patch also introduces a couple of small changes enabling the use of
TProofPlayerLocal to process selectors in the local session using the PROOF
iteration mechanism; this is very useful for optimizations studies.
Finally, a couple of missing protections were added in TProofPlayer.
Revision
42882 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 6 11:10:21 2012 UTC (2 years, 11 months ago) by
ganis
File length: 19450 byte(s)
Diff to
previous 39696
From Sangsu Ryu and me.
Add support for paralell dataset verification. This is implemented via a dedicated
TSelector (TSelVerifyDataSet) which is run over the list of files in the dataset via
TPacketizerFile. The file order is preserved using the recently introduced index in
TFileInfo.
Parallel verification is the now the default. For test and comparison purposes it is
still possible to verify datasets serially by specifying 'S' in the option field of
VerifyDataSet and RegisterDataSet .
To avoid/reduce code duplication some of the static functions in TDataSetManager have
refactorized. The overall behavior should be unchanged.
Revision
39696 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 13 21:35:34 2011 UTC (3 years, 7 months ago) by
pcanal
File length: 19370 byte(s)
Diff to
previous 39074
From David Dagenhart:
Implement an option to allow ROOT to use the LZMA compression
algorithm to compress data instead of the ZLIB compression algorithm.
LZMA compression typically results in smaller files, but takes more
CPU time to compress data. To use the new feature, the external XZ
package must be installed when ROOT is configured and built:
Download 5.0.3 from here http://tukaani.org/xz/
and make sure to configure with fPIC:
./configure CFLAGS='-fPIC'
Then the client C++ code must call routines to explicitly request LZMA
compression.
ZLIB compression is still the default.
Setting the Compression Level and Algorithm:
There are three equivalent ways to set the compression level and
algorithm. For example, to set the compression to the LZMA algorithm
and compression level 5.
1. TFile f(filename, option, title);
f.SetCompressionSettings(ROOT::CompressionSettings(ROOT::kLZMA, 5));
OR
2. TFile f(filename, option, title, ROOT::CompressionSettings(ROOT::kLZMA, 5));
OR
3. TFile f(filename, option, title);
f.SetCompressionAlgorithm(ROOT::kLZMA);
f.SetCompressionLevel(5);
These methods work for TFile, TBranch, TMessage, TSocket, and TBufferXML.
The compression algorithm and level settings only affect compression of
data after they have been set. TFile passes its settings to its branches
only at the time the branches are created. This can be overidden by
explicitly setting the level and algorithm for the branch. These classes
also have the following methods to access the algorithm and level for
compression.
Int_t GetCompressionAlgorithm() const;
Int_t GetCompressionLevel() const;
Int_t GetCompressionSettings() const;
If the compression level is set to 0, then no compression will be
done. All of the currently supported algorithms allow the level to be
set to any value from 1 to 9. The higher the level, the larger the
compression factors will be (smaller compressed data size). The
tradeoff is that for higher levels more CPU time is used for
compression and possibly more memory. The ZLIB algorithm takes less
CPU time during compression than the LZMA algorithm, but the LZMA
algorithm usually delivers higher compression factors.
The header file core/zip/inc/Compression.h declares the function
"CompressionSettings" and the enumeration for the algorithms.
Currently the following selections can be made for the algorithm:
kZLIB (1), kLZMA (2), kOldCompressionAlgo (3), and kUseGlobalSetting
(0). The last option refers to an older interface used to control the
algorithm that is maintained for backward compatibility. The following
function is defined in core/zip/inc/Bits.h and it set the global
variable.
R__SetZipMode(int algorithm)
If the algorithm is set to kUseGlobalSetting (0), the global variable
controls the algorithm for compression operations. This is the
default and the default value for the global variable is kZLIB.
Note that the LZMA algorithm will only be available if the LZMA
libraries from the XZ package were available when the ROOT executable
being used was configured and built. If you are building ROOT and want
LZMA then you must do something similar to the following if XZ is not
already installed on your system.
Download XZ version 5.0.3 from http://tukaani.org/xz/
unwind the tarball
cd xz-5.0.3
./configure CFLAGS='-fPIC'
make
ROOT was tested with version 5.0.3 of XZ. It might work with earlier
versions and will probably work with later versions of XZ.
Then either use "make install" to put the library and headers into
/usr/local/* so the scripts building ROOT can find them or use
arguments to the ROOT configure script to point to the proper
directories containing the XZ library and headers. These are the
library and the included header (lzma.h includes other headers in the
lzma subdirectory).
src/liblzma/.libs/liblzma.a
src/liblzma/api/lzma.h
WARNING: Data compressed with the LZMA algorithm cannot be read by
ROOT executables that were not built with LZMA support.
Revision
39074 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue May 3 09:30:02 2011 UTC (3 years, 8 months ago) by
ganis
File length: 19129 byte(s)
Diff to
previous 38570
From S. Ryu:
- Fix issue affecting the result of subsequent worker activation/deactivation requests when worker
ordinal numbers were not in increasing order
From me:
- Add support for block requests for worker activation/deactivation (much more efficient, because
FindUniqueSlaves is only called once); the list of workers to have their statis changed is comma-
separated.
- Make DeactivateWorker and ActivateWorker to return != 0 when something went wrong, with -2 indicating
that at least one of the worker ordinal numbers in the list is invalid (e.g. '0.134' for a cluster
with 20 workers only).
Revision
38570 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Mar 23 12:05:17 2011 UTC (3 years, 10 months ago) by
ganis
File length: 19129 byte(s)
Diff to
previous 36592
Add support for log file truncation (see Savannah #74730).
Truncation is disabled by default. Enabling is controlled by the rootrc variable
ProofServ.LogFileMaxSize {<bytes>|<kilobytes>K|<megabytes>M|<gigabytes>G}
indicating the max number of bytes. The number can be followed by a {K,M,G} to indicate
KBytes, MBytes or GBytes, respectively. So, for example, the following setting in the
xproofd configuration file sets a limit to 100MBytes:
xpd.putrc ProofServ.LogFileMaxSize 100M
Revision
36592 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Nov 11 10:43:17 2010 UTC (4 years, 2 months ago) by
ganis
File length: 18976 byte(s)
Diff to
previous 36537
- Terminate a proofserv after it has been stopped for large memory footprint and it has
sent the results back
- Notify information about which object was being merged if a memory exception happens
during merging on the master
- Allow to set the HWM and stop-process fractions via the environment
Revision
36537 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 8 16:16:41 2010 UTC (4 years, 2 months ago) by
ganis
File length: 18871 byte(s)
Diff to
previous 36475
Patch to improve the diagnostic in case of exceptions. Information about the event and file being
processed at the moment the exception was raised is sent to the client, e.g.
0.5: caught exception triggered by signal '1' while processing dset:'EventTree',
file:'http://root.cern.ch/files/data/event_3.root', event:1 - check logs for possible stacktrace
The patch also fixes a problem with submergers observed when a worker was stopped because above
the memory limits: this worker was established as merger but could not do the work, for obvious
reasons, freezing the session.
Revision
36475 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 2 13:23:08 2010 UTC (4 years, 2 months ago) by
rdm
File length: 18745 byte(s)
Diff to
previous 34101
From Gerri:
add the possibility to control the resident and virtual memory of a proofserv
using 'ulimit', which has less limitations and more flexibility than setrlimit.
Revision
34101 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 24 12:12:59 2010 UTC (4 years, 7 months ago) by
ganis
File length: 18468 byte(s)
Diff to
previous 33824
Following a request from ALICE, implement a timer to terminate idle sessions. The timeout
value is controlled by the variable ProofServ.IdleTimeout (value in seconds). For example,
to set a timeout of 20 mins, enter the following in the xproofd config file:
### Setup an idle timeout
xpd.putrc ProofServ.IdleTimeout 1200
The timer is disabled by default.
Revision
33824 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 10 14:43:40 2010 UTC (4 years, 7 months ago) by
ganis
File length: 17645 byte(s)
Diff to
previous 33749
- Add new directive 'xpd.datadir' to better control the user data directories and their
permission settings. The syntax is 'xpd.datadir <datadir> <opts>' where <datadir> is
the root path for the datadirs (the final datadir will be <datadir>/<group>/<user>) and
<opts> can contain a combination of the following letters: 'g', 'a', 'M', 'W' ["gW"];
the lower case control the write permissions: 'g' group, 'a' everybody; the upper case
control where the data dirs shoudl be created: 'W' on the workers, 'M' on the master,
"WM" everywhere.
- Better handle the removal of empty data directories in TProofServ and derivatives
- In TProofOutputFile:
- extend support for URL options
- correctly take the URL options into account
This should solve a problem observed in the new ALICE CAF.
Revision
33749 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 4 13:56:55 2010 UTC (4 years, 7 months ago) by
ganis
File length: 17595 byte(s)
Diff to
previous 33369
Patch implementing a few changes/fixes in the package management
- Add support for arguments in the SETUP function: it is possible now to pass a string
Int_t SETUP(const char *opt)
or a list of objects
Int_t SETUP(TList *optls);
Two interface methods have been added to support this:
Int_t TProof::EnablePackage(const char *package, const char *opt, Bool_t notOnClient);
Int_t TProof::EnablePackage(const char *package, TList *optls, Bool_t notOnClient);
- Make sure the return code from BUILD.sh is properly checked and execution flow stoped on
failure; this required the introduction of a check on the return code of TSystem::ClosePipe().
- Allow building packages in the global package directories if 'write' permissions are available;
this facilitates management of global packages from an authorized account.
Revision
33280 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 28 14:45:01 2010 UTC (4 years, 8 months ago) by
ganis
File length: 17356 byte(s)
Diff to
previous 31683
Restructuring of the syslog logging functionality.
The activation is still controlled by the ProofServ.LogToSysLog switch, but now two
levels of logging are provided:
2 log everything to syslog (local5)
1 log only a record for each action performed by the client; by default the
format is
user message-type realtime cputime
When relevant, some information is added, like file name, selector name,
package name, bytes read, etc ...
This may be useful to get a quick monitoring of the cluster activities.
The format of the message to syslog has also been sligthly modified and made more compact;
example
Apr 28 15:50:53 pcphsft64 proofm-0[15086]: ganis 5 0.006 0.010 .!pwd
Apr 28 16:14:52 pcphsft64 proofm-0[18814]: ganis 1012 0.000 0.000 h1analysis.C
Apr 28 16:14:52 pcphsft64 proofm-0[18814]: ganis 1013 0.031 0.000
Apr 28 16:14:52 pcphsft64 proofm-0[18814]: ganis 1012 0.000 0.000 h1analysis.h
Apr 28 16:14:52 pcphsft64 proofm-0[18814]: ganis 1013 0.029 0.010
Apr 28 16:15:02 pcphsft64 proofm-0[18814]: ganis 1015 7.097 0.090 0 283813 22554834 5.920
for the action records, and
Apr 28 16:37:44 pcphsft64 proofm-0[20907]: ganis: Info:<TXProofServ::Terminate>: starting session termination operations ...
for an Info message.
Revision
31683 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Dec 8 16:12:08 2009 UTC (5 years, 1 month ago) by
ganis
File length: 17148 byte(s)
Diff to
previous 31505
TProofPlayerRemote
- In SendSelector, add missing option kCpBin when sending the selector source; the binary files were
never retrieved, even if present and valid
TProofPlayerSlave
- In Process, fix a problem with cache directory locking while building the selector; the net effect
was that each worker process was re-buidling its own selector binary.
- Fix warning from the nightlies (variable possibly clobbered by `longjmp' or `vfork')
TProofServ
- Fix the order in which the log file is sent in asynchronous processing; the wrong order was screwing
up an immediate synchronous query submission after an asynchronous run; this case occured, for example,
in 'stressProof'
Revision
31505 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 2 13:20:06 2009 UTC (5 years, 1 month ago) by
brun
File length: 17045 byte(s)
Diff to
previous 31012
From Katerina Opocenska:
Patch implementing the new submerger functionality to speed up the merging phase.
At the end of the query, a set of workers are promoted submergers and assigned a
sub-set of workers to merge. Once each sub-merger has merged its sub-set of workers,
it sends its result to the master, which merges the partial results into the final
set of results.
The determination of the sub-mergers is always done dynamically, based on the recent
performance of workers. An optimal (i.e. giving the highest speed-up) number can be
calculated analytically under simpleassumptions.
Merging via submergers is by default disabled. To enable it, with the optimal number
of sub-mergers, one should set the integer parameter 'PROOF_UseMergers' to 0, i.e.
proof->SetParameter("PROOF_UseMergers", 0)
To force S sub-mergers (regardless of the optimal number) do
proof->SetParameter("PROOF_UseMergers", S)
Revision
31012 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Nov 6 16:08:19 2009 UTC (5 years, 2 months ago) by
ganis
File length: 16655 byte(s)
Diff to
previous 30859
Save the status of a 'proofserv' session into a new file in the 'activesessions' area.
The full path of the new file is
<admin_path>/.xproofd.<port>/activesessions/<user>.<group>.<pid>.status
The status indicates whether the session is idle, running or queued.
The status is updated every 'checkfq' secs (see xpd.proofservmgr; default 30 s).
The status is dumped by the reader thread of TXProofServ and therefore its r/w access
is protected.
This change is needed for PoD optimizations.
Revision
30859 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Oct 24 14:53:07 2009 UTC (5 years, 3 months ago) by
ganis
File length: 16220 byte(s)
Diff to
previous 30174
Patch for improved performance monitoring. The 'Rate Plot' button in the progress
dialog is renamed 'Performance plot' and shows up to 4 plots with the event/sec,
the average read chunck size, the number of active workers and the number of active
PROOF sessions on the cluster, all as a function of processing time.
The read chunck size plot allows to monitor the usage of the cache.
The istantaneous processing rate (event/sec) is now better estimate: a few issues
with the normalizing times have ben solved, removing the artificial structures that
were observed.
The possibility to set a max packet time length is introduced (default 30 s); this
can be changes with the parameter PROOF_MaxPacketTime.
The size of the cache is also taken into account to optimize the use of the cache.
The parameter PROOF_UseParallelUnzip has been introduced to toggle the use of the
parallel unzip (default off for now).
A page describing the new performance plots is under preparation at
http://root.cern.ch/drupal/content/progress-dialog
Revision
30174 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Sep 15 14:24:56 2009 UTC (5 years, 4 months ago) by
ganis
File length: 15704 byte(s)
Diff to
previous 29133
Patch to support automatic creation of a dataset out of files created on the
worker nodes by worker processes. The implementation is an extension of the
functionality of the class TProofOutputFile used for merging via file.
TProofOutputFile:
- Add options to steer the creation of a dataset; the extemded meaning of the
constructor arguments and anew constructor are described in
http://root.cern.ch/drupal/content/handling-large-outputs-root-files
The dataset (i.e. TFileCollection) assembing happens on the master via the
Merge method. Information about the node location of the file is happended
to the relevant TFileInfo as a special "file:" url with option "node=<host-fqdn>:<port>"
This information is needed to facilitate the cleaning of the files once they
are not needed anymore.
The dataset can be just created and returned in the output list, or registered
and/or verified.
TProofPlayerRemote::MergeOutputFiles:
- Correctly transmit the dataset object into the output list
TProofServ:
- Create the new subdirectory 'data' either in the sandbox or followinf ProofServ.DataDir .
- Register and verify the created dataset if required
TProof:
- Implement two new methods:
- ShowData(), to show the content of the assigned 'data' directory
- ClearData(UInt_t what, const char *dsname), to clear all or a subset of the files
in the assigned 'data' directory
Additional changes:
- TProofPlayer:
- Correctly handle the feedback canvas
- Set the drawing attributes
- TProofServ:
- Replace all 'Form("...")' with TString::Format("...") .
- Use the new tag kCollect in debug statements where relevant
- TProof:
- Use the new tag kCollect in debug statements where relevant
Revision
29133 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 22 12:28:50 2009 UTC (5 years, 7 months ago) by
brun
File length: 15265 byte(s)
Diff to
previous 28856
From Gerri:
Rename TProofDataSetManager and TProofDataSetManagerFile to TDataSetManager and TDataSetManagerFile.
These classes are managers of TFileCollection which do not have anything strictly dependent on PROOF.
So far they were only used internally by PROOF, but they are going to be exposed to more public for generic
file collection handling. It is the right time to give them a more proper name.
Revision
28856 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Jun 9 11:49:43 2009 UTC (5 years, 7 months ago) by
ganis
File length: 15275 byte(s)
Diff to
previous 27034
A few optimizations in transferring and merging of the results:
- Output objects are added to the same TMessage until a HWM is reached
(default 1MB; controlled by 'ProofServ.MsgSizeHWM'); this limits the
number of transfers in the case of large numbers of small objects.
- Reasonably small histograms (GetSize() < MsgSizeHWM) are merged in
one-go at the end instead of one-by-one to exploit the better performance
of TH1::Merge on the full list of histos.
- Add possibility to compress the messages; this is controlled by
ProofServ.CompressMessage <compression_level>
The default is still 'no compression' but this will allow to study the
impact of compression.
In addition, a sort of 'progress' counter for merging is now shown on the
client:
root [n] p->Process(...)
...
Mst-0: merging output objects ... / (4 workers still sending)
This asserts socket activity and fixes the timeout problems during long
merging phases reported on the forum.
Revision
27034 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Dec 19 15:30:06 2008 UTC (6 years, 1 month ago) by
ganis
File length: 14898 byte(s)
Diff to
previous 26993
From Jan: prepare for query-enqueing support:
- add new protocol message to resume an enqueued session
- add framework to process enqueued queries after a resume: the processing part on masters
has been moved to a new method ProcessNext which is called also after a resume request.
- remove duplication in the definition of message types for SendCoordinator
(same codes defined in XProofProtocol.h and TXSocket.h)
Revision
26993 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 17 16:01:04 2008 UTC (6 years, 1 month ago) by
rdm
File length: 14793 byte(s)
Diff to
previous 26951
From Gerri:
- Fix a problem with filtering the SvcMsg in TProofMgrLite::GetSessionLogs
- Fix a weird backward-compatibility problem in TProof::SendFile (found
and needed by ALICE)
- Update the release notes
Revision
26951 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Dec 16 12:04:38 2008 UTC (6 years, 1 month ago) by
ganis
File length: 14765 byte(s)
Diff to
previous 26678
- Fix an issue with uploading packages in rapid sequence
- Fix an issue with TProof::Load() in forced compilation mode ('++')
- Fix an issue with grep in log retrieval
- Fix an issue with ownership of the xrootd log file
- Make sure the XrdProofdProofServMgr::fCurrentSessions is always up-to-date
Revision
26678 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Dec 5 14:53:23 2008 UTC (6 years, 1 month ago) by
ganis
File length: 14632 byte(s)
Diff to
previous 26381
Patch to:
- Add the possibility to set upper limits on the virtual memory (changes in TProofPlayer,
TProofServ, TXProofServ and pmain (separate patch)).
If enebled, the session gets firts a warning when it reaches 80% of the limit, and then
processing is stopped whenit exceeds 95% of the limit, sending back the results.
Also, the memory footprint is notified when the session is terminated.
- Make sure the the active valuse in XrdProofWorker are always correct; this was not the
case for dynamic startup as the notification at the end of the query was not done.
This information is crucial or the scheduler.
The way the information is stored in XrdProofdProofServ had to be modified
and a new internal message type (kReleaseWorker) added.
(Changes in several proofd classes, TProof, TXsocket and TXProofServ)
- Fix problem with setting a static upper limit of the sessions that can be started,
and enable this functionality for the dynamic mode (changes in XrdProofSched)
- Remove a deleted worker from all the lists in TProof::MarkBad to avoid later attempts of use.
- Better control the use of the internal pipe for socket readiness notification in TXSocket.
Revision
26381 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Nov 22 17:15:24 2008 UTC (6 years, 2 months ago) by
ganis
File length: 14275 byte(s)
Diff to
previous 26268
- Fixes for the asynchronous mode:
- Fully localize the general handling of the input messages into two new methods
TProof::HandleInputMessage(TMessage *) and TProofServ::HandleSocketInput(TMessage *);
these methods are callable form any place that needs to intercept some messages and
do something specific on them (examples are: TProofServ::GetNextPacket, TXProofServ::Get,
TPacketizerAdaptive::ValidateFiles, ...); this allows to remove several duplications
and to make sure that no message is lost or wrongly dispatched.
- Simplify the recursive infrastructure for TProofServ::HandleSocketInput; in particular,
TProofServ::HandleSocketInputDuringProcess is removed as it is a special case of the
HandleSocketInput, saving duplications.
- Always use a kPROOF_CHECKFILE message in replies to check file operations; failures were
signaled with kPROOF_FATAL which may have some undesired side-effects, depending on the
timing
- Add support for one level of recursivity in TProof::Collect .
- Fix a problem with TProof::Finalize when called with default arguments (on the last query)
- In TProof::SendFile, send to unique workers only in the "cache" option is specified
- Remove the call to Finalize for DrawSelect queries, as it is done via the feedback
mechanisms
- Additions/fixes in test/stressProof.cxx:
- New test for the asynchronous mode;
- Fine-tuning of the progress display in batch mode;
Revision
26268 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 18 16:27:09 2008 UTC (6 years, 2 months ago) by
ganis
File length: 14173 byte(s)
Diff to
previous 26258
- Fix a few problems with the input data machinery (thanks to C. Cheshkov)
- Use the SingeKey option to write out containers
- Send the input data after the call to Begin() to catch up also the objects
possibly added in there
- Fully enable the input data machinery also for PROOF-Lite; this required to
move some calls from TProofServ to TProof to avoid duplications.
- Additions to test/stressProof.C
- Add a test for the input data machinery
- Add some progress info for the tests involving processing
Revision
25969 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Oct 27 12:08:33 2008 UTC (6 years, 2 months ago) by
ganis
File length: 14324 byte(s)
Diff to
previous 25918
- Make the static ErrorHandler public so that it can be used in other places, e.g. TProofPlayerLite
- Add the possibility to control to which I/O stream the ErrorHandler logs should go (default stderr)
Revision
25918 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Oct 22 15:00:04 2008 UTC (6 years, 3 months ago) by
ganis
File length: 14211 byte(s)
Diff to
previous 25841
First version of PROOFLite, a 2-tier realization of PROOF intended for multi-core
machines. In PROOFLite the client start directly the workers; no daemon is required.
Currently this works on Unix only. Some issues with the internal connection still need
to be sorted out on Windows.
To start a session just use
TProof::Open("") or TProof::Open("lite")
From there on everything should be as in normal PROOF. To start a standard PROOF
session (i.e. via daemons) on the localhost use TProof::Open("localhost").
(The default meaning of "" in TProof::Open() is controlled via the rc value
'Proof.LocalDefault').
To run stressProof with PROOFLite pas the argument 'lite':
./stressProof lite
By default, PROOFLite sets the number of workers equal to the number of cores.
The number of workers is controlled by the rc variable 'ProofLite.Workers'; it can
also be set on the fly with the option "workers=<n>", e.g. TProof::Open("lite?workers=2").
This patch also introduces the rc variable 'Proof.Sandbox' to control the local location
of the sandboxes. The default has been changed from "~/proof" to "~/.proof" to avoid
interferences with possible users' working areas. The sandbox contains the directory
'packages' (with the clients packages; used by standard PROOF and PROOF Lite),
and some directories used only by PROOFLite: 'cache' (binary cache) and 'datasets' with
the datasets. The location of these directories is controlled by the rc variables
'Proof.PackageDir', 'Proof.CacheDir' and 'Proof.DataSetDir'.
The working directory for PROOFLite is in the form workdir=<sandbox>/path-to-current-dir where
~/path/to/current/dir is the directory from where the session in launched. The directory
<workdir> contains the session working dirs (in the form 'session-<host>-<time>-<pid>'),
with the symlink 'last-lite-session' pointing to the last started session, and the 'queries'
directory, with the results of queries started from the current directory.
Revision
25720 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Oct 7 10:13:37 2008 UTC (6 years, 3 months ago) by
ganis
File length: 13531 byte(s)
Diff to
previous 25704
Move the code for query-result management into a new class TQueryResultManager,
so that it can be used in other places (e.g. prooflite).
Revision
25704 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Oct 5 21:44:36 2008 UTC (6 years, 3 months ago) by
ganis
File length: 14485 byte(s)
Diff to
previous 25273
TProof:
- Add support for input data objects. These are objects which are available via the input list
but - being potentially large - are distributed to the cluster in optimized mode, i.e. via a
ROOT file copied only to unique nodes. These objects are *not* automatically saved into the
TQueryResult object.
Three new public methods have been added to TProof:
- AddInputData(TObject *, Bool_t = kFALSE) flags an input object as 'data input object'
- ClearInputData(TObject * = 0) clears the list of input data objects
- ClearInputData(const char *name) removes object 'name' from the list of input
data objects
- SetInputDataFile(const char *filepath) specifies the file from where to take the
input data objects; object in the input data
list, if any, are also send over.
New internal methods:
- BroadcastFile(const char *file, Int_t opt, const char *rfile, TList *wrks);
- BroadcastFile(const char *file, Int_t opt, const char *rfile = 0, ESlaves list = kAllUnique);
- void SendInputDataFile();
- Fixes:
- in Process(TFileCollection *, ...), remove the TFileCollection object from the input list
after processing, to avoid destruction problems in case of multiple query submission.
TProofServ:
- Add support for input data objects.
New methods:
- Int_t GetInputData(TList *input);
- Int_t SaveInputData(TQueryResult *qr);
- Int_t SendInputData(TQueryResult *qr);
- Optimize the number of input list instantiations
- Improve readibility of HandleProcess by separating out the code to prepare the dataset (get it
from input list, from dataset manager, etc ...) into a new method AssertDataSet(TDSet *, TList *).
TProofQueryResult:
- add missing protection
Revision
25273 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Aug 27 08:56:06 2008 UTC (6 years, 4 months ago) by
rdm
File length: 14253 byte(s)
Diff to
previous 24739
From Jan:
- A new optional version of PROOF with dynamic worker startup.
It can be enabled by the admin or a user with 'Proof.DynamicStartup'.
A session starts only on the master. When a query processing starts
at the master TXProofServ::GetWorkers() is called. It receives a
list of machines from the scheduler and the workers on the
machines are started. The environment is copied from the master
to the workers. It includes: the include and dynamic library paths,
the set of enabled packages as well as the macros loaded by the user.
- A new method TProof::AddWorkers(TList *workers) was added. It adds
the workers just before the query.
- A packet resubmitting mechanism. When a worker dies all the packets
that it processed are resubmitted.
- In TPacketizerAdaptive: fixing initialization of fgMaxSlaveCnt. By
default it was initialized twice.
Revision
24719 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 9 07:07:25 2008 UTC (6 years, 6 months ago) by
ganis
File length: 14029 byte(s)
Diff to
previous 24695
Patch refactorizing the XrdProofd plugin.
The class XrdProofdProtocol is now in charge only of the operations strctly related
to the XProofd protocol. All auxilliary services have been moved to dedicated service
classes controlled by XrdProofdManager. In particular:
- XrdProofdClientMgr handles the clients (represented by instances of XrdProofdClient)
including login, authentication and access control
- XrdProofdProofServMgr handles the PROOF sessions (represented by instances of
XrdProofdProofServ) including creation, attachment, detachment, destruction and
environment setting
- XrdProofdNetMgr handles connections between instances of XProofd running on different
nodes
- XrdProofdPriorityMgr handles the session priorities
A special effort has been done to get rid of all possible internal dead-lock situations.
Internal actions on clients and sessions are now all asynchronous, governed by internal
pipes.
The new plugin also offers new functionality, among which:
- a XProofd admin area, located under <xrd.admin>/.xproof.<port>, keeps information about
active and terminated sessions, and the minimal state for active clients. This is used
to reguraly check the client and session activity, to cleanup orphalin sessions and
to shutdown inactive client connections.
In particular this allows a more solid implementation of Reset, which now exists in two
flavours: 'soft', TProof::Reset(<masterurl>), (default) asks the sessions to terminate gently;
'hard', TProof::Reset(<masterurl>,kTRUE) schedules all sessions for forced termination.
- support for automatic attempts for reconnection in the case the daemon restarts.
This allows to reconfigure the plugin by xrootd restart w/o affecting the running
sessions.
- support the definition of workers via config file directive, getting de facto rid of
the proof.conf file.
- domain+level control of printout message; the format has been improved: in particular
all information messages contain now the tag 'xpd-I' and all error messages the tag
'xpd-E', so that they can easily be grepped out from the log file.
The Wiki 'XrdProofdDirectives Directives' page has been updated with the new directives.
Revision
24589 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 27 14:17:08 2008 UTC (6 years, 6 months ago) by
ganis
File length: 14227 byte(s)
Diff to
previous 24574
- Add support for an external definition of the session tag: this will be needed
by external PROOF session launchers, when available
- Check also the env ROOTRCFILE for 'rootrc' configuration.
Revision
24574 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 26 17:06:03 2008 UTC (6 years, 6 months ago) by
ganis
File length: 14086 byte(s)
Diff to
previous 23075
Implement selective sending of logs from workers to master to avoid duplicating
too many text lines on the master log. Logs are now sent only after Exec, Print
requests and in case an error (level >= kError) occured. Of course, the full logs
can always be retrieved via TProofMgr::GetSessionLogs .
Also, remove TXProofServ::SendLogFile, which was reduced to be a duplication of
TProofServ::SendLogFile .
Revision
23075 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 9 09:10:25 2008 UTC (6 years, 9 months ago) by
ganis
File length: 14003 byte(s)
Diff to
previous 22635
From Jan-Fiete Grosse-Oetringhaus (ALICE) and me.
Patch introducing the improved dataset manager developed by ALICE.
The client side interface is basically unchanged, the biggest change is the renaming
of TProof::CreateDataSet to TProof::RegisterDataSet.
On the server side the relevant code has ben extracted from TProofServ and hidden
behind an inteface class TProofDataSetManager describing the functionality needed by
TProofServ, and an implementation class TProofDataSetManagerFile handling dataset
meta data saved into ROOT files. TProofDataSetManagerFile is instantiated via the
plugin manager and by default it handles the <sand_box>/datasets area.
This separation is needed to allow loading implementation based on different backends;
for example, ATLAS foresees to use a MySQL-based backend.
A TWiki page with information and examples in on preparation.
Revision
20758 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Nov 11 21:29:49 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proof/inc/TProofServ.h
File length: 13914 byte(s)
Diff to
previous 20685
Several small fixes and few additions imported from 'branches/dev/proof':
- XrdProofdProtocol:
+ Fix a problem in CleanuProofServ sometimes preventing proper cleanup
+ In ReadBuffer, do not trigger an error on empty files
+ Add missing protection against an empty string while setting the ROOT version
- XrdProofdManager:
+ Define the list of unique nodes to avoid broadcasting twice to the same node
- XrdProofWorker: improve matching technique
- TProof: add setter and getter for Int_t parameter
- TProofServ:
+ do not flush the log file in SendAsynMessage: this recent change
made the logs "disappear" form the ProgressDialog
+ Activate control on the max number of query result kept
- TPacketizerUnit: fix a problem with filling the histof for PerfStats
- TProofPlayer: call Begin() also during selecter re-initialization
Revision
20682 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 6 15:51:59 2007 UTC (7 years, 2 months ago) by
ganis
Original Path:
trunk/proof/inc/TProofServ.h
File length: 13719 byte(s)
Diff to
previous 19826
Import branches/dev/proof r20654
Summary:
+ Improvements
- Add support for SSH SOCKS4 tunnelling; the local port for the tunnel can be specified
in the master URL, e.g. TProof::Open("master/?tunnel:8000")
- Add the possibility to plot the estimated instantaneous rate
- Add "PROOF_ForceLocal" parameter to the !TPacketizerAdaptive; if set to 1, all the data
are processed locally.
- Add support for remote grep functionality while retrieving logs (needed by the forthcoming
memory monitor)
+ Bug fixes
- Several small fixes to revive the multi-master mode.
- XrdProofdProtocol:
- add missing lock to the client instance in SendMsg to avoid screwing up requests from
workers on the same machine
- lock the mutex of the requester when setting prorities
- add notification during Reset
- fix problem with the detection of the 'allow' directive
- fix problem with the parsing of the return value from XrdProofServProxy::TerminateProofServ()
- always use the effective user to retrieve info fom another server
(XrdProofdManager::GetProofConn is now used)
- fix possible dead-locks from debug notifications done after hard-killing a session
- re-enable the garbage collector thread of the connection manager in XrdProofConn to
fix a problem with closing physical connections;
- fix a problem with !CleanupProofServ in the case of a non-privileged daemon running
in multi-user mode
- introduce a timeout when waiting for the startup of a 'proofserv'.
- XrdProofConn: init mutex in the ctor; lock in SendRecv
- XrdProofSched: add support for using the priorities defined in the group manager to define
the number of workers for sessions
- TProof:
- Broadcast priorities to unique nodes only
- timeout after 5 mins the initial Collect to avoid clients getting stuck at this stage
- add support for generic timeout in Collect (disabled by default)
- fix a problem with SendFile.
- TXProofServ: add a call to !TProof::InterruptCurrentMonitor() in Terminate() to stop
infinite loops in Collect
- TXSocket:
- Implement a flag to interrupt a TXSocket while waiting for messages
- Split the session creation timeout in 4 attempts: the total timeout is the same but
it may circumvent occasional forking problems.
- XROOTD:
- fix a potential (possibly accademic) memory leak in the client
- fix a access permission problem with Kerberos ticket forwarding
- fix bug preventing 'locate' to work properly
- re-enable optimized 'locate'
Revision
19145 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 22 17:16:35 2007 UTC (7 years, 7 months ago) by
ganis
Original Path:
trunk/proof/inc/TProofServ.h
File length: 13633 byte(s)
Diff to
previous 19099
Add support for global package directories. The colon-separated list of directories
to be searched for is defined by the directive
Proof.GlobalPackageDirs globaldir1[:globaldir2[:globaldir3[:...]]]
which can be included in the xrootd config file via the xpd.putrc directive
xpd.putrc Proof.GlobalPackageDirs globaldir1[:globaldir2[:globaldir3[:...]]]
The directories must be readable by the user (a check is made in TProofServ::Setup).
Packages are searched first in the user local cache and then in the global
directories (in the given order, so that the first occurence is used).
The content of the global package directories is listed by TProof::ShowPackages().
The functionality is available also locally on the client. The Proof.GlobalPackageDirs
has to be specified in the users $HOME/.rootrc .
Global packages can be directly enabled via TProof::EnablePackage(<package_name>).
Revision
19099 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Jun 21 08:47:42 2007 UTC (7 years, 7 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 13523 byte(s)
Diff to
previous 19093
added new data member fGroupPriority. This priority is obtained by
GetPriority() which makes an SQL query in the proofpriority table, which
has the format:
CREATE TABLE proofpriority (
id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
group VARCHAR(32) NOT NULL,
priority INT
)
This prority is then forwarded to the xrootd on the workers.
Revision
18860 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed May 23 09:10:19 2007 UTC (7 years, 8 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 13128 byte(s)
Diff to
previous 18349
From Gerri:
- Add method SendAsynMessage(const char *, Bool_t lfeed) to send asynchronous
messages to the client; this allows the repetition of the same code
in many places.
- Add new member fPrefix which is initialized in the ctor to be, for example,
"master-0" and used everywhere, included the Error handler: this allows
to avoid recreating the same prefix in many places in the code.
- Add method FlushLogFile() allowing to hide from the client messages which may
important for debugging, but not so relevant for normal operations.
- Set/Unset the kWriteV3 bit in TDSet when relevant (i.e. when the client
protocol is 12 or less).
Revision
18349 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Mar 19 14:43:26 2007 UTC (7 years, 10 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 12892 byte(s)
Diff to
previous 18320
From Gerri:
- TProof::Load(const char *macro, Bool_t notOnClient) where 'macro' is the
name of a macro or the implementation file of a class (in which case the
corresponding header should have .h or .hh extension). If notOnClient
is kTRUE (default is kFALSE) the class/macro is loaded only
on master and workers.
- Re-usage of valid binaries from macros or selectors (they are cached under
<SandBox>/cache), this allows to save a few seconds while running a query
with an unchanged and compiled selector.
- Exports ROOTINCLUDEPATH with the current content of gSystem->GetIncludePath()
just before executing BUILD.sh so that the path can be used for build
purposes (bug #18079)
Revision
17680 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Feb 4 17:39:44 2007 UTC (7 years, 11 months ago) by
brun
Original Path:
trunk/proof/inc/TProofServ.h
File length: 12293 byte(s)
Diff to
previous 17562
-Move typedefs previously in Htypes.h and Gtypes.h to Rtypes.h
-Move enum EColorType to Rtypes.h
-Move enum EMarkerStyle to TAttMarker
-Move enum ELineStyle to TAttLine
-Replace all references to Htypes.h or Gtypes.h by Rtypes.h
Revision
17562 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jan 29 15:11:11 2007 UTC (7 years, 11 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 12340 byte(s)
Diff to
previous 16955
Frrom Gerri:
There is a new classes TProofServLogHandler allowing to redirect
the output to the client in real-time. It has two main constructors,
one taking an external command (like BUILD.sh) and opening a pipe,
the other taking a pointer to a open file. TProofServ uses the class
via the guard class TProofServLogHandlerGuard to better localize the
usage. This redirection is not always activated. For the time being it
is active for Process and Cache actions,and it can be deactivated via
TProof::SetRealTimeLog(kFALSE)
(for BUILD.sh - executed externally - is always ON).
In particular, with the patch applied, the outputs from BUILD.shi
will scroll down on the screen, and clicking on "Show logs" in the
dialog box during process shows the messages and allows to see what
the session is doing: this is quite nice, because getting
an empty window from "Show Logs" is frustrating, in my opinion.
So, all this fixes the bug/request #16845.
The patch also fixes bug #22555 (hanging while enabling a package
without BUILD.sh) and several other small bugs (a couple of unitialized
variables) or typos.
It also cleanups the API a bit (we had SetQueryType and SetQueryMode
doing exactly the same thing ...).
Revision
16955 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Nov 27 14:14:24 2006 UTC (8 years, 1 month ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 10635 byte(s)
Diff to
previous 16557
From Gerri:
This is the patch to move TDSet and TChainProof in 'proof' and remove
any dependence on PROOF in 'tree'. I have renamed TChainProof as TProofChain,
for consistency. TProofChain is now deriving from TChain and overrides only
the required methods. It is loaded via PluginManager by TChain in SetProof.
TQueryResult stores now all the input-related objects in fInputList. There
is a new method
TObject *TQueryResult::GetInputObject(const char *classname)
to retrieve the first instance of a class from the input list, so for the
TDSet:
TQueryResult *qr;
TDSet *dset = (TDSet *) qr->GetInputObject("TDSet")
The changes in TSessionViewer are needed to adapt to this new way of
accessing TDSets.
Revision
16557 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Oct 19 12:38:07 2006 UTC (8 years, 3 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 10622 byte(s)
Diff to
previous 16414
From Gerri:
Major set of robustness fixes, improving error reporting and reducing the
startup failures especially observed on AMD64. Summary of changes:
- XrdProofdProtocol.cxx, .h
- Remove zombie treatment: done by XrdScheduler::Reaper()
- Check the modification date of the PROOF config file so that changes
can be picked up on the fly
- Fix problem with the definition of the image
- Improved tracing (see etc/proof/xpd.cf.sample)
- Several bug fixes
- XrdProofConn
- fix problem with marshalling in case of retries.
- TProofServ, TXProofServ
- Add return codes to CreateServer() and Setup() for proper termination
in case of errors
- TProof
- Make sure that TXProofMgr is the last one in the list of sockets for proper
cleaning
Revision
15857 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jul 26 14:28:59 2006 UTC (8 years, 6 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 10638 byte(s)
Diff to
previous 15656
From Gerri:
TVirtualProof.h
- Add abstract methods methods to communicate the data set status
(signal + sender)
pmain.cxx
- Add call to Exit(0). T(X)ProofServ::Terminate() just interrupt the event
loop (insteadof exiting) to avoid problems when the terminate is run in
a separate thread (like in xproofd).
proof/inc/TProof.h
- Add methods methods to communicate the data set status (signal + sender)
- Increase protocol number to 11 (new merging technology)
proof/inc/TProofPlayer.h, .cxx
- Implement new merging technology for the output list to reduce memory
consumption: add methods to directly merge received objects in the
output list
- Polish cleaning of the various temporary lists and objects
TProofServ.h, .cxx
- Implement new merging technology for the output list to reduce memory
consumption: objects are sent one-by-one
- Do not scan for previous query results at Setup (do it when the client
requests for it first); thsi allows to speed up start-up when a large number
of queries is present and also allos to avoid warning due dictionaries
not yet loaded.
- Add method for a global cleanup of old query results.
- Keep in memory only meta information about processed queries (i.e. not the
output list) to avoid large memory consumption in case of large
output lists
- In Terminate, change the call to Exit(0) to ExitLoop() ; Exit(0) is
called in pmain.cxx .
TProof.cxx
- Implement new merging technology for the output list to reduce memory
consumption: objects are received one-by-one and directly merged in the
output list
- Add protection in parallel startup against invelid slaves.
- Remove check on directory existence in UploadDataSet.
- Add methods to handle notifications from data set verification
- Fix inconsistency with notification in activation/deactivation
of non-existing workers
TXProofServ.cxx
- In Terminate, change the call to Exit(0) to ExitLoop() ; Exit(0) is
called in pmain.cxx.
TQueryResult.cxx
- Make a owned copy of the input list (instead of just storing the pointer) to
allow proper cleaning.
Revision
15656 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 3 09:33:50 2006 UTC (8 years, 6 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 10600 byte(s)
Diff to
previous 15524
From Jan:
This patch adds following issues:
- public dataset management
- separation of directories for different users
- adding TProofServ::HandleDataSets method
- spliting communication in UploadDataSet in two parts (checking name
and saving)
- I also added these methods:
virtual Int_t UploadDataSet(const char *dataset,
TList *files,
const char *dest = 0,
Int_t opt = kAskUser,
TList *skippedFiles = 0) = 0;
virtual Int_t CreateDataSet(const char *dataset,
TList *files,
Int_t opt = kAskUser) = 0;
- and added arguments to
TList *GetDataSets(const char *dir = 0);
void ShowDataSets(const char *dir = 0);
Revision
15507 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 21 16:18:27 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 10425 byte(s)
Diff to
previous 15338
From Gerri:
The patch implements the following new functionality:
- Shutdown of idle sessions when quitting ROOT; if not idle the session
is shutdown as soon as processing terminates; it is possible to delay
the automatic shutdown by a tunable amount of time via a new directive
in the xrootd config file 'xpd.shutdown' (see xpd.cf.example).
- New static method TProof::Reset(const char *url) to hard-cleanup an
user area (implemented for Xproofd only); this implied quite some
additions in XrdProofdProtocol.
- Possibility to temporary remove / re-insert a worker from the active list
TProof::DeactivateWorker(const char *ord)
TProof::ActivateWorker(const char *ord)
with 'ord' the ordinal numer, e.g. "0.3" .
- A default url for the disk pool
const char *TVirtualProof::GetDataPoolUrl()
It is set by default to a string send back by the master, but it can
be changed via
void TVirtualProof::SetDataPoolUrl(const char *url)
Jan should then adapt his code to use this as default.
- Posibility to timeout Collect; for now this is only used at forced
termination.
The patch includes many small changes and bug fixes. One of the most important
is in TXNetSystem (a bug preventing correct treatment of multiple request
to the same redirector).
Revision
15338 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jun 5 22:51:14 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 9894 byte(s)
Diff to
previous 15320
From Gerri:
base/inc/TVirtualProof.h, proof/inc/TProof.h, proof/src/TProof.cxx
proof/src/TProofProgressDialog.cxx:
- fix a problem with the progress dialog when detaching a session:
a new signal CloseProgressDialog() is introduced, as well as the
possibility to empty the list of submitted queries at once via
TProof::Remove().
proofx/inc/TXHandler.h, .cxx:
- add generic argument to HandleInput() and HandleError().
proofx/inc/TXProofMgr.h, .cxx:
- move to new signature for HandleInput(), HandleError().
proof/inc/TSlave.h:
- add method to flush underlying socket.
proofx/inc/TXSlave.h, .cxx:
- move to new signature for HandleInput(), HandleError().
- implement new method to flush underlying socket.
proofx/inc/TXSocket.h, .cxx:
- Add method to flush socket.
- Handle interrupt messages in the reader thread.
- Remove interrupt semaphore (not needed any more).
- Remove timeout argument from GetInterrupt (not needed any more).
proofx/inc/TXProofServ.h, .cxx:
- move to new signature for HandleInput(), HandleError().
- implement interrupt handling in HandleInput().
- implement dedicated version of LockSession to deal with new format
of session tag (this is a bug fix).
proofd/inc/XrdProofConn.h, .cxx:
- Add mutex protection in the SendRecv method (it can be called from both
the reader and main threads).
proofd/src/XrdProofdProtocol.cxx:
- use an asynchronous message (instead of a signal) for interrupts.
proofx/inc/LinkDef.h, TXSocketHandler.h, .cxx, TXUnixSocket.cxx:
- add TXSocketHandler in the dictionary.
- cosmetics.
Revision
15320 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 2 23:38:20 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 9894 byte(s)
Diff to
previous 15312
From Gerri:
- Get rid of the env to communicate the file descriptor to TProofServ.
Also, get rid of Reads. I had to add an argument to the TProofServ
(and TXProofServ) constructor with the 'FILE *' of the open log file.
Using a global required to link with libProof(x) which we droped recently.
- Drop any (for the time being) unused AFS stuff from TProofServ (this
is postponed).
Revision
15312 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Jun 2 15:14:35 2006 UTC (8 years, 7 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 10037 byte(s)
Diff to
previous 14752
From Gerri:
large PROOF patch. Main issues addressed:
- Output redirection to log file at 'proofserv' startup (instead of
after Setup())
- Log messages activated at session creation
- Session sandbox restructuring in such a way that the unique session
tag, created by the top master, uniquely identifies the session on
all nodes, for example on top master:
<User_Sandbox>/session-<top_master_session_tag>/
master-0-<top_master_session_tag>/master-0-<top_master_session_tag>.log
On worker 0.0
<User_Sandbox>/session-<top_master_session_tag>/
worker-0.0-<worker_0.0_session_tag>/worker-0.0-<worker_0.0_session_tag>.log
- Automatic creation of the dataset dir
- Improved handling of error conditions
- Reorganization of TProofServ and XrdProofdProtocol
- Several bug fixes, in particular in XrdProofdProtocol
Revision
14752 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Apr 19 10:57:44 2006 UTC (8 years, 9 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 9432 byte(s)
Diff to
previous 14356
From Gerri:
This is the big patch.
Comments:
- Client side:
- Remove use of plugin manager to avoid unwanted locks
- Improved treatment of error conditions
- Server side:
- Several new functionalities described in the release notes
Revision
13616 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Dec 10 16:51:57 2005 UTC (9 years, 1 month ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 9117 byte(s)
Diff to
previous 12733
From Gerri:
introduction of major new PROOF features. Main features:
- PROOF session manager which allows disconnect/reconnect from running
sessions
- use xrootd/xproofd as connection daemon
Much more extensive description in release notes.
Revision
12722 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Sep 16 08:48:39 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 9013 byte(s)
Diff to
previous 12561
From Gerri:
Big patch supporting query queueing on the PROOF master, interogation and
interaction with the queue. Retrieval, finalization and archival of query
results and multiple PROOF sessions. Some other fixes:
- TMacro.h, .cxx
add method Checksum to get the MD5 of the current context
(needed for optimized selector reinitialization)
- Changes needed to create and fill the list of data sets (TROOT,
TDSet, TChain, TChainProof)
- A new method TSelector::IsStandardDraw() to check if a selector
is for standard draw actions
- TEventIter: fix for the missing last update (see above)
- Few protections in TPacketizers destructors
- Fix for code conventions in TPerfStats (the other will follow)
- Occasional missing last update of the progress bar (systematically
reproduced with SetParallel(1)): all this part is quite involved
and I have found a solution which does not harm but perhaps is not
the most elegant.
- SetParallel(0): the progress bar was not working at all in such a
case, since no packetizer is involved. Still it is useful to see
that things are not stuck, so I have added some progress messages
there too. The problem is that I have not found a way to get the total
number of entries from a data set without doing all the machinery
done in the packetizer.
Revision
12561 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Aug 30 10:25:29 2005 UTC (9 years, 4 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 6922 byte(s)
Diff to
previous 12477
From Gerri:
This is the patch to add support for asynchronous queries.
The principal modifications are in TProof and TProofPlayer.
In TProof, I have separated the parsing of the received messages
from the selection of the socket (new method CollectInputFrom()):
the new method is called directly by TProofInputHandler::Notify()
in the case of asynchronous mode.
Also in TProof I have added a method to redirect the logs to a
temporary file, to avoid having them on the main window while doing
something else. The method is TProof::RedirectLog(), it uses freopen,
fdopen and dup2, and works well back and forth.
The content of the temp file can be displayed into a log window
(new GUI class TProofProgressLog), via a new button in the
progress dialog ("Show Logs").
In TProofPlayer I have created a new method Finalize() with the
final part of Process(): this is necessary to be able to finalize
the queries processed asychronously.
There are several many other small changes and fixes: a couple
of non-backward comaptibilities introduced with the previous
patch, many small issues in the GUI (the stop and abort buttons
were never disabled ...).
Also, in view of multi-sessions, it does not make much sense to
have one progress dialog per query: one per session is enough;
I have added a method ResetProgressDialog() to reset it when a
new query is started.
Finally, as anticipation of the next patch, this patch includes
support for a list of queries created by the master and described
by the new class TProofQuery.
The list is displayed by TProof::ShowQueries().
Revision
12477 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Aug 15 15:57:18 2005 UTC (9 years, 5 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 6741 byte(s)
Diff to
previous 12337
From Gerri + some mods by me:
- This patch gets rid of TFTP, slightly modifying SendFile and CheckFile.
- The factorization of the technology dependent parts of TProof, TSlave
and TProofServ. In particular, I have moved the technicalities of Ping
and Interrupt from TProof to two new virtual methods in TSlave, which
will be overwritten in TXSlave.
In TSlave I have introduced the virtual method Init which will be
overwritten by TXSlave and is able to act on an open connection for
optimal fallback a la TXNetFile.
Slaves are created by TProof via the static method TSlave::Create
which uses the plugin manager to detect the presence of TXSlave.
In TProofServ I have made the technology-dependent methods
virtual (Setup, HandleSocketInput, HandleUrgentData, HandleSigPipe,
HandleSocketInputDuringProcess).
Revision
12337 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 18 16:20:53 2005 UTC (9 years, 6 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 6701 byte(s)
Diff to
previous 12127
From Gerri:
Big patch restructuring the authentication code so libCore does not depend
on libssl etc. anymore. Isolate the current authentication code behind a
plugin library to be loaded on demand.
Setup of a framework able to manage both sets of authentication modules
(the current one and the one based on xrdsec, coming soon).
Cleanup all direct reference to authentication in TSlave and TProofServ,
allowing for significant simplification for the forthcoming changes
in PROOF for XPD.
New module: auth
New files: base/inc/TVirtualAuth.h
auth/Module.Mk
auth/inc/LinkDefRoot.h
auth/inc/TRootAuth.h auth/inc/TRootSecContext.h
auth/src/TRootAuth.cxx auth/src/TRootSecContext.cxx
Moved files:
net/inc -> auth/inc : TAuthenticate.h THostAuth.h DaemonUtils.h AuthConst.h
net/src -> auth/src : TAuthenticate.cxx THostAuth.cxx DaemonUtils.cxx
Revision
12123 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 22 20:18:12 2005 UTC (9 years, 7 months ago) by
brun
Original Path:
trunk/proof/inc/TProofServ.h
File length: 6625 byte(s)
Diff to
previous 11717
From Constantin Loizides
This patch implements:
- decentralized, automatic mutex initialization,
see R__LOCKGUARD2 in TVirtualMutex
- PROOF parallel startup fixes
- more thread protection (in base, cont, meta, rest to be done)
- cleanups
From Eddy Offermann:
TString::Atoi and Atof are made const.
Revision
11047 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 7 18:02:37 2005 UTC (9 years, 11 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 6376 byte(s)
Diff to
previous 9182
From Maarten and Kristjan Gulbrandsen:
Here is the patch with the restructured PROOF code. It parameterizes
PROOF startup, currently implementing "Standard", Condor, MLM and PEAC.
It can easily be extended with other "TProof" classes, e.g. AliEn Proof.
Or what ever next Grid implementation or scheduler comes along.
The MLM, Multi Level Master, code uses a basic packetizer, based on a
MSD (massstorage domain) tag. In MLM mode we have multi-tier client
server setup, like:
client -> super master -> sub master 0 -> slave[0..N]
...
-> sub master M -> slave[0..N]
It also contains a large number of smaller fixes. Credit for most of
the work goes to Kris!
We have tested this on multipe platforms (RH7.3, RH9, SL3) and in
multiple configurations. Should be stable.
Revision
9182 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Jun 13 16:26:36 2004 UTC (10 years, 7 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 6035 byte(s)
Diff to
previous 8561
From Maarten and Kristjan Gulbrandsen:
PROOF and file I/O performance monitoring framework. The TVirtualPerfStats
abstract class allows adding monitoring events in base classes (TFile and
TNetfile). The performance monitoring info allows detailed study of PROOF
performance (which files accessed by which slaves, latency, read/write times,
packet size, etc.). The result is send back to the client in the form of a
memory based TTree.
Remove TProofStats.h and TProofStats.cxx, .d, .o from proof/inc and src.
Revision
8561 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sun Apr 11 18:18:01 2004 UTC (10 years, 9 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 5982 byte(s)
Diff to
previous 8239
From Maarten and me:
in proofd make a dup of the socket away from the stdin, stdout and stderr
standard descriptors to avoid that printfs go to the socket (before this
was done in proofserv itself but this was too late in case there were
prints in the global object ctors.
Also provide TProofServ::GetSessionDir() method.
Also set PROOF_SANDBOX env variable (to SessionDir), this env var can
be used in the package setup scripts.
Revision
8239 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Feb 19 00:11:19 2004 UTC (10 years, 11 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 5919 byte(s)
Diff to
previous 7534
Big authentication patch by Gerri:
General purpose:
Add support for 'authenticated' sockets on client side
and prepare for support of server authenticated sockets.
Main changes:
o TSocket, TPsocket:
Added support of 'authenticated' sockets: authentication
is delegated to TSocket by calling a static method
CreateAuthSocket() returning a TSocket*.
TSocket is now the only class calling TAuthenticate.
o TFTP, TNetFile, TSlave:
modified to cope with new socket authentication scheme.
o TAuthDetails replaced by TSecContext containing:
- local authentication info
- expiration time
- pointer to a security context for the method (if any)
- information for cleanup (via TSecContextCleanup)
o A list of valid TSecContext has been added to TROOT,
accessible via gROOT->GetListOfSecContexts().
o THostAuth rewritten using arrays to avoid multiple use
of 'new'; functionality extended by the addition of
several new methods.
o TAuthenticate:
- Loading of information from .rootauthrc has been optimized;
this allowed to simplify method signatures and drop several
methods. Also collection and transmission of authentication
information from TSlave to TProofServ gets simplified by the
new scheme.
- Added support for server dependent directives in .rootauthrc
- Added check for changes in .rootauthrc each time a new
TAuthenticate is instantiated, so that changes in the file
are picked-up interactively.
- Checking for an existing TSecContext (previously TAuthDetails)
has been improved
- Improved diagnostics for SshAuth
- Enlarged support for authentication method indication in protocol:
UsrPwd: rootup/proofup
SRP: roots/proofs (already supported)
Krb5: rootk/proofk (already supported)
Globus: rootg/proofg
SSH: rootsh/proofsh
UigGid: rootug/proofug
o GlobusAuth.cxx, Krb5Auth.cxx, SRPAuth.cxx
- Adapted to new class TSecContext
- Added support for improved search for reusable TSecContext
- Drop globals used to store established security contexts
(saved in TSecContext) and related methods
o Automatic creation of <RootDir>/etc/system.rootauthrc from
compilation flags; this has priority on old envs
'Rootd.Authentication' and 'Proofd.Authentication' and is
superseded by $HOME/.rootauthrc
o Automatic creation of <RootDir>/etc/system.authdaemonrc from
compilation flags; this is superseded by $HOME/.rootauthrc
o rpdutils:
- several almost identical methods moved in from rootd/proofd
- several new methods to simplify manipulation of the
authentication tab file
- moved (and improved) check for running of sshd in RpdSshAuth,
performed only in case of failure (to avoid spurious messages
printed by sshd in /var/log/messages)
o proofd.cxx, rootd.cxx
- increased protocol version
- all initialization stuff (included authentication and
login) done via RpdInitSession
- postpone opening of parallels sockets after authentication
(to solve problem of Kerberos/Globus authentication with
parallel sockets)(rootd)
- Fixed problem with kerberos and globus authentication in rootd
with parallel sockets.
Revision
7534 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Nov 7 03:29:42 2003 UTC (11 years, 2 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 6033 byte(s)
Diff to
previous 7409
From Gerri:
Index: README/README.AUTH
o Update explaining tutorial use
Index: base/src/TFile.cxx
o ::Open(): call TNetFile also if same host but different user
(to access files protected by password)
Index: globusauth/src/GlobusAuth.cxx
o Aestethic changes (shortening of a few lines)
Index: krb5auth/src/Krb5Auth.cxx
o Simplify two debug printouts
o Change debug print to stderr in Info(...; add gDebug control.
Index: net/inc/THostAuth.h
o Modify default ctor
o Add method HasMethod to test method availability
Index: net/src/TAuthenticate.cxx
o make sure that if fgUser (and nothing else) is set, this is
the default user name
o Make sure that requiring roots/rootk uses correctly details
found for those methods in .rootauthrc
o Fix return of GetPromptUser()
o Fix problem with wild cards
o Add check for wildcard host names in method GetHostAuth
o Remove debug printout in method ReadAuthRc
Index: net/src/THostAuth.cxx
o Modify default ctor
o Add method HaveMethod to test method availability
o Aestethic changes (shortening of a few lines)
Index: net/src/TNetFile.cxx
o Remove precompiler unused #if's in method ConnectServer
Index: proof/inc/TProofServ.h
o Remove method ReadProofAuth, add method RecvHostAuth
(To receive directly the authentication info in ProofServ
instead of reading it from temporary files
/usr/tmp/proofserv.<id>)
Index: proof/src/TProofServ.cxx
o All mods needed to remove method ReadProofAuth and add
method RecvHostAuth (see above).
Modified methods: Setup, CollectAuthInfo .
o Fix bug in CollectAuthInfo switching ReUse off by default
for method UsrPwd
o Removing automatic choice of UidGid for local slaves
o Setting order of default UidGid authentication to last
o Aestethic changes in CheckAuth
Index: proof/src/TSlave.cxx
o transmit passwd for UsrPwd and SRP authentication also to slaves
o All mods needed to send auth info to ProofServ instead of
proofd (see above)
Index: proofd/src/proofd.cxx
o Fix compiling problem with namespaces for Kerberos
o Remove redundant 'access' calls
o Authenticate: do not recive auth info (done in ProofServ)
o ProofExec: send TmpDir to proofserv (for key location)
o Define gSystemdaemonc as $ROOTETCDIR/system.rootdaemonrc
Index: rootd/src/rootd.cxx
o Define gSystemdaemonc as $ROOTETCDIR/system.rootdaemonrc
Index: rpdutils/inc/rpdp.h
o Rename gAuthAllow as gSystemDaemonRc
o Rename kDaemonAccess as kDaemonRc
Index: rpdutils/src/rpdutils.cxx
o Fix compiling problem with namespaces for Kerberos
o Define kDaemonRc as .rootdaemonrc (it was daemon.access)
o Add support for $HOME/.rootdaemonrc
o Add support for daemon dependent access rules
o Aestethic changes in CheckAuth (drop commented lines)
Index: srputils/src/SRPAuth.cxx
o set static flags only when authentication is successful
Index: tutorials/TestAuth.C
o Test using TFTP instead of TNetFile (faster and runs also
with local host fqdn)
o Add test for kerberos
Revision
7409 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Oct 7 21:09:55 2003 UTC (11 years, 3 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 6034 byte(s)
Diff to
previous 7406
From Gerri:
Index: README/README.AUTH
o Updated
Index: globusauth/src/GlobusAuth.cxx
o Drop use of environment variables (DEFAULTUSER,AUTHREUSE,PROMPTUSER)
Index: krb5auth/src/Krb5Auth.cxx
o Drop use of environment variables (DEFAULTUSER,AUTHREUSE,PROMPTUSER)
Index: net/inc/TAuthenticate.h
Index: net/src/TAuthenticate.cxx
o Drop use of environment variables (DEFAULTUSER,AUTHREUSE,PROMPTUSER)
o Fix problem with defaults
o Fix problem with method ReadAuthRc when reading an empty .rootauthrc
o Fix problem with method ReadAuthRc when updating information for an
existing THostAuth
Index: proof/inc/TProof.h
o Propagation of new fSRPPwd flag
Index: proof/inc/TProofServ.h
o Propagation of new fSRPPwd flag
Index: proof/src/TProofServ.cxx
o Propagation of new fSRPPwd flag
Index: proof/src/TSlave.cxx
o Propagation of new fSRPPwd flag
Index: rpdutils/src/rpdutils.cxx
o Fix small problem with uninitialized variables to avoid
compiler complains (from Ioannis Papadopoulos).
Index: srputils/src/SRPAuth.cxx
o Drop use of environment variables (DEFAULTUSER,AUTHREUSE,PROMPTUSER)
Revision
7406 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Oct 7 14:03:03 2003 UTC (11 years, 3 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 5966 byte(s)
Diff to
previous 7262
From Gerri:
clib/inc/rsalib.h, clib/src/rsalib.c
o Random seed initialization moved from rsalib.c to in rootd.cxx,
proofd.cxx and TAuthenticate.
o Introduced counter to avoid infinite loops during key generation
etc/rpdauth.allow
o Introduced default with all authentication methods allowed in
rpdauth.allow
globusauth/src/GlobusAuth.cxx
o Secure RSA public transmission to servers (receiving a server
key)
o Added test on Argc() before use of the Argv() arguments
o Use isatty to check if interactive session
krb5auth/src/Krb5Auth.cxx
o Secure RSA public transmission to servers (receiving a server
key)
o Added test on Argc() before use of the Argv() arguments
o Use isatty to check if interactive session
net/inc/TAuthenticate.h, net/src/TAuthenticate.cxx
o Secure RSA public transmission to servers (receiving a server
key)
o Added possibility for a "default" THostAuth instantiation valid
for all hosts if anything more specific is not found.
Used by TProofServ/TAuthenticate for host without specific
authentication directives in proof.conf .
o Random seed initialization moved from rsalib.c to in rootd.cxx,
proofd.cxx and TAuthenticate.
o Introduced counter to avoid infinite loops during key generation
o Fixed potential problem with SSH master-to-slave authentication;
force failure if, for some reason, the slave is not accepting
the standard key files (depending on the user ssh config
directives, this could cause interactive prompt for password
on the master, therefore hanging the system).
Specific directives to avoid prompting are passed via command
line with the '-o' ssh option.
o Fixed problem with forming the message used to notify an SSH
failure
o Use mkstemp instead of tmpnam
o Use isatty to check if interactive session
o Avoid creating duplicates of TAuthDetails instantiation for
offset = -1
o Fixed a few inconsistency of debug levels error messages after
failed authentications
o Fixed a few inconsistency between authentication method names
and ROOT environment variable names
o Fixed problem with null passwd sending in ClearAuth
o Fixed problem with method attempting order during negotiation
o Add possibility to avoid check on localhost for testing purposes
(in method GetAuthMeth)
net/src/THostAuth.cxx
o Added possibility for a "default" THostAuth instantiation valid
for all hosts if anything more specific is not found.
Used by TProofServ/TAuthenticate for host without specific
authentication directives in proof.conf .
proof/inc/TProof.h, proof/src/TProof.cxx
o Moved collection of Authentication information for slaves from
TProof to TProofServ
proof/inc/TProofServ.h, proof/src/TProofServ.cxx
o Improved transmission of RSA public key between TSlave and
TProofServ
o Moved collection of Authentication information for slaves
from TProof to TProofServ
o Added test on Argc() before use of the Argv() arguments
proof/src/TSlave.cxx
o Improved transmission of RSA public key between TSlave and
TProofServ.
Added possibility to securely forward the SRP password to
the master (ROOT environment variable: Proof.SendSRPPwd,
default 0 - off).
o Removed checks on Proofd.Authentication in TSlave; security
stuff is setup in TProofServ and used in TAuthenticate
proofd/src/proofd.cxx
o Random seed initialization moved from rsalib.c to in rootd.cxx,
proofd.cxx and TAuthenticate.
rootd/src/rootd.cxx
o Random seed initialization moved from rsalib.c to in rootd.cxx,
proofd.cxx and TAuthenticate.
rpdutils/inc/rpdp.h, rpdutils/src/rpdutils.cxx
o Secure RSA public transmission from clients (sending a server
key)
o Fixed ambiguity with method index resolving from name due to
the fact that 'usrpwd' contains the string 'srp'
o Add file existence check before trying open
o Add a few debug messages
o Fixed problem with domain checking in daemon.access
('cern.ch uidgid' or '.cern.ch uidgid' were not accepted)
o Fixed problem with null passwd receiveing in RpdUser
rpdutils/src/ssh.cxx
o Fixed function name in debug message in ssh.cxx
o Use mkstemp instead of mktemp
srputils/src/SRPAuth.cxx
o Secure RSA public transmission to servers (receiving a server
key)
Revision
6243 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Mar 4 17:29:29 2003 UTC (11 years, 10 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 5780 byte(s)
Diff to
previous 5401
implementation of package manager. Package manager allows the distribution,
installation and setting up of user code in the PROOF sandbox environment.
The package manager distributes so called PAR files, which are standard
tarred and gziped files, but with extension .par. A file "mylibs.par" is
required to untar into a directory "mylibs". A PAR file contains in addition
to the user's code and libraries one special directory PROOF-INF that contains
two scripts: BUILD.sh and SETUP.C. BUILD.sh is a shell script that is
used to build the user's libraries on the PROOF cluster (which might be
of a different architecture/compiler then the user's machine). SETUP.C is
a ROOT macro that is used to load the user's libraries, built by BUILD.sh,
and to do any other kind of setup needed to run with the user libraries.
These scripts are executed in the package directory (not the PROOF-INF
directory). An example of a BUILD.sh is:
#! /bin/sh
make libEvent.so
An example of SETUP.C is:
void SETUP()
{
gSystem->Load("libEvent");
}
The package manager API is:
gProof->UploadPackage()
gProof->EnablePackage()
gProof->ClearPackage()
Revision
4190 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Sat Mar 16 18:36:52 2002 UTC (12 years, 10 months ago) by
rdm
Original Path:
trunk/proof/inc/TProofServ.h
File length: 5050 byte(s)
Diff to
previous 4167
ProofServErrorHandler() now adds to the Info, Warning and Error messages
the master or slave id. Remove private Info version since now in TObject.
Fix handling of kMESS_CINT messages, correctly forward using Exec() instead
of SendCommand() which did not relay the scripts (like in ".x aap.C") to
the slaves.
Revision
1205 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Dec 13 15:13:57 2000 UTC (14 years, 1 month ago) by
brun
Original Path:
trunk/proof/inc/TProofServ.h
File length: 5029 byte(s)
Diff to
previous 1075
W A R N I N G !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================================================
A very long list of changes in this pre-release of version 3.00.
We have modified the signature of many functions (in particular TObject)
to introduce more constness in the system.
You must change your code if your class derives from TObject and uses
one of the modified functions such as ls, Print, Compare, Hash, etc.
The modified functions in TObject have the following signature:
virtual TObject *Clone() const;
virtual Int_t Compare(const TObject *obj) const;
virtual void Delete(Option_t *option=""); // *MENU*
virtual void DrawClass() const; // *MENU*
virtual void DrawClone(Option_t *option="") const; // *MENU*
virtual void Dump() const; // *MENU*
virtual TObject *FindObject(const TObject *obj) const;
virtual char *GetObjectInfo(Int_t px, Int_t py) const;
virtual ULong_t Hash() const;
virtual void Inspect() const; // *MENU*
virtual Bool_t IsEqual(const TObject *obj) const;
virtual void ls(Option_t *option="") const;
virtual void Print(Option_t *option="") const;
A similar operation has been done with classes such as TH1, TVirtualPad,
TTree, etc.
This form allows you to request diffs between any two revisions of this file.
For each of the two "sides" of the diff,
enter a numeric revision.
