You are here

ROOT Version v5-26-00 Proof

This branch consist of the latest 5-26-00-patches integrated with the latest PROOF develpments from the SVN trunk. The branch is developed at

https://root.cern.ch/svn/root/branches/dev/proof/branches/v5-26-00-proof
Tags on the branch are created under
https://root.cern.ch/svn/root/branches/dev/proof/tags
The version on the development branch has cycle +49 with respect to the reference ROOT tag, e.g. 5.26/49 . The first tag on the branch has version 5.26/50. The next development cycle 5.26/51. And so on. Tags have always even cycle numbers.

Changes in the head of v5-26-00-proof wrt v5-26-00-proof-04

  • New functionality
  • Improvements
    • Add support for the ROOTrc variable 'Proof.UseMergers' to trigger the use of submergers for merging. This has the same meaning of the parameter 'PROOF_UseMergers'. The setting can be always superseeded by the user setting of the 'PROOF_UseMergers' in the input list (or of the local ProofUseMergers setting).
    • Add test in stressProof for package argument setting.
  • Fixes
    • Lock the package directory during the execution of the modified SETUP to avoid clashes between multiple workers on the same machine.
    • Correctly register the status code of building and loading packages on the workers, and transmit it to the client.
    • Fix problem with packet re-assignment in case of a worker death. Some packets were processed twice or more times.
    • Fix problem with transmission of EnablePackage arguments to workers.
    • Fix an issue with a semaphore closing actions and asynchronous processing.This should remove a problem sometimes happening at the end of stressProof where the main application goes into timeout when closing the PROOF session.

Tag v5-26-00-proof-04

Details

Changes in v5-26-00-proof-04 wrt v5-26-00-proof-02

  • New functionality
    • Identify which TSelector data members point to which output list object on the workers, and if consistent set the client's data members to point to the corresponding objects of the merged output list.
  • Improvements
    • Add support for arguments in the SETUP function: it is possible now to pass a string
                  Int_t SETUP(const char *opt)
      or a list of objects
                  Int_t SETUP(TList *optls);
      Two interface methods have been added to support this:
                  Int_t TProof::EnablePackage(const char *package, const char *opt, Bool_t notOnClient);
                  Int_t TProof::EnablePackage(const char *package, TList *optls, Bool_t notOnClient);
    • Allow building packages in the global package directories if 'write' permissions are available; this facilitates management of global packages from an authorized account.
    • Optimize the validation step in the case not all the entries are required. The validation step is stopped as soon as the requested number of events is reached. If the parameter "PROOF_ValidateByFile" is set to 1, the number of files is exactly what needed; otherwise the number of files may exceed the number of fles needed by #workers-1.
    • New Xrootd version 20100602-0830 .
  • Fixes
    • Make sure the return code from BUILD.sh is properly checked and execution flow stop on failure;
    • Fix a subtle bug affecting the (possibly rare) case when not all entries are required and # entries does not correspond to an complete subset of files (e.g. # entries = 1001000 with files of 100000 entries each).
    • Couple of fixes in h1analysis (see #33866);
    • In TXSocket, make sure that the socket cannot be closed (and deleted) while processing an asynchronous message (in a separated thread).
    • Fix issue with the ROOT_VERSION_CODE preventing library compatibility with 5.26/00x.

Tag v5-26-00-proof-02

Details

Changes in v5-26-00-proof-02 wrt v5-26-00-proof-00

  • New functionality
    • Add possibility to disable the graphic progress dialog via a call to TProof::SetProgressDialog(kFALSE).
  • Improvements
    • Updated man pages for the PQ2 tools
    • New tutorial 'friends' to illustrate the use of TTree friends
    • New packetizer TPacketizerFile to generating packets which contain a single file path to be used in processing single files. Used, for example, in tasks generating files, like in the forthcoming PROOF bench or the "friends" tutorial.
    • In TFileMerger, implement merging of THStack objects as done already in hadd
    • Remove data directory in TProofServ::Terminate (and alike) when empty: avoid having many empty directories around (patch #33693)
  • Fixes
    • In TXProofMgr import fix #33639: fix problem affecting TProofMgr::Find.
    • Import fix #33640: fix a few issues affecting the usage of tree friends in PROOF and a bug affecting the locality check for files in TDSet.
    • In TProof::ClearData() import fix #33641: fix problem observed when the dataset repository is empty.
    • In TProof import fix #33648: fix an issue with the workers names in TSlaveInfo in PROOF-Lite .
    • Fix a few issue affecting test/ProofBench on MacOsX
    • In Xrootd, import fix #33677 fixing a fake authentication failure with proxies created by voms-proxy-init when the input certificates are PKCS12-formatted
    • In TProofLite, make sure that the default sandbox is always ~/.proof and that the package lock file is under gSystem->TempDirectory()

Tag v5-26-00-proof-00

Details

Changes in v5-26-00-proof-00 wrt v5-26-00-patches

  • New functionality
    • Add support for processing many datasets in one go in TProof::Process(const char *dataset, ...).
      Two options are provided:
       - 'grand dataset':   the datasets are added up and considered as a single dataset (syntax: "dataset1|dataset2|...")
       - 'keep separated': the datasets are processed one after the other; the user is notified in the selector of the change of dataset so she/he has the opportunity to separate the results. A new packetizer, TPacketizerMulti, has been developed for this case: it basically contains a list of standard packetizers (one for each dataset) and loops over them (syntax: "dataset1,dataset2,..." or dataset1 dataset2 ...").
      In both cases, entry-list can be applied using the syntax "dataset The datasets to be processed can also be specified on one or multiple lines in a text file.
      See http://root.cern.ch/drupal/content/working-data-sets for more details.
    • Add support for automatic download of a package when available on the master but not locally. The downloaded packages are store under /packages/downloaded and automatically checked for updates against the master repository. If a local version of the same package is created (using the UploadPackage) the entry in downloaded is cleared, so that the behaviour is unchanged.
      The new functionality is described in http://root.cern.ch/drupal/content/working-packages-par-files .
    • Add the possibility to remap the server for the files in a dataset. This allows, for example, to reuse the dataset information for the same files stored in a different cluster.
    • Add a local cache for TDataSetManagerFile. This is mainly used to improve the speed of TDataSetManager::ShowDataSets, which is run very often by users and may be very slow if the number of dataset is large. The cache is also used to cache frequently received dataset objects.
    • Add the possibility to audit the activity on the nodes via syslog. See http://root.cern.ch/drupal/content/enabling-query-monitoring.
  • Improvements
    • Improve support for valgrind runs in PROOF-Lite
    • Add the possibility to add files to a dataset. This is achieved with a new option 'U' (for update) to RegisterDataSet.
    • Add methof TProof::GetStatistics to allow the client to retrieve the correct values of fBytesRead, fRealTime and fCpuTime at any moment; this will be used to setup a sort of ROOTmarks in stressProof .
    • Several improvements in the test program 'stressProof' and in the tutorials under 'tutorials/proof'
    • Avoid contacting the DNS when initializing TProofMgr as base class of TProofMgrLite: it is not needed and it may introduce long startup delays.
    • Make TProof::LogViewer("") start the viewer for a Lite session, in parallel to whats happen for TProof::Open("").
    • Several improvements in the handling of wild cards in the dataset manager; for example, issuing a GetDataSet(...) on a dataset URI containign wild cards will return a grand dataset sum of all the datasets matching the URI.
    • Add options to get a list of all dataset registered names from ScanDataSets (option kList; the result is a TMap of {TObjString, TObjString} with the second TObjString empty).
    • Improved version of the PQ2 scripts; the scripts now invoke a dedicated ROOT application (named pq2) available under $ROOTSYS/bin .
    • Add support for recursive reading of group config files via the 'include sub-file' directive. This allows to have a common part and, for example, customize differently the quotas.
  • Fixes
    • Fix a bug in error status transmission which avoid session freezing in some cases
    • FIx a few issues in libXrdProofd.so with handling of connection used for admin operation: this should solve some cases where the daemon was not responding.
    • Fix a few memory leaks showing up when running several queries in the same session
    • Fix a few issues affecting the new sub-merging option
    • Fix an issue preventing proper real-time notification during VerifyDataSet
    • Fix an issue with TQueryResult ordering (was causing random 'stressProof' failures)
    • Fix an issue with TProof::AskStatistics (fBytesRead, fRealTime and fCpuTime were not correctly filled on the client; the values on the master, displayed by TProof::Print were correct).
    • Fix several small issues affecting the handling of global package directories
    • Fix an issue with socket handling in the main event-loop while sendign or receiving files via TProofMgr.
    • Fix a problem counting valid nodes in sequential or 'masteronly' mode, generating the fake error message "GoParallel: attaching to candidate!"
    • Fix a few issues with the length of Unix socket paths affecting PROOF-Lite and xproofd on MacOsX
    • Fix an issue with the release of file descriptors when recovering sessions .
    • Fix an issue with a fake error message ("Error in <:cd>: No such file root:/") in PROOF-Lite when issuing TProof::SetParallel().
    • Fix a problem with negative values for 'workers still sending' in PROOF-Lite .
    • Fix locking issue while building packages locally.
    • Fix issue setting permission and ownership of the dataset user directories.