You are here

Working with packages (PAR files)

Working with packages (PAR files)

The PAR file technology addresses the problem of uploading and enabling a package , a set of files which, after proper enabling, provide additional, usually well defined, functionality. In this section we describe what is a PAR file and the TProof interface to work with these files. The interface has been somewhat simplified starting with ROOT 5.27/02, where the automatism to download the needed packages has been introduced. The situation up to 5.26/00 (and related patches) is described in the last part.


Content of this section:

  1. What is a PAR file?
  2. The BUILD.sh script
  3. The SETUP.C function
  4. Listing available packages
  5. The client downloaded cache
  6. Enabling a package
  7. Downloading a package
  8. Uploading a package
  9. Removing packages
  10. Differences in ROOT version 5.26/00 and older

To be used the packages must have been built in dedicated directories on the client and on the cluster nodes. Each user has its own package directory on the client (default $HOME/.proof/packages) and on the cluster (default /packages). Both clients and cluster administrators can defined global package directories, where stable packages of public interest could go.

The ':' separated list of global package directories are defined via the rootrc environment variable Proof.GlobalPackageDirs:

Proof.GlobalPackageDirs  /pool/globalpack-PH1:/pool/globalpack-PH2

Each global package directory gets automatically labeled with 'Gn' where 'n' is the order of appearance in the list (e.g. G0 will refer to '/pool/globalpack-PH1' in the example above). This label is used to address a specific package when working with the PROOF package interface described below. Of course, any operation which modifies the directory content requires the appropriate privileges; so, typically, only a subset of people can modify the global package directories.

Before continuing we need to have a closer look to what a PAR file is.

1. What is a PAR file?

A PAR file is an archive containing all the files needed to build and setup a package. A package is uniquely identified by its name packname . The PAR file packname.par of a package is a gzipped tarball of a well defined directory structure:

  • packname
    Source/binary files, header files, makefile
  • packname /PROOF-INF
    1. BUILD.sh
      Script to build the package; typically if executes 'make' in the upstream source directory; may be missing.
    2. SETUP.C
      Macro to enable the package; this is the place where to handle the dependencies .

A simple example of PAR file can generate following the instructions in $ROOTSYS/test/ProofBench/README . The resulting event.par has the following content

pcepsft43:~$ tar tzvf $ROOTSYS/test/ProofBench/event.par
drwxr-xr-x ganis/sf          0 2007-02-21 19:38:16 event/
-rw-r--r-- ganis/sf      14712 2007-02-21 19:38:16 event/Event.cxx
-rw-r--r-- ganis/sf       7873 2007-02-21 19:38:16 event/Event.h
-rw-r--r-- ganis/sf        259 2007-02-21 19:38:16 event/EventLinkDef.h
-rw-r--r-- ganis/sf      13440 2007-02-21 19:38:16 event/Makefile
-rw-r--r-- ganis/sf      13390 2007-02-21 19:38:16 event/Makefile.arch
drwxr-xr-x ganis/sf          0 2007-02-21 19:38:16 event/PROOF-INF/
-rwxr-xr-x ganis/sf         29 2007-02-21 19:38:16 event/PROOF-INF/BUILD.sh
-rw-r--r-- ganis/sf         47 2007-02-21 19:38:16 event/PROOF-INF/SETUP.C

2. The BUILD.sh script

If present, the BUILD.sh script is executed (via TSystem->Exec(...)) to build the package. The return code of the script is checked for failures with 0 indicating success and != 0 indicating failure. The example shows the BUILD.sh script in the 'tutorials/proof/event.par' example package.

#! /bin/sh
# Build libEvent library.

if test ! "x$ROOTPROOFLITE" = "x"; then
   echo "event-BUILD: PROOF-Lite node (session has $ROOTPROOFLITE workers)"
elif test ! "x$ROOTPROOFCLIENT" = "x"; then
   echo "event-BUILD: PROOF client"
else
   echo "event-BUILD: standard PROOF node"
fi

if [ "" = "clean" ]; then
   make distclean
   exit 0
fi

make
rc=$?
echo "rc=$?"
if [ $? != "0" ] ; then
   exit 1
fi
exit 0

The BUILD.sh must of course be executable. The environment variables ROOTPROOFLITE and ROOTPROOFCLIENT allow to find out whether the BUILD.sh is executed in PROOF-Lite or a client session, respectively, so that dedicated actions may be taken if needed. They are available starting from ROOT 5.28/00a .

3. The SETUP.C function

If present, the SETUP.C function is executed to setup the package; this includes all the settings needed to properly use the package (e.g. dependency library loading). The basic signature of the SETUP() function is

Int_t SETUP()
To properly stop the execution flow the function should return -1 on failure and 0 on success.
Starting with ROOT version 5.27/04 to additional signatures are supported for SETUP():
Int_t SETUP(const char *opt)
Int_t SETUP(TList *optls)
The 'opt' string or the 'optls' object list can be passed via TProof::EnablePackage (see below).
The example shows the SETUP.C function in the 'tutorials/proof/event.par' example package.

 

Int_t SETUP()
{
   if (gSystem->Getenv("ROOTPROOFLITE")) {
      Printf("event-SETUP: PROOF-Lite node (session has %s workers)",
                                   gSystem->Getenv("ROOTPROOFLITE"));
   } else if (gSystem->Getenv("ROOTPROOFCLIENT")) {
      Printf("event-SETUP: PROOF client");
   } else {
      Printf("event-SETUP: standard PROOF node");
   }
   if (gSystem->Load("libEvent") == -1)
      return -1;
   return 0;
}
Again, the environment variables ROOTPROOFLITE and ROOTPROOFCLIENT (available starting from ROOT 5.28/00a ) allow to find out whether the SETUP.C is executed in PROOF-Lite or a client session, respectively, so that dedicated actions may be taken if needed.

4. Listing available packages

Once a PROOF session has been started, one of the first things to do is to see which packages are already avalaible on the cluster. This is achieved by issuing a call to

TProof::ShowPackages(Bool_t all = kFALSE)

for example:

root [0] p = TProof::Open("alicecaf.cern.ch:41093")
Starting master: opening connection ...
Starting master: OK
Opening connections to workers: OK (13 workers)
Setting up worker servers: OK (13 workers)
PROOF set to parallel mode (13 workers)
(class TProof*)0x2230da0
root [1] p->ShowPackages()
*** Package cache client:/home/ganis/.proof/packages ***
total 32
drwxr-xr-x 3 ganis ganis  4096 2010-03-29 14:29 ANALYSIS
lrwxrwxrwx 1 ganis ganis    50 2010-03-29 14:29 ANALYSIS.par -> /home/ganis/.proof/packages/downloaded/ANALYSIS.par
drwxr-xr-x 3 ganis ganis 12288 2010-03-29 14:28 ESD
lrwxrwxrwx 1 ganis ganis    45 2010-03-29 14:28 ESD.par -> /home/ganis/.proof/packages/downloaded/ESD.par
drwxr-xr-x 3 ganis ganis  4096 2010-03-29 14:30 event
lrwxrwxrwx 1 ganis ganis    59 2010-03-29 14:29 event.par -> /home/ganis/local/root/trunk/root/tutorials/proof/event.par
drwxr-xr-x 3 ganis ganis 12288 2010-03-29 12:37 STEERBase
lrwxrwxrwx 1 ganis ganis    51 2010-03-29 14:27 STEERBase.par -> /home/ganis/.proof/packages/downloaded/STEERBase.par

*** Global Package cache G0 lxfsrd0506.cern.ch:/pool/proofbox-xpd3/alicecaf/packages ***
total 428
drwxr-xr-x  3 alicecaf z2    690 Mar 29 13:54 ANALYSIS
-rw-r--r--  1 alicecaf z2  40517 Mar 29 13:52 ANALYSIS.par
drwxr-xr-x  3 alicecaf z2  16384 Mar 29 13:54 ESD
-rw-r--r--  1 alicecaf z2 193055 Mar 29 13:52 ESD.par
drwxr-xr-x  3 alicecaf z2  16384 Mar 29 13:53 STEERBase
-rw-r--r--  1 alicecaf z2 165116 Mar 29 13:52 STEERBase.par

*** Package cache lxfsrd0506.cern.ch:/pool/proofbox-xpd3/ganis/packages ***
total 12
drwxr-xr-x  3 ganis sf  196 Mar 29 14:54 event
-rw-r--r--  1 ganis sf 9505 Mar 29 14:54 event.par
root [2]

The argument controls whether the command has to be executed also on the workers or only on the client and master; the workers should have the same packages as the master, so the default is 'false'. In the above example we see that locally there is no global directory on the client, that there is a global directory 'G0' at /pool/proofbox-xpd3/alicecaf/packages on the cluster with three packages, and that there is only one private package 'event' available on the cluster.

3. The client downloaded cache

Starting with ROOT 5.27/02, clients may download the packages needed by the query from the master. The PAR files for the downloaded packages are cached under the 'downloaded' sub-directory of the package directory (default $HOME/.proof/packages/downloaded'). The downloaded packages are automatically updated whenever the copy on the master is updated. If the client uploads a private package with same name (for example a privately modified version of an existing package), the download cache is automatically cleaned of the related package so that the automatic cross-check for updates in the master is disabled.

In the example above, the packages STEERBase, ESD and ANALYSIS have been downloaded from the master, while the package event has been uploaded by the client.

4. Enabling a package

A package that ShowPackages() lists as available on the cluster can be enabled (i.e. loaded - and built, if needed) by calling one of the following three methods:

TProof::EnablePackage(const char *packname, Bool_t notOnClient = kFALSE)
TProof::EnablePackage(const char *packname, TList *optls = 0, Bool_t notOnClient = kFALSE)
TProof::EnablePackage(const char *packname, const char *opt, Bool_t notOnClient = kFALSE)

The first argument contains the name of the package (if present, the extension '.par' is automatically stripped off). The 'notOnClient' argument can be used to avoid enabling the package locally (i.e. on the client). The 'opt' or 'optls' arguments allow to specify, respectively, a string or a list of objects to be passed to the SETUP function (see above).

A package not available yet on the client is downloaded automatically in the client local .download cache. A package that was previously downloaded (i.e. whose PAR file is stored locally in the .download cache) is checked for updates and eventually downloaded again. When a package is downloaded from the master repository, any local reference to the package is removed and a full build started again.

On the cluster, enabling of a package is done in parallel: the master first forwards the enabling request to the worker nodes and then starts its own build. The logs from the building operations on the master are feedback to the client in real-time; those from the worker nodes come after.

The list of enabled packages can be displayed with

TProof::ShowEnabledPackages(Bool_t all = kFALSE)

Again, only the packages enabled on the master are shown by default. If 'all' is KTRUE the requests are run also on the worker nodes.

For example:

root [0] p = TProof::Open("alicecaf.cern.ch:41093")
Starting master: opening connection ...            
Starting master: OK                                                 
Opening connections to workers: OK (13 workers)                     
Setting up worker servers: OK (13 workers)                          
PROOF set to parallel mode (13 workers)                             
(class TProof*)0x20fb140                                            
root [1] p->ShowEnabledPackages()                                   
*** Enabled packages on client on pcphsft64                         
*** Enabled packages ***                                            
root [2] p->EnablePackage("STEERBase")
Info in <:downloadpackage>: 'STEERBase' cross-checked against master repository (local path: /home/ganis/.proof/packages/.download/STEERBase.par)                                                                                                                        
make: `libSTEERBase.so' is up to date.                                                                                                  
Mst-0: building STEERBase ...                                                                                                           
Mst-0: make: `libSTEERBase.so' is up to date.                                                                                           
Wrk-0.0: building STEERBase ...                                                                                                         
Wrk-0.0: make: `libSTEERBase.so' is up to date.                                                                                         
...
(Int_t)0                                                                                                                                
root [3] p->EnablePackage("ESD")
Info in <:downloadpackage>: 'ESD' cross-checked against master repository (local path: /home/ganis/.proof/packages/.download/ESD.par)                                                                                                                                    
make: `libESD.so' is up to date.                                                                                                        
Mst-0: building ESD ...                                                                                                                 
Mst-0: make: `libESD.so' is up to date.                                                                                                 
Wrk-0.6: building ESD ...                                                                                                               
Wrk-0.6: make: `libESD.so' is up to date.                                                                                               
...                                                                                              
(Int_t)0                                                                                                                                
root [4] p->EnablePackage("ANALYSIS")
Info in <:downloadpackage>: 'ANALYSIS' cross-checked against master repository (local path: /home/ganis/.proof/packages/.download/ANALYSIS.par)
make: Nothing to be done for `default-target'.
Mst-0: building ANALYSIS ...
Mst-0: make: Nothing to be done for `default-target'.
Wrk-0.11: building ANALYSIS ...
Wrk-0.11: make: Nothing to be done for `default-target'.
...
(Int_t)0
root [5] p->ShowEnabledPackages()
*** Enabled packages on client on pcphsft64
STEERBase
ESD
ANALYSIS
*** Enabled packages ***
STEERBase
ESD
ANALYSIS
root [6]

5. Downloading a package

As mentioned in the discussion of TProof::EnablePackage, packages are automatically downloaded when needed. However, one can force download of a package issuing directly the dedicated method:

TProof::DownloadPackage(const char *par, const char *dstdir = 0)

The default destination directory is the downloaded sub-directory in the packages directory (default $HOME/.proof/packages/downloaded). This functionality allows to download packages from the master repository at user will, for example it allows to create a local repository of packages for further distribution.

6. Uploading a package

A new package (or a new version of a package) can be uploaded to the session via

TProof::UploadPackage(const char *parfile, EUploadPackageOpt opt = kUntar)

The first argument contains the path to the PAR file (with or without '.par' extension). The first time a PAR file is uploaded, it is cached by master and worker sessions in the user's sandbox. The next time the user requests the upload, the MD5 of the PAR file on the farm nodes is checked and the file is uploaded only if changed.

The second argument indicates the way the new version of and existing PAR file has to be treated. By default PROOF just untars the PAR file, so that binaries of unchanged source files can be re-used during build. if 'opt' is set to 'kRemoveOld' then the exiting package directory is removed from the sandbox.

The 'upload' functionality is useful for private packages or for modified version of common packages.

7. Removing packages

A package can be removed with

TProof::ClearPackage(const char *packname)

This operation deletes both the building directory and the PAR file, both on the client and the cluster. Non backup is provided for cleared packages, so this method has to be used with care. After TProof::ClearPackage an TProof::UploadPackage is needed before re-enabling the package.

The following removes all installed packages

TProof::ClearPackages()

8. Differences in ROOT version 5.26/00 and older

The functionality in the previous sub-sections is fully available only starting from ROOT version 5.27/02. The DownloadPackage and related functionality (automatic download; client downloaded cache) are missing from previous versions. DownloadPackage uses the sandbox access interface introduced in ROOT 5.25/02, which therefore can be used in ROOT 5.26/00 to retrieve the relevant files. Furthermore, in ROOT 5.27/02 a few bugs affecting the usage of global package directories have been fixed. These fixes have been included in the patch version 5.26/00c .