Loading files to proof worker nodes

ROOT
1

Dear experts,

I’m running a TPySelector on a proof-lite setup and I pass configuration parameters via pickled python dictionaries.

From the TProof documentation I found the Load() function which allows me to load the file to the workers and then read it in the TPySelector.

However, because the file is not a ROOT macro, the call to TProof::Load() returns the following warning for each worker node:

15:23:30 26551 Wrk-0.19 | Info in <TProofServLite::HandleCache>: loading macro parameters.pkl ...
In file included from input_line_11:1:
/grid_mnt/vol__vol_U__u/llr/cms/pigard/.proof/cache/parameters.pkl:2:1: error: expected ')'
S'outputName'
^
/grid_mnt/vol__vol_U__u/llr/cms/pigard/.proof/cache/parameters.pkl:1:1: note: to match this '('
(dp0
^
/grid_mnt/vol__vol_U__u/llr/cms/pigard/.proof/cache/parameters.pkl:1:2: error: C++ requires a type specifier for all declarations
(dp0
 ^
/grid_mnt/vol__vol_U__u/llr/cms/pigard/.proof/cache/parameters.pkl:42:3: error: expected ';' after top level declarator

Is there a better way to load non-macro files to the worker nodes?

Thanks for your insights!
Philipp

2

Dear Philipp,

I’ll have a look if and how we can extend the TProof::Load functionality.
In the meantime you may consider the PAR mechanism, which allows to do generic things (root.cern.ch/working-packages-par-files).
Otherwise, is there any solution you can design around the input list (root.cern.ch/input-list)?

G Ganis

3

Dear Ganis,

thank you for the reply and information on the other methods to upload data to the nodes.

I might consider an approach based on PAR files, but pickling dictionaries has the advantage of working in non-proof environments.

The input list might be a solution, if I wrap the dictionaries with a TObject. On the other hand, the current solution works, and the only issue is a clogged/messy log file.

Thank you again for your support.
Philipp