Root as a service proposal

[joel.vogt]

Dear all

We (some people at the DGS-RP Group) are developing a Python-based system to share code snippets over the network. We started this project because some of our group wanted to use numpy et al. from within the Jython Pixelman (http://aladdin.utef.cvut.cz/ofat/others ... elman.html) console. This system is in its infancy; currently it only supports Python. What we would like to do next is add support for Root. The aim would be to have an open “Root as a service” platform. Root developers can publish Root algorithms as remote procedures so other scientists can call and reuse these functions remotely.
Does something similar already exist? Would others also find such a system useful? … We’d love to get some feedback/comments!
Thanks

Joël

Source: https://github.com/joelvogt/CERNthon

[wlav]

Joël

We (some people at the DGS-RP Group) are developing a Python-based system to share code snippets over the network. We started this project because some of our group wanted to use numpy et al. from within the Jython Pixelman (http://aladdin.utef.cvut.cz/ofat/others ... elman.html) console.

just so that I understand it: you ran into the lack of a Python C-API in Jython and that's why you chose this approach? (That also means that technically, the remote process does not need to be python, right?)

For data streaming (as per the last slide in the .pdf you provided): PyROOT allows the pickling of ROOT/C++ objects. This is not super efficient (objects are written into a buffer, then copied into a string for pickling), but it does allow for easy transfer of basic C++ data objects through a python layer.

Some such was done by LHCb (using http://www.parallelpython.com/ IIRC).

In a similar vein, there is a problem with extension modules and PyPy in the same way as for Jython (just to a lesser degree). Offloading to a PyPy process, over the network or locally, would then allow a speedup, without loosing the ability of using all extension modules one wants.

Cheers,
Wim

[joel.vogt]

Hi Wim

Thanks for your prompt and comprehensive reply!

just so that I understand it: you ran into the lack of a Python C-API in Jython and that's why you chose this approach? (That also means that technically, the remote process does not need to be python, right?)

Yes, exactly. The C-API issue with Jython was the initial motivation to build a workaround. Thanks for pointing out parallel python and PYROOT's pickling abilities. This is a super helpful information.
The other part of this proposal is to have an open "Root" (et al.) services platform. When we started working on CERNthon (that's how we call our system), the idea came up that we could share the algorithms on the CERNthon server with others. Also we have some great Root code, which is only being used by the developer himself, because others in the group use different frameworks. That's why we'd like to make that Root code too accessible remotely. A bit like a remote library to call domain specific algorithms.
What's your thought on that?

Cheers,

Joël