Python loop over entries in a chain

Hello ROOTs,

I've been investigating different ways to loop over entries of a TChain and am not quite sure I understand what I see. Maybe someone has some insight.

I made up some TTrees with many entries and a few branches. In my tests I am just interested in looking at one branch. I assume things would generalise to loading several branches but to keep things simple one branch should do.

The first thing to do is find out how long it takes to loop as many times as there are entries in the chain to see what the overhead of looping is in python.

The naive way to loop is to turn off all branches except for the one I am interested in.

To improve on this the recommended thing is to call LoadTree() and then branch->GetEntry() on the branch one is interested in. For comparison this is done in C++ (looping and everything, this is probably the fastest possible) and in python. Then there are some variations on this.

What times do I get?

noop looping: 1.1s
naive: 2.8s
c++ load branch: 0.7s
python everything but GetEntry() on the branch: 4.9s python load branch: 7s

Trying to understand why getbranch in python is so much slower I moved parts of the hard work to C++, namely everything except for the GetBranch() is done in C++ this reduced the time things take to 5.5s.

I assume all the hard work is done somewhere inside GetEntry() or LoadTree() or somewhere, so why does it take so much longer calling these "hard worker" function form python then from c++? I would expect things to take the time it takes to perform this many iterations in python, ~1s, plus the time to do the full thing in C++, ~ 0.7s, so something like two seconds. But not even the "naive", which one might assume is the most common way of doing things and hence most optimised, gets close to that. Maybe there is some overhead in making a call from python -> C++ but that can be minimized by making just one call.

Am I not comparing apples with apples? Making some silly mistake in my code somewhere? Some recommendations how to speed things up?

Ultimately I have some code which will load a branch on demand and was hoping to get the same speed up as one gets by turning the branches off by hand (which is error prone and tedious). I am planning to contribute this to ROOT because I think it is super useful but I would like to understand why things are slow first.

I have attached all my code, it should be fairly straight forward to run. It generates the trees I used and then runs things a few times to get an idea of how long they take.

Tim

-- 
http://tim.jottit.com/






text/x-csrc attachment: t.C



application/octet-stream attachment: t.py