Re: Friend trees of "filtered Trees"

On Sun, Jun 21, 2009 at 1:18 PM, Tim Head <betatim_at_gmail.com> wrote:

> Hello all (for real this time),
>
> 2009/6/19 Salvatore Rappoccio <rappocc_at_fnal.gov>:
> > On Fri, Jun 19, 2009 at 5:30 PM, Tim Head <betatim_at_gmail.com> wrote:
> >
> > Hi, Tim,
> >
> > Thanks for the suggestion, but unfortunately in our case this solution
> > doesn't scale sufficiently to help us.
> >
>
> What kind of benchmarks did you do to find this out? My trees
> currently seem to be acceptable (hundreds of MB) but I would be
> interested to know when to expect this falling over.
>
> Tim

The issue for us would be that we will have the following situation:

• A huge dataset (many hundreds of GB, in principle) that is skimmed down to very few events, and most branches dropped. This is expected to be on the tens of GB level or less.
• The user does work with the smaller dataset locally.
• If they don't understand something, they access the "full" dataset if necessary via the grid (or on some mass storage, in the case of FNAL it would be DCACHE).
• At CMS there are two ways to do an analysis, with our "full framework" (which has read/write capability, access to DB and geometry, etc) and our "light framework" which is basically object libraries on top of root, plus some added capability. Typically the user will do selection with the "full framework" via the grid, and then do their analysis with the "light framework". Oftentimes the user doesn't have the exact configuration to do the skim in the "light framework", and simply wants to access the information via "run,event" mapping.

The example from Philippe will be much more extensible in our use case (creating a "run,event" mapping between the trees).

Thanks for the suggestion, though! I appreciate it.

Sal

>
>
> > Thanks for the advice anyway, though!
> >
> > Sal
> >
>
>
>
> --
> http://tim.jottit.com/
>
Received on Sun Jun 21 2009 - 20:15:44 CEST