[root] / trunk / proof / proofplayer / inc / TProofPlayer.h Repository:
ViewVC logotype

Log of /trunk/proof/proofplayer/inc/TProofPlayer.h

Parent Directory Parent Directory


Links to HEAD: (view) (download) (as text) (annotate)
Sticky Revision:

Revision 45348 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Aug 2 12:23:54 2012 UTC (2 years, 5 months ago) by ganis
File length: 19773 byte(s)
Diff to previous 45283

     Consolidation of the automatic output saving functionality
     - Fix an issue with saving output from memory on the master
     - Remove some duplications and fix several small bugs

Revision 45283 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Jul 30 10:42:38 2012 UTC (2 years, 5 months ago) by ganis
File length: 19677 byte(s)
Diff to previous 44397

   Patch automatizing the usage of the file-based technology to handle outputs.
   After this patch PROOF users can define as an option to Process an output file for their results
   and choose if the merging goes via file or in memory. User can also decide to save theit results
   in a set of files identified by a name (dataset).
   Administrators can force merging-via-file or can make it conditional to the resident memory exceeding
   a given threshold.

   The new options are documented at
                                         http://root.cern.ch/drupal/content/handling-outputs

   Since TProofOutputFile is now used by TProof, the class has been moved from proof/proofplayer
   (libProofPlayer) to proof/proof (libProof).
   The selector tutorials/proof/ProofSimple.h,.C has been modified to allo testing the new options,
   and a new test has been added to stressProof.

Revision 44397 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu May 31 13:28:14 2012 UTC (2 years, 7 months ago) by ganis
File length: 19193 byte(s)
Diff to previous 44046
   Use by default the official Merge methods for histograms which correctly treats
   histograms with labelled axis.
   Simple addition was performed for histograms with same axis to overcome a
   perfomance penalty due to additional general consistency checks present in
   the first version of TH1::Merge. This performance issue has been fixed starting
   with 5.32.

   Should the issue reported in Savannah #94926.

Revision 44046 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue May 1 11:34:04 2012 UTC (2 years, 8 months ago) by ganis
File length: 19183 byte(s)
Diff to previous 44010
   Simplify enabling of basic feedback.
   - In TDrawFeedback::Feedback, call method Draw() of objects not identified as TH1
     derivation. This allows user-defined objects implementing Draw to be displayed
     via this utility class (request on the forum).
   - In TPerfStats, add new histogram TH1I with the packets being processed per worker;
     currently this is 0 or 1 but may change in future; it shows which workers are
     still active, which may be useful for debugging at the end of query.
   - Add new class TStatsFeedback to display in one canvas the distributions for
     the events processed per workers, the packets processed per workers and the
     packets being processed per worker.
   - In TProof::Process, add support for switches "fb=name1,name2,name3,... " or
     "feedback=name1,name2,name3,... " in the option field. This enables feedback
     for the specified objects, creating a TDrawFeedback object attached to the session.
     Feedback for the specified objects is disabled at the end of the query and the
     created TDrawFeedback is destroyed.
     The special name 'stats' enables the three feedback histograms required by
     TStatsFeedback, and uses a TStatsFeedback instead of TDrawFeedback.
     The feedback specs are removed from the option field passed to TProofPlayer::Process.

Revision 44010 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sun Apr 29 23:07:48 2012 UTC (2 years, 8 months ago) by ganis
File length: 19062 byte(s)
Diff to previous 43044
  Several optimizations in the low level PROOF event loop (TProofPlayer::Process):
  - Per-event calls to TEventIter::GetNextEvent are replaced by per-packet calls
    to a new function TEventIter::GetNextPacket and a for loop over the returned
    range;
  - Progress information is filled in on per-packet pace just before asking for
    a new packet, instead of being filled for each event;
  - In TProofProgressStatus, the relevant setters in were inlined, and the (expensive)
    calls to SetLastUpdate() reduced to the minimum required;
  - Status of objects is checked using bits instead of functions calls;
  - Check whether to dump memory info is done before calling CheckMemUsage.
  These optimizations allow to reduce dramatically the overhead introduced by the
  operations PROOF needs to perform during the event loop. A measurement of the
  overhead can be obtained from a very light computational task, for example,
  generating one random number and filling one histogram; executing this task
  within a PROOF-Lite session with 1 worker now takes only 1.8 times the time
  required by a straight loop in the parent ROOT session; the same number before
  was about 13. 

  The patch also introduces a couple of small changes enabling the use of
  TProofPlayerLocal to process selectors in the local session using the PROOF
  iteration mechanism; this is very useful for optimizations studies.

  Finally, a couple of missing protections were added in TProofPlayer.

Revision 43044 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Feb 17 14:34:52 2012 UTC (2 years, 11 months ago) by ganis
File length: 18070 byte(s)
Diff to previous 40186

    From Taesoo Kim and me.

    Add support for selector-by-object processing in PROOF.
    The selector object, created and configured locally by the user, is added to the input
    list and recuperated from there on the worker machines for processing. Any input list
    setting in the selector itself is not streamed but temporarly moved to then standard
    input list, so that user can use the selector input list as container of processing
    information if they find convenient to do so.
    Process(...) methods with the file name argument replaced by 'TSelector *' have 
    introduced where relevant (TProof, TProofPlayer and their derivatives, TDSet). 
    Two tests for the new functionality have been added to test/stressProof.cxx (tests 25
    and 26).

Revision 40186 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Jul 11 12:00:50 2011 UTC (3 years, 6 months ago) by ganis
File length: 17245 byte(s)
Diff to previous 39442
  - Record in TStatus also the max memory usage on the master and printed via TStatus::Print;
    this allow a quick visualisation of the overall memroy usage at the end of the query:

          root [] TStatus *st = (TStatus *) p->GetOutputList()->FindObject("PROOF_Status");
          root [] st->Print();
          OBJ: TStatus    PROOF_Status    OK
            Max worker virtual memory: 394.90 MB   Max worker resident memory: 229.47 MB 
            Max master virtual memory: 1104.55 MB  Max master resident memory: 940.73 MB 

  - Fix issue in TProofPlayer::HandleHistogram preventing proper histogram cleaning right
    after merging when using TH1::Add; histogram were still destroyed at the end of the
    query, but there was no memory advantage in TH1::Add wrt TH1::Merge.

Revision 39442 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri May 27 09:13:25 2011 UTC (3 years, 7 months ago) by ganis
File length: 17229 byte(s)
Diff to previous 36592
  In TProofPlayerRemote::HandleHistograms, add support for an option to call directly TH1::Add
  if the incoming histogram has the same axis of the one in memory. This allows to avoid going
  through the whole Merge machinery for the 99.9% of the cases, considerably speeding up
  histogram merging and significantly reducing memory requirements on the master.
  This new way is on by default and is controlled by the rootrc 'ProofPlayer.UseTH1Merge' and
  by the input list parameter PROOF_UseTH1Merge .

Revision 36592 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Nov 11 10:43:17 2010 UTC (4 years, 2 months ago) by ganis
File length: 17017 byte(s)
Diff to previous 36537
   - Terminate a proofserv after it has been stopped for large memory footprint and it has
     sent the results back
   - Notify information about which object was being merged if a memory exception happens
     during merging on the master
   - Allow to set the HWM and stop-process fractions via the environment

Revision 36537 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Nov 8 16:16:41 2010 UTC (4 years, 2 months ago) by ganis
File length: 17015 byte(s)
Diff to previous 34081
  Patch to improve the diagnostic in case of exceptions. Information about the event and file being
  processed at the moment the exception was raised is sent to the client, e.g.

0.5: caught exception triggered by signal '1' while processing dset:'EventTree',
   file:'http://root.cern.ch/files/data/event_3.root', event:1 - check logs for possible stacktrace

  The patch also fixes a problem with submergers observed when a worker was stopped because above
  the memory limits: this worker was established as merger but could not do the work, for obvious
  reasons, freezing the session.

Revision 34081 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Jun 23 10:59:01 2010 UTC (4 years, 7 months ago) by ganis
File length: 16879 byte(s)
Diff to previous 33867
Make sure that the record used by the memory monitor is written on the master in all cases

Revision 33867 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Jun 14 10:02:39 2010 UTC (4 years, 7 months ago) by ganis
File length: 16832 byte(s)
Diff to previous 31505
  From Axel:
  Identify which TSelector data members point to which output list object
  on the workers, and if consistent set the client's data members to point
  to the corresponding objects of the merged output list.

Revision 31505 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Dec 2 13:20:06 2009 UTC (5 years, 1 month ago) by brun
File length: 16728 byte(s)
Diff to previous 30900
From Katerina Opocenska:

    Patch implementing the new submerger functionality to speed up the merging phase.
    At the end of the query, a set of workers are promoted submergers and assigned a
    sub-set of workers to merge. Once each sub-merger has merged its sub-set of workers,
    it sends its result to the master, which merges the partial results into the final
    set of results.
    The determination of the sub-mergers is always done dynamically, based on the recent
    performance of workers. An optimal (i.e. giving the highest speed-up) number can be
    calculated analytically under simpleassumptions.

    Merging via submergers is by default disabled. To enable it, with the optimal number
    of sub-mergers, one should set the integer parameter 'PROOF_UseMergers' to 0, i.e.

                     proof->SetParameter("PROOF_UseMergers", 0)

    To force S sub-mergers (regardless of the optimal number) do

                     proof->SetParameter("PROOF_UseMergers", S)

Revision 30900 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Oct 28 14:10:02 2009 UTC (5 years, 2 months ago) by ganis
File length: 16692 byte(s)
Diff to previous 30859
Enable the new performance monitoring also for multi-master setups

Revision 30859 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sat Oct 24 14:53:07 2009 UTC (5 years, 3 months ago) by ganis
File length: 16568 byte(s)
Diff to previous 30174
  Patch for improved performance monitoring. The 'Rate Plot' button in the progress
  dialog is renamed 'Performance plot' and shows up to 4 plots with the event/sec,
  the average read chunck size, the number of active workers and the number of active
  PROOF sessions on the cluster, all as a function of processing time.

  The read chunck size plot allows to monitor the usage of the cache.

  The istantaneous processing rate (event/sec) is now better estimate: a few issues
  with the normalizing times have ben solved, removing the artificial structures that
  were observed.

  The possibility to set a max packet time length is introduced (default 30 s); this
  can be changes with the parameter PROOF_MaxPacketTime.
  The size of the cache is also taken into account to optimize the use of the cache.

  The parameter PROOF_UseParallelUnzip has been introduced to toggle the use of the 
  parallel unzip (default off for now).

  A page describing the new performance plots is under preparation at 
           http://root.cern.ch/drupal/content/progress-dialog

Revision 30174 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Sep 15 14:24:56 2009 UTC (5 years, 4 months ago) by ganis
File length: 16085 byte(s)
Diff to previous 27234
   Patch to support automatic creation of a dataset out of files created on the
   worker nodes by worker processes. The implementation is an extension of the
   functionality of the class TProofOutputFile used for merging via file.

   TProofOutputFile:
    - Add options to steer the creation of a dataset; the extemded meaning of the
      constructor arguments and anew constructor are described in

        http://root.cern.ch/drupal/content/handling-large-outputs-root-files

      The dataset (i.e. TFileCollection) assembing happens on the master via the
      Merge method. Information about the node location of the file is happended
      to the relevant TFileInfo as a special "file:" url with option "node=<host-fqdn>:<port>"
      This information is needed to facilitate the cleaning of the files once they
      are not needed anymore.
      The dataset can be just created and returned in the output list, or registered
      and/or verified.

   TProofPlayerRemote::MergeOutputFiles:
    - Correctly transmit the dataset object into the output list

   TProofServ:
    - Create the new subdirectory 'data' either in the sandbox or followinf ProofServ.DataDir .
    - Register and verify the created dataset if required

   TProof:
    - Implement two new methods:
      - ShowData(), to show the content of the assigned 'data' directory
      - ClearData(UInt_t what, const char *dsname), to clear all or a subset of the files
        in the assigned 'data' directory

   Additional changes:
    - TProofPlayer:
      - Correctly handle the feedback canvas
      - Set the drawing attributes
    - TProofServ:
      - Replace all 'Form("...")' with TString::Format("...") .
      - Use the new tag kCollect in debug statements where relevant 
    - TProof:
      - Use the new tag kCollect in debug statements where relevant

Revision 27234 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Jan 26 07:57:14 2009 UTC (5 years, 11 months ago) by ganis
File length: 15908 byte(s)
Diff to previous 26381
   TProofDraw
     - Do not try to get "htemp" from the input list; this removes a useless warning

   TProofPlayer
     - Improve the treatment of histogram merging in case of small statistics, i.e. when
       the autobinning is not or only partially active. The largest stat histogram is
       taken as base for the merging. This is implemented both for output and feedback
       histograms, and avoids the large axis boundaries variations observed in such cases.

Revision 26381 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sat Nov 22 17:15:24 2008 UTC (6 years, 2 months ago) by ganis
File length: 15859 byte(s)
Diff to previous 25970
  - Fixes for the asynchronous mode:
    - Fully localize the general handling of the input messages into two new methods
      TProof::HandleInputMessage(TMessage *) and TProofServ::HandleSocketInput(TMessage *);
      these methods are callable form any place that needs to intercept some messages and
      do something specific on them (examples are: TProofServ::GetNextPacket, TXProofServ::Get,
      TPacketizerAdaptive::ValidateFiles, ...); this allows to remove several duplications
      and to make sure that no message is lost or wrongly dispatched.
    - Simplify the recursive infrastructure for TProofServ::HandleSocketInput; in particular,
      TProofServ::HandleSocketInputDuringProcess is removed as it is a special case of the
      HandleSocketInput, saving duplications.
    - Always use a kPROOF_CHECKFILE message in replies to check file operations; failures were
      signaled with kPROOF_FATAL which may have some undesired side-effects, depending on the
      timing
    - Add support for one level of recursivity in TProof::Collect .
    - Fix a problem with TProof::Finalize when called with default arguments (on the last query)
    - In TProof::SendFile, send to unique workers only in the "cache" option is specified
    - Remove the call to Finalize for DrawSelect queries, as it is done via the feedback
      mechanisms  

  - Additions/fixes in test/stressProof.cxx:
    - New test for the asynchronous mode;
    - Fine-tuning of the progress display in batch mode;

Revision 25970 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Oct 27 12:11:26 2008 UTC (6 years, 2 months ago) by ganis
File length: 15867 byte(s)
Diff to previous 25918
   - Fine-tune output redirection so that the text progress appears on the screen when running
     PROOF-Lite in batch mode.

Revision 25918 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Oct 22 15:00:04 2008 UTC (6 years, 3 months ago) by ganis
File length: 15650 byte(s)
Diff to previous 25827
   First version of PROOFLite, a 2-tier realization of PROOF intended for multi-core
   machines. In PROOFLite the client start directly the workers; no daemon is required.

   Currently this works on Unix only. Some issues with the internal connection still need
   to be sorted out on Windows.

   To start a session just use

            TProof::Open("")  or TProof::Open("lite")

   From there on everything should be as in normal PROOF. To start a standard PROOF
   session (i.e. via daemons) on the localhost use TProof::Open("localhost").
   (The default meaning of "" in TProof::Open() is controlled via the rc value
    'Proof.LocalDefault').

   To run stressProof with PROOFLite pas the argument 'lite':

            ./stressProof lite

   By default, PROOFLite sets the number of workers equal to the number of cores.
   The number of workers is controlled by the rc variable 'ProofLite.Workers'; it can
   also be set on the fly with the option "workers=<n>", e.g. TProof::Open("lite?workers=2").

   This patch also introduces the rc variable 'Proof.Sandbox' to control the local location
   of the sandboxes. The default has been changed from "~/proof" to "~/.proof" to avoid
   interferences with possible users' working areas. The sandbox contains the directory
   'packages' (with the clients packages; used by standard PROOF and PROOF Lite),
   and some directories used only by PROOFLite: 'cache' (binary cache) and 'datasets' with 
   the datasets. The location of these directories is controlled by the rc variables
   'Proof.PackageDir', 'Proof.CacheDir' and 'Proof.DataSetDir'.
   The working directory for PROOFLite is in the form workdir=<sandbox>/path-to-current-dir where
   ~/path/to/current/dir is the directory from where the session in launched. The directory
   <workdir> contains the session working dirs (in the form 'session-<host>-<time>-<pid>'),
   with the symlink 'last-lite-session' pointing to the last started session, and the 'queries'
   directory, with the results of queries started from the current directory.

Revision 25827 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Oct 15 14:02:59 2008 UTC (6 years, 3 months ago) by ganis
File length: 15457 byte(s)
Diff to previous 25273
   From Jan:

   - Added the possibility to handle removed workers and partly processed packets. When a worker is stopped
     while processing a packet it finishes, the current event and the rest of the packet is reassigned to another
     worker. This is done via two interfaces:
       - TVirtualPacketizer::AddProcessed(TSlave *sl, TProofProgressStatus *st, TList **)
       - TVirtualPacketizer::ReassignPacket.
   - New class TProofProgressStatus used to keep the query progress stauts in all the TProofPlayer objects and in
     TPacketizerAdaptive::TSlaveStat. This class is also used to structure the relevant information send in
     kPROOF_GETPACKET and kPROOF_STOPPROCESS messages.
   - The class TPacketizerProgressive is removed completely.
   - The PROOF protocol version is increased to 19: this is to handle the changes in the kPROOF_STOPPROCESS and
     kPROOF_GETPACKET messages in Master - worker communication.

Revision 25273 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Aug 27 08:56:06 2008 UTC (6 years, 4 months ago) by rdm
File length: 15184 byte(s)
Diff to previous 25105
From Jan:
- A new optional version of PROOF with dynamic worker startup.
  It can be enabled by the admin or a user with 'Proof.DynamicStartup'.
  A session starts only on the master. When a query processing starts
  at the master TXProofServ::GetWorkers() is called. It receives a 
  list of machines from the scheduler and the workers on the
  machines are started. The environment is copied from the master
  to the workers. It includes: the include and dynamic library paths,
  the set of enabled packages as well as the macros loaded by the user.

- A new method TProof::AddWorkers(TList *workers) was added. It adds
  the workers just before the query.

- A packet resubmitting mechanism. When a worker dies all the packets
  that it processed are resubmitted.

- In TPacketizerAdaptive: fixing initialization of fgMaxSlaveCnt. By
  default it was initialized twice.

Revision 25105 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sun Aug 10 07:52:54 2008 UTC (6 years, 5 months ago) by ganis
File length: 15056 byte(s)
Diff to previous 24779
Minimal changes to make possible to initialize TProofPlayerLocal in a local session

Revision 24779 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Jul 10 23:19:12 2008 UTC (6 years, 6 months ago) by rdm
File length: 14865 byte(s)
Diff to previous 23310
remove useless "!!!cleanup!!!" message to syslog.

Revision 23310 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Apr 18 10:22:33 2008 UTC (6 years, 9 months ago) by ganis
File length: 14908 byte(s)
Diff to previous 22635
Fix a weird mem leak due to missing brackets in a nested if-else construct

Revision 22635 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Mar 13 10:50:20 2008 UTC (6 years, 10 months ago) by rdm
File length: 14872 byte(s)
Diff to previous 22583
move all PROOF related libraries under the new proof directory.

Revision 22583 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Mar 11 07:16:50 2008 UTC (6 years, 10 months ago) by brun
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 14872 byte(s)
Diff to previous 21467
From Gerri:
        - TXNetFile: fix warning when compiling with an external XROOTD
        - XrdProofSched: calculating the number of busy CPUs on the base of all started sessions
           (instead of the sessions in running state).
        - TProofPlayer.h: fix some descriptions
        - Make SavePerfInfo.C compilable

Revision 21467 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Dec 18 17:44:44 2007 UTC (7 years, 1 month ago) by ganis
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 14197 byte(s)
Diff to previous 20882
Add/correct doc statements where needed

Revision 20882 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Nov 19 11:31:26 2007 UTC (7 years, 2 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 13822 byte(s)
Diff to previous 20682
Set property svn:eol-style LF on all source and Makefiles. This should avoid
problems with Win32 line endings ending up in the repository. All MS tools
support LF eols fine.

Revision 20682 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Nov 6 15:51:59 2007 UTC (7 years, 2 months ago) by ganis
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 13822 byte(s)
Diff to previous 20307
  Import branches/dev/proof r20654

  Summary:

  + Improvements
    - Add support for SSH SOCKS4 tunnelling; the local port for the tunnel can be specified
      in the master URL, e.g. TProof::Open("master/?tunnel:8000")
    - Add the possibility to plot the estimated instantaneous rate
    - Add "PROOF_ForceLocal" parameter to the !TPacketizerAdaptive; if set to 1, all the data
      are processed locally.
    - Add support for remote grep functionality while retrieving logs (needed by the forthcoming
      memory monitor)
  + Bug fixes
    - Several small fixes to revive the multi-master mode.
    - XrdProofdProtocol:
      - add missing lock to the client instance in SendMsg to avoid screwing up requests from
        workers on the same machine
         - lock the mutex of the requester when setting prorities
         - add notification during Reset
         - fix problem with the detection of the 'allow' directive
         - fix problem with the parsing of the return value from XrdProofServProxy::TerminateProofServ()
         - always use the effective user to retrieve info fom another server
           (XrdProofdManager::GetProofConn is now used)
         - fix possible dead-locks from debug notifications done after hard-killing a session
         - re-enable the garbage collector thread of the connection manager in XrdProofConn to
           fix a problem with closing physical connections;
         - fix a problem with !CleanupProofServ in the case of a non-privileged daemon running
           in multi-user mode
         - introduce a timeout when waiting for the startup of a 'proofserv'.
    - XrdProofConn: init mutex in the ctor; lock in SendRecv
    - XrdProofSched: add support for using the priorities defined in the group manager to define
      the number of workers for sessions
      - TProof:
         - Broadcast priorities to unique nodes only
         - timeout after 5 mins the initial Collect to avoid clients getting stuck at this stage
         - add support for generic timeout in Collect (disabled by default)
         - fix a problem with SendFile.
      - TXProofServ: add a call to !TProof::InterruptCurrentMonitor() in Terminate() to stop 
        infinite loops in Collect
      - TXSocket:
         - Implement a flag to interrupt a TXSocket while waiting for messages
         - Split the session creation timeout in 4 attempts: the total timeout is the same but
           it may circumvent occasional forking problems.
      - XROOTD: 
         - fix a potential (possibly accademic) memory leak in the client
         - fix a access permission problem with Kerberos ticket forwarding
         - fix bug preventing 'locate' to work properly
         - re-enable optimized 'locate'

Revision 20307 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Oct 11 10:58:50 2007 UTC (7 years, 3 months ago) by ganis
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 12757 byte(s)
Diff to previous 19826
  Import branches/dev/proof r20306

  Summary (see branch logs for more details):

  - Set of changes related to CPU quota control including
    - broadcast of centrally determined priorities
    - mechanism to renice processes in a quantitative way
  - New packetizer for non-tree based analysis and related API in TProof (from L. Tran-Thanh)
  - Support for merging output objects saved in files on the workers (from L. Tran-Thanh and me)
  - Improve version binary compatibility checks using also the SVN revision number
   (when available) to define the running version.
  - Extend the version binary compatibility checks also to the cached selector binaries.
  - Extend TDSet::Lookup so that in case of missing files, it can remove them from the
    dataset (option removeMissing must be set).
  - Move the data set lookup to the TProofPlayerRemote::Process.
  - Handle properly the case of incomple datasets: if the file is not found in the lookup
    don't try to validate it; add it, instead, to a 'missingFiles' list returned in the
    output list (fixing bug #28800 in Savannah).

Revision 19826 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Sep 19 19:56:11 2007 UTC (7 years, 4 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 12555 byte(s)
Diff to previous 19825
imported svn:keywords Id property

Revision 19825 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Sep 19 19:49:10 2007 UTC (7 years, 4 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 12607 byte(s)
Diff to previous 19335
remove :$ from tag line

Revision 19335 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Jul 13 13:22:57 2007 UTC (7 years, 6 months ago) by ganis
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 12617 byte(s)
Diff to previous 19314
From Anna:
- Add support for entry-lists to PROOF.
From me:
- Cleanup usage of TEntryList/TEventList arguments in the TProof/TProofPlayer API.

Revision 19314 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Jul 11 15:35:05 2007 UTC (7 years, 6 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 12703 byte(s)
Diff to previous 19249
From Gerri:
put the PROOF Draw() related methods in the TProofDraw classes in an
own library libProofDraw outside libProofPlayer. This reduces considerably
the memory footprint in case Selectors don't use canvas related methods.

Revision 19249 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Jul 3 16:26:44 2007 UTC (7 years, 6 months ago) by ganis
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 12479 byte(s)
Diff to previous 18941
Improve determination of the query initialization time.

Revision 18941 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Jun 5 05:47:25 2007 UTC (7 years, 7 months ago) by ganis
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 12323 byte(s)
Diff to previous 18829
  TAdaptivePacketizer (from Jan)
   - New version featuring better performance and robustness.
     Changes include:
     - simplified and better metric for finding the most loaded file node;
     - adding a new static member for defining whether network bandwidth or
       hard disk transfer is greater (TAdaptivePacketizer::fgNetworkFasterThanHD);
     - it can be set via "proof->SetParameter("PROOF_NetworkFasterThanHD", (Long_t)0);";
     - fixing the confusion with fProcTime, used also in the progress information;
     - adding TAdaptivePacketizer::fCumProcTime;
     - improvements in comments.

  TProofPlayer.cxx, .h
   - Remove a few unused includes
   - Make TAdaptivePacketizer the default
   - Improve comments

Revision 18829 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon May 21 00:22:51 2007 UTC (7 years, 8 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 11571 byte(s)
Diff to previous 18327
use TProof::GetParameter() utility to get parameters from the input list.

Revision 18327 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Mar 19 01:36:56 2007 UTC (7 years, 10 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 11571 byte(s)
Diff to previous 18320
more libProof restructuring. Now libProof is also independent of libHist.
Last remaining step is to remove libTree dependency. On startup proofserv.exe
now loads 9 libs, was 16.

Revision 18320 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sat Mar 17 18:04:02 2007 UTC (7 years, 10 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 11345 byte(s)
Diff to previous 18301
move TProofDraw to proofplayer. Also move the TProofServ GetTreeHeader code
to proofplayer.

Revision 18301 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Mar 16 17:06:19 2007 UTC (7 years, 10 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 11910 byte(s)
Diff to previous 17585
first step in important restructuring of proofserv.exe to do the same as was
done with root.exe. At startup proofserv.exe should depend only on the MAINLIBS
and not much else.

Revision 17585 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Jan 30 16:34:54 2007 UTC (7 years, 11 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 12059 byte(s)
Diff to previous 16870
From Gerri:
- call TSystem::DispatchOneEvent() via an asynchronous timer
  (period 1s) instead of at the end of each processed event;
  this reduces considerably the overhead when processing many
  CPU-light events
- remove some left-over hardcoded settings of the feedback period
- cleanup the stop/abort timer infrastructure

Revision 16870 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Nov 22 14:16:55 2006 UTC (8 years, 2 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 11829 byte(s)
Diff to previous 16775
remove a bunch of warnings about Long64_t being truncated to Int_t.

Revision 16775 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Nov 15 17:45:55 2006 UTC (8 years, 2 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 11817 byte(s)
Diff to previous 15857
From Gerri and Bertrand:
Fixes and new features developed in preparation for the Intel quad-core
launch demo:
- Add instantaneous event and MB processing rates in the Progress signal
  and make necessary modification to exploit it in the Progress Dialog.
- Add possibility to change the Progress and Feedback periods
- A number of small improvements and bug fixes

Revision 15857 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Jul 26 14:28:59 2006 UTC (8 years, 6 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 11004 byte(s)
Diff to previous 15849
From Gerri:
TVirtualProof.h
  - Add abstract methods methods to communicate the data set status
    (signal + sender)

pmain.cxx
  - Add call to Exit(0). T(X)ProofServ::Terminate() just interrupt the event
    loop (insteadof exiting) to avoid problems when the terminate is run in
    a separate thread (like in xproofd).

proof/inc/TProof.h
  - Add methods methods to communicate the data set status (signal + sender)
  - Increase protocol number to 11 (new merging technology)

proof/inc/TProofPlayer.h, .cxx
  - Implement new merging technology for the output list to reduce memory
    consumption: add methods to directly merge received objects in the
    output list
  - Polish cleaning of the various temporary lists and  objects

TProofServ.h, .cxx
  - Implement new merging technology for the output list to reduce memory
    consumption: objects are sent one-by-one
  - Do not scan for previous query results at Setup (do it when the client
    requests for it first); thsi allows to speed up start-up when a large number
    of queries is present and also allos to avoid warning due dictionaries
    not yet loaded.
  - Add method for a global cleanup of old query results.
  - Keep in memory only meta information about processed queries (i.e. not the
    output list) to avoid large memory consumption in case of large
    output lists
  - In Terminate, change the call to Exit(0) to ExitLoop() ; Exit(0) is
    called in pmain.cxx .

TProof.cxx
  - Implement new merging technology for the output list to reduce memory
    consumption: objects are received one-by-one and directly merged in the
    output list
  - Add protection in parallel startup against invelid slaves.
  - Remove check on directory existence in UploadDataSet.
  - Add methods to handle notifications from data set verification
  - Fix inconsistency with notification in activation/deactivation
    of non-existing workers

TXProofServ.cxx
  - In Terminate, change the call to Exit(0) to ExitLoop() ; Exit(0) is
    called in pmain.cxx.

TQueryResult.cxx
  - Make a owned copy of the input list (instead of just storing the pointer) to
    allow proper cleaning.

Revision 15849 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Jul 26 13:36:44 2006 UTC (8 years, 6 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 10629 byte(s)
Diff to previous 15507
Instead of providing default implementations for copy ctor's and assignment
operators we better make them private (and not implemented) to avoid people
from accidentally using them. Especially the collections classes and the
TQObject derived classes. In these classes the default implementations would
cause havoc due to multiple deletions of the same objects and other
potential memory corruptions.

Revision 15507 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Jun 21 16:18:27 2006 UTC (8 years, 7 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 10765 byte(s)
Diff to previous 15138
From Gerri:
The patch implements the following new functionality:

  - Shutdown of idle sessions when quitting ROOT; if not idle the session
    is shutdown as soon as processing terminates; it is possible to delay
    the automatic shutdown by a tunable amount of time via a new directive
    in the xrootd config file 'xpd.shutdown' (see xpd.cf.example).

  - New static method TProof::Reset(const char *url) to hard-cleanup an
    user area (implemented for Xproofd only); this implied quite some
    additions in XrdProofdProtocol.

  - Possibility to temporary remove / re-insert a worker from the active list
       TProof::DeactivateWorker(const char *ord)
       TProof::ActivateWorker(const char *ord)
    with 'ord' the ordinal numer, e.g. "0.3" .

  - A default url for the disk pool
       const char *TVirtualProof::GetDataPoolUrl()
    It is set by default to a string send back by the master, but it can
    be changed via
       void  TVirtualProof::SetDataPoolUrl(const char *url)

    Jan should then adapt his code to use this as default.

  - Posibility to timeout Collect; for now this is only used at forced
    termination.

The patch includes many small changes and bug fixes. One of the most important
is in TXNetSystem (a bug preventing correct treatment of multiple request
to the same redirector).

Revision 15138 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue May 23 07:43:55 2006 UTC (8 years, 8 months ago) by brun
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 10379 byte(s)
Diff to previous 15046
Implement copy constructors and assignement operators required following Federico's changes

Revision 15046 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon May 15 09:45:03 2006 UTC (8 years, 8 months ago) by brun
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 10243 byte(s)
Diff to previous 12829
From Gerri:
Fix coding convention violations in krb5auth, rpdutils and
   proof

Revision 12829 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sat Sep 24 11:33:41 2005 UTC (9 years, 4 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 10228 byte(s)
Diff to previous 12730
From Gerri:
This is the patch for Marek's problem.

Basically, ReinitSelector() does not need to be called if running in
synchronous mode (new argument to TProofPlayer::Finalize()), and the aclic
mode must be recorded and used when reinitialization is needed.

It remains the problem with ::GetSelector that I mentioned in the
previous mail, which affects, for example, cases when one runs once
with "<selec>.C+", modifies <selec>.C and runs again forgetting the
'+'.

Revision 12730 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sat Sep 17 13:55:15 2005 UTC (9 years, 4 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 10184 byte(s)
Diff to previous 12722
From Gerri:
coding convention fixes.

Revision 12722 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Sep 16 08:48:39 2005 UTC (9 years, 4 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 10117 byte(s)
Diff to previous 12561
From Gerri:
Big patch supporting query queueing on the PROOF master, interogation and
interaction with the queue. Retrieval, finalization and archival of query
results and multiple PROOF sessions. Some other fixes:
- TMacro.h, .cxx
  add method Checksum to get the MD5 of the current context
  (needed for optimized selector reinitialization)

- Changes needed to create and fill the list of data sets (TROOT,
  TDSet, TChain, TChainProof)

- A new method TSelector::IsStandardDraw() to check if a selector
  is for standard draw actions

- TEventIter: fix for the missing last update (see above)

- Few protections in TPacketizers destructors

- Fix for code conventions in TPerfStats (the other will follow)

- Occasional missing last update of the progress bar (systematically
  reproduced with SetParallel(1)): all this part is quite involved
  and I have found a solution which does not harm but perhaps is not
  the most elegant.

- SetParallel(0): the progress bar was not working at all in such a
  case, since no packetizer is involved. Still it is useful to see
  that things are not stuck, so I have added some progress messages
  there too. The problem is that I have not found a way to get the total
  number of entries from a data set without doing all the machinery
  done in the packetizer.

Revision 12561 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Aug 30 10:25:29 2005 UTC (9 years, 4 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 9038 byte(s)
Diff to previous 11988
From Gerri:
This is the patch to add support for asynchronous queries.

The principal modifications are in TProof and TProofPlayer.

In TProof, I have separated the parsing of the received messages
from the selection of the socket (new method CollectInputFrom()):
the new method is called directly by TProofInputHandler::Notify()
in the case of asynchronous mode.
Also in TProof I have added a method to redirect the logs to a
temporary file, to avoid having them on the main window while doing
something else. The method is TProof::RedirectLog(), it uses freopen,
fdopen and dup2, and works well back and forth.
The content of the temp file can be displayed into a log window
(new GUI class TProofProgressLog), via a new button in the
progress dialog ("Show Logs").

In TProofPlayer I have created a new method Finalize() with the
final part of Process(): this is necessary to be able to finalize
the queries processed asychronously.

There are several many other small changes and fixes: a couple
of non-backward comaptibilities introduced with the previous
patch, many small issues in the GUI (the stop and abort buttons
were never disabled ...).
Also, in view of multi-sessions, it does not make much sense to
have one progress dialog per query: one per session is enough;
I have added a method ResetProgressDialog() to reset it when a
new query is started.

Finally, as anticipation of the next patch, this patch includes
support for a list of queries created by the master and described
by the new class TProofQuery.
The list is displayed by TProof::ShowQueries().

Revision 11988 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Jun 7 20:28:32 2005 UTC (9 years, 7 months ago) by brun
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 8974 byte(s)
Diff to previous 11717
From Maarten:
Patch fixes several THashList related problems in PROOF.

Revision 11717 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon May 2 11:00:39 2005 UTC (9 years, 8 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 9029 byte(s)
Diff to previous 11449
From Guenter Kickinger:
Add proper stop and cancel feature in PROOF.

Revision 11449 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Mar 30 04:07:29 2005 UTC (9 years, 9 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 8605 byte(s)
Diff to previous 11319
From Maarten:
remove references to gProof.

Revision 11319 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sun Mar 13 15:06:50 2005 UTC (9 years, 10 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 8039 byte(s)
Diff to previous 11281
use EmitVA() for signal/slot of Progress(Long64_t,Long64_t).

Revision 11281 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Mar 10 17:57:05 2005 UTC (9 years, 10 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 8041 byte(s)
Diff to previous 11047
From Marek Biskup:
major new PROOF developments:
 - access to PROOF directly via the TChain interface (creating a TDSet
   in the background).
 - access to PROOF based trees via the TTreeViewer
 - fully implemented the TChain/TDSet::Draw() interface.
 - PROOF sessions can now be browsed via the TBrowser

Revision 11047 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Feb 7 18:02:37 2005 UTC (9 years, 11 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 7911 byte(s)
Diff to previous 9615
From Maarten and Kristjan Gulbrandsen:
Here is the patch with the restructured PROOF code. It parameterizes
PROOF startup, currently implementing "Standard", Condor, MLM and PEAC.
It can easily be extended with other "TProof" classes, e.g. AliEn Proof.
Or what ever next Grid implementation or scheduler comes along.

The MLM, Multi Level Master, code uses a basic packetizer, based on a
MSD (massstorage domain) tag. In MLM mode we have multi-tier client
server setup, like:

client -> super master -> sub master 0 -> slave[0..N]
                          ...
                       -> sub master M -> slave[0..N]

It also contains a large number of smaller fixes. Credit for most of
the work goes to Kris!

We have tested this on multipe platforms (RH7.3, RH9, SL3) and in
multiple configurations. Should be stable.

Revision 9615 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Jul 29 10:54:55 2004 UTC (10 years, 5 months ago) by brun
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 6524 byte(s)
Diff to previous 9459

*****************WARNING*****************
With this mega patch, we introduce support for TTree/TChain  with more than
2 billion entries. Several class data members have been changed from
Int_t (or Stat_t) to Long64_t.
==>Trees written with this new version cannot be read with older versions
******************************************

TBranch:
========
 - replace the members with type Int_t or Stat_t by Long64_t
    Long64_t    fEntryNumber;     //  Current entry number (last one filled in this branch)
    Long64_t    fReadEntry;       //! Current entry number when reading
    Long64_t    fEntries;         //  Number of entries
    Long64_t    fTotBytes;        //  Total number of bytes in all leaves before compression
    Long64_t    fZipBytes;        //  Total number of bytes in all leaves after compression
    Long64_t   *fBasketEntry;     //[fMaxBaskets] Table of first entry in eack basket

 - corresponding changes in the member functions
    virtual Int_t     GetEntry(Long64_t entry=0, Int_t getall = 0);
    virtual Int_t     GetEntryExport(Long64_t entry, Int_t getall, TClonesArray *list, Int_t n);
            Int_t     GetEvent(Long64_t entry=0) {return GetEntry(entry);}
            Long64_t *GetBasketEntry() const {return fBasketEntry;}
            Long64_t  GetReadEntry()   const {return fReadEntry;}
            Long64_t  GetTotalSize()   const;
            Long64_t  GetTotBytes()    const {return fTotBytes;}
            Long64_t  GetZipBytes()    const {return fZipBytes;}
            Long64_t  GetEntryNumber() const {return fEntryNumber;}
            Long64_t  GetEntries()     const {return fEntries;}

TBranch::Streamer has been modified to read old files and automatically translate
the old types to the new types.
The new version of Streamer uses the TClass::ReadBuffer/WriteBuffer.

TBranch::Print has been modified to take into account the new data types.

ClassDef version increased to 10.

TBranchClones:
==============
    virtual Int_t    GetEntry(Long64_t entry=0, Int_t getall = 0);

TBranchElement:
===============
            Int_t    GetEntry(Long64_t entry=0, Int_t getall = 0);

TBranchObject:
==============
    virtual Int_t    GetEntry(Long64_t entry=0, Int_t getall = 0);

TChain:
=======
    Long64_t    *fTreeOffset;       //[fTreeOffsetLen]Array of variables

    virtual Int_t     Add(const char *name, Long64_t nentries=kBigNumber);
    virtual Int_t     AddFile(const char *name, Long64_t nentries=kBigNumber);
    virtual Long64_t  Draw(const char *varexp, const TCut &selection, Option_t *option=""
                       ,Long64_t nentries=kBigNumber, Long64_t firstentry=0);
    virtual Long64_t  Draw(const char *varexp, const char *selection, Option_t *option=""
                     ,Long64_t nentries=kBigNumber, Long64_t firstentry=0); // *MENU*
    virtual Long64_t  GetChainEntryNumber(Long64_t entry) const;
    virtual Long64_t  GetEntries() const;
    virtual Int_t     GetEntry(Long64_t entry=0, Int_t getall=0);
            Long64_t  LoadTree(Long64_t entry);
    virtual void      Loop(Option_t *option="",Long64_t nentries=kBigNumber, Long64_t firstentry=0); // *MENU*
    virtual Long64_t  Merge(const char *name);
    virtual Long64_t  Merge(TCollection *list);
    virtual Long64_t  Merge(TFile *file, Int_t basketsize, Option_t *option="");
    virtual Long64_t  Process(const char *filename,Option_t *option="", Long64_t nentries=kBigNumber, Long64_t firstentry=0); // *MENU*
    virtual Long64_t  Process(TSelector *selector,Option_t *option="",  Long64_t nentries=kBigNumber, Long64_t firstentry=0);

ClassDef version changed from 4 to 5

TSelector
=========
   virtual Bool_t      ProcessCut(Long64_t /*entry*/) { return kTRUE; }
   virtual void        ProcessFill(Long64_t /*entry*/) { }
   virtual Bool_t      Process(Long64_t /*entry*/) { return kFALSE; }

TSelectorCint
=============
   virtual Bool_t      ProcessCut(Long64_t entry);
   virtual void        ProcessFill(Long64_t entry);
   virtual Bool_t      Process(Long64_t entry);

TSelectorDraw
=============
    Long64_t       fDraw;           //! Last entry loop number when object was drawn
    Long64_t       fSelectedRows;   //  Number of selected entries
    Long64_t       fOldEstimate;    //  value of Tree fEstimate when selector is called
    Double_t      *fV1;             //![fSelectedRows]Local buffer for variable 1
    Double_t      *fV2;             //![fSelectedRows]Local buffer for variable 2
    Double_t      *fV3;             //![fSelectedRows]Local buffer for variable 3
    Double_t      *fV4;             //![fSelectedRows]Local buffer for variable 4
    Double_t      *fW;              //![fSelectedRows]Local buffer for weights

    virtual Long64_t  GetSelectedRows() const {return fSelectedRows;}
    virtual Bool_t    Process(Long64_t /*entry*/) { return kFALSE; }
    virtual void      ProcessFill(Long64_t entry);
    virtual void      ProcessFillMultiple(Long64_t entry);
    virtual void      ProcessFillObject(Long64_t entry);
    virtual void      SetEstimate(Long64_t n);

TTree
=====
Modified data types
    Long64_t       fEntries;           //  Number of entries
    Long64_t       fTotBytes;          //  Total number of bytes in all branches before compression
    Long64_t       fZipBytes;          //  Total number of bytes in all branches after compression
    Long64_t       fSavedBytes;        //  Number of autosaved bytes
    Long64_t       fMaxEntryLoop;      //  Maximum number of entries to process
    Long64_t       fMaxVirtualSize;    //  Maximum total size of buffers kept in memory
    Long64_t       fAutoSave;          //  Autosave tree when fAutoSave bytes produced
    Long64_t       fEstimate;          //  Number of entries to estimate histogram limits
    Long64_t       fChainOffset;       //! Offset of 1st entry of this Tree in a TChain
    Long64_t       fReadEntry;         //! Number of the entry being processed
    Long64_t       fTotalBuffers;      //! Total number of bytes in branch buffers
    Long64_t       fDebugMin;          //! First entry number to debug
    Long64_t       fDebugMax;          //! Last entry number to debug

New function signatures
    virtual Long64_t     AutoSave(Option_t *option="");
    virtual TTree       *CloneTree(Long64_t nentries=-1, Option_t *option="");
    virtual Long64_t     CopyEntries(TTree *tree, Long64_t nentries=-1);
    virtual TTree       *CopyTree(const char *selection, Option_t *option=""
                          ,Long64_t nentries=1000000000, Long64_t firstentry=0);
    virtual Long64_t     Draw(const char *varexp, const TCut &selection, Option_t *option=""
                          ,Long64_t nentries=1000000000, Long64_t firstentry=0);
    virtual Long64_t     Draw(const char *varexp, const char *selection, Option_t *option=""
                          ,Long64_t nentries=1000000000, Long64_t firstentry=0); // *MENU*
    virtual Long64_t     Fit(const char *funcname ,const char *varexp, const char *selection="",Option_t *option="" ,Option_t *goption=""
                          ,Long64_t nentries=1000000000, Long64_t firstentry=0); // *MENU*
    virtual Long64_t     GetChainEntryNumber(Long64_t entry) const {return entry;}
    virtual Long64_t     GetChainOffset() const { return fChainOffset; }
            Long64_t     GetDebugMax()  const {return fDebugMax;}
            Long64_t     GetDebugMin()  const {return fDebugMin;}
    virtual Long64_t     GetEntries() const   {return fEntries;}
    virtual Long64_t     GetEntriesFast() const   {return fEntries;}
    virtual Long64_t     GetEntriesFriend() const;
    virtual Long64_t     GetEstimate() const { return fEstimate; }
    virtual Int_t        GetEntry(Long64_t entry=0, Int_t getall=0);
            Int_t        GetEvent(Long64_t entry=0, Int_t getall=0) {return GetEntry(entry,getall);}
    virtual Long64_t     GetEntryNumberWithBestIndex(Int_t major, Int_t minor=0) const;
    virtual Long64_t     GetEntryNumberWithIndex(Int_t major, Int_t minor=0) const;
    virtual Long64_t     GetEntryNumber(Long64_t entry) const;
    virtual Long64_t     GetMaxEntryLoop() const {return fMaxEntryLoop;}
    static  Long64_t     GetMaxTreeSize();
    virtual Long64_t     GetMaxVirtualSize() const {return fMaxVirtualSize;}
    virtual Long64_t     GetReadEntry()  const {return fReadEntry;}
    virtual Long64_t     GetReadEvent()  const {return fReadEntry;}
    virtual Long64_t     GetSelectedRows() {return GetPlayer()->GetSelectedRows();}
    virtual Long64_t     GetTotBytes() const {return fTotBytes;}
    virtual Long64_t     GetZipBytes() const {return fZipBytes;}
    virtual Long64_t     LoadTree(Long64_t entry);
    virtual Long64_t     LoadTreeFriend(Long64_t entry, TTree *T);
    virtual Long64_t     Merge(TCollection *list);
    TPrincipal          *Principal(const char *varexp="", const char *selection="", Option_t *option="np"
                                   ,Long64_t nentries=1000000000, Long64_t firstentry=0);
    virtual Long64_t     Process(const char *filename,Option_t *option="", Long64_t nentries=1000000000, Long64_t firstentry=0); // *MENU*
    virtual Long64_t     Process(TSelector *selector, Option_t *option="", Long64_t nentries=1000000000, Long64_t firstentry=0);
    virtual Long64_t     Project(const char *hname, const char *varexp, const char *selection="", Option_t *option=""
                          ,Long64_t nentries=1000000000, Long64_t firstentry=0);
    virtual TSQLResult  *Query(const char *varexp="", const char *selection="", Option_t *option=""
                          ,Long64_t nentries=1000000000, Long64_t firstentry=0);
    virtual Long64_t     Scan(const char *varexp="", const char *selection="", Option_t *option=""
                          ,Long64_t nentries=1000000000, Long64_t firstentry=0); // *MENU*
    virtual void         SetDebug(Int_t level=1, Long64_t min=0, Long64_t max=9999999); // *MENU*
    virtual void         SetEntries(Long64_t n);
    virtual void         SetEstimate(Long64_t nentries=10000);
    virtual void         SetMaxEntryLoop(Long64_t maxev=1000000000) {fMaxEntryLoop = maxev;} // *MENU*
    static  void         SetMaxTreeSize(Long64_t maxsize=1900000000);
    virtual void         SetMaxVirtualSize(Long64_t size=0) {fMaxVirtualSize = size;} // *MENU*
    virtual void         Show(Long64_t entry=-1, Int_t lenmax=20);
    virtual Long64_t     UnbinnedFit(const char *funcname ,const char *varexp, const char *selection="",Option_t *option=""
                          ,Long64_t nentries=1000000000, Long64_t firstentry=0);

TTree::Streamer has been modified to real old files.
TTree::Print has been modified to take into account the new data types.

ClassDef version number incremented to 13.

TVirtualTreePlayer
==================
    virtual TTree         *CopyTree(const char *selection, Option_t *option=""
                            ,Long64_t nentries=1000000000, Long64_t firstentry=0) = 0;
    virtual Long64_t       DrawScript(const char* wrapperPrefix,
                                      const char *macrofilename, const char *cutfilename,
                                      Option_t *option, Long64_t nentries, Long64_t firstentry) = 0;
    virtual Long64_t       DrawSelect(const char *varexp, const char *selection, Option_t *option
                            ,Long64_t nentries, Long64_t firstentry) = 0;
    virtual Long64_t       Fit(const char *formula ,const char *varexp, const char *selection,Option_t *option ,Option_t *goption
                            ,Long64_t nentries, Long64_t firstentry) = 0;
    virtual Long64_t       GetSelectedRows() const = 0;
    virtual TPrincipal    *Principal(const char *varexp="", const char *selection="", Option_t *option="np"
                           ,Long64_t nentries=1000000000, Long64_t firstentry=0) = 0;
    virtual Long64_t       Process(const char *filename,Option_t *option="", Long64_t nentries=1000000000, Long64_t firstentry=0) = 0;
    virtual Long64_t       Process(TSelector *selector,Option_t *option="",  Long64_t nentries=1000000000, Long64_t firstentry=0) = 0;
    virtual Long64_t       Scan(const char *varexp, const char *selection, Option_t *option
                            ,Long64_t nentries, Long64_t firstentry) = 0;
    virtual TSQLResult    *Query(const char *varexp, const char *selection, Option_t *option
                            ,Long64_t nentries, Long64_t firstentry) = 0;
    virtual void           SetEstimate(Long64_t n) = 0;
    virtual Long64_t       UnbinnedFit(const char *formula ,const char *varexp, const char *selection,Option_t *option
                            ,Long64_t nentries, Long64_t firstentry) = 0;

TTreeFormula
============
   TLeaf*      GetLeafWithDatamember(const char* topchoice, const char* nextchice, Long64_t readentry) const;
   Bool_t      BranchHasMethod(TLeaf* leaf, TBranch* branch,
                               const char* method,const char* params,
                               Long64_t readentry) const;

TTreePlayer
===========
    Long64_t       fSelectedRows;    //  Number of selected entries

    virtual TTree    *CopyTree(const char *selection, Option_t *option
                       ,Long64_t nentries, Long64_t firstentry);
    virtual Long64_t  DrawScript(const char* wrapperPrefix,
                                 const char *macrofilename, const char *cutfilename,
                                 Option_t *option, Long64_t nentries, Long64_t firstentry);
    virtual Long64_t  DrawSelect(const char *varexp, const char *selection, Option_t *option
                                 ,Long64_t nentries, Long64_t firstentry);
    virtual Long64_t  Fit(const char *formula ,const char *varexp, const char *selection,Option_t *option ,
                          Option_t *goption ,Long64_t nentries, Long64_t firstentry);
    virtual Long64_t  GetSelectedRows() const {return fSelectedRows;}
    TPrincipal       *Principal(const char *varexp, const char *selection, Option_t *option
                       ,Long64_t nentries, Long64_t firstentry);
    virtual Long64_t  Process(const char *filename,Option_t *option, Long64_t nentries, Long64_t firstentry);
    virtual Long64_t  Process(TSelector *selector,Option_t *option,  Long64_t nentries, Long64_t firstentry);
    virtual Long64_t  Scan(const char *varexp, const char *selection, Option_t *option
                       ,Long64_t nentries, Long64_t firstentry);
    virtual TSQLResult *Query(const char *varexp, const char *selection, Option_t *option
                         ,Long64_t nentries, Long64_t firstentry);
    virtual void      SetEstimate(Long64_t n);
    virtual Long64_t  UnbinnedFit(const char *formula ,const char *varexp, const char *selection,Option_t *option
                       ,Long64_t nentries, Long64_t firstentry);

The generated code by TTreePlayer::MakeCode, MakeClass, MakeProxy
uses Long64_t instead of Int_t

TTreeViewer
===========
    Long64_t      Process(const char* filename, Option_t *option="", Long64_t nentries=1000000000, Long64_t firstentry=0); // *MENU*
    void          SetCurrentRecord(Long64_t entry);

THbookBranch
============
    virtual Int_t    GetEntry(Long64_t entry=0, Int_t getall=0);
    virtual void     SetEntries(Long64_t n) {fEntries=n;}

THbookTree
==========
    virtual Int_t    GetEntry(Long64_t entry=0, Int_t getall=0);
    virtual void     InitBranches(Long64_t entry);
    virtual void     SetEntries(Long64_t n);

TProofDraw
==========
    virtual Bool_t   Process(Long64_t /*entry*/);

TProofPlayer
============
    virtual Long64_t  Process(TDSet *set,
    virtual Long64_t  DrawSelect(TDSet *set, const char *varexp,
    Long64_t          Process(TDSet *set, const char *selector,
    Long64_t          DrawSelect(TDSet *set, const char *varexp,

Revision 9459 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Jul 9 01:34:51 2004 UTC (10 years, 6 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 6521 byte(s)
Diff to previous 9322
From Maarten:
- Implement TStatus to hold and communicate the status between players
  (master and slave) and selectors
- Use of TStatus in player
- Use TStatus in TProofDraw and cleanup

Revision 9322 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Jun 25 17:27:09 2004 UTC (10 years, 7 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 6441 byte(s)
Diff to previous 9182
From Maarten:
- new class TParameter
- new class TDrawFeedback
- fixes to the feedback of performance histograms
- selection of the packetizer
- cleanups

Example commands:

gROOT->Proof()
... package setup ...

TDSet *d = make_tdset(2)

gEnv->SetValue("Proof.StatsTrace",1)
gEnv->SetValue("Proof.StatsHist",1)
TList *fb = new TList
fb->SetName("FeedbackList")
fb->Add(new TObjString("EventsHist"))
fb->Add(new TObjString("CpuTimeHist"))
TDrawFeedback *dfb = new TDrawFeedback((TProof*)gProof)
gProof->AddInput(fb)
d->Process("EventTree_Proc.C","")

Revision 9182 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sun Jun 13 16:26:36 2004 UTC (10 years, 7 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 6328 byte(s)
Diff to previous 8962
From Maarten and Kristjan Gulbrandsen:
PROOF and file I/O performance monitoring framework. The TVirtualPerfStats
abstract class allows adding monitoring events in base classes (TFile and
TNetfile). The performance monitoring info allows detailed study of PROOF
performance (which files accessed by which slaves, latency, read/write times,
packet size, etc.). The result is send back to the client in the form of a
memory based TTree.

Remove TProofStats.h and TProofStats.cxx, .d, .o from proof/inc and src.

Revision 8962 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue May 18 11:32:49 2004 UTC (10 years, 8 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 6469 byte(s)
Diff to previous 8380
From Maarten and Kristjan Gulbrandsen:
Performance histograms and trace tree for monitoring PROOF peformance.
Enabled via gEnv.

Revision 8380 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Mar 12 14:54:24 2004 UTC (10 years, 10 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 6245 byte(s)
Diff to previous 8366
work around problem accessing protected method from embedded TCleanup class.

Revision 8366 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Mar 11 18:47:00 2004 UTC (10 years, 10 months ago) by brun
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 6174 byte(s)
Diff to previous 8362
A few more updates from Maarten

Revision 8362 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Mar 11 11:02:55 2004 UTC (10 years, 10 months ago) by brun
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 6153 byte(s)
Diff to previous 7664
From Maarten Ballintijn
first patch implementing the most basic 1d histogramming using PROOF.

Revision 7664 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Nov 26 21:48:27 2003 UTC (11 years, 2 months ago) by brun
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 5886 byte(s)
Diff to previous 6519
From Philippe:

This update makes any selector created by the TTreePlayer directly from a source file local to the current TTreePlayer
object (instead of being a static).

It also insures that TProofPlayer and TTreePlayer do not call delete for an object of
a class which library was unloaded (since the code is gone, this would lead to a core
dump).

Revision 6519 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu May 1 17:51:42 2003 UTC (11 years, 8 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 5821 byte(s)
Diff to previous 6421
implementation of TProof::StopProcess() and propagation infrastructure.
By Maarten.

Revision 6421 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Apr 11 11:48:11 2003 UTC (11 years, 9 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 5649 byte(s)
Diff to previous 6321
Big (unsigned) long long support patch. Following provided:
- basic typedefs in Rtypes.h: Long64_t and ULong64_t
- basic Long64 I/O support in TBuffer
- automatic Long64 I/O support in TStreamerInfo
- Long64 byteswap in Bytes.h
- Long64 type handling in classes like TDataMember, TDataType, TCint,
  TROOT, etc
- Removal of obsolete Long64_t typedefs in many PROOF classes
No changes for non-Long64 data types (no backward incompatibilies).
I/O tested for Long64 basic type, and static and dynamic arrays using
handcoded streamers, rootcint generated streamers and automatic StreamerInfo
streamers.

Revision 6321 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Mar 18 14:29:59 2003 UTC (11 years, 10 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 5674 byte(s)
Diff to previous 5944
PROOF patch from Maarten. Introduction of Draw() skeleton.

Revision 5944 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Jan 20 10:25:58 2003 UTC (12 years ago) by brun
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 5205 byte(s)
Diff to previous 5692
*** empty log message ***

Revision 5692 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Nov 28 18:38:12 2002 UTC (12 years, 1 month ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 5172 byte(s)
Diff to previous 5596
latest patch from Maarten, adding support for a list of feedback objects
that can be displayed while the query is running. Also better packet
size algorithm.

Revision 5596 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Nov 15 20:02:56 2002 UTC (12 years, 2 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 5117 byte(s)
Diff to previous 5480
more mods for dynamic feedback.

Revision 5480 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Oct 25 01:23:38 2002 UTC (12 years, 3 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 4288 byte(s)
Diff to previous 5401
this patch implements a signal Progress(total,processed) in the proofplayer.
The patch provides some other small bug fixes and an implementation of
file specs without host.

Revision 5401 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Oct 7 10:43:51 2002 UTC (12 years, 3 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 4041 byte(s)
Diff to previous 5083
added code to trace the latency and cpu time every work packet took.
By Maarten.

Revision 5083 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Aug 7 15:53:54 2002 UTC (12 years, 5 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 3999 byte(s)
Diff to previous 4385
few bug fixes by Maarten.

Revision 4385 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Fri Apr 19 18:24:02 2002 UTC (12 years, 9 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 3981 byte(s)
Diff to previous 4215
Major PROOF updates from both Maarten and me. Coming close to a working
system.

Revision 4215 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Thu Mar 21 16:11:03 2002 UTC (12 years, 10 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 3863 byte(s)
Diff to previous 4167
more PROOF patches from Maarten.

Revision 4167 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Wed Mar 13 01:52:21 2002 UTC (12 years, 10 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 3774 byte(s)
Diff to previous 3962
PROOF synchronization with Maarten. Addition in TProof using std::map.
Might not yet compile on all platforms. Need quick feedback to fix.

Revision 3962 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Feb 12 17:53:18 2002 UTC (12 years, 11 months ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 3722 byte(s)
Diff to previous 3699
PROOF merge with Maarten.

Revision 3699 - (view) (download) (as text) (annotate) - [select for diffs]
Added Fri Jan 18 14:24:10 2002 UTC (13 years ago) by rdm
Original Path: trunk/proofplayer/inc/TProofPlayer.h
File length: 3285 byte(s)
relocate some proof file to remove circular dependencies between the
treeplayer and proof shared libraries. Also rename TPlayer to TProofPlayer
et al. Also remove Begin() from TSelector, use Begin(0) instead. This to
be backward compatible with existing TSelector scripts that don't have
Begin() and therefor give warnings about Begin() being hidden in the base
class.

This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, enter a numeric revision.

  Diffs between and
  Type of Diff should be a

Sort log by:

Subversion Admin
ViewVC Help
Powered by ViewVC 1.0.9