Log of /trunk/proof/proof/inc/TProofProgressStatus.h

---

Revision 44010 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sun Apr 29 23:07:48 2012 UTC (2 years, 8 months ago) by ganis
File length: 4091 byte(s)
Diff to previous 38810

  Several optimizations in the low level PROOF event loop (TProofPlayer::Process):
  - Per-event calls to TEventIter::GetNextEvent are replaced by per-packet calls
    to a new function TEventIter::GetNextPacket and a for loop over the returned
    range;
  - Progress information is filled in on per-packet pace just before asking for
    a new packet, instead of being filled for each event;
  - In TProofProgressStatus, the relevant setters in were inlined, and the (expensive)
    calls to SetLastUpdate() reduced to the minimum required;
  - Status of objects is checked using bits instead of functions calls;
  - Check whether to dump memory info is done before calling CheckMemUsage.
  These optimizations allow to reduce dramatically the overhead introduced by the
  operations PROOF needs to perform during the event loop. A measurement of the
  overhead can be obtained from a very light computational task, for example,
  generating one random number and filling one histogram; executing this task
  within a PROOF-Lite session with 1 worker now takes only 1.8 times the time
  required by a straight loop in the parent ROOT session; the same number before
  was about 13. 

  The patch also introduces a couple of small changes enabling the use of
  TProofPlayerLocal to process selectors in the local session using the PROOF
  iteration mechanism; this is very useful for optimizations studies.

  Finally, a couple of missing protections were added in TProofPlayer.

---

Revision 38810 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Apr 12 16:22:59 2011 UTC (3 years, 9 months ago) by ganis
File length: 4039 byte(s)
Diff to previous 30859

  Patch to correctly honour selector abort status settings in PROOF. Currently only
  the TSelector::kAbortProcess was handled by stopping processing. In particular
  TSelector::kAbortFile was ignored; this recently created some problems in ALICE
  with corrupted files, with repeated attempts to read events eventually leading to
  bad_alloc exceptions.

  This patch also fixes other related issues, in particular with the reporting of the
  non-processed {files, events} in the final 'MissingFiles' list. This list should
  now account much more precisely of the number of events which could not be processed.

  It also fixes a problem with the final update of the progress information affecting
  occasionally cases with skipped events.

---

Revision 30859 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sat Oct 24 14:53:07 2009 UTC (5 years, 3 months ago) by ganis
File length: 3926 byte(s)
Diff to previous 25827

  Patch for improved performance monitoring. The 'Rate Plot' button in the progress
  dialog is renamed 'Performance plot' and shows up to 4 plots with the event/sec,
  the average read chunck size, the number of active workers and the number of active
  PROOF sessions on the cluster, all as a function of processing time.

  The read chunck size plot allows to monitor the usage of the cache.

  The istantaneous processing rate (event/sec) is now better estimate: a few issues
  with the normalizing times have ben solved, removing the artificial structures that
  were observed.

  The possibility to set a max packet time length is introduced (default 30 s); this
  can be changes with the parameter PROOF_MaxPacketTime.
  The size of the cache is also taken into account to optimize the use of the cache.

  The parameter PROOF_UseParallelUnzip has been introduced to toggle the use of the 
  parallel unzip (default off for now).

  A page describing the new performance plots is under preparation at 
           http://root.cern.ch/drupal/content/progress-dialog

---

Revision 25827 - (view) (download) (as text) (annotate) - [select for diffs]
Added Wed Oct 15 14:02:59 2008 UTC (6 years, 3 months ago) by ganis
File length: 2830 byte(s)

   From Jan:

   - Added the possibility to handle removed workers and partly processed packets. When a worker is stopped
     while processing a packet it finishes, the current event and the rest of the packet is reassigned to another
     worker. This is done via two interfaces:
       - TVirtualPacketizer::AddProcessed(TSlave *sl, TProofProgressStatus *st, TList **)
       - TVirtualPacketizer::ReassignPacket.
   - New class TProofProgressStatus used to keep the query progress stauts in all the TProofPlayer objects and in
     TPacketizerAdaptive::TSlaveStat. This class is also used to structure the relevant information send in
     kPROOF_GETPACKET and kPROOF_STOPPROCESS messages.
   - The class TPacketizerProgressive is removed completely.
   - The PROOF protocol version is increased to 19: this is to handle the changes in the kPROOF_STOPPROCESS and
     kPROOF_GETPACKET messages in Master - worker communication.