Log of /trunk/proof/proofplayer/inc/TPacketizer.h

---

Revision 33781 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Jun 8 14:13:39 2010 UTC (4 years, 7 months ago) by ganis
File length: 4278 byte(s)
Diff to previous 30870

   - Optimize the validation step in the case not all the entries are required.
     The validation step is stopped as soon as the requested number of events is reached.
     If the parameter "PROOF_ValidateByFile" is set to 1, the number of files is exactly what
     needed; otherwise the number of files may exceed the number of fles needed by #workers-1
     (this is the default because additional, serial, checks are needed to ensure that only
      the files really required are open).
     This feature was requested in the context of ALICE reconstruction.
     This new feature is used as an example in the "eventproc" tutorial in runProof.C .
   - The patch also fixes a subtle bug affecting the (possibly rare) case when not all entries
     are required and # entries does not correspond to an complete subset of files (e.g.
     # entries = 1001000 with files of 100000 entries each). The effect was uncomplete
     processing (skipped events, magenta bar) or a session freeze.

---

Revision 30870 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Mon Oct 26 15:16:00 2009 UTC (5 years, 2 months ago) by ganis
File length: 4232 byte(s)
Diff to previous 30859

   Use the improved estimation of the current rate also for TPacketizer and TPacketizerUnit.
   Fixes also a problem with the progress bar with these packetizers appeared after the last
   check-in.

---

Revision 30859 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Sat Oct 24 14:53:07 2009 UTC (5 years, 3 months ago) by ganis
File length: 4186 byte(s)
Diff to previous 25827

  Patch for improved performance monitoring. The 'Rate Plot' button in the progress
  dialog is renamed 'Performance plot' and shows up to 4 plots with the event/sec,
  the average read chunck size, the number of active workers and the number of active
  PROOF sessions on the cluster, all as a function of processing time.

  The read chunck size plot allows to monitor the usage of the cache.

  The istantaneous processing rate (event/sec) is now better estimate: a few issues
  with the normalizing times have ben solved, removing the artificial structures that
  were observed.

  The possibility to set a max packet time length is introduced (default 30 s); this
  can be changes with the parameter PROOF_MaxPacketTime.
  The size of the cache is also taken into account to optimize the use of the cache.

  The parameter PROOF_UseParallelUnzip has been introduced to toggle the use of the 
  parallel unzip (default off for now).

  A page describing the new performance plots is under preparation at 
           http://root.cern.ch/drupal/content/progress-dialog

---

Revision 25827 - (view) (download) (as text) (annotate) - [select for diffs]
Added Wed Oct 15 14:02:59 2008 UTC (6 years, 3 months ago) by ganis
File length: 4148 byte(s)

   From Jan:

   - Added the possibility to handle removed workers and partly processed packets. When a worker is stopped
     while processing a packet it finishes, the current event and the rest of the packet is reassigned to another
     worker. This is done via two interfaces:
       - TVirtualPacketizer::AddProcessed(TSlave *sl, TProofProgressStatus *st, TList **)
       - TVirtualPacketizer::ReassignPacket.
   - New class TProofProgressStatus used to keep the query progress stauts in all the TProofPlayer objects and in
     TPacketizerAdaptive::TSlaveStat. This class is also used to structure the relevant information send in
     kPROOF_GETPACKET and kPROOF_STOPPROCESS messages.
   - The class TPacketizerProgressive is removed completely.
   - The PROOF protocol version is increased to 19: this is to handle the changes in the kPROOF_STOPPROCESS and
     kPROOF_GETPACKET messages in Master - worker communication.