Log of /trunk/proof/proofplayer/inc/TPerfStats.h
Parent Directory
Revision
44046 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue May 1 11:34:04 2012 UTC (2 years, 8 months ago) by
ganis
File length: 6224 byte(s)
Diff to
previous 40699
Simplify enabling of basic feedback.
- In TDrawFeedback::Feedback, call method Draw() of objects not identified as TH1
derivation. This allows user-defined objects implementing Draw to be displayed
via this utility class (request on the forum).
- In TPerfStats, add new histogram TH1I with the packets being processed per worker;
currently this is 0 or 1 but may change in future; it shows which workers are
still active, which may be useful for debugging at the end of query.
- Add new class TStatsFeedback to display in one canvas the distributions for
the events processed per workers, the packets processed per workers and the
packets being processed per worker.
- In TProof::Process, add support for switches "fb=name1,name2,name3,... " or
"feedback=name1,name2,name3,... " in the option field. This enables feedback
for the specified objects, creating a TDrawFeedback object attached to the session.
Feedback for the specified objects is disabled at the end of the query and the
created TDrawFeedback is destroyed.
The special name 'stats' enables the three feedback histograms required by
TStatsFeedback, and uses a TStatsFeedback instead of TDrawFeedback.
The feedback specs are removed from the option field passed to TProofPlayer::Process.
Revision
40699 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Thu Aug 25 10:36:29 2011 UTC (3 years, 5 months ago) by
rdm
File length: 6129 byte(s)
Diff to
previous 40186
From Gerri:
Patch adding support for backend-dependent record formatting of PROOF
monitoring. This is achieved by introducing a new layer, described by
the abstract interface TProofMonSender, with the record format defined
in the backend implemenattion (currently TProofMonSenderML, for MonaLisa,
and TProofMonSenderSQL, for SQL backends).
Currently three types of records are sent: 'summary' (derived from what
was currently posted), 'dataset', with entries per dataset processed
in the query, and 'files', with entries per file processed in the query.
In SQL terms, each of this records corresponds to a different table.
Sending of any of the three records can be toggled independently.
Record format have been versionned, so that previous versions of the
'summary' record can be still sent chosing the corresponding version.
The latest version of 'summary' has been augmented with information
about the number of missing files and the ROOT version.
The headers of the relevant methods in TProofMonSenderSQL and
TProofMonSenderML contain the details about the records.
The Web doc is up-to-date.
The patch also add to TSQMonitoring::SendParameters the possibility to
do a bulk insert. This feature is used by TProofMonSenderSQL when
sending dataset or files information.
Revision
40186 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Jul 11 12:00:50 2011 UTC (3 years, 6 months ago) by
ganis
File length: 6093 byte(s)
Diff to
previous 38709
- Record in TStatus also the max memory usage on the master and printed via TStatus::Print;
this allow a quick visualisation of the overall memroy usage at the end of the query:
root [] TStatus *st = (TStatus *) p->GetOutputList()->FindObject("PROOF_Status");
root [] st->Print();
OBJ: TStatus PROOF_Status OK
Max worker virtual memory: 394.90 MB Max worker resident memory: 229.47 MB
Max master virtual memory: 1104.55 MB Max master resident memory: 940.73 MB
- Fix issue in TProofPlayer::HandleHistogram preventing proper histogram cleaning right
after merging when using TH1::Add; histogram were still destroyed at the end of the
query, but there was no memory advantage in TH1::Add wrt TH1::Merge.
Revision
38180 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 21 14:38:22 2011 UTC (3 years, 11 months ago) by
rdm
File length: 5496 byte(s)
Diff to
previous 38055
From Gerri:
Fix a few issues in SQL PROOF monitoring:
- In TSQLMonitoringWriter::SendParameters, drop ''' around field names in
the INSERT string; also use TString::Format(...) instead of Form(...)
where relevant;
- In TPerfStats:
- call 'proofgroup' instead of 'group' the field with the PROOF group
(interference with the 'group' keyword in SQL);
- add new field 'querytag' VARCHAR(64) with the unique query tag
- in WriteQueryLog fill also the field 'totevents'
- in PacketEvent, add switch to control whether to send te information
to the monitoring system on per packet level (may be too much for SQL).
The switch is called fMonitorPerPacket and it is globally controlled
by the rootrc variable 'Proof.MonitorPerPacket' and at
session level with the parameter PROOF_MonitorPerPacket .
Revision
38055 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Fri Feb 11 22:06:25 2011 UTC (3 years, 11 months ago) by
pcanal
File length: 5404 byte(s)
Diff to
previous 30965
Introduce TVirtualPerfStats::FileUnzipEvent to be able to keep track of the cost of unzipping and use this in TTreePerfStas and TBakset ... This give a good picture of where the time in unzip or in unstreaming
Revision
30953 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Tue Nov 3 08:42:49 2009 UTC (5 years, 2 months ago) by
ganis
File length: 5314 byte(s)
Diff to
previous 22635
If enabled, send monitoring information from the master at each GetNextPacket
(at each call of TPerfStat::PacketEvent) to allow extrnal real-time progress
monitoring.
Revision
18956 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Jun 6 10:03:48 2007 UTC (7 years, 7 months ago) by
rdm
Original Path:
trunk/proofplayer/inc/TPerfStats.h
File length: 5023 byte(s)
Diff to
previous 18336
write PROOF query log info in a SQL DB. This info can be used for quota
and user priority calculation. The data is stored in a table "proofquerylog"
with the definition:
CREATE TABLE proofquerylog (
id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
user VARCHAR(128) NOT NULL,
group VARCHAR(128),
begin DATETIME,
end DATETIME,
walltime INT,
cputime FLOAT,
bytesread BIGINT,
events BIGINT
)
This information will only be stored if in the xpd.cf file the option
xpd.querylogdb mysql://lxb6041.cern.ch/proof alice alicaf
is specified.
Revision
16775 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Wed Nov 15 17:45:55 2006 UTC (8 years, 2 months ago) by
rdm
Original Path:
trunk/proofplayer/inc/TPerfStats.h
File length: 4709 byte(s)
Diff to
previous 16144
From Gerri and Bertrand:
Fixes and new features developed in preparation for the Intel quad-core
launch demo:
- Add instantaneous event and MB processing rates in the Progress signal
and make necessary modification to exploit it in the Progress Dialog.
- Add possibility to change the Progress and Feedback periods
- A number of small improvements and bug fixes
Revision
11047 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Modified
Mon Feb 7 18:02:37 2005 UTC (9 years, 11 months ago) by
rdm
Original Path:
trunk/proofplayer/inc/TPerfStats.h
File length: 4315 byte(s)
Diff to
previous 9182
From Maarten and Kristjan Gulbrandsen:
Here is the patch with the restructured PROOF code. It parameterizes
PROOF startup, currently implementing "Standard", Condor, MLM and PEAC.
It can easily be extended with other "TProof" classes, e.g. AliEn Proof.
Or what ever next Grid implementation or scheduler comes along.
The MLM, Multi Level Master, code uses a basic packetizer, based on a
MSD (massstorage domain) tag. In MLM mode we have multi-tier client
server setup, like:
client -> super master -> sub master 0 -> slave[0..N]
...
-> sub master M -> slave[0..N]
It also contains a large number of smaller fixes. Credit for most of
the work goes to Kris!
We have tested this on multipe platforms (RH7.3, RH9, SL3) and in
multiple configurations. Should be stable.
Revision
9182 -
(
view)
(
download)
(
as text)
(
annotate)
-
[select for diffs]
Added
Sun Jun 13 16:26:36 2004 UTC (10 years, 7 months ago) by
rdm
Original Path:
trunk/proofplayer/inc/TPerfStats.h
File length: 4333 byte(s)
From Maarten and Kristjan Gulbrandsen:
PROOF and file I/O performance monitoring framework. The TVirtualPerfStats
abstract class allows adding monitoring events in base classes (TFile and
TNetfile). The performance monitoring info allows detailed study of PROOF
performance (which files accessed by which slaves, latency, read/write times,
packet size, etc.). The result is send back to the client in the form of a
memory based TTree.
Remove TProofStats.h and TProofStats.cxx, .d, .o from proof/inc and src.
This form allows you to request diffs between any two revisions of this file.
For each of the two "sides" of the diff,
enter a numeric revision.
