Re: Root files, directories & objects - WinNT (pt 2)

From: Mariusz Stanczak (MStanczak@usa.net)
Date: Fri Dec 03 1999 - 02:37:02 MET


On Thu, 2 Dec 1999 15:31:22 +0100 (MET), Rene Brun wrote:

>Hi Mariusz,
>It is difficult to come with a diagnostic without more information.
>You do not mention how many keys you have created (10, 10 millions ?)
   There are over 25,000 directories each with < 300 objects, on average
(6,686,217 objects in total), though some directories contain closer to 1000
objects.  The size of each object in memory is 52 bytes, and on file 94 (48
bytes for the object + 46 for key).  The total file size is close to 1GB and
an update run of about 10,000 objects (each in a separate directory)
consumed close to 2GB of virtual memory.

>When you write an object to a file via obj->Write
>a TKey object is created in memory (about 70 bytes). The current directory
>holds a list of TKeys.
   As per the code fragment, I force the (re)write of the directory header
and then I exit the directory.  I presume that all the structures in memory
pertaining to that directory would be freed at that point.
>> 		if (strcmp(t, prev_t) != 0) {
>> 			gDirectory->Write(NULL, TObject::kOverwrite);


>What is your estimation of your "claimed leak" wrt the number of
>objects written ?
   Almost all of the allocated memory thoughout the run.  Here are the stats
you asked for (with appologies to the group):

***** Memory and object stats at start of Root *****
Heap statistics
        size       alloc        free        diff
================================================
           8           1           0           1
          16           4           0           4
          20           1           0           1
          28           4           1           3
          36           2           0           2
          40           2           0           2
          44           3           0           3
          72           1           0           1
         168           1           0           1
         404           2           0           2
------------------------------------------------
Total:              1804         296        1508
================================================


Object statistics
class                     cnt    on heap     size    total size    heap size
============================================================================
TOrdCollection              5          5       40           200          200
TClassTable                 1          1       12            12           12
THashTable                  2          2       36            72           72
THashList                   2          2       44            88           88
TEnv                        1          1       24            24           24
TObjectTable                1          1       24            24           24
TROOT                       1          0      240           240            0
TWinNTSystem                1          1      664           664          664
----------------------------------------------------------------------------
Total:                     14         13     1084          1324         1084
============================================================================

[NOTE: the object stats are as above and constant throughout the test run,
so I omit them below, but the memory stats do tell the story]

***** Memory stats after 100 dirs and 47199 objects *****
Heap statistics
        size       alloc        free        diff
================================================
           8          14           0          14
          14          98           0          98
          16      122890       28070       94820
          18         120          90          30
          19         272         204          68
          20       54667       54666           1
          21      141597       94398       47199
          28       94613       47300       47313
          36         268          70         198
          40       31819        7000       24819
          44         199           0         199
          72          14           0          14
          80       47398         101       47297
          84          98           0          98
         168           1           0           1
         404         198          70         128
        1604          70           0          70
------------------------------------------------
Total:         123965957   115163754     8802203
================================================

***** Memory stats after 200 dirs and 71447 objects *****
Heap statistics
        size       alloc        free        diff
================================================
           8          14           0          14
          14         197           0         197
          16      178595       34883      143712
          17         168         126          42
          18         324         243          81
          19         296         222          74
          20       81029       81028           1
          21      214341      142894       71447
          28      143307       71647       71660
          36         483          87         396
          40       45102        8699       36403
          44         397           0         397
          72          14           0          14
          80       71844         200       71644
          84         197           0         197
         168           1           0           1
         404         396          87         309
        1588           1           0           1
        1604          86           0          86
------------------------------------------------
Total:         187415929   174101746    13314183
================================================

***** Memory stats after 300 dirs and 92134 objects *****
Heap statistics
        size       alloc        free        diff
================================================
           8          14           0          14
          14         296           0         296
          16      227984       42502      185482
          17         444         333         111
          18         444         333         111
          19         296         222          74
          20      104033      104032           1
          21      276402      184268       92134
          28      184879       92433       92446
          36         700         106         594
          40       58225       10599       47626
          44         595           0         595
          72          14           0          14
          80       92729         299       92430
          84         296           0         296
         168           1           0           1
         404         594         106         488
        1588           1           0           1
        1604         105           0         105
------------------------------------------------
Total:         241688614   224447829    17240785
================================================
.
.
.
***** Memory stats after 1000 dirs and 275981 objects *****
Heap statistics
        size       alloc        free        diff
================================================
           8          14           0          14
          14         993           3         990
          15           8           6           2
          16      687239      131252      555987
          17        2432        1824         608
          18         980         735         245
          19         376         282          94
          20      312990      312989           1
          21      827943      551962      275981
          28      553959      276973      276986
          36        2307         327        1980
          40      177367       32702      144665
          44        1981           0        1981
          72          16           2          14
          80      277962         992      276970
          84         989           0         989
         168           1           0           1
         404        1980         327        1653
        1556           2           0           2
        1588           4           0           4
        1604         309           0         309
        1636          12           0          12
------------------------------------------------
Total:         724332415   672475420    51856995
================================================

At this point the system shows the program to use 89MB of physical memory
and 112MB of virtual memory, and the file size is 37MB.

>
>Rene Brun

Thanks,

/Mariusz



This archive was generated by hypermail 2b29 : Tue Jan 04 2000 - 00:43:43 MET