Re: is TTreeSQL dumping the whole table ?

From: Tristan Beau <beau_at_in2p3.fr>
Date: Fri, 3 Feb 2006 15:34:56 +0100

Hi,

On 3 Feb 2006, at 14:43, Philippe Canal wrote:

> Hi Tristan,
>
> What is the output of valgrind if you exist after only 2 draws.
> Also when use the command line:
>
>   valgrind --leak-check=full root.exe

At the end (after .q) I get :




root [5] .q
==18948==
==18948== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 22
from 2)
==18948== malloc/free: in use at exit: 175 bytes in 3 blocks.
==18948== malloc/free: 1,056 allocs, 1,053 frees, 736,097 bytes
allocated.
==18948== For counts of detected errors, rerun with: -v
==18948== searching for pointers to 3 not-freed blocks.
==18948== checked 146,644 bytes.
==18948==
==18948== LEAK SUMMARY:
==18948== definitely lost: 0 bytes in 0 blocks.
==18948== possibly lost: 0 bytes in 0 blocks.
==18948== still reachable: 175 bytes in 3 blocks.
==18948== suppressed: 0 bytes in 0 blocks.
==18948== Reachable blocks (those to which a pointer was found) are
not shown.
==18948== To see them, rerun with: --show-reachable=yes


I also tried with "-v" and "--show-reachable=yes" options, as suggested. Then I get :



root [4] .q
--19104-- discard syms at 0x411A000-0x4125000 in /lib/ libnss_files-2.3.4.so due to munmap()
==19104==
==19104== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 21
from 1)
--19104--
--19104-- supp: 21 dl_relocate_object
==19104== malloc/free: in use at exit: 175 bytes in 3 blocks.
==19104== malloc/free: 1,056 allocs, 1,053 frees, 736,097 bytes
allocated.
==19104==
==19104== searching for pointers to 3 not-freed blocks.
==19104== checked 147,364 bytes.
==19104==
==19104== 23 bytes in 1 blocks are still reachable in loss record 1 of 3
==19104== at 0x400446D: malloc (vg_replace_malloc.c:149)
==19104== by 0x2F743D: XauFileName (in /usr/X11R6/lib/libX11.so.6.2)
==19104== by 0x2F7232: XauGetBestAuthByAddr (in /usr/X11R6/lib/
libX11.so.6.2)
==19104== by 0x2CD36A: _X11TransConnectDisplay (in /usr/X11R6/lib/
libX11.so.6.2)
==19104== by 0x2DE552: XOpenDisplay (in /usr/X11R6/lib/libX11.so.6.2)
==19104== by 0x804A972: PopupLogo(bool) (in /usr/local/ROOT/
root_headcvs.gcc_3.4.3/bin/root)
==19104== by 0x8049D94: main (in /usr/local/ROOT/
root_headcvs.gcc_3.4.3/bin/root)
==19104==
==19104==
==19104== 48 bytes in 1 blocks are still reachable in loss record 2 of 3
==19104== at 0x400446D: malloc (vg_replace_malloc.c:149)
==19104== by 0x1FDEAF: gaih_inet (in /lib/tls/libc-2.3.4.so)
==19104== by 0x2000B0: getaddrinfo (in /lib/tls/libc-2.3.4.so)
==19104== by 0x30AC0C: (within /usr/X11R6/lib/libX11.so.6.2)
==19104== by 0x30BF30: _X11TransConnect (in /usr/X11R6/lib/
libX11.so.6.2)
==19104== by 0x2CCFD2: _X11TransConnectDisplay (in /usr/X11R6/lib/
libX11.so.6.2)
==19104== by 0x2DE552: XOpenDisplay (in /usr/X11R6/lib/libX11.so.6.2)
==19104== by 0x804A972: PopupLogo(bool) (in /usr/local/ROOT/
root_headcvs.gcc_3.4.3/bin/root)
==19104== by 0x8049D94: main (in /usr/local/ROOT/
root_headcvs.gcc_3.4.3/bin/root)
==19104==
==19104==
==19104== 104 bytes in 1 blocks are still reachable in loss record 3
of 3
==19104== at 0x400446D: malloc (vg_replace_malloc.c:149)
==19104== by 0x30AB54: (within /usr/X11R6/lib/libX11.so.6.2)
==19104== by 0x30BF30: _X11TransConnect (in /usr/X11R6/lib/
libX11.so.6.2)
==19104== by 0x2CCFD2: _X11TransConnectDisplay (in /usr/X11R6/lib/
libX11.so.6.2)
==19104== by 0x2DE552: XOpenDisplay (in /usr/X11R6/lib/libX11.so.6.2)
==19104== by 0x804A972: PopupLogo(bool) (in /usr/local/ROOT/
root_headcvs.gcc_3.4.3/bin/root)
==19104== by 0x8049D94: main (in /usr/local/ROOT/
root_headcvs.gcc_3.4.3/bin/root)
==19104==
==19104== LEAK SUMMARY:
==19104== definitely lost: 0 bytes in 0 blocks.
==19104== possibly lost: 0 bytes in 0 blocks.
==19104== still reachable: 175 bytes in 3 blocks.
==19104== suppressed: 0 bytes in 0 blocks.
--19104--  memcheck: sanity checks: 118 cheap, 5 expensive
--19104--  memcheck: auxmaps: 0 auxmap entries (0k, 0M) in use
--19104--  memcheck: auxmaps: 0 searches, 0 comparisons
--19104--  memcheck: secondaries: 33 issued (2112k, 2M)
--19104--  memcheck: secondaries: 41 accessible and distinguished  
(2624k, 2M)
--19104--     tt/tc: 11,317 tt lookups requiring 11,832 probes
--19104--     tt/tc: 11,317 fast-cache updates, 4 flushes
--19104-- translate: new        5,448 (111,727 -> 1,811,203; ratio  
162:10) [0 scs]
--19104-- translate: dumped     0 (0 -> ??)
--19104-- translate: discarded  118 (2,156 -> ??)
--19104-- scheduler: 5,925,805 jumps (bb entries).
--19104-- scheduler: 118/9,603 major/minor sched events.
--19104--    sanity: 119 cheap, 5 expensive checks.
--19104--    exectx: 30,011 lists, 168 contexts (avg 0 per list)
--19104-- exectx: 2,130 searches, 1,963 full compares (921 per 1000) --19104-- exectx: 3 cmp2, 63 cmp4, 0 cmpAll

Moreover, according to top,

1) After launchin root with this command, root.exe takes 26 Mo
2) After the TSQLServer::Connect statement, takes 27 Mo
3) After the TTreeSQL constructor, takes 674 Mo (and takes a lot of  
seconds to end)
4) After first Draw(), takes 747 Mo (and takes few seconds to create the histo)
5) After second Draw(), takes 913 Mo (and takes a lot of seconds to create this new histo)

Last thing, each time, root.exe sends " Select * from atable " whereas it should be better to send " Select afield from atable " to reduce incomming data. Or it should ask only once the full table, even if it might be a memory problem if the table is big. I checked this point with the "show processlist;" on the mysql server side.

   Hope it helps you to understand what's happening...

     Cheers,

                                          Tristan

>
> Cheers,
> Philippe.
>
> -----Original Message-----
> From: Tristan Beau [mailto:beau_at_in2p3.fr]
> Sent: Friday, February 03, 2006 2:27 AM
> To: Philippe Canal
> Cc: roottalk_at_pcroot.cern.ch
> Subject: Re: [ROOT] is TTreeSQL dumping the whole table ?
>
>
> Hi,
>
> On 2 Feb 2006, at 15:35, Philippe Canal wrote:
>
>> I'll try to see if I can spot any memory leak.  Could you try running
>> with valgrind?
>
> Well, I don't know valgrind. Anyway, I've installed and tried it. But
> I am using the interactive mode of ROOT, and the valgrind output is
> not so interesting...
>
> #########################################################
>
>   $ valgrind --leak-check=full root
>
> ==18136== Memcheck, a memory error detector.
> ==18136== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et
> al.
> ==18136== Using LibVEX rev 1471, a library for dynamic binary
> translation.
> ==18136== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
> ==18136== Using valgrind-3.1.0, a dynamic binary instrumentation
> framework.
> ==18136== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et
> al.
> ==18136== For more details, rerun with: -v
> ==18136==
>    *******************************************
>    *                                         *
>    *        W E L C O M E  to  R O O T       *
>    *                                         *
>    *   Version   5.09/01  16 December 2005   *
>    *                                         *
>    *  You are welcome to visit our Web site  *
>    *          http://root.cern.ch            *
>    *                                         *
>    *******************************************
>
> FreeType Engine v2.1.9 used to render TrueType fonts.
> Compiled on 1 February 2006 for linux with thread support.
>
> CINT/ROOT C/C++ Interpreter version 5.16.7, January 19, 2006
> Type ? for help. Commands must be C++ statements.
> Enclose multiple statements between { }.
> root [0] TSQLServer *s = TSQLServer::Connect("mysql://aserver/
> adb","user","pwd");
> root [1] TTreeSQL *t = new TTreeSQL(s,"adb","atable");
> root [2] t->Draw("field1","")
> <TCanvas::MakeDefCanvas>: created default TCanvas with name c1
> (Long64_t)742562
> root [3] t->Draw("field2","")
> (Long64_t)742562
> root [4] t->Draw("field3","")
> Error in <TTreeFormula::Compile>:  Bad numerical expression : "T3Rate"
> (Long64_t)(-1)
> root [5] t->Draw("field4:field5","")
> (Long64_t)742562
> root [6] t->Draw("field6:field7","")
> ==18136==
> ==18136== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 22
> from 2)
> ==18136== malloc/free: in use at exit: 175 bytes in 3 blocks.
> ==18136== malloc/free: 1,056 allocs, 1,053 frees, 736,097 bytes
> allocated.
> ==18136== For counts of detected errors, rerun with: -v
> ==18136== searching for pointers to 3 not-freed blocks.
> ==18136== checked 146,644 bytes.
> ==18136==
> ==18136== LEAK SUMMARY:
> ==18136==    definitely lost: 0 bytes in 0 blocks.
> ==18136==      possibly lost: 0 bytes in 0 blocks.
> ==18136==    still reachable: 175 bytes in 3 blocks.
> ==18136==         suppressed: 0 bytes in 0 blocks.
> ==18136== Reachable blocks (those to which a pointer was found) are
> not shown.
> ==18136== To see them, rerun with: --show-reachable=yes
>
> #######################
>
> After the 6th command, the system killed the interactive root due to
> memory full...
>
> How should I use valgrind to test the memory leak ?
>
>                                 Tristan
>
> ---
> Tristan Beau       http://www.apc.univ-paris7.fr/~beau/
> AstroParticules et Cosmologie
> 11 pl. M. Berthelot, 75231 Paris cedex 05
> tel: +33 1 44 27 14 46 ,        fax: +33 1 43 54 69 89
>
>
>


                                Tristan

---
Tristan Beau       http://www.apc.univ-paris7.fr/~beau/
AstroParticules et Cosmologie
11 pl. M. Berthelot, 75231 Paris cedex 05 tel: +33 1 44 27 14 46 , fax: +33 1 43 54 69 89 Received on Fri Feb 03 2006 - 15:35:16 MET

This archive was generated by hypermail 2.2.0 : Mon Jan 01 2007 - 16:31:57 MET