Re: invalid free in XrdProofdProofServMgr::Recover()

    Dear Thiemo,

    Thans for this report. I will try to reproduce it with your files and debug it.

    G. Ganis

Thiemo Nagel wrote:
> Hello,
>
> I stumbled upon a reproducible invalid free in
> XrdProofdProofServMgr::Recover(). A valgrind transcript of the xrootd
> session is below. It seems that something was corrupted in my
> /tmp/.x* directories because moving them out of the way resolved the
> problem. (I have attached a tarball of these, in case you're
> interested.)
>
> Kind regards,
>
> Thiemo
>
>
>
> Copr. 2007 Stanford University, xrootd version 2.9.0 build 20081122-0000
> ++++++ xrootd protocol initialization started.
> Config warning: 'xrootd.seclib' not specified; strong authentication
> disabled!
> Config warning: 'xrootd.fslib' not specified; using native file system.
> Copr. 2007 Stanford University/SLAC sfs (Standard File System) v 9.0n
> Config warning: 'xrootd.prepare logdir' not specified; prepare
> tracking disabled.
> Config warning: only '/tmp' will be exported.
> ------ xrootd protocol initialization completed.
> 090122 16:14:28 001 xpd-I: Manager::Config: configuring
> 090122 16:14:28 001 xpd-I: Manager::Config: listening on port 1093
> 090122 16:14:28 001 xpd-I: Manager::Config: using temp dir: /tmp
> 090122 16:14:28 001 xpd-I: Manager::Config: role set to: any
> 090122 16:14:28 001 xpd-I: Manager::Config: admin path set to:
> /tmp/.xproofd.1093
> 090122 16:14:28 001 xpd-I: Manager::Config: PROOF pool:
> root://nailgun.e18.physik.tu-muenchen.de
> 090122 16:14:28 001 xpd-I: Manager::Config: PROOF pool namespace:
> /proofpool
> 090122 16:14:28 001 xpd-I: Manager::LoadScheduler: scheduler loaded:
> type: default
> 090122 16:14:28 001 xpd-I: Manager::Config: user config files are
> disabled
> 090122 16:14:28 001 xpd-I: Manager::Config: list of superusers: tnagel
> 090122 16:14:28 001 xpd-I: Manager::Config: bare lib path for
> proofserv: /usr/lib/debug:/usr/X11R6/lib/debug
> 090122 16:14:28 001 xpd-I: Group::Print: +++ Group: default
> 090122 16:14:28 001 xpd-I: Group::Print: +++ Priority: -1, fraction: -1
> 090122 16:14:28 001 xpd-I: Group::Print: +++ End of Group: default
> 090122 16:14:29 001 xpd-I: NetMgr::Config: configuring
> 090122 16:14:29 001 xpd-I: NetMgr::Config: PROOF config file: none
> 090122 16:14:29 001 xpd-I: NetMgr::Config: 4 worker nodes defined
> 090122 16:14:29 001 xpd-I: PriorityMgr::Config: configuring
> 090122 16:14:29 001 xpd-I: PriorityMgr::Config: no priority changes
> requested
> 090122 16:14:29 001 xpd-I: PriorityMgr::Config: poller thread started
> 090122 16:14:29 001 xpd-I: ROOTMgr::Config: configuring
> 090122 16:14:29 001 xpd-I: ROOTMgr::Config: ROOT dist: '5.22/00
> 5.22/00 /afs/e18/sw/ROOT/root_v5.22.00.gcc412' validated
> 090122 16:14:29 001 xpd-I: ClientMgr::Config: configuring
> 090122 16:14:29 001 xpd-I: ClientMgr::Config: clients admin path set
> to: /tmp/.xproofd.1093/clients
> 090122 16:14:29 001 xpd-I: ClientMgr::Config: XRD seclib not
> specified; strong authentication disabled
> 090122 16:14:29 001 xpd-I: ClientMgr::Config: cron thread started
> 090122 16:14:29 001 xpd-I: ProofServMgr::Config: configuring
> 090122 16:14:29 001 xpd-I: ProofServMgr::Config: setting internal
> timeout to 10 secs
> 090122 16:14:29 001 xpd-I: ProofServMgr::Config: client sessions kept
> idle for 0 secs after disconnection
> 090122 16:14:29 001 xpd-I: ProofServMgr::Config: active sessions admin
> path set to: /tmp/.xproofd.1093/activesessions
> 090122 16:14:29 001 xpd-I: ProofServMgr::Config: terminated sessions
> admin path set to /tmp/.xproofd.1093/terminatedsessions
> 090122 16:14:29 001 xpd-I: ProofServMgr::PrepareSessionRecovering:
> session recover thread started
> 090122 16:14:30 29867 xpd-I: ProofServCron: next full sessions check
> in 30 secs
> 090122 16:14:30 001 xpd-I: ProofServMgr::Config: cron thread started
> 090122 16:14:30 001 xpd-I: Manager::Config: manager cron thread started
> 090122 16:14:30 001 xpd-I: Protocol::Configure: global manager created
> 090122 16:14:30 001 xpd-I: Protocol::Configure: xproofd protocol
> version 0.5 build 20081122-0000 successfully loaded
> ------ xrootd anon_at_nailgun.e18.physik.tu-muenchen.de:1094
> initialization completed.
> 090122 16:14:30 29867 xpd-I: tnagel.29659:30_at_localhost:
> ClientMgr::MapClient: user tnagel logged-in (privileged); type: Internal
> 090122 16:14:30 29867 xpd-I: tnagel.29657:36_at_localhost:
> ClientMgr::MapClient: user tnagel logged-in (privileged); type: Internal
> 090122 16:14:30 29867 xpd-I: tnagel.29655:37_at_localhost:
> ClientMgr::MapClient: user tnagel logged-in (privileged); type: Internal
> ==29867==
> ==29867== Thread 13:
> ==29867== Invalid read of size 4
> ==29867== at 0x499D890:
> XrdProofdProofServMgr::Recover(XpdClientSessions*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x499DFE3:
> XrdProofdProofServMgr::RecoverActiveSessions() (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x499E409: XrdProofdProofServRecover(void*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x808A2DC: XrdSysThread_Xeq (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> ==29867== by 0x405923F: start_thread (in
> /lib/tls/i686/cmov/libpthread-2.3.6.so)
> ==29867== by 0x424949D: clone (in /lib/tls/i686/cmov/libc-2.3.6.so)
> ==29867== Address 0x5A4C514 is 0 bytes after a block of size 36 alloc'd
> ==29867== at 0x401DB31: operator new(unsigned)
> (vg_replace_malloc.c:163)
> ==29867== by 0x499EB7C: XrdProofdProofServMgr::ResolveSession(char
> const*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x49A0B90:
> XrdProofdProofServMgr::PrepareSessionRecovering() (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x49A257D: XrdProofdProofServMgr::Config(bool) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x498515A: XrdProofdManager::Config(bool) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x49B58C4: XrdProofdProtocol::Configure(char*,
> XrdProtocol_Config*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x49B641B: XrdgetProtocol (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x8078D2C: XrdProtLoad::getProtocol(char const*, char
> const*, char*, XrdProtocol_Config*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> ==29867== by 0x8078DD8: XrdProtLoad::Load(char const*, char const*,
> char*, XrdProtocol_Config*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> ==29867== by 0x807DA0C: XrdConfig::Setup(char*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> ==29867== by 0x807E8EA: XrdConfig::Configure(int, char**) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> ==29867== by 0x8076E24: main (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> ==29867==
> ==29867== Invalid free() / delete / delete[]
> ==29867== at 0x401CCBC: operator delete(void*)
> (vg_replace_malloc.c:244)
> ==29867== by 0x499D8A5:
> XrdProofdProofServMgr::Recover(XpdClientSessions*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x499DFE3:
> XrdProofdProofServMgr::RecoverActiveSessions() (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x499E409: XrdProofdProofServRecover(void*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x808A2DC: XrdSysThread_Xeq (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> ==29867== by 0x405923F: start_thread (in
> /lib/tls/i686/cmov/libpthread-2.3.6.so)
> ==29867== by 0x424949D: clone (in /lib/tls/i686/cmov/libc-2.3.6.so)
> ==29867== Address 0x5A4C50C is 28 bytes inside a block of size 36
> alloc'd
> ==29867== at 0x401DB31: operator new(unsigned)
> (vg_replace_malloc.c:163)
> ==29867== by 0x499EB7C: XrdProofdProofServMgr::ResolveSession(char
> const*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x49A0B90:
> XrdProofdProofServMgr::PrepareSessionRecovering() (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x49A257D: XrdProofdProofServMgr::Config(bool) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x498515A: XrdProofdManager::Config(bool) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x49B58C4: XrdProofdProtocol::Configure(char*,
> XrdProtocol_Config*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x49B641B: XrdgetProtocol (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/lib/libXrdProofd.so.5.22)
> ==29867== by 0x8078D2C: XrdProtLoad::getProtocol(char const*, char
> const*, char*, XrdProtocol_Config*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> ==29867== by 0x8078DD8: XrdProtLoad::Load(char const*, char const*,
> char*, XrdProtocol_Config*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> ==29867== by 0x807DA0C: XrdConfig::Setup(char*) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> ==29867== by 0x807E8EA: XrdConfig::Configure(int, char**) (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> ==29867== by 0x8076E24: main (in
> /afs/e18.ph.tum.de/sw/ROOT/root_v5.22.00.gcc412/bin/xrootd)
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Recover: problems accepting
> callback:
> 090122 16:14:30 29867 xpd-E: ProofServMgr::Accept: session pointer
> undefined or socket invalid: 0

-- 
+--------------------------------------------------------------------------+
  Gerardo GANIS    PH Department, CERN
        address    CERN, CH 1211 Geneve 23  
                   room: 32-RC-017, tel / fax: +412276 76439 / 69133
         e-mail    gerardo.ganis_at_cern.ch
+--------------------------------------------------------------------------+