RE:Re: RE:Re: [ROOT] CINT thread safety hac

From: Masaharu Goto (MXJ02154@nifty.ne.jp)
Date: Wed May 23 2001 - 16:05:24 MEST


Hello Christoph,

I tried further on the libcintX.so multi-threading issue.

After some effort, I could run more than one Cint sessions
in one process using the libcintX.so scheme. This is good.
I also made it possible that each thread can load additional
shared libraries by '#include "xxx.dll"' or equivalent means.

As you reported, sometimes it works and sometimes it does not.
I investigated it and came to following understanding. It is 
likely that segmentation violation occurs in conjunction with
order of loading/unloading additional shared library that is
common to more than one thread. 

    main prog ------> libcint1.so ---------> myshl.so
              ------> libcint2.so --------->

For example , in above fiture, there are 2 threads and each
one loads independent libcintX.so. However, both Cint loads
same myshl.so.  When one of the thread finishes the job,
it tries to unload myshl.so while the other one is still
using it. In single thread environment, this has to be
handled without problem. However, with multi-thread, this
might be causing segv errors.

So far, I suspect there are 2 ways to workaround this problem.

1) Do not dlclose() the libcintX.so. Keep it in a table and
   reuse when new task is scheduled.

2) Do not load further shared library from libcintX.so. You need
   to link everything statically into libcintX.so.

3) Duplicate also the additional shared library. In above case
   duplicate myshl.so as myshl1.so and myshl2.so.


At this moment, I'll leave this issue open. Since, I do not see
convincing advantage over fork + shared memory solution. I 
understand this is because you already have an existing framework
and it forces this scheme.  But, can you consider using system()
function and shared memory? Or is it really inevitable to stick
on this scheme?  Will you let me understand better?

For enabling libcintX.so multi-threading, I added some changes to
G__ci.h and shl.c. We do not lose shared library binary
compatibility unless user defines a special compile flag or
command line option.  platform/linux_RH6.2_mtso is added to
try this feature. However, using this configuration is not
recommended for ordinary users.


Thank you
Masaharu Goto


>Date: Mon, 21 May 2001 20:34:34 +0300
>From: Christoph Bugel <chris@uxsrvc.tti-telecom.com>
>To: Masaharu Goto <MXJ02154@nifty.ne.jp>
>Subject: Re: RE:Re: [ROOT] CINT thread safety hack?
>
>
>Hello Masaharu,
>
>Actually the idea was of a collegue of mine, here in TTI.  On the one hand, w
e
>know it a 'not very elegant' (to say the least) workaround, but on the other
>hand - I can't think of a reason why it shouldn't work.  We need to use this 
on
>most platforms, and I already tried this on Linux/gcc-3.0-snapshot,
>Solaris/CC5, and NT4/msvc. On Solaris it already seems to work quite well.
>
>
>I read somewhere in the CINT documentation that you recommend people to
>'register' when they are using cint.
>So I would like to register myself and my employer as a cint user :)
>
>I am using cint as part of a larger C++ framework that our company uses as a
>basis to develop our applications. TTI-Telecom is a software company, with +-
>500 employees. We are selling solutions (servers + software) to telephony
>companies all over the world. We need 'scripting capabilities' in our
>framework, and we intend to use cint for this. I am a programmer working at
>TTI, and my job is currently to implement the scripting capabilities of our
>framework.
>
>Thanks a lot :-)
>Yours,
>
>Christoph Bugel
>TTI Telecom
>www.tti-telecom.com
>
>
>
>On Thu 2001-05-10, Masaharu Goto wrote:
>> Hello Christoph,
>> 
>> This is a very interesting idea. As far as I know,
>> nobody has tried it.   Whether or not this works depends
>> on a shared library symbol resolution scheme in your
>> operating system.  If the operating system always resolves
>> symbols within own shared library, it works. However,
>> if symbols are resolved among multiple libraries, this won't
>> work. I have a feeling that this works for Windows and
>> IBM AIX, but not for most of the other platforms.  
>> 
>> I'll try this myself as soon as I can find my time.
>> 
>> Will you teach me which platform are you using?
>> 
>> 
>> Thank you
>> Masaharu Goto
>> 
>> 
>> >Date: Thu, 10 May 2001 13:24:46 +0300
>> >From: Christoph Bugel <chris@uxsrvc.tti-telecom.com>
>> >To: Valeri Fine <fine@bnl.gov>
>> >Cc: roottalk@pcroot.cern.ch, MXJ02154@nifty.ne.jp
>> >Subject: Re: [ROOT] CINT thread safety hack?
>> >
>> >Hi Valeri,
>> >
>> >Actually I don't use ROOT at all! (I installed it, out of curiosity of
>> >course, and was very impressed) but I 'just' need a C++ interpreter, and
>> >this mailing list is for CINT as well. (I think (?)).  My shared library
>> >doesn't contain ROOT. I just download the most recent cintX.Y.Z.tar.gz
>> >every now and then from root.cern.ch, and compile it. (and play around
>> >with the Makefiles as needed to create a shared library).  You mentioned
>> >the 'CINT Thread' - with other threads sending messages to it. I don't
>> >need other thread talking with the CINT thread.  But on the other hand,
>> >I need more than one 'CINT thread'!!  Each CINT is totally independent,
>> >and runs a totally different C++ function. You are right to ask - why
>> >don't I do it in different processes, when there is nothing to share
>> >anyway. The answer is that this code has to fit into an existing
>> >framework with a very object oriented && multithreaded architecture, it
>> >is just a given fact that I have to comply with..
>> >
>> >I still didn't run it under gdb, admittedly.
>> >
>> >I understand that cint does dynamic loading and unloading of dlls at
>> >runtime. But since I don't use those features (I think I don't --  I
>> >don't use any libraries such as iostream, STL, etc, and I dont do any
>> >#pragma compile or any other #pragma, I just run a C++ function from a
>> >file -- nothing 'dynamic') I hoped I could succeed to run simultaneous
>> >CINT threads without a problem, provided I could isolate them from each
>> >other by loading them into different memory locations (as described
>> >below) -- effectively multiplexing their global variables. I know this
>> >has been discussed before, but my boss wanted to try out anyway. After
>> >all, in theory it should (probably) work. Actually, it already works,
>> >some times.. I managed to load two 'instances' of cint, each running a
>> >different program, only right now this succeeds sometimes and crashes
>> >sometimes :(
>> >
>> >Christoph
>> >
>> >
>> >
>> >On Wed 2001-05-09, Valeri Fine wrote:
>> >
>> >> Hi, Chrisoph
>> >> 
>> >> >
>> >> > I am currently trying out an *ugly* workaround to run multiple cint's
>> >> > in multiple threads. They don't need to share anything between each
>> >> > other. Each cint thread will just load a some file and execute a few C
>> >> > functions from it.  If anyone can point out why the following method
>> >> > will break - please do!
>> >> 
>> >>   The question is which thread is responsible for the dictionaries of al
l
>> >>   ROOT classes?
>> >> 
>> >>   Did you assume each your share library will include the entire ROOT ?
>> >>   Then why do you need all of this, it is much simpler just to run the
>> >> separate
>> >>   processes.
>> >> 
>> >>  On another hand the Windows version of ROOT is 4 threads application by
>> >> now.
>> >>  One of them drives "CINT", others are to sent mesages to the "Cint" 
thread
>> .
>> >>  Somehow it works,
>> >> 
>> >> 
>> >> > The workaroud is as following: I compiled cint into a shared library
>> >> > (.so). My process copies this cint.so into a few identical copies with
>> >> > *different* names -- one for each thread. for example cint.1.so,
>> >> > cint.2.so, cint.3.so, etc.  Then, each thread does a dlopen on its own
>> >> > copy of cint and looks up (dlsym) the symbols it needs (like
>> >> > G__init_cint, G__loadfile, G__exec_text, etc). Because the dlls have
>> >> > different names, the code will be entirely loaded each time, so
>> >> > cint.1.so and cint.2.so are each loaded into another memory location,
>> >> > and all variables used by cint are multiplexed - even global and stati
c
>> >> > ones.
>> >> 
>> >>   If variables are to be multiplexed then why do you need the threads.
>> >>   I am guessing you do want your threads share information via common
>> >>   memory.
>> >> 
>> >> > This method worked for some simple tests (without cint). This method
>> >> > does not protect shared resources such as files, but then I think sinc
e
>> >> > cint generates random temporary filenames two thread won't use the sam
e
>> >> > filename anyway.  It almost works for me already, only I just got a
>> >> > segfault, and I wondered if I forgot something. (Usually these
>> >> > segfaults are my own fault) did I miss something important?
>> >> 
>> >>  Did you try this under debugger. What did it say ?
>> >> 
>> >>     Valeri
>> >
>> >-- 
>
>-- 



This archive was generated by hypermail 2b29 : Tue Jan 01 2002 - 17:50:46 MET