Re: Threads continued ...

From: Marc Hemberger (hemberg@clri6a.gsi.de)
Date: Wed Aug 25 1999 - 16:58:16 MEST


Dear Valeri,

thanks for your comments.

We worked here in collaboration together with Victor, who gave us his
original sources. We implemented his mechanisms on protecting the Canvas
and Pad operations so that it should work also when operations on a canvas
are performed by different threads (refering to TPad *gPadMain and
*gThreadXAR, which synchronizes the XActions on the canvas). By this
mechanism, a thread safe design to ROOT is applied.

I don't talk about creating an object from within a thread. I only talk
about having a globally instantiated canvas and accessing this canvas then
from within 2 threads. This is an easy to imagine task which occurs in
several DAQ's. 

The mechanism provided by Victor is working in principle very well. But
only (in our working environment at GSI) on single processor machines (may
be depending also on our Linux kernel and library combinations)!! 

For this reason we wanted to have some feedback from different people on
boiling down the problem to a simple X-program with threads (without the
ROOT mechanisms), giving us more general informations about the problem.

Sincerely,

Marc



On Tue, 24 Aug 1999, Valeri Fine (Faine) wrote:

> 
> Hello Marc,
> 
> > Using the TThread classes provided in the framework of ROOT (e.g. 2.22),
> > we found, that on our working environment here at GSI (Linux Debian 2.1,
> > kernel 2.0.36, glibc 2.0, XFree 3.3.2 on SMP machines with two CPUs) all
> > actions on the TCanvas from within threads causes nearly instantaniously a
> > segmentation violation.
> > 
> 
>   I'd like to say that TTread class is just a tool to solve problem. 
>   But, first, one should understand where those problems come from.
> 
>   Working with ROOT TCanvas the code does work with the real pointers to 
>   the user defined arbitrary object calling an arbitrary user-defined method of that
>   object. In multithread environment one can easily  image the scenario when two
>   or more concurrent threads are playing with one and the same user objects
>   causing the  crash of entire application.
>   This is to say each ROOT (including all user's classes)  must be thread safe
>   unless a special "thread safe design is applied". It is not enough just to protect 
>   the pointer to the current TCanvas with "critical section" /"semaphore" or some
>   other synch tools . (TThread class is a kind of such tool too).
> 
>   A separate issue is  "CINT static variables". Those are not thread safe either. 
>   ROOT calls CINT internally by various occassions.
> 
>   This is to say to be useful in the multithread env, ROOT should be first "sliced" by
>   "functional threads". By this scheme some "ROOT" service should be provided by
>    a single dedicated  thread only.  (For example CINT API must be under control of
>    one and only one thread etc, all TCanvas'es must be created by another dedicated
>    thread only etc) (But it may entailed such sliced ROOT will be more then two times
>    a slow as a single thread one. And one may decide it is for the best to use two
>    separate processes  rather the two concurrent threads )
> 
>   TThread class does allow one to create a thread object to run some methods
>    / functions in parallel but it is those classes / functions responsibility to keep
>    the entire ROOT application running with no crash. TThread class does give 
>    some tool to synchronize an access to some shared object. TThread class does
>    provided an example how it can be used (to access "TPad *gPad" global pointer,
>    for  example ).
>    But this tool can not protect an application from the crash on its own. Just because
>    the instance of that class is created. It doesn't work "by automatic" and "by miracle"
> 
>    May be Victor Perevoztchikov, an author of the  TThread class, can provide
>    a comment  on his implementation as well.
> 
>   Hope this helps,
>                                  Valery
> 
> > Having also access to single CPU machines we found, that the identical
> > program (which starts several threads and each accesses the canvas) works!
> > 
> > Thus the mechanism provided by ROOT to protect the canvas from accesses by
> > different threads at the same time works in principle, but there seems to
> > be a problem in the interplay between ROOT and the Xlib. Since the X11R6
> > is stated to be thread-safe, we wanted now to check on the experimental
> > SMP-support in the kernel in use.
> > 
> > So I created a simple C-program, based only on calls to the Xlib and to
> > the pthreads, which reproduces the error on SMP-machines but runs on
> > single-CPU machines.
> > 
> > We would greatly appreciate any help from anybody outside, who has access
> > to an SMP-machine with a newer kernel (like 2.2.x). Just compile the
> > attached program on your machine and try wether it runs stable, also
> > introducing different timing values (by the sleep/usleep). Any response to
> > that problem is greatly appreciated. 
> > 
> > We try currently to draw our conclusions from the findings we have here
> > about the use of threads within our GO4 project.
> > 
> > Greetings,
> > 
> > Marc
> > 
> > 
> > ----------------------------------------
> > Dr. Marc Hemberger
> > 
> > GSI, Abteilung DV&EE
> > Planckstr. 1
> > 64291 Darmstadt
> > 
> > M.Hemberger@gsi.de
> > 
> > 
> > 
> 
> 
> --------------------------------------------------------------------------------
> 
> 
> > /* X lib program with multithreading,
> >    the timing introduced by the sleep/usleep commands is essential for the
> >    program
> >    */
> >    
> > #include <stdio.h>
> > #include <X11/Xlib.h>
> > #include <X11/Xutil.h>
> > #include <X11/Xresource.h>
> > #include <X11/keysym.h>
> > 
> > #include <math.h>
> > #include <pthread.h>
> > #define EXIT_FAILURE 1
> > pthread_t tid1, tid2, tid3;
> > pthread_attr_t t_attr;
> > pthread_mutex_t mut1;
> > pthread_mutexattr_t attr;
> > int counter=0;
> > Bool propagate = False;
> > 
> > Display *display;  /* Server or display to which a connection will be made */
> > int screen;        /* Screen number to use on that display */
> > Window win;        /* The applications one and only window */
> > XEvent event;      /* The event data structure */
> > GC gc1, gc2;       /* Graphics context data structures */
> > XGCValues gs1,gs2; /* Graphics context settings */
> > char *message;     /* Message printed in window */
> > char mess[] = { "000000000" };
> > static char progDefaultMessage[] = {"A Simple Xlib Program"};
> > 
> > extern int counter;
> > 
> > void setUpGCs();
> > void drawInToWindow();
> > void drawCountWindow();
> > void thread1();
> > void thread2();
> > void thread3();
> > void threadex();
> > 
> > int main(argc, argv)
> >     int argc; char **argv;
> > {
> >   int count, *rc;
> >   /* Attempt to connect to the server and use the default screen */
> >   if ((display = XOpenDisplay(NULL)) == NULL) {
> >     perror("Can't connect to server");
> >     exit(1);
> >   }
> >   screen = DefaultScreen(display);
> > 
> >   /* Create a window */
> >   win = XCreateSimpleWindow(display, RootWindow(display, screen),
> >     100, 200, 400, 300, 1, BlackPixel(display, screen),
> >     WhitePixel(display, screen));
> >   
> >   /* Set up which event types the window will handle */
> >   XSelectInput(display, win, ExposureMask | ButtonPressMask);
> >   
> >   /* Graphics contexts */
> >   setUpGCs();
> >   
> >   /* Attempt to get message from user-defined resource setting */
> >   if ((message = XGetDefault(display, "win", "message")) == NULL)
> >     message = progDefaultMessage;
> >   
> >   /* Map the window in order for it to be displayed */
> >   XMapWindow(display, win);
> > 
> >   threadex();
> >   
> >   /* Enter an infinite loop, awaiting and responding to the chosen events */
> >   count = 0;
> >   while (1) {
> >     XNextEvent(display, &event);
> >     switch (event.type) {
> >     case Expose: drawInToWindow(); break; /* Draw/re-draw everything */
> >     case ButtonPress: exit(0); break; /* Quit on mouse button press */
> >     default: break;
> >     }
> >     if(counter > 99999)
> >       {
> > if(pthread_mutex_lock(&mut1) == -1) {
> >   perror("pthread_mutex_lock");
> >   exit(EXIT_FAILURE);
> > }
> > counter = 0;
> > count ++;
> > if(pthread_mutex_unlock(&mut1) == -1) {
> >   perror("pthread_mutex_unlock");
> >   exit(EXIT_FAILURE);
> > }
> > fprintf(stderr, "counter reset %d\n", count); 
> >       }
> >   }
> > }
> > 
> > void setUpGCs()
> >     /* Set up 2 graphics context resources in the display server */
> >     /* gc1 for message and rectangle; gc2 for ellipse */
> > {
> >   /* Define a bitmap for the stipple pattern used to fill the ellipse */
> > #define stipple_width 3
> > #define stipple_height 3
> >   static char stipple_bits[] = {0x02, 0x05, 0x02};
> >   Pixmap stipple;
> >   
> >   /* Create the stipple for ellipse fill pattern of gc2 */
> >   if ((stipple = XCreateBitmapFromData(display, RootWindow(display, screen),
> >        stipple_bits, stipple_width, stipple_height)) == 0) {
> >     perror("Can't create bitmap");
> >     exit(1);
> >   }
> >   
> >   gc1 = XCreateGC(display, win, 0, &gs1); /* 0 -> use defaults */
> >   gc2 = XCreateGC(display, win, 0, &gs2); /* 0 -> use defaults */
> >   /* Now override some of the defaults */
> >   XSetForeground(display, gc1, BlackPixel(display, screen));
> >   XSetBackground(display, gc1, WhitePixel(display, screen));
> >   XSetLineAttributes(display, gc1, 2, LineSolid, CapRound, JoinRound);
> >   XSetStipple(display, gc2, stipple);
> >   XSetFillStyle(display, gc2, FillOpaqueStippled);
> > }
> > 
> > void drawInToWindow()
> > {
> >   /* Print message, draw restangle and filled ellipse as defined */
> >   /* by the GC */
> >   XDrawString(display, win, gc1, 130, 20, message, strlen(message));
> >   XDrawRectangle(display, win, gc1, 40, 40, 320, 220);
> >   XFillArc(display, win, gc2, 50, 50, 300, 200, 0, 360*64);
> > }
> > 
> > void drawCountWindow()
> > {
> >   XSendEvent(display, win, propagate, ExposureMask, &event);
> > }
> > 
> > void thread1()
> > {
> >   static int count = 0;
> >   while(1) {
> >     if(pthread_mutex_lock(&mut1) == -1) {
> >       perror("pthread_mutex_lock");
> >       exit(EXIT_FAILURE);
> >     }
> >     count++;
> >     counter = counter + 1;
> >     /* fprintf(stderr, "T1=%02d %02d ", counter, count); */
> >     if(pthread_mutex_unlock(&mut1) == -1) {
> >       perror("pthread_mutex_unlock");
> >       exit(EXIT_FAILURE);
> >     }
> >     XFillRectangle(display, win, gc1, 308, 8, 50, 15);
> >     sprintf(mess, "%s %05d", "T1", counter);
> >     XSetForeground(display, gc1, WhitePixel(display, screen));
> >     XDrawString(display, win, gc1, 310, 20, mess, strlen(mess));
> >     XSetForeground(display, gc1, BlackPixel(display, screen));
> >     XSendEvent(display, win, propagate, ExposureMask, &event);
> >     sleep(1);                         /* <======= introduce tming here! */
> >     /* usleep(100000); *//* <== this does not work !! */
> >   }
> > }
> > 
> > void thread2()
> > {
> >   static int count = 0;
> >   while(1) {
> >     if(pthread_mutex_lock(&mut1) == -1) {
> >       perror("pthread_mutex_lock");
> >       exit(EXIT_FAILURE);
> >     }
> >     count++;
> >     counter = counter + 1;
> >     /* fprintf(stderr, "\tT2=%02d %02d ", counter, count); */
> >     if(pthread_mutex_unlock(&mut1) == -1) {
> >       perror("pthread_mutex_unlock");
> >       exit(EXIT_FAILURE);
> >     }
> >     XFillRectangle(display, win, gc1, 48, 8, 50, 15);
> >     sprintf(mess, "%s %05d", "T2", counter);
> >     XSetForeground(display, gc1, WhitePixel(display, screen));
> >     XDrawString(display, win, gc1, 50, 20, mess, strlen(mess));
> >     XSetForeground(display, gc1, BlackPixel(display, screen));
> >     XSendEvent(display, win, propagate, ExposureMask, &event);
> >     usleep(378000);                  /* <======= introduce tming here! */
> >     /* usleep(200000); *//* <== this does not work !! */
> >   }
> > }
> > 
> > void thread3()
> > {
> >   static int count = 0;
> >   while(1) {
> >     if(pthread_mutex_lock(&mut1) == -1) {
> >       perror("pthread_mutex_lock");
> >       exit(EXIT_FAILURE);
> >     }
> >     count++;
> >     counter = counter + 1;
> >     /* fprintf(stderr, "\tT3=%02d %02d ", counter, count); */
> >     if(pthread_mutex_unlock(&mut1) == -1) {
> >       perror("pthread_mutex_unlock");
> >       exit(EXIT_FAILURE);
> >     }
> >     XSendEvent(display, win, propagate, ExposureMask, &event);
> >     usleep(734000);                /* <======= introduce tming here! */
> >     /* usleep(150000); */ /* <== this does not work !! */
> >   }
> > }
> > 
> > void threadex()
> > {
> >   if(pthread_mutexattr_init(&attr) == -1) {
> >     perror("pthread_mutexattr_create");
> >     exit(EXIT_FAILURE);
> >   }
> >   if(pthread_mutex_init(&mut1, &attr) == -1) {
> >     perror("pthread_mutex_init");
> >     exit(EXIT_FAILURE);
> >   }
> >   if(pthread_attr_init(&t_attr) == -1) {
> >     perror("pthread_attr_create");
> >     exit(EXIT_FAILURE);
> >   }
> >   if(pthread_create(&tid1, &t_attr, thread1, 0) == -1) {
> >     perror("pthread_create(1)");
> >     exit(EXIT_FAILURE);
> >   }
> >   if(pthread_create(&tid2, &t_attr, thread2, 0) == -1) {
> >     perror("pthread_create(2)");
> >     exit(EXIT_FAILURE);
> >   }
> >   if(pthread_create(&tid3, &t_attr, thread3, 0) == -1) {
> >     perror("pthread_create(3)");
> >     exit(EXIT_FAILURE);
> >   }
> >     
> >   /*
> >      for(;;)
> >      pause();
> >   */
> > }
> > 
> > 
> 
> 
> --------------------------------------------------------------------------------
> 
> 
> > # ========================= Ultrix: ===========================
> > #
> >    CC      = egcc
> >    O       = o
> > # To enable debugging use '-g' option for compile step:
> > #  CFLAGS  = -g -D_CFORTRAN2_VERSION_ -gen_feedback 
> > #  CFLAGS  = -g -D__STDC__ -Dconst=" " -D__LANGUAGE_C -D__mips -DDEBUG
> >    CFLAGS  = -Wall -O3
> > # cc CFLAGS = -w0 -O3 or -g0 -g1 -g2 -g3  on levi
> > # To enable profiling use '-p' option for link step:
> > #   LFLAGS  = -p1 -pg
> > #   LFLAGS  = -Wl,-yprob_
> >    LFLAGS  = 
> >    LINK    = egcc
> >    OUTPUT  = -o ### The blank before the '#' is required!
> >    MATHLIB = -lm
> >    EXE     =
> >    LIB     = a
> >    F77COMP = g77 -c
> >    F77     = g77
> >    FFLAGS  = -static -g -warn unused -O3
> > #  for profiling enable -g3
> > #   FFLAGS  = -static -g -g3
> > 
> > # Symbols for linking with libraries:
> > 
> > #   FPLIB = -L/us0/vlk/hemberg/fpack -lfpack08604
> > #   CERNLIB = -L/cern/pro/lib -lkernlib -lpacklib \
> > #             -lUfor -lfor -lFutil -lm -lots -lc
> >    XLIB = -L/usr/X11/lib -lX11 -lXm -lXaw -lXp -lXpm
> >    VOGLLIB= -L/u/hemberg/ftp/vogle-3.0/src -lvogle
> > #   MIZZILIB= -L/us0/vlk/hemberg/ccl -lccl
> >    DIVLIB= -lpthread
> >    GRAFLIB= -L/cern/pro/lib -lgraflib -lgrafX11 
> > #   PGPLOT= -lcpgplot -lpgplot
> >    MATHLIB = -lm
> >    CPPLIB = -lstdc++
> > 
> > winthread2$(EXE): winthread2.$(O)
> > $(LINK) $(LFLAGS) winthread2.$(O) $(DIVLIB) $(XLIB) $(OUTPUT) winthread2
> > 
> > winthread2.$(O): winthread2.c
> > 
> > 
> 
> 



This archive was generated by hypermail 2b29 : Tue Jan 04 2000 - 00:43:38 MET