Re: sending large messages over sockets in non blocking mode

From: Fons Rademakers (rdm@pcsalo.cern.ch)
Date: Fri Apr 09 1999 - 00:37:19 MEST


Hi Judith,

   John is right. The problem is that when using non-blocking I/O the
splitting send does not work correctly since I don't handle the EWOULDBLOCK
case in TUnixSystem::UnixSend(). The best work around is to make the socket
blocking just before sending a large buffer and unblock it afterwards.

Cheers, Fons.



> 
> hi John,
> 
> Root takes care of splitting the message into pieces that matches the 
> buffersize of the system  and sending them (see Fons' email to roottalk). 
> 
> I suspect (if I didn't do anything wrong), that something goes wrong with
> the synchronizing of these packages if the socket is in non-splitting mode.
> 
> cheers,
> Judith
> 
> On Wed, 7 Apr 1999, John Zweizig wrote:
> 
> > It sounds like your problem is that you don't have a large enough system
> > send buffer. The usual way to fix this is with setsockopt(SO_SNDBUF, nnnn)
> > (nnn is the buffer size to be used), but I don't know how to translate
> > this into root TSockets (I don't see it documented in the online TSocket
> > class description) and you run the risk of getting errors if the buffer
> > hasn't been fully transferred. The easy way around this is to use a
> > blocking socket for your sends. The send() function then makes sure
> > everything works out OK. This should only take a few extra milliseconds
> > while the data are pushed out over the ethernet (or whatever you use) and 
> > you won't be able to respond to any further requests during that time anyway. 
> > Best regards,
> > 
> > John
> > 
> > On Wed, 7 Apr 1999, Judith Katzy wrote:
> > 
> > > 
> > > hi there,
> > > 
> > > I'd like to send messages of about 300k in a client server environment 
> > > using TSocket and TServerSocket.
> > > Everything works fine as long as I use the blocking mode for 
> > > the socket. It also works, if I use the non-blocking mode and just send
> > > small objects (280 bytes). 
> > > I get the following error on the server side, when I use the non-blocking
> > > mode with a 300k messages:
> > > 
> > > root [1] .x server.C                                               
> > > received mess 'waiting'
> > > SysError in <TUnixSystem::UnixSend>: send (Resource temporarily
> > > unavailable)
> > > Error in <TUnixSystem::SendRaw>: cannot send buffer
> > > server: error occcurred during sending event
> > > 
> > > Attached, I send the macros server.C and client.C.
> > > 
> > > My suspecion is, that something goes wrong when I send several 
> > > packets with TCP/IP in a non-blocking mode, even when  the
> > > sending is synchronized.
> > > 
> > > thanks for any help,
> > > Judith
> > > 
> > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > macro serroot.C
> > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > {
> > > // just a large object (buf size about 300k), I take a data event here
> > > TPhEvent *ev = dst->GetNextEvent() ;
> > > TMessage smess(kMESS_OBJECT);
> > > smess->WriteObject(ev);
> > > TServerSocket *ss = new TServerSocket(9091, kTRUE);
> > > TSocket *s0 = ss->Accept();
> > > s0->SetOption(kNoBlock,1);
> > > Int_t fSend,idx;
> > > 
> > > Char_t str[32];
> > > Int_t fRecv, fSend;
> > > 
> > > while (1) 
> > > {
> > >   fRecv = s0->Recv(str,32);
> > >   if(fRecv != -4) 
> > >     { 
> > >       // client request new event
> > >       idx = !strcmp(str, "waiting") ? 0 : 1;
> > >        if(idx == 0)
> > > 	{     
> > > 	  printf("received mess '%s'\n",str);
> > > 	  //***fSend = s0->Send("xxx");	  
> > > 	  UInt_t what = smess->What();
> > > 	  // synchronize sending
> > > 	  smess->SetWhat(what || kMESS_ACK);
> > > 	  fSend = s0->Send(smess);
> > > 	  if(fSend==-1) 
> > >               printf("server: error occcurred during sending event\n");
> > > 	}
> > >     }
> > > }
> > > s0->Close();
> > > ss->Close();
> > > }
> > > 
> > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > macro client.C
> > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > 
> > > {
> > >    TSocket *csock = new TSocket("judithpc.phy.uic.edu", 9091);
> > >    csock->Send("waiting");  
> > >    TMessage *mess;
> > >    Int_t fRecv;
> > >    Int_t idx;
> > >    Char_t cstr[256];
> > >    while (1) {
> > >      //**     fRecv = csock->Recv(cstr,32);
> > >      fRecv = csock->Recv(mess);
> > >      if(fRecv!=-4){     
> > >       if(fRecv<0)
> > > 	{
> > > 	  printf("error message received, error code %d -return\n ",fRecv);
> > > 	  return;
> > > 	} 
> > >       //**printf("received %s\n",cstr);
> > >       TPhEvent *event = (TPhEvent *)mess->ReadObject(mess->GetClass());
> > >       printf("received %d, event %d\n",mess->What(),event->GetNEvent());
> > >       csock->Send("waiting");
> > >       delete mess;
> > >      }
> > >    }
> > >   csock->Close();
> > > }
> > > 
> > > 
> > > 
> > > 
> > > 
> > 


-- 
Org:    CERN, European Laboratory for Particle Physics.
Mail:   1211 Geneve 23, Switzerland
E-Mail: Fons.Rademakers@cern.ch              Phone: +41 22 7679248
WWW:    http://root.cern.ch/~rdm/            Fax:   +41 22 7677910



This archive was generated by hypermail 2b29 : Tue Jan 04 2000 - 00:43:31 MET