BidirMMapPipe creates a bidirectional channel between the current process and a child it forks.
- Author
- Manuel Schiller manue.nosp@m.l.sc.nosp@m.hille.nosp@m.r@ni.nosp@m.khef..nosp@m.nl
- Date
- 2013-07-07
This class creates a bidirectional channel between this process and a child it creates with fork().
The channel is comrised of a small shared pool of buffer memory mmapped into both process spaces, and two pipes to synchronise the exchange of data. The idea behind using the pipes at all is to have some primitive which we can block on without having to worry about atomic operations or polling, leaving these tasks to the OS. In case the anonymous mmap cannot be performed on the OS the code is running on (for whatever reason), the code falls back to mmapping /dev/zero, mmapping a temporary file, or (if those all fail), a dynamically allocated buffer which is then transmitted through the pipe(s), a slightly slower alternative (because the data is copied more often).
The channel supports five major operations: read(), write(), flush(), purge() and close(). Reading and writing may block until the required buffer space is available. Writes may queue up data to be sent to the other end until either enough pages are full, or the user calls flush which forces any unsent buffers to be sent to the other end. flush forces any data that is to be sent to be sent. purge discards any buffered data waiting to be read and/or sent. Closing the channel on the child returns zero, closing it on the parent returns the child's exit status.
The class also provides operator<< and operator>> for C++-style I/O for basic data types (bool, char, short, int, long, long long, float, double and their unsigned counterparts). Data is transmitted binary (i.e. no formatting to strings like std::cout does). There are also overloads to support C-style zero terminated strings and std::string. In terms of performance, the former is to be preferred.
If the caller needs to multiplex input and output to/from several pipes, the class provides the poll() method which allows to block until an event occurs on any of the polled pipes.
After the BidirMMapPipe is closed, no further operations may be performed on that object, save for the destructor which may still be called.
If the BidirMMapPipe has not properly been closed, the destructor will call close. However, the exit code of the child is lost in that case.
Closing the object causes the mmapped memory to be unmapped and the two pipes to be closed. We also install an atexit handler in the process of creating BidirMMapPipes. This ensures that when the current process terminates, a SIGTERM signal is sent to the child processes created for all unclosed pipes to avoid leaving zombie processes in the OS's process table.
BidirMMapPipe creation, closing and destruction are thread safe. If the BidirMMapPipe is used in more than one thread, the other operations have to be protected with a mutex (or something similar), though.
End of file (other end closed its pipe, or died) is indicated with the eof() method, serious I/O errors set a flags (bad(), fail()), and also throw exceptions. For normal read/write operations, they can be suppressed (i.e. error reporting only using flags) with a constructor argument.
Technicalities:
- there is a pool of mmapped pages, half the pages are allocated to the parent process, half to the child
- when one side has accumulated enough data (or a flush forces dirty pages out to the other end), it sends these pages to the other end by writing a byte containing the page number into the pipe
- the other end (which has the pages mmapped, too) reads the page number(s) and puts the corresponding pages on its busy list
- as the other ends reads, it frees busy pages, and eventually tries to put them on the its list; if a page belongs to the other end of the connection, it is sent back
- lists of pages are sent across the pipe, not individual pages, in order to minimise the number of read/write operations needed
- when mmap works properly, only one bytes containing the page number of the page list head is sent back and forth; the contents of that page allow to access the rest of the page list sent, and page headers on the list tell the receiving end if the page is free or has to be added to the busy list
- when mmap does not work, we transfer one byte to indicate the head of the page list sent, and for each page on the list of sent pages, the page header and the page payload is sent (if the page is free, we only transmit the page header, and we never transmit more payload than the page actually contains)
- in the child, all open BidirMMapPipes but the current one are closed. this is done for two reasons: first, to conserve file descriptors and address space. second, if more than one process is meant to use such a BidirMMapPipe, synchronisation issues arise which can lead to bugs that are hard to find and understand. it's much better to come up with a design which does not need pipes to be shared among more than two processes.
Here is a trivial example of a parent and a child talking to each other over a BidirMMapPipe:
#include <string>
#include <iostream>
#include <cstdlib>
{
while (pipe.
good() && !pipe.
eof()) {
std::string str;
pipe >> str;
if (!pipe) return -1;
if (!str.empty()) {
std::cout << "[CHILD] : read: " << str << std::endl;
str = "... early in the morning?";
}
if (str.empty()) break;
if (!pipe) return -1;
std::cout << "[CHILD] : wrote: " << str << std::endl;
}
return 0;
}
{
int retVal = childexec(*p);
delete p;
std::exit(retVal);
}
return p;
}
{
std::cout << "[PARENT]: simple challenge-response test, one child:" <<
std::endl;
for (int i = 0; i < 5; ++i) {
std::string str("What shall we do with a drunken sailor...");
if (!*pipe) return -1;
std::cout << "[PARENT]: wrote: " << str << std::endl;
*pipe >> str;
if (!*pipe) return -1;
std::cout << "[PARENT]: read: " << str << std::endl;
}
std::cout <<
"[PARENT]: exit status of child: " << pipe->
close() <<
std::endl;
delete pipe;
return 0;
}
header file for BidirMMapPipe, a class which forks off a child process and serves as communications c...
BidirMMapPipe creates a bidirectional channel between the current process and a child it forks.
int close()
flush buffers, close pipe
bool good() const
status of stream is good
bool isChild() const
return if this end of the pipe is the child end
bool eof() const
true if end-of-file
void flush()
flush buffers with unwritten data
BidirMMapPipe(bool useExceptions=true, bool useSocketpair=false)
constructor (forks!)
int main(int argc, char **argv)
static constexpr double s
When designing your own protocols to use over the pipe, there are a few things to bear in mind:
- Do as http does: When building a request, send all the options and properties of that request with the request itself in a single go (one flush). Then, the server has everything it needs, and hopefully, it'll shut up for a while and to let the client do something useful in the meantime... The same goes when the server replies to the request: include everything there is to know about the result of the request in the reply.
- The expensive operation should be the request that is made, all other operations should somehow be formulated as options or properties to that request.
- Include a shutdown handshake in whatever protocol you send over the pipe. That way, you can shut things down in a controlled way. Otherwise, and depending on your OS's scheduling quirks, you may catch a SIGPIPE if one end closes its pipe while the other is still trying to read.
Definition at line 379 of file BidirMMapPipe.h.
ROOT 6.18/05 - Reference Guide Generated on Wed Apr 13 2022 01:54:05 (GVA Time) using Doxygen 1.9.4 (15dae504a167d838873053cc96417d8beae78df7).