Previously, we'd moved from using anonymous pipes to communicate with our subprocess, to named pipes with unique names. Problem solved, right? Sadly not!
Rather than create completely unique names, I created names that were guaranteed to be unique for the lifetime of the sub-process. Generating totally unique names just seemed like overkill! So, I create one sub-process, then the next one, and use the same unique pipe name each time. As long as the sub-process has died, and the parent's end of the pipes are closed, the old pipe will go away, and a new pipe with the same name can be created.
Furthermore, our sub-process manager is multi-threaded. So, we can fire off multiple sub-proceses at the same time. However, the pipes weren't being closed properly (so we couldn't reuse the names) when run in multi-threaded mode. Why?
It's all to do with the way Win32 passes handles to sub-processes. In order to get a handle into a sub-process, we have to make the handle inheritable, and then spawn the sub-process with 'inherit all inheritable handles'. Unfortunately, if you spawn two subprocesses around the same time, both children will inherit the handles (even if they're not wired up to stdin/out/err in that particular process). So, the other process is holding the pipe open.
We can reduce the window by closing the pipe as soon as the sub-process is fired off, but the race is still there. We could then hold a lock during the spawn step, but it's slightly icky to introduce the extra synchronisation. Personally, I think it'd have been nice to control exactly which handles are passed to which subprocess, but this looks like a weakness of the Win32 API. Unix-style APIs, using fork/exec allow you to control the exact handle (file descriptor) set-up for the child. Gah.
Another funkier approach that I didn't try is to spawn the sub-process suspended, use DuplicateHandle to poke the handles into the sub-process, and then unsuspend the process, to spawn a subprocess with exactly the handles required.
So what did I do? I just made the pipe names globally unique.
Posted 2009-09-15.