Fun with pipes on Win32 (2)

Previously, we'd moved from using anonymous pipes to communicate with our subprocess, to named pipes with unique names. Problem solved, right? Sadly not!

Rather than create completely unique names, I created names that were guaranteed to be unique for the lifetime of the sub-process. Generating totally unique names just seemed like overkill! So, I create one sub-process, then the next one, and use the same unique pipe name each time. As long as the sub-process has died, and the parent's end of the pipes are closed, the old pipe will go away, and a new pipe with the same name can be created.

Furthermore, our sub-process manager is multi-threaded. So, we can fire off multiple sub-proceses at the same time. However, the pipes weren't being closed properly (so we couldn't reuse the names) when run in multi-threaded mode. Why?

It's all to do with the way Win32 passes handles to sub-processes. In order to get a handle into a sub-process, we have to make the handle inheritable, and then spawn the sub-process with 'inherit all inheritable handles'. Unfortunately, if you spawn two subprocesses around the same time, both children will inherit the handles (even if they're not wired up to stdin/out/err in that particular process). So, the other process is holding the pipe open.

We can reduce the window by closing the pipe as soon as the sub-process is fired off, but the race is still there. We could then hold a lock during the spawn step, but it's slightly icky to introduce the extra synchronisation. Personally, I think it'd have been nice to control exactly which handles are passed to which subprocess, but this looks like a weakness of the Win32 API. Unix-style APIs, using fork/exec allow you to control the exact handle (file descriptor) set-up for the child. Gah.

Another funkier approach that I didn't try is to spawn the sub-process suspended, use DuplicateHandle to poke the handles into the sub-process, and then unsuspend the process, to spawn a subprocess with exactly the handles required.

So what did I do? I just made the pipe names globally unique.

Posted 2009-09-15.