Simon Frankau's blog

Fun with pipes on Win32 (1)

Most of my work nowadays is writing Haskell, but to interop with other systems we have COM server which fires off the Haskell processes and communicates with them through pipes - we could do things less indirectly, but this is a nice, simple way of decoupling the components. However, the pipes have been really quite irritating to deal with.

First off, Win32 anonymous pipes (the kind of pipes you'd expect to use to communicate with a child process) don't support WaitForMultipleObjects. This is really quite irritating if you want to support select-style reading from your sub-process. Why might you want to do this? Well, the alternatives are:

  • Just do blocking I/O. Except if you're reading from stdout and stderr on the sub-process, if you're blocking on one handle, and the child process writes a pipe's-worth to the other handle, you have deadlock.
  • Do a polling loop, non-blocking reading one, then the other, with a little sleep inbetween. This is no solution, I'm just mentioning it because it's such a bad idea.
  • Fire off an extra thread or two, just to blocking read the handles. Yuck, yuck, yuck. All the extra set-up of starting heavyweight threads, with a meg or so of address space, makes this heavy and unscalable (ok, there's plenty of overhead in starting a subprocess, but it's the principle!).

So, instead I tried WaitForMultipleObjects-ing on the handle. MSDN said I couldn't, but MSDN lies about lots of stuff. It turns out MSDN was telling the truth, although it wasn't immediately obvious. It basically immediately returns (without error), so we then perform a blocking read on the pipe, leading to the deadlock described above, as we wait to read one pipe, and the child waits to write the other.

The solution? Use named pipes, with unique names. This link contains the basic source, which you may want to adjust for your personal use, while complaining that the Win32 API really should do this right itself, especially since it's such a common usage case.

Posted 2009-09-15.

Full index

Latest posts

RSS

Mail me at random.user@arbitrary.name.