Skip to content

Subprocess Pipe Deadlock in Blender Addons

This document explains a critical issue encountered when running background processes from within a Blender modal operator and how it was resolved.

The Issue: Pipe Deadlock

When running an external command (like argos run) from a Blender addon, the communication typically happens via pipes (stdin, stdout, and stderr). A common pitfall occurs when handling large amounts of data:

  1. Limited OS Buffers: Operating systems allocate a fixed-size buffer for pipes (often 64KB).
  2. Full Buffers: If the subprocess writes more than 64KB to stdout or stderr without the parent process (Blender) reading it, the subprocess blocks, waiting for space to become available.
  3. Circular Wait: If Blender is simultaneously waiting for the subprocess to finish (p.poll() or p.wait()) or is busy trying to write a large amount of data to stdin that the subprocess isn't yet reading, a deadlock occurs. Neither process can move forward.

In our case, large OBJ files being piped through stdin and large results coming back through stdout were frequently triggering this deadlock, causing Blender to stall indefinitely even though the process was still "running" in the background.

The Resolution

To prevent these deadlocks, the communication must be handled asynchronously:

1. Asynchronous I/O Threads

Instead of reading or writing directly from the main Blender thread, we now use background threads for all pipe operations: - _write_input: A dedicated thread that writes data to stdin and, crucially, closes the pipe immediately after finishing. This sends an EOF signal that many algorithms require to start processing. - _enqueue_output: Separate threads for stdout and stderr that continuously read data into a thread-safe queue.Queue. This ensures the OS pipe buffers are always being drained, regardless of what Blender is doing.

2. Binary Mode Communication

By switching to binary mode (removing text=True and bufsize=1), we eliminate the overhead of line-buffering and encoding checks during the transfer. This significantly improves performance for large datasets and avoids potential deadlocks caused by encoding-related stalls.

3. Incremental Queue Draining

The Blender modal operator now polls these queues during every timer tick (0.1s). This allows Blender to collect data incrementally without blocking the UI, ensuring that the background threads never get stuck because their internal queues are full.

4. Robust Finalization

Once the process exits, a final "drain" of the queues ensures all remaining output is collected before the data is decoded and imported into the viewport.

By decoupling the I/O from the main thread and ensuring all pipes are handled concurrently, the addon can now safely process datasets of any size.