Skip to content

Conversation

@pperanich
Copy link
Collaborator

No description provided.

@griffinmilsap
Copy link
Collaborator

Love this addition!

Currently Subscribers/InputStreams take TWO parameters to dictate leaky behavior; one boolean to set it as a leaky queue and another to dictate how many messages should be queued before leaking occurs. This feels like it should just be one parameter, and I'd argue for just the leaky boolean.

  1. max_queue is bounded by num_buffers for a particular Publisher/channel connection
  2. Any particular subscriber can receive messages from multiple Publisher/channel connections that may have different num_buffers
  3. Pausing due to backpressure is handled by a Publisher's perception of backpressure. In order for a leaky subscriber to NOT cause backpressure, max_queue must be set to a MAXIMUM value of num_buffers - 1

I believe it's unexpected behavior when a leaky subscriber still causes backpressure because its max_queue size is too large, so I'd like to just set max_queue = 1 when its a leaky queue. This is morally equivalent to double buffering.

One very important note -- because of this, we actually end up with a weird edge case when someone sets num_buffers = 1 enforcing a "soft-realtime" condition with minimal latency -- leaky subscribers will also only buffer one message, but the publisher will still see this buffering and cause backpressure. While this is unexpected behavior, I do not believe there's any way to handle this edge case in the expected way. As such, it might be useful to update the documentation (and maybe issue a warning) when users run any publisher with num_buffers = 1 to indicate that backpressure will be enforced regardless of subscriber leakiness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants