tree 43cfbbb426e70088ab1adf02ef06f83c64867ae6
parent 9bb5ce78dacf6ba6bd216cb19da60bddf9f17574
author Jean de Klerk <deklerk@google.com> 1553033527 -0600
committer Jean de Klerk <deklerk@google.com> 1553609829 +0000

pubsub: cap bundler byte limit

Bundler uses BufferedByteLimit as the weighted semaphore value which controls
how many items are handled. Each item gets handled in a goroutine.

Currently, we use maxInt, which means that the bundler will happily
accept and try to process way more items than most systems have memory
for. Let's cap it at 10 * MaxPublishRequestBytes. This is an arbitrary
number chosen because:

- It's a reasonable amount of messages handled even if each message is
max size. (10)
- It's 500MB, which is a small enough amount of memory for most all
environments to handle.

We might want to expose this under PublishSettings, but for now we'll
wait and see if any users request it before complicating those settings
further.

The result for users is that if they do,

```
pr := t.Publish(ctx, &pubsub.Message{Data: []byte(id)})
<-pr.Ready()
if _, err := pr.Get(ctx); err != nil {
    panic(err)
}
```

`err` will result in `bundler reached buffered byte limit` when the limit
is reached.

Fixes #1273

Change-Id: I7e16018f89a9e4f29b3ee5336b31460d130fd780
Reviewed-on: https://code-review.googlesource.com/c/gocloud/+/39170
Reviewed-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Chris Broadfoot <cbro@google.com>