hypothesis / h / 8611 / 1
68%
master: 97%

LAST BUILD BRANCH: realtime-update-v1-h-only-1
DEFAULT BRANCH: master
Ran
Files 87
Run time 2s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 68.155% (-0.04%) from 68.193%
8611.1

push

travis-ci

nickstenning 
streamer: Don't silently stop processing on exceptions

Prior to this commit, if an exception is thrown by a message handler
(i.e. `h.streamer.broadcast_annotation_message` or
`h.streamer.broadcast_session_change_message`) then the worst possible
combination of events will occur:

1. NSQ will not be notified, and the message will thus not be requeued.
2. The greenlet processing the internal event queue will crash. There is
   no support in the code for restarting this greenlet, and thus all
   WebSocket clients connected to this worker (or that connect
   subsequently) will stop receiving updates.
3. The readers connected to NSQ will continue to receive messages and
   push them onto the internal event queue. These messages will never be
   handled, and will continue to stack up in memory until, presumably,
   the process OOMs.

All in all, not a terribly happy situation.

This commit completely alters how we handle queue messages, fixing all
three issues. The main changes are as follows:

- The internal queue is gone. Instead, we spin up a greenlet on app boot
  which is responsible, in turn, for starting the queue reader
  processes.
- The handling of all messages is done synchronously in the `on_message`
  handlers of the readers. In the event of an exception, `gnsq` will
  handle requeueing the message with NSQ.
- If any reader greenlet terminates (an event which should never occur)
  then all reader greenlets are killed and restarted. A message is also
  written to the logs to indicate that this happened.

There are numerous further simplifications which can now be made to this
code, but I've omitted them from this commit in the interests of making
the changes as clear as possible.

2504 of 3674 relevant lines covered (68.15%)

0.68 hits per line

Source Files on job 8611.1