fixes#11950https://github.com/coder/coder/issues/11950#issuecomment-1987756088 explains the bug
We were also calling into `Unlisten()` and `Close()` while holding the mutex. I don't believe that `Close()` depends on the notification loop being unblocked, but it's hard to be sure, and the safest thing to do is assume it could block.
So, I added a unit test that fakes out `pq.Listener` and sends a bunch of notifies every time we call into it to hopefully prevent regression where we hold the mutex while calling into these functions.
It also removes the use of a `context.Context` to stop the PubSub -- it must be explicitly `Closed()`. This simplifies a bunch of the logic, and is how we use the pubsub anyway.
Fixes#12030
This is a good example of the kind of thing I'd like to address with a time-testing lib. The problem is that there is a race between the watchdog starting it's timer and the test incrementing the time. What would make this easier is if the time-testing library could wait for and assert the call to start the timer before incrementing the time.
adds a watchdog to our pubsub and runs it for Coder server.
If the watchdog times out, it triggers a graceful exit in `coder server` to give any provisioner jobs a chance to shut down.
c.f. #11950
Annoyingly, prometheus Registry collects metrics async, which is causing our test to be racy. They also don't export enough from the Metric interface for us to replicate a synchronous collect, so we have to use Eventually to test.
Adds prometheus metrics to PGPubsub for monitoring its health and performance in production.
Related to #11950 --- additional diagnostics to help figure out what's happening
* chore: add /v2 to import module path
go mod requires semantic versioning with versions greater than 1.x
This was a mechanical update by running:
```
go install github.com/marwan-at-work/mod/cmd/mod@latest
mod upgrade
```
Migrate generated files to import /v2
* Fix gen
* chore: rename store to dbmock for consistency
* chore: remove redundant dbtype package
This wasn't necessary and forked how we do DB types.
* chore: separate pubsub into a new package
This didn't need to be in database and was bloating it.
Colin Adler
Danny Kopping
Cian Johnston
Spike Curtis
Dean Sheather
Kyle Carberry