Commit Graph

85 Commits

Author SHA1 Message Date
Kayla Washburn c194119689
chore: rename `AwaitTemplateVersionJobCompleted` and `AwaitWorkspaceBuildJobCompleted` (#10003) 2023-10-03 11:02:56 -06:00
Cian Johnston 93ef696b57
refactor(agent): add agenttest.New helper function (#9812)
* Adds agenttest.New() helper function
* Makes sure agent gets closed on test cleanup
* Makes sure you don't forget to set session token
* Sets the agent and client logger automatically
2023-09-26 12:05:19 +01:00
Kyle Carberry 1262eef2c0
feat: add support for `coder_script` (#9584)
* Add basic migrations

* Improve schema

* Refactor agent scripts into it's own package

* Support legacy start and stop script format

* Pipe the scripts!

* Finish the piping

* Fix context usage

* It works!

* Fix sql query

* Fix SQL query

* Rename `LogSourceID` -> `SourceID`

* Fix the FE

* fmt

* Rename migrations

* Fix log tests

* Fix lint err

* Fix gen

* Fix story type

* Rename source to script

* Fix schema jank

* Uncomment test

* Rename proto to TimeoutSeconds

* Fix comments

* Fix comments

* Fix legacy endpoint without specified log_source

* Fix non-blocking by default in agent

* Fix resources tests

* Fix dbfake

* Fix resources

* Fix linting I think

* Add fixtures

* fmt

* Fix startup script behavior

* Fix comments

* Fix context

* Fix cancel

* Fix SQL tests

* Fix e2e tests

* Interrupt on Windows

* Fix agent leaking script process

* Fix migrations

* Fix stories

* Fix duplicate logs appearing

* Gen

* Fix log location

* Fix tests

* Fix tests

* Fix log output

* Show display name in output

* Fix print

* Return timeout on start context

* Gen

* Fix fixture

* Fix the agent status

* Fix startup timeout msg

* Fix command using shared context

* Fix timeout draining

* Change signal type

* Add deterministic colors to startup script logs

---------

Co-authored-by: Muhammad Atif Ali <atif@coder.com>
2023-09-25 16:47:17 -05:00
Mathias Fredriksson 530dd9d247
fix(coderd): subscribe to workspace when streaming agent logs to detect outdated build (#9729)
Fixes #9721
2023-09-19 20:02:27 +03:00
Cian Johnston 653488e8ee
fix(coderd): emit CollectedAt as UTC in convertWorkspaceAgentMetadata (#9700) 2023-09-18 09:17:18 +01:00
Cian Johnston 65db7a71b7
feat(coderd/database/dbtestutil): set default database timezone to non-UTC in unit tests (#9672)
- Adds dbtestutil.WithTimezone(tz) to allow setting the timezone for a test database.
- Modifies our test database setup code to pick a consistently weird timezone for the database.
- Adds the facility randtz.Name() to pick a random timezone which is consistent across subtests (via sync.Once).
- Adds a linter rule to warn against setting the test database timezone to UTC.
2023-09-15 09:01:32 +01:00
Mathias Fredriksson 19d7da3d24
refactor(coderd/database): split `Time` and `Now` into `dbtime` package (#9482)
Ref: #9380
2023-09-01 16:50:12 +00:00
Jon Ayers ee24260614
feat: allow configuring display apps from template (#9100) 2023-08-30 14:53:42 -05:00
Mathias Fredriksson 487bdc2e08
fix(coderd): allow `workspaceAgentLogs` follow to return on non-latest-build (#9382) 2023-08-28 19:46:42 +00:00
Spike Curtis 60d5002eb6
refactor: change template archive extraction to be on provisioner (#9264)
* refactor provisionersdk protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisioners to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisionerd to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor tests & proto renames

* Fixes from self-review

Signed-off-by: Spike Curtis <spike@coder.com>

* appease fmt & link

Signed-off-by: Spike Curtis <spike@coder.com>

* code review fixes & e2e fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* More fmt

Signed-off-by: Spike Curtis <spike@coder.com>

* Code review fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* new gen; use uuid for session workdir

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert nix-based gen CI task until dogfood is on nix

Signed-off-by: Spike Curtis <spike@coder.com>

* revert deleting dogfood Docker stuff

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert "revert deleting dogfood Docker stuff"

This reverts commit 9762158167.

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-25 06:10:15 +00:00
Ammar Bandukwala 630ec55c48
fix(coderd): remove rate limits from agent metadata (#9308)
Include the full update message in the PubSub notification so that
we don't have to refresh metadata from the DB and can avoid rate
limiting.
2023-08-24 15:18:42 -05:00
Cian Johnston 5d4a17717f
refactor(coderd): fetch owner information when authorizing workspace agent (#9123)
* Refactors the existing httpmw tests to use dbtestutil so that we can test them against a real database if desired,
* Modifies the GetWorkspaceAgentByAuthToken to return the owner and associated roles, removing the need for additional queries
2023-08-21 15:49:26 +01:00
Kyle Carberry 22e781eced
chore: add /v2 to import module path (#9072)
* chore: add /v2 to import module path

go mod requires semantic versioning with versions greater than 1.x

This was a mechanical update by running:
```
go install github.com/marwan-at-work/mod/cmd/mod@latest
mod upgrade
```

Migrate generated files to import /v2

* Fix gen
2023-08-18 18:55:43 +00:00
Dean Sheather 07fd73c4a0
chore: allow multiple agent subsystems, add exectrace (#8933) 2023-08-08 22:10:28 -07:00
Dean Sheather c575292ba6
fix: fix tailnet netcheck issues (#8802) 2023-08-02 01:50:43 +10:00
Kyle Carberry bd944e0d21
chore: rename startup logs to agent logs (#8649)
* chore: rename startup logs to agent logs

This also adds a `source` property to every agent log. It
should allow us to group logs and display them nicer in
the UI as they stream in.

* Fix migration order

* Fix naming

* Rename the frontend

* Fix tests

* Fix down migration

* Match enums for workspace agent logs

* Fix inserting log source

* Fix migration order

* Fix logs tests

* Fix psql insert
2023-07-28 15:57:23 +00:00
Dean Sheather 2f0a9996e7
chore: add derpserver to wsproxy, add proxies to derpmap (#7311) 2023-07-27 02:21:04 +10:00
Kyle Carberry 98164f687e
fix!: remove startup logs eof for streaming (#8528)
* fix: remove startup logs eof for streaming

We have external utilities like logstream-kube that may send
logs after an agent shuts down unexpectedly to report additional
information. In a recent change we stopped accepting these logs,
which broke these utilities.

In the future we'll rename startup logs to agent logs or something
more generalized so this is less confusing in the future.

* fix(cli/cliui): handle never ending startup log stream in Agent

---------

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2023-07-18 09:57:29 -06:00
Mathias Fredriksson b73f9d8e86
feat: add computed workspace and agent health fields to the api (#8280) 2023-07-10 12:40:11 +03:00
Kyle Carberry 34467a3289
feat: add github device flow for authentication (#8232)
* feat: add github device flow for authentication

This will allow us to add a GitHub OAuth provider out-of-the-box
to reduce setup requirements.

* Improve askpass view

* Add routes to improve clarity of git auth

* Redesign the git auth page

* Refactor to add a page view

* Fix sideways layout

* Remove legacy notify

* Fix git auth redirects

* Add E2E tests

* Fix route documentation

* Fix imports

* Remove unused imports

* Fix E2E web test

* Fix friendly message appearance

* Fix layout shifting for full-screen sign-in

* Fix height going to 100%

* Fix comments
2023-06-29 18:58:01 +00:00
Mathias Fredriksson d3c39b60c9
feat: add agent log streaming and follow provisioner format (#8170) 2023-06-28 10:54:13 +02:00
Dean Sheather a28d422c35
feat: add flag to disable all direct connections (#7936) 2023-06-21 22:02:05 +00:00
Mathias Fredriksson 8dac0356ed
refactor: replace startup script logs EOF with starting/ready time (#8082)
This commit reverts some of the changes in #8029 and implements an
alternative method of keeping track of when the startup script has ended
and there will be no more logs.

This is achieved by adding new agent fields for tracking when the agent
enters the "starting" and "ready"/"start_error" lifecycle states. The
timestamps simplify logic since we don't need understand if the current
state is before or after the state we're interested in. They can also be
used to show data like how long the startup script took to execute. This
also allowed us to remove the EOF field from the logs as the
implementation was problematic when we returned the EOF log entry in the
response since requesting _after_ that ID would give no logs and the API
would thus lose track of EOF.
2023-06-20 14:41:55 +03:00
Mathias Fredriksson 0c5077464b
fix: avoid missed logs when streaming startup logs (#8029)
* feat(coderd,agent): send startup log eof at the end

* fix(coderd): fix edge case in startup log pubsub

* fix(coderd): ensure startup logs are closed on lifecycle state change (fallback)

* fix(codersdk): fix startup log channel shared memory bug

* fix(site): remove the EOF log line
2023-06-16 17:14:22 +03:00
Mathias Fredriksson 1d0fae83a2
fix(coderd): prevent lost messages in `watchWorkspaceAgentMetadata` (#7934)
* fix(codersdk): wait for subscription in WatchWorkspaceAgentMetadata
* fix(coderd): subscribe before sending initial metadata event
* test(coderd): add retries to TestWorkspaceAgent_Metadata to avoid flake
2023-06-13 12:21:06 +00:00
Jon Ayers 00a2413c03
feat: add telemetry support for workspace agent subsystem (#7579) 2023-05-17 22:49:25 -05:00
Kyle Carberry 11ac3be4d8
fix: redirect to login on gitauth invalid session (#7416) 2023-05-04 17:19:44 +00:00
Ammar Bandukwala 80bf042528
chore(coderd): remove timing check (#7144) 2023-04-17 17:40:02 +00:00
Dean Sheather 5398150c25
chore: move workspace apps tests to new package (#7025)
* chore: move workspace apps tests to new package
* chore: move reconnecting pty to apptest package
2023-04-06 16:35:27 -05:00
Ammar Bandukwala 24d8644c0b
chore: de-flake TestWorkspaceAgent_Metadata (round 2) (#7039)
This time, we keep the timing / "racey" tests, but avoid running
them in the harsher CI conditions.
2023-04-06 21:10:13 +00:00
Ammar Bandukwala 2637f5501b
chore: de-flake TestWorkspaceAgent_Metadata (#7036) 2023-04-06 17:20:16 +00:00
Colin Adler 8683169e71
fix: agent metadata flake (#6962) 2023-04-03 04:45:23 +00:00
Ammar Bandukwala ca4fa81570
feat: add agent metadata (#6614) 2023-03-31 15:26:19 -05:00
Kyle Carberry 5c1dc1b7fe
fix: iterate through all workspace updates on logs overflow (#6885)
This was causing some flakes!
2023-03-30 10:05:45 -05:00
Ammar Bandukwala 2bd6d2908e
feat: convert entire CLI to clibase (#6491)
I'm sorry.
2023-03-23 17:42:20 -05:00
Kyle Carberry cb7375450b
feat: add startup script logs to the ui (#6558)
* Add startup script logs to the database

* Add coderd endpoints for startup script logs

* Push startup script logs from agent

* Pull startup script logs on frontend

* Rename queries

* Add constraint

* Start creating log sending loop

* Add log sending to the agent

* Add tests for streaming logs

* Shorten notify channel name

* Add FE

* Improve bulk log performance

* Finish UI display

* Fix startup log visibility

* Add warning for overflow

* Fix agent queue logs overflow

* Display staartup logs in a virtual DOM for performance

* Fix agent queue with loads of logs

* Fix authorize test

* Remove faulty test

* Fix startup and shutdown reporting error

* Fix gen

* Fix comments

* Periodically purge old database entries

* Add test fixture for migration

* Add Storybook

* Check if there are logs when displaying features

* Fix startup component overflow gap

* Fix startup log wrapping

---------

Co-authored-by: Asher <ash@coder.com>
2023-03-23 14:09:13 -05:00
Kyle Carberry df31636e72
feat: pass `access_token` to `coder_git_auth` resource (#6713)
This allows template authors to leverage git auth to perform
custom actions, like clone repositories.
2023-03-22 19:37:08 +00:00
Kyle Carberry aaa3b31a0b
chore: add echo helper to create an agent with token (#6576)
This should reduce some LOC and duplication in tests!
2023-03-21 18:03:38 +00:00
Mathias Fredriksson 22e3ff96be
feat(agent): Add shutdown lifecycle states and shutdown_script support (#6139)
* feat(api): Add agent shutdown lifecycle states

* feat(agent): Add shutdown_script support

* feat(agent): Add shutdown_script timeout

* feat(site): Support new agent lifecycle states

---

Co-authored-by: Marcin Tojek <marcin@coder.com>
2023-03-06 21:34:00 +02:00
Kyle Carberry 2ff1c6d613
feat: add agent stats for different connection types (#6412)
This allows us to track when our extensions are used, when the
web terminal is used, and average connection latency to the agent.
2023-03-02 08:06:00 -06:00
Kyle Carberry 05e449943d
chore: convert agent stats to use a table (#6374)
* chore: convert workspace agent stats from json to table

* chore: convert agent stats to use a table

Backwards compatibility becomes hard when all agent stats are in a JSON blob.
We also want to query this table for new agents that are failing health checks
so we can display it in the UI.

* Fix migration using default values
2023-02-28 13:33:33 -06:00
Kyle Carberry 7f226d4f90
feat: add support for `coder_git_auth` data source (#6334)
* Add git auth providers schema

* Pipe git auth providers to the schema

* Add git auth providers to the API

* Add gitauth endpoint to query authenticated state

* Add endpoint to query git state

* Use BroadcastChannel to automatically authenticate with Git

* Add error validation for submitting the create workspace form

* Fix panic on template dry-run

* Add tests for the template version Git auth endpoint

* Show error if no gitauth is configured

* Add gitauth to cliui

* Fix unused method receiver

* Fix linting errors

* Fix dbauthz querier test

* Fix make gen

* Add JavaScript test for git auth

* Fix bad error message

* Fix provisionerd test race

See https://github.com/coder/coder/actions/runs/4277960646/jobs/7447232814

* Fix requested changes

* Add comment to CreateWorkspacePageView
2023-02-27 10:18:19 -06:00
Mathias Fredriksson a414de9e81
fix(tailnet): Improve tailnet setup and agentconn stability (#6292)
* fix(tailnet): Improve start and close to detect connection races

* fix: Prevent agentConn use before ready via AwaitReachable

* fix(tailnet): Ensure connstats are closed on conn close

* fix(codersdk): Use AwaitReachable in DialWorkspaceAgent

* fix(tailnet): Improve logging via slog.Helper()
2023-02-24 13:11:28 +02:00
Ammar Bandukwala f05609b4da
chore: format Go more aggressively 2023-02-18 18:32:09 -06:00
Kyle Carberry 7ad87505c8
chore: move agent functions from `codersdk` into `agentsdk` (#5903)
* chore: rename `AgentConn` to `WorkspaceAgentConn`

The codersdk was becoming bloated with consts for the workspace
agent that made no sense to a reader. `Tailnet*` is an example
of these consts.

* chore: remove `Get` prefix from *Client functions

* chore: remove `BypassRatelimits` option in `codersdk.Client`

It feels wrong to have this as a direct option because it's so infrequently
needed by API callers. It's better to directly modify headers in the two
places that we actually use it.

* Merge `appearance.go` and `buildinfo.go` into `deployment.go`

* Merge `experiments.go` and `features.go` into `deployment.go`

* Fix `make gen` referencing old type names

* Merge `error.go` into `client.go`

`codersdk.Response` lived in `error.go`, which is wrong.

* chore: refactor workspace agent functions into agentsdk

It was odd conflating the codersdk that clients should use
with functions that only the agent should use. This separates
them into two SDKs that are closely coupled, but separate.

* Merge `insights.go` into `deployment.go`

* Merge `organizationmember.go` into `organizations.go`

* Merge `quota.go` into `workspaces.go`

* Rename `sse.go` to `serversentevents.go`

* Rename `codersdk.WorkspaceAppHostResponse` to `codersdk.AppHostResponse`

* Format `.vscode/settings.json`

* Fix outdated naming in `api.ts`

* Fix app host response

* Fix unsupported type

* Fix imported type
2023-01-29 15:47:24 -06:00
Mathias Fredriksson 138887de7e
feat: Add workspace agent lifecycle state reporting (#5785) 2023-01-24 14:24:27 +02:00
Kyle Carberry 8b73844f69
feat: Validate Git tokens before consuming them (#5167)
* feat: Validate Git tokens before consuming them

This works the exact same way that the Git credential manager does. It ensures the user token is valid before returning it to the client.

It's been manually tested on GitHub, GitLab, and BitBucket.

* Fix requested changes
2022-11-29 12:08:27 -06:00
Colin Adler ae38bbeab6
chore: refactor agent stats streaming (#5112) 2022-11-18 16:46:53 -06:00
Mathias Fredriksson c1ecc91aab
feat: Add fallback troubleshooting URL for coder agents (#5005) 2022-11-16 12:53:02 +02:00
Kyle Carberry fc0a493b72
feat: Add `no_refresh` option to Git auth configs (#5097)
This allows organizations to disable refreshing Git tokens
and instead prompt for authentication again.
2022-11-15 21:06:13 +00:00