Commit Graph

302 Commits

Author SHA1 Message Date
Mathias Fredriksson 7d07e670ca
chore: Improve test cleanup (#3112) 2022-07-22 15:14:45 +03:00
David Wahler f7ea016494
Pass git configuration variables via terraform (#3034)
* Pass workspace owner email address to provisioner

* Remove owner_email and owner_username fields from agent metadata

* Add Git environment variables to example templates

* Remove "owner_name" field from provisioner metadata, use username instead

* Remove Git configuration from most templates, add documentation

* Proofreading/typo fixes from @mafredri

* Update example templates to latest version of terraform-provider-coder
2022-07-19 13:24:06 -05:00
Spike Curtis 22febc749a
provisionerd sends failed or complete last (#2732)
* provisionerd sends failed or complete last

Signed-off-by: Spike Curtis <spike@coder.com>

* Move runner into package

Signed-off-by: Spike Curtis <spike@coder.com>

* Remove jobRunner interface

Signed-off-by: Spike Curtis <spike@coder.com>

* renames and slight reworking from code review

Signed-off-by: Spike Curtis <spike@coder.com>

* Reword comment about okToSend

Signed-off-by: Spike Curtis <spike@coder.com>
2022-07-01 09:55:46 -07:00
Kyle Carberry d9da96cad0
fix: Add test for SCP (#2692)
* fix: Elongate agent disconnect timeout in tests

This will fix the flake seen here:
https://github.com/coder/coder/runs/7071719863?check_suite_focus=true

* fix: Add test for SCP

This was hanging due to the stdin pipe never being closed.
A test has been added to make sure it works!
2022-06-27 17:41:53 +01:00
Colin Adler 6aed58f486
feat: add ssh support over wireguard (#2642) 2022-06-24 16:21:46 -05:00
Colin Adler 05b67ab1cf
feat: peer wireguard (#2445) 2022-06-24 10:25:01 -05:00
Jon Ayers ee5918217b
fix: cleanup reaper implementation (#2563)
- Clean up the agent/reaper API to be a more isolated and reusable package.
2022-06-21 18:01:34 -05:00
Spike Curtis 0aa66b4296
Lock the fake database during transactions (#2478)
* Lock the fake database during transactions

Signed-off-by: Spike Curtis <spike@coder.com>

* Add ut for fake database transactions

Signed-off-by: Spike Curtis <spike@coder.com>

* fix lint

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix lint macOS

Signed-off-by: Spike Curtis <spike@coder.com>
2022-06-17 13:50:11 -07:00
Jon Ayers 18973a65c1
fix: Add reaper to coder agent (#2441)
* fix: Add reaper to coder agent

- The coder agent runs as PID 1 in some of our Docker workspaces.
  In such cases it is the responsibility of the init process to
  reap dead processes. Failing to do so can result in an inability
  to create new processes by running out of PIDs.

  This PR adds a reaper to our agent that is only spawned if it
  detects that it is PID1.
2022-06-17 11:51:46 -05:00
Kyle Carberry b9f3fe49cb
fix: Start login shells on macOS and Linux (#2437)
This appends `-l` to the shell command on macOS and Linux.
It also adds environment variable expansion to allow for
chaining from `coder_agent.env`.
2022-06-17 05:54:45 +00:00
Kyle Carberry 66cf59bbe1
fix: Apply environment variables to startup script (#2099)
This was stopping `coder` from being in the path, and allowed
applications started in the script to bypass injected environmnet
variables like `GIT_SSH_COMMAND`.
2022-06-06 14:20:25 -05:00
Kyle Carberry 013f028e55
feat: Add workspace application support (#1773)
* feat: Add app support

This adds apps as a property to a workspace agent.

The resource is added to the Terraform provider here:
https://github.com/coder/terraform-provider-coder/pull/17

Apps will be opened in the dashboard or via the CLI
with `coder open <name>`. If `command` is specified, a
terminal will appear locally and in the web. If `target`
is specified, the browser will open to an exposed instance
of that target.

* Compare fields in apps test

* Update Terraform provider to use relative path

* Add some basic structure for routing

* chore: Remove interface from coderd and lift API surface

Abstracting coderd into an interface added misdirection because
the interface was never intended to be fulfilled outside of a single
implementation.

This lifts the abstraction, and attaches all handlers to a root struct
named `*coderd.API`.

* Add basic proxy logic

* Add proxying based on path

* Add app proxying for wildcards

* Add wsconncache

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* Add workspace route proxying endpoint

- Makes the workspace conn cache concurrency-safe
- Reduces unnecessary open checks in `peer.Channel`
- Fixes the use of a temporary context when dialing a workspace agent

* Add embed errors

* chore: Refactor site to improve testing

It was difficult to develop this package due to the
embed build tag being mandatory on the tests. The logic
to test doesn't require any embedded files.

* Add test for error handler

* Remove unused access url

* Add RBAC tests

* Fix dial agent syntax

* Fix linting errors

* Fix gen

* Fix icon required

* Adjust migration number

* Fix proxy error status code

* Fix empty db lookup
2022-06-04 15:13:37 -05:00
Kyle Carberry d0ed107b08
fix: Add command to reconnecting PTY (#1860)
This fixes #1708 and opens the door for PTYs to execute
non-shell commands!
2022-05-27 14:51:20 -05:00
Garrett Delfosse 1361c1357a
feat: inject USER into shells (#1818) 2022-05-26 18:01:47 -05:00
Garrett Delfosse 781f3d0641
fix: use dir over full path for coder bin (#1795) 2022-05-26 19:05:46 +00:00
Garrett Delfosse 3052a6d88e
Add coder executable to PATH (#1771) 2022-05-26 12:59:41 -05:00
Mathias Fredriksson 527f1f3bc3
feat: Add SSH agent forwarding support to coder agent (#1548)
* feat: Add SSH agent forwarding support to coder agent

* feat: Add forward agent flag to `coder ssh`

* refactor: Share setup between SSH tests, sync goroutines

* feat: Add test for `coder ssh --forward-agent`

* fix: Fix test flakes and implement Deans suggestion for helpers

* fix: Add example to config-ssh

* fix: Allow forwarding agent via -A

Co-authored-by: Cian Johnston <cian@coder.com>
2022-05-25 21:28:10 +03:00
Colin Adler 4543a3b277
fix: log after test exit in `TestAgent/StartupScript` (#1726)
```
$ go test ./agent/ -v -run TestAgent/StartupScript -count 1
=== RUN   TestAgent
=== PAUSE TestAgent
=== CONT  TestAgent
=== RUN   TestAgent/StartupScript
=== PAUSE TestAgent/StartupScript
=== CONT  TestAgent/StartupScript
    t.go:56: 2022-05-24 20:22:39.648 [INFO]	<agent.go:112>	connected
--- PASS: TestAgent (0.00s)
    --- PASS: TestAgent/StartupScript (0.17s)
PASS
panic: Log in goroutine after TestAgent/StartupScript has completed: 2022-05-24 20:22:39.651 [WARN]	<agent.go:130>	agent script failed ...
"error": run:
             github.com/coder/coder/agent.(*agent).runStartupScript
                 /home/colin/Projects/coder/coder/agent/agent.go:183
           - signal: killed
```
2022-05-24 16:03:42 -05:00
Cian Johnston c2f74f3cc2
chore: avoid concurrent usage of t.FailNow (#1683)
* chore: golangci: add linter rule to report usage of t.FailNow inside goroutines
* chore: avoid t.FailNow in goroutines to appease the race detector
2022-05-24 08:58:39 +01:00
Cian Johnston d72c45e483
refactor: workspace autostop_schedule -> ttl (#1578)
Co-authored-by: G r e y <grey@coder.com>
2022-05-19 15:09:27 -04:00
David Wahler a50a6e8638
fix: Make TestAgent and TestWorkspaceAgentPTY less flaky (#1562) 2022-05-18 17:06:17 +00:00
Dean Sheather 9141be3656
feat: add port-forward subcommand (#1350) 2022-05-19 00:10:40 +10:00
Kyle Carberry 3d96785bf5
fix: Add lock around read/write of circular buffer (#1258)
This fixes a race seen in:
https://github.com/coder/coder/runs/6260926628?check_suite_focus=true#step:10:666
2022-05-02 17:31:04 +00:00
Kyle Carberry e531c0930c
fix: Write agent logs to "/tmp/coder-agent.log" for debugging (#1239)
It was difficult to obtain logs for the agent if it failed to
start for some reason. Now they'll go to a consistent spot!
2022-05-02 16:36:51 +00:00
Kyle Carberry 8701e0084c
feat: Update Terraform provider to support "dir" in "coder_agent" (#1219)
This allows users to specify a starting directory for shell sessions.
2022-05-02 10:27:34 -05:00
Kyle Carberry b948f2dab5
fix: Use environment variables for agent authentication (#1238)
* fix: Update GIT_COMMITTER_NAME to use username

This was a mistake when adding the committer fields 🤦.

* fix: Use environment variables for agent authentication

Using files led to situations where running "coder server --dev" would
break `gitssh`. This is applicable in a production environment too. Users
should be able to log into another Coder deployment from their workspace.

Users can still set "CODER_URL" if they'd like with agent env vars!
2022-04-30 16:40:30 +00:00
Kyle Carberry eb606924ab
fix: Update GIT_COMMITTER_NAME to use username (#1237)
This was a mistake when adding the committer fields 🤦.
2022-04-29 20:51:11 -05:00
Kyle Carberry 2acdd3b44f
fix: Add "GIT_COMMITTER_*" to remove Git prompt (#1236) 2022-04-30 01:22:17 +00:00
Kyle Carberry 81577f120a
feat: Add web terminal with reconnecting TTYs (#1186)
* feat: Add web terminal with reconnecting TTYs

This adds a web terminal that can reconnect to resume sessions!
No more disconnects, and no more bad bufferring!

* Add xstate service

* Add the webpage for accessing a web terminal

* Add terminal page tests

* Use Ticker instead of Timer

* Active Windows mode on Windows
2022-04-29 17:30:10 -05:00
Kyle Carberry 8d85d80a55
fix: Use proper shutdown signal for systemd (#1179)
Coder was being killed instantly, which caused builds
to randomly fail!
2022-04-25 21:09:11 -05:00
Kyle Carberry 744a00a55d
feat: Add GIT_COMMITTER information to agent env vars (#1171)
This makes setting up git a bit simpler, and users
can always override these values!

We'll probably add a way to disable our Git integration
anyways, so these could be part of that.
2022-04-25 20:03:54 -05:00
Kyle Carberry 29d55887f9
fix: Update initial window size on SSH TTY (#1174)
It required a window resize before to trigger
a size update. This fixes it!
2022-04-25 23:02:54 +00:00
Kyle Carberry 587cbac498
fix: Swap height and width for TTY size (#1161)
This was causing the TTY to be real wonky on Windows.
It didn't seem to have an effect on Linux, but I suspect
that's because of escape codes.
2022-04-25 15:30:02 -05:00
Kyle Carberry 33b58a0363
fix: Use forward slashes on Windows for gitssh (#1146) 2022-04-25 19:41:52 +00:00
Kyle Carberry 64348882a0
test: Loop if content length is zero for Windows (#1151)
On Windows it seems the file can be created before the
content has been written, which results in comparing
an empty string.
2022-04-25 19:01:49 +00:00
Kyle Carberry a2dd618849
feat: Use environment variables and startup script in agent (#1147)
These values were ignored. Environment variables are applied to
new sessions, and are refreshed on reconnect. This is cool because
a workspace could be updated with new environment variables without
requiring a complete start/stop.

The startup script is only ran once regardless of changes, which
feels like the expected behavior.
2022-04-25 18:30:39 +00:00
Kyle Carberry 5e6e626ace
test: Disable TestAgent/SessionTTY on Windows (#1148)
Either our ConPTY implementation is unstable, or something is
flakey with how it sends output. I'm not sure how our
implementation would sometimes work, so it's best to disable
this for CI stability for now.
2022-04-25 17:53:45 +00:00
Kyle Carberry 185d97a65b
fix: Increase ptytest buffer to resolve flake (#1141)
From investigating the following run:
https://github.com/coder/coder/runs/6156348454?check_suite_focus=true

It's believed that the cause is data being discarded due to the buffer
filling up. This _might_ fix it, but not entirely sure.
2022-04-25 12:14:16 -05:00
G r e y 5ce06769cd
chore: replace todos with issues (#1066) 2022-04-19 12:16:57 -04:00
Kyle Carberry d202f20fdb
feat: Add TURN proxying to enable offline deployments (#1000)
* Add turnconn

* Add option for passing ICE servers

* Log TURN remote address

* Add TURN server to coder start
2022-04-18 22:40:25 +00:00
Kyle Carberry e5a1c305d3
fix: Remove quotes on GitSSH (#1043) 2022-04-18 15:12:39 -05:00
Kyle Carberry 3304db08dd
fix: Agent/SessionTTY flake waiting for terminal prompt (#1002)
For an unknown reason, the prompt wouldn't appear on Windows
randomly in CI. This shouldn't be a necessary check anyways,
because terminal input will be buffered.
2022-04-14 15:55:43 +00:00
Kyle Carberry e8b1a57929
feat: Add support for VS Code and JetBrains Gateway via SSH (#956)
* Improve CLI documentation

* feat: Allow workspace resources to attach multiple agents

This enables a "kubernetes_pod" to attach multiple agents that
could be for multiple services. Each agent is required to have
a unique name, so SSH syntax is:

`coder ssh <workspace>.<agent>`

A resource can have zero agents too, they aren't required.

* Add tree view

* Improve table UI

* feat: Allow workspace resources to attach multiple agents

This enables a "kubernetes_pod" to attach multiple agents that
could be for multiple services. Each agent is required to have
a unique name, so SSH syntax is:

`coder ssh <workspace>.<agent>`

A resource can have zero agents too, they aren't required.

* Rename `tunnel` to `skip-tunnel`

This command was `true` by default, which causes
a confusing user experience.

* Add disclaimer about editing templates

* Add help to template create

* Improve workspace create flow

* Add end-to-end test for config-ssh

* Improve testing of config-ssh

* Fix workspace list

* feat: Add support for VS Code and JetBrains Gateway via SSH

This fixes various bugs that made this not work:
- Incorrect max message size in `peer`
- Incorrect reader buffer size in `peer`
- Lack of SFTP support in `agent`
- Lack of direct-tcpip support in `agent`
- Misuse of command from session. It should always use the shell
- Blocking on SSH session, only allowing one at a time

Fixes #833 too.

* Fix config-ssh command with socat
2022-04-12 00:17:18 +00:00
Kyle Carberry fb9dc4f346
feat: Improve resource preview and first-time experience (#946)
* Improve CLI documentation

* feat: Allow workspace resources to attach multiple agents

This enables a "kubernetes_pod" to attach multiple agents that
could be for multiple services. Each agent is required to have
a unique name, so SSH syntax is:

`coder ssh <workspace>.<agent>`

A resource can have zero agents too, they aren't required.

* Add tree view

* Improve table UI

* feat: Allow workspace resources to attach multiple agents

This enables a "kubernetes_pod" to attach multiple agents that
could be for multiple services. Each agent is required to have
a unique name, so SSH syntax is:

`coder ssh <workspace>.<agent>`

A resource can have zero agents too, they aren't required.

* Rename `tunnel` to `skip-tunnel`

This command was `true` by default, which causes
a confusing user experience.

* Add disclaimer about editing templates

* Add help to template create

* Improve workspace create flow

* Add end-to-end test for config-ssh

* Improve testing of config-ssh

* Fix workspace list

* Fix config ssh tests

* Update cli/configssh.go

Co-authored-by: Cian Johnston <public@cianjohnston.ie>

* Fix requested changes

* Remove socat requirement

* Fix resources not reading in TTY

Co-authored-by: Cian Johnston <public@cianjohnston.ie>
2022-04-11 18:54:30 -05:00
Garrett Delfosse 38f074254b
feat: wrap ssh with coder key (#894) 2022-04-07 22:40:27 +00:00
Kyle Carberry 6612e3c9c7
feat: Add config-ssh command (#735)
* feat: Add config-ssh command

Closes #254 and #499.

* Fix Windows support
2022-03-30 17:59:54 -05:00
Kyle Carberry 82dfd6c72f
feat: Add UI for awaiting agent connections (#578)
* feat: Add stage to build logs

This adds a stage property to logs, and refactors the job logs
cliui.

It also adds tests to the cliui for build logs!

* feat: Add stage to build logs

This adds a stage property to logs, and refactors the job logs
cliui.

It also adds tests to the cliui for build logs!

* feat: Add config-ssh and tests for resiliency

* Rename "Echo" test to "ImmediateExit"

* Fix Terraform resource agent association

* Fix logs post-cancel

* Fix select on Windows

* Remove terraform init logs

* Move timer into it's own loop

* Fix race condition in provisioner jobs

* Fix requested changes
2022-03-28 19:19:28 -05:00
Kyle Carberry 6be949a88e
feat: Separate workspace agent for tests (#567)
This adds tests for Google Cloud authentication, and lays
the ground-work for future agent auth types in the future.
2022-03-25 14:48:08 -05:00
Kyle Carberry c451f4e685
feat: Add templates to create working release (#422)
* Add templates

* Move API structs to codersdk

* Back to green tests!

* It all works, but now with tea! 🧋

* It works!

* Add cancellation to provisionerd

* Tests pass!

* Add deletion of workspaces and projects

* Fix agent lock

* Add clog

* Fix linting errors

* Remove unused CLI tests

* Rename daemon to start

* Fix leaking command

* Fix promptui test

* Update agent connection frequency

* Skip login tests on Windows

* Increase tunnel connect timeout

* Fix templater

* Lower test requirements

* Fix embed

* Disable promptui tests for Windows

* Fix write newline

* Fix PTY write newline

* Fix CloseReader

* Fix compilation on Windows

* Fix linting error

* Remove bubbletea

* Cleanup readwriter

* Use embedded templates instead of serving over API

* Move templates to examples

* Improve workspace create flow

* Fix Windows build

* Fix tests

* Fix linting errors

* Fix untar with extracting max size

* Fix newline char
2022-03-22 13:17:50 -06:00
Kyle Carberry bf0ae8f573
feat: Refactor API routes to use UUIDs instead of friendly names (#401)
* Add client for agent

* Cleanup code

* Fix linting error

* Rename routes to be simpler

* Rename workspace history to workspace build

* Refactor HTTP middlewares to use UUIDs

* Cleanup routes

* Compiles!

* Fix files and organizations

* Fix querying

* Fix agent lock

* Cleanup database abstraction

* Add parameters

* Fix linting errors

* Fix log race

* Lock on close wait

* Fix log cleanup

* Fix e2e tests

* Fix upstream version of opencensus-go

* Update coderdtest.go

* Fix coverpkg

* Fix codecov ignore
2022-03-07 11:40:54 -06:00
Kyle Carberry 55784d3565
fix: Remove ICEServer proxying from client to server (#400)
This made testing simple, but enabled insecure behavior. This allows
the listener to fetch ICEServers from a remote location, which will
likely be coderd.
2022-03-06 18:41:02 -06:00
Kyle Carberry 91bf8636fb
feat: Add workspace agent for SSH (#318)
* feat: Add workspace agent for SSH

This adds the initial agent that supports TTY
and execution over SSH. It functions across MacOS,
Windows, and Linux.

This does not handle the coderd interaction yet,
but does setup a simple path forward.

* Fix pty tests on Windows

* Fix log race

* Lock around dial error to fix log output

* Fix context return early

* fix: Leaking yamux session after HTTP handler is closed

Closes #317. We depended on the context canceling the yamux connection,
but this isn't a sync operation. Explicitly calling close ensures the
handler waits for yamux to complete before exit.

* Lock around close return

* Force failure with log

* Fix failed handler

* Upgrade dep

* Fix defer inside loops

* Fix context cancel for HTTP requests

* Fix resize
2022-02-18 23:13:32 -06:00