Commit Graph

65 Commits

Author SHA1 Message Date
Kyle Carberry bbe2baf3f6
fix: Ignore all hidden files and folders in archive (#4382)
This also adds a suite of tests to ensure this cannot happen again!
2022-10-06 00:36:45 +00:00
Dean Sheather 2a66395fb7
feat: use app wildcards for apps if configured (#4263)
* feat: use app wildcards for apps if configured

* feat: relative_path -> subdomain

- rename relative_path -> subdomain when referring to apps
    - migrate workspace_apps.relative_path to workspace_apps.subdomain
- upgrade coder/coder terraform module to 0.5.0
2022-10-05 19:23:01 +00:00
Kyle Carberry 06d7e368ab
fix: Ignore hidden folders when archiving (#4370)
Fixes #4369.
2022-10-04 22:27:14 +00:00
Garrett Delfosse 4c8be34d81
feat: add health check monitoring to workspace apps (#4114) 2022-09-23 15:51:04 -04:00
Bruno Quaresma 214e59452f
feat: Show custom resource icons in the UI (#4020) 2022-09-13 11:32:59 -03:00
Kyle Carberry 6e20f9c729
fix: Recursively ignore hidden folders (#3997)
Fixes #3938.
2022-09-11 15:13:20 -05:00
Bruno Quaresma 8a94b72c7d
feat: Allow hide resources (#3977) 2022-09-09 16:38:00 -03:00
Ammar Bandukwala a09ffd6c0d
feat: show better error on invalid template upload (#3847)
* feat: show better error on invalid template upload

* Fix tests
2022-09-02 22:48:40 +00:00
Ammar Bandukwala 30f8fd9b95
Daily Active User Metrics (#3735)
* agent: add StatsReporter

* Stabilize protoc
2022-09-01 14:58:23 -05:00
Kyle Carberry b0fe9bcdd1
chore: Upgrade to Go 1.19 (#3617)
This is required as part of #3505.
2022-08-21 22:32:53 +00:00
Kyle Carberry 9c12b4ed8e
chore: Add nix shell for simple development setup (#3399)
* chore: Add nix shell for simple development setup

This enables contributors using Nix to set up their environment with ease.

* improve nix style, flake output schema

* fix error message

* Update scripts/build_go_slim.sh

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>

* Update scripts/build_go_slim.sh

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>

* Add UTC default for timezone and remove unnecessary goreleaser dependency

* Skip TZ test if localtime does not exist

Co-authored-by: Charlie Moog <moogcharlie@gmail.com>
Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2022-08-08 15:49:12 +00:00
David Wahler 8a2811210a
feat: Add backend API support for resource metadata (#3242)
* Initial support for metadata in provisioner API and Terraform provisioner

* add support for nullable metadata fields

* handle metadata fields in provisionerd and API
2022-08-01 16:53:05 -05:00
Denbeigh Stevens 43b8cf04f0
fix: remove pipefail from standard shell options (#3269)
This isn't well-supported by every POSIX shell anyways.
2022-07-28 18:50:04 +00:00
Mathias Fredriksson d27076cac7
fix: Improve `coder server` shutdown procedure (#3246)
* fix: Improve `coder server` shutdown procedure

This commit improves the `coder server` shutdown procedure so that all
triggers for shutdown do so in a graceful way without skipping any
steps.

We also improve cancellation and shutdown of services by ensuring
resources are cleaned up at the end.

Notable changes:
- We wrap `cmd.Context()` to allow us to control cancellation better
- We attempt graceful shutdown of the http server (`server.Shutdown`)
  because it's less abrupt (compared to `shutdownConns`)
- All exit paths share the same shutdown procedure (except for early
  exit)
- `provisionerd`s are now shutdown concurrently instead of one at a
  time, the also now get a new context for shutdown because
  `cmd.Context()` may be cancelled
- Resources created by `newProvisionerDaemon` are cleaned up
- Lifecycle `Executor` exits its goroutine on context cancellation

Fixes #3245
2022-07-27 18:21:21 +03:00
Mathias Fredriksson 6916d34458
fix: Fix cleanup in test helpers, prefer `defer` in tests (#3113)
* fix: Change uses of t.Cleanup -> defer in test bodies

Mixing t.Cleanup and defer can lead to unexpected order of execution.

* fix: Ensure t.Cleanup is not aborted by require

* chore: Add helper annotations
2022-07-25 19:22:02 +03:00
David Wahler f7ea016494
Pass git configuration variables via terraform (#3034)
* Pass workspace owner email address to provisioner

* Remove owner_email and owner_username fields from agent metadata

* Add Git environment variables to example templates

* Remove "owner_name" field from provisioner metadata, use username instead

* Remove Git configuration from most templates, add documentation

* Proofreading/typo fixes from @mafredri

* Update example templates to latest version of terraform-provider-coder
2022-07-19 13:24:06 -05:00
Kyle Carberry 8e17254785
fix: Add test for wrapping init script with single quotes (#2979)
This ensures our initialization script works with single  uotes.
2022-07-13 17:43:48 -05:00
Cian Johnston 0f5f30b6f6
fix: make agent scripts easier to troubleshoot (#2922)
- Adds distinct exit statuses to the bootstrap scripts
- Makes the bootstrap scripts loop forever trying to download the coder agent
- Surfaces and logs the status codes returned by the download tool
2022-07-13 10:17:40 +01:00
Kyle Carberry 6f34cbff1e
fix: Use double quotes for `trap` signal (#2956)
Frequently callers will wrap our shell script in `sh -c ''`.
Having single quotes on our `trap` led to a syntax error when
doing this.
2022-07-13 01:09:59 +00:00
Kyle Carberry dde51f1caa
fix: Force trap to always succeed due to incompatibility (#2953)
There are some instances of Linux that don't support trap. We should
ignore the failure in those cases.
2022-07-12 23:31:25 +00:00
Kyle Carberry 1973786335
fix: Add `trap` to agent startup script to sleep on failure (#2873)
* fix: Add `trap` to agent startup script to sleep on failure

The Docker Terraform provider removes containers immediately on exit, making
it difficult to debug a failed container start with Coder. This will sleep on
exit and output a friendly log, which should assist with debugging failures.

* Update provisionersdk/agent.go

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>

* Update provisionersdk/agent.go

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2022-07-11 14:10:06 +00:00
Cian Johnston 8a59178e7e
provisionersdk: extract and embed agent bootstrap scripts (#2886)
This commit extracts the existing hard-coded agent scripts to their own files and embeds them using go:embed.
We can more easily use e.g. shellcheck to validate our agent scripts.
2022-07-11 12:43:14 +01:00
Mathias Fredriksson 6c1208e3db
feat: Clean up coder agent path in `ps` listing (#2453)
This commit changes the `coder agent` path in `ps` listing from
`/tmp/tmp.coderwWs87Y/coder agent` to `./coder agent`.

The path is also updated to `/tmp/coder.wWs87Y`.

There were two options considered for turning `./coder agent` into
`coder agent`:

1. Run `exec -a coder /path/to/coder agent`
2. Run `PATH=/path/to:$PATH exec coder agent`

Option 1 is not supported by `dash`, and thus discarded.

Option 2 duplicates functionality in `coder agent` which _appends_ the
path, here we would want to _prepend_ it to ensure we're starting the
downloaded `coder` binary in case there is a binary with a conflicting
name on the system.

Fixes #2407
2022-06-17 19:37:47 +03:00
Colin Adler 961ddad925
fix: use `command -v ` instead of `which` in agent bootstrap (#2307)
Certain distros don't ship with `which` (arch) and `command -v` is
built-in to the shell, so this is much more compatible.
2022-06-13 21:20:15 -05:00
Colin Adler dad42fe712
feat: gzip static http server assets (#2272) 2022-06-13 13:14:22 -05:00
Kyle Carberry 013f028e55
feat: Add workspace application support (#1773)
* feat: Add app support

This adds apps as a property to a workspace agent.

The resource is added to the Terraform provider here:
https://github.com/coder/terraform-provider-coder/pull/17

Apps will be opened in the dashboard or via the CLI
with `coder open <name>`. If `command` is specified, a
terminal will appear locally and in the web. If `target`
is specified, the browser will open to an exposed instance
of that target.

* Compare fields in apps test

* Update Terraform provider to use relative path

* Add some basic structure for routing

* chore: Remove interface from coderd and lift API surface

Abstracting coderd into an interface added misdirection because
the interface was never intended to be fulfilled outside of a single
implementation.

This lifts the abstraction, and attaches all handlers to a root struct
named `*coderd.API`.

* Add basic proxy logic

* Add proxying based on path

* Add app proxying for wildcards

* Add wsconncache

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* Add workspace route proxying endpoint

- Makes the workspace conn cache concurrency-safe
- Reduces unnecessary open checks in `peer.Channel`
- Fixes the use of a temporary context when dialing a workspace agent

* Add embed errors

* chore: Refactor site to improve testing

It was difficult to develop this package due to the
embed build tag being mandatory on the tests. The logic
to test doesn't require any embedded files.

* Add test for error handler

* Remove unused access url

* Add RBAC tests

* Fix dial agent syntax

* Fix linting errors

* Fix gen

* Fix icon required

* Adjust migration number

* Fix proxy error status code

* Fix empty db lookup
2022-06-04 15:13:37 -05:00
Kyle Carberry 1fa50a9da1
fix: Race when writing to a closed pipe (#1916) 2022-06-01 07:59:03 -05:00
Cian Johnston c2f74f3cc2
chore: avoid concurrent usage of t.FailNow (#1683)
* chore: golangci: add linter rule to report usage of t.FailNow inside goroutines
* chore: avoid t.FailNow in goroutines to appease the race detector
2022-05-24 08:58:39 +01:00
Mathias Fredriksson ad9bdb7bd1
fix: More robust provisionersdk agent init scripts (#1551)
Related #1544

Co-authored-by: Dean Sheather <dean@deansheather.com>
2022-05-19 13:02:42 +00:00
Kyle Carberry a77da8445e
fix: Resolve symlinks being written with size 0 in tar (#1488)
Solution found here:
https://stackoverflow.com/questions/38454850/getting-write-too-long-error-when-trying-to-create-tar-gz-file-from-file-and-d

Symlink's were being written with a size of 0, which surfaced an error
for write too long.
2022-05-16 20:26:23 +00:00
Ben Potter f5817248de
feat: arm(v7/64) builds for releases and agent scripts (#1337)
* feat: build armv7 linux releases

* upload ARM binaries to bin

* Only build arm 7 for Linux

* add ARM agent scripts

* fix: specify armv7 to match tf provider

* append arm version to slim builds

* use descript armv7 binary

* Add script mappings for each architecture

Co-authored-by: kylecarbs <kyle@carberry.com>
2022-05-11 09:44:43 -05:00
Kyle Carberry 8701e0084c
feat: Update Terraform provider to support "dir" in "coder_agent" (#1219)
This allows users to specify a starting directory for shell sessions.
2022-05-02 10:27:34 -05:00
Kyle Carberry b948f2dab5
fix: Use environment variables for agent authentication (#1238)
* fix: Update GIT_COMMITTER_NAME to use username

This was a mistake when adding the committer fields 🤦.

* fix: Use environment variables for agent authentication

Using files led to situations where running "coder server --dev" would
break `gitssh`. This is applicable in a production environment too. Users
should be able to log into another Coder deployment from their workspace.

Users can still set "CODER_URL" if they'd like with agent env vars!
2022-04-30 16:40:30 +00:00
Kyle Carberry db7ed4d019
fix: Add resiliency to daemon connections (#1116)
Connections could fail when massive payloads were transmitted.
This fixes an upstream bug in dRPC where the connection would
end with a context canceled if a message was too large.

This adds retransmission of completion and failures too. If
Coder somehow loses connection with a provisioner daemon,
upon the next connection the state will be properly reported.
2022-04-24 20:33:19 -05:00
Steven Masley da3681246e
chore: Bump protoc to 3.20.0 (#1104)
* chore: Bump protoc to 3.20.0
* Make gen with 3.20.0 protoc
2022-04-23 01:53:22 +00:00
Kyle Carberry 65d77383d0
fix: Allow nested Terraform resources (#1093)
This fixes the dependency tree by adding recursion. It
now finds indirect connections and associates it with
an agent.

An example is attached which surfaced this issue.
2022-04-20 12:28:48 -05:00
Kyle Carberry e35a4fdcf0
fix: Disable Windows Defender on agent binary (#1095)
Windows (reasonably) detected our CLI as a virus due to the name
being "sshd" for VS Code support. See:
https://github.com/microsoft/vscode-remote-release/issues/5699

This disables monitoring for our binary prior to run, which
fixes our Windows example.
2022-04-20 15:01:35 +00:00
Kyle Carberry 104a3c6b9c
fix: Prefix tmp directory for agent download with "coder" (#1038)
This makes it easier to find the temporary dir for the coder binary.
2022-04-15 15:30:37 -05:00
Garrett Delfosse 027d89dd9b
chore: Add alias `coder agent` (#986) 2022-04-13 20:55:55 +00:00
Kyle Carberry e8b310166f
fix: Remove resource addresses (#982)
These were added under the impression that there was significant
user-experience impact if multiple resources share the same name.

This hasn't proven to be true yet, so figured we'd take this out
until it becomes necessary.
2022-04-12 14:38:02 -05:00
Kyle Carberry db9d5b7e8c
fix: Rename `coder` to `sshd` on Windows for VS Code Remote support (#974)
On Windows, VS Code Remote requires a parent process of the
executing shell to be named sshd, otherwise it fails. See:
https://github.com/microsoft/vscode-remote-release/issues/5699
2022-04-11 21:14:30 -05:00
Kyle Carberry fb9dc4f346
feat: Improve resource preview and first-time experience (#946)
* Improve CLI documentation

* feat: Allow workspace resources to attach multiple agents

This enables a "kubernetes_pod" to attach multiple agents that
could be for multiple services. Each agent is required to have
a unique name, so SSH syntax is:

`coder ssh <workspace>.<agent>`

A resource can have zero agents too, they aren't required.

* Add tree view

* Improve table UI

* feat: Allow workspace resources to attach multiple agents

This enables a "kubernetes_pod" to attach multiple agents that
could be for multiple services. Each agent is required to have
a unique name, so SSH syntax is:

`coder ssh <workspace>.<agent>`

A resource can have zero agents too, they aren't required.

* Rename `tunnel` to `skip-tunnel`

This command was `true` by default, which causes
a confusing user experience.

* Add disclaimer about editing templates

* Add help to template create

* Improve workspace create flow

* Add end-to-end test for config-ssh

* Improve testing of config-ssh

* Fix workspace list

* Fix config ssh tests

* Update cli/configssh.go

Co-authored-by: Cian Johnston <public@cianjohnston.ie>

* Fix requested changes

* Remove socat requirement

* Fix resources not reading in TTY

Co-authored-by: Cian Johnston <public@cianjohnston.ie>
2022-04-11 18:54:30 -05:00
Kyle Carberry 19b4323512
feat: Allow workspace resources to attach multiple agents (#942)
This enables a "kubernetes_pod" to attach multiple agents that
could be for multiple services. Each agent is required to have
a unique name, so SSH syntax is:

`coder ssh <workspace>.<agent>`

A resource can have zero agents too, they aren't required.
2022-04-11 16:06:15 -05:00
Colin Adler 3abb87ddb6
chore: remove usage of ioutil (#642)
It was deprecated as of 1.17.
2022-03-29 14:59:32 -05:00
Kyle Carberry a06821c103
feat: Update Coder Terraform Provider to v0.2.1 (#563)
This update exposes the workspace name and owner, and changes
authentication methods to be explicit. Implicit authentication
added unnecessary complexity and introduced inconsistency.
2022-03-25 16:34:45 +00:00
Kyle Carberry c451f4e685
feat: Add templates to create working release (#422)
* Add templates

* Move API structs to codersdk

* Back to green tests!

* It all works, but now with tea! 🧋

* It works!

* Add cancellation to provisionerd

* Tests pass!

* Add deletion of workspaces and projects

* Fix agent lock

* Add clog

* Fix linting errors

* Remove unused CLI tests

* Rename daemon to start

* Fix leaking command

* Fix promptui test

* Update agent connection frequency

* Skip login tests on Windows

* Increase tunnel connect timeout

* Fix templater

* Lower test requirements

* Fix embed

* Disable promptui tests for Windows

* Fix write newline

* Fix PTY write newline

* Fix CloseReader

* Fix compilation on Windows

* Fix linting error

* Remove bubbletea

* Cleanup readwriter

* Use embedded templates instead of serving over API

* Move templates to examples

* Improve workspace create flow

* Fix Windows build

* Fix tests

* Fix linting errors

* Fix untar with extracting max size

* Fix newline char
2022-03-22 13:17:50 -06:00
Colin Adler 706e60bb3b
chore: pluralize table names (#463) 2022-03-17 18:10:46 -05:00
Kyle Carberry 18c929c8ab
feat: Use open-source Terraform Provider (#403)
This removes our internal Terraform Provider, and opens
it to the world!
2022-03-07 17:39:00 -06:00
Kyle Carberry bf0ae8f573
feat: Refactor API routes to use UUIDs instead of friendly names (#401)
* Add client for agent

* Cleanup code

* Fix linting error

* Rename routes to be simpler

* Rename workspace history to workspace build

* Refactor HTTP middlewares to use UUIDs

* Cleanup routes

* Compiles!

* Fix files and organizations

* Fix querying

* Fix agent lock

* Cleanup database abstraction

* Add parameters

* Fix linting errors

* Fix log race

* Lock on close wait

* Fix log cleanup

* Fix e2e tests

* Fix upstream version of opencensus-go

* Update coderdtest.go

* Fix coverpkg

* Fix codecov ignore
2022-03-07 11:40:54 -06:00
Kyle Carberry b6017a7cc5
feat: Add destroy to workspace provision job (#373)
* ci: Update DataDog GitHub branch to fallback to GITHUB_REF

This was detecting branches, but not our "main" branch before.
Hopefully this fixes it!

* Add basic Terraform Provider

* Rename post files to upload

* Add tests for resources

* Skip instance identity test

* Add tests for ensuring agent get's passed through properly

* Fix linting errors

* Add echo path

* Fix agent authentication

* fix: Convert all jobs to use a common resource and agent type

This enables a consistent API for project import and provisioned resources.

* Add "coder_workspace" data source

* feat: Remove magical parameters from being injected

This is a much cleaner abstraction. Explicitly declaring the user
parameters for each provisioner makes for significantly simpler
testing.

* feat: Add graceful exits to provisionerd

Terraform (or other provisioners) may need to cleanup state, or
cancel actions before exit. This adds the ability to gracefully
exit provisionerd.

* Fix cancel error check

* feat: Add destroy to workspace provision job

This enables the full flow of create/update/delete.
2022-02-28 19:20:36 +00:00