Commit Graph

34 Commits

Author SHA1 Message Date
Kyle Carberry b0fe9bcdd1
chore: Upgrade to Go 1.19 (#3617)
This is required as part of #3505.
2022-08-21 22:32:53 +00:00
Mathias Fredriksson 3fdae47b87
fix: Shadow err in TestProvision_Cancel to fix test race (#3579)
Fixes #3574
2022-08-19 11:56:28 +03:00
Mathias Fredriksson f1423450bd
fix: Allow terraform provisions to be gracefully cancelled (#3526)
* fix: Allow terraform provisions to be gracefully cancelled

This change allows terraform commands to be gracefully cancelled on
Unix-like platforms by signaling interrupt on provision cancellation.

One implementation detail to note is that we do not necessarily kill a
running terraform command immediately even if the stream is closed. The
reason for this is to allow for graceful cancellation even in such an
event. Currently the timeout is set to 5 minutes by default.

Related: #2683

The above issue may be partially or fully fixed by this change.

* fix: Remove incorrect minimumTerraformVersion variable

* Allow init to return provision complete response
2022-08-18 17:03:55 +03:00
Mathias Fredriksson fb9fca8bc9
fix: Ensure terraform tests have a cache path and logger (#3161)
* fix: Ensure terraform tests have a cache path and logger

* fix: Protect against concurrent `terraform init`
2022-08-04 20:37:07 +03:00
Spike Curtis 54a585dbf6
Log provisioner outputs from TestProvision_ExtraEnv (#2427)
Signed-off-by: Spike Curtis <spike@coder.com>
2022-06-17 14:47:05 -07:00
Spike Curtis 552dad6919
Remove tfexec, allow TF_ environment vars and log them (#2264)
* Remove tfexec, allow TF_ environment vars and log them

Signed-off-by: Spike Curtis <spike@coder.com>

* fixup: commented code, long lines

Signed-off-by: Spike Curtis <spike@coder.com>

* rename executor methods to remove get

Signed-off-by: Spike Curtis <spike@coder.com>

* don't log terraform environment variables we don't know are safe

Signed-off-by: Spike Curtis <spike@coder.com>

* Disable linting of fake secret

Signed-off-by: Spike Curtis <spike@coder.com>

* drop parse support and move logger into terraform package

Signed-off-by: Spike Curtis <spike@coder.com>

* disable testpackage linter on internal package test

Signed-off-by: Spike Curtis <spike@coder.com>
2022-06-16 17:50:39 +00:00
Dean Sheather 45eb1b4980
feat: improve terraform template parsing errors (#2331) 2022-06-16 04:12:17 +10:00
Kyle Carberry 14701498c9
fix: Improve Terraform agent<->resource association testing (#2187) 2022-06-08 17:40:34 -05:00
Kyle Carberry 013f028e55
feat: Add workspace application support (#1773)
* feat: Add app support

This adds apps as a property to a workspace agent.

The resource is added to the Terraform provider here:
https://github.com/coder/terraform-provider-coder/pull/17

Apps will be opened in the dashboard or via the CLI
with `coder open <name>`. If `command` is specified, a
terminal will appear locally and in the web. If `target`
is specified, the browser will open to an exposed instance
of that target.

* Compare fields in apps test

* Update Terraform provider to use relative path

* Add some basic structure for routing

* chore: Remove interface from coderd and lift API surface

Abstracting coderd into an interface added misdirection because
the interface was never intended to be fulfilled outside of a single
implementation.

This lifts the abstraction, and attaches all handlers to a root struct
named `*coderd.API`.

* Add basic proxy logic

* Add proxying based on path

* Add app proxying for wildcards

* Add wsconncache

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* fix: Race when writing to a closed pipe

This is such an intermittent race it's difficult to track,
but regardless this is an improvement to the code.

* Add workspace route proxying endpoint

- Makes the workspace conn cache concurrency-safe
- Reduces unnecessary open checks in `peer.Channel`
- Fixes the use of a temporary context when dialing a workspace agent

* Add embed errors

* chore: Refactor site to improve testing

It was difficult to develop this package due to the
embed build tag being mandatory on the tests. The logic
to test doesn't require any embedded files.

* Add test for error handler

* Remove unused access url

* Add RBAC tests

* Fix dial agent syntax

* Fix linting errors

* Fix gen

* Fix icon required

* Adjust migration number

* Fix proxy error status code

* Fix empty db lookup
2022-06-04 15:13:37 -05:00
Kyle Carberry 74329f479f
fix: Use Terraform address to index resource + agent association (#1727)
Closes #1705.

There was an issue in the implementation brought by #1577 by not trimming
the array value when resources use counts. This should fix it, and adds
a test to be sure!
2022-05-24 20:00:34 -05:00
Cian Johnston c2f74f3cc2
chore: avoid concurrent usage of t.FailNow (#1683)
* chore: golangci: add linter rule to report usage of t.FailNow inside goroutines
* chore: avoid t.FailNow in goroutines to appease the race detector
2022-05-24 08:58:39 +01:00
Dean Sheather adb7d20c16
feat: skip terraform destroy if there is no state when deleting (#1594) 2022-05-20 14:07:23 +10:00
Kyle Carberry 43c6bff5ae
fix: Use "terraform state pull" instead of "terraform show" (#1262)
Although the terraform-exec docs don't indicate this, the result of
"terraform show" isn't actually the state... it's a trimmed version
of the state that excludes resource identifiers, essentially removing
all state that did exist.

Tests will be written to ensure Terraform state reconciliation can occur.
This will happen in another PR, as dogfood is currently broken because of this.
2022-05-02 20:02:38 +00:00
Kyle Carberry d139a16446
fix: Allow remote state to be used with Terraform (#1242)
The Terraform Provisioner depended on the statefile content
being at a specific path, which disallowed the use of external
state providers. This fixes it!
2022-05-02 15:41:27 +00:00
Kyle Carberry 8701e0084c
feat: Update Terraform provider to support "dir" in "coder_agent" (#1219)
This allows users to specify a starting directory for shell sessions.
2022-05-02 10:27:34 -05:00
Kyle Carberry b948f2dab5
fix: Use environment variables for agent authentication (#1238)
* fix: Update GIT_COMMITTER_NAME to use username

This was a mistake when adding the committer fields 🤦.

* fix: Use environment variables for agent authentication

Using files led to situations where running "coder server --dev" would
break `gitssh`. This is applicable in a production environment too. Users
should be able to log into another Coder deployment from their workspace.

Users can still set "CODER_URL" if they'd like with agent env vars!
2022-04-30 16:40:30 +00:00
Kyle Carberry 65d77383d0
fix: Allow nested Terraform resources (#1093)
This fixes the dependency tree by adding recursion. It
now finds indirect connections and associates it with
an agent.

An example is attached which surfaced this issue.
2022-04-20 12:28:48 -05:00
Kyle Carberry 19b4323512
feat: Allow workspace resources to attach multiple agents (#942)
This enables a "kubernetes_pod" to attach multiple agents that
could be for multiple services. Each agent is required to have
a unique name, so SSH syntax is:

`coder ssh <workspace>.<agent>`

A resource can have zero agents too, they aren't required.
2022-04-11 16:06:15 -05:00
Kyle Carberry 21fdb80825
fix: Parsing dynamic values for agent results in error (#564)
The logic required a constant value before, which disallowed dynamic
value injection into the agent. This isn't an accurate limitation,
so inverting the logic resolves it.
2022-03-25 17:21:39 +00:00
Kyle Carberry a06821c103
feat: Update Coder Terraform Provider to v0.2.1 (#563)
This update exposes the workspace name and owner, and changes
authentication methods to be explicit. Implicit authentication
added unnecessary complexity and introduced inconsistency.
2022-03-25 16:34:45 +00:00
Kyle Carberry c451f4e685
feat: Add templates to create working release (#422)
* Add templates

* Move API structs to codersdk

* Back to green tests!

* It all works, but now with tea! 🧋

* It works!

* Add cancellation to provisionerd

* Tests pass!

* Add deletion of workspaces and projects

* Fix agent lock

* Add clog

* Fix linting errors

* Remove unused CLI tests

* Rename daemon to start

* Fix leaking command

* Fix promptui test

* Update agent connection frequency

* Skip login tests on Windows

* Increase tunnel connect timeout

* Fix templater

* Lower test requirements

* Fix embed

* Disable promptui tests for Windows

* Fix write newline

* Fix PTY write newline

* Fix CloseReader

* Fix compilation on Windows

* Fix linting error

* Remove bubbletea

* Cleanup readwriter

* Use embedded templates instead of serving over API

* Move templates to examples

* Improve workspace create flow

* Fix Windows build

* Fix tests

* Fix linting errors

* Fix untar with extracting max size

* Fix newline char
2022-03-22 13:17:50 -06:00
Kyle Carberry 18c929c8ab
feat: Use open-source Terraform Provider (#403)
This removes our internal Terraform Provider, and opens
it to the world!
2022-03-07 17:39:00 -06:00
Kyle Carberry 8acd58a095
fix: Use plan to detect resource agent association (#381)
* ci: Update DataDog GitHub branch to fallback to GITHUB_REF

This was detecting branches, but not our "main" branch before.
Hopefully this fixes it!

* Add basic Terraform Provider

* Rename post files to upload

* Add tests for resources

* Skip instance identity test

* Add tests for ensuring agent get's passed through properly

* Fix linting errors

* Add echo path

* Fix agent authentication

* fix: Convert all jobs to use a common resource and agent type

This enables a consistent API for project import and provisioned resources.

* Add "coder_workspace" data source

* feat: Remove magical parameters from being injected

This is a much cleaner abstraction. Explicitly declaring the user
parameters for each provisioner makes for significantly simpler
testing.

* feat: Add graceful exits to provisionerd

Terraform (or other provisioners) may need to cleanup state, or
cancel actions before exit. This adds the ability to gracefully
exit provisionerd.

* Fix cancel error check

* feat: Add destroy to workspace provision job

This enables the full flow of create/update/delete.

* fix: Use plan to detect resource agent association

Before this used the configuration object which detected all resources
regardless of count.
2022-02-28 19:32:13 +00:00
Kyle Carberry e5c95552cd
feat: Remove magical parameters from being injected (#371)
* ci: Update DataDog GitHub branch to fallback to GITHUB_REF

This was detecting branches, but not our "main" branch before.
Hopefully this fixes it!

* Add basic Terraform Provider

* Rename post files to upload

* Add tests for resources

* Skip instance identity test

* Add tests for ensuring agent get's passed through properly

* Fix linting errors

* Add echo path

* Fix agent authentication

* fix: Convert all jobs to use a common resource and agent type

This enables a consistent API for project import and provisioned resources.

* Add "coder_workspace" data source

* feat: Remove magical parameters from being injected

This is a much cleaner abstraction. Explicitly declaring the user
parameters for each provisioner makes for significantly simpler
testing.
2022-02-28 18:26:01 +00:00
Kyle Carberry 35ae532f7c
feat: Add Terraform Provider for agent resources (#368)
* ci: Update DataDog GitHub branch to fallback to GITHUB_REF

This was detecting branches, but not our "main" branch before.
Hopefully this fixes it!

* Add basic Terraform Provider

* Rename post files to upload

* Add tests for resources

* Skip instance identity test

* Add tests for ensuring agent get's passed through properly

* Fix linting errors

* Add echo path

* Fix agent authentication

* Update codersdk/files.go

Co-authored-by: Bryan <bryan@coder.com>

Co-authored-by: Bryan <bryan@coder.com>
2022-02-28 17:16:44 +00:00
Kyle Carberry 795bba2af4
feat: Add dry run for provisioners (#178)
* refactor: Rename ProjectParameter to ProjectVersionParameter

This was confusing with ParameterValue before. It still is a bit,
but this should help distinguish scope.

* Add project version resources table

* Allow project parameters to optionally have user and workspace

* Add dry run for provisioners

* Add resource detection on project import
2022-02-07 19:35:18 -06:00
Kyle Carberry 75468fad48
chore: Add testpackage linter (#156)
Enforces a consistent test package layout.
This makes it difficult to test internal
functionality, which I believe promotes
healthy decomposition, and minimal package
exports.
2022-02-04 12:06:06 -06:00
Kyle Carberry 3ba8242764
feat: Add provisionerd service (#127)
This brings an async service that parses and
provisions to life! It's separated from coderd
intentionally to allow for simpler testing.

Integration with coderd will come in another PR!
2022-02-01 12:15:54 -06:00
Kyle Carberry ac617e1fa8
feat: Update generated schema for provisionerd (#121) 2022-01-31 23:36:15 -06:00
Kyle Carberry 9db5fb0952
refactor: Improve handshake resiliency of peer (#95)
* fix: Synchronize peer logging with a channel

We were depending on the close mutex to properly
report connection state. This ensures the RTC
connection is properly closed before returning.

* Disable pion logging

* Remove buffer

* Try ICE servers

* Remove flushed

* Add diagram explaining handshake

* Fix candidate accept ordering

* Add debug logging to peerbroker

* Fix send ordering

* Lock adding ICE candidate

* Add test for negotiating out of order

* Reduce connection to a single negotiation channel

* Improve test times by pre-installing Terraform

* Lock remote session description being applied

* Organize conn

* Revert to multi-channel setup

* Properly close ICE gatherer

* Improve comments

* Try removing buffered candidates

* Buffer local and remote messages

* Log dTLS transport state

* Add pion logging
2022-01-30 20:11:18 -06:00
Kyle Carberry b3c5bb3576
feat: Compute project build parameters (#82)
* feat: Add parameter and jobs database schema

This modifies a prior migration which is typically forbidden,
but because we're pre-production deployment I felt grouping
would be helpful to future contributors.

This adds database functions that are required for the provisioner
daemon and job queue logic.

* feat: Compute project build parameters

Adds a projectparameter package to compute build-time project
values for a provided scope.

This package will be used to return which variables are being
used for a build, and can visually indicate the hierarchy to
a user.

* Fix terraform provisioner

* Improve naming, abstract inject to consume scope

* Run CI on all branches
2022-01-29 17:45:42 -06:00
Kyle Carberry 2654a93132
chore: Fix golangci-lint configuration and patch errors (#34)
* chore: Fix golangci-lint configuration and patch errors

Due to misconfiguration of a linting rules directory, our linter has not been
working properly. This change fixes the configuration issue, and all remaining
linting errors.

* Fix race in peer logging

* Fix race and return

* Lock on bufferred amount low

* Fix mutex lock
2022-01-20 10:00:13 -06:00
Kyle Carberry 6a919aea79
feat: Add authentication and personal user endpoint (#29)
* feat: Add authentication and personal user endpoint

This contribution adds a lot of scaffolding for the database fake
and testability of coderd.

A new endpoint "/user" is added to return the currently authenticated
user to the requester.

* Use TestMain to catch leak instead

* Add userpassword package

* Add WIP

* Add user auth

* Fix test

* Add comments

* Fix login response

* Fix order

* Fix generated code

* Update httpapi/httpapi.go

Co-authored-by: Bryan <bryan@coder.com>

Co-authored-by: Bryan <bryan@coder.com>
2022-01-20 13:46:51 +00:00
Kyle Carberry 7c260f88d1
feat: Create provisioner abstraction (#12)
* feat: Create provisioner abstraction

Creates a provisioner abstraction that takes prior art from the Terraform plugin system. It's safe to assume this code will change a lot when it becomes integrated with provisionerd.

Closes #10.

* Ignore generated files in diff view

* Check for unstaged file changes

* Install protoc-gen-go

* Use proper drpc plugin version

* Fix serve closed pipe

* Install sqlc with curl for speed

* Fix install command

* Format CI action

* Add linguist-generated and closed pipe test

* Cleanup code from comments

* Add dRPC comment

* Add Terraform installer for cross-platform

* Build provisioner tests on Linux only
2022-01-08 11:24:02 -06:00