Commit Graph

89 Commits

Author SHA1 Message Date
Danny Kopping 06e042acfa
chore: apply shellcheck recommendation which was causing "make lint" to fail locally (#12972) 2024-04-16 13:06:08 +02:00
Colin Adler 4d5a7b2d56
chore(codersdk): move all tailscale imports out of `codersdk` (#12735)
Currently, importing `codersdk` just to interact with the API requires
importing tailscale, which causes builds to fail unless manually using
our fork.
2024-03-26 12:44:31 -05:00
Marcin Tojek a7d9d87ba2
docs: use scale testing utility (#12643) 2024-03-22 11:33:31 +00:00
Cian Johnston 1cc51b009a
chore(examples): remove deprecated startup_script_timeout and shutdown_script_timeout (#12104)
Removes deprecated startup_script_timeout and shutdown_script_timeout from our example templates.

Co-authored-by: Muhammad Atif Ali <atif@coder.com>
2024-02-12 14:29:41 +00:00
Mathias Fredriksson 83eea2d323
feat(scaletest/templates): add support for concurrent scenarios (#11753) 2024-01-30 14:54:54 +02:00
Mathias Fredriksson 3e89ba23e5
test(scaletest): fix websocket error during close (#11879)
Fixes #11735
2024-01-29 13:42:30 +02:00
Mathias Fredriksson 52c08a98bb
test(scaletest): fix worksapcebuild retry (#11836) 2024-01-26 15:49:19 +02:00
Mathias Fredriksson 02124758fb
feat(cli/exp): extend scaletest create-workspaces with --retry option (#11825)
Part of #11801
2024-01-26 11:29:48 +00:00
Mathias Fredriksson 73e6bbff7e
feat(cli/exp): add app testing to scaletest workspace-traffic (#11633) 2024-01-19 15:20:19 +02:00
Garrett Delfosse b21da38bea
chore: deprecate template create command in favor of template push (#11390) 2024-01-05 21:04:14 +00:00
Spike Curtis 65290997c1
chore: disable failing metrics check until it can be fixed (#11361) 2024-01-02 05:39:48 +00:00
Mathias Fredriksson e300b036be
feat(scaletest): add greedy agent test to runner (#10559) 2023-12-05 12:37:10 +02:00
Mathias Fredriksson 99151183bc
feat(scaletest): replace bash with dd in ssh/rpty traffic and use pseudorandomness (#10821)
Fixes #10795
Refs #8556
2023-11-30 19:30:12 +02:00
Mathias Fredriksson 8f39ec5cc3
fix(scaletest/templates): fix bugs and improve debugging (#10316) 2023-10-17 15:18:49 +00:00
Mathias Fredriksson 43f26dfec5
feat(scaletest/templates): run all scenarios even on failure (#10290)
We now also end all failed phases and add the `error` tag in Grafana.
2023-10-16 18:18:05 +03:00
Mathias Fredriksson 8ffe0e22b6
feat(scaletest/templates): gather pod logs at the end of a scale test (#10288) 2023-10-16 13:50:04 +00:00
Mathias Fredriksson 3c49290dd7
feat(scaletest/templates): add comment parameter (#10285) 2023-10-16 13:28:23 +00:00
Mathias Fredriksson 493e2bd2ac
feat(scaletest/templates): add repo branch parameter (#10279) 2023-10-16 12:25:32 +00:00
Cian Johnston dd86100f33
fix(scaletest): fix flake in Test_Runner/Cleanup (#10252)
* fix(scaletest/createworkspaces): address flake in Test_Runner/CleanupPendingBuild

* fix(scaletest): pass io.Writer to Cleanup()

* add some extra logs to workspacebuild cleanup

* fixup! fix(scaletest): pass io.Writer to Cleanup()

* remove race

* fmt

* address PR comments
2023-10-16 12:37:12 +01:00
Cian Johnston e5198a25a6
feat(scaletest): annotate scaletest pod when scaletest is in progress (#10235)
This PR modifies the scaletest-runner template to add a pod annotation to the scaletest runner pod.

The annotation key is set to com.coder.scaletest.phase and the annotation value is one of preparing, running, or complete.

This will allow checking if a scaletest is in progress, and preventing any operations that would interrupt a running scaletest.

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2023-10-12 14:36:15 +01:00
Cian Johnston e829cbf2db
fix(scaletest/dashboard): fix early exit due to validate (#10212) 2023-10-11 11:51:06 +00:00
Cian Johnston ed8092c83d
fix(scaletest/createworkspaces): address race condition between agent closer and cleanup (#10210) 2023-10-11 12:10:51 +01:00
Cian Johnston b3471bd23a
fix(scaletest/dashboard): increase viewport size and handle deadlines (#10197)
- Set viewport size to avoid responsive mode
- Added way more debug logging
- Added facility to write a screenshot on error in verbose mode.
- Added a deadline for each iteraction of clicking on and waiting for a thing.
2023-10-11 11:10:08 +01:00
Cian Johnston 2506415def
chore(scaletest/templates/scaletest-runner): fix dashboard command invocation, autoscale provisioners (#10177)
add --retries on kubectl cp
remove --count parameter to scaletest dashboard
scale provisioners up and down

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2023-10-10 15:33:55 +01:00
Cian Johnston 5673aca408
feat(cli): add --parameter flag to exp scaletest command (#10132) 2023-10-09 14:08:24 +01:00
Mathias Fredriksson 5d5a7da67f
fix(scaletest): output error and trace instead of {} for json output (#10075) 2023-10-05 13:31:54 +00:00
Kayla Washburn c194119689
chore: rename `AwaitTemplateVersionJobCompleted` and `AwaitWorkspaceBuildJobCompleted` (#10003) 2023-10-03 11:02:56 -06:00
Cian Johnston 9aac15212b
fix(cli): remove exp scaletest from slim binary (#9934)
- Removes the `exp scaletest` command from the slim binary 
- Updates scaletest-runner template to fetch the full binary from the running Coder instance
2023-10-03 15:13:04 +01:00
Mathias Fredriksson 4b0565c895
feat(scaletest): add service banner status for runner (#9945) 2023-10-02 14:37:38 +03:00
Mathias Fredriksson e6d2ddb54b
feat(scaletest): add apps to scaletest template (#9944) 2023-10-02 14:37:16 +03:00
Cian Johnston 1c48610d56
feat(scaletest/dashboard): integrate chromedp (#9927)
* Adds a set of actions to automatically interact with a Coder instance using chromedp
* Integrates the chromedp actions into the scaletest dashboard command,
* Re-enables the previously disabled unit tests for scaletest/dashboard
* Removes previous dashboard actions based around codersdk
2023-10-02 10:40:17 +01:00
Cian Johnston 72e8f88af3
feat(scaletest/terraform): add cert-manager, otel, and TLS (#9894) 2023-09-27 17:41:07 +01:00
Mathias Fredriksson 68738771b9
feat(scaletest): create automated pprof dumps during scaletest (#9887) 2023-09-27 15:58:43 +03:00
Mathias Fredriksson d8515f02af
feat(scaletest): add grafana annotations and slack reporting (#9852)
Fixes #9575
Fixes #9576
2023-09-27 11:44:11 +00:00
Cian Johnston 399b428149
chore(scaletest): update scaletest terraform to match big.cdr.dev (#9860)
- Removes usage of null_resources as no longer required
- Updates terraform in both infra/ and k8s/ to match
- Updates provisionerd deployment to use helm chart
2023-09-27 09:02:35 +01:00
Cian Johnston 93ef696b57
refactor(agent): add agenttest.New helper function (#9812)
* Adds agenttest.New() helper function
* Makes sure agent gets closed on test cleanup
* Makes sure you don't forget to set session token
* Sets the agent and client logger automatically
2023-09-26 12:05:19 +01:00
Cian Johnston a1f3a6b606
refactor(scaletest/terraform): break up infra creation and k8s resource provisioning (#9824)
Breaks up scaletest/terraform into {infra, k8s}.
2023-09-22 15:55:47 +01:00
Mathias Fredriksson bc97eaa41b
feat(scaletest): add scaletest-runner template (#9662)
Closes #9571
2023-09-15 13:23:37 +00:00
Ammar Bandukwala b63dfe7b75
perf(cli): optimize CPU consumption of help pages (#9607)
This change reduces the CPU consumption of --help by ~50%.

Also, this change removes ANSI escape codes from our golden files. I
don't think those were worth the inability to parallelize golden file tests and
global state fragility.
2023-09-14 19:48:29 -05:00
Cian Johnston 3be783b319
fix(scaletest/workspacetraffic): wait for non-zero metrics before cancelling in TestRun (#9663) 2023-09-13 15:54:56 +01:00
Spike Curtis e7a231e44f
fix: fix test flake introduced by #9264 (#9330)
* Fix test flake introduced by #9264

Signed-off-by: Spike Curtis <spike@coder.com>

* change check to match suffix

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-25 10:48:42 +00:00
Spike Curtis 60d5002eb6
refactor: change template archive extraction to be on provisioner (#9264)
* refactor provisionersdk protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisioners to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor provisionerd to use new protocol

Signed-off-by: Spike Curtis <spike@coder.com>

* refactor tests & proto renames

* Fixes from self-review

Signed-off-by: Spike Curtis <spike@coder.com>

* appease fmt & link

Signed-off-by: Spike Curtis <spike@coder.com>

* code review fixes & e2e fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* More fmt

Signed-off-by: Spike Curtis <spike@coder.com>

* Code review fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* new gen; use uuid for session workdir

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert nix-based gen CI task until dogfood is on nix

Signed-off-by: Spike Curtis <spike@coder.com>

* revert deleting dogfood Docker stuff

Signed-off-by: Spike Curtis <spike@coder.com>

* Revert "revert deleting dogfood Docker stuff"

This reverts commit 9762158167.

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-25 06:10:15 +00:00
Kyle Carberry 22e781eced
chore: add /v2 to import module path (#9072)
* chore: add /v2 to import module path

go mod requires semantic versioning with versions greater than 1.x

This was a mechanical update by running:
```
go install github.com/marwan-at-work/mod/cmd/mod@latest
mod upgrade
```

Migrate generated files to import /v2

* Fix gen
2023-08-18 18:55:43 +00:00
Spike Curtis c66befb548
fix(scaletest): change IP range to non-reserved in GCP (#9146)
Signed-off-by: Spike Curtis <spike@coder.com>
2023-08-17 13:11:02 +04:00
Asher b993cab49a
fix: use screen for reconnecting terminal sessions on Linux if available (#8640)
* Add screen backend for reconnecting ptys

The screen portion is a port from wsep.  There is an interface that lets
you choose between screen and the previous method.  By default it will
choose screen if it is installed but this can be overidden (mostly for
tests).

The tests use a scanner instead of a reader now because the reader will
loop infinitely at the end of a stream.

Replace /bin/bash with bash since bash is not always in /bin.

* Remove connection_id from reconnecting PTY logger

This serves multiple connections so it makes no sense to scope it to a
single connection.

Also lets us use "connection_id" when logging write errors instead of
"other_conn_id".

* Use PATH to test buffered reconnecting pty
2023-08-14 11:19:13 -08:00
Cian Johnston ef9d84c723
fix(scaletest): cleanup: attempt to cancel in-progress jobs (#9080)
This change modifies the cleanup behaviour to make a best-effort attempt to cancel the in-progress scaletest workspace build jobs before deleting them.
2023-08-14 12:43:45 +01:00
Cian Johnston 32829080ac
feat(cli): add dashboard load test command (#8723) 2023-07-27 09:40:13 +01:00
Cian Johnston 68a46198d3
fix(scaletest): deploy external provisionerd (#8618)
* scaletest: stop kubernetes_secret from being constantly recreated
* scaletest: ensure we do not get auto-upgraded
* scaletest: add external provisionerd deployment, the lazy way
2023-07-20 11:38:46 +01:00
Cian Johnston 278527cff4
feat(scaletest): add option to send traffic over SSH (#8521)
- Refactors the metrics logic to avoid needing to pass in a whole prometheus registry
- Adds an --ssh option to the workspace-traffic command to send SSH traffic

Fixes #8242
2023-07-18 12:17:11 +01:00
Cian Johnston cdf9b9045f
fix(scaletest/terraform): fix prometheus namespace deps, disable auto-upgrade (#8490)
* hotfix(scaletest/terraform): fix prometheus namespace deps, disable auto-upgrade

* fixup! hotfix(scaletest/terraform): fix prometheus namespace deps, disable auto-upgrade
2023-07-13 10:54:57 +01:00