Commit Graph

111 Commits

Author SHA1 Message Date
Ammar Bandukwala 465fe8658d
chore: skip timing-sensistive AgentMetadata test in the standard suite (#7237)
* chore: skip timing-sensistive AgentMetadata test in the standard suite

* Add test-timing target

* fix windows?

* Works on my Windows desktop?

* Use tag system

* fixup! Use tag system
2023-05-02 10:41:41 +00:00
Marcin Tojek bb0a38b161
feat: Implement aggregator for agent metrics (#7259) 2023-04-27 12:34:00 +02:00
Spike Curtis b6666cf1cf
chore: tailnet debug logging (#7260)
* Enable discovery (disco) debug

Signed-off-by: Spike Curtis <spike@coder.com>

* Better debug on reconnectingPTY

Signed-off-by: Spike Curtis <spike@coder.com>

* Agent logging in appstest

Signed-off-by: Spike Curtis <spike@coder.com>

* More reconnectingPTY logging

Signed-off-by: Spike Curtis <spike@coder.com>

* Add logging to coordinator

Signed-off-by: Spike Curtis <spike@coder.com>

* Update agent/agent.go

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>

* Update agent/agent.go

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>

* Update agent/agent.go

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>

* Update agent/agent.go

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>

* Clarify logs; remove unrelated changes

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2023-04-27 13:59:01 +04:00
Colin Adler 3eb7f06bf1
feat(agent): add http debug routes for magicsock (#7287) 2023-04-26 13:01:49 -05:00
Spike Curtis 29cbc5404a Reconnecting PTY waits for command output or EOF (#7279)
Signed-off-by: Spike Curtis <spike@coder.com>
2023-04-26 09:02:06 +04:00
Spike Curtis daee91c6dc
refactor: PTY & SSH (#7100)
* Add ssh tests for longoutput, orphan

Signed-off-by: Spike Curtis <spike@coder.com>

* PTY/SSH tests & improvements

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix some tests

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix linting

Signed-off-by: Spike Curtis <spike@coder.com>

* fmt

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix windows test

Signed-off-by: Spike Curtis <spike@coder.com>

* Windows copy test

Signed-off-by: Spike Curtis <spike@coder.com>

* WIP Windows pty handling

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix truncation tests

Signed-off-by: Spike Curtis <spike@coder.com>

* Appease linter/fmt

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix typo

Signed-off-by: Spike Curtis <spike@coder.com>

* Rework truncation test to not assume OS buffers

Signed-off-by: Spike Curtis <spike@coder.com>

* Disable orphan test on Windows --- uses sh

Signed-off-by: Spike Curtis <spike@coder.com>

* agent_test running SSH in pty use ptytest.Start

Signed-off-by: Spike Curtis <spike@coder.com>

* More detail about closing pseudoconsole on windows

Signed-off-by: Spike Curtis <spike@coder.com>

* Code review fixes

Signed-off-by: Spike Curtis <spike@coder.com>

* Rearrange ptytest method order

Signed-off-by: Spike Curtis <spike@coder.com>

* Protect pty.Resize on windows from races

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix windows bugs

Signed-off-by: Spike Curtis <spike@coder.com>

* PTY doesn't extend PTYCmd

Signed-off-by: Spike Curtis <spike@coder.com>

* Fix windows types

Signed-off-by: Spike Curtis <spike@coder.com>

---------

Signed-off-by: Spike Curtis <spike@coder.com>
2023-04-24 14:53:57 +04:00
Kyle Carberry f39e6a79de
feat: add support for X11 forwarding (#7205)
* feat: add support for X11 forwarding

* Only run X forwarding on Linux

* Fix piping

* Fix comments
2023-04-21 15:52:40 +00:00
Muhammad Atif Ali bb43713d38
fix: VSCode desktop connection (#7120)
Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2023-04-14 17:32:18 +03:00
Mathias Fredriksson 0224426e5b
refactor(agent): Move SSH server into agentssh package (#7004)
Refs: #6177
2023-04-06 19:39:22 +03:00
Kyle Carberry bc18f6c113
fix: add `CODER_AGENT_TAILNET_LISTEN_PORT` for specifying a static tailnet port (#6980)
Fixes #5175.
2023-04-03 16:20:19 +00:00
Ammar Bandukwala 34debbf837
fix(agent): prevent goroutine pile up in reportMetadataLoop (#6957) 2023-04-01 16:34:42 -05:00
Ammar Bandukwala ca4fa81570
feat: add agent metadata (#6614) 2023-03-31 15:26:19 -05:00
Mathias Fredriksson 90d18dd2e5
fix(agent): Close stdin and stdout separately to fix pty output loss (#6862)
Fixes #6656
Closes #6840
2023-03-29 21:58:38 +03:00
Mathias Fredriksson 891bbda995
fix(agent): More protection for lost output of SSH PTY commands (#6833)
Fixes #6656 (part 2)
2023-03-28 09:11:15 +00:00
Mathias Fredriksson 76bdde7f1b
fix(agent): Prevent SSH TTYs from losing command output on exit (#6777) 2023-03-24 18:23:41 +00:00
Kyle Carberry cb7375450b
feat: add startup script logs to the ui (#6558)
* Add startup script logs to the database

* Add coderd endpoints for startup script logs

* Push startup script logs from agent

* Pull startup script logs on frontend

* Rename queries

* Add constraint

* Start creating log sending loop

* Add log sending to the agent

* Add tests for streaming logs

* Shorten notify channel name

* Add FE

* Improve bulk log performance

* Finish UI display

* Fix startup log visibility

* Add warning for overflow

* Fix agent queue logs overflow

* Display staartup logs in a virtual DOM for performance

* Fix agent queue with loads of logs

* Fix authorize test

* Remove faulty test

* Fix startup and shutdown reporting error

* Fix gen

* Fix comments

* Periodically purge old database entries

* Add test fixture for migration

* Add Storybook

* Check if there are logs when displaying features

* Fix startup component overflow gap

* Fix startup log wrapping

---------

Co-authored-by: Asher <ash@coder.com>
2023-03-23 14:09:13 -05:00
Benjamin Sejas 7076dee522
feat(agent): Add SSH max timeout option for coder agent (#6596)
* feat(agent): Add SSH max timeout option for coder agent

* Fix lint and update test golden snapshot
2023-03-15 09:08:50 -05:00
Steven Masley 2abae42cec
feat: Ignore agent pprof port in listening ports (#6515)
* feat: Ignore agent pprof port in listening ports
2023-03-09 10:53:00 -06:00
Kyle Carberry 5304b4e483
feat: add connection statistics for workspace agents (#6469)
* fix: don't make session counts cumulative

This made for some weird tracking... we want the point-in-time
number of counts!

* Add databasefake query for getting agent stats

* Add deployment stats endpoint

* The query... works?!?

* Fix aggregation query

* Select from multiple tables instead

* Fix continuous stats

* Increase period of stat refreshes

* Add workspace counts to deployment stats

* fmt

* Add a slight bit of responsiveness

* Fix template version editor overflow

* Add refresh button

* Fix font family on button

* Fix latest stat being reported

* Revert agent conn stats

* Fix linting error

* Fix tests

* Fix gen

* Fix migrations

* Block on sending stat updates

* Add test fixtures

* Fix response structure

* make gen
2023-03-08 21:05:45 -06:00
Kyle Carberry 29ced72cda
chore: fix stats leaking in tests (#6478)
See https://github.com/coder/coder/actions/runs/4350254306/jobs/7601134509
2023-03-07 04:09:02 +00:00
Mathias Fredriksson 22e3ff96be
feat(agent): Add shutdown lifecycle states and shutdown_script support (#6139)
* feat(api): Add agent shutdown lifecycle states

* feat(agent): Add shutdown_script support

* feat(agent): Add shutdown_script timeout

* feat(site): Support new agent lifecycle states

---

Co-authored-by: Marcin Tojek <marcin@coder.com>
2023-03-06 21:34:00 +02:00
Kyle Carberry 2ff1c6d613
feat: add agent stats for different connection types (#6412)
This allows us to track when our extensions are used, when the
web terminal is used, and average connection latency to the agent.
2023-03-02 08:06:00 -06:00
Kyle Carberry 05e449943d
chore: convert agent stats to use a table (#6374)
* chore: convert workspace agent stats from json to table

* chore: convert agent stats to use a table

Backwards compatibility becomes hard when all agent stats are in a JSON blob.
We also want to query this table for new agents that are failing health checks
so we can display it in the UI.

* Fix migration using default values
2023-02-28 13:33:33 -06:00
Kyle Carberry 3d8b77d6f1
chore: improve clarity of the agent logs (#6345)
I looked through these logs when debugging and there was a bit of spam!
2023-02-27 09:20:24 -06:00
Mathias Fredriksson 677721e4a1
fix(tailnet): Skip nodes without DERP, avoid use of RemoveAllPeers (#6320)
* fix(tailnet): Skip nodes without DERP, avoid use of RemoveAllPeers
2023-02-24 18:16:29 +02:00
Colin Adler 4432cd08d6
chore: update tailscale (#6091) 2023-02-09 21:43:18 -06:00
Kyle Carberry 691495d761
feat: add `expanded_directory` to the agent for extension support (#6087)
This will enable opening the default `dir` of an agent in
the VS Code extension!
2023-02-07 21:35:09 +00:00
Kyle Carberry f6effdb63e
fix: redirect the user to the home directory if dir is not set (#6085)
This was blocking SSH connections from being established if a dir
that wasn't created yet is set.
2023-02-07 20:28:41 +00:00
Kyle Carberry 2c2bbcc019
chore: update tests to support fish (#6023)
* fix: update tests to add fish support

* Track connections for SSH sessions to prevent leaks

* Revert SSH conn handling
2023-02-03 12:25:11 -06:00
Mathias Fredriksson e6f5623627
chore: Rename agent statistics server to http api server (#5961) 2023-02-01 20:05:57 +02:00
Mathias Fredriksson f4d6afb01d
feat(agent): Allow specifying log directory via flag or env (#5915) 2023-01-30 18:39:52 +02:00
Kyle Carberry 0d08065488
fix: use a waitgroup to ensure all connections are cleaned up in agent (#5910)
* fix: use a waitgroup to ensure all connections are cleaned up in agent

There was a race where connections would be created at the same time as close.
The `net.Conn` produced by Tailscale doesn't close then the listener does.

* Remove accidental test
2023-01-29 17:20:30 -06:00
Kyle Carberry 7ad87505c8
chore: move agent functions from `codersdk` into `agentsdk` (#5903)
* chore: rename `AgentConn` to `WorkspaceAgentConn`

The codersdk was becoming bloated with consts for the workspace
agent that made no sense to a reader. `Tailnet*` is an example
of these consts.

* chore: remove `Get` prefix from *Client functions

* chore: remove `BypassRatelimits` option in `codersdk.Client`

It feels wrong to have this as a direct option because it's so infrequently
needed by API callers. It's better to directly modify headers in the two
places that we actually use it.

* Merge `appearance.go` and `buildinfo.go` into `deployment.go`

* Merge `experiments.go` and `features.go` into `deployment.go`

* Fix `make gen` referencing old type names

* Merge `error.go` into `client.go`

`codersdk.Response` lived in `error.go`, which is wrong.

* chore: refactor workspace agent functions into agentsdk

It was odd conflating the codersdk that clients should use
with functions that only the agent should use. This separates
them into two SDKs that are closely coupled, but separate.

* Merge `insights.go` into `deployment.go`

* Merge `organizationmember.go` into `organizations.go`

* Merge `quota.go` into `workspaces.go`

* Rename `sse.go` to `serversentevents.go`

* Rename `codersdk.WorkspaceAppHostResponse` to `codersdk.AppHostResponse`

* Format `.vscode/settings.json`

* Fix outdated naming in `api.ts`

* Fix app host response

* Fix unsupported type

* Fix imported type
2023-01-29 15:47:24 -06:00
Mathias Fredriksson 138887de7e
feat: Add workspace agent lifecycle state reporting (#5785) 2023-01-24 14:24:27 +02:00
Colin Adler d2ae16dd22
fix: routinely ping agent websocket to ensure liveness (#5824) 2023-01-23 20:05:29 +00:00
Kyle Carberry 9f6edab53b
feat: replace vscodeipc with vscodessh (#5645)
The VS Code extension has been refactored to use VS Code
Remote SSH instead of using the private API.

This changes the structure to continue using SSH, but
output network information periodically to a file.
2023-01-10 04:23:17 +00:00
Dean Sheather f1fe2b5c06
feat: add GPG forwarding to coder ssh (#5482) 2023-01-06 07:52:19 +00:00
Mathias Fredriksson 012a9e759e
fix: Avoid deadlock in AgentReportStats Close during agent Close (#5415)
Since AgentReportStats takes a stats function which was doing mutex
locking on agent shutdown, it was possible for there to be a deadlock
depending on how the AgentReportsStats Close function is implemented.

This mostly seems to happen on Windows test runners as it's pretty hard
to hit this edge case. The bug currently only exists in the test
implementation of AgentReportStats, however, this was refactored to be
more robust in case of future changes.
2022-12-14 18:45:46 +02:00
Mathias Fredriksson 4fc4c01cea
fix: Enable reconnectingpty loadtest and fix/improve logging (#5403)
* fix: Enable reconnectingpty loadtest and fix/improve logging

This commit re-enabled reconnectingpty loadtests after a logging
refactor of `(*agent).handleReconnectingPTY`. The reasons the tests were
flaking was that `logger.Error` was being called and `slogtest` failing
the test.

We could have set the option for `slogtest` to disable failing, but that
could hide real issues. The current approach improves reconnectingpty
logging overall and provides more insight into what's happening. It's
expected that reconnectingpty sessions fail after the agent is closed,
so calling `logger.Error` at that point is not wanted.

Ref: #5322
2022-12-13 21:28:07 +02:00
Mathias Fredriksson 88bb901283
fix: Close tailnet if agent is closed during creation (#5375) 2022-12-12 11:26:49 +00:00
Marcin Tojek d3200382f6
fix: agent panics on closed network (#5295)
* fix: agent panics on closed network

* Remove a.network = network

* Fix

* Fix

* Fix
2022-12-05 23:18:23 +01:00
Mathias Fredriksson fa641554e8
fix: Improve agent connection tracking when agent is closed (#5253) 2022-12-02 16:24:40 +02:00
Mathias Fredriksson eff99f78fa
feat: Add support for MOTD file in coder agents (#5147) 2022-11-24 12:22:20 +00:00
Colin Adler ae38bbeab6
chore: refactor agent stats streaming (#5112) 2022-11-18 16:46:53 -06:00
Dean Sheather 69e8c9e7b4
feat: add reconnectingpty loadtest (#5083) 2022-11-17 16:57:15 +00:00
Mathias Fredriksson e72927f3ab
fix: Avoid running shell twice in coder agent (#5061)
The users login shell would be executed as:

	/bin/bash -c '/bin/bash -l'

This simplifies the command for login shells so that the executed
command is:

	/bin/bash -l
2022-11-14 14:01:22 +02:00
Mathias Fredriksson c515085450
fix: Unify context usage for agent cmd and logs (#5059) 2022-11-14 13:48:44 +02:00
Mathias Fredriksson f017548a9c
fix: Return correct exit code for SFTP sessions (#5044)
Fixes #5038
2022-11-13 23:22:50 +02:00
Ammar Bandukwala 73f91e4690
ci: use big runners (#4990)
* chore: Close idle connections on test cleanup

It's possible that this was the source of a leak on Windows...

* ci: use big runners

* fix: Improve tailnet connections by reducing timeouts

This awaits connection ping before running a dial. Before,
we were hitting the TCP retransmission and handshake timeouts,
which could intermittently add 1 or 5 seconds to a connection
being initialized.

* Add logging to Startupscript test

* Add better logging

* Write startup script logs to fs dir

* Fix startup script test

* Fix startup script test

* Reduce test timeout

* Use central tmp dir in agent

* Adjust output

* Skip startup script test on Windows

Co-authored-by: Kyle Carberry <kyle@carberry.com>
2022-11-13 14:23:23 -06:00
Kyle Carberry 8e743d28c8
fix: Use instance identity session token for git subcommands (#4884)
This broke using gitssh with instance identity!
2022-11-04 09:44:36 -07:00