docs: add `coder-logstream-kube` docs and update k8s example template (#8675)

* docs: add kubestream docs and update example template * make fmt * add to template's README * add the seperate page * make fmt * make fmt * add namespace to deployment resource * fmt * add screenshots * link docs in template * Add remaining examples * enable deployments requirement * Update deployment-logs.md * rewording * fix spelling * how? * cleanup * Update docs/platforms/kubernetes/deployment-logs.md Co-authored-by: Ben Potter <ben@coder.com> * suggestions * use online link * refine --------- Co-authored-by: Eric <ericpaulsen@coder.com> Co-authored-by: Ben Potter <ben@coder.com>
2023-07-24 18:51:25 +03:00 · 2023-07-24 18:51:25 +03:00 · 6929792a58
parent b7ced94f0f
commit 6929792a58
8 changed files with 171 additions and 62 deletions
--- a/docs/platforms/kubernetes/coder-logstream-kube-logs-normal.png
+++ b/docs/platforms/kubernetes/coder-logstream-kube-logs-normal.png
--- a/docs/platforms/kubernetes/coder-logstream-kube-logs-pod-crashed.png
+++ b/docs/platforms/kubernetes/coder-logstream-kube-logs-pod-crashed.png
--- a/docs/platforms/kubernetes/coder-logstream-kube-logs-quota-exceeded.png
+++ b/docs/platforms/kubernetes/coder-logstream-kube-logs-quota-exceeded.png
--- a/docs/platforms/kubernetes/coder-logstream-kube-logs-wrong-image.png
+++ b/docs/platforms/kubernetes/coder-logstream-kube-logs-wrong-image.png
--- a/docs/platforms/kubernetes/deployment-logs.md
+++ b/docs/platforms/kubernetes/deployment-logs.md
@ -0,0 +1,59 @@
+# Kubernetes event logs
+
+To stream Kubernetes events into your workspace startup logs, you can use Coder's [`coder-logstream-kube`](https://github.com/coder/coder-logstream-kube) tool. `coder-logstream-kube` provides useful information about the workspace pod or deployment, such as:
+
+- Causes of pod provisioning failures, or why a pod is stuck in a pending state.
+- Visibility into when pods are OOMKilled, or when they are evicted.
+
+## Prerequisites
+
+`coder-logstream-kube` works best with the [`kubernetes_deployment`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/deployment) Terraform resource, which requires the `coder` service account to have permission to create deployments. For example, if you use [Helm](https://coder.com/docs/v2/latest/install/kubernetes#install-coder-with-helm) to install Coder, you should set `coder.serviceAccount.enableDeployments=true` in your `values.yaml`
+
+```diff
+coder:
+serviceAccount:
+    workspacePerms: true
+-   enableDeployments: false
+   enableDeployments: true
+    annotations: {}
+    name: coder
+```
+
+> Note: This is only required for Coder versions < 0.28.0, as this will be the default value for Coder versions >= 0.28.0
+
+## Installation
+
+Install the `coder-kubestream-logs` helm chart on the cluster where the deployment is running.
+
+```shell
+helm repo add coder-logstream-kube https://helm.coder.com/logstream-kube
+helm install coder-logstream-kube coder-logstream-kube/coder-logstream-kube \
+    --namespace coder \
+    --set url=<your-coder-url-including-http-or-https>
+```
+
+## Example logs
+
+Here is an example of the logs you can expect to see in the workspace startup logs:
+
+### Normal pod deployment
+
+![normal pod deployment](./coder-logstream-kube-logs-normal.png)
+
+### Wrong image
+
+![Wrong image name](./coder-logstream-kube-logs-wrong-image.png)
+
+### Kubernetes quota exceeded
+
+![Kubernetes quota exceeded](./coder-logstream-kube-logs-quota-exceeded.png)
+
+### Pod crash loop
+
+![Pod crash loop](./coder-logstream-kube-logs-pod-crashed.png)
+
+## How it works
+
+Kubernetes provides an [informers](https://pkg.go.dev/k8s.io/client-go/informers) API that streams pod and event data from the API server.
+
+coder-logstream-kube listens for pod creation events with containers that have the CODER_AGENT_TOKEN environment variable set. All pod events are streamed as logs to the Coder API using the agent token for authentication. For more details, see the [coder-logstream-kube](https://github.com/coder/coder-logstream-kube) repository.
--- a/docs/platforms/kubernetes/index.md
+++ b/docs/platforms/kubernetes/index.md
@ -18,7 +18,7 @@ From the dashboard, import the Kubernetes starter template:

 In the next screen, set the following template variables:

- use_kubeconfig: `false` (The ServiceAccount will authorize Coder to create pods on your cluster)
+- `use_kubeconfig`: `false` (The ServiceAccount will authorize Coder to create pods on your cluster)
 - `namespace`: `coder` (or whatever namespace you deployed Coder on)

 ![Variables for Kubernetes template](../../images/platforms/kubernetes/template-variables.png)
--- a/examples/templates/kubernetes/README.md
+++ b/examples/templates/kubernetes/README.md
@ -7,7 +7,23 @@ icon: /icon/k8s.png

 # Getting started

-This template creates a pod running the `codercom/enterprise-base:ubuntu` image.
+This template creates a deplyment running the `codercom/enterprise-base:ubuntu` image.
+
+## Prerequisites
+
+This template uses [`kubernetes_deployment`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/deployment) terraform resource, which requires the `coder` service account to have permission to create deploymnets. For example if you are using [helm](https://coder.com/docs/v2/latest/install/kubernetes#install-coder-with-helm) to install Coder, you should set `coder.serviceAccount.enableDeployments=true` in your `values.yaml`
+
+```diff
+coder:
+serviceAccount:
+    workspacePerms: true
+-   enableDeployments: false
+   enableDeployments: true
+    annotations: {}
+    name: coder
+```
+
+> Note: This is only required for Coder versions < 0.28.0, as this will be the default value for Coder versions >= 0.28.0

 ## Authentication

@ -62,7 +78,7 @@ You may want to deploy workspaces on a cluster outside of the Coder control plan

 ## Namespace

-The target namespace in which the pod will be deployed is defined via the `coder_workspace`
+The target namespace in which the deployment will be deployed is defined via the `coder_workspace`
 variable. The namespace must exist prior to creating workspaces.

 ## Persistence
@ -96,3 +112,16 @@ resource "coder_agent" "main" {
 `code-server` is installed via the `startup_script` argument in the `coder_agent`
 resource block. The `coder_app` resource is defined to access `code-server` through
 the dashboard UI over `localhost:13337`.
+
+## Deployment logs
+
+To stream kubernetes pods events from the deployment, you can use Coder's [`coder-logstream-kube`](https://github.com/coder/coder-logstream-kube) tool. This can stream logs from the deployment to Coder's workspace startup logs. You just need to install the `coder-logstream-kube` helm chart on the cluster where the deployment is running.
+
+```shell
+helm repo add coder-logstream-kube https://helm.coder.com/logstream-kube
+helm install coder-logstream-kube coder-logstream-kube/coder-logstream-kube \
+    --namespace coder \
+    --set url=<your-coder-url-including-http-or-https>
+```
+
+For detailed instructions, see [Deployment logs](https://coder.com/docs/v2/latest/platforms/kubernetes/deployment-logs)
--- a/examples/templates/kubernetes/main.tf
+++ b/examples/templates/kubernetes/main.tf
@ -2,11 +2,11 @@ terraform {
  required_providers {
    coder = {
      source  = "coder/coder"
-      version = "~> 0.7.0"
+      version = "~> 0.11.0"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
-      version = "~> 2.18"
+      version = "~> 2.22"
    }
  }
 }
@ -198,7 +198,7 @@ resource "kubernetes_persistent_volume_claim" "home" {
      "app.kubernetes.io/name"     = "coder-pvc"
      "app.kubernetes.io/instance" = "coder-pvc-${lower(data.coder_workspace.me.owner)}-${lower(data.coder_workspace.me.name)}"
      "app.kubernetes.io/part-of"  = "coder"
-      // Coder specific labels.
+      //Coder-specific labels.
      "com.coder.resource"       = "true"
      "com.coder.workspace.id"   = data.coder_workspace.me.id
      "com.coder.workspace.name" = data.coder_workspace.me.name
@ -220,8 +220,12 @@ resource "kubernetes_persistent_volume_claim" "home" {
  }
 }

-resource "kubernetes_pod" "main" {
+resource "kubernetes_deployment" "main" {
  count = data.coder_workspace.me.start_count
+  depends_on = [
+    kubernetes_persistent_volume_claim.home
+  ]
+  wait_for_rollout = false
  metadata {
    name      = "coder-${lower(data.coder_workspace.me.owner)}-${lower(data.coder_workspace.me.name)}"
    namespace = var.namespace
@ -229,73 +233,90 @@ resource "kubernetes_pod" "main" {
      "app.kubernetes.io/name"     = "coder-workspace"
      "app.kubernetes.io/instance" = "coder-workspace-${lower(data.coder_workspace.me.owner)}-${lower(data.coder_workspace.me.name)}"
      "app.kubernetes.io/part-of"  = "coder"
-      // Coder specific labels.
-      "com.coder.resource"       = "true"
-      "com.coder.workspace.id"   = data.coder_workspace.me.id
-      "com.coder.workspace.name" = data.coder_workspace.me.name
-      "com.coder.user.id"        = data.coder_workspace.me.owner_id
-      "com.coder.user.username"  = data.coder_workspace.me.owner
+      "com.coder.resource"         = "true"
+      "com.coder.workspace.id"     = data.coder_workspace.me.id
+      "com.coder.workspace.name"   = data.coder_workspace.me.name
+      "com.coder.user.id"          = data.coder_workspace.me.owner_id
+      "com.coder.user.username"    = data.coder_workspace.me.owner
    }
    annotations = {
      "com.coder.user.email" = data.coder_workspace.me.owner_email
    }
  }
+
  spec {
-    security_context {
-      run_as_user = "1000"
-      fs_group    = "1000"
+    # replicas = data.coder_workspace.me.start_count
+    replicas = 1
+    selector {
+      match_labels = {
+        "app.kubernetes.io/name" = "coder-workspace"
+      }
    }
-    container {
-      name              = "dev"
-      image             = "codercom/enterprise-base:ubuntu"
-      image_pull_policy = "Always"
-      command           = ["sh", "-c", coder_agent.main.init_script]
-      security_context {
-        run_as_user = "1000"
-      }
-      env {
-        name  = "CODER_AGENT_TOKEN"
-        value = coder_agent.main.token
-      }
-      resources {
-        requests = {
-          "cpu"    = "250m"
-          "memory" = "512Mi"
-        }
-        limits = {
-          "cpu"    = "${data.coder_parameter.cpu.value}"
-          "memory" = "${data.coder_parameter.memory.value}Gi"
+
+    template {
+      metadata {
+        labels = {
+          "app.kubernetes.io/name" = "coder-workspace"
        }
      }
-      volume_mount {
-        mount_path = "/home/coder"
-        name       = "home"
-        read_only  = false
-      }
-    }
+      spec {
+        security_context {
+          run_as_user = 1000
+          fs_group    = 1000
+        }

-    volume {
-      name = "home"
-      persistent_volume_claim {
-        claim_name = kubernetes_persistent_volume_claim.home.metadata.0.name
-        read_only  = false
-      }
-    }
+        container {
+          name              = "dev"
+          image             = "codercom/enterprise-base:ubuntu"
+          image_pull_policy = "Always"
+          command           = ["sh", "-c", coder_agent.main.init_script]
+          security_context {
+            run_as_user = "1000"
+          }
+          env {
+            name  = "CODER_AGENT_TOKEN"
+            value = coder_agent.main.token
+          }
+          resources {
+            requests = {
+              "cpu"    = "250m"
+              "memory" = "512Mi"
+            }
+            limits = {
+              "cpu"    = "${data.coder_parameter.cpu.value}"
+              "memory" = "${data.coder_parameter.memory.value}Gi"
+            }
+          }
+          volume_mount {
+            mount_path = "/home/coder"
+            name       = "home"
+            read_only  = false
+          }
+        }

+        volume {
+          name = "home"
+          persistent_volume_claim {
+            claim_name = kubernetes_persistent_volume_claim.home.metadata.0.name
+            read_only  = false
+          }
+        }

-    affinity {
-      pod_anti_affinity {
-        // This affinity attempts to spread out all workspace pods evenly across
-        // nodes.
-        preferred_during_scheduling_ignored_during_execution {
-          weight = 1
-          pod_affinity_term {
-            topology_key = "kubernetes.io/hostname"
-            label_selector {
-              match_expressions {
-                key      = "app.kubernetes.io/name"
-                operator = "In"
-                values   = ["coder-workspace"]
+        affinity {
+          // This affinity attempts to spread out all workspace pods evenly across
+          // nodes.
+          pod_anti_affinity {
+            preferred_during_scheduling_ignored_during_execution {
+              weight = 1
+              pod_affinity_term {
+                topology_key = "kubernetes.io/hostname"
+                label_selector {
+                  match_expressions {
+                    key      = "app.kubernetes.io/name"
+                    operator = "In"
+                    values   = ["coder-workspace"]
+                  }
+                }
              }
            }
          }