Documentation

Sandboxing for untrusted commands on Kubernetes.

Shiitake is an HTTP server that fans commands out to a pool of resident, resource-bounded worker containers. Each command runs in its own worker, which resets to a clean slate between commands — so there is no state bleed from one caller to the next. The server is the only ingress; workers never bind a public port.

Shiitake is generic: it has no knowledge of any particular application. You bring the toolchain image, drop in the worker binary as its entrypoint, and the server dispatches commands to the pool.

Overview

The project is a small Rust workspace plus two thin clients:

Architecture

One server process is the single entry point. It accepts POST /api/v1/exec, picks an idle worker from the pool, and hands the command to it over a WebSocket on the dispatch listener. The worker runs the command, streams stdout/stderr straight to capture files on a shared volume, reports the result, and then resets its sandbox before serving the next command.

Workers never bind a port — they dial the dispatcher, at whatever SHIITAKE_DISPATCH_URL says, presenting a bearer token on the upgrade. That address is the only thing that changes between running the server and the workers in one Pod and running them in two.

flowchart TB
  client["Client"]

  subgraph cluster["Kubernetes cluster"]
    api["Kubernetes API"]
    subgraph pod["Shiitake Pod(s)"]
      server["server"]
      subgraph workers["worker pool"]
        w0["worker-0"]
        w1["worker-1"]
        wn["worker-N"]
      end
      vol[("shared capture volume")]
    end
  end

  client -->|"exec"| server
  server -->|"dispatch (ws + bearer)"| w0
  server --> w1
  server --> wn
  server -.->|"get pods"| api
  w0 -->|"write"| vol
  w1 --> vol
  wn --> vol
  server -.->|"read"| vol
      

POST /api/v1/exec  →  Server (only ingress)  →  Worker pool  →  one command per worker
                                                      ↑               ↓ resets between commands
                                                      └───── re-advertised idle ─────┘

Resident workers & the clean slate

A worker connects once and serves commands in a loop. Between commands it resets its sandbox to reproduce the effect of a container teardown without the cost of one:

• Process sweep. The worker is its container's pid 1, so it SIGKILLs every other process in the PID namespace — no leftover background processes from the previous command.
• Scratch clear. It empties the scratch directories listed in SHIITAKE_RESET_PATHS (symlink-safe), e.g. /tmp, /var/tmp, /dev/shm.
• IPC removal. It removes SysV IPC objects and re-reads /proc/sysvipc to verify they are gone.

If a reset cannot be trusted clean — a process won't die, a scratch clear fails, or an IPC object survives — the worker recycles into a fresh container rather than serving on a dirty sandbox. Staying resident avoids the kubelet CrashLoopBackOff that per-command container exits would otherwise incur.

Output capture

The worker redirects the command's stdout/stderr file descriptors straight into per-stream capture files — the kernel writes to disk, so neither the worker nor the server buffers output in memory. The server reads it back with HTTP Range support. Storage is bounded only by the volume; output sizes are exported as metrics so a runaway command is observable rather than silently truncated.

Local quickstart

Run the server and a worker locally with Cargo, then dispatch a command:

# Terminal 1 — the server (the only ingress)
SHIITAKE_AUTH_TOKEN=dev-token SHIITAKE_DISPATCH_TOKEN=dev-dispatch \
  SHIITAKE_CAPTURE_ROOT=/tmp/capture \
  cargo run --bin shiitake-server

# Terminal 2 — one worker, joining the pool
# (SHIITAKE_DISPATCH_URL defaults to ws://127.0.0.1:8090/dispatch)
SHIITAKE_WORKER_ID=worker-0 SHIITAKE_DISPATCH_TOKEN=dev-dispatch \
  SHIITAKE_CAPTURE_ROOT=/tmp/capture \
  cargo run --bin shiitake-worker

# Terminal 3 — wait for the pool, then dispatch a command
# (-f fails on the 503 /ready returns while no worker has registered)
until curl -sf localhost:8080/api/v1/ready >/dev/null; do sleep 1; done
curl -s -H "Authorization: Bearer dev-token" \
  -X POST localhost:8080/api/v1/exec \
  -d '{"command": "echo hello from a sandboxed worker"}'

Installation

Shiitake is distributed as two OCI images published to GitHub Container Registry:

The server image is a minimal Alpine wrapping a static musl binary; it makes outbound TLS calls to the Kubernetes API (for OOM detection) but never executes user commands itself. The worker is shipped only as a binary so you can layer it into whatever toolchain image your commands need — see building a custom worker image.

Deploying on Kubernetes

The simplest deployment is a single Pod running one server container and N worker containers. Two things must be shared across them:

• The pod network namespace — so workers reach the dispatcher on 127.0.0.1, the default SHIITAKE_DISPATCH_URL. Within a Pod this is automatic.
• A capture volume — mounted into the server and every worker at the same path (SHIITAKE_CAPTURE_ROOT). Use an emptyDir, or a persistent volume if the capture files need to outlive the Pod for reasons of your own — the server itself purges the volume at startup, since handles live in memory and nothing on it can be served after a restart.

Server and workers can also run as separate Pods, which is what lets pod-scoped controls differ between them.

A minimal single-Pod spec looks like this:

# one server + workers in one Pod, sharing netns + a capture volume
spec:
  serviceAccountName: shiitake        # bound to the pod-reader Role below
  containers:
    - name: server
      image: ghcr.io/tenzailabs/shiitake-server
      env:
        - name: SHIITAKE_AUTH_TOKEN
          valueFrom: { secretKeyRef: { name: shiitake, key: token } }
        - name: SHIITAKE_DISPATCH_TOKEN
          valueFrom: { secretKeyRef: { name: shiitake, key: dispatch-token } }
        - name: SHIITAKE_CAPTURE_ROOT
          value: /capture
        - name: POD_NAME
          valueFrom: { fieldRef: { fieldPath: metadata.name } }
        - name: POD_NAMESPACE
          valueFrom: { fieldRef: { fieldPath: metadata.namespace } }
      readinessProbe:                             # gated on the pool: 503 until a worker registers
        httpGet: { path: /api/v1/ready, port: 8080 }
      livenessProbe:                              # process-level only; an empty pool is not a restart
        httpGet: { path: /api/v1/health, port: 8080 }
      volumeMounts:
        - { name: capture, mountPath: /capture }
    - name: worker-0
      image: your-registry/your-worker:latest   # your toolchain + the worker binary
      restartPolicy: Always                       # recycle this container on its own
      env:
        - { name: SHIITAKE_WORKER_ID,   value: worker-0 }
        - name: SHIITAKE_DISPATCH_TOKEN
          valueFrom: { secretKeyRef: { name: shiitake, key: dispatch-token } }
        - { name: SHIITAKE_CAPTURE_ROOT, value: /capture }
        - { name: SHIITAKE_RESET_PATHS,  value: "/tmp,/var/tmp,/dev/shm" }
      volumeMounts:
        - { name: capture, mountPath: /capture }
  volumes:
    - { name: capture, emptyDir: {} }

RBAC for the OOM probe

The server detects out-of-memory kills by reading the kubelet's container status from the Kubernetes API. Bind its ServiceAccount to a Role that can get pods in its own namespace — nothing more:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata: { name: shiitake-pod-reader }
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get"]

Example Helm chart

The repository ships a reference chart under tests/chart that deploys either topology (a topology value picks one) — server + a configurable worker count, the pod-reader RBAC, the dispatch Service and worker NetworkPolicy for the split shape, an optional OpenTelemetry Collector, and tunable resources.limits. It is built for the end-to-end suite, so treat it as a worked example rather than a production-hardened chart.

Topologies

Two supported shapes. The binaries, the wire protocol and the HTTP API are identical in both — what changes is where the containers sit and how the worker addresses the dispatcher.

Why split them

A NetworkPolicy selects a Pod, not a container. While the server and the workers share one, every egress the server legitimately needs — the Kubernetes API to classify an OOM-killed worker, a collector to export telemetry to — is necessarily also granted to the worker containers, which are the ones running arbitrary commands. Per-container controls such as the service-account token mount can be kept narrow, but network reach cannot. Hardening the workers and giving the server what it needs pull in opposite directions, and the operator has to choose one.

Split apart, the workers get a policy of their own — no ingress, egress only to DNS and the dispatcher — while the server pod keeps its reach. Neither side gives anything up.

What changes

• Dispatch is addressed by URL. Bind the server's dispatch listener with SHIITAKE_DISPATCH_HOST=0.0.0.0, put a cluster-internal Service in front of it, and point the workers at it with SHIITAKE_DISPATCH_URL.
• Dispatch is authenticated. SHIITAKE_DISPATCH_TOKEN is required on both sides and checked on the HTTP upgrade, before a WebSocket exists — being loopback-bound is no longer what protects that path. It is a separate secret from SHIITAKE_AUTH_TOKEN: a worker never needs the API's token.
• Capture must span both Pods. The worker writes the capture files and the server reads them back, so SHIITAKE_CAPTURE_ROOT has to name the same storage in both. An emptyDir cannot do this.
• Worker ids must stay unique. A Deployment of worker Pods shares one Pod template, so take the id from the downward API (metadata.name) rather than a literal.
• Workers report their own Pod. Wire POD_NAME, POD_NAMESPACE and SHIITAKE_CONTAINER_NAME into the worker; it puts them on its Hello so the server's OOM probe queries the worker's Pod rather than its own.
• The dispatch Service must publish not-ready addresses. The server is not ready until SHIITAKE_MIN_READY_WORKERS workers have registered, and they register through this Service. A Service that routed only to ready endpoints would deadlock the two on each other — the server waiting for workers, the workers unable to reach it. Set publishNotReadyAddresses: true.

The manifests

Four objects, plus the pod-reader RBAC the server already needs. A working chart for both topologies lives in tests/chart.

# 1. the server — dispatch bound wide, and its own Pod
apiVersion: apps/v1
kind: Deployment
metadata: { name: shiitake, labels: { app: shiitake } }
spec:
  replicas: 1
  selector: { matchLabels: { app: shiitake } }
  template:
    metadata: { labels: { app: shiitake } }
    spec:
      serviceAccountName: shiitake          # bound to the pod-reader Role
      containers:
        - name: server
          image: ghcr.io/tenzailabs/shiitake-server
          ports:
            - { name: api,      containerPort: 8080 }
            - { name: dispatch, containerPort: 8090 }
          env:
            - { name: SHIITAKE_DISPATCH_HOST, value: "0.0.0.0" }   # not loopback any more
            - name: SHIITAKE_AUTH_TOKEN
              valueFrom: { secretKeyRef: { name: shiitake, key: token } }
            - name: SHIITAKE_DISPATCH_TOKEN
              valueFrom: { secretKeyRef: { name: shiitake, key: dispatch-token } }
            - { name: SHIITAKE_CAPTURE_ROOT, value: /capture }
            - name: POD_NAME
              valueFrom: { fieldRef: { fieldPath: metadata.name } }
            - name: POD_NAMESPACE
              valueFrom: { fieldRef: { fieldPath: metadata.namespace } }
          readinessProbe:
            httpGet: { path: /api/v1/ready, port: 8080 }
          livenessProbe:
            httpGet: { path: /api/v1/health, port: 8080 }
          volumeMounts:
            - { name: capture, mountPath: /capture }
      volumes:
        - name: capture
          persistentVolumeClaim: { claimName: shiitake-capture }   # ReadWriteMany
---
# 2. how the workers find it. publishNotReadyAddresses is required:
#    the server is not ready until workers register, and they register here.
apiVersion: v1
kind: Service
metadata: { name: shiitake-dispatch }
spec:
  publishNotReadyAddresses: true
  selector: { app: shiitake }
  ports:
    - { name: dispatch, port: 8090, targetPort: dispatch }

# 3. worker Pods: their own id, the dispatch Service, and no API-server reach
apiVersion: apps/v1
kind: Deployment
metadata: { name: shiitake-workers, labels: { app: shiitake-worker } }
spec:
  replicas: 8                               # pool size — one worker per Pod
  selector: { matchLabels: { app: shiitake-worker } }
  template:
    metadata: { labels: { app: shiitake-worker } }
    spec:
      automountServiceAccountToken: false    # workers never talk to the API server
      containers:
        - name: worker
          image: your-registry/your-worker:latest
          env:
            - name: SHIITAKE_WORKER_ID          # unique per replica
              valueFrom: { fieldRef: { fieldPath: metadata.name } }
            - { name: SHIITAKE_DISPATCH_URL,   value: "ws://shiitake-dispatch:8090/dispatch" }
            - name: SHIITAKE_DISPATCH_TOKEN
              valueFrom: { secretKeyRef: { name: shiitake, key: dispatch-token } }
            - { name: SHIITAKE_CONTAINER_NAME, value: worker }
            - { name: SHIITAKE_CAPTURE_ROOT,   value: /capture }
            - { name: SHIITAKE_RESET_PATHS,    value: "/tmp,/var/tmp,/dev/shm" }
            - name: POD_NAME
              valueFrom: { fieldRef: { fieldPath: metadata.name } }
            - name: POD_NAMESPACE
              valueFrom: { fieldRef: { fieldPath: metadata.namespace } }
          volumeMounts:
            - { name: capture, mountPath: /capture }   # same path as the server
      volumes:
        - name: capture
          persistentVolumeClaim: { claimName: shiitake-capture }
---
# 4. the policy that is now expressible: DNS and the dispatcher, nothing else
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: shiitake-worker }
spec:
  podSelector:
    matchLabels: { app: shiitake-worker }
  policyTypes: ["Ingress", "Egress"]
  ingress: []
  egress:
    - to:
        - namespaceSelector: {}
          podSelector:
            matchLabels: { k8s-app: kube-dns }
      ports:
        - { protocol: UDP, port: 53 }
        - { protocol: TCP, port: 53 }
    - to:
        - podSelector:
            matchLabels: { app: shiitake }
      ports:
        - { protocol: TCP, port: 8090 }

Failure model

Once the server and the workers are separate Pods, either can die without the other. What each side does about it:

Two consequences worth designing around. A command killed by a server restart is not retried for you — the caller sees a handle that 404s and decides what to do. And if that caller retries, it runs the command a second time; shiitake guarantees a request is dispatched once, not that your command is idempotent.

Building a custom worker image

The worker is application-agnostic and ships as a static binary so it can run inside your toolchain. Copy it out of the published worker image into your own base image and set it as the entrypoint:

FROM your/toolchain:latest

# Pin by immutable digest — GHCR tags (incl. release tags) are mutable,
# so a `COPY --from` by tag can be silently moved. Get the digest from
# the package page or `docker buildx imagetools inspect <image>:<tag>`.
COPY --from=ghcr.io/tenzailabs/shiitake-worker@sha256:<digest> \
     /usr/local/bin/shiitake-worker /usr/local/bin/shiitake-worker

ENTRYPOINT ["/usr/local/bin/shiitake-worker"]

A few things to get right in a custom image:

• Ship bash. Commands run as bash -c <command>, so your base image must provide bash.
• Include the tools your commands need. The worker clears the command environment, so any external binary must be on the PATH you pass in the request (see the API).
• List scratch paths to reset. Set SHIITAKE_RESET_PATHS to every directory that must be wiped between commands, and omit anything that must persist. Shiitake never assumes a path layout — clearing the wrong directory, or failing to clear a writable one, is your call to make.

Both the server and the worker are built for *-unknown-linux-musl, so the worker binary is fully static and runs in any Linux base image, including scratch (though you'll still want bash and your toolchain).

Configuration

Both binaries take their configuration from flags with SHIITAKE_* environment fallbacks.

Server

Worker

HTTP API

The public API is versioned under /api/v1. Every /exec route requires Authorization: Bearer <token>; only /health and /ready are unauthenticated. The worker dispatch endpoint (/dispatch) is a separate internal router on its own listener, with its own bearer token (SHIITAKE_DISPATCH_TOKEN) and is never publicly reachable.

Spawn a command. Returns {handle, started_at} with 202 Accepted; 429 if the pool has no idle worker. Request body:

{
  "command": "python3 -c 'print(2 + 2)'",
  "workdir": "/tmp",
  "timeout": 300.0,
  "env": { "PATH": "/usr/bin:/bin" },
  "drop_to": { "uid": 1000, "gid": 1000, "supplementary_gids": [], "umask": 7 }
}

Only command is required. Notes:

• command is a single string, run verbatim as bash -c <command> — use ordinary shell syntax for pipes, redirects, and multi-statement scripts. There is no argv array; the worker always goes through a shell.
• The command runs with only the env you pass — the worker clears its own environment first. Include PATH for any command that calls an external binary (bash builtins like echo work without it).
• timeout is in seconds (default 300). workdir defaults to the server's SHIITAKE_DEFAULT_WORKDIR.
• drop_to is optional; see privilege drop.

Status of a handle: status (running / completed / timeout / oomkilled / error), exit_code, exit_signal, exit_cause, timestamps, and per-stream byte counters.

Read captured output. Serves the capture file with HTTP Range support (206 / 416); tail the last N bytes with Range: bytes=-N.

Terminate the command (SIGTERM → SIGKILL the process group). Idempotent on already-terminal handles.

Liveness. Always 200 while the server is serving, plus a pool snapshot (workers_idle, workers_inflight). Unauthenticated. An empty pool is not a reason to restart the process, so this endpoint deliberately ignores the pool — use /ready to gate traffic.

Readiness. 200 once at least SHIITAKE_MIN_READY_WORKERS workers are registered, 503 otherwise — so an orchestrator probe can gate on the status code alone and keep traffic off a pod whose workers have not connected (or have all died). Unauthenticated. The body carries the same verdict:

{
  "ready": true,
  "service": "shiitake",
  "workers_idle": 7,
  "workers_inflight": 1,
  "workers_required": 1
}

A worker counts as registered whether it is idle or running a command, so a fully-busy pool stays ready — gating on idle workers alone would pull a pod out of rotation exactly when it is doing the most work. Raise SHIITAKE_MIN_READY_WORKERS to report unready below some fraction of a large pool rather than only at zero.

Exit causes

A finished handle reports one exit_cause:

The dispatch protocol

The contract between the server and a worker. You do not need this to run shiitake — drop in the shipped worker binary and it speaks it for you. You need it to write your own worker, or to reason about what crosses the wire when the two are in different Pods.

Connecting

The worker dials SHIITAKE_DISPATCH_URL and presents its bearer token on the HTTP upgrade request, not as a frame:

GET /dispatch HTTP/1.1
Upgrade: websocket
Authorization: Bearer <SHIITAKE_DISPATCH_TOKEN>

A wrong or missing token fails the handshake with 401, so an unauthenticated peer never gets to send a frame. After the upgrade every message is a JSON text frame with a kind discriminator. The worker sends Hello first and the server dispatches only after it arrives.

Frames

// worker → server, immediately after the upgrade.
// `location` is optional: omit it and the server assumes the worker is a
// container of its OWN Pod named after worker_id (the single-Pod case).
// In the two-Pod topology it is required for OOM detection to work.
{"kind": "hello",
 "worker_id": "shiitake-workers-6d4b8f9c7-x2k9p",
 "location": {"pod": "shiitake-workers-6d4b8f9c7-x2k9p",
              "namespace": "shiitake",
              "container": "worker"}}

// server → worker. drop_to is optional; absent means run as the worker's uid.
{"kind": "execute",
 "request_id": "0f3c…",
 "command": "echo hi",
 "working_dir": "/tmp",
 "env": {"PATH": "/usr/bin:/bin"},
 "timeout_secs": 300.0,
 "drop_to": {"uid": 1000, "gid": 1000, "supplementary_gids": [], "umask": 7}}

// server → worker
{"kind": "cancel", "request_id": "0f3c…"}

// worker → server. Output is NOT here — it is already on the capture volume.
// `error` is set only when the worker could not run the command at all.
{"kind": "result",
 "request_id": "0f3c…",
 "exit_code": 0,
 "exit_signal": null,
 "timed_out": false,
 "cancelled": false,
 "usage": {"memory_peak_bytes": 1048576, "memory_limit_bytes": 536870912,
           "cpu_user_seconds": 0.01, "cpu_system_seconds": 0.00},
 "error": null}

Capture, not streaming

Command output never crosses this socket. The worker redirects the child's stdout/stderr file descriptors straight into files, and the server reads those back over HTTP. Both sides mount the same volume at the same path (SHIITAKE_CAPTURE_ROOT) and agree on this layout:

<capture root>/<request_id>/stdout
<capture root>/<request_id>/stderr

The worker creates both files at exec start, so a known handle with no file means the command produced nothing on that stream — not an error. The server stats them for the byte counters it reports and serves them with HTTP range support. This is why the split topology needs a ReadWriteMany volume: the reader and the writer are in different Pods.

Liveness

The server sends WebSocket pings to every registered worker — idle and in-flight — and evicts one that goes silent, reconciling its handles. A worker answers pings while waiting, while running a command, and across its between-command reset. In the other direction the worker gives up on a session after SHIITAKE_LEASE_TIMEOUT of silence; see the failure model for what it does next, which depends on whether a command was running.

Clients

Two first-party clients wrap the HTTP API so you don't hand-roll requests.

Python (httpx)

from shiitake.client import AsyncShiitakeClient

async with AsyncShiitakeClient("http://shiitake:8080", auth_token="...") as c:
    result = await c.run({"command": "echo hi", "env": {"PATH": "/bin"}})
    print(result.stdout)

The client also exposes lower-level spawn / status / read / kill calls and an AsyncHandle with wait() and slurp() helpers. A synchronous ShiitakeClient is available too.

Rust (reqwest)

use shiitake_rs::{Client, ExecRequest};

let client = Client::new("http://shiitake:8080", Some(token));
let handle = client.spawn(&ExecRequest { command: "echo hi".into(), ..Default::default() }).await?;
let status = client.status(&handle.handle).await?;

The request/response types are re-exported from shiitake-server-api, so callers depend on just the one client crate.

Observability

Telemetry lives entirely in the server — the worker stays lean (no OTel dependencies). When OTEL_EXPORTER_OTLP_ENDPOINT is set, the server exports over OTLP:

• Traces — one shiitake.exec span per command.
• Metrics — shiitake_-prefixed gauges/counters for exit cause, duration, memory and CPU (from the worker's cgroup readings), per-stream output sizes, capture-volume free space, and pool occupancy.

Without an endpoint set, the server logs to stdout only. The OTLP exporter is plaintext-only by design (no TLS feature, to keep the static musl link free of aws-lc-rs); terminate TLS at a sidecar collector if you need it.

Interactive PTY

Alongside the fire-and-forget /exec, a worker can host an interactive terminal — a persistent shell on a real TTY (job control, vim, resize, tab-completion) with a live, bidirectional byte stream. A PTY session pins one worker for its whole lifetime rather than returning it to the pool per command, so it costs the pool one worker until it ends.

Opening a session

GET /api/v1/pty is a WebSocket upgrade, bearer-gated exactly like /exec (SHIITAKE_AUTH_TOKEN). The first message is a JSON open control frame; after it, binary frames are the byte stream — client → server is stdin, server → client is pty output — and JSON frames stay control. The server closes the socket when the shell exits.

// client → server, the FIRST message. `command` empty uses the worker's
// default shell (SHIITAKE_PTY_SHELL). `drop_to` is optional, as on /exec.
{"op": "open",
 "command": ["bash", "-i"],
 "working_dir": "/tmp",
 "env": {"PATH": "/usr/bin:/bin"},
 "cols": 120, "rows": 40,
 "drop_to": {"uid": 1000, "gid": 1000, "supplementary_gids": []}}

// client → server, at any time — reflow the tty
{"op": "resize", "cols": 80, "rows": 24}

The worker injects TERM=xterm-256color and runs the shell in working_dir under the caller-supplied drop_to — the same isolation as a command, just persistent. Scrollback is the client's; nothing is captured to disk. The server↔worker dispatch protocol carries matching pty_open / pty_resize / pty_close / pty_exit frames for anyone writing their own worker.

The default shell

When the open frame carries no command, the worker spawns SHIITAKE_PTY_SHELL (whitespace-split argv), defaulting to bash -i. Pointing it at a transparent tmux makes every terminal a tmux session without any protocol change:

# on the worker
SHIITAKE_PTY_SHELL="tmux new-session -A -D -s main"

Naming the session's user

drop_to is numeric, but a human terminal reads better as a login name than a bare uid. So drop_to accepts an optional name (plus create_home): while still privileged, before the drop, the worker best-effort ensures a matching /etc/passwd entry (and a home seeded from /etc/skel) so whoami, id -un, and the shell prompt resolve the uid to that name. The home lives at <SHIITAKE_HOME_ROOT>/<name> (default root /home), and the worker sets the shell's HOME to it so the directory, the passwd home, and HOME agree.

• Idempotent — skipped when the uid or name already resolves.
• Never a root command — the worker only names the uid it is already about to become; it runs no caller command as root, and never names uid 0.
• Cleaned on reset — the between-session reset removes every passwd entry and home the worker created, so a pooled worker never accumulates names, and one session never sees another's.

Shiitake stays identity-agnostic: it reflects whatever drop_to asks for. Deciding the uid and the name — e.g. mapping an authenticated principal to a stable uid and a display name — is the caller's policy.

Pinning & capacity

A session takes one idle worker and holds it until the shell exits or the client disconnects; on the way out the worker resets before rejoining the pool, so the next caller never inherits a dirty sandbox. If no worker is idle the upgrade closes with a try-again code. /health reports workers_interactive alongside workers_idle / workers_inflight, so you can see the capacity terminals are holding.

Security considerations

Shiitake runs untrusted commands by design. Its isolation properties — and where they stop — are worth understanding before you expose it.

What Shiitake provides

• Single ingress, authenticated. Every /exec route requires the bearer token (SHIITAKE_AUTH_TOKEN); the server refuses to start without one. Treat the token as a secret — mount it from a Kubernetes Secret, not a plaintext env literal.
• Workers never face the network. Workers bind no port at all — they dial out to the dispatcher. The dispatch endpoint is a separate listener behind its own bearer token (SHIITAKE_DISPATCH_TOKEN, required), bound to loopback when server and workers share a Pod and to a cluster-internal Service when they do not. Never expose it beyond the cluster.
• Pod-scoped controls, where you want them. Running the server and the workers as separate Pods lets a NetworkPolicy, a service-account mount, a RuntimeClass or a seccomp profile apply to the containers running arbitrary commands without also applying to the server — which needs egress to the Kubernetes API and to a telemetry collector that the workers have no business having.
• Per-command isolation. Each command runs in its own resource-bounded worker container; it cannot see or touch another command's processes, files, or output. Between commands the worker resets to a clean slate (process sweep, scratch clear, IPC removal), and if that reset can't be trusted the worker recycles into a fresh container instead of serving dirty.
• Resource bounds. Memory and CPU are enforced at the container level via the Pod's resources.limits. An allocation bomb kills only its own worker — the server and every other worker keep running. The kill is surfaced as oom_container, never silently lost.
• Identity-agnostic privilege drop. An /exec request may carry a drop_to directive (uid, gid, supplementary gids, umask); the worker applies it in the post-fork pre_exec hook before exec. Shiitake never decides identities — your embedding layer maps its own auth to drop_to.
• Clean command environment. The worker clears its own environment before running a command, so the worker's secrets and config don't leak into the command. The command sees only the env in the request.

What you must handle

• The reset list is yours to get right. The clean-slate guarantee is only as good as SHIITAKE_RESET_PATHS. Any writable directory that isn't listed — or any state outside a filesystem the reset can reach — persists across commands. When in doubt, lean on SHIITAKE_RESTART_AFTER for periodic full container teardown.
• The process sweep needs pid 1. The worker sweeps leftover processes only when it is its container's pid 1 (its own PID namespace). Don't run it behind an init shim or share a PID namespace across workers.
• Output storage is unbounded. Capture files grow until the volume is full; sizes are reported as metrics rather than capped. Size the capture volume for your workloads and alert on free space.
• Capture is shared. The server and every worker mount the same capture volume. The worker writes; the server reads. Don't widen those mounts beyond what's needed.
• No TLS on the API itself. Terminate TLS at an ingress / sidecar in front of the server. The OTLP exporter is likewise plaintext-only.

Development & testing

cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace                # unit + in-process integration tests

# full k3d cluster end-to-end (docker, k3d, kubectl, helm, python3, uv)
bash tests/build.sh && bash tests/setup.sh && bash tests/run.sh

Local + CI e2e tooling (k3d, kubectl, python) is managed by mise — run mise install. The Rust toolchain is pinned in rust-toolchain.toml. See AGENTS.md for the architecture notes and gotchas that keep the static musl build and the worker's clean-slate guarantee intact.

License

Shiitake is licensed under the Apache License 2.0. Contributions are welcome — open an issue or a pull request; by contributing you agree your contributions are licensed under the same terms.