Greentic · Phase E · Workload Identity

The Kubernetes + Vault demo, explained simply

A Greentic digital worker runs in Kubernetes and reads its own passwords directly from HashiCorp Vault — proving its identity with nothing but the badge Kubernetes gives the pod. No password is ever copied into the cluster. This page explains, in plain words, what each step does and shows the real proof from a live run.

✓ Verified end-to-end on 2026-06-29

The whole idea in one sentence

Instead of handing the worker its secrets, we put the secrets in a vault, give the worker an ID badge, and let the worker fetch its own secrets at runtime — so a snapshot of Kubernetes contains zero sensitive values.

On this page
  1. Why this is different
  2. The picture
  3. Tools you use (and one trap)
  4. Step 0 — Wake & teach Vault
  5. Step 1 — Make the environment
  6. Step 2 — Apply the blueprint
  7. Step 3 — Put secrets in Vault
  8. Step 4 — Start it on the cluster
  9. Step 5 — Verify (real output)
  10. The blueprint file

Why this is different

Two ways to give a worker a secret — only one keeps the cluster clean.

The old way (dev-store bridge)This demo (Vault workload identity)
The deploy tool copies the secret value into a Kubernetes Secret object, and an init-container injects it into the pod. The deploy tool writes only the worker's identity and a pointer to Vault. The worker fetches the value itself at boot.
Anyone with kubectl get secret can read the password (base64 ≠ encryption). kubectl get secret shows nothing sensitive. The value only ever lives inside Vault, encrypted.
The pod is trusted because the secret was pushed to it. The pod proves who it is (its Kubernetes ServiceAccount), and Vault hands back only the secrets that identity is allowed to read.

The picture

Kubernetes cluster · namespace “greentic” Operator (your laptop) greentic-deployer-dev seeds via a short tunnel HashiCorp Vault KV (encrypted values) transit (wrap/unwrap) k8s auth + read-only rule Worker pod identity: SA gtc-worker GREENTIC_SECRETS_BACKEND = vault (+ VAULT_ADDR) no secret value mounted ① seed (encrypted) ② login with SA badge ③ read + decrypt inbound webhook → 200 / 401
① The operator seeds secrets into Vault (encrypted, through a short-lived tunnel). ② At boot the worker logs into Vault using its Kubernetes ServiceAccount. ③ Vault returns only the secrets the worker's read-only rule allows; the worker decrypts them in memory.

Tools you use (and one trap)

You installed the binaries with gtc-dev install --release nextgen-deployer. They already contain everything this demo needs.

⚠ Use greentic-deployer-dev, not gtc-dev op

gtc-dev op … routes to greentic-operator-dev, and the version pinned in this release does not contain the Vault fixes. The fixes (render Vault identity #392, seed Vault via op secrets put #394, scope the webhook secret to the env owner #395) live in greentic-deployer-dev.

So every command below calls greentic-deployer-dev op … directly. We verified the installed binary really carries those fixes.

The serving image is separate

The worker runs the container ghcr.io/greenticai/greentic-start-distroless:develop — not your local greentic-start-dev. It must be newer than greentic-start #305 (2026‑06‑26). The image loaded in the cluster was built at 2026-06-26T17:40Z — 15 minutes after #305 — so it is good.

0Wake & teach Vault

One-time per cluster — and again any time the Vault pod restarts.

kubectl exec into the vault pod → enable transit, k8s login, a read-only rule, and a role

kubectl -n greentic exec -i deploy/vault -- env VAULT_ADDR=http://127.0.0.1:8200 VAULT_TOKEN=root sh -s <<'SH'
vault secrets enable -path=transit transit || true
vault write -f transit/keys/greentic
vault auth enable kubernetes || true
vault write auth/kubernetes/config kubernetes_host=https://kubernetes.default.svc
vault policy write gtc-worker-ro - <<'POL'
path "secret/data/greentic/vault-demo/tenant-default/*"     { capabilities = ["read"] }
path "secret/metadata/greentic/vault-demo/tenant-default/*" { capabilities = ["read","list"] }
path "transit/decrypt/greentic"                             { capabilities = ["update"] }
POL
vault write auth/kubernetes/role/gtc-worker bound_service_account_names=gtc-worker bound_service_account_namespaces=greentic policies=gtc-worker-ro ttl=1h
vault audit enable file file_path=stdout || true
SH
Plain

Open the safe and set the rules: turn on the encryption engine, turn on “log in with a Kubernetes badge”, and write a rule that says “the worker may only read this one environment's drawer, and may unlock wrapped keys.” Then issue that rule to the badge named gtc-worker.

Does

Enables transit (envelope encryption), enables the Kubernetes auth method, writes the read-only policy gtc-worker-ro scoped to vault-demo / tenant-default, binds it to the gtc-worker ServiceAccount, and turns on the audit log so we can prove what the worker reads.

Why

This Vault runs in dev mode — it keeps everything in memory, so a pod restart wipes it. That is exactly what happened on this machine, which is why we re-run this step. (Dev mode is for demos only.)

1Make the environment

A named, tenant-owned environment in a store under the demo folder — so it never collides with your other environments.

create the env in an isolated store

# fish shell:
set -gx STORE /home/vampik/greenticai/my_demos/k8s-vault-demo/state/.greentic/environments

greentic-deployer-dev op --store-root $STORE env create vault-demo --tenant-org tenant-default --name vault-demo
Plain

Open a fresh, labelled filing cabinet that belongs to the customer tenant-default, and keep it in this demo's own folder so it can't mix with anything else.

Does

Creates the environment vault-demo owned by tenant-default inside a private store at --store-root $STORE.

Why

The runtime refuses to serve secrets across tenants: a Vault env must be owned by the same tenant the bot serves, or it fails closed. The plain local env has no owner, so we use a named, owned one. The next step (apply) updates an existing env — it won't create a named one for you.

2Apply the blueprint

One file, one command — it does the work of ten manual commands.

op env apply — the whole wiring in a single shot

greentic-deployer-dev op --store-root $STORE \
  --answers /home/vampik/greenticai/my_demos/k8s-vault-demo/manifest/env-manifest.json \
  env apply --yes
Plain

Hand over one blueprint and let the tool build the setup: trust the operator's signing key, use Vault for secrets, use Kubernetes to deploy, run the webchat-bot app, and add a Telegram doorway that points at it.

Does

In one run it bootstraps the trust root, binds the secrets→Vault and deployer→Kubernetes packs, deploys the bundle (stage + warm + 100% traffic), and adds + links the Telegram endpoint. All of this is recorded in the store — nothing touches the cluster yet.

Why

The manifest replaces ten separate op commands with one declarative document you can keep in git and re-apply safely (re-running an unchanged manifest is a no-op).

Why the bot-token secret is not in the blueprint

For a named Vault env, apply checks each secret against the bound backend at planning time — but the Vault pack is only bound during that same run, so a secret listed in the manifest fails with “no secrets env-pack bound.” (The reference dev-store demo gets away with it only because it targets local, which auto-seeds default packs.) So for Vault we seed secrets in the next step, after the pack is bound.

3Put the secrets in Vault

Two values go straight into Vault, encrypted — never into Kubernetes.

open a short tunnel to Vault, then push two secrets through Greentic's secure path

# admin tunnel to Vault (only while seeding)
kubectl -n greentic port-forward svc/vault 8200:8200 &

# the webhook secret's name is tied to the endpoint id created in step 2 — look it up
set EPID (greentic-deployer-dev op --store-root $STORE messaging endpoint list vault-demo | jq -r '.result.endpoints[0].endpoint_id')

# 1) the Telegram bot token (fixed name)
env VAULT_ADDR=http://127.0.0.1:8200 VAULT_TOKEN=root \
  greentic-deployer-dev op --store-root $STORE --answers seed-tok.json secrets put

# 2) the per-endpoint webhook secret (name includes the endpoint id)
env VAULT_ADDR=http://127.0.0.1:8200 VAULT_TOKEN=root \
  greentic-deployer-dev op --store-root $STORE --answers seed-wh.json secrets put
Plain

Drop the two real passwords into the safe through a sealed slot. Greentic wraps each value in encryption before it's stored, so even inside Vault it's never sitting in the clear — and the cluster never sees it at all.

Does

Uses op secrets put (the #394 Vault path) which transit-encrypts each value and writes it to the exact KV path the worker will read. VAULT_ADDR/VAULT_TOKEN come from the environment (the tunnel + the admin token); the Vault mounts/paths come from the env's binding, so the seeded path matches the worker's read path.

Why

The webhook secret's name is derived from the endpoint's generated id and, for Vault, its value is not auto-written (the operator must seed it). The bot token's name is fixed. The tunnel reaches the pod's own loopback, so seeding works while the worker (which uses its in-cluster identity) is untouched.

4Start it on the cluster

Turn the blueprint into running pods.

reconcile → renders the worker with its Vault identity

greentic-deployer-dev op --store-root $STORE env reconcile vault-demo

# wait for the worker
kubectl -n greentic rollout status deploy/<gtc-worker-…> --timeout=240s
Plain

Press go. Kubernetes starts the worker, clips on its gtc-worker ID badge, and tells it “your secrets live in Vault.” The worker boots and fetches its own keys.

Does

Renders and applies the Kubernetes resources (15 of them): the worker Deployment with serviceAccountName: gtc-worker, GREENTIC_SECRETS_BACKEND=vault and VAULT_ADDR — and no gtc-dev-secrets Secret.

Why

Apply (step 2) only updated the store. Reconcile is the part that actually converges the live cluster to match it.

5Verify — the real output

Captured from the live run on this machine, 2026-06-29.

CheckResult
A. No secret value in the clusternone ✓kubectl get secret → “No resources found”
B. Worker wears the Vault identity SA = gtc-worker, GREENTIC_SECRETS_BACKEND = vault, VAULT_K8S_ROLE = gtc-worker
C. Worker pulled the bundle & activated it revisions_active = 1
D. Right password accepted, wrong one rejectedHTTP 200 (correct token) · HTTP 401 in 1.8 ms (wrong token)
E. Vault's own log shows the worker reading its secretssee below ↓

Vault audit — exactly what the worker did (the proof)

1 update  auth/kubernetes/login                                   ← worker logs in with its SA badge
1 read    secret/.../messaging-01kw8wng…/webhook_secret           ← reads the webhook secret (named by endpoint id)
1 read    secret/.../messaging-telegram/telegram_bot_token        ← reads the bot token
2 update  transit/decrypt/greentic                                ← unwraps both values in memory

# (the create + transit/encrypt lines in the log are the operator's seed in step 3, not the worker)
Plain

Vault's logbook shows the worker walking up, showing its badge, opening only its own drawer, and unwrapping its two keys. Nobody pushed anything to it — it fetched its own secrets, and only the ones it's allowed to.

What this proves

A real worker in Kubernetes resolved both secret:// references from Vault using only its pod identity, authenticated an inbound webhook with a Vault-stored secret (HTTP 200), and the whole time the cluster held no secret values. That's the Phase E goal, end to end.

The blueprint file

manifest/env-manifest.json — what one op env apply replaces.

SectionReplaces these manual commands
environment + trust_rootenv create* + trust-root bootstrap
packs[] (secrets→vault, deployer→k8s)env-packs add × 2
bundles[] (webchat-bot, OCI ref)bundles add + revisions stage + warm + traffic set
messaging_endpoints[]messaging endpoint add + link-bundle

*Apply updates an existing named env but won't create one — that's why Step 1 runs env create first. The two answer files next to the manifest (secrets-vault-answers.json, deployer-k8s-answers.json) supply the Vault address/role and the Kubernetes namespace/image.

Greentic · deterministic digital-worker OS. Vault here runs in dev mode for the demo only — in production Vault is sealed, lives in its own namespace, and the same workload-identity flow applies unchanged.