kubernetes
control planeorchestration layer the agent stood up live; ready in ~6 seconds on a bare gpu host.
./flight-recorder
an agent built the infrastructure, shipped the pipeline, and let a panel that doesn't flatter score the work.
human-directed · agent-executedthis is the recorded flight of one autonomous build — a cluster, a pipeline, a chart, and a review it could not grade itself. every act below is timestamped and replayable. nothing was staged after the fact, including the part where the first draft scored 48 and then climbed.
recording — 6 acts, unedited- who
- one autonomous agent, unsupervised, no second take.
- did what
- provisioned the cluster, wrote the pipeline, authored the chart, then sat through a panel review it could not grade itself.
- expected
- a flight recording, not a highlight reel — every act timestamped and left exactly as found.
act i — infrastructure
a cluster came alive in under two minutes.
- who
- the orchestrating agent + the gpu-node cluster host.
- did what
- authored and ran the provisioning live; the control plane came ready in ~6s; the observability stack deployed as pods.
- expected
- a full kubernetes + monitoring stack in ~70 seconds, recorded end to end.
no scaffold, no template repo — the agent wrote the provisioning steps against a bare gpu host and ran them live. the control plane reported ready in roughly six seconds; prometheus and grafana followed as ordinary pods, not an afterthought bolted on later. the whole stack was standing, monitored, in about seventy seconds, start to finish, on tape.
rec_act-i.tape
provisioning log
$ agent provision --profile gpu-node --with observability › control plane ready 6.1s › node join 11.4s › prometheus + grafana up 41.8s › stack healthy 70.2s
act ii — ci/cd
a pipeline that runs before the runner exists.
- who
- a self-hosted ci runner on the orchestrator + the ci service.
- did what
- the agent wrote a reusable pipeline; the runner ran a linter, 25 offline tests, and a helm-chart lint gate.
- expected
- a commit that queues a build before the runner exists, then goes green in seconds.
ci run #114
queued build #114 (runner not yet online) › runner registered 2.0s › lint pass 3.1s › tests (25 offline) pass 9.4s › helm lint gate pass 1.8s ✓ green 16.3s
the pipeline definition was written before there was anything to run it — the commit queued a build against a runner that didn't exist yet. once the self-hosted runner came online it picked up the job on its own: a linter pass, twenty-five offline tests, and a helm-chart lint gate as the last word before anything ships. green in sixteen seconds.
rec_act-ii.tape
act iii — multi-agent review
three reviewers, one judge, zero flattery.
- who
- three frontier ai models as independent reviewers + a separate judge model, orchestrated by langgraph.
- did what
- each reviewer critiqued the same code in parallel; an aggregator tallied findings; the judge ruled with a 0–100 consensus score; a retired model was detected and skipped and a failed reviewer became a finding, not a crash.
- expected
- a scored, fault-tolerant consensus review that does not flatter its own input.
no reviewer sees another reviewer's notes — three models read the same diff cold, in parallel, and file independent findings. an aggregator merges the lists without softening them; a separate judge model rules on a 0–100 consensus score. the panel is built to survive its own reviewers failing: mid-run, one model came back retired, and the run kept going. that incident is its own section, below.
rec_act-iii.tape
act iv — the authored artifact
the panel scored the first draft 48, then watched it climb.
- who
- the agent as author + the review panel (trinity / the council).
- did what
- wrote a helm chart; the panel scored the first draft 48 then 68 after revision; the agent triaged findings like a maintainer — rejected one false blocker with an empirical test, accepted the real fixes including de-privileging the container.
- expected
- a reviewed, hardened, non-root deployment that passes ci.
the climb — consensus score, first draft to green
48first draft
68after acting on the review
fixedrest closed by hand
greenci passed
the point is the trajectory. the panel scored the owner's first draft 48/100 — a starting line, not a verdict. he acted on the findings, argued down the one that didn't hold up with an empirical test, and fixed the ones that did. the second pass rose to 68. what was left, including running the container as non-root, he closed by hand — and the pipeline went green. honest review didn't punish the chart; it drove the score up. the review works.
rec_act-iv.tape
act v — logs
and the logs completed the picture.
- who
- the orchestrating agent + the gpu-node's grafana.
- did what
- installed loki plus a per-node log shipper, then auto-wired it into the same grafana as the metrics and traces.
- expected
- logs, metrics and traces in one pane — the full lgtm stack.
metrics and traces were already flowing; the missing piece was logs. one helm release brought up loki with a collector tailing every pod, and a labelled config had grafana import the datasource on its own — no dashboard surgery. now a single grafana explores metrics, traces and logs together: the flight recorder has its full black box.
loki — logs complete lgtm
$ helm upgrade --install loki grafana/loki-stack -n logging --create-namespace › STATUS: deployed $ kubectl -n logging get pods › loki-0 1/1 Running › loki-promtail-... 1/1 Running $ kubectl apply -f loki-datasource.yaml # labelled grafana_datasource=1 › grafana sidecar → imported "loki" datasource › grafana explore · loki · {namespace="monitoring"} · live pod logs ✔ LGTM complete: loki · grafana · tempo · prometheus
act vi — identity
and identity closed the loop.
- who
- the agent + a keycloak identity provider on the cluster.
- did what
- deployed keycloak, created a realm and an oidc client, and issued a signed jwt via client-credentials.
- expected
- oauth2 / oidc / jwt the platform can actually enforce.
a platform that runs agents has to know who is calling. one deployment brought up keycloak as an openid-connect provider; a realm and a confidential client later, a service exchanged its credentials for an rs256-signed jwt — the same oauth2 / oidc / jwt flow that gates real api and tool access.
keycloak — oidc identity
$ kubectl apply -f keycloak.yaml # oidc identity provider › keycloak 1/1 Running · serving oidc discovery $ create realm "forge" + oidc client "forge-demo" › realm → 201 › client → 201 $ curl .../realms/forge/protocol/openid-connect/token # client_credentials › jwt issued — {"alg":"RS256","typ":"JWT"} › iss=.../realms/forge · azp=forge-demo · typ=Bearer ✔ oidc end-to-end: realm → client → signed jwt
act vii — gcp
and then the same, on someone else’s cloud.
the platform’s primitives don’t belong to one provider. to prove it, the whole pattern was re-provisioned on google cloud with terraform — a live cloud run service, cloud storage, artifact registry and a workload-identity service account — in a few hours, agent-driven.
▸ read the full GCP deep-diveincident log
when it broke.
a flight recorder is only honest if it keeps the parts that don't go well. this is the one failure on tape.
- who
- the aggregator + the judge model, mid-review.
- did what
- one reviewer came back retired and unreachable; the aggregator logged the failure as a finding instead of halting the run; the judge scored consensus on the two voices left standing.
- expected
- a panel that keeps its shape when a member drops out, instead of quietly going quiet.
panel run — error handling
› reviewer_b: model retired (410) › aggregator: logged as finding, run continues › reviewer_a: ok reviewer_c: ok › judge: scoring on 2/3 reviewers ✓ consensus score returned — no crash
./stack
the parts, named.
nothing here is a mock. every card below is a piece that actually ran, in the acts above.
helm
the authored chartwritten by the agent, scored 48 then 68 by the panel, hardened by hand after.
prometheus + grafana
observabilitydeployed as ordinary pods alongside the workload, not bolted on after launch.
self-hosted runner
ci computecame online after the build was already queued and picked the job up on its own.
pytest
test floor25 offline tests plus a helm-chart lint gate as the last word before anything ships.
langgraph
review orchestrationruns three reviewers in parallel and routes their findings to an aggregator and judge.
three frontier models
independent reviewersread the same diff cold, with no visibility into each other's notes.
judge model
consensus scoringrules on a 0–100 score from the aggregated findings — including when a reviewer fails.
non-root container
the real fixthe one finding that mattered most: de-privileging the runtime before ci went green.
./governance
the council — how the panel stays honest.
trinity is the review panel's structure, not its brand name: independence in, one accountable score out.
independent by construction
each reviewer sees the code, not the other reviewers. agreement isn't manufactured by a shared context window.
aggregation before judgment
findings are tallied first, then handed to a separate judge model — the same entity never both reports and rules.
fault-tolerant by design
a retired model is detected and skipped; a failed reviewer becomes a logged finding, never a crashed run.
the score is a floor, not a wall
48 wasn't a rejection — it was a starting line the owner was expected to climb from, and did, to 68 and then green.