feat: move distribution builds to platform Docker builder
This commit is contained in:
npc0-hue
+2
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-07-30
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
## Context
|
||||
|
||||
Two independently reasonable decisions currently deadlock the build path.
|
||||
|
||||
`platform/service/job_channel.go` strips `distribution.build` from any component-authenticated session, so a generated run cannot claim build work. `platform/service/distributions.go` derives `generate-run` availability from `svc.endpointSupports(endpoint, domain.JobCapabilityDistributionBuild)`, falling back to `run endpoint cannot build distributions`. An instance bound to its own generated run therefore fails the availability check permanently.
|
||||
|
||||
`platform/service/distributions.go` picks the builder endpoint as `instance.DeploymentTargetID` when set, otherwise `instance.RunEndpointID`. Both resolve to machine-side endpoints, so building depends on a hand-maintained privileged worker being registered and online.
|
||||
|
||||
`platform/service/distribution_build_jobs.go` decrypts the component key and returns `AuthKey` in `DistributionBuildInput`. Any endpoint claiming a build job receives that plaintext credential.
|
||||
|
||||
`platform/validator/server_lifecycle.go` does not require a deployment target; it only requires `profileKey` when `runEndpointId` is supplied. The creation-time requirement is imposed by `platform_web/components/ServerDeploymentWorkflow.tsx`.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
Goals: make build execution a platform responsibility with no dependency on machine-side endpoint state; reduce server creation to plugin type and server name; keep the generated-run build restriction as a security boundary; stop shipping plaintext auth keys to machine-side endpoints for builds.
|
||||
|
||||
Non-goals: changing the channel model for control/jobs/logs/artifacts; changing how a registered run executes game lifecycle work; adding billing, cloud host sales, or provider workflows; removing deployment target selection from post-creation instance management.
|
||||
|
||||
## Decisions
|
||||
|
||||
### Platform-owned Docker builder
|
||||
|
||||
The platform owns a builder that runs each distribution build in a container from a pinned image, with the run source snapshot mounted read-only and a per-job output directory mounted writable. Container-per-build keeps the existing plugin/job workspace isolation guarantee from `run-build-download-flow` and keeps the Go toolchain out of the platform runtime image.
|
||||
|
||||
Alternative considered: building in-process with the platform's own Go toolchain. Rejected because it makes the toolchain a hard platform deployment dependency and gives build code the platform process's filesystem and credential reach. The container boundary is what makes it safe to hold the auth key on the platform side.
|
||||
|
||||
Builder readiness is a platform-level probe, not a run endpoint capability. When the builder is unavailable, the unavailable reason names the builder so the operator is not sent looking at run endpoints.
|
||||
|
||||
### Availability derivation
|
||||
|
||||
`generate-run` and `generate-client-manager` availability becomes: plugin declares the capability, runtime bindings are complete, platform builder is ready. The `endpointSupports(..., JobCapabilityDistributionBuild)` term is removed from both actions. `bindingsComplete` and the plugin declaration checks stay as they are.
|
||||
|
||||
### Build job identity
|
||||
|
||||
Build jobs remain jobs with `distribution.build` capability so idempotency, artifact ownership (`ArtifactOwnerKindJob`), progress projection, and the `projectDistributionBuildResult` verification path are preserved unchanged. The change is who executes them: the platform builder claims and completes them internally instead of a machine-side endpoint claiming over the job channel. The existing artifact-scope assertions in `validateDistributionBuildResult` continue to guard the result.
|
||||
|
||||
The capability-stripping guard in `job_channel.go` stays. With platform-side execution it becomes redundant for correctness but remains as defense in depth: a machine-side endpoint must never be assignable build work even if a future dispatch path regresses.
|
||||
|
||||
### Secret handling
|
||||
|
||||
`GetDistributionBuildInput` remains for legacy machine-side flows already in the field, but platform-executed builds resolve the component key internally and never place it in a job-channel response. The key reaches the builder container through the per-job input file rather than an API response, so it is never transmitted to a machine-side endpoint.
|
||||
|
||||
### Creation form
|
||||
|
||||
The deployment target selector is removed from the create branch of `ServerDeploymentWorkflow.tsx` rather than made optional. Leaving an optional selector preserves the original defect: the listed endpoints are still wrong choices at creation time. The run endpoint selector on the non-create branch is unaffected. The `saveAsDraft` special case for the create branch loses its reason to exist for target selection and is simplified accordingly.
|
||||
|
||||
Backend validation already permits this, so no relaxation is needed there. `deploymentTargetId` remains accepted by the create DTO for post-creation and programmatic flows.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
Docker becomes a platform deployment dependency for building. Mitigation: builder readiness is probed and surfaced as an explicit unavailable reason, so a platform without Docker degrades to "cannot build" with a clear cause rather than a misleading endpoint capability message.
|
||||
|
||||
Existing instances carry `DeploymentTargetID` values pointing at privileged workers. Those bindings stay valid for non-build work; only build routing stops consulting them.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
Availability derivation and platform-side execution land together, since changing availability alone would surface an action that cannot execute. The creation-form change is independent and can land in the same change without ordering constraints.
|
||||
|
||||
## Open Questions
|
||||
|
||||
Whether the builder image is built from this repository or pinned from a registry is left to implementation, provided the image reference is pinned rather than floating.
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
## Why
|
||||
|
||||
Creating a server instance currently forces the owner to pick a deployment target before any run executor exists. The target dropdown lists already-registered run endpoints, but the intended flow is create server → platform builds run → operator executes run on the machine → run registers back. At creation time there is nothing correct to select, so the field can only be filled with an unrelated endpoint or bypassed with the draft checkbox.
|
||||
|
||||
Distribution building is also routed through machine-side run endpoints, while a generated run is intentionally stripped of `distribution.build` authority. Both restrictions are individually sound, but together they mean an instance bound to its own generated run can never build again: `generate-run` reports `run endpoint cannot build distributions`. Building only works when a separately maintained privileged worker endpoint happens to be registered and online, which makes the platform's core build path depend on hand-maintained machine state.
|
||||
|
||||
The current build dispatch additionally hands the plaintext component `authKey` to whichever endpoint claims the build job, so a privileged worker accumulates credentials for every server it has ever built. Moving builds into a platform-owned Docker builder removes that credential egress path instead of widening it.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Remove deployment target selection from the server creation form. Creation requires only game plugin type and server name; the run endpoint binding is established when the generated run registers itself.
|
||||
- Keep deployment target and runtime profile selection available as optional actions on an already-created instance, never as creation prerequisites.
|
||||
- Move `distribution.build` execution into a platform-owned Docker builder. The platform builds run and client-manager packages itself and no longer dispatches build jobs to machine-side run endpoints.
|
||||
- Keep the generated-run build restriction intact as a security boundary; `generate-run` availability must no longer depend on any run endpoint advertising `distribution.build`.
|
||||
- Stop exposing plaintext component auth keys over the job channel for builds executed by the platform builder.
|
||||
- Add tests proving an instance bound only to its own generated run can still generate a new run distribution.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `platform-side-distribution-builds`: Covers platform-owned Docker build execution, creation-time field requirements, and run-endpoint-independent build availability.
|
||||
|
||||
### Modified Capabilities
|
||||
- `run-distribution-and-client-managers`: Build execution moves from machine-side run endpoints to the platform Docker builder; generated-run build restriction is preserved.
|
||||
- `run-build-download-flow`: Build source snapshotting and artifact download must work without a privileged worker endpoint.
|
||||
|
||||
## Impact
|
||||
|
||||
- Affected roots: `platform/`, `platform_web/`, `scripts/`.
|
||||
- Affected behavior: server creation validation, `generate-run` and `generate-client-manager` availability, job channel build dispatch, build input secret exposure.
|
||||
- Verification requires `scripts/check-structure.sh`, platform tests, frontend tests, OpenSpec strict validation, and a local proof that run generation succeeds on an instance whose only endpoint is its own generated run.
|
||||
+68
@@ -0,0 +1,68 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Server creation requires only plugin type and server name
|
||||
The system SHALL require only the game plugin type and the server name to create a server instance, and SHALL NOT require a deployment target, run endpoint, or runtime profile at creation time.
|
||||
|
||||
#### Scenario: Creation form field set
|
||||
- **WHEN** an owner opens the server creation workflow
|
||||
- **THEN** the form requires plugin type and server name only, and presents no deployment target or run endpoint selector as a creation prerequisite
|
||||
|
||||
#### Scenario: Creation without any registered endpoint
|
||||
- **WHEN** an owner creates a server instance while no run endpoint is registered for that instance
|
||||
- **THEN** creation succeeds and the instance is created without a deployment target binding
|
||||
|
||||
#### Scenario: Binding established by run registration
|
||||
- **WHEN** a generated run for that instance registers itself with the platform
|
||||
- **THEN** the platform binds the instance to that run endpoint without the owner having pre-selected it
|
||||
|
||||
#### Scenario: Target selection remains available after creation
|
||||
- **WHEN** an owner opens an already-created instance
|
||||
- **THEN** deployment target and runtime profile selection remain available as optional actions on that instance
|
||||
|
||||
### Requirement: Distribution builds execute in a platform-owned Docker builder
|
||||
The platform SHALL execute `distribution.build` work in a platform-owned Docker builder and SHALL NOT dispatch distribution build jobs to machine-side run endpoints.
|
||||
|
||||
#### Scenario: Run distribution build execution
|
||||
- **WHEN** an owner requests run generation for a server instance
|
||||
- **THEN** the platform builds the package in its own Docker builder and records the resulting artifact against the build job
|
||||
|
||||
#### Scenario: Client-manager distribution build execution
|
||||
- **WHEN** an owner requests client-manager generation for a server instance
|
||||
- **THEN** the platform builds the package in its own Docker builder and records the resulting artifact against the build job
|
||||
|
||||
#### Scenario: Build failure reporting
|
||||
- **WHEN** a platform Docker build fails
|
||||
- **THEN** the distribution status becomes failed, the build job reports a failure, and the failure reason excludes host paths and secret values
|
||||
|
||||
### Requirement: Build availability is independent of run endpoint capabilities
|
||||
The system SHALL determine `generate-run` and `generate-client-manager` availability from plugin declarations, runtime bindings, and platform builder readiness, and SHALL NOT require any run endpoint to advertise `distribution.build`.
|
||||
|
||||
#### Scenario: Instance bound only to its own generated run
|
||||
- **WHEN** a server instance's only run endpoint is its own generated run, which holds no distribution-build authority
|
||||
- **THEN** `generate-run` remains available and a new run distribution can be generated
|
||||
|
||||
#### Scenario: No privileged worker endpoint registered
|
||||
- **WHEN** no run endpoint advertising `distribution.build` is registered or online
|
||||
- **THEN** run generation still succeeds through the platform Docker builder
|
||||
|
||||
#### Scenario: Builder unavailable
|
||||
- **WHEN** the platform Docker builder is unavailable
|
||||
- **THEN** the unavailable reason names the platform builder rather than a run endpoint capability
|
||||
|
||||
### Requirement: Generated runs hold no distribution-build authority
|
||||
The system SHALL continue to deny distribution-build work to component-authenticated generated runs. This restriction is a security boundary and SHALL NOT be relaxed to unblock building.
|
||||
|
||||
#### Scenario: Generated run claims a build
|
||||
- **WHEN** a component-authenticated generated run claims work advertising `distribution.build`
|
||||
- **THEN** the platform does not assign distribution build work to that run
|
||||
|
||||
### Requirement: Platform builds do not expose plaintext component auth keys over the job channel
|
||||
The system SHALL keep component auth keys inside the platform when builds are executed by the platform Docker builder, and SHALL NOT return plaintext auth keys to machine-side run endpoints for distribution builds.
|
||||
|
||||
#### Scenario: Build input secret handling
|
||||
- **WHEN** the platform builder assembles a package requiring a component auth key
|
||||
- **THEN** the key is resolved inside the platform and is not transmitted to any machine-side run endpoint
|
||||
|
||||
#### Scenario: Generated package still authenticates
|
||||
- **WHEN** a package built by the platform builder registers with the platform
|
||||
- **THEN** its embedded credential and key generation are accepted as before
|
||||
@@ -0,0 +1,40 @@
|
||||
## 1. Server creation form
|
||||
|
||||
- [x] 1.1 Remove the deployment target selector from the create branch of `platform_web/components/ServerDeploymentWorkflow.tsx`, keeping the run endpoint selector on the non-create branch unchanged.
|
||||
- [x] 1.2 Simplify the create-branch step gating so it no longer depends on `deploymentTargetId` or on `saveAsDraft` for target selection.
|
||||
- [x] 1.3 Keep `deploymentTargetId` accepted in `platform_web/schemas/serverManagement.ts` and the create DTO for post-creation and programmatic flows.
|
||||
- [x] 1.4 Update or add frontend tests proving creation submits with plugin type and server name only.
|
||||
|
||||
## 2. Platform Docker builder
|
||||
|
||||
- [x] 2.1 Add a platform-owned builder that executes a distribution build in a container from a pinned image, with run source mounted read-only and a per-job output directory mounted writable.
|
||||
- [x] 2.2 Add a builder readiness probe and expose its unavailable reason as a platform-builder reason, not a run endpoint capability reason.
|
||||
- [x] 2.3 Route `distribution.build` job execution to the platform builder so the job is claimed and completed internally instead of over the job channel.
|
||||
- [x] 2.4 Preserve job idempotency, `ArtifactOwnerKindJob` artifact ownership, progress projection, and the existing `validateDistributionBuildResult` artifact-scope assertions.
|
||||
- [x] 2.5 Keep build workspaces isolated per plugin and per job as required by `run-build-download-flow`.
|
||||
|
||||
## 3. Build availability derivation
|
||||
|
||||
- [x] 3.1 Remove the `endpointSupports(..., JobCapabilityDistributionBuild)` term from `generate-run` and `generate-client-manager` availability in `platform/service/distributions.go`.
|
||||
- [x] 3.2 Derive availability from plugin declaration, complete runtime bindings, and builder readiness, keeping existing binding reasons intact.
|
||||
- [x] 3.3 Stop resolving a machine-side builder endpoint for build dispatch in `GenerateRunDistribution` and the client-manager build path.
|
||||
|
||||
## 4. Secret handling
|
||||
|
||||
- [x] 4.1 Resolve component auth keys inside the platform for builder-executed builds and pass them to the container through the per-job input rather than a job-channel response.
|
||||
- [x] 4.2 Keep the `distribution.build` capability-stripping guard in `platform/service/job_channel.go` as defense in depth.
|
||||
- [x] 4.3 Add a test proving builder-executed builds do not return a plaintext auth key to a machine-side endpoint.
|
||||
|
||||
## 5. Tests and verification
|
||||
|
||||
- [x] 5.1 Add a platform test proving an instance whose only endpoint is its own generated run can generate a new run distribution.
|
||||
- [x] 5.2 Add a platform test proving run generation succeeds with no endpoint advertising `distribution.build` registered or online.
|
||||
- [x] 5.3 Keep `TestCoreServiceComponentRunCannotClaimDistributionBuild` passing.
|
||||
- [x] 5.4 Add a builder-unavailable test proving the reason names the platform builder.
|
||||
- [x] 5.5 Run `scripts/check-structure.sh`, platform tests, frontend tests, and `openspec validate platform-side-docker-distribution-builds --strict`.
|
||||
- [x] 5.6 Prove the flow end to end in local debug: create a server with plugin type and name only, generate a run, download and execute it, confirm registration and heartbeat.
|
||||
|
||||
## 6. Documentation
|
||||
|
||||
- [x] 6.1 Add server creation field rules and platform-side build ownership rules to `AGENTS.md`.
|
||||
- [x] 6.2 Document builder configuration values operators must provide or may tune.
|
||||
@@ -0,0 +1,71 @@
|
||||
# platform-side-distribution-builds Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change platform-side-docker-distribution-builds. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Server creation requires only plugin type and server name
|
||||
The system SHALL require only the game plugin type and the server name to create a server instance, and SHALL NOT require a deployment target, run endpoint, or runtime profile at creation time.
|
||||
|
||||
#### Scenario: Creation form field set
|
||||
- **WHEN** an owner opens the server creation workflow
|
||||
- **THEN** the form requires plugin type and server name only, and presents no deployment target or run endpoint selector as a creation prerequisite
|
||||
|
||||
#### Scenario: Creation without any registered endpoint
|
||||
- **WHEN** an owner creates a server instance while no run endpoint is registered for that instance
|
||||
- **THEN** creation succeeds and the instance is created without a deployment target binding
|
||||
|
||||
#### Scenario: Binding established by run registration
|
||||
- **WHEN** a generated run for that instance registers itself with the platform
|
||||
- **THEN** the platform binds the instance to that run endpoint without the owner having pre-selected it
|
||||
|
||||
#### Scenario: Target selection remains available after creation
|
||||
- **WHEN** an owner opens an already-created instance
|
||||
- **THEN** deployment target and runtime profile selection remain available as optional actions on that instance
|
||||
|
||||
### Requirement: Distribution builds execute in a platform-owned Docker builder
|
||||
The platform SHALL execute `distribution.build` work in a platform-owned Docker builder and SHALL NOT dispatch distribution build jobs to machine-side run endpoints.
|
||||
|
||||
#### Scenario: Run distribution build execution
|
||||
- **WHEN** an owner requests run generation for a server instance
|
||||
- **THEN** the platform builds the package in its own Docker builder and records the resulting artifact against the build job
|
||||
|
||||
#### Scenario: Client-manager distribution build execution
|
||||
- **WHEN** an owner requests client-manager generation for a server instance
|
||||
- **THEN** the platform builds the package in its own Docker builder and records the resulting artifact against the build job
|
||||
|
||||
#### Scenario: Build failure reporting
|
||||
- **WHEN** a platform Docker build fails
|
||||
- **THEN** the distribution status becomes failed, the build job reports a failure, and the failure reason excludes host paths and secret values
|
||||
|
||||
### Requirement: Build availability is independent of run endpoint capabilities
|
||||
The system SHALL determine `generate-run` and `generate-client-manager` availability from plugin declarations, runtime bindings, and platform builder readiness, and SHALL NOT require any run endpoint to advertise `distribution.build`.
|
||||
|
||||
#### Scenario: Instance bound only to its own generated run
|
||||
- **WHEN** a server instance's only run endpoint is its own generated run, which holds no distribution-build authority
|
||||
- **THEN** `generate-run` remains available and a new run distribution can be generated
|
||||
|
||||
#### Scenario: No privileged worker endpoint registered
|
||||
- **WHEN** no run endpoint advertising `distribution.build` is registered or online
|
||||
- **THEN** run generation still succeeds through the platform Docker builder
|
||||
|
||||
#### Scenario: Builder unavailable
|
||||
- **WHEN** the platform Docker builder is unavailable
|
||||
- **THEN** the unavailable reason names the platform builder rather than a run endpoint capability
|
||||
|
||||
### Requirement: Generated runs hold no distribution-build authority
|
||||
The system SHALL continue to deny distribution-build work to component-authenticated generated runs. This restriction is a security boundary and SHALL NOT be relaxed to unblock building.
|
||||
|
||||
#### Scenario: Generated run claims a build
|
||||
- **WHEN** a component-authenticated generated run claims work advertising `distribution.build`
|
||||
- **THEN** the platform does not assign distribution build work to that run
|
||||
|
||||
### Requirement: Platform builds do not expose plaintext component auth keys over the job channel
|
||||
The system SHALL keep component auth keys inside the platform when builds are executed by the platform Docker builder, and SHALL NOT return plaintext auth keys to machine-side run endpoints for distribution builds.
|
||||
|
||||
#### Scenario: Build input secret handling
|
||||
- **WHEN** the platform builder assembles a package requiring a component auth key
|
||||
- **THEN** the key is resolved inside the platform and is not transmitted to any machine-side run endpoint
|
||||
|
||||
#### Scenario: Generated package still authenticates
|
||||
- **WHEN** a package built by the platform builder registers with the platform
|
||||
- **THEN** its embedded credential and key generation are accepted as before
|
||||
Reference in New Issue
Block a user