fix: refresh run source during distribution builds
This commit is contained in:
@@ -6,7 +6,7 @@ This repository is the game server management platform workspace. It replaces th
|
|||||||
- `platform_web/`: management console frontend.
|
- `platform_web/`: management console frontend.
|
||||||
- `plugins/`: game management plugin workspace. A plugin defines how to create and manage one server type, and one installed plugin can create many server instances.
|
- `plugins/`: game management plugin workspace. A plugin defines how to create and manage one server type, and one installed plugin can create many server instances.
|
||||||
|
|
||||||
The machine-side executor source lives in the separate `git@git.npc0.com:admin343/run.git` repository. Local debug and Docker workflows can keep the editable source at `./run` as an ignored nested checkout, or use another path through `RUN_SOURCE_DIR` / legacy `RUN_REPO_DIR`. Local debug snapshots that source into `.local-debug` before building or starting run.
|
The machine-side executor source lives in the separate `git@git.npc0.com:admin343/run.git` repository. Local debug and Docker workflows can keep the editable source at `./run` as an ignored nested checkout, or use another path through `RUN_SOURCE_DIR` / legacy `RUN_REPO_DIR`. Run distributions refresh the configured repository revision during each platform build, then inject package configuration and compile in the isolated builder workspace.
|
||||||
|
|
||||||
## Product Scope
|
## Product Scope
|
||||||
|
|
||||||
@@ -75,7 +75,6 @@ Run executor checks are owned by the separate run checkout:
|
|||||||
Run the API-backed local debug workspace when you need platform, run, platform_web, and the dev plugin fixture together:
|
Run the API-backed local debug workspace when you need platform, run, platform_web, and the dev plugin fixture together:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone git@git.npc0.com:admin343/run.git run # once, if the ignored local checkout is missing
|
|
||||||
scripts/dev-start.sh
|
scripts/dev-start.sh
|
||||||
scripts/dev-smoke.sh
|
scripts/dev-smoke.sh
|
||||||
```
|
```
|
||||||
@@ -83,7 +82,7 @@ scripts/dev-smoke.sh
|
|||||||
See `docs/local-debug-workspace.md` for ports, disposable data roots, log files, reset steps, smoke evidence, and the required browser walkthrough. This workflow treats frontend local auth fallback as a verification failure.
|
See `docs/local-debug-workspace.md` for ports, disposable data roots, log files, reset steps, smoke evidence, and the required browser walkthrough. This workflow treats frontend local auth fallback as a verification failure.
|
||||||
Use `LOCAL_DEBUG_SELF_START=true scripts/dev-smoke.sh` when you need the smoke command to own the temporary local stack for the duration of the verification.
|
Use `LOCAL_DEBUG_SELF_START=true scripts/dev-smoke.sh` when you need the smoke command to own the temporary local stack for the duration of the verification.
|
||||||
|
|
||||||
For local debug, prefer the managed scripts because they snapshot run source into the closed `.local-debug` build bucket and start the bootstrap worker from the built binary:
|
For local debug, prefer the managed scripts because they prepare the Run checkout, start the platform-owned builder, and start the bootstrap worker from the current source:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
scripts/dev-start.sh
|
scripts/dev-start.sh
|
||||||
@@ -94,7 +93,6 @@ scripts/dev-start.sh
|
|||||||
Use the root compose file for a local all-in-one deployment:
|
Use the root compose file for a local all-in-one deployment:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone git@git.npc0.com:admin343/run.git run # once, if the ignored local checkout is missing
|
|
||||||
docker compose up --build
|
docker compose up --build
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -166,7 +164,10 @@ Most common edits:
|
|||||||
- Log body persistence: `PLATFORM_LOG_BODY_BACKEND=file`, `PLATFORM_LOG_DIR`.
|
- Log body persistence: `PLATFORM_LOG_BODY_BACKEND=file`, `PLATFORM_LOG_DIR`.
|
||||||
- Run editable source checkout: `RUN_SOURCE_DIR=./run` by default; legacy `RUN_REPO_DIR` is accepted as an alias; `run/` is ignored by the browser repository.
|
- Run editable source checkout: `RUN_SOURCE_DIR=./run` by default; legacy `RUN_REPO_DIR` is accepted as an alias; `run/` is ignored by the browser repository.
|
||||||
- Run closed build bucket: `RUN_BUILD_BUCKET_ROOT=.local-debug/run/build-buckets`.
|
- Run closed build bucket: `RUN_BUILD_BUCKET_ROOT=.local-debug/run/build-buckets`.
|
||||||
- Run build source snapshot: `RUN_BUILD_SOURCE_ROOT=.local-debug/run/build-buckets/source/current`.
|
- Run source repository: `RUN_SOURCE_REPOSITORY=git@git.npc0.com:admin343/run.git`.
|
||||||
|
- Run source revision: `RUN_SOURCE_REVISION=main`.
|
||||||
|
- Platform builder source repository: `PLATFORM_BUILDER_SOURCE_REPOSITORY` (defaults to the Run repository).
|
||||||
|
- Platform builder source revision: `PLATFORM_BUILDER_SOURCE_REVISION` (defaults to `main`).
|
||||||
- Run bootstrap binary: `RUN_BOOTSTRAP_BIN=.local-debug/run/build-buckets/bootstrap/bin/run`.
|
- Run bootstrap binary: `RUN_BOOTSTRAP_BIN=.local-debug/run/build-buckets/bootstrap/bin/run`.
|
||||||
- Run worker mode: `RUN_MODE=worker`.
|
- Run worker mode: `RUN_MODE=worker`.
|
||||||
- Run-to-platform URL: `RUN_PLATFORM_URL=http://127.0.0.1:8080` locally, `http://platform:8080` in Docker.
|
- Run-to-platform URL: `RUN_PLATFORM_URL=http://127.0.0.1:8080` locally, `http://platform:8080` in Docker.
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ The local debug workspace runs the real platform API, run worker, platform_web c
|
|||||||
- Platform listens on `http://127.0.0.1:18080` by default.
|
- Platform listens on `http://127.0.0.1:18080` by default.
|
||||||
- platform_web listens on `http://127.0.0.1:5173` by default and proxies `/api/v1` plus `/healthz` to platform.
|
- platform_web listens on `http://127.0.0.1:5173` by default and proxies `/api/v1` plus `/healthz` to platform.
|
||||||
- Run editable source is loaded from `RUN_SOURCE_DIR` / legacy `RUN_REPO_DIR`, defaulting to the ignored nested `./run` checkout.
|
- Run editable source is loaded from `RUN_SOURCE_DIR` / legacy `RUN_REPO_DIR`, defaulting to the ignored nested `./run` checkout.
|
||||||
- Run source is snapshotted into `.local-debug/run/build-buckets/source/current`; the platform mounts that snapshot read-only into the dedicated `browser-platform-distribution-builder:1.0.0` image, and the local bootstrap worker is built into `.local-debug/run/build-buckets/bootstrap/bin/run` before registering as `run-local-debug`.
|
- Run source is kept as an independent checkout. The platform-owned builder refreshes the configured repository revision for every Run build, exports a clean per-job source workspace, injects package configuration, and compiles before publishing the artifact. The local bootstrap worker is built from the checkout before registering as `run-local-debug`.
|
||||||
- Disposable state lives under `.local-debug/`.
|
- Disposable state lives under `.local-debug/`.
|
||||||
- Logs live under `.local-debug/logs/`.
|
- Logs live under `.local-debug/logs/`.
|
||||||
- PIDs live under `.local-debug/pids/`.
|
- PIDs live under `.local-debug/pids/`.
|
||||||
@@ -30,7 +30,6 @@ Use the managed local debug scripts for browser-facing verification. Do not star
|
|||||||
## Start
|
## Start
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone git@git.npc0.com:admin343/run.git run # once, if the ignored local checkout is missing
|
|
||||||
scripts/dev-start.sh
|
scripts/dev-start.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -59,7 +58,9 @@ Key platform variables:
|
|||||||
- `PLATFORM_LOG_DIR=.local-debug/platform/logs`
|
- `PLATFORM_LOG_DIR=.local-debug/platform/logs`
|
||||||
- `PLATFORM_BUILDER_DOCKER_BINARY=docker`
|
- `PLATFORM_BUILDER_DOCKER_BINARY=docker`
|
||||||
- `PLATFORM_BUILDER_IMAGE=browser-platform-distribution-builder:1.0.0`
|
- `PLATFORM_BUILDER_IMAGE=browser-platform-distribution-builder:1.0.0`
|
||||||
- `PLATFORM_BUILDER_SOURCE_DIR=.local-debug/run/build-buckets/source/current`
|
- `PLATFORM_BUILDER_SOURCE_DIR=./run`
|
||||||
|
- `PLATFORM_BUILDER_SOURCE_REPOSITORY=git@git.npc0.com:admin343/run.git`
|
||||||
|
- `PLATFORM_BUILDER_SOURCE_REVISION=main`
|
||||||
- `PLATFORM_BUILDER_WORKSPACE_DIR=.local-debug/platform/distribution-builds`
|
- `PLATFORM_BUILDER_WORKSPACE_DIR=.local-debug/platform/distribution-builds`
|
||||||
- `PLATFORM_BUILDER_CACHE_DIR=.local-debug/platform/distribution-build-cache`
|
- `PLATFORM_BUILDER_CACHE_DIR=.local-debug/platform/distribution-build-cache`
|
||||||
- `PLATFORM_BUILDER_TIMEOUT_SECONDS=1800`
|
- `PLATFORM_BUILDER_TIMEOUT_SECONDS=1800`
|
||||||
@@ -70,7 +71,8 @@ Key run variables:
|
|||||||
- `RUN_SOURCE_DIR=./run`
|
- `RUN_SOURCE_DIR=./run`
|
||||||
- `RUN_REPO_DIR=./run` as a legacy alias for `RUN_SOURCE_DIR`
|
- `RUN_REPO_DIR=./run` as a legacy alias for `RUN_SOURCE_DIR`
|
||||||
- `RUN_BUILD_BUCKET_ROOT=.local-debug/run/build-buckets`
|
- `RUN_BUILD_BUCKET_ROOT=.local-debug/run/build-buckets`
|
||||||
- `RUN_BUILD_SOURCE_ROOT=.local-debug/run/build-buckets/source/current`
|
- `RUN_SOURCE_REPOSITORY=git@git.npc0.com:admin343/run.git`
|
||||||
|
- `RUN_SOURCE_REVISION=main`
|
||||||
- `RUN_BOOTSTRAP_BIN=.local-debug/run/build-buckets/bootstrap/bin/run`
|
- `RUN_BOOTSTRAP_BIN=.local-debug/run/build-buckets/bootstrap/bin/run`
|
||||||
- `RUN_MODE=worker`
|
- `RUN_MODE=worker`
|
||||||
- `RUN_PLATFORM_URL=http://127.0.0.1:18080`
|
- `RUN_PLATFORM_URL=http://127.0.0.1:18080`
|
||||||
|
|||||||
@@ -33,6 +33,8 @@ PLATFORM_ARTIFACT_DIR=.platform-data/artifacts
|
|||||||
PLATFORM_BUILDER_DOCKER_BINARY=docker
|
PLATFORM_BUILDER_DOCKER_BINARY=docker
|
||||||
PLATFORM_BUILDER_IMAGE=browser-platform-distribution-builder:1.0.0
|
PLATFORM_BUILDER_IMAGE=browser-platform-distribution-builder:1.0.0
|
||||||
PLATFORM_BUILDER_SOURCE_DIR=../run
|
PLATFORM_BUILDER_SOURCE_DIR=../run
|
||||||
|
PLATFORM_BUILDER_SOURCE_REPOSITORY=git@git.npc0.com:admin343/run.git
|
||||||
|
PLATFORM_BUILDER_SOURCE_REVISION=main
|
||||||
PLATFORM_BUILDER_WORKSPACE_DIR=.platform-data/distribution-builds
|
PLATFORM_BUILDER_WORKSPACE_DIR=.platform-data/distribution-builds
|
||||||
PLATFORM_BUILDER_TIMEOUT_SECONDS=1800
|
PLATFORM_BUILDER_TIMEOUT_SECONDS=1800
|
||||||
# URL embedded into generated Run and client-manager packages.
|
# URL embedded into generated Run and client-manager packages.
|
||||||
|
|||||||
+7
-3
@@ -56,7 +56,9 @@ Runtime configuration:
|
|||||||
- `PLATFORM_SECRET_ENVELOPE_KEY`: external secret used to derive the AES-GCM component-key envelope key; use at least 32 random characters and keep it stable across restarts.
|
- `PLATFORM_SECRET_ENVELOPE_KEY`: external secret used to derive the AES-GCM component-key envelope key; use at least 32 random characters and keep it stable across restarts.
|
||||||
- `PLATFORM_BUILDER_DOCKER_BINARY`: Docker-compatible CLI used by the platform builder, default `docker`.
|
- `PLATFORM_BUILDER_DOCKER_BINARY`: Docker-compatible CLI used by the platform builder, default `docker`.
|
||||||
- `PLATFORM_BUILDER_IMAGE`: prebuilt, explicitly versioned or digest-pinned builder image, default `browser-platform-distribution-builder:1.0.0`; floating tags such as `latest` are rejected.
|
- `PLATFORM_BUILDER_IMAGE`: prebuilt, explicitly versioned or digest-pinned builder image, default `browser-platform-distribution-builder:1.0.0`; floating tags such as `latest` are rejected.
|
||||||
- `PLATFORM_BUILDER_SOURCE_DIR`: read-only Run source snapshot containing `go.mod`; this is required for builder readiness.
|
- `PLATFORM_BUILDER_SOURCE_DIR`: persistent Run checkout location used by the platform builder. The checkout may be absent at startup; each Run build creates it when needed, fetches `PLATFORM_BUILDER_SOURCE_REVISION`, exports that revision into the isolated job workspace, injects package configuration, and then compiles.
|
||||||
|
- `PLATFORM_BUILDER_SOURCE_REPOSITORY`: Run Git repository used when the checkout is absent or needs a build-time refresh.
|
||||||
|
- `PLATFORM_BUILDER_SOURCE_REVISION`: branch, tag, or commit fetched for each Run build; defaults to `main`.
|
||||||
- `PLATFORM_BUILDER_WORKSPACE_DIR`: private per-plugin/per-job build workspace, default `<PLATFORM_DATA_DIR>/distribution-builds`.
|
- `PLATFORM_BUILDER_WORKSPACE_DIR`: private per-plugin/per-job build workspace, default `<PLATFORM_DATA_DIR>/distribution-builds`.
|
||||||
- `PLATFORM_BUILDER_CACHE_DIR`: persistent Go build/module cache, default `<PLATFORM_DATA_DIR>/distribution-build-cache`; it contains no job inputs or component keys.
|
- `PLATFORM_BUILDER_CACHE_DIR`: persistent Go build/module cache, default `<PLATFORM_DATA_DIR>/distribution-build-cache`; it contains no job inputs or component keys.
|
||||||
- `PLATFORM_BUILDER_TIMEOUT_SECONDS`: positive build deadline, default `1800`.
|
- `PLATFORM_BUILDER_TIMEOUT_SECONDS`: positive build deadline, default `1800`.
|
||||||
@@ -66,12 +68,14 @@ Build the dedicated toolchain image before enabling distribution generation:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker build --pull -t browser-platform-distribution-builder:1.0.0 distribution-builder
|
docker build --pull -t browser-platform-distribution-builder:1.0.0 distribution-builder
|
||||||
export PLATFORM_BUILDER_SOURCE_DIR=/absolute/path/to/read-only/run-source-snapshot
|
export PLATFORM_BUILDER_SOURCE_DIR=/absolute/path/to/run-checkout
|
||||||
|
export PLATFORM_BUILDER_SOURCE_REPOSITORY=git@git.npc0.com:admin343/run.git
|
||||||
|
export PLATFORM_BUILDER_SOURCE_REVISION=main
|
||||||
export PLATFORM_BUILDER_IMAGE=browser-platform-distribution-builder:1.0.0
|
export PLATFORM_BUILDER_IMAGE=browser-platform-distribution-builder:1.0.0
|
||||||
go run ./cmd/platform
|
go run ./cmd/platform
|
||||||
```
|
```
|
||||||
|
|
||||||
Each build runs in a separate read-only container. The platform mounts source and per-job input read-only, mounts only the job build/output directories writable, and passes the component auth key through a mode-`0600` input file. The key is not sent through the machine-side job channel or Docker arguments. Production deployments may use an internal-registry `image@sha256:...` reference; the selected image must already exist in the Docker daemon because builds run with `--pull never`.
|
Each build runs in a separate read-only container. For Run, the platform first exports the fetched revision into a clean per-job source directory; source and per-job input are mounted read-only, only the job build/output directories are writable, and the component auth key is passed through a mode-`0600` input file. The key is not sent through the machine-side job channel or Docker arguments. Production deployments may use an internal-registry `image@sha256:...` reference; the selected image must already exist in the Docker daemon because builds run with `--pull never`.
|
||||||
|
|
||||||
MySQL configuration example:
|
MySQL configuration example:
|
||||||
|
|
||||||
|
|||||||
@@ -52,6 +52,8 @@ func NewRouterFromConfig(cfg config.Config) (http.Handler, error) {
|
|||||||
DockerBinary: cfg.BuilderDockerBinary,
|
DockerBinary: cfg.BuilderDockerBinary,
|
||||||
Image: cfg.BuilderImage,
|
Image: cfg.BuilderImage,
|
||||||
SourceDir: cfg.BuilderSourceDir,
|
SourceDir: cfg.BuilderSourceDir,
|
||||||
|
SourceRepositoryURL: cfg.BuilderSourceRepository,
|
||||||
|
SourceRevision: cfg.BuilderSourceRevision,
|
||||||
WorkspaceDir: cfg.BuilderWorkspaceDir,
|
WorkspaceDir: cfg.BuilderWorkspaceDir,
|
||||||
CacheDir: cfg.BuilderCacheDir,
|
CacheDir: cfg.BuilderCacheDir,
|
||||||
Timeout: time.Duration(cfg.BuilderTimeoutSeconds) * time.Second,
|
Timeout: time.Duration(cfg.BuilderTimeoutSeconds) * time.Second,
|
||||||
|
|||||||
@@ -12,6 +12,8 @@ const defaultAddr = ":8080"
|
|||||||
const defaultDataDir = ".platform-data"
|
const defaultDataDir = ".platform-data"
|
||||||
const defaultStorageBackend = "file"
|
const defaultStorageBackend = "file"
|
||||||
const defaultBuilderDockerBinary = "docker"
|
const defaultBuilderDockerBinary = "docker"
|
||||||
|
const defaultBuilderSourceRepository = "git@git.npc0.com:admin343/run.git"
|
||||||
|
const defaultBuilderSourceRevision = "main"
|
||||||
|
|
||||||
// defaultBuilderImage names an explicit toolchain version rather than a
|
// defaultBuilderImage names an explicit toolchain version rather than a
|
||||||
// floating tag such as latest. Operators who want digest pinning override
|
// floating tag such as latest. Operators who want digest pinning override
|
||||||
@@ -35,6 +37,8 @@ type Config struct {
|
|||||||
BuilderDockerBinary string
|
BuilderDockerBinary string
|
||||||
BuilderImage string
|
BuilderImage string
|
||||||
BuilderSourceDir string
|
BuilderSourceDir string
|
||||||
|
BuilderSourceRepository string
|
||||||
|
BuilderSourceRevision string
|
||||||
BuilderWorkspaceDir string
|
BuilderWorkspaceDir string
|
||||||
BuilderCacheDir string
|
BuilderCacheDir string
|
||||||
BuilderTimeoutSeconds int
|
BuilderTimeoutSeconds int
|
||||||
@@ -93,6 +97,8 @@ func Load() Config {
|
|||||||
BuilderDockerBinary: defaultString(strings.TrimSpace(os.Getenv("PLATFORM_BUILDER_DOCKER_BINARY")), defaultBuilderDockerBinary),
|
BuilderDockerBinary: defaultString(strings.TrimSpace(os.Getenv("PLATFORM_BUILDER_DOCKER_BINARY")), defaultBuilderDockerBinary),
|
||||||
BuilderImage: defaultString(strings.TrimSpace(os.Getenv("PLATFORM_BUILDER_IMAGE")), defaultBuilderImage),
|
BuilderImage: defaultString(strings.TrimSpace(os.Getenv("PLATFORM_BUILDER_IMAGE")), defaultBuilderImage),
|
||||||
BuilderSourceDir: strings.TrimSpace(os.Getenv("PLATFORM_BUILDER_SOURCE_DIR")),
|
BuilderSourceDir: strings.TrimSpace(os.Getenv("PLATFORM_BUILDER_SOURCE_DIR")),
|
||||||
|
BuilderSourceRepository: defaultString(strings.TrimSpace(os.Getenv("PLATFORM_BUILDER_SOURCE_REPOSITORY")), defaultBuilderSourceRepository),
|
||||||
|
BuilderSourceRevision: defaultString(strings.TrimSpace(os.Getenv("PLATFORM_BUILDER_SOURCE_REVISION")), defaultBuilderSourceRevision),
|
||||||
BuilderWorkspaceDir: builderWorkspaceDir,
|
BuilderWorkspaceDir: builderWorkspaceDir,
|
||||||
BuilderCacheDir: builderCacheDir,
|
BuilderCacheDir: builderCacheDir,
|
||||||
BuilderTimeoutSeconds: defaultPositiveInt(strings.TrimSpace(os.Getenv("PLATFORM_BUILDER_TIMEOUT_SECONDS")), defaultBuilderTimeoutSeconds),
|
BuilderTimeoutSeconds: defaultPositiveInt(strings.TrimSpace(os.Getenv("PLATFORM_BUILDER_TIMEOUT_SECONDS")), defaultBuilderTimeoutSeconds),
|
||||||
|
|||||||
@@ -21,6 +21,8 @@ func TestLoadUsesDefaultAddress(t *testing.T) {
|
|||||||
t.Setenv("PLATFORM_BUILDER_DOCKER_BINARY", "")
|
t.Setenv("PLATFORM_BUILDER_DOCKER_BINARY", "")
|
||||||
t.Setenv("PLATFORM_BUILDER_IMAGE", "")
|
t.Setenv("PLATFORM_BUILDER_IMAGE", "")
|
||||||
t.Setenv("PLATFORM_BUILDER_SOURCE_DIR", "")
|
t.Setenv("PLATFORM_BUILDER_SOURCE_DIR", "")
|
||||||
|
t.Setenv("PLATFORM_BUILDER_SOURCE_REPOSITORY", "")
|
||||||
|
t.Setenv("PLATFORM_BUILDER_SOURCE_REVISION", "")
|
||||||
t.Setenv("PLATFORM_BUILDER_WORKSPACE_DIR", "")
|
t.Setenv("PLATFORM_BUILDER_WORKSPACE_DIR", "")
|
||||||
t.Setenv("PLATFORM_BUILDER_CACHE_DIR", "")
|
t.Setenv("PLATFORM_BUILDER_CACHE_DIR", "")
|
||||||
t.Setenv("PLATFORM_BUILDER_TIMEOUT_SECONDS", "")
|
t.Setenv("PLATFORM_BUILDER_TIMEOUT_SECONDS", "")
|
||||||
@@ -35,7 +37,7 @@ func TestLoadUsesDefaultAddress(t *testing.T) {
|
|||||||
if cfg.MetadataPath != filepath.Join(".platform-data", "metadata.json") || cfg.LogDir != filepath.Join(".platform-data", "logs") {
|
if cfg.MetadataPath != filepath.Join(".platform-data", "metadata.json") || cfg.LogDir != filepath.Join(".platform-data", "logs") {
|
||||||
t.Fatalf("unexpected default storage paths: %+v", cfg)
|
t.Fatalf("unexpected default storage paths: %+v", cfg)
|
||||||
}
|
}
|
||||||
if cfg.BuilderDockerBinary != defaultBuilderDockerBinary || cfg.BuilderImage != defaultBuilderImage || cfg.BuilderSourceDir != "" || cfg.BuilderWorkspaceDir != filepath.Join(".platform-data", "distribution-builds") || cfg.BuilderCacheDir != filepath.Join(".platform-data", "distribution-build-cache") || cfg.BuilderTimeoutSeconds != defaultBuilderTimeoutSeconds {
|
if cfg.BuilderDockerBinary != defaultBuilderDockerBinary || cfg.BuilderImage != defaultBuilderImage || cfg.BuilderSourceDir != "" || cfg.BuilderSourceRepository != defaultBuilderSourceRepository || cfg.BuilderSourceRevision != defaultBuilderSourceRevision || cfg.BuilderWorkspaceDir != filepath.Join(".platform-data", "distribution-builds") || cfg.BuilderCacheDir != filepath.Join(".platform-data", "distribution-build-cache") || cfg.BuilderTimeoutSeconds != defaultBuilderTimeoutSeconds {
|
||||||
t.Fatalf("unexpected default builder config: %+v", cfg)
|
t.Fatalf("unexpected default builder config: %+v", cfg)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -54,6 +56,8 @@ func TestLoadUsesConfiguredAddress(t *testing.T) {
|
|||||||
t.Setenv("PLATFORM_BUILDER_DOCKER_BINARY", "/usr/local/bin/docker")
|
t.Setenv("PLATFORM_BUILDER_DOCKER_BINARY", "/usr/local/bin/docker")
|
||||||
t.Setenv("PLATFORM_BUILDER_IMAGE", "registry.example.test/distribution-builder:2.0.0")
|
t.Setenv("PLATFORM_BUILDER_IMAGE", "registry.example.test/distribution-builder:2.0.0")
|
||||||
t.Setenv("PLATFORM_BUILDER_SOURCE_DIR", "/srv/run-source")
|
t.Setenv("PLATFORM_BUILDER_SOURCE_DIR", "/srv/run-source")
|
||||||
|
t.Setenv("PLATFORM_BUILDER_SOURCE_REPOSITORY", "https://git.example.test/admin/run.git")
|
||||||
|
t.Setenv("PLATFORM_BUILDER_SOURCE_REVISION", "release")
|
||||||
t.Setenv("PLATFORM_BUILDER_WORKSPACE_DIR", "/srv/distribution-builds")
|
t.Setenv("PLATFORM_BUILDER_WORKSPACE_DIR", "/srv/distribution-builds")
|
||||||
t.Setenv("PLATFORM_BUILDER_CACHE_DIR", "/srv/distribution-build-cache")
|
t.Setenv("PLATFORM_BUILDER_CACHE_DIR", "/srv/distribution-build-cache")
|
||||||
t.Setenv("PLATFORM_BUILDER_TIMEOUT_SECONDS", "900")
|
t.Setenv("PLATFORM_BUILDER_TIMEOUT_SECONDS", "900")
|
||||||
@@ -65,7 +69,7 @@ func TestLoadUsesConfiguredAddress(t *testing.T) {
|
|||||||
if cfg.StorageBackend != "mysql" || cfg.MySQLDSN != "platform:platform@tcp(127.0.0.1:3306)/platform?parseTime=true" || cfg.DataDir != "/tmp/platform-data" || cfg.MetadataPath != "/tmp/platform-metadata.json" || cfg.LogDir != "/tmp/platform-logs" || cfg.LogBodyBackend != "file" || cfg.BootstrapAdminEmail != "admin@example.test" || cfg.BootstrapAdminPassword != "configured-secret" || cfg.SecretEnvelopeKey != "configured-envelope-key-at-least-32-bytes" {
|
if cfg.StorageBackend != "mysql" || cfg.MySQLDSN != "platform:platform@tcp(127.0.0.1:3306)/platform?parseTime=true" || cfg.DataDir != "/tmp/platform-data" || cfg.MetadataPath != "/tmp/platform-metadata.json" || cfg.LogDir != "/tmp/platform-logs" || cfg.LogBodyBackend != "file" || cfg.BootstrapAdminEmail != "admin@example.test" || cfg.BootstrapAdminPassword != "configured-secret" || cfg.SecretEnvelopeKey != "configured-envelope-key-at-least-32-bytes" {
|
||||||
t.Fatalf("unexpected configured storage: %+v", cfg)
|
t.Fatalf("unexpected configured storage: %+v", cfg)
|
||||||
}
|
}
|
||||||
if cfg.BuilderDockerBinary != "/usr/local/bin/docker" || cfg.BuilderImage != "registry.example.test/distribution-builder:2.0.0" || cfg.BuilderSourceDir != "/srv/run-source" || cfg.BuilderWorkspaceDir != "/srv/distribution-builds" || cfg.BuilderCacheDir != "/srv/distribution-build-cache" || cfg.BuilderTimeoutSeconds != 900 {
|
if cfg.BuilderDockerBinary != "/usr/local/bin/docker" || cfg.BuilderImage != "registry.example.test/distribution-builder:2.0.0" || cfg.BuilderSourceDir != "/srv/run-source" || cfg.BuilderSourceRepository != "https://git.example.test/admin/run.git" || cfg.BuilderSourceRevision != "release" || cfg.BuilderWorkspaceDir != "/srv/distribution-builds" || cfg.BuilderCacheDir != "/srv/distribution-build-cache" || cfg.BuilderTimeoutSeconds != 900 {
|
||||||
t.Fatalf("unexpected configured builder: %+v", cfg)
|
t.Fatalf("unexpected configured builder: %+v", cfg)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -127,6 +131,8 @@ func clearPlatformEnv(t *testing.T) {
|
|||||||
"PLATFORM_BUILDER_DOCKER_BINARY",
|
"PLATFORM_BUILDER_DOCKER_BINARY",
|
||||||
"PLATFORM_BUILDER_IMAGE",
|
"PLATFORM_BUILDER_IMAGE",
|
||||||
"PLATFORM_BUILDER_SOURCE_DIR",
|
"PLATFORM_BUILDER_SOURCE_DIR",
|
||||||
|
"PLATFORM_BUILDER_SOURCE_REPOSITORY",
|
||||||
|
"PLATFORM_BUILDER_SOURCE_REVISION",
|
||||||
"PLATFORM_BUILDER_WORKSPACE_DIR",
|
"PLATFORM_BUILDER_WORKSPACE_DIR",
|
||||||
"PLATFORM_BUILDER_CACHE_DIR",
|
"PLATFORM_BUILDER_CACHE_DIR",
|
||||||
"PLATFORM_BUILDER_TIMEOUT_SECONDS",
|
"PLATFORM_BUILDER_TIMEOUT_SECONDS",
|
||||||
|
|||||||
@@ -51,12 +51,15 @@ type DockerDistributionBuilderConfig struct {
|
|||||||
DockerBinary string
|
DockerBinary string
|
||||||
Image string
|
Image string
|
||||||
SourceDir string
|
SourceDir string
|
||||||
|
SourceRepositoryURL string
|
||||||
|
SourceRevision string
|
||||||
WorkspaceDir string
|
WorkspaceDir string
|
||||||
CacheDir string
|
CacheDir string
|
||||||
Timeout time.Duration
|
Timeout time.Duration
|
||||||
PlatformURL string
|
PlatformURL string
|
||||||
CommandRunner func(ctx context.Context, name string, args ...string) ([]byte, error)
|
CommandRunner func(ctx context.Context, name string, args ...string) ([]byte, error)
|
||||||
CommandStream func(ctx context.Context, name string, args []string, onLine func(string)) ([]byte, error)
|
CommandStream func(ctx context.Context, name string, args []string, onLine func(string)) ([]byte, error)
|
||||||
|
GitCommandRunner func(ctx context.Context, name string, args ...string) ([]byte, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
// DockerDistributionBuilder runs each build in a container from a pinned image,
|
// DockerDistributionBuilder runs each build in a container from a pinned image,
|
||||||
@@ -64,6 +67,7 @@ type DockerDistributionBuilderConfig struct {
|
|||||||
// writable.
|
// writable.
|
||||||
type DockerDistributionBuilder struct {
|
type DockerDistributionBuilder struct {
|
||||||
config DockerDistributionBuilderConfig
|
config DockerDistributionBuilderConfig
|
||||||
|
sourceMu sync.Mutex
|
||||||
}
|
}
|
||||||
|
|
||||||
func NewDockerDistributionBuilder(config DockerDistributionBuilderConfig) *DockerDistributionBuilder {
|
func NewDockerDistributionBuilder(config DockerDistributionBuilderConfig) *DockerDistributionBuilder {
|
||||||
@@ -77,6 +81,9 @@ func NewDockerDistributionBuilder(config DockerDistributionBuilderConfig) *Docke
|
|||||||
if config.CommandRunner == nil {
|
if config.CommandRunner == nil {
|
||||||
config.CommandRunner = runCommandCombined
|
config.CommandRunner = runCommandCombined
|
||||||
}
|
}
|
||||||
|
if config.GitCommandRunner == nil {
|
||||||
|
config.GitCommandRunner = runCommandCombined
|
||||||
|
}
|
||||||
if config.CommandStream == nil && !customRunner {
|
if config.CommandStream == nil && !customRunner {
|
||||||
config.CommandStream = runCommandStreamCombined
|
config.CommandStream = runCommandStreamCombined
|
||||||
}
|
}
|
||||||
@@ -142,8 +149,13 @@ func (builder *DockerDistributionBuilder) Readiness() (bool, string) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return false, "platform builder run source directory is invalid"
|
return false, "platform builder run source directory is invalid"
|
||||||
}
|
}
|
||||||
|
if _, err := os.Stat(source); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return false, "platform builder run source directory is not accessible"
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(builder.config.SourceRepositoryURL) == "" {
|
||||||
if _, err := os.Stat(filepath.Join(source, "go.mod")); err != nil {
|
if _, err := os.Stat(filepath.Join(source, "go.mod")); err != nil {
|
||||||
return false, "platform builder run source directory does not contain a run checkout"
|
return false, "platform builder run source repository is not configured"
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if strings.TrimSpace(builder.config.WorkspaceDir) == "" {
|
if strings.TrimSpace(builder.config.WorkspaceDir) == "" {
|
||||||
return false, "platform builder workspace directory is not configured"
|
return false, "platform builder workspace directory is not configured"
|
||||||
@@ -199,6 +211,15 @@ func (builder *DockerDistributionBuilder) BuildWithProgress(input domain.Distrib
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, validationError("platform builder workspace directory is invalid")
|
return nil, validationError("platform builder workspace directory is invalid")
|
||||||
}
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), builder.config.Timeout)
|
||||||
|
defer cancel()
|
||||||
|
if input.ComponentKind == domain.DistributionComponentRun {
|
||||||
|
// Keep a shared checkout stable while Docker reads it. The checkout is
|
||||||
|
// refreshed and exported below for this build, rather than captured when
|
||||||
|
// the platform process starts.
|
||||||
|
builder.sourceMu.Lock()
|
||||||
|
defer builder.sourceMu.Unlock()
|
||||||
|
}
|
||||||
cacheDir, err := builder.cacheDir(workspaceDir)
|
cacheDir, err := builder.cacheDir(workspaceDir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, validationError("platform builder cache directory is invalid")
|
return nil, validationError("platform builder cache directory is invalid")
|
||||||
@@ -212,15 +233,23 @@ func (builder *DockerDistributionBuilder) BuildWithProgress(input domain.Distrib
|
|||||||
outputDir := filepath.Join(jobDir, "output")
|
outputDir := filepath.Join(jobDir, "output")
|
||||||
inputDir := filepath.Join(jobDir, "input")
|
inputDir := filepath.Join(jobDir, "input")
|
||||||
buildDir := filepath.Join(jobDir, "build")
|
buildDir := filepath.Join(jobDir, "build")
|
||||||
|
sourceMountDir := filepath.Join(jobDir, "source")
|
||||||
goBuildCacheDir := filepath.Join(cacheDir, "go-build")
|
goBuildCacheDir := filepath.Join(cacheDir, "go-build")
|
||||||
goModCacheDir := filepath.Join(cacheDir, "go-mod")
|
goModCacheDir := filepath.Join(cacheDir, "go-mod")
|
||||||
for _, directory := range []string{outputDir, inputDir, buildDir, goBuildCacheDir, goModCacheDir} {
|
for _, directory := range []string{outputDir, inputDir, buildDir, sourceMountDir, goBuildCacheDir, goModCacheDir} {
|
||||||
if err := os.MkdirAll(directory, 0o700); err != nil {
|
if err := os.MkdirAll(directory, 0o700); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
defer func() { _ = os.RemoveAll(jobDir) }()
|
defer func() { _ = os.RemoveAll(jobDir) }()
|
||||||
reportBuilderProgress(progress, 14, "env_check: platform builder workspace prepared")
|
reportBuilderProgress(progress, 14, "env_check: platform builder workspace prepared")
|
||||||
|
if input.ComponentKind == domain.DistributionComponentRun {
|
||||||
|
preparedSourceDir, err := builder.prepareRunSource(ctx, sourceDir, sourceMountDir, input, progress)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
sourceMountDir = preparedSourceDir
|
||||||
|
}
|
||||||
|
|
||||||
// The auth key reaches the container through a per-job input file, never
|
// The auth key reaches the container through a per-job input file, never
|
||||||
// through a job-channel response to a machine-side endpoint or a container
|
// through a job-channel response to a machine-side endpoint or a container
|
||||||
@@ -261,9 +290,7 @@ func (builder *DockerDistributionBuilder) BuildWithProgress(input domain.Distrib
|
|||||||
if outputName == "" || filepath.Base(outputName) != outputName {
|
if outputName == "" || filepath.Base(outputName) != outputName {
|
||||||
return nil, validationError("distribution build input has an invalid output filename")
|
return nil, validationError("distribution build input has an invalid output filename")
|
||||||
}
|
}
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), builder.config.Timeout)
|
args := builder.containerArgs(input, sourceMountDir, inputDir, buildDir, outputDir, goBuildCacheDir, goModCacheDir, outputName)
|
||||||
defer cancel()
|
|
||||||
args := builder.containerArgs(input, sourceDir, inputDir, buildDir, outputDir, goBuildCacheDir, goModCacheDir, outputName)
|
|
||||||
reportBuilderProgress(progress, 18, "git_sync: platform builder container starting")
|
reportBuilderProgress(progress, 18, "git_sync: platform builder container starting")
|
||||||
output, err := builder.runBuildCommand(ctx, args, progress)
|
output, err := builder.runBuildCommand(ctx, args, progress)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -290,6 +317,98 @@ func (builder *DockerDistributionBuilder) BuildWithProgress(input domain.Distrib
|
|||||||
return packageClientManagerDistribution(input.PackageFormat, outputName, binary, configPayload)
|
return packageClientManagerDistribution(input.PackageFormat, outputName, binary, configPayload)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// prepareRunSource implements the source phase of a Jenkins-style build. A
|
||||||
|
// configured checkout is created when absent, fetched on every build, and
|
||||||
|
// exported at FETCH_HEAD into the per-job workspace. The archive keeps local
|
||||||
|
// uncommitted files and the checkout's .git directory out of the executable.
|
||||||
|
func (builder *DockerDistributionBuilder) prepareRunSource(ctx context.Context, sourceDir, destination string, input domain.DistributionBuildInput, progress func(DistributionBuildProgress)) (string, error) {
|
||||||
|
repository := strings.TrimSpace(input.RepositoryURL)
|
||||||
|
if repository == "" {
|
||||||
|
repository = strings.TrimSpace(builder.config.SourceRepositoryURL)
|
||||||
|
}
|
||||||
|
revision := strings.TrimSpace(input.SourceRevision)
|
||||||
|
if revision == "" {
|
||||||
|
revision = strings.TrimSpace(builder.config.SourceRevision)
|
||||||
|
}
|
||||||
|
if revision == "" {
|
||||||
|
revision = "main"
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := os.Stat(filepath.Join(sourceDir, ".git")); errors.Is(err, os.ErrNotExist) {
|
||||||
|
if repository == "" {
|
||||||
|
if _, sourceErr := os.Stat(filepath.Join(sourceDir, "go.mod")); sourceErr != nil {
|
||||||
|
return "", validationError("platform builder has no Run source checkout or repository")
|
||||||
|
}
|
||||||
|
return sourceDir, nil
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(filepath.Dir(sourceDir), 0o700); err != nil {
|
||||||
|
return "", validationError("platform builder could not create the Run source parent")
|
||||||
|
}
|
||||||
|
reportBuilderProgress(progress, 20, "git_sync: creating Run source checkout")
|
||||||
|
if _, err := builder.config.GitCommandRunner(ctx, "git", "clone", "--no-checkout", repository, sourceDir); err != nil {
|
||||||
|
return "", validationError("platform builder could not clone the Run source")
|
||||||
|
}
|
||||||
|
} else if err != nil {
|
||||||
|
return "", validationError("platform builder could not inspect the Run source checkout")
|
||||||
|
}
|
||||||
|
|
||||||
|
reportBuilderProgress(progress, 26, "git_sync: fetching Run source revision")
|
||||||
|
if _, err := builder.config.GitCommandRunner(ctx, "git", "-C", sourceDir, "fetch", "--depth", "1", "origin", revision); err != nil {
|
||||||
|
return "", validationError("platform builder could not update the Run source revision")
|
||||||
|
}
|
||||||
|
archivePayload, err := builder.config.GitCommandRunner(ctx, "git", "-C", sourceDir, "archive", "--format=tar", "FETCH_HEAD")
|
||||||
|
if err != nil {
|
||||||
|
return "", validationError("platform builder could not export the Run source revision")
|
||||||
|
}
|
||||||
|
if err := extractRunSourceArchive(archivePayload, destination); err != nil {
|
||||||
|
return "", validationError("platform builder could not prepare the Run source workspace")
|
||||||
|
}
|
||||||
|
reportBuilderProgress(progress, 34, "git_sync: Run source revision prepared")
|
||||||
|
return destination, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func extractRunSourceArchive(payload []byte, destination string) error {
|
||||||
|
reader := tar.NewReader(bytes.NewReader(payload))
|
||||||
|
for {
|
||||||
|
header, err := reader.Next()
|
||||||
|
if errors.Is(err, io.EOF) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
name := filepath.Clean(filepath.FromSlash(header.Name))
|
||||||
|
if name == "." || filepath.IsAbs(name) || name == ".." || strings.HasPrefix(name, ".."+string(os.PathSeparator)) {
|
||||||
|
return errors.New("Run source archive contains an unsafe path")
|
||||||
|
}
|
||||||
|
target := filepath.Join(destination, name)
|
||||||
|
if header.FileInfo().IsDir() {
|
||||||
|
if err := os.MkdirAll(target, header.FileInfo().Mode().Perm()); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !header.FileInfo().Mode().IsRegular() {
|
||||||
|
return errors.New("Run source archive contains an unsupported file")
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(filepath.Dir(target), 0o700); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
file, err := os.OpenFile(target, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, header.FileInfo().Mode().Perm())
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_, copyErr := io.Copy(file, reader)
|
||||||
|
closeErr := file.Close()
|
||||||
|
if copyErr != nil {
|
||||||
|
return copyErr
|
||||||
|
}
|
||||||
|
if closeErr != nil {
|
||||||
|
return closeErr
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func (builder *DockerDistributionBuilder) cacheDir(workspaceDir string) (string, error) {
|
func (builder *DockerDistributionBuilder) cacheDir(workspaceDir string) (string, error) {
|
||||||
configured := strings.TrimSpace(builder.config.CacheDir)
|
configured := strings.TrimSpace(builder.config.CacheDir)
|
||||||
if configured == "" {
|
if configured == "" {
|
||||||
|
|||||||
@@ -94,6 +94,66 @@ func TestDockerDistributionBuilderReadinessNamesPlatformBuilderFailures(t *testi
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestDockerDistributionBuilderRefreshesRunSourceAtBuildTime(t *testing.T) {
|
||||||
|
sourceDir := filepath.Join(t.TempDir(), "run-checkout")
|
||||||
|
workspaceDir := t.TempDir()
|
||||||
|
var gitCalls [][]string
|
||||||
|
builder := NewDockerDistributionBuilder(DockerDistributionBuilderConfig{
|
||||||
|
DockerBinary: "docker-test",
|
||||||
|
Image: "browser-platform-distribution-builder:1.0.0",
|
||||||
|
SourceDir: sourceDir,
|
||||||
|
SourceRepositoryURL: "git@example.test:admin/run.git",
|
||||||
|
SourceRevision: "main",
|
||||||
|
WorkspaceDir: workspaceDir,
|
||||||
|
GitCommandRunner: func(_ context.Context, name string, args ...string) ([]byte, error) {
|
||||||
|
if name != "git" {
|
||||||
|
t.Fatalf("unexpected source command %q", name)
|
||||||
|
}
|
||||||
|
gitCalls = append(gitCalls, append([]string(nil), args...))
|
||||||
|
if len(args) >= 4 && args[0] == "clone" {
|
||||||
|
if err := os.MkdirAll(filepath.Join(args[len(args)-1], ".git"), 0o700); err != nil {
|
||||||
|
t.Fatalf("create fake checkout: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(args) >= 4 && args[0] == "-C" && args[2] == "archive" {
|
||||||
|
return runSourceArchive(t), nil
|
||||||
|
}
|
||||||
|
return nil, nil
|
||||||
|
},
|
||||||
|
CommandRunner: func(_ context.Context, name string, args ...string) ([]byte, error) {
|
||||||
|
if name != "docker-test" {
|
||||||
|
t.Fatalf("unexpected container runtime %q", name)
|
||||||
|
}
|
||||||
|
if len(args) > 0 && (args[0] == "version" || args[0] == "image") {
|
||||||
|
return []byte("27.0.0"), nil
|
||||||
|
}
|
||||||
|
outputDir := builderMountHostPath(t, args, "/workspace/output")
|
||||||
|
if err := os.WriteFile(filepath.Join(outputDir, "run"), []byte("compiled-run"), 0o700); err != nil {
|
||||||
|
t.Fatalf("write fake build output: %v", err)
|
||||||
|
}
|
||||||
|
return nil, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
payload, err := builder.Build(domain.DistributionBuildInput{
|
||||||
|
JobID: "job-source-refresh",
|
||||||
|
ComponentKind: domain.DistributionComponentRun,
|
||||||
|
PluginID: "game.scum",
|
||||||
|
TargetOS: "linux",
|
||||||
|
TargetArch: "amd64",
|
||||||
|
OutputFilename: "run",
|
||||||
|
AuthKey: "component-key",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("build refreshed Run source: %v", err)
|
||||||
|
}
|
||||||
|
if string(payload) != "compiled-run" {
|
||||||
|
t.Fatalf("unexpected built payload %q", payload)
|
||||||
|
}
|
||||||
|
if len(gitCalls) != 3 || gitCalls[0][0] != "clone" || gitCalls[1][0] != "-C" || gitCalls[1][2] != "fetch" || gitCalls[2][2] != "archive" {
|
||||||
|
t.Fatalf("expected clone, fetch, and archive during build, got %#v", gitCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestDockerDistributionBuilderKeepsSecretInIsolatedInput(t *testing.T) {
|
func TestDockerDistributionBuilderKeepsSecretInIsolatedInput(t *testing.T) {
|
||||||
sourceDir := createBuilderSource(t)
|
sourceDir := createBuilderSource(t)
|
||||||
workspaceDir := t.TempDir()
|
workspaceDir := t.TempDir()
|
||||||
@@ -143,6 +203,11 @@ func TestDockerDistributionBuilderKeepsSecretInIsolatedInput(t *testing.T) {
|
|||||||
if bytes.Contains(script, []byte(secret)) {
|
if bytes.Contains(script, []byte(secret)) {
|
||||||
t.Fatal("build script must not embed the component auth key")
|
t.Fatal("build script must not embed the component auth key")
|
||||||
}
|
}
|
||||||
|
for _, ordered := range [][2]string{{"cp -R /workspace/source/. /workspace/build/run-source/", "workspace_seed_generated.go"}, {"workspace_seed_generated.go", "go build -trimpath -ldflags"}} {
|
||||||
|
if bytes.Index(script, []byte(ordered[0])) >= bytes.Index(script, []byte(ordered[1])) {
|
||||||
|
t.Fatalf("Run build script must prepare source before injecting config and compiling: %q before %q", ordered[0], ordered[1])
|
||||||
|
}
|
||||||
|
}
|
||||||
planPayload, err := os.ReadFile(filepath.Join(inputDir, "autonomous-lifecycle-plan.json"))
|
planPayload, err := os.ReadFile(filepath.Join(inputDir, "autonomous-lifecycle-plan.json"))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("read autonomous lifecycle plan input: %v", err)
|
t.Fatalf("read autonomous lifecycle plan input: %v", err)
|
||||||
@@ -339,6 +404,22 @@ func createBuilderSource(t *testing.T) string {
|
|||||||
return sourceDir
|
return sourceDir
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func runSourceArchive(t *testing.T) []byte {
|
||||||
|
t.Helper()
|
||||||
|
var payload bytes.Buffer
|
||||||
|
writer := tar.NewWriter(&payload)
|
||||||
|
if err := writer.WriteHeader(&tar.Header{Name: "go.mod", Mode: 0o600, Size: int64(len("module browser.local/run\n"))}); err != nil {
|
||||||
|
t.Fatalf("write source archive header: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := writer.Write([]byte("module browser.local/run\n")); err != nil {
|
||||||
|
t.Fatalf("write source archive body: %v", err)
|
||||||
|
}
|
||||||
|
if err := writer.Close(); err != nil {
|
||||||
|
t.Fatalf("close source archive: %v", err)
|
||||||
|
}
|
||||||
|
return payload.Bytes()
|
||||||
|
}
|
||||||
|
|
||||||
func builderMountHostPath(t *testing.T, args []string, containerSuffix string) string {
|
func builderMountHostPath(t *testing.T, args []string, containerSuffix string) string {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
for index := 0; index+1 < len(args); index++ {
|
for index := 0; index+1 < len(args); index++ {
|
||||||
|
|||||||
+15
-46
@@ -37,11 +37,15 @@ export RUN_VERSION="${RUN_VERSION:-0.1.0-local-debug}"
|
|||||||
export RUN_REGISTRATION_TOKEN="${RUN_REGISTRATION_TOKEN:-local-debug-registration}"
|
export RUN_REGISTRATION_TOKEN="${RUN_REGISTRATION_TOKEN:-local-debug-registration}"
|
||||||
export RUN_WORKSPACE_ROOT="${RUN_WORKSPACE_ROOT:-$LOCAL_DEBUG_ROOT/run/workspace}"
|
export RUN_WORKSPACE_ROOT="${RUN_WORKSPACE_ROOT:-$LOCAL_DEBUG_ROOT/run/workspace}"
|
||||||
export RUN_BUILD_BUCKET_ROOT="${RUN_BUILD_BUCKET_ROOT:-$LOCAL_DEBUG_ROOT/run/build-buckets}"
|
export RUN_BUILD_BUCKET_ROOT="${RUN_BUILD_BUCKET_ROOT:-$LOCAL_DEBUG_ROOT/run/build-buckets}"
|
||||||
export RUN_BUILD_SOURCE_ROOT="${RUN_BUILD_SOURCE_ROOT:-$RUN_BUILD_BUCKET_ROOT/source/current}"
|
export RUN_BUILD_SOURCE_ROOT="${RUN_BUILD_SOURCE_ROOT:-$RUN_SOURCE_DIR}"
|
||||||
export RUN_BOOTSTRAP_BIN="${RUN_BOOTSTRAP_BIN:-$RUN_BUILD_BUCKET_ROOT/bootstrap/bin/run}"
|
export RUN_BOOTSTRAP_BIN="${RUN_BOOTSTRAP_BIN:-$RUN_BUILD_BUCKET_ROOT/bootstrap/bin/run}"
|
||||||
export PLATFORM_BUILDER_DOCKER_BINARY="${PLATFORM_BUILDER_DOCKER_BINARY:-docker}"
|
export PLATFORM_BUILDER_DOCKER_BINARY="${PLATFORM_BUILDER_DOCKER_BINARY:-docker}"
|
||||||
export PLATFORM_BUILDER_IMAGE="${PLATFORM_BUILDER_IMAGE:-browser-platform-distribution-builder:1.0.0}"
|
export PLATFORM_BUILDER_IMAGE="${PLATFORM_BUILDER_IMAGE:-browser-platform-distribution-builder:1.0.0}"
|
||||||
export PLATFORM_BUILDER_SOURCE_DIR="${PLATFORM_BUILDER_SOURCE_DIR:-$RUN_BUILD_SOURCE_ROOT}"
|
export RUN_SOURCE_REPOSITORY="${RUN_SOURCE_REPOSITORY:-git@git.npc0.com:admin343/run.git}"
|
||||||
|
export RUN_SOURCE_REVISION="${RUN_SOURCE_REVISION:-main}"
|
||||||
|
export PLATFORM_BUILDER_SOURCE_DIR="${PLATFORM_BUILDER_SOURCE_DIR:-$RUN_SOURCE_DIR}"
|
||||||
|
export PLATFORM_BUILDER_SOURCE_REPOSITORY="${PLATFORM_BUILDER_SOURCE_REPOSITORY:-$RUN_SOURCE_REPOSITORY}"
|
||||||
|
export PLATFORM_BUILDER_SOURCE_REVISION="${PLATFORM_BUILDER_SOURCE_REVISION:-$RUN_SOURCE_REVISION}"
|
||||||
export PLATFORM_BUILDER_WORKSPACE_DIR="${PLATFORM_BUILDER_WORKSPACE_DIR:-$PLATFORM_DATA_DIR/distribution-builds}"
|
export PLATFORM_BUILDER_WORKSPACE_DIR="${PLATFORM_BUILDER_WORKSPACE_DIR:-$PLATFORM_DATA_DIR/distribution-builds}"
|
||||||
export PLATFORM_BUILDER_CACHE_DIR="${PLATFORM_BUILDER_CACHE_DIR:-$PLATFORM_DATA_DIR/distribution-build-cache}"
|
export PLATFORM_BUILDER_CACHE_DIR="${PLATFORM_BUILDER_CACHE_DIR:-$PLATFORM_DATA_DIR/distribution-build-cache}"
|
||||||
export PLATFORM_BUILDER_TIMEOUT_SECONDS="${PLATFORM_BUILDER_TIMEOUT_SECONDS:-1800}"
|
export PLATFORM_BUILDER_TIMEOUT_SECONDS="${PLATFORM_BUILDER_TIMEOUT_SECONDS:-1800}"
|
||||||
@@ -142,51 +146,16 @@ local_debug_prepare_run_lifecycle_templates() {
|
|||||||
local_debug_seed_plugin_lifecycle_template game.scum run-local
|
local_debug_seed_plugin_lifecycle_template game.scum run-local
|
||||||
}
|
}
|
||||||
|
|
||||||
local_debug_require_bucket_path() {
|
local_debug_prepare_run_build_source() {
|
||||||
local target="$1"
|
if [[ -e "$RUN_SOURCE_DIR" && ! -d "$RUN_SOURCE_DIR/.git" ]]; then
|
||||||
local parent
|
printf 'run source path is not a git checkout: %s\n' "$RUN_SOURCE_DIR" >&2
|
||||||
parent="$(dirname "$target")"
|
|
||||||
mkdir -p "$parent"
|
|
||||||
local resolved_parent
|
|
||||||
resolved_parent="$(cd "$parent" && pwd -P)"
|
|
||||||
local resolved="$resolved_parent/$(basename "$target")"
|
|
||||||
case "$resolved" in
|
|
||||||
"$LOCAL_DEBUG_ROOT"/* | /private/tmp/browser-local-debug-*/* | /tmp/browser-local-debug-*/*)
|
|
||||||
return 0
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
printf 'refusing to clear non-local-debug build bucket path: %s\n' "$target" >&2
|
|
||||||
printf 'set RUN_BUILD_BUCKET_ROOT/RUN_BUILD_SOURCE_ROOT under %s or an approved browser-local-debug temp root\n' "$LOCAL_DEBUG_ROOT" >&2
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
|
|
||||||
ensure_local_run_repo() {
|
|
||||||
if [[ ! -f "$RUN_SOURCE_DIR/go.mod" ]]; then
|
|
||||||
printf 'run source repo not found at %s\n' "$RUN_SOURCE_DIR" >&2
|
|
||||||
printf 'clone git@git.npc0.com:admin343/run.git into %s/run or set RUN_SOURCE_DIR/RUN_REPO_DIR to another independent run checkout\n' "$LOCAL_DEBUG_ROOT_DIR" >&2
|
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
}
|
if [[ ! -d "$RUN_SOURCE_DIR/.git" ]]; then
|
||||||
|
mkdir -p "$(dirname "$RUN_SOURCE_DIR")"
|
||||||
local_debug_prepare_run_build_source() {
|
git clone --no-checkout "$RUN_SOURCE_REPOSITORY" "$RUN_SOURCE_DIR"
|
||||||
ensure_local_run_repo
|
fi
|
||||||
local_debug_require_bucket_path "$RUN_BUILD_SOURCE_ROOT"
|
git -C "$RUN_SOURCE_DIR" fetch --depth 1 origin "$RUN_SOURCE_REVISION"
|
||||||
rm -rf "$RUN_BUILD_SOURCE_ROOT"
|
|
||||||
mkdir -p "$RUN_BUILD_SOURCE_ROOT"
|
|
||||||
rsync -a --delete \
|
|
||||||
--exclude '.git/' \
|
|
||||||
--exclude '.local-debug/' \
|
|
||||||
--exclude '.run-workspace/' \
|
|
||||||
--exclude '.tmp/' \
|
|
||||||
--exclude 'node_modules/' \
|
|
||||||
"$RUN_SOURCE_DIR"/ "$RUN_BUILD_SOURCE_ROOT"/
|
|
||||||
cat >"$RUN_BUILD_SOURCE_ROOT/.local-debug-source.json" <<EOF
|
|
||||||
{
|
|
||||||
"sourceDir": "$RUN_SOURCE_DIR",
|
|
||||||
"buildSourceRoot": "$RUN_BUILD_SOURCE_ROOT",
|
|
||||||
"createdBy": "scripts/local-debug/env.sh"
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
}
|
}
|
||||||
|
|
||||||
local_debug_prepare_distribution_builder() {
|
local_debug_prepare_distribution_builder() {
|
||||||
@@ -211,7 +180,7 @@ local_debug_build_bootstrap_run() {
|
|||||||
local_debug_prepare_run_build_source
|
local_debug_prepare_run_build_source
|
||||||
mkdir -p "$(dirname "$RUN_BOOTSTRAP_BIN")"
|
mkdir -p "$(dirname "$RUN_BOOTSTRAP_BIN")"
|
||||||
(
|
(
|
||||||
cd "$RUN_BUILD_SOURCE_ROOT"
|
cd "$RUN_SOURCE_DIR"
|
||||||
env GOCACHE="$GOCACHE" go build -trimpath -o "$RUN_BOOTSTRAP_BIN" ./cmd/run
|
env GOCACHE="$GOCACHE" go build -trimpath -o "$RUN_BOOTSTRAP_BIN" ./cmd/run
|
||||||
)
|
)
|
||||||
chmod +x "$RUN_BOOTSTRAP_BIN"
|
chmod +x "$RUN_BOOTSTRAP_BIN"
|
||||||
|
|||||||
@@ -46,6 +46,8 @@ RUN_BOOTSTRAP_BIN=$RUN_BOOTSTRAP_BIN
|
|||||||
PLATFORM_BUILDER_DOCKER_BINARY=$PLATFORM_BUILDER_DOCKER_BINARY
|
PLATFORM_BUILDER_DOCKER_BINARY=$PLATFORM_BUILDER_DOCKER_BINARY
|
||||||
PLATFORM_BUILDER_IMAGE=$PLATFORM_BUILDER_IMAGE
|
PLATFORM_BUILDER_IMAGE=$PLATFORM_BUILDER_IMAGE
|
||||||
PLATFORM_BUILDER_SOURCE_DIR=$PLATFORM_BUILDER_SOURCE_DIR
|
PLATFORM_BUILDER_SOURCE_DIR=$PLATFORM_BUILDER_SOURCE_DIR
|
||||||
|
PLATFORM_BUILDER_SOURCE_REPOSITORY=$PLATFORM_BUILDER_SOURCE_REPOSITORY
|
||||||
|
PLATFORM_BUILDER_SOURCE_REVISION=$PLATFORM_BUILDER_SOURCE_REVISION
|
||||||
PLATFORM_BUILDER_WORKSPACE_DIR=$PLATFORM_BUILDER_WORKSPACE_DIR
|
PLATFORM_BUILDER_WORKSPACE_DIR=$PLATFORM_BUILDER_WORKSPACE_DIR
|
||||||
PLATFORM_BUILDER_CACHE_DIR=$PLATFORM_BUILDER_CACHE_DIR
|
PLATFORM_BUILDER_CACHE_DIR=$PLATFORM_BUILDER_CACHE_DIR
|
||||||
PLATFORM_BUILDER_TIMEOUT_SECONDS=$PLATFORM_BUILDER_TIMEOUT_SECONDS
|
PLATFORM_BUILDER_TIMEOUT_SECONDS=$PLATFORM_BUILDER_TIMEOUT_SECONDS
|
||||||
@@ -277,6 +279,8 @@ start_self_hosted_stack() {
|
|||||||
PLATFORM_BUILDER_DOCKER_BINARY="$PLATFORM_BUILDER_DOCKER_BINARY" \
|
PLATFORM_BUILDER_DOCKER_BINARY="$PLATFORM_BUILDER_DOCKER_BINARY" \
|
||||||
PLATFORM_BUILDER_IMAGE="$PLATFORM_BUILDER_IMAGE" \
|
PLATFORM_BUILDER_IMAGE="$PLATFORM_BUILDER_IMAGE" \
|
||||||
PLATFORM_BUILDER_SOURCE_DIR="$PLATFORM_BUILDER_SOURCE_DIR" \
|
PLATFORM_BUILDER_SOURCE_DIR="$PLATFORM_BUILDER_SOURCE_DIR" \
|
||||||
|
PLATFORM_BUILDER_SOURCE_REPOSITORY="$PLATFORM_BUILDER_SOURCE_REPOSITORY" \
|
||||||
|
PLATFORM_BUILDER_SOURCE_REVISION="$PLATFORM_BUILDER_SOURCE_REVISION" \
|
||||||
PLATFORM_BUILDER_WORKSPACE_DIR="$PLATFORM_BUILDER_WORKSPACE_DIR" \
|
PLATFORM_BUILDER_WORKSPACE_DIR="$PLATFORM_BUILDER_WORKSPACE_DIR" \
|
||||||
PLATFORM_BUILDER_CACHE_DIR="$PLATFORM_BUILDER_CACHE_DIR" \
|
PLATFORM_BUILDER_CACHE_DIR="$PLATFORM_BUILDER_CACHE_DIR" \
|
||||||
PLATFORM_BUILDER_TIMEOUT_SECONDS="$PLATFORM_BUILDER_TIMEOUT_SECONDS" \
|
PLATFORM_BUILDER_TIMEOUT_SECONDS="$PLATFORM_BUILDER_TIMEOUT_SECONDS" \
|
||||||
|
|||||||
@@ -193,7 +193,7 @@ printf 'platform: %s\n' "$(local_debug_platform_url)"
|
|||||||
printf 'platform_web: %s\n' "$(local_debug_web_url)"
|
printf 'platform_web: %s\n' "$(local_debug_web_url)"
|
||||||
printf 'run source: %s\n' "$RUN_SOURCE_DIR"
|
printf 'run source: %s\n' "$RUN_SOURCE_DIR"
|
||||||
printf 'run build bucket: %s\n' "$RUN_BUILD_BUCKET_ROOT"
|
printf 'run build bucket: %s\n' "$RUN_BUILD_BUCKET_ROOT"
|
||||||
printf 'run build source snapshot: %s\n' "$RUN_BUILD_SOURCE_ROOT"
|
printf 'run source checkout: %s\n' "$RUN_SOURCE_DIR"
|
||||||
printf 'run bootstrap binary: %s\n' "$RUN_BOOTSTRAP_BIN"
|
printf 'run bootstrap binary: %s\n' "$RUN_BOOTSTRAP_BIN"
|
||||||
printf 'platform builder image: %s\n' "$PLATFORM_BUILDER_IMAGE"
|
printf 'platform builder image: %s\n' "$PLATFORM_BUILDER_IMAGE"
|
||||||
printf 'platform builder workspace: %s\n' "$PLATFORM_BUILDER_WORKSPACE_DIR"
|
printf 'platform builder workspace: %s\n' "$PLATFORM_BUILDER_WORKSPACE_DIR"
|
||||||
@@ -221,6 +221,8 @@ start_service platform "$ROOT_DIR/platform" env \
|
|||||||
PLATFORM_BUILDER_DOCKER_BINARY="$PLATFORM_BUILDER_DOCKER_BINARY" \
|
PLATFORM_BUILDER_DOCKER_BINARY="$PLATFORM_BUILDER_DOCKER_BINARY" \
|
||||||
PLATFORM_BUILDER_IMAGE="$PLATFORM_BUILDER_IMAGE" \
|
PLATFORM_BUILDER_IMAGE="$PLATFORM_BUILDER_IMAGE" \
|
||||||
PLATFORM_BUILDER_SOURCE_DIR="$PLATFORM_BUILDER_SOURCE_DIR" \
|
PLATFORM_BUILDER_SOURCE_DIR="$PLATFORM_BUILDER_SOURCE_DIR" \
|
||||||
|
PLATFORM_BUILDER_SOURCE_REPOSITORY="$PLATFORM_BUILDER_SOURCE_REPOSITORY" \
|
||||||
|
PLATFORM_BUILDER_SOURCE_REVISION="$PLATFORM_BUILDER_SOURCE_REVISION" \
|
||||||
PLATFORM_BUILDER_WORKSPACE_DIR="$PLATFORM_BUILDER_WORKSPACE_DIR" \
|
PLATFORM_BUILDER_WORKSPACE_DIR="$PLATFORM_BUILDER_WORKSPACE_DIR" \
|
||||||
PLATFORM_BUILDER_CACHE_DIR="$PLATFORM_BUILDER_CACHE_DIR" \
|
PLATFORM_BUILDER_CACHE_DIR="$PLATFORM_BUILDER_CACHE_DIR" \
|
||||||
PLATFORM_BUILDER_TIMEOUT_SECONDS="$PLATFORM_BUILDER_TIMEOUT_SECONDS" \
|
PLATFORM_BUILDER_TIMEOUT_SECONDS="$PLATFORM_BUILDER_TIMEOUT_SECONDS" \
|
||||||
|
|||||||
Reference in New Issue
Block a user