From 65da73eff0205c3217862c77f489c036008c2e4e Mon Sep 17 00:00:00 2001 From: npc0-hue Date: Tue, 1 Sep 2026 14:23:58 +0800 Subject: [PATCH] Clarify live relay run logging --- AGENTS.md | 6 +++--- README.md | 12 ++++++------ cmd/run/main.go | 2 +- protocol/artifact.md | 8 ++++---- protocol/control.md | 2 +- protocol/log-ingest.md | 4 ++-- protocol/protected-request.md | 2 +- runtime/process_supervisor.go | 2 +- 8 files changed, 19 insertions(+), 19 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 38b156e..654252a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,11 +8,11 @@ Do not rebuild the old all-in-one WebSocket model. Keep these workloads separate - Control: hello, heartbeat, version, capabilities, capacity. - Job: claim, ack, progress, result, cancel, reconcile. -- Logs: local spool, batch upload, sequence acknowledgement, retry. +- Logs: current supervised stdout/stderr live relay through `/api/v1/run/logs/relay`; no local durable log spool, cache, resend backlog, or sequence-ack dependency. - Artifacts: chunks, checksums, resume, throttling. - Game client bridge: optional in-game command/snapshot channel. -Artifact transfer must not block control heartbeats, job result reporting, or log upload. +Artifact transfer must not block control heartbeats, job result reporting, or current live log relay. ## Structure Rules @@ -30,6 +30,6 @@ Run must not contain game-specific lifecycle logic. Do not add hardcoded game na If a lifecycle flow needs "install if missing", "stop before update", "validate via SteamCMD", or "start with game-specific arguments", run should execute the plugin-declared action through generic capabilities. Fix missing generic capabilities in run when necessary, but keep the game policy and concrete commands in the plugin action assets. -Run should supervise plugin-declared start commands generically: hide started process windows when the operating system supports it, capture the supervised stdout/stderr streams, and upload those streams through log ingest. Do not replace plugin-declared process output with game-specific file inspection. +Run should supervise plugin-declared start commands generically: hide started process windows when the operating system supports it, capture the supervised stdout/stderr streams, and push only current observed output through live log relay. Do not persist supervised process output in a local log database/cache, do not wait for platform delivery acknowledgement, and do not replace plugin-declared process output with game-specific file inspection. Do not introduce per-game runtime files such as `scum_deployment.go`. Prefer generic action execution, dependency helpers, process supervision, and manifest-declared capability checks that any plugin can use. diff --git a/README.md b/README.md index 36cb1b4..b43d958 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ Machine-side executor for scoped server operations. - Register with platform and report heartbeat, version, capabilities, and capacity. - Claim and execute jobs for server lifecycle, file/config work, backups, updates, and bounded database or command work. -- Collect server logs into local spool and upload acknowledged batches. +- Relay current supervised stdout/stderr to platform live subscribers without local durable log spool, cache, resend backlog, or delivery acknowledgement dependency. - Transfer artifacts with chunking, checksums, resume, throttling, and low priority. - Optionally coordinate with a game client bridge when a specific game requires in-game commands or snapshots. @@ -18,13 +18,13 @@ Implementation should use dedicated directories for: - `protocol/`: control, job, log, artifact, and game-client bridge DTOs. - `domain/`: executor domain types. - `runtime/`: local execution and server process orchestration. -- `spool/`: local durable log/job/artifact queues. +- `spool/`: local artifact queues plus legacy compatibility utilities; current supervised process logs must use live relay, not durable local spool/cache. - `artifact/`: chunk transfer implementation. - `logingest/`: log collectors and uploaders. - `config/`: configuration structures and loading. - `shared/`: small shared helpers. -Logs and artifacts must have separate queues and priority controls. +Live logs and artifacts must stay separate; artifact queues must not delay current log relay. ## Development Baseline @@ -72,7 +72,7 @@ In Docker, `RUN_PLATFORM_URL` must be `http://platform:8080` because `platform` Current executable behavior includes smoke mode plus worker mode. Worker mode registers with platform, opens a signed persistent control event stream for lightweight wakeups, sends heartbeat metadata, claims lifecycle jobs, acknowledges leases, reports bounded progress, executes scoped `process.install`, `process.start`, and `process.stop` command templates inside per-server workspaces, polls cancellation, submits terminal results, and reconciles active jobs. -Lifecycle templates are JSON files addressed by logical keys under the server workspace. They resolve to direct executable/argument vectors, not shell strings. Absolute paths, parent traversal, raw credentials, direct sockets, shell launchers, unsafe environment keys, and unsafe output are rejected or redacted. Plugin-declared Windows `.cmd` and `.bat` assets are launched through a bounded `cmd.exe` adapter and remain under the same process supervisor. Process identity journals are namespaced by Run endpoint, server, plugin, and component profile, so multiple Run services cannot overwrite one another; a restart migrates matching legacy state, reopens the persisted output files, reconciles the PID, and resumes stdout/stderr tailing. Process stdout/stderr is written to the log spool, and lifecycle result metadata is queued through artifact hooks so control heartbeat and job result submission stay independent from log and artifact work. +Lifecycle templates are JSON files addressed by logical keys under the server workspace. They resolve to direct executable/argument vectors, not shell strings. Absolute paths, parent traversal, raw credentials, direct sockets, shell launchers, unsafe environment keys, and unsafe output are rejected or redacted. Plugin-declared Windows `.cmd` and `.bat` assets are launched through a bounded `cmd.exe` adapter and remain under the same process supervisor. Process identity journals are namespaced by Run endpoint, server, plugin, and component profile, so multiple Run services cannot overwrite one another; a restart migrates matching legacy process state and resumes observing stdout/stderr from the current supervised process. Process stdout/stderr is pushed through best-effort live relay, and lifecycle result metadata is queued through artifact hooks so control heartbeat and job result submission stay independent from log and artifact work. The control event stream carries only small hints such as `control.ready`, `control.heartbeat`, and `job.changed`. It never carries assignments, logs, artifact chunks, file bodies, host paths, credentials, or direct sockets; Run still fetches work through the durable job claim channel after a wake event. @@ -94,6 +94,6 @@ Worker mode now dispatches distribution capabilities in addition to lifecycle wo - `dependencies.install`: executes only typed install plans addressed under `dependencies/install/...`; arbitrary shell snippets are rejected before execution. - `logs.backfill`: advances historical log cursors for declared sources and returns a cursor/result artifact ref instead of embedding large log bodies in job results. -Declared file log sources use a tailer with offset checkpoints and redaction before entries enter the durable log channel. FTP/rsync, SQL read, RCON command, and file transfer adapters are represented as bounded envelopes with scoped input or artifact refs. Long transfers remain lower priority than heartbeat, job ack/result, cancellation polling, reconcile, and log acknowledgement. +Declared file log sources used by explicit maintenance jobs remain bounded and redacted, but current supervised process output uses live relay only. FTP/rsync, SQL read, RCON command, and file transfer adapters are represented as bounded envelopes with scoped input or artifact refs. Long transfers remain lower priority than heartbeat, job ack/result, cancellation polling, reconcile, and current live log relay. -Protected SQL, RCON, and management-program requests use a separate signed one-time input route after Run claims a single-attempt fenced job. Run rechecks approval, expiry, server/endpoint/fence, capability kind, and logical transport/target bindings before dispatching to a local handler. Request text, private connection configuration, and response bodies do not enter assignments, journals, or terminal results. Management-program stdout/stderr is redacted into the durable `management-program` log source rather than file execution logs. See [`protocol/protected-request.md`](protocol/protected-request.md). +Protected SQL, RCON, and management-program requests use a separate signed one-time input route after Run claims a single-attempt fenced job. Run rechecks approval, expiry, server/endpoint/fence, capability kind, and logical transport/target bindings before dispatching to a local handler. Request text, private connection configuration, and response bodies do not enter assignments, journals, or terminal results. Management-program stdout/stderr is redacted before live relay and must not be stored as a Run-owned durable log body. See [`protocol/protected-request.md`](protocol/protected-request.md). diff --git a/cmd/run/main.go b/cmd/run/main.go index 11a0fe2..7a28967 100644 --- a/cmd/run/main.go +++ b/cmd/run/main.go @@ -59,7 +59,7 @@ func main() { } if cfg.Mode == "worker" { - log.Printf("RUN phase=startup status=worker platform=%s endpoint=%s server=%s plugin=%s workspace=%s spool=%s maxJobs=%d", diagnosticPlatformAddress(cfg.PlatformURL), diagnosticValue(cfg.RunEndpointID), diagnosticValue(cfg.ServerInstanceID), diagnosticValue(cfg.PluginID), diagnosticValue(cfg.WorkspaceRoot), diagnosticValue(cfg.SpoolRoot), cfg.MaxJobs) + log.Printf("RUN phase=startup status=worker platform=%s endpoint=%s server=%s plugin=%s workspace=%s artifactQueue=%s maxJobs=%d", diagnosticPlatformAddress(cfg.PlatformURL), diagnosticValue(cfg.RunEndpointID), diagnosticValue(cfg.ServerInstanceID), diagnosticValue(cfg.PluginID), diagnosticValue(cfg.WorkspaceRoot), diagnosticValue(cfg.SpoolRoot), cfg.MaxJobs) log.Printf("RUN phase=workspace_seed status=materializing present=%t workspace=%s componentKey=%s", cfg.WorkspaceSeed != "", diagnosticValue(cfg.WorkspaceRoot), diagnosticValue(cfg.ComponentKey)) if err := runruntime.MaterializeWorkspaceSeed(cfg); err != nil { fmt.Fprintf(os.Stderr, "initialize plugin workspace assets: %v\n", err) diff --git a/protocol/artifact.md b/protocol/artifact.md index ee8f5cc..fcef861 100644 --- a/protocol/artifact.md +++ b/protocol/artifact.md @@ -1,6 +1,6 @@ # Run Artifact Contract -Artifacts move files and large payloads between platform and run without blocking logs or control. +Artifacts move files and large payloads between platform and run without blocking current live logs or control. ## Implemented Routes @@ -34,9 +34,9 @@ Run stores unacknowledged `ArtifactChunkUploadRequest` payloads in the local art - Transfers must be resumable. - Transfers must be checksummed. - Artifact concurrency must be limited. -- Artifact transfer must not block control heartbeat, job ack/result, or log upload. -- Artifact transfer is lower priority than control, job lifecycle metadata, and durable log ingest. -- Slow or retrying artifact chunks must not prevent log spool acknowledgement cleanup or terminal job result submission. +- Artifact transfer must not block control heartbeat, job ack/result, or current live log relay. +- Artifact transfer is lower priority than control, job lifecycle metadata, and current live log relay. +- Slow or retrying artifact chunks must not delay live log relay or terminal job result submission. - Control, job, and log routes must reject artifact chunk payloads or transport details rather than accepting them through lightweight channel payloads. Run artifact queues use owner-only atomic JSON entries and retain chunks until an exact transfer/artifact/index acknowledgement covers them. A low-priority uploader retries pending chunks independently of control, jobs, and logs. diff --git a/protocol/control.md b/protocol/control.md index 0c26c09..951c681 100644 --- a/protocol/control.md +++ b/protocol/control.md @@ -24,7 +24,7 @@ Control is the lightweight high-priority channel between run and platform. - Control payloads must be small. - Control events wake Run only; they must not carry job assignments or execution input. - Control must not carry logs, artifact chunks, or long job result bodies. -- Control must have priority over job execution, log upload, and artifact transfer. +- Control must have priority over job execution, current live log relay, and artifact transfer. - Heartbeat capacity summaries must remain metadata-only and must not mention or carry heavy channel payloads. ## Deferred Channels diff --git a/protocol/log-ingest.md b/protocol/log-ingest.md index b0e5191..58da6e9 100644 --- a/protocol/log-ingest.md +++ b/protocol/log-ingest.md @@ -18,9 +18,9 @@ channel. Game plugins own durable log storage and analysis. - `LogBatchIngestRequest`: run ID, session token, server instance ID, stream ID, source, sequence range, compression metadata, checksum, and bounded entries. - `LogEntry`: sequence, timestamp, level, line, parser metadata, and redaction state. -- `LogBatchIngestResponse`: accepted sequence range, latest acknowledged sequence, duplicate flag, retry hint, and server time. +- `LogBatchIngestResponse`: accepted sequence range, latest stream metadata sequence, compatibility duplicate/retry fields, and server time. - `LogStreamCursorRequest`: stream ID, sequence cursor, and limit. -- `LogStreamCursorResponse`: ordered entries, next cursor, and latest acknowledged sequence. +- `LogStreamCursorResponse`: compatibility ordered entries, next cursor, and latest stored metadata sequence. - `LogStreamEventResponse`: safe browser event containing server ID, stream metadata, latest sequence, and one log entry. Autonomous lifecycle output uses diff --git a/protocol/protected-request.md b/protocol/protected-request.md index 7f119bd..6310f57 100644 --- a/protocol/protected-request.md +++ b/protocol/protected-request.md @@ -33,6 +33,6 @@ diagnostics without forwarding handler errors or response bodies. These errors affect only the current request. Management-program stdout and stderr are bounded, redacted, and sent to the -durable log channel with source `management-program` and streams +live log relay with source `management-program` and streams `management-program.stdout` / `management-program.stderr`. They are not file execution logs and are never embedded in job result content. diff --git a/runtime/process_supervisor.go b/runtime/process_supervisor.go index 4b04a2f..5f6d2e8 100644 --- a/runtime/process_supervisor.go +++ b/runtime/process_supervisor.go @@ -542,7 +542,7 @@ func (supervisor *OSManagedProcessSupervisor) tailOutput(ctx context.Context, ta log.Printf("RUN phase=process.managed.output status=line job=%s pid=%d stream=%s line=%q", safeOptional(identity.JobID), identity.PID, stream, RedactText(text)) break } else { - log.Printf("RUN phase=process.managed.output status=spool_retry job=%s pid=%d stream=%s error=%s", safeOptional(identity.JobID), identity.PID, stream, RedactText(sinkErr.Error())) + log.Printf("RUN phase=process.managed.output status=sink_retry job=%s pid=%d stream=%s error=%s", safeOptional(identity.JobID), identity.PID, stream, RedactText(sinkErr.Error())) } select { case <-ctx.Done():