Clarify live relay run logging
This commit is contained in:
npc0-hue
@@ -8,11 +8,11 @@ Do not rebuild the old all-in-one WebSocket model. Keep these workloads separate
|
||||
|
||||
- Control: hello, heartbeat, version, capabilities, capacity.
|
||||
- Job: claim, ack, progress, result, cancel, reconcile.
|
||||
- Logs: local spool, batch upload, sequence acknowledgement, retry.
|
||||
- Logs: current supervised stdout/stderr live relay through `/api/v1/run/logs/relay`; no local durable log spool, cache, resend backlog, or sequence-ack dependency.
|
||||
- Artifacts: chunks, checksums, resume, throttling.
|
||||
- Game client bridge: optional in-game command/snapshot channel.
|
||||
|
||||
Artifact transfer must not block control heartbeats, job result reporting, or log upload.
|
||||
Artifact transfer must not block control heartbeats, job result reporting, or current live log relay.
|
||||
|
||||
## Structure Rules
|
||||
|
||||
@@ -30,6 +30,6 @@ Run must not contain game-specific lifecycle logic. Do not add hardcoded game na
|
||||
|
||||
If a lifecycle flow needs "install if missing", "stop before update", "validate via SteamCMD", or "start with game-specific arguments", run should execute the plugin-declared action through generic capabilities. Fix missing generic capabilities in run when necessary, but keep the game policy and concrete commands in the plugin action assets.
|
||||
|
||||
Run should supervise plugin-declared start commands generically: hide started process windows when the operating system supports it, capture the supervised stdout/stderr streams, and upload those streams through log ingest. Do not replace plugin-declared process output with game-specific file inspection.
|
||||
Run should supervise plugin-declared start commands generically: hide started process windows when the operating system supports it, capture the supervised stdout/stderr streams, and push only current observed output through live log relay. Do not persist supervised process output in a local log database/cache, do not wait for platform delivery acknowledgement, and do not replace plugin-declared process output with game-specific file inspection.
|
||||
|
||||
Do not introduce per-game runtime files such as `scum_deployment.go`. Prefer generic action execution, dependency helpers, process supervision, and manifest-declared capability checks that any plugin can use.
|
||||
|
||||
@@ -6,7 +6,7 @@ Machine-side executor for scoped server operations.
|
||||
|
||||
- Register with platform and report heartbeat, version, capabilities, and capacity.
|
||||
- Claim and execute jobs for server lifecycle, file/config work, backups, updates, and bounded database or command work.
|
||||
- Collect server logs into local spool and upload acknowledged batches.
|
||||
- Relay current supervised stdout/stderr to platform live subscribers without local durable log spool, cache, resend backlog, or delivery acknowledgement dependency.
|
||||
- Transfer artifacts with chunking, checksums, resume, throttling, and low priority.
|
||||
- Optionally coordinate with a game client bridge when a specific game requires in-game commands or snapshots.
|
||||
|
||||
@@ -18,13 +18,13 @@ Implementation should use dedicated directories for:
|
||||
- `protocol/`: control, job, log, artifact, and game-client bridge DTOs.
|
||||
- `domain/`: executor domain types.
|
||||
- `runtime/`: local execution and server process orchestration.
|
||||
- `spool/`: local durable log/job/artifact queues.
|
||||
- `spool/`: local artifact queues plus legacy compatibility utilities; current supervised process logs must use live relay, not durable local spool/cache.
|
||||
- `artifact/`: chunk transfer implementation.
|
||||
- `logingest/`: log collectors and uploaders.
|
||||
- `config/`: configuration structures and loading.
|
||||
- `shared/`: small shared helpers.
|
||||
|
||||
Logs and artifacts must have separate queues and priority controls.
|
||||
Live logs and artifacts must stay separate; artifact queues must not delay current log relay.
|
||||
|
||||
## Development Baseline
|
||||
|
||||
@@ -72,7 +72,7 @@ In Docker, `RUN_PLATFORM_URL` must be `http://platform:8080` because `platform`
|
||||
|
||||
Current executable behavior includes smoke mode plus worker mode. Worker mode registers with platform, opens a signed persistent control event stream for lightweight wakeups, sends heartbeat metadata, claims lifecycle jobs, acknowledges leases, reports bounded progress, executes scoped `process.install`, `process.start`, and `process.stop` command templates inside per-server workspaces, polls cancellation, submits terminal results, and reconciles active jobs.
|
||||
|
||||
Lifecycle templates are JSON files addressed by logical keys under the server workspace. They resolve to direct executable/argument vectors, not shell strings. Absolute paths, parent traversal, raw credentials, direct sockets, shell launchers, unsafe environment keys, and unsafe output are rejected or redacted. Plugin-declared Windows `.cmd` and `.bat` assets are launched through a bounded `cmd.exe` adapter and remain under the same process supervisor. Process identity journals are namespaced by Run endpoint, server, plugin, and component profile, so multiple Run services cannot overwrite one another; a restart migrates matching legacy state, reopens the persisted output files, reconciles the PID, and resumes stdout/stderr tailing. Process stdout/stderr is written to the log spool, and lifecycle result metadata is queued through artifact hooks so control heartbeat and job result submission stay independent from log and artifact work.
|
||||
Lifecycle templates are JSON files addressed by logical keys under the server workspace. They resolve to direct executable/argument vectors, not shell strings. Absolute paths, parent traversal, raw credentials, direct sockets, shell launchers, unsafe environment keys, and unsafe output are rejected or redacted. Plugin-declared Windows `.cmd` and `.bat` assets are launched through a bounded `cmd.exe` adapter and remain under the same process supervisor. Process identity journals are namespaced by Run endpoint, server, plugin, and component profile, so multiple Run services cannot overwrite one another; a restart migrates matching legacy process state and resumes observing stdout/stderr from the current supervised process. Process stdout/stderr is pushed through best-effort live relay, and lifecycle result metadata is queued through artifact hooks so control heartbeat and job result submission stay independent from log and artifact work.
|
||||
|
||||
The control event stream carries only small hints such as `control.ready`, `control.heartbeat`, and `job.changed`. It never carries assignments, logs, artifact chunks, file bodies, host paths, credentials, or direct sockets; Run still fetches work through the durable job claim channel after a wake event.
|
||||
|
||||
@@ -94,6 +94,6 @@ Worker mode now dispatches distribution capabilities in addition to lifecycle wo
|
||||
- `dependencies.install`: executes only typed install plans addressed under `dependencies/install/...`; arbitrary shell snippets are rejected before execution.
|
||||
- `logs.backfill`: advances historical log cursors for declared sources and returns a cursor/result artifact ref instead of embedding large log bodies in job results.
|
||||
|
||||
Declared file log sources use a tailer with offset checkpoints and redaction before entries enter the durable log channel. FTP/rsync, SQL read, RCON command, and file transfer adapters are represented as bounded envelopes with scoped input or artifact refs. Long transfers remain lower priority than heartbeat, job ack/result, cancellation polling, reconcile, and log acknowledgement.
|
||||
Declared file log sources used by explicit maintenance jobs remain bounded and redacted, but current supervised process output uses live relay only. FTP/rsync, SQL read, RCON command, and file transfer adapters are represented as bounded envelopes with scoped input or artifact refs. Long transfers remain lower priority than heartbeat, job ack/result, cancellation polling, reconcile, and current live log relay.
|
||||
|
||||
Protected SQL, RCON, and management-program requests use a separate signed one-time input route after Run claims a single-attempt fenced job. Run rechecks approval, expiry, server/endpoint/fence, capability kind, and logical transport/target bindings before dispatching to a local handler. Request text, private connection configuration, and response bodies do not enter assignments, journals, or terminal results. Management-program stdout/stderr is redacted into the durable `management-program` log source rather than file execution logs. See [`protocol/protected-request.md`](protocol/protected-request.md).
|
||||
Protected SQL, RCON, and management-program requests use a separate signed one-time input route after Run claims a single-attempt fenced job. Run rechecks approval, expiry, server/endpoint/fence, capability kind, and logical transport/target bindings before dispatching to a local handler. Request text, private connection configuration, and response bodies do not enter assignments, journals, or terminal results. Management-program stdout/stderr is redacted before live relay and must not be stored as a Run-owned durable log body. See [`protocol/protected-request.md`](protocol/protected-request.md).
|
||||
|
||||
+1
-1
@@ -59,7 +59,7 @@ func main() {
|
||||
}
|
||||
|
||||
if cfg.Mode == "worker" {
|
||||
log.Printf("RUN phase=startup status=worker platform=%s endpoint=%s server=%s plugin=%s workspace=%s spool=%s maxJobs=%d", diagnosticPlatformAddress(cfg.PlatformURL), diagnosticValue(cfg.RunEndpointID), diagnosticValue(cfg.ServerInstanceID), diagnosticValue(cfg.PluginID), diagnosticValue(cfg.WorkspaceRoot), diagnosticValue(cfg.SpoolRoot), cfg.MaxJobs)
|
||||
log.Printf("RUN phase=startup status=worker platform=%s endpoint=%s server=%s plugin=%s workspace=%s artifactQueue=%s maxJobs=%d", diagnosticPlatformAddress(cfg.PlatformURL), diagnosticValue(cfg.RunEndpointID), diagnosticValue(cfg.ServerInstanceID), diagnosticValue(cfg.PluginID), diagnosticValue(cfg.WorkspaceRoot), diagnosticValue(cfg.SpoolRoot), cfg.MaxJobs)
|
||||
log.Printf("RUN phase=workspace_seed status=materializing present=%t workspace=%s componentKey=%s", cfg.WorkspaceSeed != "", diagnosticValue(cfg.WorkspaceRoot), diagnosticValue(cfg.ComponentKey))
|
||||
if err := runruntime.MaterializeWorkspaceSeed(cfg); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "initialize plugin workspace assets: %v\n", err)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Run Artifact Contract
|
||||
|
||||
Artifacts move files and large payloads between platform and run without blocking logs or control.
|
||||
Artifacts move files and large payloads between platform and run without blocking current live logs or control.
|
||||
|
||||
## Implemented Routes
|
||||
|
||||
@@ -34,9 +34,9 @@ Run stores unacknowledged `ArtifactChunkUploadRequest` payloads in the local art
|
||||
- Transfers must be resumable.
|
||||
- Transfers must be checksummed.
|
||||
- Artifact concurrency must be limited.
|
||||
- Artifact transfer must not block control heartbeat, job ack/result, or log upload.
|
||||
- Artifact transfer is lower priority than control, job lifecycle metadata, and durable log ingest.
|
||||
- Slow or retrying artifact chunks must not prevent log spool acknowledgement cleanup or terminal job result submission.
|
||||
- Artifact transfer must not block control heartbeat, job ack/result, or current live log relay.
|
||||
- Artifact transfer is lower priority than control, job lifecycle metadata, and current live log relay.
|
||||
- Slow or retrying artifact chunks must not delay live log relay or terminal job result submission.
|
||||
- Control, job, and log routes must reject artifact chunk payloads or transport details rather than accepting them through lightweight channel payloads.
|
||||
|
||||
Run artifact queues use owner-only atomic JSON entries and retain chunks until an exact transfer/artifact/index acknowledgement covers them. A low-priority uploader retries pending chunks independently of control, jobs, and logs.
|
||||
|
||||
+1
-1
@@ -24,7 +24,7 @@ Control is the lightweight high-priority channel between run and platform.
|
||||
- Control payloads must be small.
|
||||
- Control events wake Run only; they must not carry job assignments or execution input.
|
||||
- Control must not carry logs, artifact chunks, or long job result bodies.
|
||||
- Control must have priority over job execution, log upload, and artifact transfer.
|
||||
- Control must have priority over job execution, current live log relay, and artifact transfer.
|
||||
- Heartbeat capacity summaries must remain metadata-only and must not mention or carry heavy channel payloads.
|
||||
|
||||
## Deferred Channels
|
||||
|
||||
@@ -18,9 +18,9 @@ channel. Game plugins own durable log storage and analysis.
|
||||
|
||||
- `LogBatchIngestRequest`: run ID, session token, server instance ID, stream ID, source, sequence range, compression metadata, checksum, and bounded entries.
|
||||
- `LogEntry`: sequence, timestamp, level, line, parser metadata, and redaction state.
|
||||
- `LogBatchIngestResponse`: accepted sequence range, latest acknowledged sequence, duplicate flag, retry hint, and server time.
|
||||
- `LogBatchIngestResponse`: accepted sequence range, latest stream metadata sequence, compatibility duplicate/retry fields, and server time.
|
||||
- `LogStreamCursorRequest`: stream ID, sequence cursor, and limit.
|
||||
- `LogStreamCursorResponse`: ordered entries, next cursor, and latest acknowledged sequence.
|
||||
- `LogStreamCursorResponse`: compatibility ordered entries, next cursor, and latest stored metadata sequence.
|
||||
- `LogStreamEventResponse`: safe browser event containing server ID, stream metadata, latest sequence, and one log entry.
|
||||
|
||||
Autonomous lifecycle output uses
|
||||
|
||||
@@ -33,6 +33,6 @@ diagnostics without forwarding handler errors or response bodies. These errors
|
||||
affect only the current request.
|
||||
|
||||
Management-program stdout and stderr are bounded, redacted, and sent to the
|
||||
durable log channel with source `management-program` and streams
|
||||
live log relay with source `management-program` and streams
|
||||
`management-program.stdout` / `management-program.stderr`. They are not file
|
||||
execution logs and are never embedded in job result content.
|
||||
|
||||
@@ -542,7 +542,7 @@ func (supervisor *OSManagedProcessSupervisor) tailOutput(ctx context.Context, ta
|
||||
log.Printf("RUN phase=process.managed.output status=line job=%s pid=%d stream=%s line=%q", safeOptional(identity.JobID), identity.PID, stream, RedactText(text))
|
||||
break
|
||||
} else {
|
||||
log.Printf("RUN phase=process.managed.output status=spool_retry job=%s pid=%d stream=%s error=%s", safeOptional(identity.JobID), identity.PID, stream, RedactText(sinkErr.Error()))
|
||||
log.Printf("RUN phase=process.managed.output status=sink_retry job=%s pid=%d stream=%s error=%s", safeOptional(identity.JobID), identity.PID, stream, RedactText(sinkErr.Error()))
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
|
||||
Reference in New Issue
Block a user