admin343/browser
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Restore durable log ingest and typed plugin projections

Browse Source
This commit is contained in:
npc0-hue
2026-09-02 10:20:30 +08:00
parent 40ac46ba17
commit 6018d8f0fc
61 changed files with 809 additions and 3157 deletions
Download Patch File Download Diff File Expand all files Collapse all files
platform/protocol/run-contracts.md
+12 -15
View File
@@ -23,7 +23,7 @@ Named control DTOs:
Control payloads must remain small and must not include logs, artifact chunks, host paths, raw credentials, direct sockets, job assignments, execution input, or long task results. Hello creates or updates run endpoint metadata and issues an in-memory platform session token. A server-scoped generated Run must use the endpoint identity reserved for its server; Platform rejects a valid component key presented for another endpoint. Registration is binding/authentication only for generated Run bootstrap and must not enqueue lifecycle or status jobs merely because Run appeared. Heartbeat requires that active session token and may request capability refresh when the fingerprint changes. The control event stream is a signed Run-only `text/event-stream` wake channel; events such as `job.changed` only tell Run to claim durable work through `/run/jobs/claim`.
Capacity reports include bounded `maxJobs`, `runningJobs`, `queuedJobs`, compatibility `logBacklogBatches`, `artifactBacklogChunks`, and enumerated pressure codes. Current Run implementations must not use `logBacklogBatches` as a durable live-log spool; capacity reports never include log bodies, artifact chunks, machine paths, PIDs, sockets, credentials, or transport endpoints.
Capacity reports include bounded `maxJobs`, `runningJobs`, `queuedJobs`, `logBacklogBatches`, `artifactBacklogChunks`, and enumerated pressure codes. They report queue and spool counts only, never log bodies, artifact chunks, machine paths, PIDs, sockets, credentials, or transport endpoints.
Control is the highest-priority run/platform path. Artifact/file transfer load must not delay heartbeat acceptance or mutate heartbeat capacity state through heavy payload fields.
@@ -65,34 +65,31 @@ The plan is build input for the generated package, not a machine-side job-channe
Autonomous lifecycle reports use `POST /api/v1/run/lifecycle/report` with the active Run session and signed envelope when required. The route accepts only bounded terminal lifecycle facts for `process.install`, `process.start`, `process.stop`, or `process.status`; it validates the server/run binding, records bounded evidence, and projects server state from Run-reported process facts without creating or completing a Platform job. A managed-process report includes an opaque `managedProcessId`, monotonic `observationSeq`, and `observedAt`; retries are idempotent and a lower sequence cannot regress a newer fact for that process.
## Live Log Relay And Compatibility Log Ingest
## Log Ingest
Implemented HTTP JSON routes:
- `POST /api/v1/run/logs/relay`
- `POST /api/v1/run/logs/batches`
- `GET /api/v1/server-instances/{id}/logs/events`
- `POST /api/v1/game-client-bridge/companion/logs/events`
- `POST /api/v1/log-streams/query`
Named log DTOs:
- `LogBatchIngestRequest`
- `LogBatchIngestResponse`
- `RunLogStreamProgressRequest` / `RunLogStreamProgressResponse`: compatibility signed Run-only cursor metadata for older durable-ingest streams. Current live relay does not depend on this route for delivery or progress.
- `RunLogStreamProgressRequest` / `RunLogStreamProgressResponse`: signed Run-only sequence recovery for a server-bound `run.<endpoint>.<server>.*` stream. The response contains only the latest acknowledged sequence.
- `LogEntry`
- `LogStreamCursorRequest`
- `LogStreamCursorResponse`
- `LogStreamEventResponse`
Current Run output uses `POST /api/v1/run/logs/relay`. The route validates the active Run session and stream binding, updates only bounded stream metadata such as latest sequence/session identity, and immediately fans entries out to live subscribers. It does not persist log bodies, does not create a resend backlog, does not require platform sequence acknowledgements for progress, and does not block lifecycle/control/job work on delivery success.
Log ingest supports bounded batches, sequence ranges, checksum validation, retry-safe duplicate acknowledgement, latest sequence tracking, cursor query, and browser SSE fan-out from already-ingested platform logs. Log payloads must not carry artifact chunks, host paths, raw credentials, direct sockets, or unbounded inline data.
Server terminal streaming uses `GET /api/v1/server-instances/{id}/logs/events`; SCUM companion streaming uses `POST /api/v1/game-client-bridge/companion/logs/events` with the component session token in the typed JSON body. Both emit only the current supervised process session and do not replay retained platform log bodies. Platform is the live relay only. The game plugin/companion owns game-log storage, semantic analysis, and console-page fan-out behavior.
Run-assigned Platform jobs use `job.<jobId>.<streamKey>` log stream IDs. Autonomous lifecycle bootstrap is Run-owned machine execution rather than a Platform job, so its durable process logs use `run.<runEndpointId>.<serverInstanceId>.<streamKey>`. Platform may auto-create those streams only after validating the active Run session and the server-to-Run binding. For retry compatibility, legacy spooled `job.autonomous-*.<streamKey>` batches are accepted as Run-owned autonomous streams without creating or completing a Platform job.
`POST /api/v1/run/logs/batches` and `POST /api/v1/log-streams/query` remain compatibility/internal maintenance contracts for older durable-ingest flows. New Run workers must not use durable local log spool/cache/resend semantics for current supervised stdout/stderr. Log payloads must not carry artifact chunks, host paths, raw credentials, direct sockets, or unbounded inline data.
Log ingest is durable and independently retried. Artifact/file transfer backlog must not prevent log batch acknowledgement, duplicate acknowledgement, cursor state updates, or spool cleanup.
The platform stores log stream metadata through `repo.Store`. Game-specific durable log bodies and derived semantic records belong to the owning game plugin; for SCUM that storage uses plugin-owned SQL tables and stores raw world coordinates without map projection or coordinate conversion.
The platform stores log stream metadata through `repo.Store` and stores log bodies through the configured `LogBodyStore`. The default `file` backend persists platform metadata to `PLATFORM_METADATA_PATH` and appends log entries to segmented JSONL files under `PLATFORM_LOG_DIR`; the `memory` backend is only for tests and disposable local development. MySQL/Postgres are appropriate for platform metadata, stream state, retention policy, indexes, and operational records, but should not be the primary row-per-log-line store for hundreds or thousands of servers. Production log bodies should move behind the same boundary to append/query backends such as ClickHouse, Loki, OpenSearch/Elasticsearch, or object-storage segments with compact indexes.
## Artifact
@@ -115,9 +112,9 @@ Named artifact DTOs:
- `ArtifactTransferCompleteResponse`
- `ArtifactResponse`
Artifact upload supports active run session validation, job/server-instance owner scoping, bounded JSON chunk payloads, per-chunk checksum validation, duplicate chunk acknowledgement, resume status, and final checksum verification before an artifact becomes available. Artifact transport is separate from control, job result, live log relay, plugin bridge, and browser file APIs.
Artifact upload supports active run session validation, job/server-instance owner scoping, bounded JSON chunk payloads, per-chunk checksum validation, duplicate chunk acknowledgement, resume status, and final checksum verification before an artifact becomes available. Artifact transport is separate from control, job result, log ingest, plugin bridge, and browser file APIs.
Artifact/file transfer is the lower-priority heavy channel. Chunk upload and completion must not block control heartbeat, job ack/result delivery, cancellation/reconcile calls, or current live log relay. Lightweight routes must reject heavy transfer payloads instead of accepting or storing them.
Artifact/file transfer is the lower-priority heavy channel. Chunk upload and completion must not block control heartbeat, job ack/result delivery, cancellation/reconcile calls, or log ingest acknowledgement. Lightweight routes must reject heavy transfer payloads instead of accepting or storing them.
## Server File Manager Transfer
@@ -131,7 +128,7 @@ Browser-facing file management uses server-instance scoped routes on Platform fo
Browser uploads are first staged as server-instance artifacts. Platform then queues a `files.write` job whose `inputRef` is `artifact://<id>` and whose execution input names the dedicated `run-file-transfer` channel. Run pulls those bytes through `POST /api/v1/run/files/input-chunk` while proving the active endpoint session plus job attempt and lease. The chunk route is fenced to the active file-write job, validates artifact ownership/checksum, and returns bounded byte ranges only.
File-manager transfer is a separate, low-priority heavy path. Slow uploads, downloads, retries, or file input chunk pulls must not block control heartbeat, job claim/ack/progress/result/cancel/reconcile, current live log relay, or artifact upload acknowledgements. Control, jobs, logs, artifacts, file transfer, and optional game-client bridge remain independently backpressured channels.
File-manager transfer is a separate, low-priority heavy path. Slow uploads, downloads, retries, or file input chunk pulls must not block control heartbeat, job claim/ack/progress/result/cancel/reconcile, durable log batch ingest, or artifact upload acknowledgements. Control, jobs, logs, artifacts, file transfer, and optional game-client bridge remain independently backpressured channels.
## Client Manager lifecycle channel
@@ -139,8 +136,8 @@ Client Manager lifecycle jobs use the independent capabilities `client-manager.d
Run materializes the declared output such as `config.yaml` from the fenced values and its own configured Platform control URL. Source template values are not credentials and must not override the generated component identity or policy. The lifecycle input never contains the component proof itself, a component session, a browser credential, a host path, or a direct socket; proof remains inside the component package and is supplied to the supervised process only through the declared environment-variable name.
Run persists staging/active/previous slots and a local journal. It rejects stale lease/attempt/generation/target fences, traversal/link/device-file archives, checksum mismatches, undeclared executables, and arbitrary shell. Terminal results use `client-manager.deployed`, `client-manager.control`, `client-manager.updated`, `client-manager.rolled-back`, `client-manager.rollback.restored`, or `client-manager.uninstalled` with logical process/health state only. A stalled Client Manager download must not delay Run heartbeat, job ack/result/cancel, or current live log relay.
Run persists staging/active/previous slots and a local journal. It rejects stale lease/attempt/generation/target fences, traversal/link/device-file archives, checksum mismatches, undeclared executables, and arbitrary shell. Terminal results use `client-manager.deployed`, `client-manager.control`, `client-manager.updated`, `client-manager.rolled-back`, `client-manager.rollback.restored`, or `client-manager.uninstalled` with logical process/health state only. A stalled Client Manager download must not delay Run heartbeat, job ack/result/cancel, or log spool acknowledgement.
## Game Client Bridge
The optional game client bridge is separate from run lifecycle, control registration, job handling, compatibility log ingest, and artifact transport. A plugin companion may subscribe to the current live log relay through its component session so it can own game-log storage and analysis without requiring Platform to persist or interpret game log bodies.
The optional game client bridge is separate from run lifecycle, control registration, job handling, log ingest, and artifact transport.