feat: 完整游戏运维功能

2026-07-18 09:04:01 +08:00
parent f3b14b7945
commit 48b8ad8d6c
187 changed files with 16607 additions and 1140 deletions
@@ -0,0 +1,53 @@
+## Context
+
+Earlier changes introduced channelized log ingest and artifact transfer contracts, but the Platform service keeps log batches, artifact transfer state, and artifact payload bytes in process memory. Metrics are currently generated from a live snapshot, backups are not represented, and Run remote access execution is a success-only placeholder. The repository already has file and MySQL metadata snapshots, signed Run requests, endpoint/session fencing, scoped plugin permissions, and safe frontend projections. This change extends those boundaries without replacing them.
+
+## Goals
+
+- Make restart behavior explicit and testable for logs, artifacts, metrics, and backups.
+- Keep large bodies outside lightweight job/control payloads and expose only bounded, redacted projections to users, plugins, and platform_web.
+- Make remote adapters declaration-driven and auditable, with cancellation and lease/attempt fencing handled by the existing job channel.
+- Preserve independent channel priorities so slow artifact or adapter work cannot delay control heartbeat, job ack/result, or log acknowledgement.
+
+## Decisions
+
+### Decision 1: File-backed durable bodies behind service interfaces
+
+Log segments, artifact chunks, and completed artifact content use owner-only files under configured private roots. Metadata and cursors remain in the existing repository snapshot (file or MySQL). Atomic temp-file rename, bounded reads, checksum verification, and startup reconstruction keep writes recoverable. Memory stores remain available for unit tests.
+
+### Decision 2: Transfer manifests are the recovery source of truth
+
+Artifact transfer sessions persist an idempotency key, owner scope, direction, size/chunk limits, received indexes, and final checksum. A restart reloads incomplete manifests and reconstructs next-missing state without exposing payload paths. A chunk is removed from the Run queue only after an acknowledgement for the exact transfer/artifact/index.
+
+### Decision 3: Retention is bounded and deterministic
+
+Log streams carry bounded retention count/age metadata; metric samples and backup records have configurable maximum records/bytes and oldest-first pruning. Pruning emits an audit record and never changes an acknowledged log sequence or an available artifact checksum. Recovery marks interrupted backups/receipts as recoverable failure rather than claiming success.
+
+### Decision 4: Metrics and backups are append-only records with safe projections
+
+Metric samples store server/run identity, timestamp, bounded numeric values, and source. Backup records store logical scope, artifact reference, checksum, size, state, and recovery/audit status. Host paths, credentials, sockets, PID, session/lease tokens, secret refs, and hashes used for fencing are excluded from response DTOs.
+
+### Decision 5: Remote adapters are a constrained registry, not a command tunnel
+
+The Platform registers adapter declarations from an installed plugin/runtime profile and authorizes a request only when owner/admin scope, server instance, selected endpoint, declared capability, and target allowlist all match. Run accepts a typed adapter kind and logical target key, validates timeout and retry bounds, checks context cancellation before and during work, and returns a safe result reference. Shell source, arbitrary command vectors, raw socket addresses, unapproved hosts, and embedded credentials are rejected. Existing Job attempt/lease/session fencing remains authoritative.
+
+### Decision 6: Lightweight routes stay isolated
+
+Control and job routes continue to reject log/artifact payload fields. Log ingest and artifact transfer use separate clients/queues. Remote adapter work is scheduled as a job and its result is metadata-only; it cannot write through control or log endpoints. Tests exercise interleaving and blocked/slow operations with bounded deadlines.
+
+## Data Flow
+
+1. Run appends a bounded log batch or artifact chunk to its owner-only spool/queue before upload.
+2. A low-priority uploader sends the batch/chunk on its dedicated route; the Platform validates signature, endpoint/session, owner scope, sequence/range/checksum, and idempotency before durable append.
+3. Platform persists metadata and body/manifests atomically, returns an acknowledgement, and the Run removes only the acknowledged item.
+4. Metrics and backup records are written through bounded service methods, pruned deterministically, and queried through authorized safe DTOs.
+5. A declared remote adapter request becomes a fenced job. Run executes only the typed adapter implementation, returns a bounded status/result ref, and Platform audits/project results after terminal fencing.
+
+## Non-Goals And Follow-Ups
+
+- No dependency installation, Run self-update, client-manager lifecycle, plugin lifecycle, production scaling/alerts, external object stores, arbitrary FTP/rsync/DB/RCON network access, or real AI provider integration.
+- Native OS process birth tokens and distributed transaction guarantees remain outside this change.
+
+## Rollback
+
+The additive protocol and repository fields are backward compatible. Removing the new capabilities stops advertising durable/adapter features and leaves old metadata untouched; incomplete transfer manifests remain private and can be retried by a compatible release.
@@ -0,0 +1,29 @@
+## Why
+
+The platform has typed log and artifact routes, but restart recovery is incomplete: log acknowledgements and artifact transfer state are not durable, metrics are derived on demand, and backups and restricted remote adapters have no durable ownership/audit model. This change makes those existing channels operationally durable while preserving the control/job/log/artifact priority boundaries established by earlier changes.
+
+## What Changes
+
+- Persist log stream batches, acknowledgement cursors, retention metadata, and bounded queries across Platform restarts; keep Run local spool files retryable and recoverable.
+- Persist artifact metadata, transfer sessions, chunk manifests, checksums, and content through a restart-safe bounded file-backed store; keep chunk retries idempotent and lower priority than logs/jobs/control.
+- Add bounded metrics samples and backup records with retention, size limits, recovery status, and audit events; expose only safe projections.
+- Add declaration-backed remote adapter requests for approved FTP/rsync/run-file/process/database/RCON operations with scoped targets, timeout/cancel/retry/fencing, and audit outcomes. No arbitrary shell, raw socket, unapproved host/credential, or bypass of Platform ownership/endpoint/session checks.
+- Add Platform and Run protocol/client/runtime contracts plus platform_web safe status projections and regression coverage.
+
+## Capabilities
+
+### New Capabilities
+
+- `durable-observability`: durable logs, artifacts, metrics, backups, retention, recovery, and safe query projections.
+- `scoped-remote-adapters`: declared and authorized remote adapter execution with bounded lifecycle and audit semantics.
+
+### Modified Capabilities
+
+- `log-ingest-pipeline`: durable acknowledgement and restart recovery replace the earlier in-memory service assumption.
+- `artifact-transfer-channel`: transfer manifests and content survive restart and retain idempotent chunk/checksum behavior.
+
+## Impact
+
+- Affects `platform/`, independent `run/`, and projection-only `platform_web/` contracts/views.
+- Adds dedicated domain, DTO, model, repository, service, protocol, validator, and runtime types; no Run source is copied into the main repository.
+- Does not implement dependency installation, Run self-update, client-manager lifecycle, plugin lifecycle, production scaling/alerts, or real third-party AI provider integration.
@@ -0,0 +1,61 @@
+## ADDED Requirements
+
+### Requirement: Durable logs recover after restart
+
+The system SHALL persist accepted log batches, acknowledged sequence state, retention metadata, and bounded query indexes so a Platform restart does not duplicate or lose acknowledged ranges.
+
+#### Scenario: Restart preserves log cursor
+
+- **WHEN** a batch is acknowledged, Platform restarts, and a caller queries after a cursor
+- **THEN** the ordered entries and latest acknowledged sequence MUST be available from the persisted store
+
+#### Scenario: Retry remains idempotent
+
+- **WHEN** Run retries an acknowledged batch with the same stream, range, and checksum
+- **THEN** Platform MUST return an idempotent acknowledgement without duplicating entries
+
+### Requirement: Run log spool is restart-safe
+
+The Run log spool SHALL atomically persist unacknowledged batches, tolerate a process restart, and remove a batch only when an acknowledgement covers its full stream range.
+
+#### Scenario: Interrupted enqueue
+
+- **WHEN** a process restarts after an incomplete temporary spool write
+- **THEN** the next spool load MUST ignore temporary files and retain every committed unacknowledged batch
+
+### Requirement: Durable artifacts recover with checksum and chunk bounds
+
+The system SHALL persist artifact metadata, transfer manifests, received chunk indexes, chunk checksums, final checksums, and bounded content so uploads can resume after restart.
+
+#### Scenario: Resume missing chunk
+
+- **WHEN** a transfer has received some chunks and Platform restarts
+- **THEN** status MUST return the same received indexes and next missing index without exposing storage paths
+
+#### Scenario: Checksum conflict is rejected
+
+- **WHEN** a retry uses a different payload or checksum for an already received chunk
+- **THEN** Platform MUST reject it and leave the original chunk and transfer state unchanged
+
+### Requirement: Metrics and backups are durable and bounded
+
+The system SHALL persist metric samples and backup records, apply explicit age/count/byte retention, support recovery status, and expose only owner-authorized safe projections.
+
+#### Scenario: Retention prunes oldest records
+
+- **WHEN** a metric or backup append exceeds its configured bound
+- **THEN** the oldest records MUST be pruned deterministically and an audit event MUST record the retention result
+
+#### Scenario: Interrupted backup is recoverable
+
+- **WHEN** a backup remains in an incomplete state during restart
+- **THEN** it MUST be projected as failed/recoverable with an audit outcome and MUST NOT claim an available artifact
+
+### Requirement: Safe queries enforce ownership
+
+The system SHALL authorize log, artifact, metric, and backup queries by platform session and server ownership/admin scope, returning bounded pages/cursors and never returning host paths, credentials, sockets, Run tokens, leases, session hashes, or secret references.
+
+#### Scenario: Cross-owner query
+
+- **WHEN** a user queries another owner's resource
+- **THEN** the service MUST reject with the existing 403 behavior and MUST NOT reveal whether private body data exists
@@ -0,0 +1,47 @@
+## ADDED Requirements
+
+### Requirement: Remote adapters are declared and scoped
+
+The system SHALL accept only typed adapter kinds and logical target keys declared by the installed plugin/runtime profile and selected Run endpoint.
+
+#### Scenario: Undeclared adapter
+
+- **WHEN** a request names an adapter or target not declared for the server and endpoint
+- **THEN** Platform MUST reject it before creating a job
+
+#### Scenario: Unsafe target data
+
+- **WHEN** a request contains shell source, raw socket addresses, host paths, credentials, or unbounded inline query/command data
+- **THEN** validation MUST reject it and MUST NOT persist the unsafe fields
+
+### Requirement: Adapter lifecycle is bounded and fenced
+
+The system SHALL enforce timeout, cancellation, retry, endpoint/session, attempt, and lease fencing using the existing job channel.
+
+#### Scenario: Cancelled adapter
+
+- **WHEN** cancellation arrives before or during adapter execution
+- **THEN** Run MUST stop at a bounded checkpoint and return a cancelled safe result; Platform MUST not apply a stale terminal result
+
+#### Scenario: Stale attempt result
+
+- **WHEN** an older attempt reports success after a newer attempt owns the lease
+- **THEN** Platform MUST reject the result and retain the newer job state
+
+### Requirement: Adapter results are auditable projections
+
+The system SHALL persist an audit event for authorization, timeout, cancellation, success, and failure outcomes and expose only adapter kind, target key, status, bounded message, and safe result references.
+
+#### Scenario: Successful scoped adapter
+
+- **WHEN** a declared adapter completes within its deadline
+- **THEN** the operator MUST see a safe status and audit summary without raw host/credential/socket details
+
+### Requirement: Channel isolation is maintained
+
+Remote adapter work and artifact transfer SHALL use lower-priority independent work paths and MUST NOT delay control heartbeat, job ack/result, or log upload acknowledgement beyond their deadlines.
+
+#### Scenario: Slow adapter and artifact transfer
+
+- **WHEN** adapter or chunk work blocks or retries
+- **THEN** control, job lifecycle, and log acknowledgement calls MUST remain independently completable
@@ -0,0 +1,29 @@
+## 1. OpenSpec And Contracts
+
+- [x] 1.1 Add durable log/artifact/metrics/backup/remote adapter domain, DTO, model, protocol, and validator contracts with forbidden-field tests.
+- [x] 1.2 Extend file/MySQL snapshot and repository interfaces for durable bodies, transfer manifests, samples, backups, and audit projections.
+- [x] 1.3 Update route/protocol/API contracts and platform_web safe types without exposing Run tokens, lease/session hashes, host paths, credentials, sockets, PID, or secret refs.
+
+## 2. Durable Logs And Artifacts
+
+- [x] 2.1 Make Platform log batches, cursors, retention, and file-backed body storage restart-safe and queryable with idempotent ack/retry.
+- [x] 2.2 Make artifact transfer sessions/chunks/content durable, bounded, resumable, checksum-verified, and idempotent across restart.
+- [x] 2.3 Make Run spool/queues atomic and restart-safe, and keep log/artifact upload scheduling independent from control/jobs.
+
+## 3. Metrics, Backups, And Audit
+
+- [x] 3.1 Persist bounded metrics samples with retention and authorized paginated safe queries.
+- [x] 3.2 Persist backup records and recovery transitions with size/checksum/retention bounds and audit events.
+- [x] 3.3 Add cross-owner/endpoint/signature/session rejection and safe projection tests for logs, artifacts, metrics, backups, and audits.
+
+## 4. Scoped Remote Adapters
+
+- [x] 4.1 Add declaration-backed adapter registry and Platform authorization/dispatch with timeout, cancel, retry, fencing, and audit semantics.
+- [x] 4.2 Implement Run typed adapter execution checkpoints without arbitrary shell, raw sockets, unapproved hosts/credentials, or direct plugin access.
+- [x] 4.3 Add remote adapter protocol/client/runtime tests for timeout, cancellation, stale attempt, wrong owner/endpoint, and safe result projection.
+
+## 5. Frontend And Verification
+
+- [x] 5.1 Add platform_web logs/artifacts/metrics/backups/adapter status, pagination/sequence/checksum/audit projections using existing theme primitives and auth handling.
+- [x] 5.2 Add regression tests for restart/recovery/retention/checksum/idempotency and control/job/log/artifact channel isolation.
+- [x] 5.3 Run plugin manifest/SDK tests, Platform Go tests, platform_web tests/typecheck/build, Run tests, strict OpenSpec validation, structure, shell/compose checks, and both repository diff checks; record only passing evidence.