browser/openspec/changes/implement-production-operations-governance/specs/production-operations-governance/spec.md

ADDED Requirements

Requirement: Platform governs production capacity

Platform SHALL make admission and scheduling decisions using persisted server bindings, endpoint capability support, current Run capacity reports, durable job pressure, and configured safety limits before dispatching production lifecycle work.

Scenario: Capacity accepts bounded work

• WHEN an authorized operator requests a plugin lifecycle, dependency, backup, update, or server lifecycle operation and an assigned endpoint is online with matching capabilities and available capacity
• THEN Platform MUST create one durable job with safe capacity metadata, audit the admission decision, and expose a safe job projection

Scenario: Capacity denies pressure safely

• WHEN the assigned endpoint is offline, stale, missing capability support, over its configured running/queued limits, or under log/artifact/backlog pressure
• THEN Platform MUST reject or defer the request with a safe retry-after/capacity reason, MUST NOT fabricate success, and MUST NOT expose raw endpoint addresses, sockets, paths, credentials, leases, or session tokens

Requirement: Run reports bounded capacity without blocking priority channels

Run SHALL report only bounded capacity dimensions on control heartbeat and job claim metadata, and SHALL keep control, job lifecycle, log ingest, artifact transfer, and optional game-client bridge channels independent.

Scenario: Artifact backlog exists during heartbeat

• WHEN artifact or component transfer backlog is present
• THEN Run MUST continue sending control heartbeat and job ack/result/cancel/reconcile metadata without embedding artifact chunks, host paths, local file names, credentials, sockets, or provider configuration

Scenario: Capacity report is sanitized

• WHEN Platform or browser views Run endpoint capacity
• THEN visible capacity MUST include only logical counts, percentages, pressure codes, and timestamps, never raw host paths, PIDs, sockets, DSNs, RCON values, secrets, leases, or direct endpoint URLs

Requirement: Alerts are durable and closeable

Platform SHALL persist alert records for production incidents including capacity pressure, endpoint staleness, failed jobs, lifecycle failures, plugin compatibility failures, unsafe request denials, and AI provider failures.

Scenario: Alert is acknowledged and resolved

• WHEN an authorized operator acknowledges and later resolves an active alert
• THEN Platform MUST update durable alert state, actor, timestamps, audit references, and safe resolution notes without mutating unrelated alerts

Scenario: Alert retry remains scoped

• WHEN an operator retries a failed alert source such as endpoint capacity refresh, plugin lifecycle check, or AI provider test
• THEN only that source is retried, busy state prevents duplicate submission, and success/failure is reflected from Platform responses rather than local timers

Requirement: Plugin lifecycle is complete and platform-mediated

Platform SHALL manage plugin lifecycle installations separately from marketplace catalog state, including install, enable, disable, upgrade, rollback, retire, dependency checks, compatibility gates, desired/current state, job linkage, audit history, and alert integration.

Scenario: Plugin upgrade is approved

• WHEN an authorized operator approves an upgrade for a server-bound plugin installation with a compatible target version, satisfied dependency checks, and endpoint capability support
• THEN Platform MUST create a durable lifecycle job and transition state only from job results or reconciliation evidence

Scenario: Plugin rollback is repeated

• WHEN rollback is retried with the same idempotency key and immutable inputs
• THEN Platform MUST return the original lifecycle/job result and MUST reject the same key with different immutable inputs

Scenario: Unsafe lifecycle input is rejected

• WHEN plugin lifecycle metadata, manifest actions, bridge requests, or job inputs contain arbitrary shell, raw credentials, direct Run endpoints, host paths, PIDs, sockets, DSNs, RCON data, unknown capabilities, or undeclared AI purposes
• THEN validation MUST reject the request before registration or dispatch

Requirement: Plugins request AI only through Platform-mediated capability

Plugin manifests, SDK calls, and platform-hosted plugin pages SHALL request AI assistance only through typed Platform bridge/API contracts that declare purpose, request ID, scoped input, and context references.

Scenario: Declared AI purpose is invoked

• WHEN a plugin invokes a declared AI purpose through the Platform bridge
• THEN Platform MUST validate plugin installation state, route permission, server scope, purpose, payload bounds, provider availability, and unsafe text before invoking a provider client

Scenario: Undeclared AI purpose is denied

• WHEN a plugin invokes an undeclared or currently unauthorized AI purpose
• THEN Platform MUST deny the request with a safe error and MUST NOT call a provider client

Requirement: Real AI Provider integration stays inside Platform

Platform SHALL support a real provider client path that reads provider keys and base URL material only inside Platform service boundaries, invokes enabled providers with bounded requests, and returns redacted responses.

Scenario: Real provider returns recommendation

• WHEN an enabled provider is selected and the provider client succeeds
• THEN Platform MUST return safe recommendation, model, usage, request ID, and optional diff preview metadata without raw API keys, bearer tokens, base URL secrets, transport headers, storage material, host paths, sockets, or credentials

Scenario: Provider fails

• WHEN provider transport, authentication, rate limit, or model invocation fails
• THEN Platform MUST persist a safe audit/alert, return a redacted failure, and MUST NOT expose raw provider URL, headers, keys, request body secrets, or stack traces

Requirement: AI config changes require reviewable diff approval

AI-generated configuration changes SHALL be persisted as reviewable recommendations or diff previews and SHALL require separate operator approval before Platform dispatches any run-side config write job.

Scenario: Diff is approved

• WHEN an authorized operator approves a current, matching AI config diff preview for the same server, plugin, actor scope, and config revision
• THEN Platform MUST dispatch one bounded config write job and link the job, approval, audit event, and source AI request

Scenario: Diff is stale or cancelled

• WHEN the config revision changes, the approval is cancelled, or the preview has expired
• THEN Platform MUST reject dispatch, preserve the recommendation for review history, and avoid run-side mutation

Requirement: Console surfaces governance without changing visual direction

platform_web SHALL surface capacity governance, alert closure, plugin lifecycle, and AI diff approval in the existing first-party operations console using real Platform APIs, permissions, confirmation, busy state, scoped retry, and failure recovery.

Scenario: Operator closes alert in the console

• WHEN an authorized operator acknowledges or resolves an alert
• THEN the UI MUST dispatch one Platform request, block duplicate submission while pending, refresh from persisted response, and show failure recovery without claiming local success

Scenario: Narrow viewport renders governance surfaces

• WHEN the console is rendered at 390px in black-mecha or magical-girl theme
• THEN capacity, alerts, lifecycle, AI diff review, confirmations, and operation rows MUST remain readable, bounded, and theme-consistent without page-local global decoration

Requirement: Verification covers production governance boundaries

The change SHALL include platform tests, run tests, plugin typecheck/tests/manifest validation, platform_web typecheck/tests/build, browser acceptance evidence, structure validation, strict OpenSpec validation, two-repository diff whitespace checks, and forbidden-field scans before tasks are marked complete.

Scenario: Verification evidence is recorded

• WHEN implementation tasks are completed
• THEN tasks.md MUST record real command/browser evidence and MUST NOT mark verification complete until those checks pass