first commit
This commit is contained in:
npc0-hue
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-07-08
|
||||
@@ -0,0 +1,66 @@
|
||||
## Context
|
||||
|
||||
The platform/run architecture already separates control registration, job lifecycle calls, durable log ingest, and artifact transfer into typed routes and protocol packages. Prior changes proved those channels individually, and the lifecycle proof showed that real plugin operations can route through platform-owned lifecycle APIs into run jobs.
|
||||
|
||||
This change hardens the cross-channel behavior. The important failure mode is not just malformed payloads; it is starvation under concurrent work. A large artifact or file transfer must not delay control heartbeat, job acknowledgement, job result delivery, or durable log spool upload. Likewise, retry queues must stay independently bounded so a blocked artifact transfer cannot consume the execution path needed for log ingest or job completion.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Prove run-side scheduling keeps control heartbeat, job ack/result, log upload, and artifact/file transfer on independently bounded paths.
|
||||
- Prove platform APIs validate and mutate state independently when log, artifact, job, and control requests interleave.
|
||||
- Add focused tests that simulate slow or large artifact/file work while verifying timely heartbeat, job ack/result, and log acknowledgement.
|
||||
- Document the channel priority and non-starvation invariants in run/platform protocol docs.
|
||||
- Preserve the existing channel APIs unless implementation reveals a contract gap that must be made explicit in the spec.
|
||||
|
||||
**Non-Goals:**
|
||||
- Do not add new plugin-facing transport or direct run access.
|
||||
- Do not add a browser UI flow unless implementation discovers an existing platform_web surface incorrectly exposes channel details.
|
||||
- Do not redesign storage backends, introduce external queues, or require distributed infrastructure.
|
||||
- Do not change artifact/log/job payload semantics except where needed to enforce bounded isolation.
|
||||
|
||||
## Decisions
|
||||
|
||||
1. Keep isolation proof local to run/platform packages before adding broader e2e tooling.
|
||||
|
||||
The current risk lives in queueing, retry, route handling, and worker scheduling. Package-level tests can deterministically simulate slow artifact uploads, retryable platform failures, and interleaved requests without relying on brittle timing from a full browser stack. A later browser acceptance suite can reuse this confidence without becoming the primary proof.
|
||||
|
||||
Alternative considered: start with a full local platform/run/browser smoke. That gives nice operator evidence but is weaker for starvation because browser timing is noisy and harder to make deterministic.
|
||||
|
||||
2. Treat control and job lifecycle calls as high-priority bounded work.
|
||||
|
||||
Heartbeats, job claim/ack/progress/result, and cancellation/reconcile calls remain small JSON payloads. They must never carry artifact chunks, file bodies, or large inline logs. Tests should assert that delayed artifact/file uploads cannot prevent these calls from completing.
|
||||
|
||||
Alternative considered: one shared retry worker for all run-to-platform calls. That is simpler, but a stuck artifact transfer could monopolize retries and delay lifecycle visibility.
|
||||
|
||||
3. Keep log ingest durable and independently retryable.
|
||||
|
||||
The log spool already persists batches until platform acknowledgement. This change should assert that log batch selection, upload, ack handling, and retry bookkeeping stay independent from artifact chunk retry queues and job result submission.
|
||||
|
||||
Alternative considered: merge log and artifact retry state because both are upload queues. That would blur priority boundaries and make it easier for large artifact payloads to starve small log acknowledgements.
|
||||
|
||||
4. Verify platform state isolation with interleaved service/API tests.
|
||||
|
||||
Platform tests should interleave control heartbeat, job ack/result, log batch ingest, and artifact transfer requests for the same run endpoint. Success means each route validates only its own contract, mutates only its own state, and preserves idempotency when requests are retried or reordered within valid channel rules.
|
||||
|
||||
Alternative considered: only test run-side clients. That would miss platform-side cross-route coupling, such as artifact completion accidentally blocking log acknowledgement state.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- Timing-sensitive tests become flaky -> Use deterministic fakes, channels, contexts, and bounded wait helpers instead of wall-clock sleeps wherever possible.
|
||||
- Hardening may reveal that current worker scheduling is too serial -> Introduce small, explicit channel executors or queue limits rather than broad worker rewrites.
|
||||
- Additional docs can drift -> Keep docs close to `run/protocol/` and `platform/protocol/` route contracts, and update them in the same implementation task as tests.
|
||||
- Full starvation proof can become too broad -> Scope the first pass to platform/run package behavior and exact commands in `tasks.md`; leave browser-wide automation to the later acceptance-suite queue item.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. Add failing tests for platform and run channel isolation around existing APIs and queues.
|
||||
2. Adjust run scheduling, retry queues, or client sequencing only where tests prove coupling.
|
||||
3. Update protocol documentation with the enforced invariants.
|
||||
4. Run platform/run tests, structure check, and strict OpenSpec validation.
|
||||
|
||||
Rollback is straightforward because expected changes are test and scheduling hardening around existing APIs. If a scheduling change regresses behavior, revert that implementation while keeping the new tests as the contract for the corrected approach.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- None currently. The implementation should stay within `run/` and `platform/` unless a failing test proves a shared contract needs a spec update.
|
||||
@@ -0,0 +1,25 @@
|
||||
## Why
|
||||
|
||||
Log ingest, artifact/file transfer, control heartbeat, and job ack/result delivery already exist as separate platform/run channels, but the current evidence mostly proves each channel in isolation. The next risk is starvation under load: a large artifact or file operation must not delay heartbeat, lifecycle acknowledgement, job result delivery, or durable log upload.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add channel-isolation requirements that define priority, bounded payloads, retry behavior, and non-starvation guarantees across run/platform channels.
|
||||
- Add run-side concurrency and queue tests proving large artifact/file work cannot block control heartbeat, job ack/result submission, or log spool upload.
|
||||
- Add platform service/API tests proving artifact/log/job/control endpoints preserve independent validation, state mutation, and idempotency under interleaved requests.
|
||||
- Add a local verification command set that exercises platform and run test suites plus strict OpenSpec validation.
|
||||
- No breaking API changes are expected; the change hardens behavior and verification around existing channel contracts.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `log-artifact-channel-isolation`: Defines cross-channel non-starvation, bounded-transfer, durable-retry, and verification guarantees for run/platform control, job, log, artifact, and file operations.
|
||||
|
||||
### Modified Capabilities
|
||||
- None.
|
||||
|
||||
## Impact
|
||||
|
||||
- Affected roots: `run/` and `platform/`.
|
||||
- Affected areas: run worker scheduling, platform client calls, log spool retry, artifact/file queue retry, platform run-facing APIs, service tests, API tests, and protocol documentation.
|
||||
- Validation impact: requires focused run/platform concurrency tests, existing package tests, `scripts/check-structure.sh`, and `openspec validate harden-log-artifact-channel-isolation --strict`.
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Run channels preserve non-starvation under large transfers
|
||||
The run executor SHALL keep control heartbeat, job acknowledgement, job result delivery, and log batch upload on bounded execution paths that are not blocked by large artifact or file transfer work.
|
||||
|
||||
#### Scenario: Artifact upload does not block lifecycle calls
|
||||
- **WHEN** a run endpoint is uploading or retrying a large artifact or file transfer
|
||||
- **THEN** control heartbeat, job acknowledgement, job progress, cancellation polling, reconciliation, and terminal job result calls MUST remain able to complete through their typed platform client methods without waiting for the transfer payload to finish
|
||||
|
||||
#### Scenario: Log upload continues during transfer pressure
|
||||
- **WHEN** artifact or file transfer chunks are queued, slow, or retrying
|
||||
- **THEN** the run log spool MUST still select bounded log batches, upload them through the log ingest client, and remove acknowledged batches independently from artifact/file queue state
|
||||
|
||||
### Requirement: Platform routes mutate only their own channel state
|
||||
The platform SHALL handle interleaved control, job, log, artifact, and file requests for the same run endpoint without one channel accepting another channel's payload or mutating another channel's state.
|
||||
|
||||
#### Scenario: Interleaved valid requests succeed independently
|
||||
- **WHEN** a registered run endpoint interleaves valid heartbeat, job ack/result, log batch ingest, and artifact chunk or completion requests
|
||||
- **THEN** each route MUST validate its own typed DTO, update only the corresponding control/job/log/artifact state, and return the same acknowledgement semantics as if the requests were sent without interleaving
|
||||
|
||||
#### Scenario: Heavy payload is rejected from lightweight routes
|
||||
- **WHEN** a control, job, or log route receives an artifact chunk, file body, host path, raw credential, direct socket, or other transport payload owned by another channel
|
||||
- **THEN** the platform MUST reject the request as a JSON validation error and MUST NOT mutate control session, job lifecycle, log acknowledgement, or artifact state
|
||||
|
||||
### Requirement: Retry queues remain independently bounded
|
||||
The run executor SHALL keep log retry state and artifact/file retry state independently bounded and independently acknowledged.
|
||||
|
||||
#### Scenario: Artifact retry backlog does not consume log retry state
|
||||
- **WHEN** artifact or file chunks remain unacknowledged after platform upload failures
|
||||
- **THEN** the artifact/file retry queue MUST retain those chunks without preventing log spool retry listing, log batch upload, or acknowledged log batch removal
|
||||
|
||||
#### Scenario: Log retry backlog does not consume artifact retry state
|
||||
- **WHEN** log batches remain unacknowledged after platform ingest failures
|
||||
- **THEN** the log spool MUST retain those batches without preventing artifact/file retry listing, chunk upload, or acknowledged artifact chunk removal
|
||||
|
||||
### Requirement: Job terminal results remain bounded and prioritized
|
||||
The run job channel SHALL submit terminal job results as bounded metadata and result references, not inline logs, artifact chunks, file bodies, host paths, raw credentials, or direct sockets.
|
||||
|
||||
#### Scenario: Terminal result arrives while transfer is active
|
||||
- **WHEN** a job finishes while artifact/file transfer work is still active or retrying
|
||||
- **THEN** run MUST submit the terminal job result through the job result endpoint with bounded result metadata and the platform MUST accept or reject it only according to job lease and idempotency rules
|
||||
|
||||
#### Scenario: Duplicate terminal result remains idempotent under pressure
|
||||
- **WHEN** run retries an equivalent terminal job result while log and artifact retries are also pending
|
||||
- **THEN** platform MUST return the accepted idempotent terminal result response and MUST NOT duplicate logs, chunks, artifacts, or unrelated job metadata
|
||||
|
||||
### Requirement: Channel isolation is documented and verified
|
||||
The change SHALL document the enforced priority and isolation rules and SHALL include deterministic platform/run tests for interleaved requests, retry independence, and large-transfer non-starvation.
|
||||
|
||||
#### Scenario: Contributor inspects channel docs
|
||||
- **WHEN** a contributor opens run or platform protocol documentation
|
||||
- **THEN** the docs MUST state that control and job lifecycle calls are lightweight, log ingest is durable and independently retried, artifact/file transfer is chunked and lower priority, and no lightweight route accepts heavy transfer payloads
|
||||
|
||||
#### Scenario: Verification commands run
|
||||
- **WHEN** the change is complete
|
||||
- **THEN** `go test ./...` from `platform/`, `go test ./...` from `run/`, `scripts/check-structure.sh`, and `openspec validate harden-log-artifact-channel-isolation --strict` MUST pass
|
||||
@@ -0,0 +1,56 @@
|
||||
## 1. Run-Side Channel Isolation
|
||||
|
||||
- [x] 1.1 Add deterministic run tests that simulate slow or retrying artifact/file transfer work while control heartbeat and job ack/progress/result calls continue through bounded client calls.
|
||||
- [x] 1.2 Add run tests proving log spool selection, upload acknowledgement, and retry cleanup continue while artifact/file chunks are queued, slow, or retrying.
|
||||
- [x] 1.3 Add run tests proving artifact/file retry listing, chunk acknowledgement, and cleanup continue while log batches are queued, slow, or retrying.
|
||||
- [x] 1.4 Update run scheduling, retry queue, or worker orchestration code only where needed to make the tests pass without exposing host paths, raw credentials, direct sockets, or large inline payloads through lightweight channels.
|
||||
- [x] 1.5 Run `cd run && GOCACHE=/private/tmp/browser-go-build-cache go test ./... -count=1` and record evidence.
|
||||
|
||||
## 2. Platform Interleaving and Validation
|
||||
|
||||
- [x] 2.1 Add platform service/API tests that interleave valid heartbeat, job ack/result, log batch ingest, and artifact transfer requests for one registered run endpoint.
|
||||
- [x] 2.2 Prove each interleaved platform route mutates only its own state and preserves existing idempotency semantics.
|
||||
- [x] 2.3 Add negative platform tests proving control, job, and log routes reject artifact chunks, file bodies, host paths, raw credentials, direct sockets, and channel-owned transport payloads from other routes.
|
||||
- [x] 2.4 Update platform validators, DTOs, service logic, or route documentation only where needed to enforce the isolation contract.
|
||||
- [x] 2.5 Run `cd platform && GOCACHE=/private/tmp/browser-go-build-cache go test ./... -count=1` and record evidence.
|
||||
|
||||
## 3. Protocol Documentation
|
||||
|
||||
- [x] 3.1 Update `run/protocol/*.md`, `run/spool/README.md`, `run/artifact/README.md`, `platform/protocol/run-contracts.md`, or `platform/api/routes.md` as needed to document channel priority and payload boundaries.
|
||||
- [x] 3.2 Confirm docs state that control/job calls remain lightweight, log ingest is durable and independently retried, artifact/file transfer is chunked and lower priority, and lightweight routes never accept heavy transfer payloads.
|
||||
|
||||
## 4. Verification and Stream Handoff
|
||||
|
||||
- [x] 4.1 Record implementation evidence in this tasks file only after each command has actually run.
|
||||
- [x] 4.2 Run `scripts/check-structure.sh` and record evidence.
|
||||
- [x] 4.3 Run `openspec validate harden-log-artifact-channel-isolation --strict` and record evidence.
|
||||
- [x] 4.4 Update `openspec/changes/architecture-delivery-stream/delivery-plan.md` to mark `harden-log-artifact-channel-isolation` complete only after evidence exists and move the next queue item to active.
|
||||
- [x] 4.5 Update `openspec/changes/architecture-delivery-stream/NEXT_CHANGE.md` with the next implementation/generator handoff after this change closes.
|
||||
|
||||
## Evidence
|
||||
|
||||
- Run-side channel isolation:
|
||||
- Added `run/api/channel_isolation_test.go`, proving heartbeat, terminal job result, and log ingest complete while an artifact chunk upload is deliberately blocked.
|
||||
- Added `run/spool/channel_isolation_test.go`, proving log acknowledgement cleanup remains independent from artifact backlog and artifact acknowledgement cleanup remains independent from log backlog.
|
||||
- No run scheduling or queue production code changes were required; existing separate client calls and separate `logs` / `artifacts` spool areas satisfied the new regression tests.
|
||||
- Initial sandbox run of `cd run && GOCACHE=/private/tmp/browser-go-build-cache go test ./api ./spool -count=1` was blocked by `httptest` loopback bind permissions after `run/spool` passed.
|
||||
- Escalated rerun of `cd run && GOCACHE=/private/tmp/browser-go-build-cache go test ./api ./spool -count=1` passed for `browser.local/run/api` and `browser.local/run/spool`.
|
||||
- Full sandbox run of `cd run && GOCACHE=/private/tmp/browser-go-build-cache go test ./... -count=1` was blocked by `httptest` loopback bind permissions in `run/api` and `run/runtime`; non-listener packages passed.
|
||||
- Escalated rerun of `cd run && GOCACHE=/private/tmp/browser-go-build-cache go test ./... -count=1` passed for `api`, `config`, `protocol`, `runtime`, and `spool`.
|
||||
|
||||
- Platform interleaving and validation:
|
||||
- Added `platform/api/channel_isolation_handlers_test.go`, proving interleaved heartbeat, job ack/result, log batch ingest, and artifact transfer requests mutate only their own channel state.
|
||||
- Added negative API coverage proving lightweight control/job/log routes reject artifact chunks, inline log arrays, host paths, raw credential fragments, direct socket strings, and heavy transfer payload fields through strict JSON decoding.
|
||||
- Added rejection-state coverage proving a rejected heavy heartbeat payload does not mutate endpoint capacity or store heavy payload text.
|
||||
- Focused command `cd platform && GOCACHE=/private/tmp/browser-go-build-cache go test ./api -run 'TestRunChannelAPI|TestLightweightRunRoutes' -count=1` passed.
|
||||
- Full command `cd platform && GOCACHE=/private/tmp/browser-go-build-cache go test ./... -count=1` passed for `api`, `config`, `domain`, `dto`, `model`, `repo`, `service`, and `validator`.
|
||||
|
||||
- Protocol documentation:
|
||||
- Updated `run/spool/README.md`, `run/artifact/README.md`, `run/protocol/artifact.md`, `run/protocol/log-ingest.md`, and `run/protocol/job.md` with channel priority, independent retry, and heavy-payload boundary rules.
|
||||
- Updated `platform/protocol/run-contracts.md` and `platform/api/routes.md` to state that control/job calls remain lightweight, log ingest is durable and independently retried, artifact/file transfer is lower priority and chunked, and lightweight routes reject heavy transfer payloads.
|
||||
|
||||
- Final gates and stream handoff:
|
||||
- `scripts/check-structure.sh` passed with `structure check passed`.
|
||||
- `openspec validate harden-log-artifact-channel-isolation --strict` passed with `Change 'harden-log-artifact-channel-isolation' is valid`; the process exited 0. PostHog telemetry flush reported `ENOTFOUND edge.openspec.dev`, which did not affect validation.
|
||||
- `openspec/changes/architecture-delivery-stream/delivery-plan.md` now marks `harden-log-artifact-channel-isolation` complete and `implement-local-debug-workspace` active.
|
||||
- `openspec/changes/architecture-delivery-stream/NEXT_CHANGE.md` now points the next generator chat at creating `implement-local-debug-workspace`, because that OpenSpec directory does not exist yet.
|
||||
Reference in New Issue
Block a user