first commit

2026-07-11 14:56:10 +08:00
commit 7e05d0a4e7
660 changed files with 78119 additions and 0 deletions
@@ -0,0 +1,36 @@
+# Design
+
+## Metadata backend
+
+`PLATFORM_STORAGE_BACKEND` selects the platform metadata store:
+
+- `file`: default local durable snapshot at `PLATFORM_METADATA_PATH`.
+- `memory`: test/disposable storage.
+- `mysql`: MySQL-backed metadata snapshot using `PLATFORM_MYSQL_DSN`.
+
+The first MySQL implementation stores one platform-owned JSON snapshot in a `platform_metadata_snapshots` table. This gives operators a real durable MySQL option now while preserving the existing `repo.Store` boundary. Later changes can normalize individual repositories into relational tables without changing handlers or services.
+
+## Log body backend
+
+`PLATFORM_LOG_BODY_BACKEND` selects log body storage separately:
+
+- empty: follows the metadata backend, except `mysql` maps to `file`.
+- `file`: segmented JSONL log files in `PLATFORM_LOG_DIR`.
+- `memory`: tests/disposable local runs.
+
+MySQL metadata storage does not imply MySQL log bodies. Hundreds or thousands of servers should use segmented files for local deployments and log-optimized stores such as ClickHouse, Loki, OpenSearch/Elasticsearch, or object-storage segments in production.
+
+## Docker guidance
+
+The root `docker-compose.yml` keeps the default file backend. It includes commented MySQL service/config blocks so operators can uncomment them when they want local MySQL metadata:
+
+```text
+PLATFORM_STORAGE_BACKEND=mysql
+PLATFORM_MYSQL_DSN=platform:platform@tcp(mysql:3306)/platform?parseTime=true
+PLATFORM_LOG_BODY_BACKEND=file
+```
+
+## Failure behavior
+
+If `PLATFORM_STORAGE_BACKEND=mysql` is set without `PLATFORM_MYSQL_DSN`, platform startup fails with a direct configuration error. If the DSN is present but the database is unreachable, startup fails fast rather than silently falling back to memory.
+
@@ -0,0 +1,20 @@
+# Add MySQL platform metadata storage
+
+## Why
+
+The platform now has durable file storage, but Docker/local configuration does not expose a real MySQL option. Operators need a clear `PLATFORM_STORAGE_BACKEND=mysql` path with commented configuration examples. They also need the deployment docs to make the log storage boundary explicit: MySQL is for platform metadata, not high-volume row-per-log-line bodies.
+
+## What Changes
+
+- Add MySQL metadata storage configuration through `PLATFORM_MYSQL_DSN`.
+- Add a MySQL-backed `repo.Store` implementation that persists platform metadata snapshots in a platform-owned table.
+- Separate metadata backend selection from log body backend selection with `PLATFORM_LOG_BODY_BACKEND`.
+- Update Docker compose/env examples with commented MySQL configuration.
+- Document how to configure MySQL locally and in Docker, and clarify that log bodies remain on `LogBodyStore`.
+
+## Impact
+
+- Operators can configure platform metadata persistence with MySQL without changing code.
+- Existing file-backed storage remains the default.
+- Logs continue to use file segments by default; production log analytics backends remain a future adapter behind `LogBodyStore`.
+
@@ -0,0 +1,45 @@
+# mysql-platform-metadata-storage Specification
+
+## ADDED Requirements
+
+### Requirement: MySQL metadata backend configuration
+
+The platform SHALL support `PLATFORM_STORAGE_BACKEND=mysql` for metadata persistence.
+
+#### Scenario: MySQL backend is configured with a DSN
+
+- **GIVEN** `PLATFORM_STORAGE_BACKEND=mysql`
+- **AND** `PLATFORM_MYSQL_DSN` points to a reachable database
+- **WHEN** the platform starts
+- **THEN** it SHALL initialize a MySQL metadata store
+- **AND** it SHALL create required metadata storage structures when missing.
+
+#### Scenario: MySQL backend is missing a DSN
+
+- **GIVEN** `PLATFORM_STORAGE_BACKEND=mysql`
+- **AND** `PLATFORM_MYSQL_DSN` is empty
+- **WHEN** the platform starts
+- **THEN** startup SHALL fail with a clear configuration error.
+
+### Requirement: Log body backend remains separate
+
+The platform SHALL configure log body storage separately from metadata storage.
+
+#### Scenario: MySQL metadata uses file log bodies by default
+
+- **GIVEN** `PLATFORM_STORAGE_BACKEND=mysql`
+- **AND** `PLATFORM_LOG_BODY_BACKEND` is empty
+- **WHEN** the platform starts
+- **THEN** log bodies SHALL use the file segmented backend
+- **AND** log entries SHALL NOT be stored as row-per-line MySQL metadata.
+
+### Requirement: MySQL configuration is documented
+
+The repository SHALL include commented MySQL examples in local env and Docker configuration docs.
+
+#### Scenario: Operator wants to configure MySQL
+
+- **GIVEN** an operator reads the env examples or README
+- **WHEN** they search for MySQL configuration
+- **THEN** they SHALL find `PLATFORM_STORAGE_BACKEND=mysql`, `PLATFORM_MYSQL_DSN`, and `PLATFORM_LOG_BODY_BACKEND=file` examples.
+
@@ -0,0 +1,23 @@
+## 1. OpenSpec Artifacts
+
+- [x] 1.1 Create proposal, design, spec, and tasks for MySQL metadata storage configuration.
+- [x] 1.2 Validate the change with `openspec validate add-mysql-platform-metadata-storage --strict`.
+
+## 2. MySQL Metadata Storage
+
+- [x] 2.1 Add platform config fields for `PLATFORM_MYSQL_DSN` and `PLATFORM_LOG_BODY_BACKEND`.
+- [x] 2.2 Implement a MySQL-backed metadata snapshot store behind `repo.Store`.
+- [x] 2.3 Wire router startup to support `PLATFORM_STORAGE_BACKEND=mysql`.
+- [x] 2.4 Add tests for MySQL config loading, missing DSN failure, and log body backend selection.
+
+## 3. Documentation And Comments
+
+- [x] 3.1 Add commented MySQL examples to root/platform env examples and Docker compose.
+- [x] 3.2 Update README docs with exact MySQL DSN examples and log storage guidance.
+
+## 4. Verification
+
+- [x] 4.1 Run `cd platform && go test ./config ./api ./repo -count=1`.
+- [x] 4.2 Run `docker compose config`.
+- [x] 4.3 Run `scripts/check-structure.sh`.
+- [x] 4.4 Run `openspec validate add-mysql-platform-metadata-storage --strict`.