chore: 清理一些无用的文件
This commit is contained in:
105
AGENTS.md
105
AGENTS.md
@@ -1,105 +0,0 @@
|
|||||||
# Repository Guidelines
|
|
||||||
|
|
||||||
## Project Structure & Module Organization
|
|
||||||
- `backend/`: Go service. `cmd/server` is the entrypoint, `internal/` contains handlers/services/repositories/server wiring, `ent/` holds Ent schemas and generated ORM code, `migrations/` stores DB migrations, and `internal/web/dist/` is the embedded frontend build output.
|
|
||||||
- `frontend/`: Vue 3 + TypeScript app. Main folders are `src/api`, `src/components`, `src/views`, `src/stores`, `src/composables`, `src/utils`, and test files in `src/**/__tests__`.
|
|
||||||
- `deploy/`: Docker and deployment assets (`docker-compose*.yml`, `.env.example`, `config.example.yaml`).
|
|
||||||
- `openspec/`: Spec-driven change docs (`changes/<id>/{proposal,design,tasks}.md`).
|
|
||||||
- `tools/`: Utility scripts (security/perf checks).
|
|
||||||
|
|
||||||
## Build, Test, and Development Commands
|
|
||||||
```bash
|
|
||||||
make build # Build backend + frontend
|
|
||||||
make test # Backend tests + frontend lint/typecheck
|
|
||||||
cd backend && make build # Build backend binary
|
|
||||||
cd backend && make test-unit # Go unit tests
|
|
||||||
cd backend && make test-integration # Go integration tests
|
|
||||||
cd backend && make test # go test ./... + golangci-lint
|
|
||||||
cd frontend && pnpm install --frozen-lockfile
|
|
||||||
cd frontend && pnpm dev # Vite dev server
|
|
||||||
cd frontend && pnpm build # Type-check + production build
|
|
||||||
cd frontend && pnpm test:run # Vitest run
|
|
||||||
cd frontend && pnpm test:coverage # Vitest + coverage report
|
|
||||||
python3 tools/secret_scan.py # Secret scan
|
|
||||||
```
|
|
||||||
|
|
||||||
## Coding Style & Naming Conventions
|
|
||||||
- Go: format with `gofmt`; lint with `golangci-lint` (`backend/.golangci.yml`).
|
|
||||||
- Respect layering: `internal/service` and `internal/handler` must not import `internal/repository`, `gorm`, or `redis` directly (enforced by depguard).
|
|
||||||
- Frontend: Vue SFC + TypeScript, 2-space indentation, ESLint rules from `frontend/.eslintrc.cjs`.
|
|
||||||
- Naming: components use `PascalCase.vue`, composables use `useXxx.ts`, Go tests use `*_test.go`, frontend tests use `*.spec.ts`.
|
|
||||||
|
|
||||||
## Go & Frontend Development Standards
|
|
||||||
- Control branch complexity: `if` nesting must not exceed 3 levels. Refactor with guard clauses, early returns, helper functions, or strategy maps when deeper logic appears.
|
|
||||||
- JSON hot-path rule: for read-only/partial-field extraction, prefer `gjson` over full `encoding/json` struct unmarshal to reduce allocations and improve latency.
|
|
||||||
- Exception rule: if full schema validation or typed writes are required, `encoding/json` is allowed, but PR must explain why `gjson` is not suitable.
|
|
||||||
|
|
||||||
### Go Performance Rules
|
|
||||||
- Optimization workflow rule: benchmark/profile first, then optimize. Use `go test -bench`, `go tool pprof`, and runtime diagnostics before changing hot-path code.
|
|
||||||
- For hot functions, run escape analysis (`go build -gcflags=all='-m -m'`) and prioritize stack allocation where reasonable.
|
|
||||||
- Every external I/O path must use `context.Context` with explicit timeout/cancel.
|
|
||||||
- When creating derived contexts (`WithTimeout` / `WithDeadline`), always `defer cancel()` to release resources.
|
|
||||||
- Preallocate slices/maps when size can be estimated (`make([]T, 0, n)`, `make(map[K]V, n)`).
|
|
||||||
- Avoid unnecessary allocations in loops; reuse buffers and prefer `strings.Builder`/`bytes.Buffer`.
|
|
||||||
- Prohibit N+1 query patterns; batch DB/Redis operations and verify indexes for new query paths.
|
|
||||||
- For hot-path changes, include benchmark or latency comparison evidence (e.g., `go test -bench` before/after).
|
|
||||||
- Keep goroutine growth bounded (worker pool/semaphore), and avoid unbounded fan-out.
|
|
||||||
- Lock minimization rule: if a lock can be avoided, do not use a lock. Prefer ownership transfer (channel), sharding, immutable snapshots, copy-on-write, or atomic operations to reduce contention.
|
|
||||||
- When locks are unavoidable, keep critical sections minimal, avoid nested locks, and document why lock-free alternatives are not feasible.
|
|
||||||
- Follow `sync` guidance: prefer channels for higher-level synchronization; use low-level mutex primitives only where necessary.
|
|
||||||
- Avoid reflection and `interface{}`-heavy conversions in hot paths; use typed structs/functions.
|
|
||||||
- Use `sync.Pool` only when benchmark proves allocation reduction; remove if no measurable gain.
|
|
||||||
- Avoid repeated `time.Now()`/`fmt.Sprintf` in tight loops; hoist or cache when possible.
|
|
||||||
- For stable high-traffic binaries, maintain representative `default.pgo` profiles and keep `go build -pgo=auto` enabled.
|
|
||||||
|
|
||||||
### Data Access & Cache Rules
|
|
||||||
- Every new/changed SQL query must be checked with `EXPLAIN` (or `EXPLAIN ANALYZE` in staging) and include index rationale in PR.
|
|
||||||
- Default to keyset pagination for large tables; avoid deep `OFFSET` scans on hot endpoints.
|
|
||||||
- Query only required columns; prohibit broad `SELECT *` in latency-sensitive paths.
|
|
||||||
- Keep transactions short; never perform external RPC/network calls inside DB transactions.
|
|
||||||
- Connection pool must be explicitly tuned and observed via `DB.Stats` (`SetMaxOpenConns`, `SetMaxIdleConns`, `SetConnMaxIdleTime`, `SetConnMaxLifetime`).
|
|
||||||
- Avoid overly small `MaxOpenConns` that can turn DB access into lock/semaphore bottlenecks.
|
|
||||||
- Cache keys must be versioned (e.g., `user_usage:v2:{id}`) and TTL should include jitter to avoid thundering herd.
|
|
||||||
- Use request coalescing (`singleflight` or equivalent) for high-concurrency cache miss paths.
|
|
||||||
|
|
||||||
### Frontend Performance Rules
|
|
||||||
- Route-level and heavy-module code splitting is required; lazy-load non-critical views/components.
|
|
||||||
- API requests must support cancellation and deduplication; use debounce/throttle for search-like inputs.
|
|
||||||
- Minimize unnecessary reactivity: avoid deep watch chains when computed/cache can solve it.
|
|
||||||
- Prefer stable props and selective rendering controls (`v-once`, `v-memo`) for expensive subtrees when data is static or keyed.
|
|
||||||
- Large data rendering must use pagination or virtualization (especially tables/lists >200 rows).
|
|
||||||
- Move expensive CPU work off the main thread (Web Worker) or chunk tasks to avoid UI blocking.
|
|
||||||
- Keep bundle growth controlled; avoid adding heavy dependencies without clear ROI and alternatives review.
|
|
||||||
- Avoid expensive inline computations in templates; move to cached `computed` selectors.
|
|
||||||
- Keep state normalized; avoid duplicated derived state across multiple stores/components.
|
|
||||||
- Load charts/editors/export libraries on demand only (`dynamic import`) instead of app-entry import.
|
|
||||||
- Core Web Vitals targets (p75): `LCP <= 2.5s`, `INP <= 200ms`, `CLS <= 0.1`.
|
|
||||||
- Main-thread task budget: keep individual tasks below ~50ms; split long tasks and yield between chunks.
|
|
||||||
- Enforce frontend budgets in CI (Lighthouse CI with `budget.json`) for critical routes.
|
|
||||||
|
|
||||||
### Performance Budget & PR Evidence
|
|
||||||
- Performance budget is mandatory for hot-path PRs: backend p95/p99 latency and CPU/memory must not regress by more than 5% versus baseline.
|
|
||||||
- Frontend budget: new route-level JS should not increase by more than 30KB gzip without explicit approval.
|
|
||||||
- For any gateway/protocol hot path, attach a reproducible benchmark command and results (input size, concurrency, before/after table).
|
|
||||||
- Profiling evidence is required for major optimizations (`pprof`, flamegraph, browser performance trace, or bundle analyzer output).
|
|
||||||
|
|
||||||
### Quality Gate
|
|
||||||
- Any changed code must include new or updated unit tests.
|
|
||||||
- Coverage must stay above 85% (global frontend threshold and no regressions for touched backend modules).
|
|
||||||
- If any rule is intentionally violated, document reason, risk, and mitigation in the PR description.
|
|
||||||
|
|
||||||
## Testing Guidelines
|
|
||||||
- Backend suites: `go test -tags=unit ./...`, `go test -tags=integration ./...`, and e2e where relevant.
|
|
||||||
- Frontend uses Vitest (`jsdom`); keep tests near modules (`__tests__`) or as `*.spec.ts`.
|
|
||||||
- Enforce unit-test and coverage rules defined in `Quality Gate`.
|
|
||||||
- Before opening a PR, run `make test` plus targeted tests for touched areas.
|
|
||||||
|
|
||||||
## Commit & Pull Request Guidelines
|
|
||||||
- Follow Conventional Commits: `feat(scope): ...`, `fix(scope): ...`, `chore(scope): ...`, `docs(scope): ...`.
|
|
||||||
- PRs should include a clear summary, linked issue/spec, commands run for verification, and screenshots/GIFs for UI changes.
|
|
||||||
- For behavior/API changes, add or update `openspec/changes/...` artifacts.
|
|
||||||
- If dependencies change, commit `frontend/pnpm-lock.yaml` in the same PR.
|
|
||||||
|
|
||||||
## Security & Configuration Tips
|
|
||||||
- Use `deploy/.env.example` and `deploy/config.example.yaml` as templates; do not commit real credentials.
|
|
||||||
- Set stable `JWT_SECRET`, `TOTP_ENCRYPTION_KEY`, and strong database passwords outside local dev.
|
|
||||||
@@ -137,8 +137,6 @@ curl -sSL https://raw.githubusercontent.com/Wei-Shaw/sub2api/main/deploy/install
|
|||||||
|
|
||||||
使用 Docker Compose 部署,包含 PostgreSQL 和 Redis 容器。
|
使用 Docker Compose 部署,包含 PostgreSQL 和 Redis 容器。
|
||||||
|
|
||||||
如果你的服务器是 **Ubuntu 24.04**,建议直接参考:`deploy/ubuntu24-docker-compose-aicodex.md`,其中包含「安装最新版 Docker + docker-compose-aicodex.yml 部署」的完整步骤。
|
|
||||||
|
|
||||||
#### 前置条件
|
#### 前置条件
|
||||||
|
|
||||||
- Docker 20.10+
|
- Docker 20.10+
|
||||||
|
|||||||
@@ -122,7 +122,7 @@ func (s *SettingRepoSuite) TestSet_EmptyValue() {
|
|||||||
func (s *SettingRepoSuite) TestSetMultiple_WithEmptyValues() {
|
func (s *SettingRepoSuite) TestSetMultiple_WithEmptyValues() {
|
||||||
// 模拟保存站点设置,部分字段有值,部分字段为空
|
// 模拟保存站点设置,部分字段有值,部分字段为空
|
||||||
settings := map[string]string{
|
settings := map[string]string{
|
||||||
"site_name": "AICodex2API",
|
"site_name": "Sub2api",
|
||||||
"site_subtitle": "Subscription to API",
|
"site_subtitle": "Subscription to API",
|
||||||
"site_logo": "", // 用户未上传Logo
|
"site_logo": "", // 用户未上传Logo
|
||||||
"api_base_url": "", // 用户未设置API地址
|
"api_base_url": "", // 用户未设置API地址
|
||||||
@@ -136,7 +136,7 @@ func (s *SettingRepoSuite) TestSetMultiple_WithEmptyValues() {
|
|||||||
result, err := s.repo.GetMultiple(s.ctx, []string{"site_name", "site_subtitle", "site_logo", "api_base_url", "contact_info", "doc_url"})
|
result, err := s.repo.GetMultiple(s.ctx, []string{"site_name", "site_subtitle", "site_logo", "api_base_url", "contact_info", "doc_url"})
|
||||||
s.Require().NoError(err, "GetMultiple after SetMultiple with empty values")
|
s.Require().NoError(err, "GetMultiple after SetMultiple with empty values")
|
||||||
|
|
||||||
s.Require().Equal("AICodex2API", result["site_name"])
|
s.Require().Equal("Sub2api", result["site_name"])
|
||||||
s.Require().Equal("Subscription to API", result["site_subtitle"])
|
s.Require().Equal("Subscription to API", result["site_subtitle"])
|
||||||
s.Require().Equal("", result["site_logo"], "empty site_logo should be preserved")
|
s.Require().Equal("", result["site_logo"], "empty site_logo should be preserved")
|
||||||
s.Require().Equal("", result["api_base_url"], "empty api_base_url should be preserved")
|
s.Require().Equal("", result["api_base_url"], "empty api_base_url should be preserved")
|
||||||
|
|||||||
@@ -1,12 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# 本地构建镜像的快速脚本,避免在命令行反复输入构建参数。
|
|
||||||
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
|
|
||||||
docker build -t sub2api:latest \
|
|
||||||
--build-arg GOPROXY=https://goproxy.cn,direct \
|
|
||||||
--build-arg GOSUMDB=sum.golang.google.cn \
|
|
||||||
-f "${SCRIPT_DIR}/Dockerfile" \
|
|
||||||
"${SCRIPT_DIR}"
|
|
||||||
@@ -112,7 +112,7 @@ POSTGRES_DB=sub2api
|
|||||||
DATABASE_PORT=5432
|
DATABASE_PORT=5432
|
||||||
|
|
||||||
# -----------------------------------------------------------------------------
|
# -----------------------------------------------------------------------------
|
||||||
# PostgreSQL 服务端参数(可选;主要用于 deploy/docker-compose-aicodex.yml)
|
# PostgreSQL 服务端参数(可选)
|
||||||
# -----------------------------------------------------------------------------
|
# -----------------------------------------------------------------------------
|
||||||
# POSTGRES_MAX_CONNECTIONS:PostgreSQL 服务端允许的最大连接数。
|
# POSTGRES_MAX_CONNECTIONS:PostgreSQL 服务端允许的最大连接数。
|
||||||
# 必须 >=(所有 Sub2API 实例的 DATABASE_MAX_OPEN_CONNS 之和)+ 预留余量(例如 20%)。
|
# 必须 >=(所有 Sub2API 实例的 DATABASE_MAX_OPEN_CONNS 之和)+ 预留余量(例如 20%)。
|
||||||
@@ -163,7 +163,7 @@ REDIS_PORT=6379
|
|||||||
# Leave empty for no password (default for local development)
|
# Leave empty for no password (default for local development)
|
||||||
REDIS_PASSWORD=
|
REDIS_PASSWORD=
|
||||||
REDIS_DB=0
|
REDIS_DB=0
|
||||||
# Redis 服务端最大客户端连接数(可选;主要用于 deploy/docker-compose-aicodex.yml)
|
# Redis 服务端最大客户端连接数(可选)
|
||||||
REDIS_MAXCLIENTS=50000
|
REDIS_MAXCLIENTS=50000
|
||||||
# Redis 连接池大小(默认 1024)
|
# Redis 连接池大小(默认 1024)
|
||||||
REDIS_POOL_SIZE=4096
|
REDIS_POOL_SIZE=4096
|
||||||
|
|||||||
@@ -1,212 +0,0 @@
|
|||||||
# =============================================================================
|
|
||||||
# Sub2API Docker Compose Test Configuration (Local Build)
|
|
||||||
# =============================================================================
|
|
||||||
# Quick Start:
|
|
||||||
# 1. Copy .env.example to .env and configure
|
|
||||||
# 2. docker-compose -f docker-compose-test.yml up -d --build
|
|
||||||
# 3. Check logs: docker-compose -f docker-compose-test.yml logs -f sub2api
|
|
||||||
# 4. Access: http://localhost:8080
|
|
||||||
#
|
|
||||||
# This configuration builds the image from source (Dockerfile in project root).
|
|
||||||
# All configuration is done via environment variables.
|
|
||||||
# No Setup Wizard needed - the system auto-initializes on first run.
|
|
||||||
# =============================================================================
|
|
||||||
|
|
||||||
services:
|
|
||||||
# ===========================================================================
|
|
||||||
# Sub2API Application
|
|
||||||
# ===========================================================================
|
|
||||||
sub2api:
|
|
||||||
image: sub2api:latest
|
|
||||||
build:
|
|
||||||
context: ..
|
|
||||||
dockerfile: Dockerfile
|
|
||||||
container_name: sub2api
|
|
||||||
restart: unless-stopped
|
|
||||||
ulimits:
|
|
||||||
nofile:
|
|
||||||
soft: 100000
|
|
||||||
hard: 100000
|
|
||||||
ports:
|
|
||||||
- "${BIND_HOST:-0.0.0.0}:${SERVER_PORT:-8080}:8080"
|
|
||||||
volumes:
|
|
||||||
# Data persistence (config.yaml will be auto-generated here)
|
|
||||||
- sub2api_data:/app/data
|
|
||||||
# Mount custom config.yaml (optional, overrides auto-generated config)
|
|
||||||
# - ./config.yaml:/app/data/config.yaml:ro
|
|
||||||
environment:
|
|
||||||
# =======================================================================
|
|
||||||
# Auto Setup (REQUIRED for Docker deployment)
|
|
||||||
# =======================================================================
|
|
||||||
- AUTO_SETUP=true
|
|
||||||
|
|
||||||
# =======================================================================
|
|
||||||
# Server Configuration
|
|
||||||
# =======================================================================
|
|
||||||
- SERVER_HOST=0.0.0.0
|
|
||||||
- SERVER_PORT=8080
|
|
||||||
- SERVER_MODE=${SERVER_MODE:-release}
|
|
||||||
- RUN_MODE=${RUN_MODE:-standard}
|
|
||||||
|
|
||||||
# =======================================================================
|
|
||||||
# Database Configuration (PostgreSQL)
|
|
||||||
# =======================================================================
|
|
||||||
- DATABASE_HOST=postgres
|
|
||||||
- DATABASE_PORT=5432
|
|
||||||
- DATABASE_USER=${POSTGRES_USER:-sub2api}
|
|
||||||
- DATABASE_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required}
|
|
||||||
- DATABASE_DBNAME=${POSTGRES_DB:-sub2api}
|
|
||||||
- DATABASE_SSLMODE=disable
|
|
||||||
- DATABASE_MAX_OPEN_CONNS=${DATABASE_MAX_OPEN_CONNS:-50}
|
|
||||||
- DATABASE_MAX_IDLE_CONNS=${DATABASE_MAX_IDLE_CONNS:-10}
|
|
||||||
- DATABASE_CONN_MAX_LIFETIME_MINUTES=${DATABASE_CONN_MAX_LIFETIME_MINUTES:-30}
|
|
||||||
- DATABASE_CONN_MAX_IDLE_TIME_MINUTES=${DATABASE_CONN_MAX_IDLE_TIME_MINUTES:-5}
|
|
||||||
|
|
||||||
# =======================================================================
|
|
||||||
# Redis Configuration
|
|
||||||
# =======================================================================
|
|
||||||
- REDIS_HOST=redis
|
|
||||||
- REDIS_PORT=6379
|
|
||||||
- REDIS_PASSWORD=${REDIS_PASSWORD:-}
|
|
||||||
- REDIS_DB=${REDIS_DB:-0}
|
|
||||||
- REDIS_POOL_SIZE=${REDIS_POOL_SIZE:-1024}
|
|
||||||
- REDIS_MIN_IDLE_CONNS=${REDIS_MIN_IDLE_CONNS:-10}
|
|
||||||
|
|
||||||
# =======================================================================
|
|
||||||
# Admin Account (auto-created on first run)
|
|
||||||
# =======================================================================
|
|
||||||
- ADMIN_EMAIL=${ADMIN_EMAIL:-admin@sub2api.local}
|
|
||||||
- ADMIN_PASSWORD=${ADMIN_PASSWORD:-}
|
|
||||||
|
|
||||||
# =======================================================================
|
|
||||||
# JWT Configuration
|
|
||||||
# =======================================================================
|
|
||||||
# Leave empty to auto-generate (recommended)
|
|
||||||
- JWT_SECRET=${JWT_SECRET:-}
|
|
||||||
- JWT_EXPIRE_HOUR=${JWT_EXPIRE_HOUR:-24}
|
|
||||||
|
|
||||||
# =======================================================================
|
|
||||||
# Timezone Configuration
|
|
||||||
# This affects ALL time operations in the application:
|
|
||||||
# - Database timestamps
|
|
||||||
# - Usage statistics "today" boundary
|
|
||||||
# - Subscription expiry times
|
|
||||||
# - Log timestamps
|
|
||||||
# Common values: Asia/Shanghai, America/New_York, Europe/London, UTC
|
|
||||||
# =======================================================================
|
|
||||||
- TZ=${TZ:-Asia/Shanghai}
|
|
||||||
|
|
||||||
# =======================================================================
|
|
||||||
# Gemini OAuth Configuration (for Gemini accounts)
|
|
||||||
# =======================================================================
|
|
||||||
- GEMINI_OAUTH_CLIENT_ID=${GEMINI_OAUTH_CLIENT_ID:-}
|
|
||||||
- GEMINI_OAUTH_CLIENT_SECRET=${GEMINI_OAUTH_CLIENT_SECRET:-}
|
|
||||||
- GEMINI_OAUTH_SCOPES=${GEMINI_OAUTH_SCOPES:-}
|
|
||||||
- GEMINI_QUOTA_POLICY=${GEMINI_QUOTA_POLICY:-}
|
|
||||||
|
|
||||||
# Built-in OAuth client secrets (optional)
|
|
||||||
# SECURITY: This repo does not embed third-party client_secret.
|
|
||||||
- GEMINI_CLI_OAUTH_CLIENT_SECRET=${GEMINI_CLI_OAUTH_CLIENT_SECRET:-}
|
|
||||||
- ANTIGRAVITY_OAUTH_CLIENT_SECRET=${ANTIGRAVITY_OAUTH_CLIENT_SECRET:-}
|
|
||||||
|
|
||||||
# =======================================================================
|
|
||||||
# Security Configuration (URL Allowlist)
|
|
||||||
# =======================================================================
|
|
||||||
# Allow private IP addresses for CRS sync (for internal deployments)
|
|
||||||
- SECURITY_URL_ALLOWLIST_ALLOW_PRIVATE_HOSTS=${SECURITY_URL_ALLOWLIST_ALLOW_PRIVATE_HOSTS:-true}
|
|
||||||
depends_on:
|
|
||||||
postgres:
|
|
||||||
condition: service_healthy
|
|
||||||
redis:
|
|
||||||
condition: service_healthy
|
|
||||||
networks:
|
|
||||||
- sub2api-network
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
|
|
||||||
interval: 30s
|
|
||||||
timeout: 10s
|
|
||||||
retries: 3
|
|
||||||
start_period: 30s
|
|
||||||
|
|
||||||
# ===========================================================================
|
|
||||||
# PostgreSQL Database
|
|
||||||
# ===========================================================================
|
|
||||||
postgres:
|
|
||||||
image: postgres:18-alpine
|
|
||||||
container_name: sub2api-postgres
|
|
||||||
restart: unless-stopped
|
|
||||||
ulimits:
|
|
||||||
nofile:
|
|
||||||
soft: 100000
|
|
||||||
hard: 100000
|
|
||||||
volumes:
|
|
||||||
- postgres_data:/var/lib/postgresql/data
|
|
||||||
environment:
|
|
||||||
# postgres:18-alpine 默认 PGDATA=/var/lib/postgresql/18/docker(位于镜像声明的匿名卷 /var/lib/postgresql 内)。
|
|
||||||
# 若不显式设置 PGDATA,则即使挂载了 postgres_data 到 /var/lib/postgresql/data,数据也不会落盘到该命名卷,
|
|
||||||
# docker compose down/up 后会触发 initdb 重新初始化,导致用户/密码等数据丢失。
|
|
||||||
- PGDATA=/var/lib/postgresql/data
|
|
||||||
- POSTGRES_USER=${POSTGRES_USER:-sub2api}
|
|
||||||
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required}
|
|
||||||
- POSTGRES_DB=${POSTGRES_DB:-sub2api}
|
|
||||||
- TZ=${TZ:-Asia/Shanghai}
|
|
||||||
networks:
|
|
||||||
- sub2api-network
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-sub2api} -d ${POSTGRES_DB:-sub2api}"]
|
|
||||||
interval: 10s
|
|
||||||
timeout: 5s
|
|
||||||
retries: 5
|
|
||||||
start_period: 10s
|
|
||||||
# 注意:不暴露端口到宿主机,应用通过内部网络连接
|
|
||||||
# 如需调试,可临时添加:ports: ["127.0.0.1:5433:5432"]
|
|
||||||
|
|
||||||
# ===========================================================================
|
|
||||||
# Redis Cache
|
|
||||||
# ===========================================================================
|
|
||||||
redis:
|
|
||||||
image: redis:8-alpine
|
|
||||||
container_name: sub2api-redis
|
|
||||||
restart: unless-stopped
|
|
||||||
ulimits:
|
|
||||||
nofile:
|
|
||||||
soft: 100000
|
|
||||||
hard: 100000
|
|
||||||
volumes:
|
|
||||||
- redis_data:/data
|
|
||||||
command: >
|
|
||||||
redis-server
|
|
||||||
--save 60 1
|
|
||||||
--appendonly yes
|
|
||||||
--appendfsync everysec
|
|
||||||
${REDIS_PASSWORD:+--requirepass ${REDIS_PASSWORD}}
|
|
||||||
environment:
|
|
||||||
- TZ=${TZ:-Asia/Shanghai}
|
|
||||||
# REDISCLI_AUTH is used by redis-cli for authentication (safer than -a flag)
|
|
||||||
- REDISCLI_AUTH=${REDIS_PASSWORD:-}
|
|
||||||
networks:
|
|
||||||
- sub2api-network
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "redis-cli", "ping"]
|
|
||||||
interval: 10s
|
|
||||||
timeout: 5s
|
|
||||||
retries: 5
|
|
||||||
start_period: 5s
|
|
||||||
|
|
||||||
# =============================================================================
|
|
||||||
# Volumes
|
|
||||||
# =============================================================================
|
|
||||||
volumes:
|
|
||||||
sub2api_data:
|
|
||||||
driver: local
|
|
||||||
postgres_data:
|
|
||||||
driver: local
|
|
||||||
redis_data:
|
|
||||||
driver: local
|
|
||||||
|
|
||||||
# =============================================================================
|
|
||||||
# Networks
|
|
||||||
# =============================================================================
|
|
||||||
networks:
|
|
||||||
sub2api-network:
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,150 +0,0 @@
|
|||||||
# =============================================================================
|
|
||||||
# Docker Compose Override Configuration Example
|
|
||||||
# =============================================================================
|
|
||||||
# This file provides examples for customizing the Docker Compose setup.
|
|
||||||
# Copy this file to docker-compose.override.yml and modify as needed.
|
|
||||||
#
|
|
||||||
# Usage:
|
|
||||||
# cp docker-compose.override.yml.example docker-compose.override.yml
|
|
||||||
# # Edit docker-compose.override.yml with your settings
|
|
||||||
# docker-compose up -d
|
|
||||||
#
|
|
||||||
# IMPORTANT: docker-compose.override.yml is gitignored and will not be committed.
|
|
||||||
# =============================================================================
|
|
||||||
|
|
||||||
# =============================================================================
|
|
||||||
# Scenario 1: Use External Database and Redis (Recommended for Production)
|
|
||||||
# =============================================================================
|
|
||||||
# Use this when you have PostgreSQL and Redis running on the host machine
|
|
||||||
# or on separate servers.
|
|
||||||
#
|
|
||||||
# Prerequisites:
|
|
||||||
# - PostgreSQL running on host (accessible via host.docker.internal)
|
|
||||||
# - Redis running on host (accessible via host.docker.internal)
|
|
||||||
# - Update DATABASE_PORT and REDIS_PORT in .env file if using non-standard ports
|
|
||||||
#
|
|
||||||
# Security Notes:
|
|
||||||
# - Ensure PostgreSQL pg_hba.conf allows connections from Docker network
|
|
||||||
# - Use strong passwords for database and Redis
|
|
||||||
# - Consider using SSL/TLS for database connections in production
|
|
||||||
# =============================================================================
|
|
||||||
|
|
||||||
services:
|
|
||||||
sub2api:
|
|
||||||
# Remove dependencies on containerized postgres/redis
|
|
||||||
depends_on: []
|
|
||||||
|
|
||||||
# Enable access to host machine services
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
|
|
||||||
# Override database and Redis connection settings
|
|
||||||
environment:
|
|
||||||
# PostgreSQL Configuration
|
|
||||||
DATABASE_HOST: host.docker.internal
|
|
||||||
DATABASE_PORT: "5678" # Change to your PostgreSQL port
|
|
||||||
# DATABASE_USER: postgres # Uncomment to override
|
|
||||||
# DATABASE_PASSWORD: your_password # Uncomment to override
|
|
||||||
# DATABASE_DBNAME: sub2api # Uncomment to override
|
|
||||||
|
|
||||||
# Redis Configuration
|
|
||||||
REDIS_HOST: host.docker.internal
|
|
||||||
REDIS_PORT: "6379" # Change to your Redis port
|
|
||||||
# REDIS_PASSWORD: your_redis_password # Uncomment if Redis requires auth
|
|
||||||
# REDIS_DB: 0 # Uncomment to override
|
|
||||||
|
|
||||||
# Disable containerized PostgreSQL
|
|
||||||
postgres:
|
|
||||||
deploy:
|
|
||||||
replicas: 0
|
|
||||||
scale: 0
|
|
||||||
|
|
||||||
# Disable containerized Redis
|
|
||||||
redis:
|
|
||||||
deploy:
|
|
||||||
replicas: 0
|
|
||||||
scale: 0
|
|
||||||
|
|
||||||
# =============================================================================
|
|
||||||
# Scenario 2: Development with Local Services (Alternative)
|
|
||||||
# =============================================================================
|
|
||||||
# Uncomment this section if you want to use the containerized postgres/redis
|
|
||||||
# but expose their ports for local development tools.
|
|
||||||
#
|
|
||||||
# Usage: Comment out Scenario 1 above and uncomment this section.
|
|
||||||
# =============================================================================
|
|
||||||
|
|
||||||
# services:
|
|
||||||
# sub2api:
|
|
||||||
# # Keep default dependencies
|
|
||||||
# pass
|
|
||||||
#
|
|
||||||
# postgres:
|
|
||||||
# ports:
|
|
||||||
# - "127.0.0.1:5432:5432" # Expose PostgreSQL on localhost
|
|
||||||
#
|
|
||||||
# redis:
|
|
||||||
# ports:
|
|
||||||
# - "127.0.0.1:6379:6379" # Expose Redis on localhost
|
|
||||||
|
|
||||||
# =============================================================================
|
|
||||||
# Scenario 3: Custom Network Configuration
|
|
||||||
# =============================================================================
|
|
||||||
# Uncomment if you need to connect to an existing Docker network
|
|
||||||
# =============================================================================
|
|
||||||
|
|
||||||
# networks:
|
|
||||||
# default:
|
|
||||||
# external: true
|
|
||||||
# name: your-existing-network
|
|
||||||
|
|
||||||
# =============================================================================
|
|
||||||
# Scenario 4: Resource Limits (Production)
|
|
||||||
# =============================================================================
|
|
||||||
# Uncomment to set resource limits for the sub2api container
|
|
||||||
# =============================================================================
|
|
||||||
|
|
||||||
# services:
|
|
||||||
# sub2api:
|
|
||||||
# deploy:
|
|
||||||
# resources:
|
|
||||||
# limits:
|
|
||||||
# cpus: '2.0'
|
|
||||||
# memory: 2G
|
|
||||||
# reservations:
|
|
||||||
# cpus: '1.0'
|
|
||||||
# memory: 1G
|
|
||||||
|
|
||||||
# =============================================================================
|
|
||||||
# Scenario 5: Custom Volumes
|
|
||||||
# =============================================================================
|
|
||||||
# Uncomment to mount additional volumes (e.g., for logs, backups)
|
|
||||||
# =============================================================================
|
|
||||||
|
|
||||||
# services:
|
|
||||||
# sub2api:
|
|
||||||
# volumes:
|
|
||||||
# - ./logs:/app/logs
|
|
||||||
# - ./backups:/app/backups
|
|
||||||
|
|
||||||
# =============================================================================
|
|
||||||
# Scenario 6: 启用宿主机 datamanagementd(数据管理)
|
|
||||||
# =============================================================================
|
|
||||||
# 说明:
|
|
||||||
# - datamanagementd 运行在宿主机(systemd 或手动)
|
|
||||||
# - 主进程固定探测 /tmp/sub2api-datamanagement.sock
|
|
||||||
# - 需要把宿主机 socket 挂载到容器内同路径
|
|
||||||
#
|
|
||||||
# services:
|
|
||||||
# sub2api:
|
|
||||||
# volumes:
|
|
||||||
# - /tmp/sub2api-datamanagement.sock:/tmp/sub2api-datamanagement.sock
|
|
||||||
|
|
||||||
# =============================================================================
|
|
||||||
# Additional Notes
|
|
||||||
# =============================================================================
|
|
||||||
# - This file overrides settings in docker-compose.yml
|
|
||||||
# - Environment variables in .env file take precedence
|
|
||||||
# - For more information, see: https://docs.docker.com/compose/extends/
|
|
||||||
# - Check the main README.md for detailed configuration instructions
|
|
||||||
# =============================================================================
|
|
||||||
222
deploy/flow.md
222
deploy/flow.md
@@ -1,222 +0,0 @@
|
|||||||
```mermaid
|
|
||||||
flowchart TD
|
|
||||||
%% Master dispatch
|
|
||||||
A[HTTP Request] --> B{Route}
|
|
||||||
B -->|v1 messages| GA0
|
|
||||||
B -->|openai v1 responses| OA0
|
|
||||||
B -->|v1beta models model action| GM0
|
|
||||||
B -->|v1 messages count tokens| GT0
|
|
||||||
B -->|v1beta models list or get| GL0
|
|
||||||
|
|
||||||
%% =========================
|
|
||||||
%% FLOW A: Claude Gateway
|
|
||||||
%% =========================
|
|
||||||
subgraph FLOW_A["v1 messages Claude Gateway"]
|
|
||||||
GA0[Auth middleware] --> GA1[Read body]
|
|
||||||
GA1 -->|empty| GA1E[400 invalid_request_error]
|
|
||||||
GA1 --> GA2[ParseGatewayRequest]
|
|
||||||
GA2 -->|parse error| GA2E[400 invalid_request_error]
|
|
||||||
GA2 --> GA3{model present}
|
|
||||||
GA3 -->|no| GA3E[400 invalid_request_error]
|
|
||||||
GA3 --> GA4[streamStarted false]
|
|
||||||
GA4 --> GA5[IncrementWaitCount user]
|
|
||||||
GA5 -->|queue full| GA5E[429 rate_limit_error]
|
|
||||||
GA5 --> GA6[AcquireUserSlotWithWait]
|
|
||||||
GA6 -->|timeout or fail| GA6E[429 rate_limit_error]
|
|
||||||
GA6 --> GA7[BillingEligibility check post wait]
|
|
||||||
GA7 -->|fail| GA7E[403 billing_error]
|
|
||||||
GA7 --> GA8[Generate sessionHash]
|
|
||||||
GA8 --> GA9[Resolve platform]
|
|
||||||
GA9 --> GA10{platform gemini}
|
|
||||||
GA10 -->|yes| GA10Y[sessionKey gemini hash]
|
|
||||||
GA10 -->|no| GA10N[sessionKey hash]
|
|
||||||
GA10Y --> GA11
|
|
||||||
GA10N --> GA11
|
|
||||||
|
|
||||||
GA11[SelectAccountWithLoadAwareness] -->|err and no failed| GA11E1[503 no available accounts]
|
|
||||||
GA11 -->|err and failed| GA11E2[map failover error]
|
|
||||||
GA11 --> GA12[Warmup intercept]
|
|
||||||
GA12 -->|yes| GA12Y[return mock and release if held]
|
|
||||||
GA12 -->|no| GA13[Acquire account slot or wait]
|
|
||||||
GA13 -->|wait queue full| GA13E1[429 rate_limit_error]
|
|
||||||
GA13 -->|wait timeout| GA13E2[429 concurrency limit]
|
|
||||||
GA13 --> GA14[BindStickySession if waited]
|
|
||||||
GA14 --> GA15{account platform antigravity}
|
|
||||||
GA15 -->|yes| GA15Y[ForwardGemini antigravity]
|
|
||||||
GA15 -->|no| GA15N[Forward Claude]
|
|
||||||
GA15Y --> GA16[Release account slot and dec account wait]
|
|
||||||
GA15N --> GA16
|
|
||||||
GA16 --> GA17{UpstreamFailoverError}
|
|
||||||
GA17 -->|yes| GA18[mark failedAccountIDs and map error if exceed]
|
|
||||||
GA18 -->|loop| GA11
|
|
||||||
GA17 -->|no| GA19[success async RecordUsage and return]
|
|
||||||
GA19 --> GA20[defer release user slot and dec wait count]
|
|
||||||
end
|
|
||||||
|
|
||||||
%% =========================
|
|
||||||
%% FLOW B: OpenAI
|
|
||||||
%% =========================
|
|
||||||
subgraph FLOW_B["openai v1 responses"]
|
|
||||||
OA0[Auth middleware] --> OA1[Read body]
|
|
||||||
OA1 -->|empty| OA1E[400 invalid_request_error]
|
|
||||||
OA1 --> OA2[json Unmarshal body]
|
|
||||||
OA2 -->|parse error| OA2E[400 invalid_request_error]
|
|
||||||
OA2 --> OA3{model present}
|
|
||||||
OA3 -->|no| OA3E[400 invalid_request_error]
|
|
||||||
OA3 --> OA4{User Agent Codex CLI}
|
|
||||||
OA4 -->|no| OA4N[set default instructions]
|
|
||||||
OA4 -->|yes| OA4Y[no change]
|
|
||||||
OA4N --> OA5
|
|
||||||
OA4Y --> OA5
|
|
||||||
OA5[streamStarted false] --> OA6[IncrementWaitCount user]
|
|
||||||
OA6 -->|queue full| OA6E[429 rate_limit_error]
|
|
||||||
OA6 --> OA7[AcquireUserSlotWithWait]
|
|
||||||
OA7 -->|timeout or fail| OA7E[429 rate_limit_error]
|
|
||||||
OA7 --> OA8[BillingEligibility check post wait]
|
|
||||||
OA8 -->|fail| OA8E[403 billing_error]
|
|
||||||
OA8 --> OA9[sessionHash sha256 session_id]
|
|
||||||
OA9 --> OA10[SelectAccountWithLoadAwareness]
|
|
||||||
OA10 -->|err and no failed| OA10E1[503 no available accounts]
|
|
||||||
OA10 -->|err and failed| OA10E2[map failover error]
|
|
||||||
OA10 --> OA11[Acquire account slot or wait]
|
|
||||||
OA11 -->|wait queue full| OA11E1[429 rate_limit_error]
|
|
||||||
OA11 -->|wait timeout| OA11E2[429 concurrency limit]
|
|
||||||
OA11 --> OA12[BindStickySession openai hash if waited]
|
|
||||||
OA12 --> OA13[Forward OpenAI upstream]
|
|
||||||
OA13 --> OA14[Release account slot and dec account wait]
|
|
||||||
OA14 --> OA15{UpstreamFailoverError}
|
|
||||||
OA15 -->|yes| OA16[mark failedAccountIDs and map error if exceed]
|
|
||||||
OA16 -->|loop| OA10
|
|
||||||
OA15 -->|no| OA17[success async RecordUsage and return]
|
|
||||||
OA17 --> OA18[defer release user slot and dec wait count]
|
|
||||||
end
|
|
||||||
|
|
||||||
%% =========================
|
|
||||||
%% FLOW C: Gemini Native
|
|
||||||
%% =========================
|
|
||||||
subgraph FLOW_C["v1beta models model action Gemini Native"]
|
|
||||||
GM0[Auth middleware] --> GM1[Validate platform]
|
|
||||||
GM1 -->|invalid| GM1E[400 googleError]
|
|
||||||
GM1 --> GM2[Parse path modelName action]
|
|
||||||
GM2 -->|invalid| GM2E[400 googleError]
|
|
||||||
GM2 --> GM3{action supported}
|
|
||||||
GM3 -->|no| GM3E[404 googleError]
|
|
||||||
GM3 --> GM4[Read body]
|
|
||||||
GM4 -->|empty| GM4E[400 googleError]
|
|
||||||
GM4 --> GM5[streamStarted false]
|
|
||||||
GM5 --> GM6[IncrementWaitCount user]
|
|
||||||
GM6 -->|queue full| GM6E[429 googleError]
|
|
||||||
GM6 --> GM7[AcquireUserSlotWithWait]
|
|
||||||
GM7 -->|timeout or fail| GM7E[429 googleError]
|
|
||||||
GM7 --> GM8[BillingEligibility check post wait]
|
|
||||||
GM8 -->|fail| GM8E[403 googleError]
|
|
||||||
GM8 --> GM9[Generate sessionHash]
|
|
||||||
GM9 --> GM10[sessionKey gemini hash]
|
|
||||||
GM10 --> GM11[SelectAccountWithLoadAwareness]
|
|
||||||
GM11 -->|err and no failed| GM11E1[503 googleError]
|
|
||||||
GM11 -->|err and failed| GM11E2[mapGeminiUpstreamError]
|
|
||||||
GM11 --> GM12[Acquire account slot or wait]
|
|
||||||
GM12 -->|wait queue full| GM12E1[429 googleError]
|
|
||||||
GM12 -->|wait timeout| GM12E2[429 googleError]
|
|
||||||
GM12 --> GM13[BindStickySession if waited]
|
|
||||||
GM13 --> GM14{account platform antigravity}
|
|
||||||
GM14 -->|yes| GM14Y[ForwardGemini antigravity]
|
|
||||||
GM14 -->|no| GM14N[ForwardNative]
|
|
||||||
GM14Y --> GM15[Release account slot and dec account wait]
|
|
||||||
GM14N --> GM15
|
|
||||||
GM15 --> GM16{UpstreamFailoverError}
|
|
||||||
GM16 -->|yes| GM17[mark failedAccountIDs and map error if exceed]
|
|
||||||
GM17 -->|loop| GM11
|
|
||||||
GM16 -->|no| GM18[success async RecordUsage and return]
|
|
||||||
GM18 --> GM19[defer release user slot and dec wait count]
|
|
||||||
end
|
|
||||||
|
|
||||||
%% =========================
|
|
||||||
%% FLOW D: CountTokens
|
|
||||||
%% =========================
|
|
||||||
subgraph FLOW_D["v1 messages count tokens"]
|
|
||||||
GT0[Auth middleware] --> GT1[Read body]
|
|
||||||
GT1 -->|empty| GT1E[400 invalid_request_error]
|
|
||||||
GT1 --> GT2[ParseGatewayRequest]
|
|
||||||
GT2 -->|parse error| GT2E[400 invalid_request_error]
|
|
||||||
GT2 --> GT3{model present}
|
|
||||||
GT3 -->|no| GT3E[400 invalid_request_error]
|
|
||||||
GT3 --> GT4[BillingEligibility check]
|
|
||||||
GT4 -->|fail| GT4E[403 billing_error]
|
|
||||||
GT4 --> GT5[ForwardCountTokens]
|
|
||||||
end
|
|
||||||
|
|
||||||
%% =========================
|
|
||||||
%% FLOW E: Gemini Models List Get
|
|
||||||
%% =========================
|
|
||||||
subgraph FLOW_E["v1beta models list or get"]
|
|
||||||
GL0[Auth middleware] --> GL1[Validate platform]
|
|
||||||
GL1 -->|invalid| GL1E[400 googleError]
|
|
||||||
GL1 --> GL2{force platform antigravity}
|
|
||||||
GL2 -->|yes| GL2Y[return static fallback models]
|
|
||||||
GL2 -->|no| GL3[SelectAccountForAIStudioEndpoints]
|
|
||||||
GL3 -->|no gemini and has antigravity| GL3Y[return fallback models]
|
|
||||||
GL3 -->|no accounts| GL3E[503 googleError]
|
|
||||||
GL3 --> GL4[ForwardAIStudioGET]
|
|
||||||
GL4 -->|error| GL4E[502 googleError]
|
|
||||||
GL4 --> GL5[Passthrough response or fallback]
|
|
||||||
end
|
|
||||||
|
|
||||||
%% =========================
|
|
||||||
%% SHARED: Account Selection
|
|
||||||
%% =========================
|
|
||||||
subgraph SELECT["SelectAccountWithLoadAwareness detail"]
|
|
||||||
S0[Start] --> S1{concurrencyService nil OR load batch disabled}
|
|
||||||
S1 -->|yes| S2[SelectAccountForModelWithExclusions legacy]
|
|
||||||
S2 --> S3[tryAcquireAccountSlot]
|
|
||||||
S3 -->|acquired| S3Y[SelectionResult Acquired true ReleaseFunc]
|
|
||||||
S3 -->|not acquired| S3N[WaitPlan FallbackTimeout MaxWaiting]
|
|
||||||
S1 -->|no| S4[Resolve platform]
|
|
||||||
S4 --> S5[List schedulable accounts]
|
|
||||||
S5 --> S6[Layer1 Sticky session]
|
|
||||||
S6 -->|hit and valid| S6A[tryAcquireAccountSlot]
|
|
||||||
S6A -->|acquired| S6AY[SelectionResult Acquired true]
|
|
||||||
S6A -->|not acquired and waitingCount < StickyMax| S6AN[WaitPlan StickyTimeout Max]
|
|
||||||
S6 --> S7[Layer2 Load aware]
|
|
||||||
S7 --> S7A[Load batch concurrency plus wait to loadRate]
|
|
||||||
S7A --> S7B[Sort priority load LRU OAuth prefer for Gemini]
|
|
||||||
S7B --> S7C[tryAcquireAccountSlot in order]
|
|
||||||
S7C -->|first success| S7CY[SelectionResult Acquired true]
|
|
||||||
S7C -->|none| S8[Layer3 Fallback wait]
|
|
||||||
S8 --> S8A[Sort priority LRU]
|
|
||||||
S8A --> S8B[WaitPlan FallbackTimeout Max]
|
|
||||||
end
|
|
||||||
|
|
||||||
%% =========================
|
|
||||||
%% SHARED: Wait Acquire
|
|
||||||
%% =========================
|
|
||||||
subgraph WAIT["AcquireXSlotWithWait detail"]
|
|
||||||
W0[Try AcquireXSlot immediately] -->|acquired| W1[return ReleaseFunc]
|
|
||||||
W0 -->|not acquired| W2[Wait loop with timeout]
|
|
||||||
W2 --> W3[Backoff 100ms x1.5 jitter max2s]
|
|
||||||
W2 --> W4[If streaming and ping format send SSE ping]
|
|
||||||
W2 --> W5[Retry AcquireXSlot on timer]
|
|
||||||
W5 -->|acquired| W1
|
|
||||||
W2 -->|timeout| W6[ConcurrencyError IsTimeout true]
|
|
||||||
end
|
|
||||||
|
|
||||||
%% =========================
|
|
||||||
%% SHARED: Account Wait Queue
|
|
||||||
%% =========================
|
|
||||||
subgraph AQ["Account Wait Queue Redis Lua"]
|
|
||||||
Q1[IncrementAccountWaitCount] --> Q2{current >= max}
|
|
||||||
Q2 -->|yes| Q2Y[return false]
|
|
||||||
Q2 -->|no| Q3[INCR and if first set TTL]
|
|
||||||
Q3 --> Q4[return true]
|
|
||||||
Q5[DecrementAccountWaitCount] --> Q6[if current > 0 then DECR]
|
|
||||||
end
|
|
||||||
|
|
||||||
%% =========================
|
|
||||||
%% SHARED: Background cleanup
|
|
||||||
%% =========================
|
|
||||||
subgraph CLEANUP["Slot Cleanup Worker"]
|
|
||||||
C0[StartSlotCleanupWorker interval] --> C1[List schedulable accounts]
|
|
||||||
C1 --> C2[CleanupExpiredAccountSlots per account]
|
|
||||||
C2 --> C3[Repeat every interval]
|
|
||||||
end
|
|
||||||
```
|
|
||||||
@@ -1,249 +0,0 @@
|
|||||||
# 后端热点 API 性能优化审计与行动计划(2026-02-22)
|
|
||||||
|
|
||||||
## 1. 目标与范围
|
|
||||||
|
|
||||||
本次文档用于沉淀后端热点 API 的性能审计结果,并给出可执行优化方案。
|
|
||||||
|
|
||||||
重点链路:
|
|
||||||
- `POST /v1/messages`
|
|
||||||
- `POST /v1/responses`
|
|
||||||
- `POST /sora/v1/chat/completions`
|
|
||||||
- `POST /v1beta/models/*modelAction`(Gemini 兼容链路)
|
|
||||||
- 相关调度、计费、Ops 记录链路
|
|
||||||
|
|
||||||
## 2. 审计方式与结论边界
|
|
||||||
|
|
||||||
- 审计方式:静态代码审阅(只读),未对生产环境做侵入变更。
|
|
||||||
- 结论类型:以“高置信度可优化点”为主,均附 `file:line` 证据。
|
|
||||||
- 未覆盖项:本轮未执行压测与火焰图采样,吞吐增益需在压测环境量化确认。
|
|
||||||
|
|
||||||
## 3. 优先级总览
|
|
||||||
|
|
||||||
| 优先级 | 数量 | 结论 |
|
|
||||||
|---|---:|---|
|
|
||||||
| P0(Critical) | 2 | 存在资源失控风险,建议立即修复 |
|
|
||||||
| P1(High) | 2 | 明确的热点 DB/Redis 放大路径,建议本迭代完成 |
|
|
||||||
| P2(Medium) | 4 | 可观收益优化项,建议并行排期 |
|
|
||||||
|
|
||||||
## 4. 详细问题清单
|
|
||||||
|
|
||||||
### 4.1 P0-1:使用量记录为“每请求一个 goroutine”,高峰下可能无界堆积
|
|
||||||
|
|
||||||
证据位置:
|
|
||||||
- `backend/internal/handler/gateway_handler.go:435`
|
|
||||||
- `backend/internal/handler/gateway_handler.go:704`
|
|
||||||
- `backend/internal/handler/openai_gateway_handler.go:382`
|
|
||||||
- `backend/internal/handler/sora_gateway_handler.go:400`
|
|
||||||
- `backend/internal/handler/gemini_v1beta_handler.go:523`
|
|
||||||
|
|
||||||
问题描述:
|
|
||||||
- 记录用量使用 `go func(...)` 直接异步提交,未设置全局并发上限与排队背压。
|
|
||||||
- 当 DB/Redis 变慢时,goroutine 数会随请求持续累积。
|
|
||||||
|
|
||||||
性能影响:
|
|
||||||
- `goroutine` 激增导致调度开销上升与内存占用增加。
|
|
||||||
- 与数据库连接池(默认 `max_open_conns=256`)竞争,放大尾延迟。
|
|
||||||
|
|
||||||
优化建议:
|
|
||||||
- 引入“有界队列 + 固定 worker 池”替代每请求 goroutine。
|
|
||||||
- 队列满时采用明确策略:丢弃(采样告警)或降级为同步短路。
|
|
||||||
- 为 `RecordUsage` 路径增加超时、重试上限与失败计数指标。
|
|
||||||
|
|
||||||
验收指标:
|
|
||||||
- 峰值 `goroutines` 稳定,无线性增长。
|
|
||||||
- 用量记录成功率、丢弃率、队列长度可观测。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 4.2 P0-2:Ops 错误日志队列携带原始请求体,存在内存放大风险
|
|
||||||
|
|
||||||
证据位置:
|
|
||||||
- 队列容量与 job 结构:`backend/internal/handler/ops_error_logger.go:38`、`backend/internal/handler/ops_error_logger.go:43`
|
|
||||||
- 入队逻辑:`backend/internal/handler/ops_error_logger.go:132`
|
|
||||||
- 请求体放入 context:`backend/internal/handler/ops_error_logger.go:261`
|
|
||||||
- 读取并入队:`backend/internal/handler/ops_error_logger.go:548`、`backend/internal/handler/ops_error_logger.go:563`、`backend/internal/handler/ops_error_logger.go:727`、`backend/internal/handler/ops_error_logger.go:737`
|
|
||||||
- 入库前才裁剪:`backend/internal/service/ops_service.go:332`、`backend/internal/service/ops_service.go:339`
|
|
||||||
- 请求体默认上限:`backend/internal/config/config.go:1082`、`backend/internal/config/config.go:1086`
|
|
||||||
|
|
||||||
问题描述:
|
|
||||||
- 队列元素包含 `[]byte requestBody`,在请求体较大且错误风暴时会显著占用内存。
|
|
||||||
- 当前裁剪发生在 worker 消费时,而不是入队前。
|
|
||||||
|
|
||||||
性能影响:
|
|
||||||
- 容易造成瞬时高内存与频繁 GC。
|
|
||||||
- 极端情况下可能触发 OOM 或服务抖动。
|
|
||||||
|
|
||||||
优化建议:
|
|
||||||
- 入队前进行“脱敏 + 裁剪”,仅保留小尺寸结构化片段(建议 8KB~16KB)。
|
|
||||||
- 队列存放轻量 DTO,避免持有大块 `[]byte`。
|
|
||||||
- 按错误类型控制采样率,避免同类错误洪峰时日志放大。
|
|
||||||
|
|
||||||
验收指标:
|
|
||||||
- Ops 错误风暴期间 RSS/GC 次数显著下降。
|
|
||||||
- 队列满时系统稳定且告警可见。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 4.3 P1-1:窗口费用检查在缓存 miss 时逐账号做 DB 聚合
|
|
||||||
|
|
||||||
证据位置:
|
|
||||||
- 候选筛选多处调用:`backend/internal/service/gateway_service.go:1109`、`backend/internal/service/gateway_service.go:1137`、`backend/internal/service/gateway_service.go:1291`、`backend/internal/service/gateway_service.go:1354`
|
|
||||||
- miss 后单账号聚合:`backend/internal/service/gateway_service.go:1791`
|
|
||||||
- SQL 聚合实现:`backend/internal/repository/usage_log_repo.go:889`
|
|
||||||
- 窗口费用缓存 TTL:`backend/internal/repository/session_limit_cache.go:33`
|
|
||||||
- 已有批量读取接口但未利用:`backend/internal/repository/session_limit_cache.go:310`
|
|
||||||
|
|
||||||
问题描述:
|
|
||||||
- 路由候选过滤阶段频繁调用窗口费用检查。
|
|
||||||
- 缓存未命中时逐账号执行聚合查询,账号多时放大 DB 压力。
|
|
||||||
|
|
||||||
性能影响:
|
|
||||||
- 路由耗时上升,数据库聚合 QPS 增长。
|
|
||||||
- 高并发下可能形成“缓存抖动 + 聚合风暴”。
|
|
||||||
|
|
||||||
优化建议:
|
|
||||||
- 先批量 `GetWindowCostBatch`,仅对 miss 账号执行批量 SQL 聚合。
|
|
||||||
- 将聚合结果批量回写缓存,降低重复查询。
|
|
||||||
- 评估窗口费用缓存 TTL 与刷新策略,减少抖动。
|
|
||||||
|
|
||||||
验收指标:
|
|
||||||
- 路由阶段 DB 查询次数下降。
|
|
||||||
- `SelectAccountWithLoadAwareness` 平均耗时下降。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 4.4 P1-2:记录用量时每次查询用户分组倍率,形成稳定 DB 热点
|
|
||||||
|
|
||||||
证据位置:
|
|
||||||
- `backend/internal/service/gateway_service.go:5316`
|
|
||||||
- `backend/internal/service/gateway_service.go:5531`
|
|
||||||
- `backend/internal/repository/user_group_rate_repo.go:45`
|
|
||||||
|
|
||||||
问题描述:
|
|
||||||
- `RecordUsage` 与 `RecordUsageWithLongContext` 每次都执行 `GetByUserAndGroup`。
|
|
||||||
- 热路径重复读数据库,且与 usage 写入、扣费路径竞争连接池。
|
|
||||||
|
|
||||||
性能影响:
|
|
||||||
- 增加 DB 往返与延迟,降低热点接口吞吐。
|
|
||||||
|
|
||||||
优化建议:
|
|
||||||
- 在鉴权或路由阶段预热倍率并挂载上下文复用。
|
|
||||||
- 引入 L1/L2 缓存(短 TTL + singleflight),减少重复 SQL。
|
|
||||||
|
|
||||||
验收指标:
|
|
||||||
- `GetByUserAndGroup` 调用量明显下降。
|
|
||||||
- 计费链路 p95 延迟下降。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 4.5 P2-1:Claude 消息链路重复 JSON 解析
|
|
||||||
|
|
||||||
证据位置:
|
|
||||||
- 首次解析:`backend/internal/handler/gateway_handler.go:129`
|
|
||||||
- 二次解析入口:`backend/internal/handler/gateway_handler.go:146`
|
|
||||||
- 二次 `json.Unmarshal`:`backend/internal/handler/gateway_helper.go:22`、`backend/internal/handler/gateway_helper.go:26`
|
|
||||||
|
|
||||||
问题描述:
|
|
||||||
- 同一请求先 `ParseGatewayRequest`,后 `SetClaudeCodeClientContext` 再做 `Unmarshal`。
|
|
||||||
|
|
||||||
性能影响:
|
|
||||||
- 增加 CPU 与内存分配,尤其对大 `messages` 请求更明显。
|
|
||||||
|
|
||||||
优化建议:
|
|
||||||
- 仅在 `User-Agent` 命中 Claude CLI 规则后再做 body 深解析。
|
|
||||||
- 或直接复用首轮解析结果,避免重复反序列化。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 4.6 P2-2:同一请求中粘性会话账号查询存在重复 Redis 读取
|
|
||||||
|
|
||||||
证据位置:
|
|
||||||
- Handler 预取:`backend/internal/handler/gateway_handler.go:242`
|
|
||||||
- Service 再取:`backend/internal/service/gateway_service.go:941`、`backend/internal/service/gateway_service.go:1129`、`backend/internal/service/gateway_service.go:1277`
|
|
||||||
|
|
||||||
问题描述:
|
|
||||||
- 同一会话映射在同请求链路被多次读取。
|
|
||||||
|
|
||||||
性能影响:
|
|
||||||
- 增加 Redis RTT 与序列化开销,抬高路由延迟。
|
|
||||||
|
|
||||||
优化建议:
|
|
||||||
- 统一在 `SelectAccountWithLoadAwareness` 内读取并复用。
|
|
||||||
- 或将上层已读到的 sticky account 显式透传给 service。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 4.7 P2-3:并发等待路径存在重复抢槽
|
|
||||||
|
|
||||||
证据位置:
|
|
||||||
- 首次 TryAcquire:`backend/internal/handler/gateway_helper.go:182`、`backend/internal/handler/gateway_helper.go:202`
|
|
||||||
- wait 内再次立即 Acquire:`backend/internal/handler/gateway_helper.go:226`、`backend/internal/handler/gateway_helper.go:230`、`backend/internal/handler/gateway_helper.go:232`
|
|
||||||
|
|
||||||
问题描述:
|
|
||||||
- 进入 wait 流程后会再做一次“立即抢槽”,与上层 TryAcquire 重复。
|
|
||||||
|
|
||||||
性能影响:
|
|
||||||
- 在高并发下增加 Redis 操作次数,放大锁竞争。
|
|
||||||
|
|
||||||
优化建议:
|
|
||||||
- wait 流程直接进入退避循环,避免重复立即抢槽。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 4.8 P2-4:`/v1/models` 每次走仓储查询与对象装配,未复用快照/短缓存
|
|
||||||
|
|
||||||
证据位置:
|
|
||||||
- 入口调用:`backend/internal/handler/gateway_handler.go:767`
|
|
||||||
- 服务查询:`backend/internal/service/gateway_service.go:6152`、`backend/internal/service/gateway_service.go:6154`
|
|
||||||
- 对象装配:`backend/internal/repository/account_repo.go:1276`、`backend/internal/repository/account_repo.go:1290`、`backend/internal/repository/account_repo.go:1298`
|
|
||||||
|
|
||||||
问题描述:
|
|
||||||
- 模型列表请求每次都落到账号查询与附加装配,缺少短时缓存。
|
|
||||||
|
|
||||||
性能影响:
|
|
||||||
- 高频请求下持续占用 DB 与 CPU。
|
|
||||||
|
|
||||||
优化建议:
|
|
||||||
- 以 `groupID + platform` 建 10s~30s 本地缓存。
|
|
||||||
- 或复用调度快照 bucket 的可用账号结果做模型聚合。
|
|
||||||
|
|
||||||
## 5. 建议实施顺序
|
|
||||||
|
|
||||||
### 阶段 A(立即,P0)
|
|
||||||
- 将“用量记录每请求 goroutine”改为有界异步管道。
|
|
||||||
- Ops 错误日志改为“入队前裁剪 + 轻量队列对象”。
|
|
||||||
|
|
||||||
### 阶段 B(短期,P1)
|
|
||||||
- 批量化窗口费用检查(缓存 + SQL 双批量)。
|
|
||||||
- 用户分组倍率加缓存/上下文复用。
|
|
||||||
|
|
||||||
### 阶段 C(中期,P2)
|
|
||||||
- 消除重复 JSON 解析与重复 sticky 查询。
|
|
||||||
- 优化并发等待重复抢槽逻辑。
|
|
||||||
- `/v1/models` 接口加入短缓存或快照复用。
|
|
||||||
|
|
||||||
## 6. 压测与验证建议
|
|
||||||
|
|
||||||
建议在预发压测以下场景:
|
|
||||||
- 场景 1:常规成功流量(验证吞吐与延迟)。
|
|
||||||
- 场景 2:上游慢响应(验证 goroutine 与队列稳定性)。
|
|
||||||
- 场景 3:错误风暴(验证 Ops 队列与内存上限)。
|
|
||||||
- 场景 4:多账号大分组路由(验证窗口费用批量化收益)。
|
|
||||||
|
|
||||||
建议监控指标:
|
|
||||||
- 进程:`goroutines`、RSS、GC 次数/停顿。
|
|
||||||
- API:各热点接口 p50/p95/p99。
|
|
||||||
- DB:QPS、慢查询、连接池等待。
|
|
||||||
- Redis:命中率、RTT、命令量。
|
|
||||||
- 业务:用量记录成功率/丢弃率、Ops 日志丢弃率。
|
|
||||||
|
|
||||||
## 7. 待补充数据
|
|
||||||
|
|
||||||
- 生产真实错误率与错误体大小分布。
|
|
||||||
- `window_cost_limit` 实际启用账号比例。
|
|
||||||
- `/v1/models` 实际调用频次。
|
|
||||||
- DB/Redis 当前容量余量与瓶颈点。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
如需进入实现阶段,建议按“阶段 A → 阶段 B → 阶段 C”分 PR 推进,每个阶段都附压测报告与回滚方案。
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
-- 修正 schema_migrations 中“本地改名”的迁移文件名
|
|
||||||
-- 适用场景:你已执行过旧文件名的迁移,合并后仅改了自己这边的文件名
|
|
||||||
|
|
||||||
BEGIN;
|
|
||||||
|
|
||||||
UPDATE schema_migrations
|
|
||||||
SET filename = '042b_add_ops_system_metrics_switch_count.sql'
|
|
||||||
WHERE filename = '042_add_ops_system_metrics_switch_count.sql'
|
|
||||||
AND NOT EXISTS (
|
|
||||||
SELECT 1 FROM schema_migrations WHERE filename = '042b_add_ops_system_metrics_switch_count.sql'
|
|
||||||
);
|
|
||||||
|
|
||||||
UPDATE schema_migrations
|
|
||||||
SET filename = '043b_add_group_invalid_request_fallback.sql'
|
|
||||||
WHERE filename = '043_add_group_invalid_request_fallback.sql'
|
|
||||||
AND NOT EXISTS (
|
|
||||||
SELECT 1 FROM schema_migrations WHERE filename = '043b_add_group_invalid_request_fallback.sql'
|
|
||||||
);
|
|
||||||
|
|
||||||
UPDATE schema_migrations
|
|
||||||
SET filename = '044b_add_group_mcp_xml_inject.sql'
|
|
||||||
WHERE filename = '044_add_group_mcp_xml_inject.sql'
|
|
||||||
AND NOT EXISTS (
|
|
||||||
SELECT 1 FROM schema_migrations WHERE filename = '044b_add_group_mcp_xml_inject.sql'
|
|
||||||
);
|
|
||||||
|
|
||||||
UPDATE schema_migrations
|
|
||||||
SET filename = '046b_add_group_supported_model_scopes.sql'
|
|
||||||
WHERE filename = '046_add_group_supported_model_scopes.sql'
|
|
||||||
AND NOT EXISTS (
|
|
||||||
SELECT 1 FROM schema_migrations WHERE filename = '046b_add_group_supported_model_scopes.sql'
|
|
||||||
);
|
|
||||||
|
|
||||||
COMMIT;
|
|
||||||
@@ -10,6 +10,7 @@ import { resolve } from 'path'
|
|||||||
function injectPublicSettings(backendUrl: string): Plugin {
|
function injectPublicSettings(backendUrl: string): Plugin {
|
||||||
return {
|
return {
|
||||||
name: 'inject-public-settings',
|
name: 'inject-public-settings',
|
||||||
|
apply: 'serve',
|
||||||
transformIndexHtml: {
|
transformIndexHtml: {
|
||||||
order: 'pre',
|
order: 'pre',
|
||||||
async handler(html) {
|
async handler(html) {
|
||||||
|
|||||||
@@ -1,20 +0,0 @@
|
|||||||
schema: spec-driven
|
|
||||||
|
|
||||||
# Project context (optional)
|
|
||||||
# This is shown to AI when creating artifacts.
|
|
||||||
# Add your tech stack, conventions, style guides, domain knowledge, etc.
|
|
||||||
# Example:
|
|
||||||
# context: |
|
|
||||||
# Tech stack: TypeScript, React, Node.js
|
|
||||||
# We use conventional commits
|
|
||||||
# Domain: e-commerce platform
|
|
||||||
|
|
||||||
# Per-artifact rules (optional)
|
|
||||||
# Add custom rules for specific artifacts.
|
|
||||||
# Example:
|
|
||||||
# rules:
|
|
||||||
# proposal:
|
|
||||||
# - Keep proposals under 500 words
|
|
||||||
# - Always include a "Non-goals" section
|
|
||||||
# tasks:
|
|
||||||
# - Break tasks into chunks of max 2 hours
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
# Project Context
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
[Describe your project's purpose and goals]
|
|
||||||
|
|
||||||
## Tech Stack
|
|
||||||
- [List your primary technologies]
|
|
||||||
- [e.g., TypeScript, React, Node.js]
|
|
||||||
|
|
||||||
## Project Conventions
|
|
||||||
|
|
||||||
### Code Style
|
|
||||||
[Describe your code style preferences, formatting rules, and naming conventions]
|
|
||||||
|
|
||||||
### Architecture Patterns
|
|
||||||
[Document your architectural decisions and patterns]
|
|
||||||
|
|
||||||
### Testing Strategy
|
|
||||||
[Explain your testing approach and requirements]
|
|
||||||
|
|
||||||
### Git Workflow
|
|
||||||
[Describe your branching strategy and commit conventions]
|
|
||||||
|
|
||||||
## Domain Context
|
|
||||||
[Add domain-specific knowledge that AI assistants need to understand]
|
|
||||||
|
|
||||||
## Important Constraints
|
|
||||||
[List any technical, business, or regulatory constraints]
|
|
||||||
|
|
||||||
## External Dependencies
|
|
||||||
[Document key external services, APIs, or systems]
|
|
||||||
@@ -1,679 +0,0 @@
|
|||||||
---
|
|
||||||
name: bug-fix-expert
|
|
||||||
description: 以"先确认、再修复"的多智能体协作方式处理缺陷,保证速度和安全。
|
|
||||||
license: MIT
|
|
||||||
compatibility: Claude Code(支持 Task 工具时启用并行协作,否则自动降级为单智能体顺序执行)。
|
|
||||||
metadata:
|
|
||||||
author: project-team
|
|
||||||
version: "4.3"
|
|
||||||
---
|
|
||||||
|
|
||||||
# Bug 修复专家(bug-fix-expert)
|
|
||||||
|
|
||||||
## 术语表
|
|
||||||
|
|
||||||
| 术语 | 定义 |
|
|
||||||
|------|------|
|
|
||||||
| **主控** | 主会话,负责协调流程、管理 worktree 生命周期、与用户沟通 |
|
|
||||||
| **子智能体** | 通过 Task 工具启动的独立 agent,执行具体任务后返回结果 |
|
|
||||||
| **角色** | 抽象职责分类(验证/分析/修复/安全/审查),映射到具体的子智能体 |
|
|
||||||
| **Beacon** | 完成信标(Completion Beacon),子智能体的结构化完成报告 |
|
|
||||||
| **Worktree** | 通过 `git worktree` 创建的隔离工作目录 |
|
|
||||||
| **三重门禁** | 交付前必须同时满足的三个条件:测试通过 + 审查通过 + 安全通过 |
|
|
||||||
|
|
||||||
## 触发条件
|
|
||||||
|
|
||||||
当以下任一条件满足时激活本技能:
|
|
||||||
|
|
||||||
- 用户明确报告 bug、异常、CI 失败、线上问题。
|
|
||||||
- 用户描述"实际行为 ≠ 预期行为"的现象。
|
|
||||||
- 代码审查报告中标记了 BUG-NNN / SEC-NNN 类问题需要修复。
|
|
||||||
- 用户显式要求"按 bug-fix-expert 流程处理"。
|
|
||||||
|
|
||||||
## 目标
|
|
||||||
|
|
||||||
以"先确认、再修复"的方式处理缺陷:
|
|
||||||
|
|
||||||
1. **先证明 bug 真实存在**(必须从多个角度确认)。
|
|
||||||
2. **若确认真实存在**:实施最佳修复方案,补齐测试,避免引入回归;修复后由独立角色审查改动,直至无明显问题。
|
|
||||||
3. **若确认不存在/无法证实**:只说明结论与证据,不修改任何代码。
|
|
||||||
|
|
||||||
## 适用范围
|
|
||||||
|
|
||||||
- **适用**:用户报告的异常、CI 失败、线上问题回溯、逻辑不符合预期、性能/并发/边界 bug 等。
|
|
||||||
- **不适用**:需求变更(应先确认产品预期)或纯重构(除非重构是修复的最小代价手段)。
|
|
||||||
|
|
||||||
## 强制原则(不可跳过)
|
|
||||||
|
|
||||||
1. **没有可重复的证据,不改代码**:至少满足"稳定复现"或"静态分析可严格证明存在"。
|
|
||||||
2. **多角度确认**:至少使用 3 种不同方式交叉验证(P0 可降至 2 种,但必须注明理由)。
|
|
||||||
3. **先写失败用例**:优先用最小化单元测试/集成测试把 bug "钉住"。
|
|
||||||
4. **修复必须带测试**:新增/完善测试覆盖 bug 场景与关键边界,确保回归保护。**改动代码的单元测试覆盖率必须 ≥ 85%**(以变更行为统计口径,非全仓覆盖率)。
|
|
||||||
5. **不引入新问题**:尽量小改动、低耦合;遵守项目既有分层与编码规范。
|
|
||||||
6. **修复与审查角色隔离**:修复者不得自审,必须由独立角色执行代码审查。
|
|
||||||
7. **安全前后双检**:修复前预扫描 + 修复后 diff 复核,两次都通过才算合格。
|
|
||||||
8. **Git 写操作必须确认**:任何会改变 Git 状态的操作必须先获得用户确认;只读诊断无需确认。**例外**:bugfix 流程中的临时 worktree 创建/删除和 `bugfix/*` 命名空间下的临时分支操作,在用户确认启动 bug 修复流程时即视为一次性授权,后续无需逐个确认。
|
|
||||||
9. **沟通与文档默认中文**:除非用户明确要求其他语言。
|
|
||||||
10. **Bug-ID 合法性校验**:Bug-ID 只允许包含字母、数字、连字符(`-`)和下划线(`_`),正则校验 `^[A-Za-z0-9_-]{1,64}$`。不符合规则的输入必须拒绝并提示用户修改。主控在构造路径和分支名前必须执行此校验。
|
|
||||||
|
|
||||||
## 严重度分级与响应策略
|
|
||||||
|
|
||||||
| 等级 | 定义 | 响应策略 |
|
|
||||||
|------|------|----------|
|
|
||||||
| **P0 — 线上崩溃/数据损坏** | 服务不可用、数据丢失/损坏、安全漏洞已被利用 | **快车道**:验证可降至 2 种交叉方式;跳过方案对比,直接最小修复;采用乐观并行(见"P0 乐观并行策略") |
|
|
||||||
| **P1 — 核心功能阻断** | 主流程不可用但服务在线、影响大量用户 | **加速道**:方案设计精简为 1-2 句权衡;验证与分析并行 |
|
|
||||||
| **P2 — 功能异常/边界问题** | 非主流程异常、边界条件触发、体验降级 | **标准道**:完整执行全部步骤 |
|
|
||||||
| **P3 — 优化/改善** | 性能可改善、代码异味、非紧急潜在风险 | **标准道**:完整执行,可排入后续迭代 |
|
|
||||||
|
|
||||||
> 默认按 P2 处理;用户明确指出严重度或从上下文可判断时自动调级。
|
|
||||||
|
|
||||||
**P0 乐观并行策略**:P0 级别可同时启动验证和修复子智能体(修复基于初步分析的"最可能根因"先行工作)。若验证子智能体返回 `FAILED`(无法证实 bug),主控必须立即通过 `TaskStop` 终止修复子智能体、清理其 worktree,并跳转到"无法证实"结论。P0 乐观并行的回滚代价是浪费修复 agent 的工作量,但换取更快的修复速度。
|
|
||||||
|
|
||||||
## 标准工作流
|
|
||||||
|
|
||||||
### 0) 信息收集
|
|
||||||
|
|
||||||
收集并复述以下信息(缺失则主动追问):
|
|
||||||
|
|
||||||
- **现象**:实际行为、报错信息/堆栈、日志片段。
|
|
||||||
- **预期**:应该发生什么?
|
|
||||||
- **环境**:版本号/分支、运行方式(本地/容器/CI)、关键配置。
|
|
||||||
- **复现步骤**:最小复现步骤与输入数据。
|
|
||||||
- **严重度**:根据影响面初步定级(P0-P3),决定后续流程节奏。
|
|
||||||
|
|
||||||
> 目标:确保"讨论的是同一个问题",避免修错。
|
|
||||||
|
|
||||||
### 1) 真实性确认(多角度交叉验证)
|
|
||||||
|
|
||||||
**核心验证(必须完成至少 3 种,P0 可降至 2 种并注明理由):**
|
|
||||||
|
|
||||||
**A. 运行复现**:按复现步骤在本地/容器复现;必要时降低变量(固定数据、关闭并发、固定随机种子)。
|
|
||||||
|
|
||||||
**B. 测试复现**:新增一个"修复前稳定失败"的最小测试(优先单测,其次集成测试)。
|
|
||||||
- 用例命名清晰,直接表达 bug。
|
|
||||||
- 失败原因明确,不依赖偶然时序。
|
|
||||||
|
|
||||||
**C. 静态交叉验证**:通过代码路径与边界条件推导 bug(空指针、越界、错误分支、并发竞态、上下文取消、事务边界、权限校验等),并与运行/测试现象一致。
|
|
||||||
|
|
||||||
**必做分析(不计入验证种类数,但每次必须执行):**
|
|
||||||
|
|
||||||
**D. 影响面评估**:分析 bug 所在代码的调用链,列出可能受影响的上下游模块。
|
|
||||||
|
|
||||||
**E. 可选补充验证(强烈建议做至少 1 项):**
|
|
||||||
|
|
||||||
- 变更输入/边界:最小值/最大值/空值/非法值/并发压力/时序变化。
|
|
||||||
- 对比历史/回归定位:优先只读方式(查看变更历史与责任行)。
|
|
||||||
- 临时诊断(不落库):局部日志、断点、计数器、trace。
|
|
||||||
|
|
||||||
#### 判定标准
|
|
||||||
|
|
||||||
| 判定 | 条件 |
|
|
||||||
|------|------|
|
|
||||||
| **真实存在** | 可稳定复现(运行或测试)且现象可解释 |
|
|
||||||
| **可严格证明存在** | 难以复现,但静态分析可严格证明必现(明显的 nil deref/越界/必走错误分支) |
|
|
||||||
| **无法证实** | 无法稳定复现,且静态分析无法给出严格证明 → **停止,不修改任何代码** |
|
|
||||||
|
|
||||||
#### 结论汇总规则
|
|
||||||
|
|
||||||
- 若验证与分析结论一致 → 进入下一步。
|
|
||||||
- 若矛盾 → 启动额外验证(上述 E 项),**最多追加 2 轮**。仍矛盾则上报用户决策。
|
|
||||||
|
|
||||||
### 2) 方案设计
|
|
||||||
|
|
||||||
至少列出 2 个可行方案(P0 可跳过对比,直选最小修复并注明理由),明确权衡:
|
|
||||||
|
|
||||||
- 影响面(改动范围、是否影响 API/DB/数据兼容性)
|
|
||||||
- 风险(并发/安全/性能/回滚复杂度)
|
|
||||||
- 可测试性(是否容易写稳定测试)
|
|
||||||
|
|
||||||
选择"最小改动且可证明正确"的方案。
|
|
||||||
|
|
||||||
### 3) 实施修复
|
|
||||||
|
|
||||||
1. 先落地最小修复(尽量不重构、不改风格)。
|
|
||||||
2. 完善测试:
|
|
||||||
- 覆盖 bug 场景(必须)
|
|
||||||
- 覆盖关键边界与回归场景(必须)
|
|
||||||
- 必要时增加集成/端到端验证(按影响面决定)
|
|
||||||
- **改动代码覆盖率门禁**:对本次修改/新增的代码,单元测试行覆盖率必须 ≥ 85%。
|
|
||||||
使用项目对应的覆盖率工具(Go: `go test -coverprofile` + 分析变更行覆盖;
|
|
||||||
JS/TS: `--collectCoverageFrom` 指定变更文件;Python: `coverage run` + `coverage report --include`)
|
|
||||||
仅统计本次变更文件中变更行的覆盖情况,不要求全仓覆盖率达标。
|
|
||||||
若因代码结构原因(如纯配置、接口声明等不可测代码)无法达到 85%,
|
|
||||||
必须在 Beacon 中说明原因和实际覆盖率。
|
|
||||||
3. 运行质量门禁(与项目 CI 对齐):
|
|
||||||
- 最小集合:受影响模块的单元测试 + 静态检查(lint/格式化/类型检查)。
|
|
||||||
- 必要时:集成测试、端到端测试、兼容性验证、性能回归检查。
|
|
||||||
- 不确定时:跑全量测试。
|
|
||||||
- **覆盖率检查**:修复完成后运行覆盖率工具,确认变更代码覆盖率 ≥ 85%,将结果写入 Beacon。
|
|
||||||
4. 若引入新失败:优先修复新失败;不要用"忽略测试/删除用例"掩盖问题。
|
|
||||||
|
|
||||||
**安全预扫描(与修复并行)**:扫描修复方案**将要触及的代码区域的修复前基线版本**,检查已有安全隐患,评估修复方案是否可能引入新风险。注意:预扫描的对象是修复前的基线代码,而非修复进行中的中间状态。
|
|
||||||
|
|
||||||
### 4) 二次审查(角色隔离,独立审查)
|
|
||||||
|
|
||||||
由独立角色(而非修复者自身)执行代码审查,至少覆盖:
|
|
||||||
|
|
||||||
- **正确性**:空指针/越界/错误处理/返回值语义/事务与上下文。
|
|
||||||
- **并发**:竞态、锁粒度、goroutine 泄漏、通道关闭时序。
|
|
||||||
- **兼容性**:API/配置/数据迁移影响,旧数据是否可读。
|
|
||||||
- **可维护性**:命名、结构、可读性、分层依赖是否违规。
|
|
||||||
- **测试质量**:是否会偶发失败?是否覆盖根因?是否能防回归?变更代码覆盖率是否 ≥ 85%?
|
|
||||||
|
|
||||||
**安全最终复核**:对修复 diff 审查鉴权/越权、注入(SQL/命令/模板)、敏感信息泄露;若修复涉及依赖变更,额外检查依赖安全。主控在启动安全复核子智能体时,必须将第 3 步安全预扫描的 Beacon 结论作为上下文传入 prompt,复核者对比两次扫描结果,确认未引入新安全问题。
|
|
||||||
|
|
||||||
**迭代规则**:发现问题 → 修复者修正 → 再次审查。**最多迭代 3 轮**,超过则上报用户重新评估方案或引入人工审查。
|
|
||||||
|
|
||||||
### 5) 交付输出
|
|
||||||
|
|
||||||
> 进入交付前必须通过**三重门禁**:测试通过 + 审查通过 + 安全通过,缺一不可(无论严重度等级)。
|
|
||||||
|
|
||||||
#### bug 确认存在并已修复
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
## Bug 修复报告
|
|
||||||
|
|
||||||
**Bug ID**:[BUG-NNN]
|
|
||||||
**严重度**:[P0🔴 / P1🟠 / P2🟡 / P3🟢]
|
|
||||||
**根因**:[触发条件 + 代码/逻辑原因,引用 file:line]
|
|
||||||
|
|
||||||
**影响面**:
|
|
||||||
- 受影响模块:[模块A → 模块B → ...]
|
|
||||||
- 受影响 API/用户:[说明]
|
|
||||||
|
|
||||||
**修复方案**:
|
|
||||||
- 改动说明:[做了什么、为何是最小且正确的修复]
|
|
||||||
- 改动文件:[file1:line, file2:line, ...]
|
|
||||||
|
|
||||||
**测试**:
|
|
||||||
- 新增/更新的测试:[测试名称 + 覆盖场景]
|
|
||||||
- 运行结果:[命令 + PASS/FAIL]
|
|
||||||
|
|
||||||
**安全扫描**:
|
|
||||||
- 预扫描:[通过/发现 N 项,已处理]
|
|
||||||
- 最终复核:[通过/发现 N 项,已处理]
|
|
||||||
|
|
||||||
**残余风险**:[仍可能存在的边界/后续建议,无则写"无"]
|
|
||||||
|
|
||||||
**回滚预案**:[P0/P1 必填:如何快速回滚]
|
|
||||||
```
|
|
||||||
|
|
||||||
#### bug 无法证实或不存在
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
## Bug 调查报告
|
|
||||||
|
|
||||||
**结论**:无法证实 / 确认不存在
|
|
||||||
**判定依据**:
|
|
||||||
- 复现尝试:[方法 + 结果]
|
|
||||||
- 测试验证:[方法 + 结果]
|
|
||||||
- 静态分析:[分析要点]
|
|
||||||
|
|
||||||
**下一步**:[需要用户补充哪些信息才能继续]
|
|
||||||
```
|
|
||||||
|
|
||||||
## 智能体协作执行
|
|
||||||
|
|
||||||
### 角色与 Task 工具映射
|
|
||||||
|
|
||||||
本技能通过 Claude Code 的 Task 工具实现多角色协作。主会话即主控,子智能体通过 Task 工具启动。**所有涉及文件写操作的子智能体必须在独立 git worktree 中工作。**
|
|
||||||
|
|
||||||
| 角色 | Task subagent_type | 并行阶段 | 需要 Worktree | 职责 |
|
|
||||||
|------|-------------------|----------|:------------:|------|
|
|
||||||
| **主控** | 主会话(不用 Task) | 全程 | 否 | 协调流程、管理 worktree 生命周期、与用户沟通、汇总结论 |
|
|
||||||
| **验证** | `general-purpose` | 第 1 步 | **是** | 在隔离 worktree 中运行复现、编写失败测试、执行测试、收集运行时证据 |
|
|
||||||
| **分析** | `Explore` | 第 1 步(与验证并行) | 否(只读) | 静态代码分析、调用链追踪、影响面评估 |
|
|
||||||
| **修复** | `general-purpose` | 第 3 步 | **是** | 在隔离 worktree 中实施修复、补齐测试、运行质量门禁 |
|
|
||||||
| **安全** | `general-purpose` | 第 3-4 步 | 否(只读扫描) | 安全预扫描(扫基线代码)+ diff 复核 |
|
|
||||||
| **审查** | `general-purpose` | 第 4 步 | **是** | 在隔离 worktree 中独立审查 diff、运行测试验证(与修复者隔离) |
|
|
||||||
|
|
||||||
### Git Worktree 强制隔离策略
|
|
||||||
|
|
||||||
#### 核心规则
|
|
||||||
|
|
||||||
1. **写操作子智能体必须使用 git worktree**:验证(写测试)、修复(改代码)、审查(验证运行)必须在独立 worktree 中操作。
|
|
||||||
2. **只读子智能体无需 worktree**:分析(Explore)和安全扫描可直接读取主工作区或指定 worktree 的路径。
|
|
||||||
3. **主控独占 worktree 生命周期**:子智能体不得自行创建、删除或合并 worktree。
|
|
||||||
|
|
||||||
#### Bug-ID 校验(主控在第 0 步强制执行)
|
|
||||||
|
|
||||||
主控在使用 Bug-ID 构造路径前,必须校验其仅包含字母、数字、连字符和下划线(正则 `^[A-Za-z0-9_-]{1,64}$`)。不符合规则时拒绝并提示用户修改。此校验防止路径穿越(`../`)、命令注入(`;`、空格)和分支名冲突。
|
|
||||||
|
|
||||||
#### 命名规范
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Worktree 路径(使用 $TMPDIR 确保跨平台一致性,macOS 上为用户私有目录)
|
|
||||||
# 注意:macOS 的 $TMPDIR 通常以 / 结尾(如 /var/folders/xx/xxxx/T/),
|
|
||||||
# 必须先去除尾部斜杠,避免路径中出现双斜杠(//)。
|
|
||||||
# 由于 Bash 不支持嵌套参数展开,需要分两步处理:
|
|
||||||
_tmpbase="${TMPDIR:-/tmp}" && _tmpbase="${_tmpbase%/}"
|
|
||||||
BUGFIX_BASE="${_tmpbase}/bugfix-$(id -u)" # 以 UID 隔离不同用户
|
|
||||||
# 完整路径:${BUGFIX_BASE}-{bug-id}-{role}
|
|
||||||
# 示例(macOS):/var/folders/xx/xxxx/T/bugfix-501-BUG-042-verifier
|
|
||||||
# 示例(Linux):/tmp/bugfix-1000-BUG-042-verifier
|
|
||||||
|
|
||||||
# 分支名
|
|
||||||
bugfix/{bug-id}/{role}
|
|
||||||
# 示例
|
|
||||||
bugfix/BUG-042/verifier
|
|
||||||
bugfix/BUG-042/fixer
|
|
||||||
```
|
|
||||||
|
|
||||||
> 使用 `$TMPDIR` 而非硬编码 `/tmp/`,原因:(1) macOS 的 `/tmp` 是 `/private/tmp` 的符号链接,会导致 `git worktree list` 输出路径与构造路径不一致;(2) macOS 的 `$TMPDIR`(形如 `/var/folders/xx/xxxx/T/`)是用户私有目录(权限 700),其他用户无法读取,避免源码泄露。
|
|
||||||
|
|
||||||
#### Worktree 生命周期(主控执行)
|
|
||||||
|
|
||||||
```text
|
|
||||||
阶段 ① 创建 worktree(主控在启动子智能体前执行)
|
|
||||||
# 创建前校验 Bug-ID 合法性(强制原则 #10)
|
|
||||||
# 重要:umask 和 git worktree add 必须在同一个 Bash 调用中执行,
|
|
||||||
# 因为 Bash 工具的 shell 状态(含 umask)不跨调用持久化。
|
|
||||||
umask 077 && git worktree add -b bugfix/{bug-id}/{role} ${BUGFIX_BASE}-{bug-id}-{role} HEAD
|
|
||||||
|
|
||||||
# 创建后禁用 worktree 的远程 push 能力(纵深防御)
|
|
||||||
git -C ${BUGFIX_BASE}-{bug-id}-{role} remote set-url --push origin PUSH_DISABLED
|
|
||||||
|
|
||||||
# 若创建失败,按以下条件分支处理:
|
|
||||||
# 情况 A — 分支已存在但无对应 worktree(上次清理不完整):
|
|
||||||
# git branch -D bugfix/{bug-id}/{role} && 重试 git worktree add
|
|
||||||
# 情况 B — worktree 路径已存在(残留目录):
|
|
||||||
# git worktree remove --force ${BUGFIX_BASE}-{bug-id}-{role}
|
|
||||||
# git branch -D bugfix/{bug-id}/{role} # 分支可能也残留
|
|
||||||
# 重试 git worktree add
|
|
||||||
# 情况 C — 磁盘空间不足:
|
|
||||||
# 尝试回退到 ~/.cache/bugfix-worktrees/bugfix-$(id -u)-{bug-id}-{role} 目录
|
|
||||||
# (需先 umask 077 && mkdir -p ~/.cache/bugfix-worktrees,确保权限 700)
|
|
||||||
# 注意:回退路径保持 "bugfix-{uid}-{bug-id}-{role}" 命名格式,
|
|
||||||
# 确保与 grep -F -- "-{bug-id}-" 清理模式兼容
|
|
||||||
# 所有情况:最多重试 1 次,仍然失败 → 降级为单智能体模式,通知用户
|
|
||||||
|
|
||||||
阶段 ② 传递路径给子智能体
|
|
||||||
主控通过 git worktree list --porcelain 获取实际创建路径(--porcelain 输出
|
|
||||||
机器可解析的格式,避免路径中含空格时被截断;同时规避符号链接导致的路径不一致),
|
|
||||||
将实际路径写入 Task prompt 中。
|
|
||||||
|
|
||||||
阶段 ③ 子智能体在 worktree 中工作
|
|
||||||
- 子智能体完成后通过完成信标(Completion Beacon)主动通知主控
|
|
||||||
- 子智能体允许在 worktree 内执行 git add 和 git commit(因为 worktree 分支
|
|
||||||
是临时隔离分支,不影响主分支;最终合并由主控在用户确认后执行)
|
|
||||||
- 子智能体禁止执行 git push / git merge / git checkout 到其他分支
|
|
||||||
|
|
||||||
阶段 ④ 主控独立验证 + 决定采纳
|
|
||||||
主控收到 Beacon 后,不可仅凭 Beacon 声明做决策,必须独立验证关键声明:
|
|
||||||
- Beacon 声明"测试通过" → 主控在 worktree 中重新运行测试确认
|
|
||||||
- Beacon 声明"变更文件" → 主控通过 git diff 独立确认实际变更范围
|
|
||||||
- Beacon 中的文件引用只允许 worktree 内的相对路径,拒绝绝对路径和含 ../ 的路径
|
|
||||||
采纳:在主工作区执行 git merge / cherry-pick / 手动应用 diff(需用户确认)
|
|
||||||
拒绝:直接清理 worktree
|
|
||||||
|
|
||||||
阶段 ⑤ 清理 worktree(流程结束时,无论成功/失败/中断)
|
|
||||||
git worktree remove --force ${BUGFIX_BASE}-{bug-id}-{role}
|
|
||||||
git branch -D bugfix/{bug-id}/{role} # 大写 -D 强制删除(临时分支可能未合并)
|
|
||||||
# 清理后校验(使用 --porcelain 确保路径解析可靠):
|
|
||||||
# 注意:使用 -F 固定字符串匹配 + "-{bug-id}-" 精确匹配(避免 BUG-1 误匹配 BUG-10)
|
|
||||||
# 使用 if/then 避免 grep 无匹配时 exit code 1 被 Bash 工具误报为错误
|
|
||||||
if git worktree list --porcelain | grep -F -- "-{bug-id}-"; then
|
|
||||||
echo "WARNING: 残留 worktree 未清理"
|
|
||||||
fi
|
|
||||||
git branch --list "bugfix/{bug-id}/*" | xargs -r git branch -D
|
|
||||||
|
|
||||||
# 若清理失败(目录被锁定等):
|
|
||||||
# 1. 等待后重试 git worktree remove --force
|
|
||||||
# 2. 仍失败:手动 rm -rf 目录,然后 git worktree prune
|
|
||||||
# 3. 记录警告并告知用户手动检查
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Worktree 安全约束
|
|
||||||
|
|
||||||
- **原子互斥**:不依赖 `grep` 预检查(存在 TOCTOU 竞态),直接执行 `git worktree add`——若目标已存在,git 本身会原子性地报错拒绝。`grep` 仅用于友好提示,不作为安全保证。
|
|
||||||
- **分支保护**:子智能体禁止直接 push 到远程或合并到主分支,创建 worktree 后主控通过 `remote set-url --push` 禁用 push 能力。
|
|
||||||
- **强制清理**:流程结束(成功/失败/中断/异常)时,主控必须执行 `git worktree list --porcelain | grep -F -- "-{bug-id}-"` 检查并清理所有该 bug 的临时 worktree 和 `bugfix/{bug-id}/*` 分支。
|
|
||||||
- **磁盘保护**:worktree 创建在 `$TMPDIR`(用户私有临时目录)下;若空间不足,回退到 `~/.cache/bugfix-worktrees/`(用户私有,权限 700),不使用系统级共享临时目录(如 `/tmp`)。回退路径同样采用 `bugfix-{uid}-{bug-id}-{role}` 命名格式,确保 `grep -F -- "-{bug-id}-"` 清理模式可匹配。
|
|
||||||
- **敏感数据保护**:子智能体禁止在测试数据中使用真实密钥/token/凭据,必须使用 mock 数据。
|
|
||||||
|
|
||||||
### 并行执行策略(含 Worktree 生命周期)
|
|
||||||
|
|
||||||
```text
|
|
||||||
第 0 步 信息收集 → 主控
|
|
||||||
├─ 校验 Bug-ID 合法性(正则 ^[A-Za-z0-9_-]{1,64}$)
|
|
||||||
├─ 确定 BUGFIX_BASE 路径
|
|
||||||
└─ 检查并清理可能残留的旧 worktree(git worktree list --porcelain | grep -F -- "-{bug-id}-")
|
|
||||||
|
|
||||||
第 1 步 真实性确认 → 并行启动
|
|
||||||
├─ 主控: git worktree add ... verifier(创建验证 worktree)
|
|
||||||
├─ Task(general-purpose:验证, run_in_background=true, max_turns=30)
|
|
||||||
│ ├─ prompt 包含 worktree 实际路径(从 git worktree list --porcelain 获取)
|
|
||||||
│ ├─ 在 worktree 中编写失败测试、运行复现
|
|
||||||
│ └─ 完成后输出 AGENT_COMPLETION_BEACON(主动通知)
|
|
||||||
├─ Task(Explore:分析, run_in_background=true, max_turns=20)
|
|
||||||
│ ├─ 只读分析,无需 worktree
|
|
||||||
│ └─ 完成后输出 AGENT_COMPLETION_BEACON(主动通知)
|
|
||||||
├─ [仅 P0] 主控: 同时创建 fixer worktree + 启动修复子智能体(乐观并行)
|
|
||||||
│ ├─ 修复基于初步分析的"最可能根因"先行工作
|
|
||||||
│ ├─ 若验证返回 FAILED → TaskStop 终止修复子智能体 + 清理其 worktree
|
|
||||||
│ └─ 若验证成功 → 乐观修复已在进行中,直接跳到第 3 步等待其完成(跳过第 2 步方案设计)
|
|
||||||
└─ 主控: 用 TaskOutput(block=false) 轮询,任一完成即处理
|
|
||||||
若验证 agent 返回 FAILED → 可通过 TaskStop 终止分析 agent(或等待其完成后忽略结果)
|
|
||||||
|
|
||||||
第 2 步 方案设计 → 主控
|
|
||||||
├─ 汇总验证+分析的 Beacon 结论
|
|
||||||
├─ 若验证 agent 写了失败测试 → 从 worktree 获取 commit hash
|
|
||||||
│ (git -C {verifier-worktree} log -1 --format="%H")
|
|
||||||
│ 然后在主分支执行 git cherry-pick {hash}(需用户确认)
|
|
||||||
├─ 清理验证 worktree
|
|
||||||
└─ 创建修复 worktree 时以最新 HEAD(含已 cherry-pick 的测试)为基点
|
|
||||||
|
|
||||||
第 3 步 实施修复 → 分步启动
|
|
||||||
├─ 主控: git worktree add ... fixer(基于包含失败测试的最新 HEAD)
|
|
||||||
├─ Task(general-purpose:修复, run_in_background=true, max_turns=40)
|
|
||||||
│ ├─ prompt 包含 worktree 路径 + 修复方案
|
|
||||||
│ ├─ 在 fixer worktree 中实施修复、补齐测试、运行门禁
|
|
||||||
│ └─ 完成后输出 AGENT_COMPLETION_BEACON(主动通知)
|
|
||||||
├─ Task(general-purpose:安全预扫描, run_in_background=true, max_turns=15)
|
|
||||||
│ ├─ 扫描修复方案将触及的代码区域的修复前基线版本(读取主工作区)
|
|
||||||
│ ├─ 注意:扫描对象是基线代码,不是 fixer worktree 中的中间状态
|
|
||||||
│ └─ 完成后输出 AGENT_COMPLETION_BEACON(主动通知)
|
|
||||||
├─ 主控: 修复 Beacon 收到后,委托 Task(Bash, max_turns=3) 在 worktree 中重跑测试(仅返回 pass/fail)
|
|
||||||
└─ 主控: 安全预扫描 + 修复验证都通过后,合并修复到主分支(需用户确认)
|
|
||||||
|
|
||||||
第 4 步 二次审查 → 并行启动
|
|
||||||
├─ 主控: git worktree add ... reviewer(基于合并修复后的最新 HEAD)
|
|
||||||
├─ Task(general-purpose:审查, run_in_background=true, max_turns=25)
|
|
||||||
│ ├─ 在 reviewer worktree 中审查 diff、运行测试
|
|
||||||
│ └─ 完成后输出 AGENT_COMPLETION_BEACON(主动通知)
|
|
||||||
├─ Task(general-purpose:安全复核, run_in_background=true, max_turns=15)
|
|
||||||
│ ├─ prompt 中包含第 3 步安全预扫描的 Beacon 结论作为对比基线
|
|
||||||
│ ├─ 对比修复 diff,执行安全检查
|
|
||||||
│ └─ 完成后输出 AGENT_COMPLETION_BEACON(主动通知)
|
|
||||||
└─ 主控: 收到两个 Beacon 后汇总审查结论
|
|
||||||
|
|
||||||
第 5 步 交付输出 → 主控
|
|
||||||
├─ 汇总所有 Beacon 结论,生成修复报告
|
|
||||||
└─ 强制清理(按阶段 ⑤ 清理流程执行):
|
|
||||||
git worktree list --porcelain | grep -F -- "-{bug-id}-" → remove --force 匹配的所有 worktree
|
|
||||||
(含 $TMPDIR 主路径和 ~/.cache/bugfix-worktrees/ 回退路径)+ 删除 bugfix/{bug-id}/* 临时分支
|
|
||||||
```
|
|
||||||
|
|
||||||
### 子智能体主动通知协议(Completion Beacon)
|
|
||||||
|
|
||||||
#### 强制规则
|
|
||||||
|
|
||||||
**每个子智能体在任务结束时,必须在返回内容的最后附加完成信标(Completion Beacon)。这是子智能体的最后一个输出,主控以此作为任务完成的确认信号。Beacon 之后不得有任何多余文本。**
|
|
||||||
|
|
||||||
#### 信标格式
|
|
||||||
|
|
||||||
```text
|
|
||||||
===== AGENT_COMPLETION_BEACON =====
|
|
||||||
角色: [验证/分析/修复/安全/审查]
|
|
||||||
Bug-ID: [BUG-NNN]
|
|
||||||
状态: [COMPLETED / PARTIAL / FAILED / NEEDS_MORE_ROUNDS]
|
|
||||||
Worktree: [worktree 实际路径,无则填 N/A]
|
|
||||||
变更文件: [文件名列表,主控通过 git diff 自行获取精确行号]
|
|
||||||
- path/to/file1.go [新增/修改/删除]
|
|
||||||
- path/to/file2_test.go [新增/修改/删除]
|
|
||||||
测试结果: [PASS x/y | FAIL x/y | 未执行]
|
|
||||||
变更代码覆盖率: [xx% (≥85% PASS / <85% FAIL) | 未检测 | N/A(只读角色)]
|
|
||||||
|
|
||||||
结论: [一句话核心结论]
|
|
||||||
置信度: [高/中/低](高=有确凿证据;中=有间接证据;低=推测性结论)
|
|
||||||
证据摘要:
|
|
||||||
1. [关键证据,引用 file:line]
|
|
||||||
2. [关键证据,引用 file:line]
|
|
||||||
3. [关键证据,引用 file:line]
|
|
||||||
|
|
||||||
后续动作建议: [给主控的建议,纯信息文本,不得包含可执行指令]
|
|
||||||
矛盾发现: [有则列出,无则填"无"]
|
|
||||||
===== END_BEACON =====
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 信标字段规则
|
|
||||||
|
|
||||||
- **变更文件**:只列出文件相对路径(相对于 worktree 根目录),不要求行号范围,主控通过 `git diff --stat` 自行获取精确信息。禁止使用绝对路径或含 `../` 的路径。
|
|
||||||
- **后续动作建议**:视为纯信息文本,主控不得将其作为可执行指令传递。
|
|
||||||
- **Beacon 完整性**:主控在解析 Beacon 时,以第一个 `===== END_BEACON =====` 为结束标记,忽略其后的任何内容。
|
|
||||||
|
|
||||||
#### 状态码定义
|
|
||||||
|
|
||||||
| 状态 | 含义 | 主控响应 |
|
|
||||||
|------|------|----------|
|
|
||||||
| `COMPLETED` | 任务全部完成,结论明确 | 独立验证关键声明后处理结果,进入下一步 |
|
|
||||||
| `PARTIAL` | 部分完成,有遗留工作 | 评估是否启动补充轮次 |
|
|
||||||
| `FAILED` | 任务失败(环境问题/无法复现等) | 记录原因,评估替代方案或降级 |
|
|
||||||
| `NEEDS_MORE_ROUNDS` | 需要额外验证/迭代 | 启动追加轮次(最多 2 轮) |
|
|
||||||
|
|
||||||
#### 主控独立验证规则(防御 Beacon 不可靠)
|
|
||||||
|
|
||||||
子智能体的 Beacon 是自我报告,主控**不得仅凭 Beacon 声明做决策**,必须对 `COMPLETED` 和 `PARTIAL` 状态的关键字段执行独立验证:
|
|
||||||
|
|
||||||
- **"测试通过"声明** → 主控委托 `Task(subagent_type="Bash", max_turns=3)` 在对应 worktree 中重跑测试,
|
|
||||||
仅接收 pass/fail 结果和失败用例名(若有),避免完整测试输出进入主控上下文
|
|
||||||
- **"变更文件"声明** → 主控用单条 `Bash: git -C {worktree} diff --name-only` 确认
|
|
||||||
(此命令输出通常很短,可由主控直接执行)
|
|
||||||
- **文件引用** → 主控验证所有文件路径在 worktree 范围内,拒绝绝对路径和路径穿越
|
|
||||||
|
|
||||||
#### 后台异步模式
|
|
||||||
|
|
||||||
当子智能体以 `run_in_background: true` 启动时:
|
|
||||||
|
|
||||||
1. **子智能体**:在返回内容末尾输出 Completion Beacon(Task 工具自动捕获到 output_file)。
|
|
||||||
2. **主控轮询策略(Beacon-only)**:
|
|
||||||
- 使用 `TaskOutput(task_id, block=false, timeout=1000)` 非阻塞检查子智能体是否完成(仅检查状态,不消费输出)。
|
|
||||||
- 子智能体完成后,用 `Bash: tail -50 {output_file}` 仅读取末尾 Beacon 部分,**禁止读取全量输出**。
|
|
||||||
- 仅当 Beacon 包含 `FAILED` / `NEEDS_MORE_ROUNDS` / 非空「矛盾发现」时,才用 `Read(offset=..., limit=100)` 定向读取失败上下文。
|
|
||||||
- 若子智能体超时未响应(参考"超时与升级机制"中的子智能体超时定义),主控通过 `Bash: tail -20 {output_file}` 检查最新输出,评估是否终止。
|
|
||||||
3. **早期终止**:若验证 agent 返回 `FAILED`(无法复现),主控可通过 `TaskStop` 终止其他正在运行的子智能体,并跳转到"无法证实"结论。
|
|
||||||
|
|
||||||
#### 通信规则
|
|
||||||
|
|
||||||
- 子智能体间不直接通信,全部经主控中转。
|
|
||||||
- 发现与预期矛盾的证据时,必须在 Beacon 的"矛盾发现"字段标注。
|
|
||||||
- 主控收到包含矛盾发现的 Beacon 后,必须暂停流程:终止所有已启动但未完成的下游子智能体,清理其 worktree,然后启动额外验证。
|
|
||||||
|
|
||||||
### 子智能体 Prompt 模板
|
|
||||||
|
|
||||||
主控启动子智能体时,必须在 Task prompt 中包含以下标准化信息:
|
|
||||||
|
|
||||||
```text
|
|
||||||
你是 Bug 修复流程中的【{角色名}】智能体。
|
|
||||||
|
|
||||||
## 任务上下文
|
|
||||||
- Bug-ID: {bug-id}
|
|
||||||
- 严重度: {P0-P3}
|
|
||||||
- Bug 描述: {现象概述}
|
|
||||||
- 你的工作目录: {worktree 实际路径,从 git worktree list --porcelain 获取}
|
|
||||||
- 允许修改的文件范围: {主控根据影响面分析预先确定的文件/目录列表,如 "backend/internal/service/*.go, backend/internal/handler/chat.go";若为"不限"则可修改任意文件}
|
|
||||||
|
|
||||||
## 项目约定(主控根据实际项目填写,以下为示例)
|
|
||||||
- 后端语言:Go | 前端框架:Vue 3 + TypeScript
|
|
||||||
- 构建命令:make build | 测试命令:make test-backend / make test-frontend
|
|
||||||
- 代码风格:Go 用 gofmt,前端用 ESLint
|
|
||||||
- 沟通与代码注释使用中文
|
|
||||||
> 注:以上为本项目默认值。主控在启动子智能体时应根据实际项目的技术栈、
|
|
||||||
> 构建系统和编码规范调整此部分内容。
|
|
||||||
|
|
||||||
## 工作指令
|
|
||||||
{角色特定的工作指令}
|
|
||||||
|
|
||||||
## 强制约束
|
|
||||||
- 使用 Read/Write/Edit 工具时,所有文件路径必须以 {worktree 路径} 为前缀
|
|
||||||
- 使用 Bash 工具时,命令中使用绝对路径,或在命令开头加 cd {worktree 路径} &&
|
|
||||||
- 禁止读写工作目录之外的文件(除非是只读分析角色读取主工作区)
|
|
||||||
- 禁止执行 git push / git merge / git checkout 到其他分支
|
|
||||||
- 允许在 worktree 内执行 git add 和 git commit(临时分支,不影响主分支)
|
|
||||||
- 修改文件必须在"允许修改的文件范围"内;若需修改范围外的文件,在 Beacon 的"后续动作建议"中说明原因并请求主控确认,不要直接修改
|
|
||||||
- 测试中禁止使用真实密钥/token/凭据,必须使用 mock 数据
|
|
||||||
- 测试中禁止使用固定端口号,使用 0 端口让 OS 分配随机端口
|
|
||||||
- 如果尝试 5 轮后仍无法完成任务,立即输出 FAILED 状态的 Beacon 并停止
|
|
||||||
- **变更代码覆盖率 ≥ 85%**:修复/验证角色完成后,必须运行覆盖率工具检测本次变更代码的行覆盖率;
|
|
||||||
低于 85% 时须补充测试直到达标,或在 Beacon 中说明无法达标的原因(如纯接口声明/配置等不可测代码)
|
|
||||||
- 返回结果必须精简:Beacon 的「证据摘要」每条不超过 80 字符
|
|
||||||
- 禁止在 Beacon 中复制大段源码,只引用 file:line
|
|
||||||
- Beacon 之前的工作过程输出(调试日志、中间推理)不需要结构化,主控不会读取这些内容
|
|
||||||
|
|
||||||
## 完成后必须做
|
|
||||||
任务完成后,你必须在返回内容的最后输出完成信标(Completion Beacon),格式如下:
|
|
||||||
===== AGENT_COMPLETION_BEACON =====
|
|
||||||
角色: {角色名}
|
|
||||||
Bug-ID: {bug-id}
|
|
||||||
状态: [COMPLETED / PARTIAL / FAILED / NEEDS_MORE_ROUNDS]
|
|
||||||
Worktree: {worktree 路径}
|
|
||||||
变更文件:
|
|
||||||
- path/to/file.go [新增/修改/删除]
|
|
||||||
测试结果: [PASS x/y | FAIL x/y | 未执行]
|
|
||||||
变更代码覆盖率: [xx% | 未检测 | N/A]
|
|
||||||
结论: [一句话核心结论]
|
|
||||||
置信度: [高/中/低]
|
|
||||||
证据摘要:
|
|
||||||
1. [关键证据,引用 file:line]
|
|
||||||
后续动作建议: [给主控的建议]
|
|
||||||
矛盾发现: [有则列出,无则填"无"]
|
|
||||||
===== END_BEACON =====
|
|
||||||
|
|
||||||
Beacon 之后不得输出任何内容。
|
|
||||||
```
|
|
||||||
|
|
||||||
### 单智能体降级模式
|
|
||||||
|
|
||||||
当环境不支持并行 Task(或任务简单无需多角色)时,主会话依次扮演所有角色:
|
|
||||||
|
|
||||||
1. **验证 + 分析**:先运行复现,再做静态分析(顺序执行)。降级模式下仍建议使用新分支隔离(`git checkout -b bugfix/{bug-id}/solo`),但不强制使用 worktree。
|
|
||||||
2. **安全预扫描**:修复前切换到"安全视角",扫描修复将触及的代码区域,记录预扫描结论。
|
|
||||||
3. **修复**:直接在主会话的隔离分支中实施。
|
|
||||||
4. **审查**:修复完成后,主会话切换到"审查视角",用 `git diff` 逐项审查清单。此时必须假设自己不是修复者,严格按清单逐条检查。同步执行安全 diff 复核,与预扫描结论对比。
|
|
||||||
5. **安全**:在审查阶段同步检查安全项。
|
|
||||||
|
|
||||||
> 降级模式下审查质量不可降低:审查清单的每一项都必须逐条确认。
|
|
||||||
> P0/P1 级别问题不建议使用降级模式(自审偏见风险),建议至少启动一个独立审查子智能体。
|
|
||||||
|
|
||||||
降级模式下每个阶段结束仍需输出简化版阶段检查点:
|
|
||||||
|
|
||||||
```text
|
|
||||||
----- 阶段检查点 -----
|
|
||||||
阶段: [验证/分析/预扫描/修复/审查]
|
|
||||||
状态: [COMPLETED / PARTIAL / FAILED / NEEDS_MORE_ROUNDS]
|
|
||||||
结论: [一句话核心结论]
|
|
||||||
置信度: [高/中/低]
|
|
||||||
证据摘要: [关键证据 1-3 条]
|
|
||||||
----- 检查点结束 -----
|
|
||||||
```
|
|
||||||
|
|
||||||
## 安全规则
|
|
||||||
|
|
||||||
### Git 操作
|
|
||||||
|
|
||||||
| 类别 | 规则 |
|
|
||||||
|------|------|
|
|
||||||
| **只读诊断** | 默认允许:查看状态/差异、搜索、查看历史与责任行 |
|
|
||||||
| **有副作用** | 必须先获得用户确认:提交、暂存、拉取/推送、切换分支、合并、变基、打标签。执行前输出变更摘要 + 影响范围 + 测试结果。**例外**:`bugfix/*` 临时分支和 worktree 的创建/删除在用户确认启动修复流程时一次性授权 |
|
|
||||||
| **破坏性** | 默认禁止:强制回退/清理/推送。用户二次确认且说明风险后方可执行 |
|
|
||||||
|
|
||||||
### 多智能体并行安全
|
|
||||||
|
|
||||||
当多个 agent 同时修复不同 bug 时:
|
|
||||||
|
|
||||||
1. **工作区隔离(强制)**:每个写操作 agent **必须**使用 git worktree 隔离工作区,禁止多个 agent 在同一工作目录并行写操作。违反此规则的子智能体结果将被主控拒绝。
|
|
||||||
2. **变更范围预声明**:主控在启动修复子智能体时,在 prompt 中预先声明该 agent 允许修改的文件范围。子智能体若需修改范围外的文件,必须在 Beacon 中标注并请求主控确认。
|
|
||||||
3. **禁止破坏性全局变更**:禁止全仓格式化、大规模重命名、批量依赖升级(除非已获用户确认)。
|
|
||||||
4. **临时产物隔离**:复现脚本、测试数据等放入 worktree 内的 `.bugfix-tmp/` 目录。清理 worktree 时使用 `--force` 参数确保连同临时产物一起删除。子智能体禁止在 worktree 外创建临时文件。
|
|
||||||
5. **并发测试安全**:子智能体编写测试时必须使用 `0` 端口让 OS 分配随机端口,使用 `os.MkdirTemp` 创建独立临时目录,禁止使用固定端口或固定临时文件名。
|
|
||||||
6. **Worktree 清理强制**:流程结束(无论成功/失败/中断)必须使用 `git worktree remove --force` 清理所有临时 worktree,然后用 `git branch -D` 删除对应的临时分支。清理后执行校验确认无残留。
|
|
||||||
7. **合并冲突处理**:主控合并 worktree 变更时若遇冲突,必须暂停并上报用户决策,不得自动解决冲突。
|
|
||||||
8. **残留清理**:每次 bug-fix-expert 流程启动时(第 0 步),主控检查是否有超过 24 小时的残留 bugfix worktree 并清理。
|
|
||||||
|
|
||||||
### 安全护栏
|
|
||||||
|
|
||||||
1. **修复前影响面分析**:分析智能体生成调用链,防止改动波及意外模块。
|
|
||||||
2. **安全前后双检**:第 3 步预扫描(扫基线代码)+ 第 4 步 diff 复核(扫修复后 diff),形成闭环。
|
|
||||||
3. **角色隔离**:审查者与修复者必须是不同的智能体/角色。
|
|
||||||
4. **矛盾即暂停**:任意两个角色结论矛盾时,主控暂停流程——终止所有进行中的下游子智能体、清理其 worktree——然后启动额外验证。
|
|
||||||
5. **三重门禁不可跳过**:测试通过 + 审查通过 + 安全通过,缺一不可(无论严重度等级)。
|
|
||||||
6. **Beacon 独立验证**:主控不得仅凭子智能体 Beacon 的自我声明做决策,必须独立验证测试结果和变更范围(详见"主控独立验证规则")。
|
|
||||||
7. **Prompt 约束为软约束**:子智能体的约束(不 push、不越界操作等)通过 Prompt 声明,属于软约束层。主控通过独立验证(检查 `git log`、`git remote -v`、`git diff`)提供纵深防御,确认子智能体未执行禁止操作。
|
|
||||||
|
|
||||||
## 超时与升级机制
|
|
||||||
|
|
||||||
| 阶段 | 超时信号 | 处理方式 |
|
|
||||||
|------|----------|----------|
|
|
||||||
| 子智能体响应 | 子智能体启动后连续 3 次 `TaskOutput(block=false)` 检查(每次间隔处理其他工作后再查)仍无完成输出 | 主控通过 `Read` 检查其 output_file 最新内容;若输出停滞(最后一行内容与上次检查相同),通过 `TaskStop` 终止并降级为主控直接执行该角色任务 |
|
|
||||||
| 真实性确认 | 矛盾验证追加超过 2 轮仍无共识 | 上报用户:当前证据 + 请求补充信息或决定是否继续 |
|
|
||||||
| 方案设计 | 所有方案风险都较高,无明显最优解 | 呈现方案对比,由用户决策 |
|
|
||||||
| 实施修复 | 修复引入的新失败无法在合理迭代内解决 | 建议回退修复或切换方案 |
|
|
||||||
| 二次审查 | 审查-修复迭代超过 3 轮仍有问题 | 建议重新评估方案或引入人工审查 |
|
|
||||||
|
|
||||||
> 注:由于 Claude Code 的 Task 工具不提供基于挂钟时间的超时机制,子智能体超时通过"轮询无进展"来判定,而非固定时间阈值。主控在等待期间应处理其他可并行的工作(如处理另一个已完成的子智能体结果),然后再回来检查。
|
|
||||||
|
|
||||||
## 上下文管理
|
|
||||||
|
|
||||||
长时间 bug 调查可能消耗大量上下文窗口,遵循以下原则:
|
|
||||||
|
|
||||||
- **Beacon-only 消费(最重要)**:主控通过 `tail -50` 仅读取子 agent 输出末尾的 Beacon,
|
|
||||||
禁止通过 `TaskOutput(block=true)` 或 `Read` 全量读取子 agent 输出。详见「上下文预算控制」。
|
|
||||||
- **独立验证委托**:测试重跑等验证操作委托给 Bash 子 agent,主控只接收 pass/fail 结论。
|
|
||||||
- **大文件用子智能体**:超过 500 行的代码分析任务,优先用 Task(Explore) 处理,避免主会话上下文膨胀。
|
|
||||||
- **阶段性摘要卡**:每完成一个步骤,输出不超过 15 行的摘要卡,后续步骤仅引用摘要卡。
|
|
||||||
- **只保留关键证据**:子智能体返回结果时只包含关键的 file:line 引用,不复制大段源码。
|
|
||||||
- **复杂度评估**:主控在第 0 步评估 bug 复杂度——对于 P2/P3 级别的简单 bug(影响单文件、根因明确),默认使用降级模式以节省上下文开销;仅当 bug 复杂(P0/P1 或跨多模块)时启用并行模式。
|
|
||||||
- **max_turns 强制**:所有子 agent 必须设置 max_turns(详见「上下文预算控制」表格)。
|
|
||||||
|
|
||||||
### 上下文预算控制(强制执行)
|
|
||||||
|
|
||||||
#### A. Beacon-only 消费模式
|
|
||||||
|
|
||||||
主控读取子 agent 结果时,**禁止读取全量输出**,必须采用 Beacon-only 模式:
|
|
||||||
|
|
||||||
1. 子 agent 以 `run_in_background=true` 启动,输出写入 output_file
|
|
||||||
2. 子 agent 完成后,主控用 Bash `tail -50 {output_file}` 只读取末尾的 Beacon 部分
|
|
||||||
3. 仅当 Beacon 状态为 `FAILED` / `NEEDS_MORE_ROUNDS` 或包含"矛盾发现"时,
|
|
||||||
才用 `Read(offset=...)` 定向读取相关段落(不超过 100 行)
|
|
||||||
4. **禁止使用 `TaskOutput(block=true)` 获取完整输出** — 这会将全量内容灌入上下文
|
|
||||||
|
|
||||||
#### B. 独立验证委托
|
|
||||||
|
|
||||||
主控的"独立验证"(重跑测试、检查 diff)不再由主控亲自执行,而是委托给轻量级验证子 agent:
|
|
||||||
|
|
||||||
| 验证项 | 委托方式 | 返回格式 |
|
|
||||||
|--------|---------|---------|
|
|
||||||
| 重跑测试 | `Task(subagent_type="Bash", max_turns=3)` | `PASS x/y` 或 `FAIL x/y + 失败用例名` |
|
|
||||||
| 检查变更范围 | `Task(subagent_type="Bash", max_turns=2)` | `git diff --name-only` 的文件列表 |
|
|
||||||
| 路径合规检查 | 主控直接用单条 Bash 命令 | 仅 pass/fail |
|
|
||||||
|
|
||||||
这样避免测试输出(可能数百行)和 diff 内容进入主控上下文。
|
|
||||||
|
|
||||||
#### C. 子 agent max_turns 约束
|
|
||||||
|
|
||||||
所有子 agent 启动时必须设置 `max_turns` 参数,防止单个 agent 输出爆炸:
|
|
||||||
|
|
||||||
| 角色 | max_turns 上限 | 说明 |
|
|
||||||
|------|---------------|------|
|
|
||||||
| 验证 | 30 | 需要写测试+运行,允许较多轮次 |
|
|
||||||
| 分析(Explore) | 20 | 只读探索,通常足够 |
|
|
||||||
| 修复 | 40 | 改代码+测试+门禁,需要较多轮次 |
|
|
||||||
| 安全扫描 | 15 | 只读扫描 |
|
|
||||||
| 审查 | 25 | 审查+可能的验证运行 |
|
|
||||||
| 独立验证(Bash) | 3 | 仅跑命令取结果 |
|
|
||||||
|
|
||||||
#### D. 阶段性上下文压缩
|
|
||||||
|
|
||||||
每完成一个工作流步骤,主控必须将该阶段结论压缩为「阶段摘要卡」(不超过 15 行),
|
|
||||||
后续步骤仅引用摘要卡,不回溯原始 Beacon:
|
|
||||||
|
|
||||||
```text
|
|
||||||
阶段摘要卡格式:
|
|
||||||
|
|
||||||
----- 阶段摘要 #{步骤号} {步骤名} -----
|
|
||||||
结论: {一句话}
|
|
||||||
关键证据: {最多 3 条,每条一行,含 file:line}
|
|
||||||
影响文件: {文件列表}
|
|
||||||
前置条件满足: [是/否]
|
|
||||||
遗留问题: {有则列出,无则"无"}
|
|
||||||
-----
|
|
||||||
```
|
|
||||||
|
|
||||||
#### E. 子 agent Prompt 精简指令
|
|
||||||
|
|
||||||
在子 agent Prompt 模板的「强制约束」部分追加以下要求:
|
|
||||||
|
|
||||||
- 返回结果必须精简:Beacon 的「证据摘要」每条不超过 80 字符
|
|
||||||
- 禁止在 Beacon 中复制大段源码,只引用 file:line
|
|
||||||
- Beacon 之前的工作过程输出(调试日志、中间推理)不需要结构化,
|
|
||||||
因为主控不会读取这些内容
|
|
||||||
@@ -1,251 +0,0 @@
|
|||||||
---
|
|
||||||
name: code-review-expert
|
|
||||||
description: >
|
|
||||||
通用代码审核专家 — 基于 git worktree 隔离的多 Agent 并行代码审核系统,集成 Context7 MCP 三重验证对抗代码幻觉。
|
|
||||||
语言无关,适用于任意技术栈(Go, Python, JS/TS, Rust, Java, C# 等)。
|
|
||||||
Use when: (1) 用户要求代码审核、code review、安全审计、性能审查,
|
|
||||||
(2) 用户说"审核代码"、"review"、"检查代码质量"、"安全检查",
|
|
||||||
(3) 用户要求对 PR、分支、目录或文件做全面质量检查,
|
|
||||||
(4) 用户提到"代码审核专家"或"/code-review-expert"。
|
|
||||||
五大审核维度:安全合规、架构设计、性能资源、可靠性数据完整性、代码质量可观测性。
|
|
||||||
自动创建 5 个 git worktree 隔离环境,派发 5 个专项子 Agent 并行审核,
|
|
||||||
通过 Context7 MCP 拉取最新官方文档验证 API 用法,消除 LLM 幻觉,
|
|
||||||
汇总后生成结构化 Markdown 审核报告,最终自动清理所有 worktree。
|
|
||||||
---
|
|
||||||
|
|
||||||
# Universal Code Review Expert
|
|
||||||
|
|
||||||
基于 git worktree 隔离 + 5 子 Agent 并行 + Context7 反幻觉验证的通用代码审核系统。
|
|
||||||
|
|
||||||
## Guardrails
|
|
||||||
|
|
||||||
- **只读审核**,绝不修改源代码,写入仅限报告文件
|
|
||||||
- **语言无关**,通过代码模式识别而非编译发现问题
|
|
||||||
- 每个子 Agent 在独立 **git worktree** 中工作
|
|
||||||
- 审核结束后**无条件清理**所有 worktree(即使中途出错)
|
|
||||||
- 问题必须给出**具体 `file:line`**,不接受泛泛而谈
|
|
||||||
- 涉及第三方库 API 的发现必须通过 **Context7 MCP** 验证,严禁凭记忆断言 API 状态
|
|
||||||
- 文件 > 500 个时自动启用**采样策略**
|
|
||||||
- **上下文保护**:严格遵循下方 Context Budget Control 规则,防止 200K 上下文耗尽
|
|
||||||
|
|
||||||
## Context Budget Control (上下文预算管理)
|
|
||||||
|
|
||||||
> **核心问题**:5 个子 Agent 并行审核时,每个 Agent 读取大量文件会快速耗尽 200K 上下文,导致审核卡住或失败。
|
|
||||||
|
|
||||||
### 预算分配策略
|
|
||||||
|
|
||||||
主 Agent 在 Phase 0 必须计算上下文预算,并分配给子 Agent:
|
|
||||||
|
|
||||||
```
|
|
||||||
总可用上下文 ≈ 180K tokens(预留 20K 给主 Agent 汇总)
|
|
||||||
每个子 Agent 预算 = 180K / 5 = 36K tokens
|
|
||||||
每个子 Agent 可读取的文件数 ≈ 36K / 平均文件大小
|
|
||||||
```
|
|
||||||
|
|
||||||
### 七项强制规则
|
|
||||||
|
|
||||||
1. **文件分片不重叠**:每个文件只分配给**一个主要维度**(按文件类型/路径自动判断),不要多维度重复审核同一文件。高风险文件(auth、crypto、payment)例外,可分配给最多 2 个维度。
|
|
||||||
|
|
||||||
2. **单文件读取上限**:子 Agent 读取单个文件时,使用 `Read` 工具的 `limit` 参数,每次最多读取 **300 行**。超过 300 行的文件分段读取,仅审核关键段落。
|
|
||||||
|
|
||||||
3. **子 Agent prompt 精简**:传递给子 Agent 的 prompt 只包含:
|
|
||||||
- 该维度的**精简检查清单**(不要传全部 170 项,只传该维度的 ~30 项)
|
|
||||||
- 文件列表(路径即可,不包含内容)
|
|
||||||
- C7 缓存中**该维度相关的**部分(不传全量缓存)
|
|
||||||
- 输出格式模板(一次,不重复)
|
|
||||||
|
|
||||||
4. **结果输出精简**:子 Agent 找到问题后只输出 JSON Lines,**不要**输出解释性文字、思考过程或总结。完成后只输出 status 行。
|
|
||||||
|
|
||||||
5. **子 Agent max_turns 限制**:每个子 Agent 使用 `max_turns` 参数限制最大轮次:
|
|
||||||
- 文件数 ≤ 10: `max_turns=15`
|
|
||||||
- 文件数 11-30: `max_turns=25`
|
|
||||||
- 文件数 31-60: `max_turns=40`
|
|
||||||
- 文件数 > 60: `max_turns=50`
|
|
||||||
|
|
||||||
6. **大仓库自动降级**:
|
|
||||||
- 文件数 > 200:减为 **3 个子 Agent**(安全+可靠性、架构+性能、质量+可观测性)
|
|
||||||
- 文件数 > 500:减为 **2 个子 Agent**(安全重点、质量重点)+ 采样 30%
|
|
||||||
- 文件数 > 1000:单 Agent 串行 + 采样 15% + 仅审核变更文件
|
|
||||||
|
|
||||||
7. **子 Agent 使用 `run_in_background`**:所有子 Agent Task 调用设置 `run_in_background=true`,主 Agent 通过 Read 工具轮询 output_file 获取结果,避免子 Agent 的完整输出回填到主 Agent 上下文。
|
|
||||||
|
|
||||||
### 文件分配算法
|
|
||||||
|
|
||||||
按文件路径/后缀自动分配到主要维度:
|
|
||||||
|
|
||||||
| 模式 | 主维度 | 辅助维度(仅高风险文件) |
|
|
||||||
|------|--------|----------------------|
|
|
||||||
| `*auth*`, `*login*`, `*jwt*`, `*oauth*`, `*crypto*`, `*secret*` | Security | Reliability |
|
|
||||||
| `*route*`, `*controller*`, `*handler*`, `*middleware*`, `*service*` | Architecture | - |
|
|
||||||
| `*cache*`, `*pool*`, `*buffer*`, `*queue*`, `*worker*` | Performance | - |
|
|
||||||
| `*db*`, `*model*`, `*migration*`, `*transaction*` | Reliability | Performance |
|
|
||||||
| `*test*`, `*spec*`, `*log*`, `*metric*`, `*config*`, `*deploy*` | Quality | - |
|
|
||||||
| 其余文件 | 按目录轮询分配到 5 个维度 | - |
|
|
||||||
|
|
||||||
### 主 Agent 汇总时的上下文控制
|
|
||||||
|
|
||||||
Phase 3 汇总时,主 Agent **不要**重新读取子 Agent 审核过的文件。仅基于子 Agent 输出的 JSON Lines 进行:
|
|
||||||
- 去重合并
|
|
||||||
- 严重等级排序
|
|
||||||
- Context7 交叉验证(仅对 critical/high 且未验证的少数发现)
|
|
||||||
- 填充报告模板
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Workflow
|
|
||||||
|
|
||||||
### Phase 0 — Scope Determination
|
|
||||||
|
|
||||||
1. **确定审核范围**(按优先级):
|
|
||||||
- 用户指定的文件/目录
|
|
||||||
- 未提交变更:`git diff --name-only` + `git diff --cached --name-only`
|
|
||||||
- 未推送提交:`git log origin/{main}..HEAD --name-only --pretty=format:""`
|
|
||||||
- 全仓库(启用采样:变更文件 → 高风险目录 → 入口文件 → 其余 30% 采样)
|
|
||||||
|
|
||||||
2. **收集项目元信息**:语言构成、目录结构、文件数量
|
|
||||||
|
|
||||||
3. **生成会话 ID**:
|
|
||||||
```bash
|
|
||||||
SESSION_ID="cr-$(date +%Y%m%d-%H%M%S)-$(openssl rand -hex 4)"
|
|
||||||
WORKTREE_BASE="/tmp/${SESSION_ID}"
|
|
||||||
```
|
|
||||||
|
|
||||||
4. 将文件分配给 5 个审核维度(每个文件可被多维度审核)
|
|
||||||
|
|
||||||
### Phase 0.5 — Context7 Documentation Warm-up (反幻觉第一重)
|
|
||||||
|
|
||||||
> 详细流程见 [references/context7-integration.md](references/context7-integration.md)
|
|
||||||
|
|
||||||
1. 扫描依赖清单(go.mod, package.json, requirements.txt, Cargo.toml, pom.xml 等)
|
|
||||||
2. 提取核心直接依赖,按优先级筛选最多 **10 个关键库**:
|
|
||||||
- P0 框架核心(web 框架、ORM)→ P1 安全相关 → P2 高频 import → P3 其余
|
|
||||||
3. 对每个库调用 `resolve-library-id` → `get-library-docs`(每库 ≤ 5000 tokens)
|
|
||||||
4. 构建 **C7 知识缓存 JSON**,传递给所有子 Agent
|
|
||||||
5. **降级**:Context7 不可用时跳过,报告标注 "未经官方文档验证"
|
|
||||||
|
|
||||||
### Phase 1 — Worktree Creation
|
|
||||||
|
|
||||||
```bash
|
|
||||||
CURRENT_COMMIT=$(git rev-parse HEAD)
|
|
||||||
for dim in security architecture performance reliability quality; do
|
|
||||||
git worktree add "${WORKTREE_BASE}/${dim}" "${CURRENT_COMMIT}" --detach
|
|
||||||
done
|
|
||||||
```
|
|
||||||
|
|
||||||
### Phase 2 — Parallel Sub-Agent Dispatch (反幻觉第二重)
|
|
||||||
|
|
||||||
**在一条消息中发出所有 Task 调用**(`subagent_type: general-purpose`),**必须设置**:
|
|
||||||
- `run_in_background: true` — 子 Agent 后台运行,结果写入 output_file,避免回填主 Agent 上下文
|
|
||||||
- `max_turns` — 按文件数量设置(见 Context Budget Control)
|
|
||||||
- `model: "sonnet"` — 子 Agent 使用 sonnet 模型降低延迟和 token 消耗
|
|
||||||
|
|
||||||
Agent 数量根据文件规模自动调整(见 Context Budget Control 大仓库降级规则)。
|
|
||||||
|
|
||||||
每个 Agent 收到:
|
|
||||||
|
|
||||||
| 参数 | 内容 |
|
|
||||||
|------|------|
|
|
||||||
| worktree 路径 | `${WORKTREE_BASE}/{dimension}` |
|
|
||||||
| 文件列表 | 该维度**独占分配**的文件(不重叠) |
|
|
||||||
| 检查清单 | 该维度对应的精简清单(~30 项,非全量 170 项) |
|
|
||||||
| C7 缓存 | 仅该维度相关的库文档摘要 |
|
|
||||||
| 输出格式 | JSON Lines(见下方) |
|
|
||||||
| 文件读取限制 | 单文件最多 300 行,使用 Read 的 limit 参数 |
|
|
||||||
|
|
||||||
每个发现输出一行 JSON:
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"dimension": "security",
|
|
||||||
"severity": "critical|high|medium|low|info",
|
|
||||||
"file": "path/to/file.go",
|
|
||||||
"line": 42,
|
|
||||||
"rule": "SEC-001",
|
|
||||||
"title": "SQL Injection",
|
|
||||||
"description": "详细描述",
|
|
||||||
"suggestion": "修复建议(含代码片段)",
|
|
||||||
"confidence": "high|medium|low",
|
|
||||||
"c7_verified": true,
|
|
||||||
"verification_method": "c7_cache|c7_realtime|model_knowledge",
|
|
||||||
"references": ["CWE-89"]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**关键规则**:
|
|
||||||
- 涉及第三方库 API 的发现,未经 Context7 验证时 `confidence` 不得为 `high`
|
|
||||||
- `verification_method == "model_knowledge"` 的发现自动降一级置信度
|
|
||||||
- 每个子 Agent 最多消耗分配的 Context7 查询预算
|
|
||||||
- 完成后输出:`{"status":"complete","dimension":"...","files_reviewed":N,"issues_found":N,"c7_queries_used":N}`
|
|
||||||
|
|
||||||
### Phase 3 — Aggregation + Cross-Validation (反幻觉第三重)
|
|
||||||
|
|
||||||
1. 等待所有子 Agent 完成
|
|
||||||
2. 合并 findings,按 severity 排序
|
|
||||||
3. **Context7 交叉验证**:
|
|
||||||
- 筛选 `c7_verified==false` 且 severity 为 critical/high 的 API 相关发现
|
|
||||||
- 主 Agent 独立调用 Context7 验证
|
|
||||||
- 验证通过 → 保留 | 验证失败 → 降级或删除(标记 `c7_invalidated`)
|
|
||||||
4. 去重(同一 file:line 合并)
|
|
||||||
5. 生成报告到 `code-review-report.md`(模板见 [references/report-template.md](references/report-template.md))
|
|
||||||
|
|
||||||
### Phase 4 — Cleanup (必须执行)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
for dim in security architecture performance reliability quality; do
|
|
||||||
git worktree remove "${WORKTREE_BASE}/${dim}" --force 2>/dev/null
|
|
||||||
done
|
|
||||||
git worktree prune
|
|
||||||
rm -rf "${WORKTREE_BASE}"
|
|
||||||
```
|
|
||||||
|
|
||||||
> 即使前面步骤失败也**必须执行**此清理。
|
|
||||||
|
|
||||||
## Severity Classification
|
|
||||||
|
|
||||||
| 等级 | 标签 | 定义 |
|
|
||||||
|------|------|------|
|
|
||||||
| P0 | `critical` | 已存在的安全漏洞或必然导致数据丢失/崩溃 |
|
|
||||||
| P1 | `high` | 高概率触发的严重问题或重大性能缺陷 |
|
|
||||||
| P2 | `medium` | 可能触发的问题或明显设计缺陷 |
|
|
||||||
| P3 | `low` | 代码质量问题,不直接影响运行 |
|
|
||||||
| P4 | `info` | 优化建议或最佳实践提醒 |
|
|
||||||
|
|
||||||
置信度:`high` / `medium` / `low`,低置信度须说明原因。
|
|
||||||
|
|
||||||
## Five Review Dimensions
|
|
||||||
|
|
||||||
每个维度对应一个子 Agent,详细检查清单见 [references/checklists.md](references/checklists.md):
|
|
||||||
|
|
||||||
1. **Security & Compliance** — 注入漏洞(10 类)、认证授权、密钥泄露、密码学、依赖安全、隐私保护
|
|
||||||
2. **Architecture & Design** — SOLID 原则、架构模式、API 设计、错误策略、模块边界
|
|
||||||
3. **Performance & Resource** — 算法复杂度、数据库性能、内存管理、并发性能、I/O、缓存、资源泄漏
|
|
||||||
4. **Reliability & Data Integrity** — 错误处理、空值安全、并发安全、事务一致性、超时重试、边界条件、优雅关闭
|
|
||||||
5. **Code Quality & Observability** — 复杂度、重复、命名、死代码、测试质量、日志、可观测性、构建部署
|
|
||||||
|
|
||||||
## Context7 Anti-Hallucination Overview
|
|
||||||
|
|
||||||
> 详细集成文档见 [references/context7-integration.md](references/context7-integration.md)
|
|
||||||
|
|
||||||
三重验证防御 5 类 LLM 幻觉:
|
|
||||||
|
|
||||||
| 幻觉类型 | 说明 | 防御层 |
|
|
||||||
|----------|------|--------|
|
|
||||||
| API 幻觉 | 错误断言函数签名 | 第一重 + 第二重 |
|
|
||||||
| 废弃幻觉 | 错误标记仍在用的 API 为 deprecated | 第二重 + 第三重 |
|
|
||||||
| 不存在幻觉 | 声称新增 API 不存在 | 第一重 + 第二重 |
|
|
||||||
| 参数幻觉 | 错误描述参数类型/默认值 | 第二重实时查 |
|
|
||||||
| 版本混淆 | 混淆不同版本 API 行为 | 第一重版本锚定 |
|
|
||||||
|
|
||||||
验证覆盖度评级:`FULL` (100% API 发现已验证) > `PARTIAL` (50%+) > `LIMITED` (<50%) > `NONE`
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
|
|
||||||
- 某个子 Agent 失败:继续汇总其他结果,报告标注不完整维度
|
|
||||||
- git worktree 创建失败:`git worktree prune` 重试 → 仍失败则回退串行模式
|
|
||||||
- Context7 不可用:跳过验证阶段,报告标注 "未经官方文档验证"
|
|
||||||
- 所有情况下 **Phase 4 清理必须执行**
|
|
||||||
|
|
||||||
## Resources
|
|
||||||
|
|
||||||
- **[references/checklists.md](references/checklists.md)** — 5 个子 Agent 的完整检查清单 (~170 项)
|
|
||||||
- **[references/context7-integration.md](references/context7-integration.md)** — Context7 MCP 集成详细流程、缓存格式、查询规范
|
|
||||||
- **[references/report-template.md](references/report-template.md)** — 审核报告 Markdown 模板
|
|
||||||
@@ -1,252 +0,0 @@
|
|||||||
# Sub-Agent Review Checklists
|
|
||||||
|
|
||||||
5 个子 Agent 的完整检查清单。每个子 Agent 在独立 git worktree 中工作。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Agent 1: Security & Compliance (安全与合规)
|
|
||||||
|
|
||||||
### 1.1 Injection (注入漏洞)
|
|
||||||
- SQL 注入:字符串拼接 SQL、未使用参数化查询
|
|
||||||
- 命令注入:exec/system/os.Command/subprocess 拼接用户输入
|
|
||||||
- XSS:未转义的用户输入写入 HTML/DOM
|
|
||||||
- XXE:XML 解析器未禁用外部实体
|
|
||||||
- SSRF:用户可控 URL 用于服务端请求,缺少白名单
|
|
||||||
- LDAP 注入:LDAP 查询拼接用户输入
|
|
||||||
- SSTI:用户输入直接传入模板引擎
|
|
||||||
- 路径穿越:文件操作中未校验 `../`
|
|
||||||
- Header 注入:HTTP 响应头拼接用户输入 (CRLF)
|
|
||||||
- Log 注入:日志中拼接未净化的用户输入
|
|
||||||
|
|
||||||
### 1.2 Authentication & Authorization
|
|
||||||
- 缺少认证:敏感 API 端点未要求身份验证
|
|
||||||
- 越权访问:缺少资源归属校验(水平越权)
|
|
||||||
- 权限提升:普通用户可执行管理员操作(垂直越权)
|
|
||||||
- 会话管理:Session fixation、不安全 cookie、缺少超时
|
|
||||||
- JWT:弱签名算法 (none/HS256)、未验证签名、token 泄露
|
|
||||||
- OAuth:开放重定向、state 缺失、token 存储不安全
|
|
||||||
- 默认凭证:代码中预设的用户名密码
|
|
||||||
|
|
||||||
### 1.3 Secrets & Sensitive Data
|
|
||||||
- 硬编码密钥:API key、密码、token、连接字符串写在源码
|
|
||||||
- 密钥泄露:.env 提交版本控制、明文密码
|
|
||||||
- 日志泄露:敏感数据出现在日志/错误信息中
|
|
||||||
- API 响应泄露:接口返回超出必要范围的用户数据
|
|
||||||
- 错误信息泄露:堆栈、内部路径、数据库结构暴露
|
|
||||||
|
|
||||||
### 1.4 Cryptography
|
|
||||||
- 弱哈希:MD5/SHA1 用于密码或安全场景
|
|
||||||
- 不安全随机数:math/rand 替代 CSPRNG
|
|
||||||
- ECB 模式:AES-ECB 等不安全加密模式
|
|
||||||
- 硬编码 IV/Salt
|
|
||||||
- 缺少完整性校验:加密但未做 HMAC/AEAD
|
|
||||||
|
|
||||||
### 1.5 Dependency Security
|
|
||||||
- 已知漏洞:依赖清单中的 CVE
|
|
||||||
- 过时依赖:已停止维护的库
|
|
||||||
- 依赖来源:非官方源、typosquatting
|
|
||||||
- 许可证合规:GPL 等传染性许可证混入商业项目
|
|
||||||
|
|
||||||
### 1.6 Privacy & Data Protection
|
|
||||||
- PII 未加密存储或传输
|
|
||||||
- 缺少数据过期/删除机制
|
|
||||||
- 跨境传输未考虑地域合规
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Agent 2: Architecture & Design (架构与设计)
|
|
||||||
|
|
||||||
### 2.1 Design Principles
|
|
||||||
- SRP:类/函数/模块承担过多职责
|
|
||||||
- OCP:修改核心逻辑而非通过扩展点添加
|
|
||||||
- LSP:子类/实现违反父类/接口契约
|
|
||||||
- ISP:接口过大,强迫实现不需要的方法
|
|
||||||
- DIP:高层模块直接依赖低层实现
|
|
||||||
|
|
||||||
### 2.2 Architectural Patterns
|
|
||||||
- 分层违规:跨层直接调用
|
|
||||||
- 循环依赖:包/模块间循环引用
|
|
||||||
- 上帝对象:单类承载过多数据和行为
|
|
||||||
- 过度抽象:不必要的工厂/策略/装饰器
|
|
||||||
- 模式误用:强行套用不适合的设计模式
|
|
||||||
- 配置管理:硬编码环境相关值
|
|
||||||
|
|
||||||
### 2.3 API Design
|
|
||||||
- 一致性:同系统 API 风格不一致
|
|
||||||
- 向后兼容:破坏性变更未版本控制
|
|
||||||
- 幂等性:写操作缺少幂等保证
|
|
||||||
- 批量操作:逐条处理导致 N+1 网络请求
|
|
||||||
- 分页:大列表缺少分页/游标
|
|
||||||
- 错误响应:格式不统一、缺少错误码
|
|
||||||
|
|
||||||
### 2.4 Error Handling Strategy
|
|
||||||
- 错误传播:底层错误未包装丢失上下文
|
|
||||||
- 错误类型:字符串替代结构化错误
|
|
||||||
- 恢复策略:缺少重试/降级/断路器
|
|
||||||
- 边界处理:系统边界缺少防御性检查
|
|
||||||
|
|
||||||
### 2.5 Module Boundaries
|
|
||||||
- 接口定义:模块间通过实现而非接口通信
|
|
||||||
- 数据共享:模块间共享可变数据结构
|
|
||||||
- 事件/消息:同步调用链过长
|
|
||||||
- 领域模型:贫血模型、逻辑散落 Service 层
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Agent 3: Performance & Resource (性能与资源)
|
|
||||||
|
|
||||||
### 3.1 Algorithm & Data Structure
|
|
||||||
- 热路径上 O(n^2) 或更高复杂度
|
|
||||||
- 不当数据结构:线性查找替代哈希
|
|
||||||
- 循环内重复计算
|
|
||||||
- 不必要的排序/遍历
|
|
||||||
|
|
||||||
### 3.2 Database Performance
|
|
||||||
- N+1 查询:循环内逐条查询
|
|
||||||
- 缺少索引:WHERE/JOIN 字段未建索引
|
|
||||||
- 全表扫描
|
|
||||||
- 大事务持锁过久
|
|
||||||
- 连接池未配置或配置不当
|
|
||||||
- SELECT * 替代指定字段
|
|
||||||
|
|
||||||
### 3.3 Memory Management
|
|
||||||
- 内存泄漏:未释放引用、全局缓存无上限
|
|
||||||
- 循环内创建大对象/切片
|
|
||||||
- 未使用缓冲 I/O、一次性读取大文件
|
|
||||||
- 循环内字符串拼接
|
|
||||||
- 高频对象未使用池化
|
|
||||||
|
|
||||||
### 3.4 Concurrency Performance
|
|
||||||
- 全局锁替代细粒度锁
|
|
||||||
- 热点资源锁竞争
|
|
||||||
- 无限制创建 goroutine/线程
|
|
||||||
- 对只读数据加锁
|
|
||||||
- 无缓冲通道导致阻塞
|
|
||||||
|
|
||||||
### 3.5 I/O Performance
|
|
||||||
- 异步上下文中阻塞调用
|
|
||||||
- HTTP 客户端未复用连接
|
|
||||||
- 大响应未压缩
|
|
||||||
- 大数据一次性加载替代流式
|
|
||||||
|
|
||||||
### 3.6 Caching
|
|
||||||
- 频繁重复计算/查询未缓存
|
|
||||||
- 缓存穿透:不存在 key 反复查 DB
|
|
||||||
- 缓存雪崩:大量 key 同时过期
|
|
||||||
- 更新后未失效缓存
|
|
||||||
- 无界缓存导致 OOM
|
|
||||||
|
|
||||||
### 3.7 Resource Leaks
|
|
||||||
- 文件句柄:打开未关闭
|
|
||||||
- HTTP response body 未关闭
|
|
||||||
- 数据库查询结果集未关闭
|
|
||||||
- Timer/Ticker/订阅未取消
|
|
||||||
- Goroutine/线程启动后永不退出
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Agent 4: Reliability & Data Integrity (可靠性与数据完整性)
|
|
||||||
|
|
||||||
### 4.1 Error Handling
|
|
||||||
- 静默吞错:空 catch、忽略返回 error
|
|
||||||
- 泛型 catch:catch(Exception e)
|
|
||||||
- 错误消息缺少上下文 (who/what/why)
|
|
||||||
- 库代码中 panic/os.Exit
|
|
||||||
- 关键路径缺少 recover/降级
|
|
||||||
|
|
||||||
### 4.2 Null Safety
|
|
||||||
- 空指针解引用:未检查 nil/null
|
|
||||||
- Optional/Maybe 未正确解包
|
|
||||||
- 空集合直接取下标
|
|
||||||
- 长链式调用中环节返回 null
|
|
||||||
|
|
||||||
### 4.3 Concurrency Safety
|
|
||||||
- 数据竞争:无保护读写共享变量
|
|
||||||
- 死锁:多锁嵌套、不一致加锁顺序
|
|
||||||
- check-then-act 未加锁
|
|
||||||
- 非线程安全 Map 并发使用
|
|
||||||
- 向已关闭 channel 发送数据
|
|
||||||
|
|
||||||
### 4.4 Transaction & Consistency
|
|
||||||
- 多步数据库操作未包裹事务
|
|
||||||
- 不恰当的事务隔离级别
|
|
||||||
- 跨服务缺少补偿/Saga
|
|
||||||
- 异步处理缺少确认/重试
|
|
||||||
- 重试产生重复数据
|
|
||||||
|
|
||||||
### 4.5 Timeout & Retry
|
|
||||||
- HTTP/DB/RPC 调用未设超时
|
|
||||||
- 无限重试或缺少退避
|
|
||||||
- 调用链超时未传递/收缩
|
|
||||||
- 缺少断路器保护
|
|
||||||
|
|
||||||
### 4.6 Boundary Conditions
|
|
||||||
- 整数溢出:大数、类型截断
|
|
||||||
- 浮点精度:金额用浮点数
|
|
||||||
- 时区未明确
|
|
||||||
- UTF-8 多字节未处理
|
|
||||||
- 空集合边界
|
|
||||||
- 并发 first/last、空队列竞态
|
|
||||||
|
|
||||||
### 4.7 Graceful Shutdown
|
|
||||||
- 缺少 SIGTERM/SIGINT 处理
|
|
||||||
- 关闭时未等待进行中请求
|
|
||||||
- 未释放 DB 连接、文件句柄
|
|
||||||
- 内存中待写数据丢失
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Agent 5: Code Quality & Observability (代码质量与可观测性)
|
|
||||||
|
|
||||||
### 5.1 Complexity
|
|
||||||
- 函数圈复杂度 > 15
|
|
||||||
- 深层嵌套 > 4 层
|
|
||||||
- 函数超过 100 行
|
|
||||||
- 参数超过 5 个
|
|
||||||
- 单文件超过 500 行
|
|
||||||
|
|
||||||
### 5.2 Duplication
|
|
||||||
- 大段相似代码 > 10 行
|
|
||||||
- 相同业务逻辑多处独立实现
|
|
||||||
- 魔法数字/字符串多处出现
|
|
||||||
|
|
||||||
### 5.3 Naming & Readability
|
|
||||||
- 不符合语言惯例的命名
|
|
||||||
- 含义模糊:data/info/temp/result
|
|
||||||
- 同一概念不同命名
|
|
||||||
- 布尔命名不是 is/has/can/should
|
|
||||||
- 不通用缩写降低可读性
|
|
||||||
|
|
||||||
### 5.4 Dead Code & Tech Debt
|
|
||||||
- 未调用的函数、未使用的变量/导入
|
|
||||||
- 被注释的代码块
|
|
||||||
- TODO/FIXME/HACK 遗留
|
|
||||||
- 使用 deprecated API
|
|
||||||
|
|
||||||
### 5.5 Test Quality
|
|
||||||
- 关键业务路径缺少测试
|
|
||||||
- 断言仅检查"不报错"
|
|
||||||
- 缺少边界和异常路径测试
|
|
||||||
- 测试间隐式依赖
|
|
||||||
- 过度 mock
|
|
||||||
- 依赖时间/网络等外部状态
|
|
||||||
|
|
||||||
### 5.6 Logging
|
|
||||||
- 关键决策点缺少日志
|
|
||||||
- ERROR 级别用于非错误场景
|
|
||||||
- 字符串拼接而非结构化日志
|
|
||||||
- 日志含密码/token/PII
|
|
||||||
- 热路径过度日志
|
|
||||||
|
|
||||||
### 5.7 Observability
|
|
||||||
- 缺少业务指标(请求量、延迟、错误率)
|
|
||||||
- 跨服务缺少 trace ID
|
|
||||||
- 缺少 liveness/readiness 探针
|
|
||||||
- 关键故障路径缺少告警
|
|
||||||
|
|
||||||
### 5.8 Build & Deploy
|
|
||||||
- 构建结果依赖环境状态
|
|
||||||
- 缺少 lock 文件
|
|
||||||
- 开发/生产配置差异未文档化
|
|
||||||
- 迁移脚本缺少回滚方案
|
|
||||||
- 大功能上线缺少 feature flag
|
|
||||||
@@ -1,169 +0,0 @@
|
|||||||
# Context7 MCP Anti-Hallucination Integration
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
Context7 MCP 提供两个工具,用于拉取第三方库的最新官方文档,消除 LLM 训练数据时效性导致的代码审核幻觉。
|
|
||||||
|
|
||||||
## Tools
|
|
||||||
|
|
||||||
### resolve-library-id
|
|
||||||
|
|
||||||
```
|
|
||||||
输入: libraryName (如 "gin", "gorm", "react", "express")
|
|
||||||
输出: Context7 兼容的 library ID (如 "/gin-gonic/gin")
|
|
||||||
```
|
|
||||||
|
|
||||||
- 必须在 `get-library-docs` 之前调用
|
|
||||||
- 用户已提供 `/org/project` 格式 ID 时可跳过
|
|
||||||
- 解析失败则记录到 `c7_failures`,跳过该库
|
|
||||||
|
|
||||||
### get-library-docs
|
|
||||||
|
|
||||||
```
|
|
||||||
输入:
|
|
||||||
- context7CompatibleLibraryID: 从 resolve-library-id 获取
|
|
||||||
- topic (可选): 聚焦主题 (如 "middleware", "hooks", "query")
|
|
||||||
- tokens (可选): 最大返回 token 数 (默认 5000)
|
|
||||||
```
|
|
||||||
|
|
||||||
- 每个库每次审核最多调用 **3 次**
|
|
||||||
- 优先用 `topic` 缩小范围
|
|
||||||
- 缓存首次查询结果,后续复用
|
|
||||||
|
|
||||||
## Three-Layer Verification
|
|
||||||
|
|
||||||
### Layer 1: Pre-Review Warm-up (Phase 0.5)
|
|
||||||
|
|
||||||
在审核开始前预热文档缓存:
|
|
||||||
|
|
||||||
1. **扫描依赖清单**:
|
|
||||||
```bash
|
|
||||||
for f in go.mod package.json requirements.txt Pipfile pyproject.toml \
|
|
||||||
Cargo.toml Gemfile pom.xml build.gradle composer.json mix.exs \
|
|
||||||
pubspec.yaml *.csproj; do
|
|
||||||
[ -f "$f" ] && echo "FOUND: $f"
|
|
||||||
done
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **提取直接依赖**(按语言):
|
|
||||||
- Go: `go.mod` require 块(排除 `// indirect`)
|
|
||||||
- Node: `package.json` 的 `dependencies`
|
|
||||||
- Python: `requirements.txt` 或 `pyproject.toml` 的 `[project.dependencies]`
|
|
||||||
- Rust: `Cargo.toml` 的 `[dependencies]`
|
|
||||||
- Java: `pom.xml` 或 `build.gradle` 的 implementation 依赖
|
|
||||||
|
|
||||||
3. **优先级筛选**(最多 10 个库):
|
|
||||||
- P0 框架核心:Web 框架、ORM、核心运行时
|
|
||||||
- P1 安全相关:认证库、加密库、JWT 库
|
|
||||||
- P2 高频使用:import 次数最多的库
|
|
||||||
- P3 其余依赖
|
|
||||||
|
|
||||||
4. **批量查询 Context7**:
|
|
||||||
```
|
|
||||||
对每个库:
|
|
||||||
id = resolve-library-id(libraryName)
|
|
||||||
如果失败 → 记录到 c7_failures, 跳过
|
|
||||||
docs = get-library-docs(id, topic="核心 API 概览", tokens=5000)
|
|
||||||
缓存到 C7 知识缓存
|
|
||||||
queries_remaining[库名] = 2
|
|
||||||
```
|
|
||||||
|
|
||||||
5. **构建缓存 JSON**:
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"session_id": "cr-20260207-143000-a1b2c3d4",
|
|
||||||
"libraries": {
|
|
||||||
"gin": {
|
|
||||||
"context7_id": "/gin-gonic/gin",
|
|
||||||
"docs_summary": "...(API 摘要)...",
|
|
||||||
"key_apis": ["gin.Context", "gin.Engine"],
|
|
||||||
"tokens_used": 5000
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"queries_remaining": { "gin": 2 },
|
|
||||||
"c7_failures": []
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
> 多个 `resolve-library-id` 可并行调用。
|
|
||||||
|
|
||||||
### Layer 2: In-Review Realtime Verification (Phase 2)
|
|
||||||
|
|
||||||
子 Agent 审核代码时的实时验证规则:
|
|
||||||
|
|
||||||
**必须验证的场景**:
|
|
||||||
1. 认为某个 API 调用方式错误 → 查 C7 确认当前版本签名
|
|
||||||
2. 认为某个 API 已废弃 → 查 C7 确认 deprecated 状态
|
|
||||||
3. 认为代码缺少某库提供的安全/性能特性 → 查 C7 确认该特性存在
|
|
||||||
4. 认为代码写法不兼容某版本 → 查 C7 拉取对应版本文档
|
|
||||||
|
|
||||||
**查询优先级**:
|
|
||||||
1. 先查 C7 知识缓存(Phase 0.5 预热结果)
|
|
||||||
2. 缓存未命中 → 调用 `get-library-docs(id, topic="{具体 API 名}")`
|
|
||||||
3. 遵守每库 3 次查询上限
|
|
||||||
|
|
||||||
**标注字段**:
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"c7_verified": true,
|
|
||||||
"c7_source": "gin.Context.JSON() accepts int status code and any interface{}",
|
|
||||||
"verification_method": "c7_cache"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
`verification_method` 取值:
|
|
||||||
- `c7_cache` — 从预热缓存验证
|
|
||||||
- `c7_realtime` — 实时调用 Context7 验证
|
|
||||||
- `model_knowledge` — 未使用 Context7(置信度自动降一级)
|
|
||||||
|
|
||||||
### Layer 3: Post-Review Cross-Validation (Phase 3)
|
|
||||||
|
|
||||||
主 Agent 汇总时的最终验证:
|
|
||||||
|
|
||||||
```
|
|
||||||
对于每个 finding:
|
|
||||||
如果 c7_verified == false 且 severity in [critical, high]:
|
|
||||||
如果涉及第三方库 API:
|
|
||||||
docs = get-library-docs(libraryID, topic="{相关 API}")
|
|
||||||
如果文档支持 Agent 判断 → c7_verified = true, 保留
|
|
||||||
如果文档与 Agent 矛盾 → 降级为 info 或删除, 标记 c7_invalidated
|
|
||||||
如果 Context7 无数据 → 保留, 标注 unverifiable
|
|
||||||
否则 (纯逻辑问题):
|
|
||||||
跳过 C7 验证, 保持原判断
|
|
||||||
```
|
|
||||||
|
|
||||||
**强制规则**:`verification_method == "model_knowledge"` 的 critical/high API 相关发现,未完成交叉验证则自动降级为 medium。
|
|
||||||
|
|
||||||
## Degradation Strategy
|
|
||||||
|
|
||||||
| 场景 | 行为 |
|
|
||||||
|------|------|
|
|
||||||
| Context7 MCP 未配置 | 跳过所有 C7 阶段,报告标注 NONE 覆盖度 |
|
|
||||||
| 网络超时 | 重试 1 次,仍失败则跳过该库 |
|
|
||||||
| `resolve-library-id` 失败 | 记录到 `c7_failures`,跳过该库 |
|
|
||||||
| 查询配额耗尽 | 使用已缓存的最佳信息 |
|
|
||||||
| 子 Agent 中 C7 调用失败 | 标注 `verification_method: "model_knowledge"`,降低置信度 |
|
|
||||||
|
|
||||||
## Report Section: Verification Statistics
|
|
||||||
|
|
||||||
审核报告中包含的 Context7 统计节:
|
|
||||||
|
|
||||||
| 指标 | 说明 |
|
|
||||||
|------|------|
|
|
||||||
| 检测到的依赖库总数 | 项目直接依赖数 |
|
|
||||||
| C7 成功解析的库 | resolve-library-id 成功数 |
|
|
||||||
| C7 解析失败的库 | 失败列表 |
|
|
||||||
| Pre-Review 查询次数 | Phase 0.5 的 get-library-docs 调用数 |
|
|
||||||
| In-Review 查询次数 | Phase 2 子 Agent 的实时查询总数 |
|
|
||||||
| Post-Review 查询次数 | Phase 3 交叉验证查询数 |
|
|
||||||
| C7 验证通过的发现数 | c7_verified == true |
|
|
||||||
| C7 纠正的误判数 | c7_invalidated 标记数 |
|
|
||||||
| 验证覆盖度评级 | FULL / PARTIAL / LIMITED / NONE |
|
|
||||||
|
|
||||||
## Anti-Hallucination Corrections Table
|
|
||||||
|
|
||||||
报告中记录被 Context7 纠正的误判:
|
|
||||||
|
|
||||||
| # | Agent | 原 Severity | 原 Title | 纠正原因 | C7 Source |
|
|
||||||
|---|-------|------------|---------|---------|-----------|
|
|
||||||
| 1 | Security | high | API deprecated | C7 文档显示该 API 在 v2.x 中仍为 stable | /lib/docs... |
|
|
||||||
@@ -1,144 +0,0 @@
|
|||||||
# Code Review Report Template
|
|
||||||
|
|
||||||
审核报告保存到项目根目录的 `code-review-report.md`,使用以下模板:
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
# Code Review Report
|
|
||||||
|
|
||||||
**Project:** {PROJECT_NAME}
|
|
||||||
**Branch:** {BRANCH}
|
|
||||||
**Commit:** {COMMIT_SHA}
|
|
||||||
**Date:** {DATE}
|
|
||||||
**Scope:** {SCOPE_DESCRIPTION}
|
|
||||||
**Files Reviewed:** {TOTAL_FILES}
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Executive Summary
|
|
||||||
|
|
||||||
| 等级 | 数量 | 占比 |
|
|
||||||
|------|------|------|
|
|
||||||
| Critical (P0) | {N} | {%} |
|
|
||||||
| High (P1) | {N} | {%} |
|
|
||||||
| Medium (P2) | {N} | {%} |
|
|
||||||
| Low (P3) | {N} | {%} |
|
|
||||||
| Info (P4) | {N} | {%} |
|
|
||||||
| **Total** | **{N}** | **100%** |
|
|
||||||
|
|
||||||
**Overall Risk:** {HIGH/MEDIUM/LOW} — {一句话总结}
|
|
||||||
**C7 Verification:** {FULL/PARTIAL/LIMITED/NONE}
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Critical Issues (P0) — Immediate Action Required
|
|
||||||
|
|
||||||
### [{RULE}] {TITLE}
|
|
||||||
- **File:** `{FILE}:{LINE}`
|
|
||||||
- **Dimension:** {DIMENSION}
|
|
||||||
- **Confidence:** {CONFIDENCE} | **C7 Verified:** {YES/NO}
|
|
||||||
- **Description:** {DESCRIPTION}
|
|
||||||
- **Suggestion:**
|
|
||||||
```{lang}
|
|
||||||
{CODE_SUGGESTION}
|
|
||||||
```
|
|
||||||
- **References:** {REFERENCES}
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## High Issues (P1) — Fix Before Next Release
|
|
||||||
|
|
||||||
{同上格式}
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Medium Issues (P2) — Plan to Fix
|
|
||||||
|
|
||||||
{同上格式}
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Low Issues (P3) — Nice to Fix
|
|
||||||
|
|
||||||
| # | Rule | File:Line | Title | Confidence |
|
|
||||||
|---|------|-----------|-------|------------|
|
|
||||||
| 1 | {RULE} | `{FILE}:{LINE}` | {TITLE} | {CONF} |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Info (P4) — Suggestions
|
|
||||||
|
|
||||||
| # | File:Line | Suggestion |
|
|
||||||
|---|-----------|------------|
|
|
||||||
| 1 | `{FILE}:{LINE}` | {SUGGESTION} |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Hotspot Analysis
|
|
||||||
|
|
||||||
| Rank | File | Issues | Critical | High | Medium |
|
|
||||||
|------|------|--------|----------|------|--------|
|
|
||||||
| 1 | {FILE} | {N} | {N} | {N} | {N} |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Dimension Summary
|
|
||||||
|
|
||||||
| 维度 | 文件数 | 问题数 | Critical | High |
|
|
||||||
|------|--------|--------|----------|------|
|
|
||||||
| Security & Compliance | {N} | {N} | {N} | {N} |
|
|
||||||
| Architecture & Design | {N} | {N} | {N} | {N} |
|
|
||||||
| Performance & Resource | {N} | {N} | {N} | {N} |
|
|
||||||
| Reliability & Data | {N} | {N} | {N} | {N} |
|
|
||||||
| Quality & Observability | {N} | {N} | {N} | {N} |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Context7 Verification Statistics
|
|
||||||
|
|
||||||
| 指标 | 数值 |
|
|
||||||
|------|------|
|
|
||||||
| 依赖库总数 | {N} |
|
|
||||||
| C7 成功解析 | {N} |
|
|
||||||
| C7 解析失败 | {N} ({FAILED_LIBS}) |
|
|
||||||
| Pre-Review 查询 | {N} |
|
|
||||||
| In-Review 查询 | {N} |
|
|
||||||
| Post-Review 查询 | {N} |
|
|
||||||
| C7 验证通过 | {N} ({%}) |
|
|
||||||
| C7 纠正误判 | {N} |
|
|
||||||
| 覆盖度评级 | {FULL/PARTIAL/LIMITED/NONE} |
|
|
||||||
|
|
||||||
### Anti-Hallucination Corrections
|
|
||||||
|
|
||||||
| # | Agent | 原 Severity | Title | 纠正原因 | C7 Source |
|
|
||||||
|---|-------|------------|-------|---------|-----------|
|
|
||||||
| 1 | {AGENT} | {SEV} | {TITLE} | {REASON} | {SOURCE} |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Recommendations
|
|
||||||
|
|
||||||
### Immediate Actions (This Sprint)
|
|
||||||
1. {P0/P1 对应行动项}
|
|
||||||
|
|
||||||
### Short-term (Next 2-3 Sprints)
|
|
||||||
1. {P2 对应行动项}
|
|
||||||
|
|
||||||
### Long-term
|
|
||||||
1. {架构级改进}
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Methodology
|
|
||||||
|
|
||||||
- **Type:** Multi-agent parallel review + Context7 anti-hallucination
|
|
||||||
- **Agents:** Security, Architecture, Performance, Reliability, Quality
|
|
||||||
- **Isolation:** Independent git worktrees per agent
|
|
||||||
- **Verification:** Context7 three-layer (warm-up → realtime → cross-validation)
|
|
||||||
- **Policy:** API findings ≥ high require C7 verification; unverified auto-downgraded
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
*Generated by Code Review Expert — Universal Multi-Agent Code Review System with Context7 Anti-Hallucination*
|
|
||||||
```
|
|
||||||
@@ -1,247 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import sys
|
|
||||||
from datetime import date
|
|
||||||
|
|
||||||
|
|
||||||
HIGH_SEVERITIES = {"high", "critical"}
|
|
||||||
REQUIRED_FIELDS = {"package", "advisory", "severity", "mitigation", "expires_on"}
|
|
||||||
|
|
||||||
|
|
||||||
def split_kv(line: str) -> tuple[str, str]:
|
|
||||||
# 解析 "key: value" 形式的简单 YAML 行,并去除引号。
|
|
||||||
key, value = line.split(":", 1)
|
|
||||||
value = value.strip()
|
|
||||||
if (value.startswith('"') and value.endswith('"')) or (
|
|
||||||
value.startswith("'") and value.endswith("'")
|
|
||||||
):
|
|
||||||
value = value[1:-1]
|
|
||||||
return key.strip(), value
|
|
||||||
|
|
||||||
|
|
||||||
def parse_exceptions(path: str) -> list[dict]:
|
|
||||||
# 轻量解析异常清单,避免引入额外依赖。
|
|
||||||
exceptions = []
|
|
||||||
current = None
|
|
||||||
with open(path, "r", encoding="utf-8") as handle:
|
|
||||||
for raw in handle:
|
|
||||||
line = raw.strip()
|
|
||||||
if not line or line.startswith("#"):
|
|
||||||
continue
|
|
||||||
if line.startswith("version:") or line.startswith("exceptions:"):
|
|
||||||
continue
|
|
||||||
if line.startswith("- "):
|
|
||||||
if current:
|
|
||||||
exceptions.append(current)
|
|
||||||
current = {}
|
|
||||||
line = line[2:].strip()
|
|
||||||
if line:
|
|
||||||
key, value = split_kv(line)
|
|
||||||
current[key] = value
|
|
||||||
continue
|
|
||||||
if current is not None and ":" in line:
|
|
||||||
key, value = split_kv(line)
|
|
||||||
current[key] = value
|
|
||||||
if current:
|
|
||||||
exceptions.append(current)
|
|
||||||
return exceptions
|
|
||||||
|
|
||||||
|
|
||||||
def pick_advisory_id(advisory: dict) -> str | None:
|
|
||||||
# 优先使用可稳定匹配的标识(GHSA/URL/CVE),避免误匹配到其他同名漏洞。
|
|
||||||
return (
|
|
||||||
advisory.get("github_advisory_id")
|
|
||||||
or advisory.get("url")
|
|
||||||
or (advisory.get("cves") or [None])[0]
|
|
||||||
or (str(advisory.get("id")) if advisory.get("id") is not None else None)
|
|
||||||
or advisory.get("title")
|
|
||||||
or advisory.get("advisory")
|
|
||||||
or advisory.get("overview")
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def iter_vulns(data: dict):
|
|
||||||
# 兼容 pnpm audit 的不同输出结构(advisories / vulnerabilities),并提取 advisory 标识。
|
|
||||||
advisories = data.get("advisories")
|
|
||||||
if isinstance(advisories, dict):
|
|
||||||
for advisory in advisories.values():
|
|
||||||
name = advisory.get("module_name") or advisory.get("name")
|
|
||||||
severity = advisory.get("severity")
|
|
||||||
advisory_id = pick_advisory_id(advisory)
|
|
||||||
title = (
|
|
||||||
advisory.get("title")
|
|
||||||
or advisory.get("advisory")
|
|
||||||
or advisory.get("overview")
|
|
||||||
or advisory.get("url")
|
|
||||||
)
|
|
||||||
yield name, severity, advisory_id, title
|
|
||||||
|
|
||||||
vulnerabilities = data.get("vulnerabilities")
|
|
||||||
if isinstance(vulnerabilities, dict):
|
|
||||||
for name, vuln in vulnerabilities.items():
|
|
||||||
severity = vuln.get("severity")
|
|
||||||
via = vuln.get("via", [])
|
|
||||||
titles = []
|
|
||||||
advisories = []
|
|
||||||
if isinstance(via, list):
|
|
||||||
for item in via:
|
|
||||||
if isinstance(item, dict):
|
|
||||||
advisories.append(
|
|
||||||
item.get("github_advisory_id")
|
|
||||||
or item.get("url")
|
|
||||||
or item.get("source")
|
|
||||||
or item.get("title")
|
|
||||||
or item.get("name")
|
|
||||||
)
|
|
||||||
titles.append(
|
|
||||||
item.get("title")
|
|
||||||
or item.get("url")
|
|
||||||
or item.get("advisory")
|
|
||||||
or item.get("source")
|
|
||||||
)
|
|
||||||
elif isinstance(item, str):
|
|
||||||
advisories.append(item)
|
|
||||||
titles.append(item)
|
|
||||||
elif isinstance(via, str):
|
|
||||||
advisories.append(via)
|
|
||||||
titles.append(via)
|
|
||||||
title = "; ".join([t for t in titles if t])
|
|
||||||
for advisory_id in [a for a in advisories if a]:
|
|
||||||
yield name, severity, advisory_id, title
|
|
||||||
|
|
||||||
|
|
||||||
def normalize_severity(severity: str) -> str:
|
|
||||||
# 统一大小写,避免比较失败。
|
|
||||||
return (severity or "").strip().lower()
|
|
||||||
|
|
||||||
|
|
||||||
def normalize_package(name: str) -> str:
|
|
||||||
# 包名只去掉首尾空白,保留原始大小写,同时兼容非字符串输入。
|
|
||||||
if name is None:
|
|
||||||
return ""
|
|
||||||
return str(name).strip()
|
|
||||||
|
|
||||||
|
|
||||||
def normalize_advisory(advisory: str) -> str:
|
|
||||||
# advisory 统一为小写匹配,避免 GHSA/URL 因大小写差异导致漏匹配。
|
|
||||||
# pnpm 的 source 字段可能是数字,这里统一转为字符串以保证可比较。
|
|
||||||
if advisory is None:
|
|
||||||
return ""
|
|
||||||
return str(advisory).strip().lower()
|
|
||||||
|
|
||||||
|
|
||||||
def parse_date(value: str) -> date | None:
|
|
||||||
# 仅接受 ISO8601 日期格式,非法值视为无效。
|
|
||||||
try:
|
|
||||||
return date.fromisoformat(value)
|
|
||||||
except ValueError:
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
parser = argparse.ArgumentParser()
|
|
||||||
parser.add_argument("--audit", required=True)
|
|
||||||
parser.add_argument("--exceptions", required=True)
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
with open(args.audit, "r", encoding="utf-8") as handle:
|
|
||||||
audit = json.load(handle)
|
|
||||||
|
|
||||||
# 读取异常清单并建立索引,便于快速匹配包名 + advisory。
|
|
||||||
exceptions = parse_exceptions(args.exceptions)
|
|
||||||
exception_index = {}
|
|
||||||
errors = []
|
|
||||||
|
|
||||||
for exc in exceptions:
|
|
||||||
missing = [field for field in REQUIRED_FIELDS if not exc.get(field)]
|
|
||||||
if missing:
|
|
||||||
errors.append(
|
|
||||||
f"Exception missing required fields {missing}: {exc.get('package', '<unknown>')}"
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
exc_severity = normalize_severity(exc.get("severity"))
|
|
||||||
exc_package = normalize_package(exc.get("package"))
|
|
||||||
exc_advisory = normalize_advisory(exc.get("advisory"))
|
|
||||||
exc_date = parse_date(exc.get("expires_on"))
|
|
||||||
if exc_date is None:
|
|
||||||
errors.append(
|
|
||||||
f"Exception has invalid expires_on date: {exc.get('package', '<unknown>')}"
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
if not exc_package or not exc_advisory:
|
|
||||||
errors.append("Exception missing package or advisory value")
|
|
||||||
continue
|
|
||||||
key = (exc_package, exc_advisory)
|
|
||||||
if key in exception_index:
|
|
||||||
errors.append(
|
|
||||||
f"Duplicate exception for {exc_package} advisory {exc.get('advisory')}"
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
exception_index[key] = {
|
|
||||||
"raw": exc,
|
|
||||||
"severity": exc_severity,
|
|
||||||
"expires_on": exc_date,
|
|
||||||
}
|
|
||||||
|
|
||||||
today = date.today()
|
|
||||||
missing_exceptions = []
|
|
||||||
expired_exceptions = []
|
|
||||||
|
|
||||||
# 去重处理:同一包名 + advisory 可能在不同字段重复出现。
|
|
||||||
seen = set()
|
|
||||||
for name, severity, advisory_id, title in iter_vulns(audit):
|
|
||||||
sev = normalize_severity(severity)
|
|
||||||
if sev not in HIGH_SEVERITIES or not name:
|
|
||||||
continue
|
|
||||||
advisory_key = normalize_advisory(advisory_id)
|
|
||||||
if not advisory_key:
|
|
||||||
errors.append(
|
|
||||||
f"High/Critical vulnerability missing advisory id: {name} ({sev})"
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
key = (normalize_package(name), advisory_key)
|
|
||||||
if key in seen:
|
|
||||||
continue
|
|
||||||
seen.add(key)
|
|
||||||
exc = exception_index.get(key)
|
|
||||||
if exc is None:
|
|
||||||
missing_exceptions.append((name, sev, advisory_id, title))
|
|
||||||
continue
|
|
||||||
if exc["severity"] and exc["severity"] != sev:
|
|
||||||
errors.append(
|
|
||||||
"Exception severity mismatch: "
|
|
||||||
f"{name} ({advisory_id}) expected {sev}, got {exc['severity']}"
|
|
||||||
)
|
|
||||||
if exc["expires_on"] and exc["expires_on"] < today:
|
|
||||||
expired_exceptions.append(
|
|
||||||
(name, sev, advisory_id, exc["expires_on"].isoformat())
|
|
||||||
)
|
|
||||||
|
|
||||||
if missing_exceptions:
|
|
||||||
errors.append("High/Critical vulnerabilities missing exceptions:")
|
|
||||||
for name, sev, advisory_id, title in missing_exceptions:
|
|
||||||
label = f"{name} ({sev})"
|
|
||||||
if advisory_id:
|
|
||||||
label = f"{label} [{advisory_id}]"
|
|
||||||
if title:
|
|
||||||
label = f"{label}: {title}"
|
|
||||||
errors.append(f"- {label}")
|
|
||||||
|
|
||||||
if expired_exceptions:
|
|
||||||
errors.append("Exceptions expired:")
|
|
||||||
for name, sev, advisory_id, expires_on in expired_exceptions:
|
|
||||||
errors.append(
|
|
||||||
f"- {name} ({sev}) [{advisory_id}] expired on {expires_on}"
|
|
||||||
)
|
|
||||||
|
|
||||||
if errors:
|
|
||||||
sys.stderr.write("\n".join(errors) + "\n")
|
|
||||||
return 1
|
|
||||||
|
|
||||||
print("Audit exceptions validated.")
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,164 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""OpenAI OAuth 灰度发布演练脚本(本地模拟)。
|
|
||||||
|
|
||||||
该脚本会启动本地 mock Ops API,调用 openai_oauth_gray_guard.py,
|
|
||||||
验证以下场景:
|
|
||||||
1) A/B/C/D 四个灰度批次均通过
|
|
||||||
2) 注入异常场景触发阈值告警并返回退出码 2(模拟自动回滚触发)
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import subprocess
|
|
||||||
import threading
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Dict, Tuple
|
|
||||||
from urllib.parse import parse_qs, urlparse
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
GUARD_SCRIPT = ROOT / "tools" / "perf" / "openai_oauth_gray_guard.py"
|
|
||||||
REPORT_PATH = ROOT / "docs" / "perf" / "openai-oauth-gray-drill-report.md"
|
|
||||||
|
|
||||||
|
|
||||||
THRESHOLDS = {
|
|
||||||
"sla_percent_min": 99.5,
|
|
||||||
"ttft_p99_ms_max": 900,
|
|
||||||
"request_error_rate_percent_max": 2.0,
|
|
||||||
"upstream_error_rate_percent_max": 2.0,
|
|
||||||
}
|
|
||||||
|
|
||||||
STAGE_SNAPSHOTS: Dict[str, Dict[str, float]] = {
|
|
||||||
"A": {"sla": 99.78, "ttft": 780, "error_rate": 1.20, "upstream_error_rate": 1.05},
|
|
||||||
"B": {"sla": 99.82, "ttft": 730, "error_rate": 1.05, "upstream_error_rate": 0.92},
|
|
||||||
"C": {"sla": 99.86, "ttft": 680, "error_rate": 0.88, "upstream_error_rate": 0.80},
|
|
||||||
"D": {"sla": 99.89, "ttft": 640, "error_rate": 0.72, "upstream_error_rate": 0.67},
|
|
||||||
"rollback": {"sla": 97.10, "ttft": 1550, "error_rate": 6.30, "upstream_error_rate": 5.60},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class _MockHandler(BaseHTTPRequestHandler):
|
|
||||||
def _write_json(self, payload: dict) -> None:
|
|
||||||
raw = json.dumps(payload, ensure_ascii=False).encode("utf-8")
|
|
||||||
self.send_response(200)
|
|
||||||
self.send_header("Content-Type", "application/json")
|
|
||||||
self.send_header("Content-Length", str(len(raw)))
|
|
||||||
self.end_headers()
|
|
||||||
self.wfile.write(raw)
|
|
||||||
|
|
||||||
def log_message(self, format: str, *args): # noqa: A003
|
|
||||||
return
|
|
||||||
|
|
||||||
def do_GET(self): # noqa: N802
|
|
||||||
parsed = urlparse(self.path)
|
|
||||||
if parsed.path.endswith("/api/v1/admin/ops/settings/metric-thresholds"):
|
|
||||||
self._write_json({"code": 0, "message": "success", "data": THRESHOLDS})
|
|
||||||
return
|
|
||||||
|
|
||||||
if parsed.path.endswith("/api/v1/admin/ops/dashboard/overview"):
|
|
||||||
q = parse_qs(parsed.query)
|
|
||||||
stage = (q.get("group_id") or ["A"])[0]
|
|
||||||
snapshot = STAGE_SNAPSHOTS.get(stage, STAGE_SNAPSHOTS["A"])
|
|
||||||
self._write_json(
|
|
||||||
{
|
|
||||||
"code": 0,
|
|
||||||
"message": "success",
|
|
||||||
"data": {
|
|
||||||
"sla": snapshot["sla"],
|
|
||||||
"error_rate": snapshot["error_rate"],
|
|
||||||
"upstream_error_rate": snapshot["upstream_error_rate"],
|
|
||||||
"ttft": {"p99_ms": snapshot["ttft"]},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
self.send_response(404)
|
|
||||||
self.end_headers()
|
|
||||||
|
|
||||||
|
|
||||||
def run_guard(base_url: str, stage: str) -> Tuple[int, str]:
|
|
||||||
cmd = [
|
|
||||||
"python",
|
|
||||||
str(GUARD_SCRIPT),
|
|
||||||
"--base-url",
|
|
||||||
base_url,
|
|
||||||
"--platform",
|
|
||||||
"openai",
|
|
||||||
"--time-range",
|
|
||||||
"30m",
|
|
||||||
"--group-id",
|
|
||||||
stage,
|
|
||||||
]
|
|
||||||
proc = subprocess.run(cmd, cwd=str(ROOT), capture_output=True, text=True)
|
|
||||||
output = (proc.stdout + "\n" + proc.stderr).strip()
|
|
||||||
return proc.returncode, output
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
server = HTTPServer(("127.0.0.1", 0), _MockHandler)
|
|
||||||
host, port = server.server_address
|
|
||||||
base_url = f"http://{host}:{port}"
|
|
||||||
|
|
||||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
|
||||||
thread.start()
|
|
||||||
|
|
||||||
lines = [
|
|
||||||
"# OpenAI OAuth 灰度守护演练报告",
|
|
||||||
"",
|
|
||||||
"> 类型:本地 mock 演练(用于验证灰度守护与回滚触发机制)",
|
|
||||||
f"> 生成脚本:`tools/perf/openai_oauth_gray_drill.py`",
|
|
||||||
"",
|
|
||||||
"## 1. 灰度批次结果(6.1)",
|
|
||||||
"",
|
|
||||||
"| 批次 | 流量比例 | 守护脚本退出码 | 结果 |",
|
|
||||||
"|---|---:|---:|---|",
|
|
||||||
]
|
|
||||||
|
|
||||||
batch_plan = [("A", "5%"), ("B", "20%"), ("C", "50%"), ("D", "100%")]
|
|
||||||
all_pass = True
|
|
||||||
for stage, ratio in batch_plan:
|
|
||||||
code, _ = run_guard(base_url, stage)
|
|
||||||
ok = code == 0
|
|
||||||
all_pass = all_pass and ok
|
|
||||||
lines.append(f"| {stage} | {ratio} | {code} | {'通过' if ok else '失败'} |")
|
|
||||||
|
|
||||||
lines.extend([
|
|
||||||
"",
|
|
||||||
"## 2. 回滚触发演练(6.2)",
|
|
||||||
"",
|
|
||||||
])
|
|
||||||
|
|
||||||
rollback_code, rollback_output = run_guard(base_url, "rollback")
|
|
||||||
rollback_triggered = rollback_code == 2
|
|
||||||
lines.append(f"- 注入异常场景退出码:`{rollback_code}`")
|
|
||||||
lines.append(f"- 是否触发回滚条件:`{'是' if rollback_triggered else '否'}`")
|
|
||||||
lines.append("- 关键信息摘录:")
|
|
||||||
excerpt = "\n".join(rollback_output.splitlines()[:8])
|
|
||||||
lines.append("```text")
|
|
||||||
lines.append(excerpt)
|
|
||||||
lines.append("```")
|
|
||||||
|
|
||||||
lines.extend([
|
|
||||||
"",
|
|
||||||
"## 3. 验收结论(6.3)",
|
|
||||||
"",
|
|
||||||
f"- 批次灰度结果:`{'通过' if all_pass else '不通过'}`",
|
|
||||||
f"- 回滚触发机制:`{'通过' if rollback_triggered else '不通过'}`",
|
|
||||||
f"- 结论:`{'通过(可进入真实环境灰度)' if all_pass and rollback_triggered else '不通过(需修复后复测)'}`",
|
|
||||||
])
|
|
||||||
|
|
||||||
REPORT_PATH.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
REPORT_PATH.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
|
||||||
|
|
||||||
server.shutdown()
|
|
||||||
server.server_close()
|
|
||||||
|
|
||||||
print(f"drill report generated: {REPORT_PATH}")
|
|
||||||
return 0 if all_pass and rollback_triggered else 1
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,213 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""OpenAI OAuth 灰度阈值守护脚本。
|
|
||||||
|
|
||||||
用途:
|
|
||||||
- 拉取 Ops 指标阈值配置与 Dashboard Overview 实时数据
|
|
||||||
- 对比 P99 TTFT / 错误率 / SLA
|
|
||||||
- 作为 6.2 灰度守护的自动化门禁(退出码可直接用于 CI/CD)
|
|
||||||
|
|
||||||
退出码:
|
|
||||||
- 0: 指标通过
|
|
||||||
- 1: 请求失败/参数错误
|
|
||||||
- 2: 指标超阈值(建议停止扩量并回滚)
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import sys
|
|
||||||
import urllib.error
|
|
||||||
import urllib.parse
|
|
||||||
import urllib.request
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import Any, Dict, List, Optional
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class GuardThresholds:
|
|
||||||
sla_percent_min: Optional[float]
|
|
||||||
ttft_p99_ms_max: Optional[float]
|
|
||||||
request_error_rate_percent_max: Optional[float]
|
|
||||||
upstream_error_rate_percent_max: Optional[float]
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class GuardSnapshot:
|
|
||||||
sla: Optional[float]
|
|
||||||
ttft_p99_ms: Optional[float]
|
|
||||||
request_error_rate_percent: Optional[float]
|
|
||||||
upstream_error_rate_percent: Optional[float]
|
|
||||||
|
|
||||||
|
|
||||||
def build_headers(token: str) -> Dict[str, str]:
|
|
||||||
headers = {"Accept": "application/json"}
|
|
||||||
if token.strip():
|
|
||||||
headers["Authorization"] = f"Bearer {token.strip()}"
|
|
||||||
return headers
|
|
||||||
|
|
||||||
|
|
||||||
def request_json(url: str, headers: Dict[str, str]) -> Dict[str, Any]:
|
|
||||||
req = urllib.request.Request(url=url, method="GET", headers=headers)
|
|
||||||
try:
|
|
||||||
with urllib.request.urlopen(req, timeout=15) as resp:
|
|
||||||
raw = resp.read().decode("utf-8")
|
|
||||||
return json.loads(raw)
|
|
||||||
except urllib.error.HTTPError as e:
|
|
||||||
body = e.read().decode("utf-8", errors="replace")
|
|
||||||
raise RuntimeError(f"HTTP {e.code}: {body}") from e
|
|
||||||
except urllib.error.URLError as e:
|
|
||||||
raise RuntimeError(f"request failed: {e}") from e
|
|
||||||
|
|
||||||
|
|
||||||
def parse_envelope_data(payload: Dict[str, Any]) -> Dict[str, Any]:
|
|
||||||
if not isinstance(payload, dict):
|
|
||||||
raise RuntimeError("invalid response payload")
|
|
||||||
if payload.get("code") != 0:
|
|
||||||
raise RuntimeError(f"api error: code={payload.get('code')} message={payload.get('message')}")
|
|
||||||
data = payload.get("data")
|
|
||||||
if not isinstance(data, dict):
|
|
||||||
raise RuntimeError("invalid response data")
|
|
||||||
return data
|
|
||||||
|
|
||||||
|
|
||||||
def parse_thresholds(data: Dict[str, Any]) -> GuardThresholds:
|
|
||||||
return GuardThresholds(
|
|
||||||
sla_percent_min=to_float_or_none(data.get("sla_percent_min")),
|
|
||||||
ttft_p99_ms_max=to_float_or_none(data.get("ttft_p99_ms_max")),
|
|
||||||
request_error_rate_percent_max=to_float_or_none(data.get("request_error_rate_percent_max")),
|
|
||||||
upstream_error_rate_percent_max=to_float_or_none(data.get("upstream_error_rate_percent_max")),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def parse_snapshot(data: Dict[str, Any]) -> GuardSnapshot:
|
|
||||||
ttft = data.get("ttft") if isinstance(data.get("ttft"), dict) else {}
|
|
||||||
return GuardSnapshot(
|
|
||||||
sla=to_float_or_none(data.get("sla")),
|
|
||||||
ttft_p99_ms=to_float_or_none(ttft.get("p99_ms")),
|
|
||||||
request_error_rate_percent=to_float_or_none(data.get("error_rate")),
|
|
||||||
upstream_error_rate_percent=to_float_or_none(data.get("upstream_error_rate")),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def to_float_or_none(v: Any) -> Optional[float]:
|
|
||||||
if v is None:
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
return float(v)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def evaluate(snapshot: GuardSnapshot, thresholds: GuardThresholds) -> List[str]:
|
|
||||||
violations: List[str] = []
|
|
||||||
|
|
||||||
if thresholds.sla_percent_min is not None and snapshot.sla is not None:
|
|
||||||
if snapshot.sla < thresholds.sla_percent_min:
|
|
||||||
violations.append(
|
|
||||||
f"SLA 低于阈值: actual={snapshot.sla:.2f}% threshold={thresholds.sla_percent_min:.2f}%"
|
|
||||||
)
|
|
||||||
|
|
||||||
if thresholds.ttft_p99_ms_max is not None and snapshot.ttft_p99_ms is not None:
|
|
||||||
if snapshot.ttft_p99_ms > thresholds.ttft_p99_ms_max:
|
|
||||||
violations.append(
|
|
||||||
f"TTFT P99 超阈值: actual={snapshot.ttft_p99_ms:.2f}ms threshold={thresholds.ttft_p99_ms_max:.2f}ms"
|
|
||||||
)
|
|
||||||
|
|
||||||
if (
|
|
||||||
thresholds.request_error_rate_percent_max is not None
|
|
||||||
and snapshot.request_error_rate_percent is not None
|
|
||||||
and snapshot.request_error_rate_percent > thresholds.request_error_rate_percent_max
|
|
||||||
):
|
|
||||||
violations.append(
|
|
||||||
"请求错误率超阈值: "
|
|
||||||
f"actual={snapshot.request_error_rate_percent:.2f}% "
|
|
||||||
f"threshold={thresholds.request_error_rate_percent_max:.2f}%"
|
|
||||||
)
|
|
||||||
|
|
||||||
if (
|
|
||||||
thresholds.upstream_error_rate_percent_max is not None
|
|
||||||
and snapshot.upstream_error_rate_percent is not None
|
|
||||||
and snapshot.upstream_error_rate_percent > thresholds.upstream_error_rate_percent_max
|
|
||||||
):
|
|
||||||
violations.append(
|
|
||||||
"上游错误率超阈值: "
|
|
||||||
f"actual={snapshot.upstream_error_rate_percent:.2f}% "
|
|
||||||
f"threshold={thresholds.upstream_error_rate_percent_max:.2f}%"
|
|
||||||
)
|
|
||||||
|
|
||||||
return violations
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
parser = argparse.ArgumentParser(description="OpenAI OAuth 灰度阈值守护")
|
|
||||||
parser.add_argument("--base-url", required=True, help="服务地址,例如 http://127.0.0.1:5231")
|
|
||||||
parser.add_argument("--admin-token", default="", help="Admin JWT(可选,按部署策略)")
|
|
||||||
parser.add_argument("--platform", default="openai", help="平台过滤,默认 openai")
|
|
||||||
parser.add_argument("--time-range", default="30m", help="时间窗口: 5m/30m/1h/6h/24h/7d/30d")
|
|
||||||
parser.add_argument("--group-id", default="", help="可选 group_id")
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
base = args.base_url.rstrip("/")
|
|
||||||
headers = build_headers(args.admin_token)
|
|
||||||
|
|
||||||
try:
|
|
||||||
threshold_url = f"{base}/api/v1/admin/ops/settings/metric-thresholds"
|
|
||||||
thresholds_raw = request_json(threshold_url, headers)
|
|
||||||
thresholds = parse_thresholds(parse_envelope_data(thresholds_raw))
|
|
||||||
|
|
||||||
query = {"platform": args.platform, "time_range": args.time_range}
|
|
||||||
if args.group_id.strip():
|
|
||||||
query["group_id"] = args.group_id.strip()
|
|
||||||
overview_url = (
|
|
||||||
f"{base}/api/v1/admin/ops/dashboard/overview?"
|
|
||||||
+ urllib.parse.urlencode(query)
|
|
||||||
)
|
|
||||||
overview_raw = request_json(overview_url, headers)
|
|
||||||
snapshot = parse_snapshot(parse_envelope_data(overview_raw))
|
|
||||||
|
|
||||||
print("[OpenAI OAuth Gray Guard] 当前快照:")
|
|
||||||
print(
|
|
||||||
json.dumps(
|
|
||||||
{
|
|
||||||
"sla": snapshot.sla,
|
|
||||||
"ttft_p99_ms": snapshot.ttft_p99_ms,
|
|
||||||
"request_error_rate_percent": snapshot.request_error_rate_percent,
|
|
||||||
"upstream_error_rate_percent": snapshot.upstream_error_rate_percent,
|
|
||||||
},
|
|
||||||
ensure_ascii=False,
|
|
||||||
indent=2,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
print("[OpenAI OAuth Gray Guard] 阈值配置:")
|
|
||||||
print(
|
|
||||||
json.dumps(
|
|
||||||
{
|
|
||||||
"sla_percent_min": thresholds.sla_percent_min,
|
|
||||||
"ttft_p99_ms_max": thresholds.ttft_p99_ms_max,
|
|
||||||
"request_error_rate_percent_max": thresholds.request_error_rate_percent_max,
|
|
||||||
"upstream_error_rate_percent_max": thresholds.upstream_error_rate_percent_max,
|
|
||||||
},
|
|
||||||
ensure_ascii=False,
|
|
||||||
indent=2,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
violations = evaluate(snapshot, thresholds)
|
|
||||||
if violations:
|
|
||||||
print("[OpenAI OAuth Gray Guard] 检测到阈值违例:")
|
|
||||||
for idx, line in enumerate(violations, start=1):
|
|
||||||
print(f" {idx}. {line}")
|
|
||||||
print("[OpenAI OAuth Gray Guard] 建议:停止扩量并执行回滚。")
|
|
||||||
return 2
|
|
||||||
|
|
||||||
print("[OpenAI OAuth Gray Guard] 指标通过,可继续观察或按计划扩量。")
|
|
||||||
return 0
|
|
||||||
|
|
||||||
except Exception as exc:
|
|
||||||
print(f"[OpenAI OAuth Gray Guard] 执行失败: {exc}", file=sys.stderr)
|
|
||||||
return 1
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,122 +0,0 @@
|
|||||||
import http from 'k6/http';
|
|
||||||
import { check } from 'k6';
|
|
||||||
import { Rate, Trend } from 'k6/metrics';
|
|
||||||
|
|
||||||
const baseURL = __ENV.BASE_URL || 'http://127.0.0.1:5231';
|
|
||||||
const apiKey = __ENV.API_KEY || '';
|
|
||||||
const model = __ENV.MODEL || 'gpt-5';
|
|
||||||
const timeout = __ENV.TIMEOUT || '180s';
|
|
||||||
|
|
||||||
const nonStreamRPS = Number(__ENV.NON_STREAM_RPS || 8);
|
|
||||||
const streamRPS = Number(__ENV.STREAM_RPS || 4);
|
|
||||||
const duration = __ENV.DURATION || '3m';
|
|
||||||
const preAllocatedVUs = Number(__ENV.PRE_ALLOCATED_VUS || 30);
|
|
||||||
const maxVUs = Number(__ENV.MAX_VUS || 200);
|
|
||||||
|
|
||||||
const reqDurationMs = new Trend('openai_oauth_req_duration_ms', true);
|
|
||||||
const ttftMs = new Trend('openai_oauth_ttft_ms', true);
|
|
||||||
const non2xxRate = new Rate('openai_oauth_non2xx_rate');
|
|
||||||
const streamDoneRate = new Rate('openai_oauth_stream_done_rate');
|
|
||||||
|
|
||||||
export const options = {
|
|
||||||
scenarios: {
|
|
||||||
non_stream: {
|
|
||||||
executor: 'constant-arrival-rate',
|
|
||||||
rate: nonStreamRPS,
|
|
||||||
timeUnit: '1s',
|
|
||||||
duration,
|
|
||||||
preAllocatedVUs,
|
|
||||||
maxVUs,
|
|
||||||
exec: 'runNonStream',
|
|
||||||
tags: { request_type: 'non_stream' },
|
|
||||||
},
|
|
||||||
stream: {
|
|
||||||
executor: 'constant-arrival-rate',
|
|
||||||
rate: streamRPS,
|
|
||||||
timeUnit: '1s',
|
|
||||||
duration,
|
|
||||||
preAllocatedVUs,
|
|
||||||
maxVUs,
|
|
||||||
exec: 'runStream',
|
|
||||||
tags: { request_type: 'stream' },
|
|
||||||
},
|
|
||||||
},
|
|
||||||
thresholds: {
|
|
||||||
openai_oauth_non2xx_rate: ['rate<0.01'],
|
|
||||||
openai_oauth_req_duration_ms: ['p(95)<3000', 'p(99)<6000'],
|
|
||||||
openai_oauth_ttft_ms: ['p(99)<1200'],
|
|
||||||
openai_oauth_stream_done_rate: ['rate>0.99'],
|
|
||||||
},
|
|
||||||
};
|
|
||||||
|
|
||||||
function buildHeaders() {
|
|
||||||
const headers = {
|
|
||||||
'Content-Type': 'application/json',
|
|
||||||
'User-Agent': 'codex_cli_rs/0.1.0',
|
|
||||||
};
|
|
||||||
if (apiKey) {
|
|
||||||
headers.Authorization = `Bearer ${apiKey}`;
|
|
||||||
}
|
|
||||||
return headers;
|
|
||||||
}
|
|
||||||
|
|
||||||
function buildBody(stream) {
|
|
||||||
return JSON.stringify({
|
|
||||||
model,
|
|
||||||
stream,
|
|
||||||
input: [
|
|
||||||
{
|
|
||||||
role: 'user',
|
|
||||||
content: [
|
|
||||||
{
|
|
||||||
type: 'input_text',
|
|
||||||
text: '请返回一句极短的话:pong',
|
|
||||||
},
|
|
||||||
],
|
|
||||||
},
|
|
||||||
],
|
|
||||||
max_output_tokens: 32,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function recordMetrics(res, stream) {
|
|
||||||
reqDurationMs.add(res.timings.duration, { request_type: stream ? 'stream' : 'non_stream' });
|
|
||||||
ttftMs.add(res.timings.waiting, { request_type: stream ? 'stream' : 'non_stream' });
|
|
||||||
non2xxRate.add(res.status < 200 || res.status >= 300, { request_type: stream ? 'stream' : 'non_stream' });
|
|
||||||
|
|
||||||
if (stream) {
|
|
||||||
const done = !!res.body && res.body.indexOf('[DONE]') >= 0;
|
|
||||||
streamDoneRate.add(done, { request_type: 'stream' });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function postResponses(stream) {
|
|
||||||
const url = `${baseURL}/v1/responses`;
|
|
||||||
const res = http.post(url, buildBody(stream), {
|
|
||||||
headers: buildHeaders(),
|
|
||||||
timeout,
|
|
||||||
tags: { endpoint: '/v1/responses', request_type: stream ? 'stream' : 'non_stream' },
|
|
||||||
});
|
|
||||||
|
|
||||||
check(res, {
|
|
||||||
'status is 2xx': (r) => r.status >= 200 && r.status < 300,
|
|
||||||
});
|
|
||||||
|
|
||||||
recordMetrics(res, stream);
|
|
||||||
return res;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function runNonStream() {
|
|
||||||
postResponses(false);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function runStream() {
|
|
||||||
postResponses(true);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function handleSummary(data) {
|
|
||||||
return {
|
|
||||||
stdout: `\nOpenAI OAuth /v1/responses 基线完成\n${JSON.stringify(data.metrics, null, 2)}\n`,
|
|
||||||
'docs/perf/openai-oauth-k6-summary.json': JSON.stringify(data, null, 2),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,167 +0,0 @@
|
|||||||
import http from 'k6/http';
|
|
||||||
import { check, sleep } from 'k6';
|
|
||||||
import { Rate, Trend } from 'k6/metrics';
|
|
||||||
|
|
||||||
const baseURL = (__ENV.BASE_URL || 'http://127.0.0.1:5231').replace(/\/$/, '');
|
|
||||||
const httpAPIKey = (__ENV.HTTP_API_KEY || '').trim();
|
|
||||||
const wsAPIKey = (__ENV.WS_API_KEY || '').trim();
|
|
||||||
const model = __ENV.MODEL || 'gpt-5.1';
|
|
||||||
const duration = __ENV.DURATION || '5m';
|
|
||||||
const timeout = __ENV.TIMEOUT || '180s';
|
|
||||||
|
|
||||||
const httpRPS = Number(__ENV.HTTP_RPS || 10);
|
|
||||||
const wsRPS = Number(__ENV.WS_RPS || 10);
|
|
||||||
const chainRPS = Number(__ENV.CHAIN_RPS || 1);
|
|
||||||
const chainRounds = Number(__ENV.CHAIN_ROUNDS || 20);
|
|
||||||
const preAllocatedVUs = Number(__ENV.PRE_ALLOCATED_VUS || 40);
|
|
||||||
const maxVUs = Number(__ENV.MAX_VUS || 300);
|
|
||||||
|
|
||||||
const httpDurationMs = new Trend('openai_http_req_duration_ms', true);
|
|
||||||
const wsDurationMs = new Trend('openai_ws_req_duration_ms', true);
|
|
||||||
const wsChainDurationMs = new Trend('openai_ws_chain_round_duration_ms', true);
|
|
||||||
const wsChainTTFTMs = new Trend('openai_ws_chain_round_ttft_ms', true);
|
|
||||||
const httpNon2xxRate = new Rate('openai_http_non2xx_rate');
|
|
||||||
const wsNon2xxRate = new Rate('openai_ws_non2xx_rate');
|
|
||||||
const wsChainRoundSuccessRate = new Rate('openai_ws_chain_round_success_rate');
|
|
||||||
|
|
||||||
export const options = {
|
|
||||||
scenarios: {
|
|
||||||
http_baseline: {
|
|
||||||
executor: 'constant-arrival-rate',
|
|
||||||
exec: 'runHTTPBaseline',
|
|
||||||
rate: httpRPS,
|
|
||||||
timeUnit: '1s',
|
|
||||||
duration,
|
|
||||||
preAllocatedVUs,
|
|
||||||
maxVUs,
|
|
||||||
tags: { path: 'http_baseline' },
|
|
||||||
},
|
|
||||||
ws_baseline: {
|
|
||||||
executor: 'constant-arrival-rate',
|
|
||||||
exec: 'runWSBaseline',
|
|
||||||
rate: wsRPS,
|
|
||||||
timeUnit: '1s',
|
|
||||||
duration,
|
|
||||||
preAllocatedVUs,
|
|
||||||
maxVUs,
|
|
||||||
tags: { path: 'ws_baseline' },
|
|
||||||
},
|
|
||||||
ws_chain_20_rounds: {
|
|
||||||
executor: 'constant-arrival-rate',
|
|
||||||
exec: 'runWSChain20Rounds',
|
|
||||||
rate: chainRPS,
|
|
||||||
timeUnit: '1s',
|
|
||||||
duration,
|
|
||||||
preAllocatedVUs: Math.max(2, Math.ceil(chainRPS * 2)),
|
|
||||||
maxVUs: Math.max(20, Math.ceil(chainRPS * 10)),
|
|
||||||
tags: { path: 'ws_chain_20_rounds' },
|
|
||||||
},
|
|
||||||
},
|
|
||||||
thresholds: {
|
|
||||||
openai_http_non2xx_rate: ['rate<0.02'],
|
|
||||||
openai_ws_non2xx_rate: ['rate<0.02'],
|
|
||||||
openai_http_req_duration_ms: ['p(95)<4000', 'p(99)<7000'],
|
|
||||||
openai_ws_req_duration_ms: ['p(95)<3000', 'p(99)<6000'],
|
|
||||||
openai_ws_chain_round_success_rate: ['rate>0.98'],
|
|
||||||
openai_ws_chain_round_ttft_ms: ['p(99)<1200'],
|
|
||||||
},
|
|
||||||
};
|
|
||||||
|
|
||||||
function buildHeaders(apiKey) {
|
|
||||||
const headers = {
|
|
||||||
'Content-Type': 'application/json',
|
|
||||||
'User-Agent': 'codex_cli_rs/0.98.0',
|
|
||||||
};
|
|
||||||
if (apiKey) {
|
|
||||||
headers.Authorization = `Bearer ${apiKey}`;
|
|
||||||
}
|
|
||||||
return headers;
|
|
||||||
}
|
|
||||||
|
|
||||||
function buildBody(previousResponseID) {
|
|
||||||
const body = {
|
|
||||||
model,
|
|
||||||
stream: false,
|
|
||||||
input: [
|
|
||||||
{
|
|
||||||
role: 'user',
|
|
||||||
content: [{ type: 'input_text', text: '请回复一个单词: pong' }],
|
|
||||||
},
|
|
||||||
],
|
|
||||||
max_output_tokens: 64,
|
|
||||||
};
|
|
||||||
if (previousResponseID) {
|
|
||||||
body.previous_response_id = previousResponseID;
|
|
||||||
}
|
|
||||||
return JSON.stringify(body);
|
|
||||||
}
|
|
||||||
|
|
||||||
function postResponses(apiKey, body, tags) {
|
|
||||||
const res = http.post(`${baseURL}/v1/responses`, body, {
|
|
||||||
headers: buildHeaders(apiKey),
|
|
||||||
timeout,
|
|
||||||
tags,
|
|
||||||
});
|
|
||||||
check(res, {
|
|
||||||
'status is 2xx': (r) => r.status >= 200 && r.status < 300,
|
|
||||||
});
|
|
||||||
return res;
|
|
||||||
}
|
|
||||||
|
|
||||||
function parseResponseID(res) {
|
|
||||||
if (!res || !res.body) {
|
|
||||||
return '';
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
const payload = JSON.parse(res.body);
|
|
||||||
if (payload && typeof payload.id === 'string') {
|
|
||||||
return payload.id.trim();
|
|
||||||
}
|
|
||||||
} catch (_) {
|
|
||||||
return '';
|
|
||||||
}
|
|
||||||
return '';
|
|
||||||
}
|
|
||||||
|
|
||||||
export function runHTTPBaseline() {
|
|
||||||
const res = postResponses(httpAPIKey, buildBody(''), { transport: 'http' });
|
|
||||||
httpDurationMs.add(res.timings.duration, { transport: 'http' });
|
|
||||||
httpNon2xxRate.add(res.status < 200 || res.status >= 300, { transport: 'http' });
|
|
||||||
}
|
|
||||||
|
|
||||||
export function runWSBaseline() {
|
|
||||||
const res = postResponses(wsAPIKey, buildBody(''), { transport: 'ws_v2' });
|
|
||||||
wsDurationMs.add(res.timings.duration, { transport: 'ws_v2' });
|
|
||||||
wsNon2xxRate.add(res.status < 200 || res.status >= 300, { transport: 'ws_v2' });
|
|
||||||
}
|
|
||||||
|
|
||||||
// 20+ 轮续链专项,验证 previous_response_id 在长链下的稳定性与时延。
|
|
||||||
export function runWSChain20Rounds() {
|
|
||||||
let previousResponseID = '';
|
|
||||||
for (let round = 1; round <= chainRounds; round += 1) {
|
|
||||||
const roundStart = Date.now();
|
|
||||||
const res = postResponses(wsAPIKey, buildBody(previousResponseID), { transport: 'ws_v2_chain' });
|
|
||||||
const ok = res.status >= 200 && res.status < 300;
|
|
||||||
wsChainRoundSuccessRate.add(ok, { round: `${round}` });
|
|
||||||
wsChainDurationMs.add(Date.now() - roundStart, { round: `${round}` });
|
|
||||||
wsChainTTFTMs.add(res.timings.waiting, { round: `${round}` });
|
|
||||||
wsNon2xxRate.add(!ok, { transport: 'ws_v2_chain' });
|
|
||||||
if (!ok) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const respID = parseResponseID(res);
|
|
||||||
if (!respID) {
|
|
||||||
wsChainRoundSuccessRate.add(false, { round: `${round}`, reason: 'missing_response_id' });
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
previousResponseID = respID;
|
|
||||||
sleep(0.01);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
export function handleSummary(data) {
|
|
||||||
return {
|
|
||||||
stdout: `\nOpenAI WSv2 对比压测完成\n${JSON.stringify(data.metrics, null, 2)}\n`,
|
|
||||||
'docs/perf/openai-ws-v2-compare-summary.json': JSON.stringify(data, null, 2),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,123 +0,0 @@
|
|||||||
import http from 'k6/http';
|
|
||||||
import { check } from 'k6';
|
|
||||||
import { Rate, Trend } from 'k6/metrics';
|
|
||||||
|
|
||||||
const pooledBaseURL = (__ENV.POOLED_BASE_URL || 'http://127.0.0.1:5231').replace(/\/$/, '');
|
|
||||||
const oneToOneBaseURL = (__ENV.ONE_TO_ONE_BASE_URL || '').replace(/\/$/, '');
|
|
||||||
const wsAPIKey = (__ENV.WS_API_KEY || '').trim();
|
|
||||||
const model = __ENV.MODEL || 'gpt-5.1';
|
|
||||||
const timeout = __ENV.TIMEOUT || '180s';
|
|
||||||
const duration = __ENV.DURATION || '5m';
|
|
||||||
const pooledRPS = Number(__ENV.POOLED_RPS || 12);
|
|
||||||
const oneToOneRPS = Number(__ENV.ONE_TO_ONE_RPS || 12);
|
|
||||||
const preAllocatedVUs = Number(__ENV.PRE_ALLOCATED_VUS || 50);
|
|
||||||
const maxVUs = Number(__ENV.MAX_VUS || 400);
|
|
||||||
|
|
||||||
const pooledDurationMs = new Trend('openai_ws_pooled_duration_ms', true);
|
|
||||||
const oneToOneDurationMs = new Trend('openai_ws_one_to_one_duration_ms', true);
|
|
||||||
const pooledTTFTMs = new Trend('openai_ws_pooled_ttft_ms', true);
|
|
||||||
const oneToOneTTFTMs = new Trend('openai_ws_one_to_one_ttft_ms', true);
|
|
||||||
const pooledNon2xxRate = new Rate('openai_ws_pooled_non2xx_rate');
|
|
||||||
const oneToOneNon2xxRate = new Rate('openai_ws_one_to_one_non2xx_rate');
|
|
||||||
|
|
||||||
export const options = {
|
|
||||||
scenarios: {
|
|
||||||
pooled_mode: {
|
|
||||||
executor: 'constant-arrival-rate',
|
|
||||||
exec: 'runPooledMode',
|
|
||||||
rate: pooledRPS,
|
|
||||||
timeUnit: '1s',
|
|
||||||
duration,
|
|
||||||
preAllocatedVUs,
|
|
||||||
maxVUs,
|
|
||||||
tags: { mode: 'pooled' },
|
|
||||||
},
|
|
||||||
one_to_one_mode: {
|
|
||||||
executor: 'constant-arrival-rate',
|
|
||||||
exec: 'runOneToOneMode',
|
|
||||||
rate: oneToOneRPS,
|
|
||||||
timeUnit: '1s',
|
|
||||||
duration,
|
|
||||||
preAllocatedVUs,
|
|
||||||
maxVUs,
|
|
||||||
tags: { mode: 'one_to_one' },
|
|
||||||
startTime: '5s',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
thresholds: {
|
|
||||||
openai_ws_pooled_non2xx_rate: ['rate<0.02'],
|
|
||||||
openai_ws_one_to_one_non2xx_rate: ['rate<0.02'],
|
|
||||||
openai_ws_pooled_duration_ms: ['p(95)<3000', 'p(99)<6000'],
|
|
||||||
openai_ws_one_to_one_duration_ms: ['p(95)<6000', 'p(99)<10000'],
|
|
||||||
},
|
|
||||||
};
|
|
||||||
|
|
||||||
function buildHeaders() {
|
|
||||||
const headers = {
|
|
||||||
'Content-Type': 'application/json',
|
|
||||||
'User-Agent': 'codex_cli_rs/0.98.0',
|
|
||||||
};
|
|
||||||
if (wsAPIKey) {
|
|
||||||
headers.Authorization = `Bearer ${wsAPIKey}`;
|
|
||||||
}
|
|
||||||
return headers;
|
|
||||||
}
|
|
||||||
|
|
||||||
function buildBody() {
|
|
||||||
return JSON.stringify({
|
|
||||||
model,
|
|
||||||
stream: false,
|
|
||||||
input: [
|
|
||||||
{
|
|
||||||
role: 'user',
|
|
||||||
content: [{ type: 'input_text', text: '请回复: pong' }],
|
|
||||||
},
|
|
||||||
],
|
|
||||||
max_output_tokens: 48,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function send(baseURL, mode) {
|
|
||||||
if (!baseURL) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
const res = http.post(`${baseURL}/v1/responses`, buildBody(), {
|
|
||||||
headers: buildHeaders(),
|
|
||||||
timeout,
|
|
||||||
tags: { mode },
|
|
||||||
});
|
|
||||||
check(res, {
|
|
||||||
'status is 2xx': (r) => r.status >= 200 && r.status < 300,
|
|
||||||
});
|
|
||||||
return res;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function runPooledMode() {
|
|
||||||
const res = send(pooledBaseURL, 'pooled');
|
|
||||||
if (!res) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
pooledDurationMs.add(res.timings.duration, { mode: 'pooled' });
|
|
||||||
pooledTTFTMs.add(res.timings.waiting, { mode: 'pooled' });
|
|
||||||
pooledNon2xxRate.add(res.status < 200 || res.status >= 300, { mode: 'pooled' });
|
|
||||||
}
|
|
||||||
|
|
||||||
export function runOneToOneMode() {
|
|
||||||
if (!oneToOneBaseURL) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const res = send(oneToOneBaseURL, 'one_to_one');
|
|
||||||
if (!res) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
oneToOneDurationMs.add(res.timings.duration, { mode: 'one_to_one' });
|
|
||||||
oneToOneTTFTMs.add(res.timings.waiting, { mode: 'one_to_one' });
|
|
||||||
oneToOneNon2xxRate.add(res.status < 200 || res.status >= 300, { mode: 'one_to_one' });
|
|
||||||
}
|
|
||||||
|
|
||||||
export function handleSummary(data) {
|
|
||||||
return {
|
|
||||||
stdout: `\nOpenAI WS 池化 vs 1:1 对比压测完成\n${JSON.stringify(data.metrics, null, 2)}\n`,
|
|
||||||
'docs/perf/openai-ws-pooling-compare-summary.json': JSON.stringify(data, null, 2),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,216 +0,0 @@
|
|||||||
import http from 'k6/http';
|
|
||||||
import { check, sleep } from 'k6';
|
|
||||||
import { Rate, Trend } from 'k6/metrics';
|
|
||||||
|
|
||||||
const baseURL = (__ENV.BASE_URL || 'http://127.0.0.1:5231').replace(/\/$/, '');
|
|
||||||
const wsAPIKey = (__ENV.WS_API_KEY || '').trim();
|
|
||||||
const wsHotspotAPIKey = (__ENV.WS_HOTSPOT_API_KEY || wsAPIKey).trim();
|
|
||||||
const model = __ENV.MODEL || 'gpt-5.3-codex';
|
|
||||||
const duration = __ENV.DURATION || '5m';
|
|
||||||
const timeout = __ENV.TIMEOUT || '180s';
|
|
||||||
|
|
||||||
const shortRPS = Number(__ENV.SHORT_RPS || 12);
|
|
||||||
const longRPS = Number(__ENV.LONG_RPS || 4);
|
|
||||||
const errorRPS = Number(__ENV.ERROR_RPS || 2);
|
|
||||||
const hotspotRPS = Number(__ENV.HOTSPOT_RPS || 10);
|
|
||||||
const preAllocatedVUs = Number(__ENV.PRE_ALLOCATED_VUS || 50);
|
|
||||||
const maxVUs = Number(__ENV.MAX_VUS || 400);
|
|
||||||
|
|
||||||
const reqDurationMs = new Trend('openai_ws_v2_perf_req_duration_ms', true);
|
|
||||||
const ttftMs = new Trend('openai_ws_v2_perf_ttft_ms', true);
|
|
||||||
const non2xxRate = new Rate('openai_ws_v2_perf_non2xx_rate');
|
|
||||||
const doneRate = new Rate('openai_ws_v2_perf_done_rate');
|
|
||||||
const expectedErrorRate = new Rate('openai_ws_v2_perf_expected_error_rate');
|
|
||||||
|
|
||||||
export const options = {
|
|
||||||
scenarios: {
|
|
||||||
short_request: {
|
|
||||||
executor: 'constant-arrival-rate',
|
|
||||||
exec: 'runShortRequest',
|
|
||||||
rate: shortRPS,
|
|
||||||
timeUnit: '1s',
|
|
||||||
duration,
|
|
||||||
preAllocatedVUs,
|
|
||||||
maxVUs,
|
|
||||||
tags: { scenario: 'short_request' },
|
|
||||||
},
|
|
||||||
long_request: {
|
|
||||||
executor: 'constant-arrival-rate',
|
|
||||||
exec: 'runLongRequest',
|
|
||||||
rate: longRPS,
|
|
||||||
timeUnit: '1s',
|
|
||||||
duration,
|
|
||||||
preAllocatedVUs: Math.max(20, Math.ceil(longRPS * 6)),
|
|
||||||
maxVUs: Math.max(100, Math.ceil(longRPS * 20)),
|
|
||||||
tags: { scenario: 'long_request' },
|
|
||||||
},
|
|
||||||
error_injection: {
|
|
||||||
executor: 'constant-arrival-rate',
|
|
||||||
exec: 'runErrorInjection',
|
|
||||||
rate: errorRPS,
|
|
||||||
timeUnit: '1s',
|
|
||||||
duration,
|
|
||||||
preAllocatedVUs: Math.max(8, Math.ceil(errorRPS * 4)),
|
|
||||||
maxVUs: Math.max(40, Math.ceil(errorRPS * 12)),
|
|
||||||
tags: { scenario: 'error_injection' },
|
|
||||||
},
|
|
||||||
hotspot_account: {
|
|
||||||
executor: 'constant-arrival-rate',
|
|
||||||
exec: 'runHotspotAccount',
|
|
||||||
rate: hotspotRPS,
|
|
||||||
timeUnit: '1s',
|
|
||||||
duration,
|
|
||||||
preAllocatedVUs: Math.max(16, Math.ceil(hotspotRPS * 3)),
|
|
||||||
maxVUs: Math.max(80, Math.ceil(hotspotRPS * 10)),
|
|
||||||
tags: { scenario: 'hotspot_account' },
|
|
||||||
},
|
|
||||||
},
|
|
||||||
thresholds: {
|
|
||||||
openai_ws_v2_perf_non2xx_rate: ['rate<0.05'],
|
|
||||||
openai_ws_v2_perf_req_duration_ms: ['p(95)<5000', 'p(99)<9000'],
|
|
||||||
openai_ws_v2_perf_ttft_ms: ['p(99)<2000'],
|
|
||||||
openai_ws_v2_perf_done_rate: ['rate>0.95'],
|
|
||||||
},
|
|
||||||
};
|
|
||||||
|
|
||||||
function buildHeaders(apiKey, opts = {}) {
|
|
||||||
const headers = {
|
|
||||||
'Content-Type': 'application/json',
|
|
||||||
'User-Agent': 'codex_cli_rs/0.104.0',
|
|
||||||
'OpenAI-Beta': 'responses_websockets=2026-02-06,responses=experimental',
|
|
||||||
};
|
|
||||||
if (apiKey) {
|
|
||||||
headers.Authorization = `Bearer ${apiKey}`;
|
|
||||||
}
|
|
||||||
if (opts.sessionID) {
|
|
||||||
headers.session_id = opts.sessionID;
|
|
||||||
}
|
|
||||||
if (opts.conversationID) {
|
|
||||||
headers.conversation_id = opts.conversationID;
|
|
||||||
}
|
|
||||||
return headers;
|
|
||||||
}
|
|
||||||
|
|
||||||
function shortBody() {
|
|
||||||
return JSON.stringify({
|
|
||||||
model,
|
|
||||||
stream: false,
|
|
||||||
input: [
|
|
||||||
{
|
|
||||||
role: 'user',
|
|
||||||
content: [{ type: 'input_text', text: '请回复一个词:pong' }],
|
|
||||||
},
|
|
||||||
],
|
|
||||||
max_output_tokens: 64,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function longBody() {
|
|
||||||
const tools = [];
|
|
||||||
for (let i = 0; i < 28; i += 1) {
|
|
||||||
tools.push({
|
|
||||||
type: 'function',
|
|
||||||
name: `perf_tool_${i}`,
|
|
||||||
description: 'load test tool schema',
|
|
||||||
parameters: {
|
|
||||||
type: 'object',
|
|
||||||
properties: {
|
|
||||||
query: { type: 'string' },
|
|
||||||
limit: { type: 'number' },
|
|
||||||
with_cache: { type: 'boolean' },
|
|
||||||
},
|
|
||||||
required: ['query'],
|
|
||||||
},
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
const input = [];
|
|
||||||
for (let i = 0; i < 20; i += 1) {
|
|
||||||
input.push({
|
|
||||||
role: 'user',
|
|
||||||
content: [{ type: 'input_text', text: `长请求压测消息 ${i}: 请输出简要摘要。` }],
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
return JSON.stringify({
|
|
||||||
model,
|
|
||||||
stream: false,
|
|
||||||
input,
|
|
||||||
tools,
|
|
||||||
parallel_tool_calls: true,
|
|
||||||
max_output_tokens: 256,
|
|
||||||
reasoning: { effort: 'medium' },
|
|
||||||
instructions: '你是压测助手,简洁回复。',
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function errorInjectionBody() {
|
|
||||||
return JSON.stringify({
|
|
||||||
model,
|
|
||||||
stream: false,
|
|
||||||
previous_response_id: `resp_not_found_${__VU}_${__ITER}`,
|
|
||||||
input: [
|
|
||||||
{
|
|
||||||
role: 'user',
|
|
||||||
content: [{ type: 'input_text', text: '触发错误注入路径。' }],
|
|
||||||
},
|
|
||||||
],
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function postResponses(apiKey, body, tags, opts = {}) {
|
|
||||||
const res = http.post(`${baseURL}/v1/responses`, body, {
|
|
||||||
headers: buildHeaders(apiKey, opts),
|
|
||||||
timeout,
|
|
||||||
tags,
|
|
||||||
});
|
|
||||||
reqDurationMs.add(res.timings.duration, tags);
|
|
||||||
ttftMs.add(res.timings.waiting, tags);
|
|
||||||
non2xxRate.add(res.status < 200 || res.status >= 300, tags);
|
|
||||||
return res;
|
|
||||||
}
|
|
||||||
|
|
||||||
function hasDone(res) {
|
|
||||||
return !!res && !!res.body && res.body.indexOf('[DONE]') >= 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function runShortRequest() {
|
|
||||||
const tags = { scenario: 'short_request' };
|
|
||||||
const res = postResponses(wsAPIKey, shortBody(), tags);
|
|
||||||
check(res, { 'short status is 2xx': (r) => r.status >= 200 && r.status < 300 });
|
|
||||||
doneRate.add(hasDone(res) || (res.status >= 200 && res.status < 300), tags);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function runLongRequest() {
|
|
||||||
const tags = { scenario: 'long_request' };
|
|
||||||
const res = postResponses(wsAPIKey, longBody(), tags);
|
|
||||||
check(res, { 'long status is 2xx': (r) => r.status >= 200 && r.status < 300 });
|
|
||||||
doneRate.add(hasDone(res) || (res.status >= 200 && res.status < 300), tags);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function runErrorInjection() {
|
|
||||||
const tags = { scenario: 'error_injection' };
|
|
||||||
const res = postResponses(wsAPIKey, errorInjectionBody(), tags);
|
|
||||||
// 错误注入场景允许 4xx/5xx,重点观测 fallback 和错误路径抖动。
|
|
||||||
expectedErrorRate.add(res.status >= 400, tags);
|
|
||||||
doneRate.add(hasDone(res), tags);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function runHotspotAccount() {
|
|
||||||
const tags = { scenario: 'hotspot_account' };
|
|
||||||
const opts = {
|
|
||||||
sessionID: 'perf-hotspot-session-fixed',
|
|
||||||
conversationID: 'perf-hotspot-conversation-fixed',
|
|
||||||
};
|
|
||||||
const res = postResponses(wsHotspotAPIKey, shortBody(), tags, opts);
|
|
||||||
check(res, { 'hotspot status is 2xx': (r) => r.status >= 200 && r.status < 300 });
|
|
||||||
doneRate.add(hasDone(res) || (res.status >= 200 && res.status < 300), tags);
|
|
||||||
sleep(0.01);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function handleSummary(data) {
|
|
||||||
return {
|
|
||||||
stdout: `\nOpenAI WSv2 性能套件压测完成\n${JSON.stringify(data.metrics, null, 2)}\n`,
|
|
||||||
'docs/perf/openai-ws-v2-perf-suite-summary.json': JSON.stringify(data, null, 2),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,149 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""轻量 secret scanning(CI 门禁 + 本地自检)。
|
|
||||||
|
|
||||||
目标:在不引入额外依赖的情况下,阻止常见敏感凭据误提交。
|
|
||||||
|
|
||||||
注意:
|
|
||||||
- 该脚本只扫描 git tracked files(优先)以避免误扫本地 .env。
|
|
||||||
- 输出仅包含 file:line 与命中类型,不回显完整命中内容(避免二次泄露)。
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Iterable, Sequence
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class Rule:
|
|
||||||
name: str
|
|
||||||
pattern: re.Pattern[str]
|
|
||||||
# allowlist 仅用于减少示例文档/占位符带来的误报
|
|
||||||
allowlist: Sequence[re.Pattern[str]]
|
|
||||||
|
|
||||||
|
|
||||||
RULES: list[Rule] = [
|
|
||||||
Rule(
|
|
||||||
name="google_oauth_client_secret",
|
|
||||||
# Google OAuth client_secret 常见前缀
|
|
||||||
# 真实值通常较长;提高最小长度以避免命中文档里的占位符(例如 GOCSPX-your-client-secret)。
|
|
||||||
pattern=re.compile(r"GOCSPX-[0-9A-Za-z_-]{24,}"),
|
|
||||||
allowlist=(
|
|
||||||
re.compile(r"GOCSPX-your-"),
|
|
||||||
re.compile(r"GOCSPX-REDACTED"),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
Rule(
|
|
||||||
name="google_api_key",
|
|
||||||
# Gemini / Google API Key
|
|
||||||
# 典型格式:AIza + 35 位字符。占位符如 'AIza...' 不会匹配。
|
|
||||||
pattern=re.compile(r"AIza[0-9A-Za-z_-]{35}"),
|
|
||||||
allowlist=(
|
|
||||||
re.compile(r"AIza\.{3}"),
|
|
||||||
re.compile(r"AIza-your-"),
|
|
||||||
re.compile(r"AIza-REDACTED"),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def iter_git_files(repo_root: Path) -> list[Path]:
|
|
||||||
try:
|
|
||||||
out = subprocess.check_output(
|
|
||||||
["git", "ls-files"], cwd=repo_root, stderr=subprocess.DEVNULL, text=True
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
return []
|
|
||||||
files: list[Path] = []
|
|
||||||
for line in out.splitlines():
|
|
||||||
p = (repo_root / line).resolve()
|
|
||||||
if p.is_file():
|
|
||||||
files.append(p)
|
|
||||||
return files
|
|
||||||
|
|
||||||
|
|
||||||
def iter_walk_files(repo_root: Path) -> Iterable[Path]:
|
|
||||||
for dirpath, _dirnames, filenames in os.walk(repo_root):
|
|
||||||
if "/.git/" in dirpath.replace("\\", "/"):
|
|
||||||
continue
|
|
||||||
for name in filenames:
|
|
||||||
yield Path(dirpath) / name
|
|
||||||
|
|
||||||
|
|
||||||
def should_skip(path: Path, repo_root: Path) -> bool:
|
|
||||||
rel = path.relative_to(repo_root).as_posix()
|
|
||||||
# 本地环境文件一般不应入库;若误入库也会被 git ls-files 扫出来。
|
|
||||||
# 这里仍跳过一些明显不该扫描的二进制。
|
|
||||||
if any(rel.endswith(s) for s in (".png", ".jpg", ".jpeg", ".gif", ".pdf", ".zip")):
|
|
||||||
return True
|
|
||||||
if rel.startswith("backend/bin/"):
|
|
||||||
return True
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def scan_file(path: Path, repo_root: Path) -> list[tuple[str, int]]:
|
|
||||||
try:
|
|
||||||
raw = path.read_bytes()
|
|
||||||
except Exception:
|
|
||||||
return []
|
|
||||||
|
|
||||||
# 尝试按 utf-8 解码,失败则当二进制跳过
|
|
||||||
try:
|
|
||||||
text = raw.decode("utf-8")
|
|
||||||
except UnicodeDecodeError:
|
|
||||||
return []
|
|
||||||
|
|
||||||
findings: list[tuple[str, int]] = []
|
|
||||||
lines = text.splitlines()
|
|
||||||
for idx, line in enumerate(lines, start=1):
|
|
||||||
for rule in RULES:
|
|
||||||
if not rule.pattern.search(line):
|
|
||||||
continue
|
|
||||||
if any(allow.search(line) for allow in rule.allowlist):
|
|
||||||
continue
|
|
||||||
rel = path.relative_to(repo_root).as_posix()
|
|
||||||
findings.append((f"{rel}:{idx} ({rule.name})", idx))
|
|
||||||
return findings
|
|
||||||
|
|
||||||
|
|
||||||
def main(argv: Sequence[str]) -> int:
|
|
||||||
parser = argparse.ArgumentParser()
|
|
||||||
parser.add_argument(
|
|
||||||
"--repo-root",
|
|
||||||
default=str(Path(__file__).resolve().parents[1]),
|
|
||||||
help="仓库根目录(默认:脚本上两级目录)",
|
|
||||||
)
|
|
||||||
args = parser.parse_args(argv)
|
|
||||||
|
|
||||||
repo_root = Path(args.repo_root).resolve()
|
|
||||||
files = iter_git_files(repo_root)
|
|
||||||
if not files:
|
|
||||||
files = list(iter_walk_files(repo_root))
|
|
||||||
|
|
||||||
problems: list[str] = []
|
|
||||||
for f in files:
|
|
||||||
if should_skip(f, repo_root):
|
|
||||||
continue
|
|
||||||
for msg, _line in scan_file(f, repo_root):
|
|
||||||
problems.append(msg)
|
|
||||||
|
|
||||||
if problems:
|
|
||||||
sys.stderr.write("Secret scan FAILED. Potential secrets detected:\n")
|
|
||||||
for p in problems:
|
|
||||||
sys.stderr.write(f"- {p}\n")
|
|
||||||
sys.stderr.write("\n请移除/改为环境变量注入,或使用明确的占位符(例如 GOCSPX-your-client-secret)。\n")
|
|
||||||
return 1
|
|
||||||
|
|
||||||
print("Secret scan OK")
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main(sys.argv[1:]))
|
|
||||||
|
|
||||||
192
tools/sora-test
192
tools/sora-test
@@ -1,192 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""
|
|
||||||
Sora access token tester.
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
tools/sora-test -at "<ACCESS_TOKEN>"
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import base64
|
|
||||||
import json
|
|
||||||
import sys
|
|
||||||
import textwrap
|
|
||||||
import urllib.error
|
|
||||||
import urllib.request
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from typing import Dict, Optional, Tuple
|
|
||||||
|
|
||||||
|
|
||||||
DEFAULT_BASE_URL = "https://sora.chatgpt.com"
|
|
||||||
DEFAULT_TIMEOUT = 20
|
|
||||||
DEFAULT_USER_AGENT = "Sora/1.2026.007 (Android 15; 24122RKC7C; build 2600700)"
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class EndpointResult:
|
|
||||||
path: str
|
|
||||||
status: int
|
|
||||||
request_id: str
|
|
||||||
cf_ray: str
|
|
||||||
body_preview: str
|
|
||||||
|
|
||||||
|
|
||||||
def parse_args() -> argparse.Namespace:
|
|
||||||
parser = argparse.ArgumentParser(
|
|
||||||
description="Test Sora access token against core backend endpoints.",
|
|
||||||
formatter_class=argparse.RawTextHelpFormatter,
|
|
||||||
epilog=textwrap.dedent(
|
|
||||||
"""\
|
|
||||||
Examples:
|
|
||||||
tools/sora-test -at "eyJhbGciOi..."
|
|
||||||
tools/sora-test -at "eyJhbGciOi..." --timeout 30
|
|
||||||
"""
|
|
||||||
),
|
|
||||||
)
|
|
||||||
parser.add_argument("-at", "--access-token", required=True, help="Sora/OpenAI access token (JWT)")
|
|
||||||
parser.add_argument(
|
|
||||||
"--base-url",
|
|
||||||
default=DEFAULT_BASE_URL,
|
|
||||||
help=f"Base URL for Sora backend (default: {DEFAULT_BASE_URL})",
|
|
||||||
)
|
|
||||||
parser.add_argument(
|
|
||||||
"--timeout",
|
|
||||||
type=int,
|
|
||||||
default=DEFAULT_TIMEOUT,
|
|
||||||
help=f"HTTP timeout seconds (default: {DEFAULT_TIMEOUT})",
|
|
||||||
)
|
|
||||||
return parser.parse_args()
|
|
||||||
|
|
||||||
|
|
||||||
def mask_token(token: str) -> str:
|
|
||||||
if len(token) <= 16:
|
|
||||||
return token
|
|
||||||
return f"{token[:10]}...{token[-6:]}"
|
|
||||||
|
|
||||||
|
|
||||||
def decode_jwt_payload(token: str) -> Optional[Dict]:
|
|
||||||
parts = token.split(".")
|
|
||||||
if len(parts) != 3:
|
|
||||||
return None
|
|
||||||
payload = parts[1]
|
|
||||||
payload += "=" * ((4 - len(payload) % 4) % 4)
|
|
||||||
payload = payload.replace("-", "+").replace("_", "/")
|
|
||||||
try:
|
|
||||||
decoded = base64.b64decode(payload)
|
|
||||||
return json.loads(decoded.decode("utf-8", errors="replace"))
|
|
||||||
except Exception:
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def ts_to_iso(ts: Optional[int]) -> str:
|
|
||||||
if not ts:
|
|
||||||
return "-"
|
|
||||||
try:
|
|
||||||
return datetime.fromtimestamp(ts, tz=timezone.utc).isoformat()
|
|
||||||
except Exception:
|
|
||||||
return "-"
|
|
||||||
|
|
||||||
|
|
||||||
def http_get(base_url: str, path: str, access_token: str, timeout: int) -> EndpointResult:
|
|
||||||
url = base_url.rstrip("/") + path
|
|
||||||
req = urllib.request.Request(url=url, method="GET")
|
|
||||||
req.add_header("Authorization", f"Bearer {access_token}")
|
|
||||||
req.add_header("Accept", "application/json, text/plain, */*")
|
|
||||||
req.add_header("Origin", DEFAULT_BASE_URL)
|
|
||||||
req.add_header("Referer", DEFAULT_BASE_URL + "/")
|
|
||||||
req.add_header("User-Agent", DEFAULT_USER_AGENT)
|
|
||||||
|
|
||||||
try:
|
|
||||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
|
||||||
raw = resp.read()
|
|
||||||
body = raw.decode("utf-8", errors="replace")
|
|
||||||
return EndpointResult(
|
|
||||||
path=path,
|
|
||||||
status=resp.getcode(),
|
|
||||||
request_id=(resp.headers.get("x-request-id") or "").strip(),
|
|
||||||
cf_ray=(resp.headers.get("cf-ray") or "").strip(),
|
|
||||||
body_preview=body[:500].replace("\n", " "),
|
|
||||||
)
|
|
||||||
except urllib.error.HTTPError as e:
|
|
||||||
raw = e.read()
|
|
||||||
body = raw.decode("utf-8", errors="replace")
|
|
||||||
return EndpointResult(
|
|
||||||
path=path,
|
|
||||||
status=e.code,
|
|
||||||
request_id=(e.headers.get("x-request-id") if e.headers else "") or "",
|
|
||||||
cf_ray=(e.headers.get("cf-ray") if e.headers else "") or "",
|
|
||||||
body_preview=body[:500].replace("\n", " "),
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
return EndpointResult(
|
|
||||||
path=path,
|
|
||||||
status=0,
|
|
||||||
request_id="",
|
|
||||||
cf_ray="",
|
|
||||||
body_preview=f"network_error: {e}",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def classify(me_status: int) -> Tuple[str, int]:
|
|
||||||
if me_status == 200:
|
|
||||||
return "AT looks valid for Sora (/backend/me == 200).", 0
|
|
||||||
if me_status == 401:
|
|
||||||
return "AT is invalid or expired (/backend/me == 401).", 2
|
|
||||||
if me_status == 403:
|
|
||||||
return "AT may be blocked by policy/challenge or lacks permission (/backend/me == 403).", 3
|
|
||||||
if me_status == 0:
|
|
||||||
return "Request failed before reaching Sora (network/proxy/TLS issue).", 4
|
|
||||||
return f"Unexpected status on /backend/me: {me_status}", 5
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
args = parse_args()
|
|
||||||
token = args.access_token.strip()
|
|
||||||
if not token:
|
|
||||||
print("ERROR: empty access token")
|
|
||||||
return 1
|
|
||||||
|
|
||||||
payload = decode_jwt_payload(token)
|
|
||||||
print("=== Sora AT Test ===")
|
|
||||||
print(f"token: {mask_token(token)}")
|
|
||||||
if payload:
|
|
||||||
exp = payload.get("exp")
|
|
||||||
iat = payload.get("iat")
|
|
||||||
scopes = payload.get("scp")
|
|
||||||
scope_count = len(scopes) if isinstance(scopes, list) else 0
|
|
||||||
print(f"jwt.iat: {iat} ({ts_to_iso(iat)})")
|
|
||||||
print(f"jwt.exp: {exp} ({ts_to_iso(exp)})")
|
|
||||||
print(f"jwt.scope_count: {scope_count}")
|
|
||||||
else:
|
|
||||||
print("jwt: payload decode failed (token may not be JWT)")
|
|
||||||
|
|
||||||
endpoints = [
|
|
||||||
"/backend/me",
|
|
||||||
"/backend/nf/check",
|
|
||||||
"/backend/project_y/invite/mine",
|
|
||||||
"/backend/billing/subscriptions",
|
|
||||||
]
|
|
||||||
|
|
||||||
print("\n--- endpoint checks ---")
|
|
||||||
results = []
|
|
||||||
for path in endpoints:
|
|
||||||
res = http_get(args.base_url, path, token, args.timeout)
|
|
||||||
results.append(res)
|
|
||||||
print(f"{res.path} -> status={res.status} request_id={res.request_id or '-'} cf_ray={res.cf_ray or '-'}")
|
|
||||||
if res.body_preview:
|
|
||||||
print(f" body: {res.body_preview}")
|
|
||||||
|
|
||||||
me_result = next((r for r in results if r.path == "/backend/me"), None)
|
|
||||||
me_status = me_result.status if me_result else 0
|
|
||||||
summary, code = classify(me_status)
|
|
||||||
print("\n--- summary ---")
|
|
||||||
print(summary)
|
|
||||||
return code
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main())
|
|
||||||
|
|
||||||
Reference in New Issue
Block a user