Commit Graph

1000 Commits

Author SHA1 Message Date
yangjianbo
2bfb16291f fix(unit): 修复 unit tag 测试编译与账号选择用例 2026-02-09 21:35:41 +08:00
yangjianbo
d367d1cde6 Merge branch 'main' into test-sora 2026-02-09 20:40:09 +08:00
yangjianbo
16131c3d3f Merge branch 'main' of https://github.com/mt21625457/aicodex2api 2026-02-09 20:26:03 +08:00
QTom
04cedce9a1 test: 为 stubAccountRepo 添加 ListCRSAccountIDs 方法实现 2026-02-09 11:40:37 +08:00
QTom
5e0d789440 feat(admin): 新增 CRS 同步预览和账号选择功能
- 后端新增 PreviewFromCRS 接口,允许用户先预览 CRS 中的账号
- 后端支持在同步时选择特定账号,不选中的账号将被跳过
- 前端重构 SyncFromCrsModal 为三步向导:输入凭据 → 预览账号 → 执行同步
- 改进表单无障碍性:添加 for/id 关联和 required 属性
- 修复 Back 按钮返回时的状态清理
- 新增 buildSelectedSet 和 shouldCreateAccount 的单元测试
- 完整的向后兼容性:旧客户端不发送 selected_account_ids 时行为不变
2026-02-09 10:39:09 +08:00
yangjianbo
d7011163b8 fix: 修复代码审核发现的安全和质量问题
安全修复(P0):
- 移除硬编码的 OAuth client_secret(Antigravity、Gemini CLI),
  改为通过环境变量注入(ANTIGRAVITY_OAUTH_CLIENT_SECRET、
  GEMINI_CLI_OAUTH_CLIENT_SECRET)
- 新增 logredact.RedactText() 对非结构化文本做敏感信息脱敏,
  覆盖 GOCSPX-*/AIza* 令牌和常见 key=value 模式
- 日志中不再打印 org_uuid、account_uuid、email_address 等敏感值

安全修复(P1):
- URL 验证增强:新增 ValidateHTTPURL 统一入口,支持 allowlist 和
  私网地址阻断(localhost/内网 IP)
- 代理回退安全:代理初始化失败时默认阻止直连回退,防止 IP 泄露,
  可通过 security.proxy_fallback.allow_direct_on_error 显式开启
- Gemini OAuth 配置校验:client_id 与 client_secret 必须同时
  设置或同时留空

其他改进:
- 新增 tools/secret_scan.py 密钥扫描工具和 Makefile secret-scan 目标
- 更新所有 docker-compose 和部署配置,传递 OAuth secret 环境变量
- google_one OAuth 类型使用固定 redirectURI,与 code_assist 对齐

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 09:58:13 +08:00
yangjianbo
fc8a39e0f5 test: 删除CI工作流,大幅提升后端单元测试覆盖率至50%+
删除因GitHub计费锁定而失败的CI工作流。
为6个核心Go源文件补充单元测试,全部达到50%以上覆盖率:
- response/response.go: 97.6%
- antigravity/oauth.go: 90.1%
- antigravity/client.go: 88.6% (新增27个HTTP客户端测试)
- geminicli/oauth.go: 91.8%
- service/oauth_service.go: 61.2%
- service/gemini_oauth_service.go: 51.9%

新增/增强8个测试文件,共计5600+行测试代码。

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 09:07:58 +08:00
erio
9a479d1b55 fix: resolve CI failures from scope removal refactor
- Fix gofmt alignment in ops_realtime_models.go
- Remove SetAntigravityQuotaScopeLimit mock from api_contract_test.go
- Add UpdateSortOrders mock to mockGroupRepoForGateway
2026-02-09 08:27:14 +08:00
erio
fc095bf054 refactor: replace scope-level rate limiting with model-level rate limiting
Merge functional changes from develop branch:
- Remove AntigravityQuotaScope system (claude/gemini_text/gemini_image)
- Replace with per-model rate limiting using resolveAntigravityModelKey
- Remove model load statistics (IncrModelCallCount/GetModelLoadBatch)
- Simplify account selection to unified priority→load→LRU algorithm
- Remove SetAntigravityQuotaScopeLimit from AccountRepository
- Clean up scope-related UI indicators and API fields
2026-02-09 08:19:01 +08:00
erio
1af06aed96 feat: shuffle accounts within same sort group to prevent thundering herd
Add post-sort shuffle for accounts with identical (priority, loadRate,
lastUsedAt) to break deterministic ordering when concurrent requests
read the same scheduler snapshot. Applies to both Antigravity and
OpenAI scheduling paths, plus the sortAccountsByPriorityAndLastUsed
helper.

Keeps upstream CallCount/ModelLoadInfo scheduling intact; shuffle is
additive and only randomises within equivalent-rank groups.
2026-02-09 07:33:17 +08:00
erio
9236936a55 feat: route AccountTypeUpstream to ForwardUpstream in Forward() entry
Without this routing guard, ForwardUpstream is never called because
Forward() always proceeds with the standard OAuth/cookie flow.
2026-02-09 07:27:10 +08:00
erio
125152460f fix: use upstream retryDelay for rate limit duration instead of fixed default
- In handleSmartRetry, use the actual upstream retryDelay to set model
  rate limit duration instead of always using the 30s default
- Return info.RetryDelay from shouldTriggerAntigravitySmartRetry when
  shouldRateLimitModel=true, so callers know the actual delay
- Extract getDefaultRateLimitDuration() and resolveResetTime() helpers
  to reduce duplication in handleUpstreamError 429 handling
- Improve debug logging with upstream_retry_delay and response body
2026-02-09 07:11:29 +08:00
erio
6d90fb0bc3 feat: detect client disconnect during streaming and continue draining upstream for billing 2026-02-09 07:06:26 +08:00
erio
b889d5017b refactor: replace Trie-based digest session store with flat cache 2026-02-09 07:02:12 +08:00
erio
72b08f9cc5 fix: ensure sticky session failover triggers cache billing exemption 2026-02-09 06:57:07 +08:00
erio
681950dadd feat: add linear delay between Antigravity account failover switches 2026-02-09 06:56:29 +08:00
erio
a67d9337b8 feat: integrate CheckErrorPolicy into Gemini error handling paths 2026-02-09 06:55:45 +08:00
erio
2f1182e8a9 feat: unified error policy for Antigravity + enable custom error codes for Gemini accounts 2026-02-09 06:54:42 +08:00
erio
cbb4d854ab fix: check type assertion in test to satisfy errcheck linter 2026-02-09 06:47:50 +08:00
erio
35598d5648 fix: parse Gemini native request format in ParseGatewayRequest for correct session hash generation
ParseGatewayRequest only parsed Anthropic format (system/messages),
ignoring Gemini native format (systemInstruction/contents). This caused
GenerateSessionHash to produce identical hashes for all Gemini sessions.

Add protocol parameter to ParseGatewayRequest to branch between
Anthropic and Gemini parsing. Update GenerateSessionHash message
traversal to extract text from both formats.
2026-02-09 06:47:22 +08:00
erio
5c76b9e45a fix: prevent sessionHash collision for different users with same messages
Mix SessionContext (ClientIP, UserAgent, APIKeyID) into
GenerateSessionHash 3rd-level fallback to differentiate requests
from different users sending identical content.

Also switch hashContent from SHA256-truncated to XXHash64 for
better performance, and optimize Trie Lua script to match from
longest prefix first.
2026-02-09 06:46:32 +08:00
erio
0b8fea4cb4 fix: clean thoughtSignature for all clients, not just CLI
Previously, thoughtSignature cleanup only applied to Gemini CLI
requests (detected via x-gemini-api-privileged-user-id header or
tmp dir pattern). This caused 400 errors for non-CLI clients when
session cache expired and they sent stale signatures.

Remove the isGeminiCLIRequest guard so all clients benefit from
proactive thoughtSignature cleanup on session binding miss.
2026-02-09 06:45:01 +08:00
Wesley Liddick
5fa93ebdc7 Merge pull request #519 from bayma888/feature/group-sort-order
feat(admin): 新增-分组管理自由拖拽排序功能
2026-02-08 18:00:22 +08:00
bayma888
bac9e2bfd5 feat(admin): add drag-and-drop group sort order
- Add `sort_order` field to groups table with migration
- Add `PUT /api/v1/admin/groups/sort-order` API for batch update
- Implement drag-and-drop UI using vue-draggable-plus
- All queries now order groups by sort_order
- Add i18n support (en/zh) for sort-related UI text
- Update test stubs to satisfy new interface methods
2026-02-08 16:53:45 +08:00
shaw
e4d74ae11d feat(ui): 用户列表页显示当前并发数
优化 /admin/users 页面的并发数列,显示「当前/最大」格式,
参考 AccountCapacityCell 的设计风格。

- 后端 UserHandler 注入 ConcurrencyService,批量查询用户当前并发数
- 新增 UserConcurrencyCell 组件,支持颜色状态(空闲灰/使用中黄/满载红)
- 前端 AdminUser 类型添加 current_concurrency 字段
2026-02-08 16:44:51 +08:00
Wesley Liddick
2185a3b674 Merge pull request #517 from touwaeriol/fix/upstream-baseurl
refactor(upstream): replace upstream account type with apikey + auto-append base_url
2026-02-08 14:03:12 +08:00
Wesley Liddick
9e3c306a5b Merge pull request #513 from touwaeriol/pr/antigravity-full-v2
feat(antigravity): comprehensive enhancements — rate limiting, scheduling & smart retry
2026-02-08 14:01:17 +08:00
erio
69816f8691 fix: remove unused upstreamHopByHopHeaders variable to pass golangci-lint 2026-02-08 13:30:39 +08:00
shaw
b4ec65785d fix: apikey类型账号test去掉oauth-2025-04-20 2026-02-08 13:26:28 +08:00
erio
fb58560d15 refactor(upstream): replace upstream account type with apikey, auto-append /antigravity
Upstream accounts now use the standard APIKey type instead of a dedicated
upstream type. GetBaseURL() and new GetGeminiBaseURL() automatically append
/antigravity for Antigravity platform APIKey accounts, eliminating the need
for separate upstream forwarding methods.

- Remove ForwardUpstream, ForwardUpstreamGemini, testUpstreamConnection
- Remove upstream branch guards in Forward/ForwardGemini/TestConnection
- Add migration 052 to convert existing upstream accounts to apikey
- Update frontend CreateAccountModal to create apikey type
- Add unit tests for GetBaseURL and GetGeminiBaseURL
2026-02-08 13:06:25 +08:00
yangjianbo
bb5a5dd65e test: 完善自动化测试体系(7个模块,73个任务)
系统性地修复、补充和强化项目的自动化测试能力:

1. 测试基础设施修复
   - 修复 stubConcurrencyCache 缺失方法和构造函数参数不匹配
   - 创建 testutil 共享包(stubs.go, fixtures.go, httptest.go)
   - 为所有 Stub 添加编译期接口断言

2. 中间件测试补充
   - 新增 JWT 认证中间件测试(有效/过期/篡改/缺失 Token)
   - 补充 rate_limiter 和 recovery 中间件测试场景

3. 网关核心路径测试
   - 新增账户选择、等待队列、流式响应、并发控制、计费、Claude Code 检测测试
   - 覆盖负载均衡、粘性会话、SSE 转发、槽位管理等关键逻辑

4. 前端测试体系(11个新测试文件,163个测试用例)
   - Pinia stores: auth, app, subscriptions
   - API client: 请求拦截器、响应拦截器、401 刷新
   - Router guards: 认证重定向、管理员权限、简易模式限制
   - Composables: useForm, useTableLoader, useClipboard
   - Components: LoginForm, ApiKeyCreate, Dashboard

5. CI/CD 流水线重构
   - 重构 backend-ci.yml 为统一的 ci.yml
   - 前后端 4 个并行 Job + Postgres/Redis services
   - Race 检测、覆盖率收集与门禁、Docker 构建验证

6. E2E 自动化测试
   - e2e-test.sh 自动化脚本(Docker 启动→健康检查→测试→清理)
   - 用户注册→登录→API Key→网关调用完整链路测试
   - Mock 模式和 API Key 脱敏支持

7. 修复预存问题
   - tlsfingerprint dialer_test.go 缺失 build tag 导致集成测试编译冲突

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 12:05:39 +08:00
erio
6ab77f5eb5 fix(upstream): passthrough response body directly instead of parsing SSE
ForwardUpstream/ForwardUpstreamGemini should pipe the upstream response
directly to the client (headers + body), not parse it as SSE stream.
2026-02-08 08:49:43 +08:00
erio
4f57d7f761 fix: add nil guard for gin.Context in header passthrough to satisfy staticcheck SA5011 2026-02-08 08:36:35 +08:00
erio
1563bd3dda feat(upstream): passthrough all client headers instead of manual header setting
Replace manual header setting (Content-Type, anthropic-version, anthropic-beta)
with full client header passthrough in ForwardUpstream/ForwardUpstreamGemini.
Only authentication headers (Authorization, x-api-key) are overridden with
upstream account credentials. Hop-by-hop headers are excluded.

Add unit tests covering header passthrough, auth override, and hop-by-hop filtering.
2026-02-08 08:33:09 +08:00
erio
77b66653ed fix(gateway): restore upstream account forwarding with dedicated methods
v0.1.74 merged upstream accounts into the OAuth path, causing requests
to hit the wrong protocol and endpoint. Add three upstream-specific
methods (testUpstreamConnection, ForwardUpstream, ForwardUpstreamGemini)
that use base_url + apiKey auth and passthrough the original body, while
reusing the existing response handling and error/retry logic.
2026-02-08 01:21:02 +08:00
yangjianbo
53e1c8b268 perf(日志): 降噪优化,将常规成功日志降级为 Debug 级别
- GIN Logger 中间件跳过 /health 和 /setup/status 的请求日志
- UsageCleanup 空闲轮询(no_task)日志降级为 slog.Debug
- Scheduler 常规 rebuild ok 日志降级为 slog.Debug
- DashboardAggregation 常规聚合完成日志降级为 slog.Debug
- TokenRefresh 无刷新活动时周期日志降级为 slog.Debug

生产环境(Info 级别)下自动静默,debug 模式下仍可见。
错误、警告类日志保持原有级别不变。

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 23:29:24 +08:00
yangjianbo
00caf0bcd8 test: 为代码审核修复添加详细单元测试(7个测试文件,50+测试用例)
新增测试文件:
- cors_test.go: CORS 条件化头部测试(12个测试,覆盖白名单/黑名单/通配符/凭证/多源/Vary)
- gateway_helper_backoff_test.go: nextBackoff 退避测试(6个测试+基准,验证指数增长/边界/抖动/收敛)
- billing_cache_jitter_test.go: jitteredTTL 抖动测试(5个测试+基准,验证范围/上界/方差/均值)
- subscription_calculate_progress_test.go: calculateProgress 纯函数测试(9个测试,覆盖日/周/月限额/超限截断/过期)
- openai_gateway_handler_test.go: SSE JSON 转义测试(7个子用例,验证双引号/反斜杠/换行符安全)

更新测试文件:
- response_transformer_test.go: 增强 generateRandomID 测试(7个测试,含并发/字符集/降级计数器)
- security_headers_test.go: 适配 GenerateNonce 新签名
- api_key_auth_test.go: 适配 NewSubscriptionService 新参数

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:14:07 +08:00
yangjianbo
9634494ba9 fix: 修复代码审核发现的10个问题(P0安全+P1数据一致性+P2性能优化)
P0: OpenAI SSE 错误消息 JSON 注入 — 使用 json.Marshal 替代 fmt.Sprintf
P1: subscription 续期包裹 Ent 事务确保原子性
P1: CSP nonce 生成处理 crypto/rand 错误,失败降级为 unsafe-inline
P1: singleflight 透传数据库真实错误,不再吞没为 not found
P1: GetUserSubscriptionsWithProgress 提取 calculateProgress 消除 N+1
P2: billing_cache/gateway_helper 迁移到 math/rand/v2 消除全局锁争用
P2: generateRandomID 降级分支增加原子计数器防碰撞
P2: CORS 非白名单 origin 不再设置 Allow-Headers/Methods/Max-Age
P2: Turnstile 验证移除 VerifyCode 空值跳过条件防绕过
P2: Redis Cluster Lua 脚本空 KEYS 添加兼容性警告注释

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:13:45 +08:00
yangjianbo
4a20a2a8ba fix: 修复批量更新凭证明细与缓存TTL抖动
- BatchUpdateCredentials 返回 success/failed/results 及 success_ids/failed_ids

- billing jitteredTTL 改为只减不增,确保TTL不超上界

- crypto/rand 失败时随机ID降级避免 panic

- OpenAI SelectAccount 失败日志去重并补充字段

- 修复两处类型断言以通过 errcheck
2026-02-07 21:18:03 +08:00
yangjianbo
fd43be8d0b merge: 合并 main 分支到 test,解决 config 和 modelWhitelist 冲突
- config.go: 保留 Sora 配置,合入 SubscriptionCache 配置
- useModelWhitelist.ts: 同时保留 soraModels 和 antigravityModels

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 20:18:07 +08:00
yangjianbo
836ba14b70 fix: 修复函数签名变更后的调用参数不匹配
- handleUpstreamError 补齐新增的三个参数 (0, "", false)
- handleStreamingResponse 移除已删除的 nil 参数

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 20:05:29 +08:00
yangjianbo
a14dfb769a Merge branch 'dev-release' 2026-02-07 19:58:00 +08:00
yangjianbo
2588fa6a8f fix(audit): 第二批审计修复 — P0 生产 Bug、安全加固、性能优化、缓存一致性、代码质量
基于 backend-code-audit 审计报告,修复剩余 P0/P1/P2 共 34 项问题:

P0 生产 Bug:
- 修复 time.Since(time.Now()) 计时逻辑错误 (P0-03)
- generateRandomID 改用 crypto/rand 替代固定索引 (P0-04)
- IncrementQuotaUsed 重写为 Ent 原子操作消除 TOCTOU 竞态 (P0-05)

安全加固:
- gateway/openai handler 错误响应替换为泛化消息,防止内部信息泄露 (P1-14)
- usage_log_repo dateFormat 参数改用白名单映射,防止 SQL 注入 (P1-16)
- 默认配置安全加固:sslmode=prefer、response_headers=true、mode=release (P1-18/19, P2-15)

性能优化:
- gateway handler 循环内 defer 替换为显式 releaseWait 闭包 (P1-02)
- group_repo/promo_code_repo Count 前 Clone 查询避免状态污染 (P1-03)
- usage_log_repo 四个查询添加 LIMIT 10000 防止 OOM (P1-07)
- GetBatchUsageStats 添加时间范围参数,默认最近 30 天 (P1-10)
- ip.go CIDR 预编译为包级变量 (P1-11)
- BatchUpdateCredentials 重构为先验证后更新 (P1-13)

缓存一致性:
- billing_cache 添加 jitteredTTL 防止缓存雪崩 (P2-10)
- DeductUserBalance/UpdateSubscriptionUsage 错误传播修复 (P2-12)
- UserService.UpdateBalance 成功后异步失效 billingCache (P2-13)

代码质量:
- search 截断改为按 rune 处理,支持多字节字符 (P2-01)
- TLS Handshake 改为 HandshakeContext 支持 context 取消 (P2-07)
- CORS 预检添加 Access-Control-Max-Age: 86400 (P2-16)

测试覆盖:
- 新增 user_service_test.go(UpdateBalance 缓存失效 6 个用例)
- 新增 batch_update_credentials_test.go(fail-fast + 类型验证 7 个用例)
- 新增 response_transformer_test.go、ip_test.go、usage_log_repo_unit_test.go、search_truncate_test.go
- 集成测试:IncrementQuotaUsed 并发测试、billing_cache 错误传播测试
- config_test.go 补充 server.mode/sslmode 默认值断言

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 19:46:42 +08:00
erio
3077fd279d feat: smart retry max 1 attempt + clear sticky session on failure
- Change antigravitySmartRetryMaxAttempts from 3 to 1 to prevent
  repeated rate limiting and long waits
- Clear sticky session binding (DeleteSessionAccountID) after smart
  retry exhaustion, so subsequent requests don't hit the same
  rate-limited account
- Add flow diagrams to Forward/ForwardGemini doc comments
- Add comprehensive unit tests covering:
  - Sticky session cleared on retry failure (429, 503, network error)
  - Sticky session NOT cleared on retry success
  - Sticky session NOT cleared for non-sticky requests (empty hash)
  - Sticky session NOT cleared on long delay path (handled by handler)
  - Nil cache safety (no panic)
  - MaxAttempts constant verification
  - End-to-end retryLoop → switchError propagation with session clear
2026-02-07 19:30:58 +08:00
shaw
6aaa4aee6a fix: 收敛 Claude Code 探测拦截并补齐回归测试 2026-02-07 19:04:08 +08:00
erio
e3748da860 fix(lint): handle errcheck for strings.Builder.WriteString 2026-02-07 18:18:15 +08:00
erio
86b503f87f refactor: remove Anthropic digest chain from Messages handler
The digest chain fallback is only needed for Gemini endpoints, not
for the Anthropic Messages API path. Remove the handler integration
while keeping the reusable service/repository layer for future use.
2026-02-07 18:01:04 +08:00
erio
50a783ff01 feat: add Anthropic sticky session digest chain matching via Trie
The previous fallback (step 3) in GenerateSessionHash hashed system +
all messages together, producing a different hash each round as the
conversation grew ([a] -> [a,b] -> [a,b,c]). This made fallback sticky
sessions ineffective for multi-turn conversations.

Implement per-message Trie digest chain matching (reusing Gemini's Trie
infrastructure) so that the previous round's chain is always a prefix
of the current round's chain, enabling reliable session affinity.
2026-02-07 18:00:56 +08:00
yangjianbo
f6ca701917 fix(oauth): SessionStore.Stop() 添加 sync.Once 防重入保护 (P1-05)
oauth 和 openai 包的 SessionStore.Stop() 直接调用 close(stopCh),
重复调用会导致 panic。使用 sync.Once 包裹确保幂等安全。

新增单元测试覆盖连续调用和 50 goroutine 并发调用场景。

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 17:39:18 +08:00
yangjianbo
a84604dceb fix(config): 禁止 server.frontend_url 携带 query/userinfo 2026-02-07 17:37:08 +08:00