Commit Graph

344 Commits

Author SHA1 Message Date
shaw
a789c8c4c7 feat: 支持opus-4.7 2026-04-17 09:37:25 +08:00
erio
db27e8f000 feat(usage): add account cost to breakdown sub-table and admin usage log
- UserBreakdownItem: add AccountCost field + SQL aggregation
- UserBreakdownSubTable: add orange account cost column
- Admin usage table: add account_cost column (after cost, default visible)
- Column settings: add account_cost toggle option
2026-04-15 15:40:40 +08:00
erio
22680dc602 test(usage): add unit tests for account_cost and fix gofmt
- Fix mock for GetModelStatsWithFilters: add account_cost column
- Add assertion: GetStatsWithFilters always returns TotalAccountCost
- New test: GetModelStatsAccountCostColumn verifies scan of AccountCost
- New test: GetGroupStatsAccountCostColumn verifies scan of AccountCost
- New test: GetStatsWithFiltersAlwaysReturnsAccountCost (no AccountID filter)
- Integration test: add TotalAccountCost/TodayAccountCost assertions
- Fix gofmt alignment in usage_log_types.go
2026-04-15 15:02:21 +08:00
erio
6ade6d30a8 feat(usage): add account cost display to admin dashboard and usage pages
- Add account_cost column to dashboard aggregation tables (migration 107)
- DashboardStats: add TotalAccountCost/TodayAccountCost fields
- ModelStat/GroupStat: add AccountCost field with SQL aggregation
- GetStatsWithFilters: always return TotalAccountCost (remove accountID filter)
- Dashboard Token cards: show user(green)/cost(orange)/standard(gray)
- Usage stats card: show account cost and standard below main value
- Model/Group distribution tables: add orange cost column
2026-04-15 15:02:21 +08:00
erio
6ac8ccde46 fix: merge 30 general improvements from release branch
Bug fixes:
- Detached context for GetAccountConcurrencyBatch (prevent all-zero on request cancel)
- Filter soft-deleted users in GetByGroupID
- Stripe CSP policy (allow Stripe.js in script-src and frame-src)
- WebSearch API key validation on save
- RECHARGING status in payment result success check
- Windows test fixes (logger Sync deadlock, config path escaping)

Feature enhancements:
- Webhook multi-instance dispatch (extractOutTradeNo + GetWebhookProvider)
- EasyPay mobile H5 payment (device param + PayURL2)
- SSE error propagation in WebSearch emulation
- AccountStatsCost DTO field for admin usage logs
- Plans sort by sort_order instead of created_at
- UsageMapHook for streaming response usage data
- apicompat Instructions field passthrough
- EffectiveLoadFactor for ops concurrency/metrics
- Usage billing RETURNING balance for notify system
- BulkUpdate mixed channel warning with details
- println to slog migration in auth cache
- Wire ProviderSet cleanup
- CI cache-dependency-path optimization

Frontend:
- Refund eligibility check per provider (canRequestRefund)
- Plan sort_order editing
- Dead code cleanup (simulate_claude_max, client_affinity)
- GroupsView platform switch guard
- channels features_config API type
- UsageView account_stats_cost export
2026-04-14 17:35:27 +08:00
erio
e8ee400a3f fix: resolve remaining lint errors for upstream CI
- Fix errcheck: brave.go resp.Body.Close, manager_test.go Encode
- Fix gofmt: payment_config_service.go
- Fix unused: use shouldFallbackGeminiModel (with modelName param) in handler
2026-04-14 12:19:44 +08:00
erio
6a08efeef9 fix: resolve upstream CI failures (lint, test, gofmt)
- Fix errcheck: handle Write/Encode return values in brave_test.go
- Fix errcheck: defer resp.Body.Close() with _ assignment in tavily.go
- Fix gofmt: payment.go, channel.go, payment_config_providers.go
- Fix unused: remove dead decodeURLValue in easypay.go
- Restore shouldFallbackGeminiModel function (deleted during cherry-pick)
- Add missing balanceNotifyService param to NewGatewayService in test
- Fix platform default test expectation (empty stays empty)
- Fix wildcard pricing test (longest prefix wins, not config order)
- Fix subscription group test (SUBSCRIPTION_REPOSITORY_UNAVAILABLE)
2026-04-14 12:11:08 +08:00
erio
9028d2085f test: add unit tests for billing, websearch, and notify systems
Billing (25 tests):
- CalculateCostUnified: nil resolver fallback, token/per_request/image modes
- GetModelPricingWithChannel: nil/partial/full channel overrides
- resolveAccountStatsCost: four-level priority chain integration tests

WebSearch (18 tests):
- PopulateWebSearchUsage: nil input, manager states, QuotaLimit nil/*int64
- ResetWebSearchUsage: nil manager error
- Manager.ResetUsage: nil Redis
- shouldEmulateWebSearch: full decision chain (8 scenarios)

Notify (36 tests):
- ParseNotifyEmails/MarshalNotifyEmails: old/new format, roundtrip
- crossedDownward: boundary values, threshold semantics
- checkQuotaDimCrossings: mixed dimensions, disabled/zero skip
2026-04-14 09:36:40 +08:00
erio
7c7292935e feat: websearch quota enhancements and balance notify hint
- QuotaLimit changed to *int64 (null=unlimited, >0=limited)
- Add reset-usage endpoint (POST /admin/settings/web-search-emulation/reset-usage)
- Show quota usage in header always (collapsed and expanded)
- Add reset quota button in expanded provider view
- Quota input: empty=unlimited with ∞ placeholder, must be >0 if set
- Add email verification hint on balance notify card
2026-04-14 09:36:40 +08:00
erio
0a4ece5f5b fix: audit round-3 — proxy safety, intervals persistence, SMTP timeout, sort fix
- Skip websearch provider when ProxyID is set but proxy not found (prevent
  silent direct connection bypass)
- Fix sortByStableRandomWeight: pair factors with items so sort.Slice swap
  keeps weights aligned
- Allow empty platform in account_stats_pricing_rules (wildcard matching),
  only force anthropic default for main model_pricing
- Add channel_account_stats_pricing_intervals table and repo layer support
  for interval-based pricing in account stats rules
- calculateTokenStatsCost now uses interval pricing when available
- Replace smtp.SendMail/tls.Dial with net.Dialer timeout (10s dial, 20s IO)
  to prevent goroutine leak on SMTP hang
- Fix gofmt formatting issues
- Web Search label: black text with red warning hint
2026-04-14 09:35:20 +08:00
erio
74f8a30f86 fix: address audit findings for websearch, email verification, and pricing
- Fix websearch provider failover: proxy error from provider-specific proxy
  now continues to next provider instead of aborting the entire loop
- Fix SMTP failure locking users out: send email first, then write cache
  and increment rate counter
- Fix notify email cache key case sensitivity: normalize to lowercase
- Add OriginalPrice validation to validatePlanPatch and validatePlanRequired
- Add empty scope validation for channel pricing rules (group_ids/account_ids)
- Add platform color to account search dropdown in channel pricing rules
2026-04-14 09:33:53 +08:00
erio
d0674e0ff9 feat(websearch): settings UI overhaul and quota improvements
- Remove Priority field, auto load-balance by quota remaining
- Replace QuotaRefreshInterval (daily/weekly/monthly) with SubscribedAt
  (subscription date, monthly lazy refresh via Redis TTL)
- Add collapsible provider cards, API key show/copy, usage progress bar
- Add test endpoint (POST /web-search-emulation/test) bypassing quota
- Wire WebSearchManagerBuilder on startup (was never called before)
- Fix nextMonthlyReset day-of-month overflow (Jan 31 → Feb 28)
- Fix non-deterministic sort in selectByQuotaWeight
- Map ProxyID in builder for provider-level proxy tracking
- Fix frontend timezone drift in subscribed_at date picker
- Fix provider deletion index shift for expandedProviders state
2026-04-14 09:23:40 +08:00
erio
60b0fa81ec fix(websearch): improve isProxyError detection and add manager tests
- Add TLS error detection to isProxyError (RecordHeaderError, handshake)
- Case-insensitive error string matching
- Add 19 unit tests for: isProviderAvailable, resolveProxyID,
  isProxyError, isProxyAvailable, selectByQuotaWeight, newHTTPClient
2026-04-14 09:22:26 +08:00
erio
499159870c fix: gofmt websearch manager 2026-04-14 09:22:25 +08:00
erio
fda61b067c feat(websearch): proxy failover, timeout, quota-weighted load balancing
- Use proxyutil.ConfigureTransportProxy for unified proxy protocol support
  (HTTP/HTTPS/SOCKS5/SOCKS5H), replacing ad-hoc HTTP-only proxy code
- Proxy errors return ErrProxyUnavailable → gateway triggers account switch
  via UpstreamFailoverError instead of fallback to direct connection
- Timeout: proxy dial 3s, TLS handshake 3s, data transfer 60s
- Mark proxy unavailable for 5 minutes in Redis on connectivity failure
- Quota-weighted load balancing: providers with quota_limit>0 are selected
  by remaining quota (weighted random); quota_limit=0 providers treated as
  0% weight and placed last
2026-04-14 09:22:25 +08:00
erio
1b53ffcac7 feat(gateway): add web search emulation for Anthropic API Key accounts
Inject web search capability for Claude Console (API Key) accounts that
don't natively support Anthropic's web_search tool. When a pure
web_search request is detected, the gateway calls Brave Search or Tavily
API directly and constructs an Anthropic-protocol-compliant SSE/JSON
response without forwarding to upstream.

Backend:
- New `pkg/websearch/` SDK: Brave and Tavily provider implementations
  with io.LimitReader, proxy support, and Redis-based quota tracking
  (Lua atomic INCR + TTL, DECR rollback on failure)
- Global config via `settings.web_search_emulation_config` (JSON) with
  in-process cache + singleflight, input validation, API key merge on
  save, and sanitized API responses
- Channel-level toggle via `channels.features_config` JSONB column
  (DB migration 101)
- Account-level toggle via `accounts.extra.web_search_emulation`
- Request interception in `Forward()` with SSE streaming response
  construction using json.Marshal (no manual string concatenation)
- Manager hot-reload: `RebuildWebSearchManager()` called on config save
  and startup via `SetWebSearchRedisClient()`
- 70 unit tests covering providers, manager, config validation,
  sanitization, tool detection, query extraction, and response building

Frontend:
- Settings → Gateway tab: Web Search Emulation config card with global
  toggle, provider list (add/remove, API key, priority, quota, proxy)
- Channels → Anthropic tab: web search emulation toggle with global
  state linkage (disabled when global off)
- Account Create/Edit modals: web search emulation toggle for API Key
  type with Toggle component
- Full i18n coverage (zh + en)
2026-04-14 09:20:39 +08:00
IanShaw027
2b70d1d332 merge upstream main into fix/bug-cleanup-main 2026-04-09 21:35:48 +08:00
IanShaw027
5f8e60a1b7 feat(table): 表格排序与搜索改为后端处理 2026-04-09 18:14:28 +08:00
IanShaw027
4de4823a65 feat(openai): 支持messages模型映射与instructions模板注入 2026-04-09 12:29:49 +08:00
Wesley Liddick
5088e91566 Merge pull request #1417 from YanzheL/fix/openai-empty-base64-image-payloads
fix: sanitize empty base64 image payloads for OpenAI requests
2026-04-08 14:20:38 +08:00
Wesley Liddick
276f499c82 Merge pull request #1418 from YanzheL/fix/1161-gemini-google-search-grounding
fix(gemini): preserve google search grounding tools
2026-04-08 14:19:57 +08:00
shaw
b2e379cf7a fix: 非流式路径在上游终态事件output为空时从delta事件重建响应内容
上游API近期更新后,response.completed终态SSE事件的output字段可能为空,
实际内容仅通过response.output_text.delta等增量事件下发。流式路径不受影响,
但chat_completions非流式路径和responses OAuth非流式路径只依赖终态事件的
output,导致返回空响应。

新增BufferedResponseAccumulator累积器,在SSE扫描过程中收集delta事件内容
(文本、function_call、reasoning),当终态output为空时补充重建。

同时修复handleChatBufferedStreamingResponse遗漏response.done事件类型的问题。
2026-04-07 19:35:56 +08:00
Wesley Liddick
c71b1d63e5 Merge pull request #1460 from imfusheng/main
解决issue#1453提到的反重力不可用问题
2026-04-05 22:01:08 +08:00
erio
62e80c602d revert: completely remove all Sora functionality 2026-04-05 17:11:01 +08:00
偷得浮生
66779f1c5f 同步ag版本号 2026-04-05 13:01:28 +08:00
偷得浮生
2c856b67ca Update oauth.go 2026-04-05 13:00:23 +08:00
erio
1b5ae71d1f fix: resolve golangci-lint issues — remove unused constants and functions, fix gofmt
- Remove unused claudeMax*Tokens constants (Claude Max feature not included)
- Remove unused UsageMapHook type, SetUsageMapHook method, and usageToMap function
- Fix gofmt formatting in channel_service.go, openai_model_mapping_test.go,
  chatcompletions_to_responses.go
2026-04-04 14:58:20 +08:00
erio
e59fa8637a fix: resolve cherry-pick compilation and test issues
- Add int64(0) param to SelectAccountWithLoadAwareness callers (signature change from channel scheduling refactor)
- Add UsageMapHook type and struct field to StreamingProcessor
- Revert Claude Max cache billing code to upstream/main (not part of channel feature)
- Revert credits overages logic to upstream/main (non-channel change)
- Remove Instructions field reference (non-channel OpenAI feature)
- Restore sora_client_handler_test.go from upstream + add channel service nil params
2026-04-04 12:38:50 +08:00
erio
3851628a43 fix: resolve golangci-lint issues
- Fix errcheck: defer rows.Close() with nolint
- Fix errcheck: type assertion with ok check in channel cache
- Fix staticcheck ST1005: lowercase error string
- Fix staticcheck SA5011: nil check cost before use in openai gateway
- Fix gofmt: format chatcompletions_to_responses.go
2026-04-04 11:17:24 +08:00
erio
d72ac92694 feat: image output token billing, channel-mapped billing source, credits balance precheck
- Parse candidatesTokensDetails from Gemini API to separate image/text output tokens
- Add image_output_tokens and image_output_cost to usage_log (migration 089)
- Support per-image-token pricing via output_cost_per_image_token from model pricing data
- Channel pricing ImageOutputPrice override works in token billing mode
- Auto-fill image_output_price in channel pricing form from model defaults
- Add "channel_mapped" billing model source as new default (migration 088)
- Bills by model name after channel mapping, before account mapping
- Fix channel cache error TTL sign error (115s → 5s)
- Fix Update channel only invalidating new groups, not removed groups
- Fix frontend model_mapping clearing sending undefined instead of {}
- Credits balance precheck via shared AccountUsageService cache before injection
- Skip credits injection for accounts with insufficient balance
- Don't mark credits exhausted for "exhausted your capacity on this model" 429s
2026-04-04 11:15:59 +08:00
erio
2555951be4 feat(channel): 渠道管理全链路集成 — 模型映射、定价、限制、用量统计
- 渠道模型映射:支持精确匹配和通配符映射,按平台隔离
- 渠道模型定价:支持 token/按次/图片三种计费模式,区间分层定价
- 模型限制:渠道可限制仅允许定价列表中的模型
- 计费模型来源:支持 requested/upstream 两种计费模型选择
- 用量统计:usage_logs 新增 channel_id/model_mapping_chain/billing_tier/billing_mode 字段
- Dashboard 支持 model_source 维度(requested/upstream/mapping)查看模型统计
- 全部 gateway handler 统一接入 ResolveChannelMappingAndRestrict
- 修复测试:同步 SoraGenerationRepository 接口、SQL INSERT 参数、scan 字段
2026-04-04 11:13:58 +08:00
erio
a51e0047b7 feat(usage): 使用记录增加计费模式字段 — 记录/展示/筛选 token/按次/图片
- DB: usage_logs 表新增 billing_mode VARCHAR(20) 列
- 后端: RecordUsage 写入时根据 image_count 判定计费模式
- 前端: 使用记录表格新增计费模式 badge 列 + 筛选下拉
2026-04-04 11:11:06 +08:00
YanzheL
095bef9554 fix(gemini): add customtools fallback metadata 2026-04-01 02:19:10 +08:00
YanzheL
936fce68d0 fix(apicompat): skip empty base64 image URLs 2026-04-01 00:46:16 +08:00
YanzheL
d978ac97f1 test(antigravity): cover mixed web search transforms 2026-04-01 00:46:14 +08:00
YanzheL
c8cfad7c00 fix(antigravity): preserve google search with function tools 2026-04-01 00:33:16 +08:00
Elysia
81ca4f12dd 修复误删的url 2026-03-28 00:55:55 +08:00
Elysia
941c469ab9 fix: use standard PKCE code verifier generation
Replace charset→base64url double-encoding with standard random
bytes→base64url approach to match official client behavior and avoid
risk control detection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 00:47:31 +08:00
Elysia
8fcd819e6f feat: add user:file_upload OAuth scope
Align OAuth scopes with upstream Claude Code client which now includes
the user:file_upload scope for file upload support.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 00:40:36 +08:00
shaw
1854050df3 feat(tls-fingerprint): 新增 TLS 指纹 Profile 数据库管理及代码质量优化
新增功能:
- 新增 TLS 指纹 Profile CRUD 管理(Ent schema + 迁移 + Admin API + 前端管理界面)
- 支持账号绑定数据库中的自定义 TLS Profile,或随机选择(profile_id=-1)
- HTTPUpstream.DoWithTLS 接口从 bool 改为 *tlsfingerprint.Profile,支持按账号指定 Profile
- AccountUsageService 注入 TLSFingerprintProfileService,统一 usage 场景与网关的 Profile 解析逻辑

代码优化:
- 删除已被 TLSFingerprintProfileService 完全取代的 registry.go 死代码(418 行)
- 提取 3 个 dialer 的重复 TLS 握手逻辑为 performTLSHandshake() 共用函数
- 修复 GetTLSFingerprintProfileID 缺少 json.Number 处理的 bug
- gateway_service.Forward 中 ResolveTLSProfile 从重试循环内重复调用改为预解析局部变量
- 删除冗余的 buildClientHelloSpec() 单行 wrapper 和 int64(e.ID) 无效转换
- tls_fingerprint_profile_cache.go 日志从 log.Printf 改为 slog 结构化日志
- dialer_capture_test.go 添加 //go:build integration 标签,防止 CI 失败
- 去重 TestProfileExpectation 类型至共享 test_types_test.go
- 修复 9 个测试文件缺少 tlsfingerprint import 的编译错误
- 修复 error_policy_integration_test.go 中 handleError 回调签名被错误替换的问题
2026-03-27 14:33:05 +08:00
QTom
975e6b1563 fix: 修复 golangci-lint 报告的 5 个问题
- gofmt: 修复 admin_service/antigravity_oauth_service/token_refresh_service 格式
- staticcheck S1009: 移除 SetUserSettingsResponse.IsSuccess 中冗余的 nil 检查
- unused: 将仅测试使用的 applyAntigravitySubscriptionResult 移至测试文件

Made-with: Cursor
2026-03-25 19:03:12 +08:00
QTom
f6fd7c83e3 feat(antigravity): 从 LoadCodeAssist 复用 TierInfo 提取 plan_type
复用已有 GetTier() 返回的 tier ID(free-tier / g1-pro-tier /
g1-ultra-tier),通过 TierIDToPlanType 映射为 Free / Pro / Ultra,
在 loadProjectIDWithRetry 中顺带提取并写入 credentials.plan_type;
前端增加 Abnormal 异常套餐红色标记。

Made-with: Cursor
2026-03-25 17:38:41 +08:00
QTom
c2965c0fb0 feat(antigravity): 自动设置隐私并支持后台手动重试
新增 Antigravity OAuth 隐私设置能力,在账号创建、刷新、导入和后台
Token 刷新路径自动调用 setUserSettings + fetchUserInfo 关闭遥测;
持久化后同步内存 Extra,错误处理改为日志记录。

Made-with: Cursor
2026-03-25 17:38:41 +08:00
Wesley Liddick
5f41b74707 Merge pull request #1242 from Ethan0x0000/feat/anthropic-openai-endpoint-compat
支持 Anthropic Responses / Chat Completions 兼容端点并完善会话一致性与错误可观测性
2026-03-24 15:16:26 +08:00
QTom
91b1d812ce feat(openai): Mobile RT 补全 plan_type、精确匹配账号、刷新时自动设置隐私
1. accounts/check 补全 plan_type:当 id_token 缺少 plan_type(如 Mobile RT),
   自动调用 accounts/check 端点获取订阅类型
2. orgID 精确匹配账号:从 JWT 提取 poid 匹配正确账号,避免 Go map
   遍历顺序随机导致 plan_type 不稳定
3. RT 刷新时设置隐私:调用 disableOpenAITraining 关闭训练数据共享,
   结果存入 extra.privacy_mode,后续跳过重复设置

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 14:50:03 +08:00
Ethan0x0000
f10e56be7e refactor(test): improve type assertions in ops endpoint context tests 2026-03-24 09:52:56 +08:00
Ethan0x0000
68f151f5c0 feat(apicompat): add Responses↔Anthropic bidirectional format conversion
Add reverse-direction converters for Anthropic platform groups to accept
OpenAI-format requests:

- ResponsesToAnthropicRequest: Responses API input → Anthropic Messages
  request with system extraction, tool/toolChoice mapping, reasoning
  effort conversion, image data URI↔base64, and consecutive role merging
- AnthropicToResponsesResponse: Anthropic response → Responses response
  with content block→output item mapping, usage, stop_reason→status
- AnthropicEventToResponsesEvents: stateful SSE stream converter
  (Anthropic streaming protocol → Responses streaming protocol)
- FinalizeAnthropicResponsesStream: synthetic termination for
  incomplete streams
2026-03-23 16:24:09 +08:00
mutuyihao
4feacf2213 fix(apicompat): support array content for system and tool messages 2026-03-21 15:34:28 +08:00
Wesley Liddick
a225a241d7 Merge pull request #1162 from remxcode/main
feat(openai): 增加 gpt-5.4-mini/nano 模型支持与定价配置
2026-03-20 13:57:47 +08:00
alfadb
8afa8c1091 fix(apicompat): 修正 Anthropic→OpenAI 推理级别映射
旧映射错误地将所有级别上移一档(medium→high, high→xhigh),
导致 effort=max 被原样透传到 OpenAI 上游并返回 400 错误。

根据两边官方 API 定义对齐:
- Anthropic: low, medium, high(默认), max
- OpenAI:    low, medium, high(默认), xhigh

新的 1:1 映射:low→low, medium→medium, high→high, max→xhigh
2026-03-20 12:01:02 +08:00