Commit Graph

34 Commits

Author SHA1 Message Date
erio
6cd7c60549 fix(channels): supported models = mapping ∪ pricing with global LiteLLM fallback
Why: channels with model pricing entries but no model mapping (e.g. azcc with
3 priced claude models, no mapping) were rendering as 未配置模型 in the
'Available Channels' page. The algorithm only iterated ModelMapping and
silently dropped any platform without a mapping entry.

Changes:
- channel.go: SupportedModels now unions mapping + pricing entries.
  For exact mapping src → target, pricing is looked up by target (the actually
  billed name), not by src.
- channel_available.go: ListAvailable enriches each entry with nil pricing
  via PricingService.GetModelPricing (global LiteLLM fallback) so the popover
  always shows a price.
- channel_service.go: NewChannelService takes *PricingService as 4th param.
- channel_test.go: rewrote 4 tests that froze the old mapping-only semantics;
  added pricing-only / mapping-target / target-missing coverage.
2026-04-23 00:45:10 +08:00
erio
4a3652ec09 refactor(channels): normalize at cache fill and eliminate frontend as-cast
- channel.go: convert normalizeBillingModelSource into a (*Channel) method for entity cohesion
- channel_service.go: normalize in populateChannelCache so every cache-backed reader (gateway, billing, future endpoints) sees the default; drop the duplicate fallback inside resolveMapping
- table: tighten Row with status?: ChannelStatus / billing_model_source?: BillingModelSource, remove the [key: string]: unknown index signature
- admin view: drop the `as ChannelStatus` / `as BillingModelSource` assertions and add statusStyleOf / billingSourceLabelOf helpers with runtime fallback so unseen values render as "-" instead of crashing
2026-04-21 14:10:53 +08:00
erio
375aefa209 refactor(channels): centralize BillingModelSource normalization and exhaustive enum maps
- service: add normalizeBillingModelSource helper, apply in Create/GetByID/Update/List/ListAvailable outputs
- handler: drop channelToResponse fallback now that service owns the default; add passthrough test
- frontend: replace ternary status/billing-source lookups with Record<Enum, ...> maps so new union members fail the build
- chip/table: drop local type aliases, reuse UserSupportedModel/UserPricingInterval directly
- tests: assert short-circuit on ListAll error, wrap-prefix preservation, and Name-based default lookup
2026-04-21 11:31:54 +08:00
erio
654cfb6480 feat(channels): add "Available Channels" aggregate view
Add a read-only aggregate view per channel: its linked groups and a
deterministic wildcard-free supported-model list with pricing details.

Backend
- service.Channel.SupportedModels(): combine ModelMapping keys with
  same-platform ModelPricing.Models; trailing "*" keys expand via
  pricing prefix match; platforms without a mapping produce no
  entries (intentional "no mapping = not shown" rule).
- Extract splitWildcardSuffix() shared with toModelEntry.
- Build a per-call pricing lookup map (platform+lowerName -> *pricing)
  to avoid O(N*M) scans in SupportedModels.
- ChannelService.ListAvailable() aggregates channels + active groups;
  filters out group IDs no longer active.
- Admin route GET /api/v1/admin/channels/available returns the full
  DTO (id, status, billing_model_source, restrict_models, groups,
  supported_models).
- User route GET /api/v1/channels/available applies three filters:
  Status==active, visible-group intersection, and platform filter
  on supported_models (prevents cross-platform leak when a channel
  links to both a user-accessible group and an inaccessible one on
  another platform). Response is a plain array (matches the
  /groups/available sibling shape). Field whitelist omits
  billing_model_source, restrict_models, ids, status, sort_order.

Frontend
- New /admin/available-channels and /available-channels views backed
  by a shared AvailableChannelsTable component (admin adds status +
  billing-source columns via slots).
- PricingRow extracted to its own SFC; SupportedModelChip references
  shared billing-mode constants in constants/channel.ts.
- Sidebar: new entry above "渠道管理" for admin; matching entry in
  user nav.
- i18n: zh + en coverage for both namespaces.

Tests
- SupportedModels: wildcard-only pricing skipped, prefix-matches-
  nothing, cross-platform bleed, case-insensitive dedup, empty
  platform mapping.
- ListAvailable: nil groupRepo, inactive-group-ID dropped, stable
  case-insensitive name sort.
- User handler: 401 on unauthenticated, visible-group intersection,
  platform filter on supported_models, JSON whitelist.
- Admin handler: full DTO including default BillingModelSource
  fallback.

Refs: issue #1729
2026-04-21 00:27:10 +08:00
erio
6ac8ccde46 fix: merge 30 general improvements from release branch
Bug fixes:
- Detached context for GetAccountConcurrencyBatch (prevent all-zero on request cancel)
- Filter soft-deleted users in GetByGroupID
- Stripe CSP policy (allow Stripe.js in script-src and frame-src)
- WebSearch API key validation on save
- RECHARGING status in payment result success check
- Windows test fixes (logger Sync deadlock, config path escaping)

Feature enhancements:
- Webhook multi-instance dispatch (extractOutTradeNo + GetWebhookProvider)
- EasyPay mobile H5 payment (device param + PayURL2)
- SSE error propagation in WebSearch emulation
- AccountStatsCost DTO field for admin usage logs
- Plans sort by sort_order instead of created_at
- UsageMapHook for streaming response usage data
- apicompat Instructions field passthrough
- EffectiveLoadFactor for ops concurrency/metrics
- Usage billing RETURNING balance for notify system
- BulkUpdate mixed channel warning with details
- println to slog migration in auth cache
- Wire ProviderSet cleanup
- CI cache-dependency-path optimization

Frontend:
- Refund eligibility check per provider (canRequestRefund)
- Plan sort_order editing
- Dead code cleanup (simulate_claude_max, client_affinity)
- GroupsView platform switch guard
- channels features_config API type
- UsageView account_stats_cost export
2026-04-14 17:35:27 +08:00
erio
9c09bd19b4 fix: websearch features_config cleanup and pricing rules validation
- Fix web_search_emulation toggle: explicitly write false for disabled
  platforms instead of leaving stale true from cloned features_config
- Extract validatePricingEntries from validateChannelConfig for reuse
- Validate account_stats_pricing_rules[].pricing in both Create and
  Update paths (negative prices, bad intervals, missing per_request price)
2026-04-14 09:35:20 +08:00
erio
7535e312e0 feat(channels): add custom account stats pricing rules
Allow channels to configure independent model pricing for account
statistics cost calculation, decoupled from user billing.

Backend:
- Migration 101: channels.apply_pricing_to_account_stats toggle,
  channel_account_stats_pricing_rules/model_pricing tables,
  usage_logs.account_stats_cost column
- resolveAccountStatsCost: match rules by group/account, then channel
  pricing, fallback to original formula when unconfigured
- Integrate into both GatewayService.recordUsageCore and
  OpenAIGatewayService.RecordUsage
- Update 8 account stats SQL queries to use
  COALESCE(account_stats_cost, total_cost) * account_rate_multiplier
- 23 unit tests for matching, pricing lookup, and cost calculation

Frontend:
- Channel edit dialog: toggle + custom rules UI with group/account
  multi-select and pricing entry cards
- API types and i18n (zh/en)
2026-04-14 09:22:12 +08:00
erio
1b53ffcac7 feat(gateway): add web search emulation for Anthropic API Key accounts
Inject web search capability for Claude Console (API Key) accounts that
don't natively support Anthropic's web_search tool. When a pure
web_search request is detected, the gateway calls Brave Search or Tavily
API directly and constructs an Anthropic-protocol-compliant SSE/JSON
response without forwarding to upstream.

Backend:
- New `pkg/websearch/` SDK: Brave and Tavily provider implementations
  with io.LimitReader, proxy support, and Redis-based quota tracking
  (Lua atomic INCR + TTL, DECR rollback on failure)
- Global config via `settings.web_search_emulation_config` (JSON) with
  in-process cache + singleflight, input validation, API key merge on
  save, and sanitized API responses
- Channel-level toggle via `channels.features_config` JSONB column
  (DB migration 101)
- Account-level toggle via `accounts.extra.web_search_emulation`
- Request interception in `Forward()` with SSE streaming response
  construction using json.Marshal (no manual string concatenation)
- Manager hot-reload: `RebuildWebSearchManager()` called on config save
  and startup via `SetWebSearchRedisClient()`
- 70 unit tests covering providers, manager, config validation,
  sanitization, tool detection, query extraction, and response building

Frontend:
- Settings → Gateway tab: Web Search Emulation config card with global
  toggle, provider list (add/remove, API key, priority, quota, proxy)
- Channels → Anthropic tab: web search emulation toggle with global
  state linkage (disabled when global off)
- Account Create/Edit modals: web search emulation toggle for API Key
  type with Toggle component
- Full i18n coverage (zh + en)
2026-04-14 09:20:39 +08:00
erio
794e817208 refactor: remove PaymentChannel, reuse upstream Channel with features field
- Delete payment_channels table and PaymentChannel Ent schema
- Add `features` column to upstream channels table (migration 095)
- Add Features field to Channel struct, input types, handler request/response
- Payment user/admin handlers now use ChannelService directly
- Remove Channel CRUD from PaymentConfigService and admin payment routes
- Remove "渠道管理" tab from admin orders page (use /admin/channels)
2026-04-14 09:15:29 +08:00
erio
37c23eccfe fix: gofmt formatting 2026-04-14 09:14:29 +08:00
erio
e374874125 feat(channel): improve cache strategy and add restriction logging
- Change channel cache TTL from 60s to 10min (reduce unnecessary DB queries)
- Actively rebuild cache after CRUD instead of lazy invalidation
- Add slog.Warn logging for channel pricing restriction blocks (4 places)
2026-04-14 09:13:53 +08:00
erio
9151d34d40 refactor(channel): split long functions, extract shared validation, move billing validation to service
- Split Update (98→25 lines), buildCache (54→20 lines), Create (51→25 lines)
  into focused sub-functions: applyUpdateInput, checkGroupConflicts,
  fetchChannelData, populateChannelCache, storeErrorCache, getOldGroupIDs,
  invalidateAuthCacheForGroups
- Extract validateChannelConfig to eliminate duplicated validation calls
  between Create and Update
- Move validatePricingBillingMode from handler to service layer for
  proper separation of concerns
- Add error logging to IsModelRestricted (was silently swallowing errors)
- Add 12 new tests: ToUsageFields, billing mode validation, antigravity
  wildcard mapping isolation, Create/Update mapping conflict integration
2026-04-05 22:32:49 +08:00
erio
c5688fef9a fix: remove cross-platform pricing/mapping leakage for antigravity groups
Antigravity groups were incorrectly matching pricing and model mapping
entries from anthropic/gemini platform tabs. Each platform should be
strictly isolated — antigravity groups only use antigravity-tagged pricing.
2026-04-05 20:42:24 +08:00
erio
1b5ae71d1f fix: resolve golangci-lint issues — remove unused constants and functions, fix gofmt
- Remove unused claudeMax*Tokens constants (Claude Max feature not included)
- Remove unused UsageMapHook type, SetUsageMapHook method, and usageToMap function
- Fix gofmt formatting in channel_service.go, openai_model_mapping_test.go,
  chatcompletions_to_responses.go
2026-04-04 14:58:20 +08:00
erio
58f758c816 feat(channel): improve cache strategy and add restriction logging
- Change channel cache TTL from 60s to 10min (reduce unnecessary DB queries)
- Actively rebuild cache after CRUD instead of lazy invalidation
- Add slog.Warn logging for channel pricing restriction blocks (4 places)
2026-04-04 11:25:01 +08:00
erio
feb6999d9a fix: channel cache fail-close, group conflict check across pages, status toggle stale data
- GetGroupPlatforms failure now stores error-TTL cache and returns error (fail-close)
- Frontend group-to-channel conflict map loads all channels instead of current page only
- Toggle channel status reloads list when active filter would hide the changed item
2026-04-04 11:25:01 +08:00
erio
71f61bbc47 fix: resolve 5 audit findings in channel/credits/scheduling
P0-1: Credits degraded response retry + fail-open
- Add isAntigravityDegradedResponse() to detect transient API failures
- Retry up to 3 times with exponential backoff (500ms/1s/2s)
- Invalidate singleflight cache between retries
- Fail-open after exhausting retries instead of 5h circuit break

P1-1: Fix channel restriction pre-check timing conflict
- Swap checkClaudeCodeRestriction before checkChannelPricingRestriction
- Ensures channel restriction is checked against final fallback groupID

P1-2: Add interval pricing validation (frontend + backend)
- Backend: ValidateIntervals() with boundary, price, overlap checks
- Frontend: validateIntervals() with Chinese error messages
- Rules: MinTokens>=0, MaxTokens>MinTokens, prices>=0, no overlap

P2: Fix cross-platform same-model pricing/mapping override
- Store cache keys using original platform instead of group platform
- Lookup across matching platforms (antigravity→anthropic→gemini)
- Prevents anthropic/gemini same-name models from overwriting each other
2026-04-04 11:25:01 +08:00
erio
1fca2bfab1 fix: address review findings for channel restriction refactoring
- Fix 7 stale comments still mentioning "限制检查" in handlers/services
- Make billingModelForRestriction explicitly list channel_mapped case
- Add slog.Warn for error swallowing in ResolveChannelMapping and
  needsUpstreamChannelRestrictionCheck
- Document sticky session upstream check exemption
2026-04-04 11:25:01 +08:00
erio
ce41afb756 refactor: move channel model restriction from handler to scheduling phase
Move the model pricing restriction check from 8 handler entry points
to the account scheduling phase (SelectAccountForModelWithExclusions /
SelectAccountWithLoadAwareness), aligning restriction with billing:

- requested: check original request model against pricing list
- channel_mapped: check channel-mapped model against pricing list
- upstream: per-account check using account-mapped model

Handler layer now only resolves channel mapping (no restriction).
Scheduling layer performs pre-check for requested/channel_mapped,
and per-account filtering for upstream billing source.
2026-04-04 11:24:48 +08:00
erio
6de1d0cb33 refactor: split buildCache into sub-functions, reduce nesting 5→2
- Extract newEmptyChannelCache() factory to deduplicate map init
- Extract expandPricingToCache() for model pricing expansion
- Extract expandMappingToCache() for model mapping expansion
- buildCache reduced from 110 to 50 lines
2026-04-04 11:21:11 +08:00
erio
8d25335b01 fix: antigravity groups now match anthropic/gemini channel pricing
Antigravity platform serves both Claude and Gemini models, but channel
pricing/mapping is configured under Anthropic/Gemini tabs. The cache
builder was using strict platform equality, causing antigravity groups
to miss all channel pricing entries, resulting in $0 billing.

Add isPlatformPricingMatch() to treat antigravity as superset of
anthropic+gemini for pricing and mapping cache indexing.
2026-04-04 11:17:48 +08:00
erio
3851628a43 fix: resolve golangci-lint issues
- Fix errcheck: defer rows.Close() with nolint
- Fix errcheck: type assertion with ok check in channel cache
- Fix staticcheck ST1005: lowercase error string
- Fix staticcheck SA5011: nil check cost before use in openai gateway
- Fix gofmt: format chatcompletions_to_responses.go
2026-04-04 11:17:24 +08:00
erio
d72ac92694 feat: image output token billing, channel-mapped billing source, credits balance precheck
- Parse candidatesTokensDetails from Gemini API to separate image/text output tokens
- Add image_output_tokens and image_output_cost to usage_log (migration 089)
- Support per-image-token pricing via output_cost_per_image_token from model pricing data
- Channel pricing ImageOutputPrice override works in token billing mode
- Auto-fill image_output_price in channel pricing form from model defaults
- Add "channel_mapped" billing model source as new default (migration 088)
- Bills by model name after channel mapping, before account mapping
- Fix channel cache error TTL sign error (115s → 5s)
- Fix Update channel only invalidating new groups, not removed groups
- Fix frontend model_mapping clearing sending undefined instead of {}
- Credits balance precheck via shared AccountUsageService cache before injection
- Skip credits injection for accounts with insufficient balance
- Don't mark credits exhausted for "exhausted your capacity on this model" 429s
2026-04-04 11:15:59 +08:00
erio
2555951be4 feat(channel): 渠道管理全链路集成 — 模型映射、定价、限制、用量统计
- 渠道模型映射:支持精确匹配和通配符映射,按平台隔离
- 渠道模型定价:支持 token/按次/图片三种计费模式,区间分层定价
- 模型限制:渠道可限制仅允许定价列表中的模型
- 计费模型来源:支持 requested/upstream 两种计费模型选择
- 用量统计:usage_logs 新增 channel_id/model_mapping_chain/billing_tier/billing_mode 字段
- Dashboard 支持 model_source 维度(requested/upstream/mapping)查看模型统计
- 全部 gateway handler 统一接入 ResolveChannelMappingAndRestrict
- 修复测试:同步 SoraGenerationRepository 接口、SQL INSERT 参数、scan 字段
2026-04-04 11:13:58 +08:00
erio
dfe3fdc1cc fix(channel): 模型限制以原始请求模型检查定价列表,而非映射后模型
开启 restrict_models 时,应用原始模型名查定价列表;
定价列表未命中即拒绝,不因通配符映射而绕过限制。
2026-04-04 11:13:57 +08:00
erio
88759407c7 feat(channel): 模型映射源支持通配符匹配
与定价通配符一致,映射源支持 * 后缀通配符(最长前缀优先):
- `*` 匹配所有模型
- `claude-*` 匹配 claude- 开头的模型
- 精确匹配优先于通配符
2026-04-04 11:13:57 +08:00
erio
3457bcbfcd fix(channel): 修复 invalidateCache 存入 typed nil 导致 loadCache panic
invalidateCache 存入 (*channelCache)(nil),类型断言 ok=true 但
指针为 nil,后续 cached.loadedAt 导致 nil pointer dereference。
在 loadCache 双重检查处增加 cached != nil 防御。
2026-04-04 11:13:56 +08:00
erio
eb385457b2 fix(channel): 全平台渠道映射覆盖 + 公共函数抽取 + 死代码清理
- 4个缺失handler入口添加渠道映射+限制检查(ChatCompletions/Responses/Gemini)
- 模型限制错误信息优化,区分"模型不可用"和"无账号"
- OpenAI RecordUsage RequestedModel 改用 OriginalModel
- ResolveChannelMappingAndRestrict/ReplaceModelInBody 抽取到 ChannelService 消除跨service重复
- validateNoDuplicateModels 按 platform:model 去重
- 删除 Channel.ResolveMappedModel 死代码和 CalculateCostWithChannel Deprecated方法
- 移除冗余nil检查,抽取 validatePricingBillingMode 公共校验
2026-04-04 11:13:56 +08:00
erio
4ea8b4cb4f refactor(channel): 抽取渠道映射公共函数 + OpenAI映射到body + 空响应修复 + 清理日志
- 抽取 ResolveChannelMappingAndRestrict 统一入口(5处→1个方法)
- 抽取 BuildModelMappingChain 到 ChannelMappingResult 方法(5处→1行调用)
- OpenAI 三入口 Forward 前应用渠道映射到请求体
- OpenAI Responses/Messages 限制检查添加错误响应
- 清理前端 3 处 console.log 调试日志
2026-04-04 11:13:56 +08:00
erio
8d03c52e15 feat(channel): 通配符定价匹配 + OpenAI BillingModelSource + 按次价格校验 + 用户端计费模式展示
- 定价查找支持通配符(suffix *),最长前缀优先匹配
- 模型限制(restrict_models)同样支持通配符匹配
- OpenAI 网关接入渠道映射/BillingModelSource/模型限制
- 按次/图片计费模式创建时强制要求价格或层级(前后端)
- 用户使用记录列表增加计费模式 badge 列
2026-04-04 11:13:43 +08:00
erio
0b1ce6be8f feat(channel): 缓存扁平化 + 网关映射集成 + 计费模式统一 + 模型限制
- 缓存按 (groupID, platform, model) 三维 key 扁平化,避免跨平台同名模型冲突
- buildCache 批量查询 group platform,按平台过滤展开定价和映射
- model_mapping 改为嵌套格式 {platform: {src: dst}}
- channel_model_pricing 新增 platform 列
- 前端按平台维度重构:每个平台独立配置分组/映射/定价
- 迁移 086: platform 列 + model_mapping 嵌套格式迁移
2026-04-04 11:09:28 +08:00
erio
ebac0dc628 feat(channel): 缓存扁平化 + 网关映射集成 + 计费模式统一 + 模型限制
- 缓存重构为 O(1) 哈希结构 (pricingByGroupModel, mappingByGroupModel)
- 渠道模型映射接入网关流程 (Forward 前应用, a→b→c 映射链)
- 新增 billing_model_source 配置 (请求模型/最终模型计费)
- usage_logs 新增 channel_id, model_mapping_chain, billing_tier 字段
- 每种计费模式统一支持默认价格 + 区间定价
- 渠道模型限制开关 (restrict_models)
- 分组按平台分类展示 + 彩色图标
- 必填字段红色星号 + 模型映射 UI
- 去除模型通配符支持
2026-04-04 11:09:01 +08:00
erio
29d58f2414 feat(channel): 模型映射 + 分组搜索 + 卡片折叠 + 冲突校验
- 渠道模型映射:新增 model_mapping JSONB 字段,在账号映射之前执行
- 分组选择:添加搜索过滤 + 平台图标
- 定价卡片:支持折叠/展开,已有数据默认折叠
- 模型冲突校验:前后端均禁止同一渠道内重复模型
- 迁移 083: channels 表添加 model_mapping 列
2026-04-04 11:06:36 +08:00
erio
91c9b8d062 feat(channel): 渠道管理系统 — 多模式定价 + 统一计费解析
Cherry-picked from release/custom-0.1.106: a9117600
2026-04-04 11:00:55 +08:00