Commit Graph

135 Commits

Author SHA1 Message Date
erio
f1297a3694 feat: add per-provider allow_user_refund control and align wildcard matching
allow_user_refund:
- Add allow_user_refund field to PaymentProviderInstance ent schema
- Migration 103: ALTER TABLE payment_provider_instances ADD COLUMN
- Cascade logic: disabling refund_enabled auto-disables allow_user_refund
- User refund validation: check provider instance allows user refund
- Admin refund validation: check provider instance allows admin refund
- Subscription refund: deduct days on refund, rollback on failure
- New endpoint: GET /payment/orders/refund-eligible-providers
- Frontend: ToggleSwitch in ProviderCard/Dialog, cascade in SettingsView

Wildcard matching:
- Change findPricingForModel from "longest prefix wins" to "config order
  priority (first match wins)", aligning with channel service behavior
2026-04-14 16:26:46 +08:00
erio
0a4ece5f5b fix: audit round-3 — proxy safety, intervals persistence, SMTP timeout, sort fix
- Skip websearch provider when ProxyID is set but proxy not found (prevent
  silent direct connection bypass)
- Fix sortByStableRandomWeight: pair factors with items so sort.Slice swap
  keeps weights aligned
- Allow empty platform in account_stats_pricing_rules (wildcard matching),
  only force anthropic default for main model_pricing
- Add channel_account_stats_pricing_intervals table and repo layer support
  for interval-based pricing in account stats rules
- calculateTokenStatsCost now uses interval pricing when available
- Replace smtp.SendMail/tls.Dial with net.Dialer timeout (10s dial, 20s IO)
  to prevent goroutine leak on SMTP hang
- Fix gofmt formatting issues
- Web Search label: black text with red warning hint
2026-04-14 09:35:20 +08:00
erio
1262654d97 feat: WebSearch tri-state, account stats pricing fix, quota cache fix, usage tooltip
WebSearch tri-state switch:
- Account-level web_search_emulation changed from bool to tri-state
  string: "default" (follow channel) / "enabled" / "disabled"
- shouldEmulateWebSearch checks channel config when account is "default"
- SQL migration converts old bool values
- Frontend select replaces toggle in Edit/CreateAccountModal

Account stats pricing:
- resolveAccountStatsCost uses upstream model (post-mapping) for matching
- Priority: custom rules → model pricing file (when toggle on) → default
- Custom rules always configurable, independent of toggle
- Account ID field changed to searchable selector filtered by platform
- Description updated to reflect new behavior

Quota notification cache fix:
- CheckAccountQuotaAfterIncrement fetches real-time account from DB
- Reconstructs pre-increment usage for accurate threshold crossing detection
- New AccountQuotaReader interface (minimal: GetByID only)

Usage tooltip:
- Per-request/image billing shows per-request price instead of $0 token price
- Token billing continues to show input/output price per million tokens
2026-04-14 09:26:08 +08:00
erio
915b7a4a56 feat(notify): convert email lists to NotifyEmailEntry struct with toggle support
- Change balance_notify_extra_emails and account_quota_notify_emails
  from []string to []NotifyEmailEntry{email, disabled, verified}
- Add per-email enable/disable toggle for both user and admin notifications
- Add PUT /user/notify-email/toggle API endpoint
- Fix critical bug: API key auth cache snapshot missing balance notify
  fields (Email, Username, BalanceNotifyEnabled, etc.), causing
  notifications to never fire on cached request paths
- Bump cache snapshot version 3→4 to invalidate stale entries
- Add SQL migration 104 to convert old format data
- Backward compatible: parseNotifyEmails auto-detects old/new format
- User balance notify: max 3 emails (primary + 2 extra)
- Admin quota notify: unlimited emails, each with toggle
2026-04-14 09:26:07 +08:00
erio
f694afbbf4 feat(notify): add percentage threshold type for balance low notification
- Add threshold_type field (fixed/percentage) to system and user settings
- Add total_recharged field to users table, auto-incremented on balance credit
- Percentage mode: effective threshold = total_recharged × percentage / 100
- User-level threshold_type inherits from system default when not set
- Update admin settings UI with radio selector (fixed amount / percentage)
- Migration: 102_add_balance_notify_threshold_type.sql
2026-04-14 09:24:17 +08:00
erio
b32d1a2c9f feat(notify): add balance low & account quota notification system
- User balance low notification: email alert when balance drops below
  configurable threshold (user email + verified extra emails)
- Account quota notification: broadcast email to admin-configured
  recipients when daily/weekly/total quota usage exceeds alert threshold
- Admin settings: global enable/disable, default threshold, quota
  notification email list (Email Settings tab)
- User profile: enable/disable, custom threshold, add/remove extra
  notification emails with verification code flow
- Account quota: per-dimension alert toggle and threshold in quota
  control card
- Trigger logic: first-crossing only (old >= threshold && new < threshold
  for balance; old < threshold && new >= threshold for quota), naturally
  prevents duplicate notifications without Redis dedup
2026-04-14 09:23:02 +08:00
erio
7535e312e0 feat(channels): add custom account stats pricing rules
Allow channels to configure independent model pricing for account
statistics cost calculation, decoupled from user billing.

Backend:
- Migration 101: channels.apply_pricing_to_account_stats toggle,
  channel_account_stats_pricing_rules/model_pricing tables,
  usage_logs.account_stats_cost column
- resolveAccountStatsCost: match rules by group/account, then channel
  pricing, fallback to original formula when unconfigured
- Integrate into both GatewayService.recordUsageCore and
  OpenAIGatewayService.RecordUsage
- Update 8 account stats SQL queries to use
  COALESCE(account_stats_cost, total_cost) * account_rate_multiplier
- 23 unit tests for matching, pricing lookup, and cost calculation

Frontend:
- Channel edit dialog: toggle + custom rules UI with group/account
  multi-select and pricing entry cards
- API types and i18n (zh/en)
2026-04-14 09:22:12 +08:00
erio
1b53ffcac7 feat(gateway): add web search emulation for Anthropic API Key accounts
Inject web search capability for Claude Console (API Key) accounts that
don't natively support Anthropic's web_search tool. When a pure
web_search request is detected, the gateway calls Brave Search or Tavily
API directly and constructs an Anthropic-protocol-compliant SSE/JSON
response without forwarding to upstream.

Backend:
- New `pkg/websearch/` SDK: Brave and Tavily provider implementations
  with io.LimitReader, proxy support, and Redis-based quota tracking
  (Lua atomic INCR + TTL, DECR rollback on failure)
- Global config via `settings.web_search_emulation_config` (JSON) with
  in-process cache + singleflight, input validation, API key merge on
  save, and sanitized API responses
- Channel-level toggle via `channels.features_config` JSONB column
  (DB migration 101)
- Account-level toggle via `accounts.extra.web_search_emulation`
- Request interception in `Forward()` with SSE streaming response
  construction using json.Marshal (no manual string concatenation)
- Manager hot-reload: `RebuildWebSearchManager()` called on config save
  and startup via `SetWebSearchRedisClient()`
- 70 unit tests covering providers, manager, config validation,
  sanitization, tool detection, query extraction, and response building

Frontend:
- Settings → Gateway tab: Web Search Emulation config card with global
  toggle, provider list (add/remove, API key, priority, quota, proxy)
- Channels → Anthropic tab: web search emulation toggle with global
  state linkage (disabled when global off)
- Account Create/Edit modals: web search emulation toggle for API Key
  type with Toggle component
- Full i18n coverage (zh + en)
2026-04-14 09:20:39 +08:00
erio
794e817208 refactor: remove PaymentChannel, reuse upstream Channel with features field
- Delete payment_channels table and PaymentChannel Ent schema
- Add `features` column to upstream channels table (migration 095)
- Add Features field to Channel struct, input types, handler request/response
- Payment user/admin handlers now use ChannelService directly
- Remove Channel CRUD from PaymentConfigService and admin payment routes
- Remove "渠道管理" tab from admin orders page (use /admin/channels)
2026-04-14 09:15:29 +08:00
shaw
f9f57e9505 fix(migrations): add 097 to restore settings.updated_at default
Legacy instances created the settings table via ent auto-migration,
which emits Go-level defaults only. Migration 005 uses CREATE TABLE
IF NOT EXISTS, so the missing SQL DEFAULT was never backfilled. This
caused 098's raw INSERT to fail with a NOT NULL violation on
updated_at. The new migration is idempotent and safe for fresh
installs (no-op) and historical instances (backfills the default).
2026-04-13 23:09:26 +08:00
erio
e3a000e0d4 refactor(payment): code standards fixes and regression repairs
Backend:
- Split payment_order.go (546→314 lines) into payment_order_lifecycle.go
- Replace magic strings with constants in factory, easypay, webhook handler
- Add rate limit/validity unit constants in payment_order_lifecycle, payment_service
- Fix critical regression: add PaymentEnabled to GetPublicSettings response
- Add missing migration 099_fix_migrated_purchase_menu_label_icon.sql

Frontend:
- Fix StripePopupView.vue: replace `as any` with typed interface, use extractApiErrorMessage
- Fix AdminOrderTable.vue: replace hardcoded column labels with i18n t() calls
- Fix SubscriptionsView.vue: replace hardcoded Today/Tomorrow with i18n
- Extract duplicate statusBadgeClass/canRefund/formatOrderDateTime to orderUtils.ts
- Add missing i18n keys: common.today, common.tomorrow, payment.orders.orderType/actions
- Remove dead PurchaseSubscriptionView.vue (replaced by PaymentView)
2026-04-11 13:16:35 +08:00
erio
e1547d7835 fix(payment): resolve PR audit issues
- Add payment navigation to AppSidebar (user orders + admin payment menu group with collapse)
- Add 5 missing nav i18n keys (myOrders, orderManagement, paymentDashboard, paymentConfig, paymentPlans)
- Renumber payment migrations 090-100 → 092-102 to avoid conflict with upstream 090/091
- Remove non-payment sora_client_enabled change, restore upstream purchase_subscription fields
- Remove extra 'data' from SettingsTab type union
2026-04-11 13:16:35 +08:00
erio
63d1860dc0 feat(payment): add complete payment system with multi-provider support
Add a full payment and subscription system supporting EasyPay (Alipay/WeChat),
Stripe, and direct Alipay/WeChat Pay providers with multi-instance load balancing.
2026-04-11 13:16:35 +08:00
IanShaw027
23c4d592f8 feat(group): 增加messages调度模型映射配置 2026-04-09 12:29:28 +08:00
erio
9c514c9808 chore: drop Sora database schema and regenerate ent code 2026-04-05 17:19:07 +08:00
erio
d72ac92694 feat: image output token billing, channel-mapped billing source, credits balance precheck
- Parse candidatesTokensDetails from Gemini API to separate image/text output tokens
- Add image_output_tokens and image_output_cost to usage_log (migration 089)
- Support per-image-token pricing via output_cost_per_image_token from model pricing data
- Channel pricing ImageOutputPrice override works in token billing mode
- Auto-fill image_output_price in channel pricing form from model defaults
- Add "channel_mapped" billing model source as new default (migration 088)
- Bills by model name after channel mapping, before account mapping
- Fix channel cache error TTL sign error (115s → 5s)
- Fix Update channel only invalidating new groups, not removed groups
- Fix frontend model_mapping clearing sending undefined instead of {}
- Credits balance precheck via shared AccountUsageService cache before injection
- Skip credits injection for accounts with insufficient balance
- Don't mark credits exhausted for "exhausted your capacity on this model" 429s
2026-04-04 11:15:59 +08:00
erio
a51e0047b7 feat(usage): 使用记录增加计费模式字段 — 记录/展示/筛选 token/按次/图片
- DB: usage_logs 表新增 billing_mode VARCHAR(20) 列
- 后端: RecordUsage 写入时根据 image_count 判定计费模式
- 前端: 使用记录表格新增计费模式 badge 列 + 筛选下拉
2026-04-04 11:11:06 +08:00
erio
0b1ce6be8f feat(channel): 缓存扁平化 + 网关映射集成 + 计费模式统一 + 模型限制
- 缓存按 (groupID, platform, model) 三维 key 扁平化,避免跨平台同名模型冲突
- buildCache 批量查询 group platform,按平台过滤展开定价和映射
- model_mapping 改为嵌套格式 {platform: {src: dst}}
- channel_model_pricing 新增 platform 列
- 前端按平台维度重构:每个平台独立配置分组/映射/定价
- 迁移 086: platform 列 + model_mapping 嵌套格式迁移
2026-04-04 11:09:28 +08:00
erio
ebac0dc628 feat(channel): 缓存扁平化 + 网关映射集成 + 计费模式统一 + 模型限制
- 缓存重构为 O(1) 哈希结构 (pricingByGroupModel, mappingByGroupModel)
- 渠道模型映射接入网关流程 (Forward 前应用, a→b→c 映射链)
- 新增 billing_model_source 配置 (请求模型/最终模型计费)
- usage_logs 新增 channel_id, model_mapping_chain, billing_tier 字段
- 每种计费模式统一支持默认价格 + 区间定价
- 渠道模型限制开关 (restrict_models)
- 分组按平台分类展示 + 彩色图标
- 必填字段红色星号 + 模型映射 UI
- 去除模型通配符支持
2026-04-04 11:09:01 +08:00
erio
29d58f2414 feat(channel): 模型映射 + 分组搜索 + 卡片折叠 + 冲突校验
- 渠道模型映射:新增 model_mapping JSONB 字段,在账号映射之前执行
- 分组选择:添加搜索过滤 + 平台图标
- 定价卡片:支持折叠/展开,已有数据默认折叠
- 模型冲突校验:前后端均禁止同一渠道内重复模型
- 迁移 083: channels 表添加 model_mapping 列
2026-04-04 11:06:36 +08:00
erio
91c9b8d062 feat(channel): 渠道管理系统 — 多模式定价 + 统一计费解析
Cherry-picked from release/custom-0.1.106: a9117600
2026-04-04 11:00:55 +08:00
QTom
aeed2eb9ad feat(group-filter): 分组账号过滤控制 — require_oauth_only + require_privacy_set
为 OpenAI/Antigravity/Anthropic/Gemini 分组新增两个布尔控制字段:
- require_oauth_only: 创建/更新账号绑定分组时拒绝 apikey 类型加入
- require_privacy_set: 调度选号时跳过 privacy 未成功设置的账号并标记 error

后端:Ent schema 新增字段 + 迁移、Group CRUD 全链路透传、
      gateway_service 与 openai_account_scheduler 两套调度路径过滤
前端:创建/编辑表单 toggle 开关(OpenAI/Antigravity/Anthropic/Gemini 平台可见)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 13:04:55 +08:00
shaw
1854050df3 feat(tls-fingerprint): 新增 TLS 指纹 Profile 数据库管理及代码质量优化
新增功能:
- 新增 TLS 指纹 Profile CRUD 管理(Ent schema + 迁移 + Admin API + 前端管理界面)
- 支持账号绑定数据库中的自定义 TLS Profile,或随机选择(profile_id=-1)
- HTTPUpstream.DoWithTLS 接口从 bool 改为 *tlsfingerprint.Profile,支持按账号指定 Profile
- AccountUsageService 注入 TLSFingerprintProfileService,统一 usage 场景与网关的 Profile 解析逻辑

代码优化:
- 删除已被 TLSFingerprintProfileService 完全取代的 registry.go 死代码(418 行)
- 提取 3 个 dialer 的重复 TLS 握手逻辑为 performTLSHandshake() 共用函数
- 修复 GetTLSFingerprintProfileID 缺少 json.Number 处理的 bug
- gateway_service.Forward 中 ResolveTLSProfile 从重试循环内重复调用改为预解析局部变量
- 删除冗余的 buildClientHelloSpec() 单行 wrapper 和 int64(e.ID) 无效转换
- tls_fingerprint_profile_cache.go 日志从 log.Printf 改为 slog 结构化日志
- dialer_capture_test.go 添加 //go:build integration 标签,防止 CI 失败
- 去重 TestProfileExpectation 类型至共享 test_types_test.go
- 修复 9 个测试文件缺少 tlsfingerprint import 的编译错误
- 修复 error_policy_integration_test.go 中 handleError 回调签名被错误替换的问题
2026-03-27 14:33:05 +08:00
Ethan0x0000
8c4a217f03 feat(ops): add endpoint/model/request_type fields to error log structs + safeUpstreamURL 2026-03-21 23:30:13 +08:00
Ethan0x0000
fe60412a17 feat(db): add requested model usage log migrations
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-03-21 01:20:23 +08:00
Wesley Liddick
241023f3fc Merge pull request #1097 from Ethan0x0000/pr/upstream-model-tracking
feat(usage): 新增 upstream_model 追踪,支持按模型来源统计与展示
2026-03-18 15:36:00 +08:00
erio
af96c8ea53 feat: map claude-haiku-4-5 variants to claude-sonnet-4-6
Update model mapping target for claude-haiku-4-5 and
claude-haiku-4-5-20251001 from claude-sonnet-4-5 to claude-sonnet-4-6.
Includes migration script, default constants, and test updates.
2026-03-18 15:03:24 +08:00
Ethan0x0000
51547fa216 feat(db): add upstream_model column to usage_logs
Add nullable VARCHAR(100) column to record the actual model sent to upstream providers when model mapping is applied. NULL means no mapping — the requested model was used as-is.

Includes migration, concurrent index for aggregation queries, Ent schema regeneration, and migration README correction (forward-only runner, not goose).
2026-03-17 19:25:17 +08:00
Ethan0x0000
eefab15958 feat: 完善使用记录端点可观测性与分布统计
将入站、上游与路径三类端点分布统一到使用记录页的一致化卡片交互中,并补齐端点元数据与统计链路,提升排障与流量分析效率。
2026-03-15 11:26:42 +08:00
ius
b764d3b8f6 Merge remote-tracking branch 'origin/main' into feat/billing-ledger-decouple-usage-log-20260312 2026-03-12 16:53:28 +08:00
ius
611fd884bd feat: decouple billing correctness from usage log batching 2026-03-12 16:53:18 +08:00
Rose Ding
1c0519f1c7 feat: add gemini 2.5 flash image support 2026-03-11 15:21:52 +08:00
Wesley Liddick
391e79f8ee Merge pull request #875 from mt21625457/fix/openai-fast-billing-clean
fix(billing): 修复 OpenAI fast 档位计费并补齐展示
2026-03-09 10:32:18 +08:00
yangjianbo
87f4ed591e fix(billing): 修复 OpenAI fast 档位计费并补齐展示
- 打通 service_tier 在 OpenAI HTTP、WS、passthrough 与 usage 记录中的传递
- 修正 priority/flex 计费逻辑,并将 fast 归一化为 priority
- 在用户端和管理端补齐服务档位与计费明细展示
- 补齐前后端测试,并修复 WS 限流信号重复持久化导致的全量回归失败

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 09:51:26 +08:00
kyx236
0c29468f90 feat(admin): 支持定时测试自动恢复并统一账号恢复入口
- 为定时测试计划增加 auto_recover 配置,补齐前后端类型、接口、仓储与数据库迁移
- 在定时测试成功后自动恢复账号 error、rate-limit 等可恢复运行时状态
- 新增 /admin/accounts/:id/recover-state 接口,合并原有重置状态与清限流操作
- 更新账号管理菜单与定时测试面板,补充自动恢复开关、说明提示和状态展示
- 补充账号恢复、限流清理与仓储同步相关测试
2026-03-08 06:59:53 +08:00
shaw
92d35409de feat: 为openai分组增加messages调度开关和默认映射模型 2026-03-07 17:02:19 +08:00
shaw
351a08f813 fix: announcement强制弹窗通知补全迁移sql 2026-03-07 15:36:18 +08:00
erio
0d6c1c7790 feat: add independent load_factor field for scheduling load calculation 2026-03-06 05:07:10 +08:00
guoyongchang
3a089242f8 feat: 支持基于 crontab 的定时账号测试
每个测试计划绑定一个账号和一个模型,按 cron 表达式定期执行测试,
保存历史结果并在前端账号管理页面中提供完整的增删改查和结果查看功能。

主要变更:
- 新增 scheduled_test_plans / scheduled_test_results 两张表及迁移
- 后端 service 层:CRUD 服务 + 后台 cron runner(每分钟扫描到期计划并发执行)
- RunTestBackground 方法通过 httptest 在内存中执行账号测试并解析 SSE 输出
- Redis leader lock + pg_try_advisory_lock 双重保障多实例部署只执行一次
- REST API:5 个管理端点(计划 CRUD + 结果查询)
- 前端 ScheduledTestsPanel 组件:计划管理、启用开关、内联编辑、结果展开查看
- 中英文 i18n 支持

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 16:06:05 +08:00
xvhuan
80ae592c23 perf(admin): optimize large-dataset loading for dashboard/users/accounts/ops 2026-03-04 13:45:49 +08:00
Wesley Liddick
22f04e72e5 Merge pull request #731 from xvhuan/fix/061-bounded-backfill-startup
fix(migrations): 061 迁移改为限时分批回填,避免启动阻塞导致 502
2026-03-03 15:08:18 +08:00
shaw
5f3debf65b chore: add migration for api key rate limit fields 2026-03-03 15:05:15 +08:00
ius
99f1e3ff35 fix(migrations): avoid startup outage from 061 full-table backfill 2026-03-03 11:01:22 +08:00
shaw
be18bc6fc3 chore: 恢复数据库迁移文件060和修正版本号 2026-02-28 22:02:01 +08:00
yangjianbo
bb664d9bbf feat(sync): full code sync from release 2026-02-28 15:01:20 +08:00
erio
a6f9f9f968 feat: replace gemini-3-pro-image with gemini-3.1-flash-image
- Add migration 060 to update model_mapping for all antigravity accounts
- Remove gemini-3-pro-image and gemini-3-pro-image-preview mappings
- Add gemini-3.1-flash-image and gemini-3.1-flash-image-preview mappings
- Update frontend usage window to show GImage for new model
- Update isImageGenerationModel to support new model
2026-02-27 09:52:50 +08:00
erio
c671e8dd1d fix: 统一gemini-3默认映射为非强制3.1 2026-02-24 23:24:48 +08:00
erio
b6fa8b8eec fix: update tests for defaultClientSecret and align migration 058
- Fix oauth_test.go and client_test.go to use defaultClientSecret
  variable instead of env var (init() already sets the default)
- Align migration 058 gemini-3-pro-high/low/preview mappings with
  constants.go (map to 3.1 versions)
2026-02-24 21:06:10 +08:00
erio
29c406dda0 feat: add migrations for sonnet-4-6 and gemini-3.1-pro model mappings
Add migration 058 to update existing Antigravity accounts with
claude-sonnet-4-6 in model_mapping. Add migration 059 to add
gemini-3.1-pro-high/low/preview mappings.
2026-02-24 19:40:30 +08:00
shaw
980fc9608f fix: 修复日志重复输出及清理冗余迁移逻辑
- logger: sinkCore 包装 tee core 时绕过了子 core 的 Check 级别过滤,
  导致每条日志同时写入 stdout 和 stderr,表现为启动日志重复显示。
  修复为正确委托 Check 给内部 tee core,sinkCore.Write 仅负责 sink 转发。
- migration 054: 移除冗余的遗留列回填逻辑,migration 009 已完成数据迁移,
  直接删除遗留列即可。
2026-02-24 11:31:19 +08:00