Commit Graph

546 Commits

Author SHA1 Message Date
IanShaw027
d08757ce9e refactor(admin): remove auth migration reports 2026-04-21 17:34:18 +08:00
IanShaw027
c624cce88e fix: unblock auth identity compat backfill migration 2026-04-21 15:56:30 +08:00
IanShaw027
0a461d8248 fix: harden auth identity legacy migrations 2026-04-21 01:30:37 +08:00
IanShaw027
ea27ac6fd7 fix: unify email identity sync and retry first-bind defaults 2026-04-21 01:00:59 +08:00
IanShaw027
7a9488ff37 Add legacy identity safety remediation migration 2026-04-21 00:59:20 +08:00
IanShaw027
bf3ef2d19a add admin user last used support 2026-04-21 00:22:17 +08:00
IanShaw027
beeab54ae3 Implement latest-used user repo queries 2026-04-21 00:17:48 +08:00
IanShaw027
5d58c7c6fb Add auth identity legacy backfill and email sync 2026-04-21 00:13:40 +08:00
IanShaw027
31d0183d45 fix: normalize repository email lookups 2026-04-20 21:51:57 +08:00
IanShaw027
c6d8592484 feat: add profile auth identity binding flow 2026-04-20 18:28:44 +08:00
IanShaw027
e9de839d87 feat: rebuild auth identity foundation flow 2026-04-20 17:39:57 +08:00
Wesley Liddick
e6e73b4f52 Merge pull request #1690 from KnowSky404/fix/ws-codex-scheduler-cache-1662
fix: preserve openai ws flags in scheduler cache
2026-04-16 17:21:32 +08:00
KnowSky404
3944b3d216 fix: preserve openai ws flags in scheduler cache 2026-04-16 02:01:50 +00:00
erio
db27e8f000 feat(usage): add account cost to breakdown sub-table and admin usage log
- UserBreakdownItem: add AccountCost field + SQL aggregation
- UserBreakdownSubTable: add orange account cost column
- Admin usage table: add account_cost column (after cost, default visible)
- Column settings: add account_cost toggle option
2026-04-15 15:40:40 +08:00
erio
22680dc602 test(usage): add unit tests for account_cost and fix gofmt
- Fix mock for GetModelStatsWithFilters: add account_cost column
- Add assertion: GetStatsWithFilters always returns TotalAccountCost
- New test: GetModelStatsAccountCostColumn verifies scan of AccountCost
- New test: GetGroupStatsAccountCostColumn verifies scan of AccountCost
- New test: GetStatsWithFiltersAlwaysReturnsAccountCost (no AccountID filter)
- Integration test: add TotalAccountCost/TodayAccountCost assertions
- Fix gofmt alignment in usage_log_types.go
2026-04-15 15:02:21 +08:00
erio
6ade6d30a8 feat(usage): add account cost display to admin dashboard and usage pages
- Add account_cost column to dashboard aggregation tables (migration 107)
- DashboardStats: add TotalAccountCost/TodayAccountCost fields
- ModelStat/GroupStat: add AccountCost field with SQL aggregation
- GetStatsWithFilters: always return TotalAccountCost (remove accountID filter)
- Dashboard Token cards: show user(green)/cost(orange)/standard(gray)
- Usage stats card: show account cost and standard below main value
- Model/Group distribution tables: add orange cost column
2026-04-15 15:02:21 +08:00
erio
58677dd53f fix: merge 5 PR-related improvements
- gateway_handler: pass ParsedRequest to RecordUsage + set in gin.Context
- channel_handler: add FeaturesConfig to CRUD (WebSearch channel toggle)
- channel_repo: features_config JSONB persistence (Create/Get/Update/List)
- security_headers: add Stripe CSP domains (script-src + frame-src)
2026-04-14 18:34:57 +08:00
erio
6ac8ccde46 fix: merge 30 general improvements from release branch
Bug fixes:
- Detached context for GetAccountConcurrencyBatch (prevent all-zero on request cancel)
- Filter soft-deleted users in GetByGroupID
- Stripe CSP policy (allow Stripe.js in script-src and frame-src)
- WebSearch API key validation on save
- RECHARGING status in payment result success check
- Windows test fixes (logger Sync deadlock, config path escaping)

Feature enhancements:
- Webhook multi-instance dispatch (extractOutTradeNo + GetWebhookProvider)
- EasyPay mobile H5 payment (device param + PayURL2)
- SSE error propagation in WebSearch emulation
- AccountStatsCost DTO field for admin usage logs
- Plans sort by sort_order instead of created_at
- UsageMapHook for streaming response usage data
- apicompat Instructions field passthrough
- EffectiveLoadFactor for ops concurrency/metrics
- Usage billing RETURNING balance for notify system
- BulkUpdate mixed channel warning with details
- println to slog migration in auth cache
- Wire ProviderSet cleanup
- CI cache-dependency-path optimization

Frontend:
- Refund eligibility check per provider (canRequestRefund)
- Plan sort_order editing
- Dead code cleanup (simulate_claude_max, client_affinity)
- GroupsView platform switch guard
- channels features_config API type
- UsageView account_stats_cost export
2026-04-14 17:35:27 +08:00
erio
d6965b0676 fix: resolve cherry-pick conflicts and restore compilation
- Restore gateway_cache.go to upstream (no lua embeds)
- Restore payment_order.go to upstream (use out_trade_no lookup)
- Restore payment_fulfillment.go to upstream (same reason)
- Add FeaturesConfig field and IsWebSearchEmulationEnabled to Channel
- Add applyAccountStatsCost wrapper function
- Add SettingKeyWebSearchEmulationConfig constant
- Add WebSearchEmulationEnabled to SystemSettings
- Add notify code rate limiting methods to EmailCache interface
- Remove AllowUserRefund references (ent schema not present)
- Fix duplicate import in payment_handler.go
- Fix wire_gen.go argument mismatches
2026-04-14 10:18:39 +08:00
erio
0a4ece5f5b fix: audit round-3 — proxy safety, intervals persistence, SMTP timeout, sort fix
- Skip websearch provider when ProxyID is set but proxy not found (prevent
  silent direct connection bypass)
- Fix sortByStableRandomWeight: pair factors with items so sort.Slice swap
  keeps weights aligned
- Allow empty platform in account_stats_pricing_rules (wildcard matching),
  only force anthropic default for main model_pricing
- Add channel_account_stats_pricing_intervals table and repo layer support
  for interval-based pricing in account stats rules
- calculateTokenStatsCost now uses interval pricing when available
- Replace smtp.SendMail/tls.Dial with net.Dialer timeout (10s dial, 20s IO)
  to prevent goroutine leak on SMTP hang
- Fix gofmt formatting issues
- Web Search label: black text with red warning hint
2026-04-14 09:35:20 +08:00
erio
a9880ee7b9 fix: round-2 audit fixes — security, code quality, and UI improvements
Security (HIGH):
- Normalize all Redis cache keys to lowercase (verifyCode, passwordReset)
- Fix verify code TTL renewal on failed attempts: use remaining TTL via
  ExpiresAt field instead of resetting to full 15-minute window
- Add 3 missing fields to diffSettings audit log (promo_code, invitation_code,
  custom_endpoints)

Code quality (MEDIUM):
- Extract filterVerifiedEmails shared helper (balance_notify_service.go)
- Add Pricing array non-empty validation for channel pricing rules
- Add platform token semantics comment in gateway_service.go
- Complete validatePlanPatch test coverage (+10 test cases)
- Replace string types with QuotaThresholdType/QuotaResetMode across frontend
- Remove duplicate getPlatformTextColor/getRateBadgeClass in ChannelsView
- Return EMAIL_NOT_FOUND error on RemoveNotifyEmail miss

UI improvements:
- Reorder cost tooltip: user billing above separator, account billing below
- Add NaN guard to accountBilled function
- Move timezone selector inline into reset-mode row (no longer standalone)
2026-04-14 09:35:05 +08:00
erio
74f8a30f86 fix: address audit findings for websearch, email verification, and pricing
- Fix websearch provider failover: proxy error from provider-specific proxy
  now continues to next provider instead of aborting the entire loop
- Fix SMTP failure locking users out: send email first, then write cache
  and increment rate counter
- Fix notify email cache key case sensitivity: normalize to lowercase
- Add OriginalPrice validation to validatePlanPatch and validatePlanRequired
- Add empty scope validation for channel pricing rules (group_ids/account_ids)
- Add platform color to account search dropdown in channel pricing rules
2026-04-14 09:33:53 +08:00
erio
ed8a9d975b fix: batch 1 audit fixes — quota SQL fixed mode, public recharge URL, WebSearch bool fallback, UpdatePlan validation
H1: incrementUsageBillingAccountQuota now uses shared dailyExpiredExpr/weeklyExpiredExpr
    constants (supporting fixed reset mode) instead of hardcoded '24 hours'/'168 hours'
H4: public settings endpoint now maps balance_low_notify_recharge_url
H6: GetWebSearchEmulationMode tolerates legacy bool values (true→enabled)
H7: UpdatePlan validates non-nil patch fields (rejects negative price, empty name, etc.)
H8: UsageTable accountBilled() helper with total_cost ?? 0 null guard
H9: AdminUsageLog TS type adds channel_id + billing_tier
M2: account.go "fixed" literals replaced with thresholdTypeFixed constant
M13: SystemSettings TS type adds web_search_emulation_enabled
UI: QuotaLimitCard title labels now use flex-1 to align with flex-1 input boxes
2026-04-14 09:32:11 +08:00
erio
b7fb2e4387 fix: audit fixes for websearch, notifications, and channel pricing
P0: fix wildcard matching test assertion (config order, not longest prefix)
P0: add TotalRecharged to auth cache snapshot (v5) for percentage threshold
P1: move pricing rules into per-platform sections in ChannelsView
P1: populate account name cache when editing existing channel rules
P1: sanitize email subject headers to prevent SMTP injection
P1: make Redis INCR+EXPIRE idempotent for rate limiting
P1: deep copy FeaturesConfig in Channel.Clone()
P2: clean up stale email="" placeholder comments
P2: replace log.Printf with slog in email_service.go
2026-04-14 09:26:32 +08:00
erio
915b7a4a56 feat(notify): convert email lists to NotifyEmailEntry struct with toggle support
- Change balance_notify_extra_emails and account_quota_notify_emails
  from []string to []NotifyEmailEntry{email, disabled, verified}
- Add per-email enable/disable toggle for both user and admin notifications
- Add PUT /user/notify-email/toggle API endpoint
- Fix critical bug: API key auth cache snapshot missing balance notify
  fields (Email, Username, BalanceNotifyEnabled, etc.), causing
  notifications to never fire on cached request paths
- Bump cache snapshot version 3→4 to invalidate stale entries
- Add SQL migration 104 to convert old format data
- Backward compatible: parseNotifyEmails auto-detects old/new format
- User balance notify: max 3 emails (primary + 2 extra)
- Admin quota notify: unlimited emails, each with toggle
2026-04-14 09:26:07 +08:00
erio
4e96a6faec fix: address audit findings for notify, websearch and security
- Fix GetByKeyForAuth missing user.FieldEmail and user.FieldUsername (notifications sent to empty address)
- Guard against empty email in collectBalanceNotifyRecipients
- Remove non-atomic TotalRecharged read-modify-write in admin balance adjustment
- HTML-escape userName/siteName/accountName in notification email templates
- Fix timer leak in ProfileBalanceNotifyCard (add onUnmounted cleanup)
- Add warning log on websearch proxy URL resolution failure
2026-04-14 09:25:49 +08:00
erio
9e33d0c4c0 fix: address audit findings for websearch and balance notification
- Fix GetByKeyForAuth not selecting balance notify fields (notifications
  never triggered in gateway path)
- Fix provider-level ProxyURL never resolved: inject ProxyRepository into
  SettingService, resolve proxy URLs when building Manager
- Fix admin manual balance adjustment not updating total_recharged
- Add threshold_type input validation (reject invalid values)
- Fix user threshold_type inheritance: custom threshold defaults to "fixed"
  instead of inheriting global type (prevents $5 being treated as 5%)
- Add try-catch for clipboard.writeText (fails on non-HTTPS)
- Add SetTotalRecharged to user Update for admin balance operations
2026-04-14 09:24:58 +08:00
erio
f694afbbf4 feat(notify): add percentage threshold type for balance low notification
- Add threshold_type field (fixed/percentage) to system and user settings
- Add total_recharged field to users table, auto-incremented on balance credit
- Percentage mode: effective threshold = total_recharged × percentage / 100
- User-level threshold_type inherits from system default when not set
- Update admin settings UI with radio selector (fixed amount / percentage)
- Migration: 102_add_balance_notify_threshold_type.sql
2026-04-14 09:24:17 +08:00
erio
b32d1a2c9f feat(notify): add balance low & account quota notification system
- User balance low notification: email alert when balance drops below
  configurable threshold (user email + verified extra emails)
- Account quota notification: broadcast email to admin-configured
  recipients when daily/weekly/total quota usage exceeds alert threshold
- Admin settings: global enable/disable, default threshold, quota
  notification email list (Email Settings tab)
- User profile: enable/disable, custom threshold, add/remove extra
  notification emails with verification code flow
- Account quota: per-dimension alert toggle and threshold in quota
  control card
- Trigger logic: first-crossing only (old >= threshold && new < threshold
  for balance; old < threshold && new >= threshold for quota), naturally
  prevents duplicate notifications without Redis dedup
2026-04-14 09:23:02 +08:00
erio
7535e312e0 feat(channels): add custom account stats pricing rules
Allow channels to configure independent model pricing for account
statistics cost calculation, decoupled from user billing.

Backend:
- Migration 101: channels.apply_pricing_to_account_stats toggle,
  channel_account_stats_pricing_rules/model_pricing tables,
  usage_logs.account_stats_cost column
- resolveAccountStatsCost: match rules by group/account, then channel
  pricing, fallback to original formula when unconfigured
- Integrate into both GatewayService.recordUsageCore and
  OpenAIGatewayService.RecordUsage
- Update 8 account stats SQL queries to use
  COALESCE(account_stats_cost, total_cost) * account_rate_multiplier
- 23 unit tests for matching, pricing lookup, and cost calculation

Frontend:
- Channel edit dialog: toggle + custom rules UI with group/account
  multi-select and pricing entry cards
- API types and i18n (zh/en)
2026-04-14 09:22:12 +08:00
erio
1b53ffcac7 feat(gateway): add web search emulation for Anthropic API Key accounts
Inject web search capability for Claude Console (API Key) accounts that
don't natively support Anthropic's web_search tool. When a pure
web_search request is detected, the gateway calls Brave Search or Tavily
API directly and constructs an Anthropic-protocol-compliant SSE/JSON
response without forwarding to upstream.

Backend:
- New `pkg/websearch/` SDK: Brave and Tavily provider implementations
  with io.LimitReader, proxy support, and Redis-based quota tracking
  (Lua atomic INCR + TTL, DECR rollback on failure)
- Global config via `settings.web_search_emulation_config` (JSON) with
  in-process cache + singleflight, input validation, API key merge on
  save, and sanitized API responses
- Channel-level toggle via `channels.features_config` JSONB column
  (DB migration 101)
- Account-level toggle via `accounts.extra.web_search_emulation`
- Request interception in `Forward()` with SSE streaming response
  construction using json.Marshal (no manual string concatenation)
- Manager hot-reload: `RebuildWebSearchManager()` called on config save
  and startup via `SetWebSearchRedisClient()`
- 70 unit tests covering providers, manager, config validation,
  sanitization, tool detection, query extraction, and response building

Frontend:
- Settings → Gateway tab: Web Search Emulation config card with global
  toggle, provider list (add/remove, API key, priority, quota, proxy)
- Channels → Anthropic tab: web search emulation toggle with global
  state linkage (disabled when global off)
- Account Create/Edit modals: web search emulation toggle for API Key
  type with Toggle component
- Full i18n coverage (zh + en)
2026-04-14 09:20:39 +08:00
erio
1c63ea1448 fix(channel): add missing features column to List query
The paginated List query was selecting 9 columns but scanning 10 fields,
missing c.features. GetByID and ListAll already included it correctly.
2026-04-14 09:16:13 +08:00
erio
794e817208 refactor: remove PaymentChannel, reuse upstream Channel with features field
- Delete payment_channels table and PaymentChannel Ent schema
- Add `features` column to upstream channels table (migration 095)
- Add Features field to Channel struct, input types, handler request/response
- Payment user/admin handlers now use ChannelService directly
- Remove Channel CRUD from PaymentConfigService and admin payment routes
- Remove "渠道管理" tab from admin orders page (use /admin/channels)
2026-04-14 09:15:29 +08:00
erio
1cd033e521 style: apply gofmt formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 09:08:00 +08:00
IanShaw027
7dc7ff22d2 fix: preserve messages dispatch config in repository hydration 2026-04-10 18:13:18 +08:00
IanShaw027
269c7a065c fix(test): restore integration expectations 2026-04-09 22:08:42 +08:00
IanShaw027
b6946e78a2 fix(lint): restore repository sort helpers 2026-04-09 21:49:10 +08:00
IanShaw027
2b70d1d332 merge upstream main into fix/bug-cleanup-main 2026-04-09 21:35:48 +08:00
Wesley Liddick
00c08c574e Merge pull request #1539 from warksu/fix/loadfactor-not-synced-to-scheduler-cache
fix: 同步 LoadFactor 到调度快照缓存
2026-04-09 21:15:40 +08:00
IanShaw027
62962c05f1 fix(lint): 修复 CI 中的 ineffassign 和 unused 代码告警,修正 group 排序集成测试兼容性 2026-04-09 19:25:08 +08:00
swark2006
118ff85fbf fix: 同步 LoadFactor 到调度快照缓存
LoadFactor 字段在构建调度快照缓存时缺失,导致调度算法
EffectiveLoadFactor() 无法使用配置的负载因子,回退到 Concurrency。
这会影响账号的负载率计算,进而影响调度优先级。

修复:在 buildSchedulerMetadataAccount 中添加 LoadFactor 字段。
2026-04-09 18:50:11 +08:00
IanShaw027
5f8e60a1b7 feat(table): 表格排序与搜索改为后端处理 2026-04-09 18:14:28 +08:00
IanShaw027
23c4d592f8 feat(group): 增加messages调度模型映射配置 2026-04-09 12:29:28 +08:00
ius
265687b56d fix: 优化调度快照缓存以避免 Redis 大 MGET 2026-04-08 10:39:15 -07:00
Wesley Liddick
bf2b590273 Merge pull request #1397 from weak-fox/fix/active-filter-excludes-rate-limited
解决账号管理中“正常”筛选包含限流中账号
2026-04-07 08:26:51 +08:00
erio
19655a15f1 fix: gofmt formatting 2026-04-05 18:56:40 +08:00
erio
f345b0f595 fix: use upstream versions of shared files and remove only Sora code
Restore gateway_service.go, setting_handler.go, routes/admin.go,
dto/settings.go, group_repo.go, api_key_repo.go, wire_gen.go to
upstream/main versions and surgically remove only Sora references.

This preserves upstream-only features (RequireOauthOnly, RequirePrivacySet,
GroupResolution, etc.) that were missing when using release branch versions.
2026-04-05 18:48:41 +08:00
erio
c6089ccb33 fix: remove Sora DI from wire_gen.go and clean remaining upstream Sora references 2026-04-05 17:50:01 +08:00
erio
1b1c08f7fb fix: remove media_type from usage_log SQL queries and test stubs
- Remove media_type column from all INSERT/SELECT/SCAN in usage_log_repo
- Remove media_type mock arg from request_type and integration tests
- Adjust scan stub value arrays from 47 to 46 elements
2026-04-05 17:22:22 +08:00
erio
0c72be0403 fix: gofmt alignment and remove media_type from usage_log repo queries 2026-04-05 17:22:22 +08:00