Commit Graph

2056 Commits

Author SHA1 Message Date
erio
44cdef7934 fix(usage): subscription billing honours group rate multiplier
Subscription-mode billing was consuming quota at TotalCost (raw) instead of
ActualCost (TotalCost * RateMultiplier), so per-group rate multipliers —
including free subscriptions (multiplier = 0) — were silently ignored.
Switch the three subscription cost writes in buildUsageBillingCommand,
finalizePostUsageBilling, and the legacy postUsageBilling fallback to
ActualCost, and add a table-driven test covering 2x / 0.5x / free multipliers
plus a balance-mode regression check.
2026-04-17 22:06:32 +08:00
erio
fd0c9a1305 fix(payment): store provider config as plaintext JSON with legacy ciphertext fallback
Without TOTP_ENCRYPTION_KEY, saved payment configs were lost on restart because
the AES round-trip failed silently. Write new records as plaintext JSON; read
path tries JSON first, falls back to legacy AES decrypt when a key is present,
and treats unreadable values as empty so admins can re-enter them via the UI.
2026-04-17 22:06:32 +08:00
github-actions[bot]
6cfdf4ec05 chore: sync VERSION to 0.1.114 [skip ci] 2026-04-17 02:51:18 +00:00
Wesley Liddick
358ff6a608 Merge pull request #1683 from FjlI5/dev-main
fix:修复上游账号为OpenAI API key时Claude Code调用缓存率低的问题
2026-04-17 10:28:12 +08:00
Wesley Liddick
41fbdba104 Merge pull request #1687 from touwaeriol/refactor/upstream-response-limit-dedup
refactor: extract ReadUpstreamResponseBody to deduplicate response read + too-large handling
2026-04-17 10:19:14 +08:00
Wesley Liddick
c22d11cedd Merge pull request #1702 from StarryKira/fix/outbox-watermark-context-dedup-1691
fix: fix outbox watermark context expiry and add in-batch group rebuild dedup
2026-04-17 10:18:56 +08:00
shaw
5d586a9f3a fix: 上游返回 KYC 身份验证要求时停止账号调度 2026-04-17 10:17:50 +08:00
shaw
a789c8c4c7 feat: 支持opus-4.7 2026-04-17 09:37:25 +08:00
Elysia
697c41a3f6 fix: create fresh context per watermark write retry attempt
Each retry in the SetOutboxWatermark loop now gets its own 5s context.
Previously a shared context could already be expired when the second or
third attempt ran, making the retries pointless.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 20:41:40 +08:00
Elysia
e44baa1094 fix: fix outbox watermark context expiry and add in-batch group rebuild dedup
Fixes #1691

- pollOutbox() reused a 10s context for SetOutboxWatermark after event
  processing could take much longer, causing "outbox watermark write
  failed: context deadline exceeded". The watermark never advanced so
  the same 200 events were reprocessed every poll cycle, spiking CPU.
  Now uses an independent 5s context with up to 3 retries (200ms apart).

- When multiple Codex accounts sharing the same 21-22 groups are all
  rate-limited in quick succession, each account_changed event triggered
  redundant bucket rebuild attempts for the same groups. Introduce
  batchSeenKey{groupID, platform} and thread a seen map through the
  handler chain; rebuildBucketsForPlatform skips (group, platform) pairs
  already rebuilt within the same poll batch (~80% fewer rebuild calls in
  the 5-accounts-same-groups scenario).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 19:09:40 +08:00
Wesley Liddick
e6e73b4f52 Merge pull request #1690 from KnowSky404/fix/ws-codex-scheduler-cache-1662
fix: preserve openai ws flags in scheduler cache
2026-04-16 17:21:32 +08:00
KnowSky404
3944b3d216 fix: preserve openai ws flags in scheduler cache 2026-04-16 02:01:50 +00:00
erio
10699eeb34 refactor: extract ReadUpstreamResponseBody to deduplicate upstream response read + too-large error handling
Consolidates 9 call sites of resolveUpstreamResponseReadLimit + readUpstreamResponseBodyLimited + ErrUpstreamResponseBodyTooLarge error handling into a single ReadUpstreamResponseBody function with TooLargeWriter callback for API-format-specific error responses (Anthropic, OpenAI, countTokens).
2026-04-16 01:53:22 +08:00
fjl5
6c89d8d35c add prompt_cache_key injection for messages→responses 2026-04-15 23:56:56 +08:00
github-actions[bot]
be7551b9f4 chore: sync VERSION to 0.1.113 [skip ci] 2026-04-15 09:34:24 +00:00
Wesley Liddick
70d0569f08 Merge pull request #1668 from tyqy12/main
修复 OpenAI 账号限流回流误判:7d 窗口可用时不因 5h 窗口为 0 回写 429
2026-04-15 16:48:48 +08:00
Wesley Liddick
1db32d692b Merge pull request #1666 from touwaeriol/feat/account-cost-display
feat(usage): add account cost display to admin dashboard and usage pages
2026-04-15 16:43:07 +08:00
erio
db27e8f000 feat(usage): add account cost to breakdown sub-table and admin usage log
- UserBreakdownItem: add AccountCost field + SQL aggregation
- UserBreakdownSubTable: add orange account cost column
- Admin usage table: add account_cost column (after cost, default visible)
- Column settings: add account_cost toggle option
2026-04-15 15:40:40 +08:00
Wesley Liddick
7451b6f9ae 修复 OpenAI 账号限流回流误判:7d 窗口可用时不因 5h 窗口为 0 回写 429 2026-04-15 15:29:52 +08:00
erio
22680dc602 test(usage): add unit tests for account_cost and fix gofmt
- Fix mock for GetModelStatsWithFilters: add account_cost column
- Add assertion: GetStatsWithFilters always returns TotalAccountCost
- New test: GetModelStatsAccountCostColumn verifies scan of AccountCost
- New test: GetGroupStatsAccountCostColumn verifies scan of AccountCost
- New test: GetStatsWithFiltersAlwaysReturnsAccountCost (no AccountID filter)
- Integration test: add TotalAccountCost/TodayAccountCost assertions
- Fix gofmt alignment in usage_log_types.go
2026-04-15 15:02:21 +08:00
erio
6ade6d30a8 feat(usage): add account cost display to admin dashboard and usage pages
- Add account_cost column to dashboard aggregation tables (migration 107)
- DashboardStats: add TotalAccountCost/TodayAccountCost fields
- ModelStat/GroupStat: add AccountCost field with SQL aggregation
- GetStatsWithFilters: always return TotalAccountCost (remove accountID filter)
- Dashboard Token cards: show user(green)/cost(orange)/standard(gray)
- Usage stats card: show account cost and standard below main value
- Model/Group distribution tables: add orange cost column
2026-04-15 15:02:21 +08:00
erio
c2108421c2 fix: gofmt payment_service.go and payment_order.go 2026-04-15 01:50:19 +08:00
erio
342dbd2e19 fix(payment): use original recharge amount in product name, not pay_amount
Product name (e.g. "快代码科技工作室 100 元") should show the user's
original recharge amount (limitAmount), not the fee-inclusive pay amount.
The gateway receives payAmount separately for actual charging.
2026-04-15 01:43:56 +08:00
erio
60614e6f74 fix: gofmt formatting and update API contract test for new fields
- Fix gofmt alignment in setting_handler.go, settings.go, payment_config_service.go
- Add payment_balance_recharge_multiplier and payment_recharge_fee_rate
  to API contract test expected JSON
2026-04-15 01:39:00 +08:00
erio
d149dbc91f fix(payment): enhance fee rate input validation and UI
Backend:
- Validate recharge_fee_rate: 0 ≤ rate ≤ 100, max 2 decimal places

Frontend settings:
- Add % suffix icon to fee rate input
- Enforce max=100, min=0, step=0.01 with 2 decimal precision
2026-04-15 01:27:24 +08:00
erio
e761d38fd1 fix(payment): integrate recharge fee rate in order flow and fix UI display
Backend:
- Use cfg.RechargeFeeRate in order creation instead of hardcoded 0
- Remove dead getFeeRate stub method
- All amounts computed server-side: order_amount, pay_amount, fee_rate

Frontend - PaymentView:
- Read recharge_fee_rate from checkout-info API (not per-method)
- Show fee breakdown only when fee_rate > 0
- Show credited amount only when multiplier ≠ 1

Frontend - Order display (user + admin):
- Fix fee_rate * 100 bug (fee_rate is already a percentage)
- OrderTable: show pay_amount as primary, fee/credited as sub-lines
- AdminOrderDetail: full breakdown (base/fee/paid/credited)
- AdminRefundDialog: label "到账金额" for clarity
- PaymentResultView: show pay_amount with fee info

Types + i18n:
- Add recharge_fee_rate to CheckoutInfoResponse
- Add fee_rate to CreateOrderResult
- Add translations: creditedAmount, fee, baseAmount, includedInPayAmount
2026-04-15 01:27:24 +08:00
erio
98140f6cac feat(payment): add recharge fee rate setting and fix provider card UI
- Add recharge_fee_rate system setting (percentage fee on top of recharge amount)
- Full backend chain: config constant, PaymentConfig struct, update validation,
  read/write persistence, DTO, handler GET/PUT responses
- Frontend: settings input with preview, i18n (zh/en), API types
- Fix provider card toggle layout: labels above switches to save width
- Fix Chinese translation: "EasyPay" → "易支付" in provider description
2026-04-15 01:27:24 +08:00
erio
60a4b9316b feat(payment): balance recharge multiplier and refund amount separation
- Add balance_recharge_multiplier system setting (e.g. 1.2 = charge 100 get 120)
- Separate order_amount (credited balance) from pay_amount (actual payment)
- Refund calculates gateway amount proportionally from pay_amount
- Frontend shows both amounts in order details, payment status, refund dialog
- Admin settings UI for configuring recharge multiplier
2026-04-15 01:27:24 +08:00
erio
8548a130c7 fix: Messages() routing refactor and subscription group test coverage
- Refactor OpenAI Messages() routing: pre-compute dispatch model using
  resolveOpenAIMessagesDispatchMappedModel + NormalizeOpenAICompatRequestedModel
  instead of try-fail-retry pattern with gin context passing
- Remove openai_messages_fallback_model context anti-pattern
- Use effectiveMappedModel directly for forward default mapped model
- Add 3 subscription group tests covering all branch paths:
  _Blocked (no active subscription → SUBSCRIPTION_REQUIRED),
  _RequiresRepo (nil repo → SUBSCRIPTION_REPOSITORY_UNAVAILABLE),
  _AllowsActiveSubscription (valid subscription → success)
2026-04-14 20:34:53 +08:00
erio
3d2027227b fix: update wire_gen.go to use ProvideSchedulerCache with config injection
wire_gen.go was calling NewSchedulerCache(redisClient) but wire.go had
been updated to register ProvideSchedulerCache(redisClient, config),
which reads SnapshotMGetChunkSize and SnapshotWriteChunkSize from config.
Without this fix, those config values were silently ignored.
2026-04-14 20:22:45 +08:00
erio
3fa5b8bca5 fix: flaky WebSocket test, usage request queue, and test improvements
- Fix flaky WebSocket passthrough test: allow StatusNormalClosure after
  client close instead of requiring NoError (race condition fix)
- Fix ratelimit 401 test: use PlatformOpenAI instead of PlatformGemini
  for OAuth token cache invalidation scenario (more accurate)
- Add usageLoadQueue: Anthropic OAuth/setup-token accounts sharing the
  same proxy exit are serialized with 1-2s jitter to prevent upstream 429
- AccountUsageCell: add module-level usage cache (5min TTL), unmounted
  safety guard, and integrate enqueueUsageRequest for throttled fetching
2026-04-14 20:13:59 +08:00
erio
63f539b382 fix: merge general improvements from release branch
Backend:
- gateway_handler: pass subject.UserID instead of int64(0) for user-level routing
- setting_handler: add missing BalanceLowNotifyRechargeURL to UpdateSettings response
- openai_gateway_service: use applyAccountStatsCost for account stats pricing integration
- embed_on: add local file override (data/public/) for embedded frontend assets

Frontend:
- useTableSelection: add batchUpdate method for batch operations
- AccountsView: virtual scrolling params, Set-based isSelected, swipe virtualization
- ProxiesView: add batchUpdate to selection and swipe-select
- BulkEditAccountModal: fix submit handler to prevent event object as argument
- SettingsView: move payload construction outside try block
- i18n: add general translation keys (saved, deleted, view, validation, allowUserRefund)
- api/client: reorder error fields for consistency
- stores/payment: clarify pollOrderStatus JSDoc
2026-04-14 19:29:37 +08:00
erio
c14d739360 fix: resolve 3 code review issues in allow_user_refund
1. PrepareRefund: block refund on provider instance lookup failure
   instead of silently skipping permission check (medium severity)

2. UpdateProviderInstance: allow enabling refund_enabled and
   allow_user_refund in the same request by checking req.RefundEnabled
   value before falling back to DB read

3. ExecuteRefund: only revoke subscription on ErrAdjustWouldExpire,
   abort on other errors (DB failure, not found) instead of
   unconditionally revoking
2026-04-14 18:41:09 +08:00
erio
58677dd53f fix: merge 5 PR-related improvements
- gateway_handler: pass ParsedRequest to RecordUsage + set in gin.Context
- channel_handler: add FeaturesConfig to CRUD (WebSearch channel toggle)
- channel_repo: features_config JSONB persistence (Create/Get/Update/List)
- security_headers: add Stripe CSP domains (script-src + frame-src)
2026-04-14 18:34:57 +08:00
erio
6ac8ccde46 fix: merge 30 general improvements from release branch
Bug fixes:
- Detached context for GetAccountConcurrencyBatch (prevent all-zero on request cancel)
- Filter soft-deleted users in GetByGroupID
- Stripe CSP policy (allow Stripe.js in script-src and frame-src)
- WebSearch API key validation on save
- RECHARGING status in payment result success check
- Windows test fixes (logger Sync deadlock, config path escaping)

Feature enhancements:
- Webhook multi-instance dispatch (extractOutTradeNo + GetWebhookProvider)
- EasyPay mobile H5 payment (device param + PayURL2)
- SSE error propagation in WebSearch emulation
- AccountStatsCost DTO field for admin usage logs
- Plans sort by sort_order instead of created_at
- UsageMapHook for streaming response usage data
- apicompat Instructions field passthrough
- EffectiveLoadFactor for ops concurrency/metrics
- Usage billing RETURNING balance for notify system
- BulkUpdate mixed channel warning with details
- println to slog migration in auth cache
- Wire ProviderSet cleanup
- CI cache-dependency-path optimization

Frontend:
- Refund eligibility check per provider (canRequestRefund)
- Plan sort_order editing
- Dead code cleanup (simulate_claude_max, client_affinity)
- GroupsView platform switch guard
- channels features_config API type
- UsageView account_stats_cost export
2026-04-14 17:35:27 +08:00
erio
f1297a3694 feat: add per-provider allow_user_refund control and align wildcard matching
allow_user_refund:
- Add allow_user_refund field to PaymentProviderInstance ent schema
- Migration 103: ALTER TABLE payment_provider_instances ADD COLUMN
- Cascade logic: disabling refund_enabled auto-disables allow_user_refund
- User refund validation: check provider instance allows user refund
- Admin refund validation: check provider instance allows admin refund
- Subscription refund: deduct days on refund, rollback on failure
- New endpoint: GET /payment/orders/refund-eligible-providers
- Frontend: ToggleSwitch in ProviderCard/Dialog, cascade in SettingsView

Wildcard matching:
- Change findPricingForModel from "longest prefix wins" to "config order
  priority (first match wins)", aligning with channel service behavior
2026-04-14 16:26:46 +08:00
erio
e8ee400a3f fix: resolve remaining lint errors for upstream CI
- Fix errcheck: brave.go resp.Body.Close, manager_test.go Encode
- Fix gofmt: payment_config_service.go
- Fix unused: use shouldFallbackGeminiModel (with modelName param) in handler
2026-04-14 12:19:44 +08:00
erio
6a08efeef9 fix: resolve upstream CI failures (lint, test, gofmt)
- Fix errcheck: handle Write/Encode return values in brave_test.go
- Fix errcheck: defer resp.Body.Close() with _ assignment in tavily.go
- Fix gofmt: payment.go, channel.go, payment_config_providers.go
- Fix unused: remove dead decodeURLValue in easypay.go
- Restore shouldFallbackGeminiModel function (deleted during cherry-pick)
- Add missing balanceNotifyService param to NewGatewayService in test
- Fix platform default test expectation (empty stays empty)
- Fix wildcard pricing test (longest prefix wins, not config order)
- Fix subscription group test (SUBSCRIPTION_REPOSITORY_UNAVAILABLE)
2026-04-14 12:11:08 +08:00
erio
4aa0070e3d fix: Stripe payment type matching in load balancer
Checkout page aggregates Stripe sub-types (card,link,alipay,wxpay) under
"stripe", but SelectInstance matched against supported_types literally,
which doesn't contain "stripe". Now matches by provider_key for Stripe.
2026-04-14 11:31:44 +08:00
erio
b42f34c359 fix: resolve test compilation errors and restore upstream VERSION
- Add missing interface methods to test stubs (RemoveGroupFromUserAllowedGroups,
  GetNotifyCodeUserRate, IncrNotifyCodeUserRate, UpdateGroupIDByUserAndGroup)
- Fix NewUserService call signatures (add 4th param)
- Fix GetAccountCount return signature (3 values)
- Update api_contract_test.go snapshots for balance_notify fields
- Restore resolveOpenAIMessagesDispatchMappedModel function
- Reset VERSION to upstream 0.1.112
2026-04-14 11:27:32 +08:00
erio
24e16b7f59 fix: restore resolveOpenAIMessagesDispatchMappedModel and reset VERSION
- Restore function deleted during cherry-pick conflict resolution
- Reset VERSION to upstream 0.1.112
2026-04-14 10:58:51 +08:00
erio
d6965b0676 fix: resolve cherry-pick conflicts and restore compilation
- Restore gateway_cache.go to upstream (no lua embeds)
- Restore payment_order.go to upstream (use out_trade_no lookup)
- Restore payment_fulfillment.go to upstream (same reason)
- Add FeaturesConfig field and IsWebSearchEmulationEnabled to Channel
- Add applyAccountStatsCost wrapper function
- Add SettingKeyWebSearchEmulationConfig constant
- Add WebSearchEmulationEnabled to SystemSettings
- Add notify code rate limiting methods to EmailCache interface
- Remove AllowUserRefund references (ent schema not present)
- Fix duplicate import in payment_handler.go
- Fix wire_gen.go argument mismatches
2026-04-14 10:18:39 +08:00
erio
9028d2085f test: add unit tests for billing, websearch, and notify systems
Billing (25 tests):
- CalculateCostUnified: nil resolver fallback, token/per_request/image modes
- GetModelPricingWithChannel: nil/partial/full channel overrides
- resolveAccountStatsCost: four-level priority chain integration tests

WebSearch (18 tests):
- PopulateWebSearchUsage: nil input, manager states, QuotaLimit nil/*int64
- ResetWebSearchUsage: nil manager error
- Manager.ResetUsage: nil Redis
- shouldEmulateWebSearch: full decision chain (8 scenarios)

Notify (36 tests):
- ParseNotifyEmails/MarshalNotifyEmails: old/new format, roundtrip
- crossedDownward: boundary values, threshold semantics
- checkQuotaDimCrossings: mixed dimensions, disabled/zero skip
2026-04-14 09:36:40 +08:00
erio
7c7292935e feat: websearch quota enhancements and balance notify hint
- QuotaLimit changed to *int64 (null=unlimited, >0=limited)
- Add reset-usage endpoint (POST /admin/settings/web-search-emulation/reset-usage)
- Show quota usage in header always (collapsed and expanded)
- Add reset quota button in expanded provider view
- Quota input: empty=unlimited with ∞ placeholder, must be >0 if set
- Add email verification hint on balance notify card
2026-04-14 09:36:40 +08:00
erio
1e6912ea2e fix: gofmt formatting across all Go source files 2026-04-14 09:36:26 +08:00
erio
9e0d12d3b0 fix: show websearch API key visibility/copy buttons for saved providers
The buttons were hidden because v-if only checked provider.api_key,
which is always empty for saved providers (backend sanitizes it).
Now also checks api_key_configured. Copy button is disabled when
no actual key is available (only configured placeholder shown).
2026-04-14 09:35:46 +08:00
erio
b402c367d3 fix: add opportunistic STARTTLS to sendMailPlain for 587 port compatibility
smtp.SendMail automatically upgrades to STARTTLS when the server
supports it. Our replacement sendMailPlain skipped this, causing
credentials to be sent in plaintext on port 587. Add STARTTLS
negotiation before Auth to restore the original security behavior.
2026-04-14 09:35:21 +08:00
erio
0a4ece5f5b fix: audit round-3 — proxy safety, intervals persistence, SMTP timeout, sort fix
- Skip websearch provider when ProxyID is set but proxy not found (prevent
  silent direct connection bypass)
- Fix sortByStableRandomWeight: pair factors with items so sort.Slice swap
  keeps weights aligned
- Allow empty platform in account_stats_pricing_rules (wildcard matching),
  only force anthropic default for main model_pricing
- Add channel_account_stats_pricing_intervals table and repo layer support
  for interval-based pricing in account stats rules
- calculateTokenStatsCost now uses interval pricing when available
- Replace smtp.SendMail/tls.Dial with net.Dialer timeout (10s dial, 20s IO)
  to prevent goroutine leak on SMTP hang
- Fix gofmt formatting issues
- Web Search label: black text with red warning hint
2026-04-14 09:35:20 +08:00
erio
9c09bd19b4 fix: websearch features_config cleanup and pricing rules validation
- Fix web_search_emulation toggle: explicitly write false for disabled
  platforms instead of leaving stale true from cloned features_config
- Extract validatePricingEntries from validateChannelConfig for reuse
- Validate account_stats_pricing_rules[].pricing in both Create and
  Update paths (negative prices, bad intervals, missing per_request price)
2026-04-14 09:35:20 +08:00
erio
a9880ee7b9 fix: round-2 audit fixes — security, code quality, and UI improvements
Security (HIGH):
- Normalize all Redis cache keys to lowercase (verifyCode, passwordReset)
- Fix verify code TTL renewal on failed attempts: use remaining TTL via
  ExpiresAt field instead of resetting to full 15-minute window
- Add 3 missing fields to diffSettings audit log (promo_code, invitation_code,
  custom_endpoints)

Code quality (MEDIUM):
- Extract filterVerifiedEmails shared helper (balance_notify_service.go)
- Add Pricing array non-empty validation for channel pricing rules
- Add platform token semantics comment in gateway_service.go
- Complete validatePlanPatch test coverage (+10 test cases)
- Replace string types with QuotaThresholdType/QuotaResetMode across frontend
- Remove duplicate getPlatformTextColor/getRateBadgeClass in ChannelsView
- Return EMAIL_NOT_FOUND error on RemoveNotifyEmail miss

UI improvements:
- Reorder cost tooltip: user billing above separator, account billing below
- Add NaN guard to accountBilled function
- Move timezone selector inline into reset-mode row (no longer standalone)
2026-04-14 09:35:05 +08:00