Commit Graph

567 Commits

Author SHA1 Message Date
erio
8cf83c984e feat(channel-monitor): aggregate history to daily rollups + soft delete
明细只保留 1 天,超过 1 天聚合到新表 channel_monitor_daily_rollups(按
monitor_id/model/bucket_date 维度),聚合保留 30 天。两张表都用 SoftDeleteMixin
软删除(DELETE 自动改为 UPDATE deleted_at = NOW())。

聚合 + 清理任务由 OpsCleanupService 的 cron 统一调度,与运维监控的清理共享
schedule(默认 0 2 * * *)和 leader lock。ChannelMonitorRunner 的 cleanupLoop
被移除,只保留 dueCheckLoop。

读取路径 ComputeAvailability* 改为 UNION 明细(今天 deleted_at IS NULL)+
聚合(过去 windowDays 天 deleted_at IS NULL),SUM(ok)/SUM(total) 自然加权
计算可用率,AVG latency 用 SUM(sum_latency_ms)/SUM(count_latency)。

watermark 表 channel_monitor_aggregation_watermark 单行(id=1),记录
last_aggregated_date,重启后从该日期 +1 继续聚合,首次为 nil 则从
today - 30d 开始回填,单次最多 35 天上限避免长事务。

raw SQL 的 ListLatestPerModel / ListLatestForMonitorIDs / ListRecentHistoryForMonitors
都补上 deleted_at IS NULL 过滤(SoftDeleteMixin interceptor 只对 ent query 生效)。

bump version to 0.1.114.28

GroupBadge 在 MonitorKeyPickerDialog 中复用平台主题色 + 倍率/专属倍率
(顺手优化)。
2026-04-21 10:10:56 +08:00
erio
a1425b457d feat(channel-monitor): redesign user dashboard as card grid
Reference check-cx UI: INTELLIGENCE MONITOR hero + 3-column card grid
with 60-point timeline bars.

Backend:
- Add PrimaryPingLatencyMs + Timeline[60] to UserMonitorView
- ListRecentHistoryForMonitors: batch CTE + ROW_NUMBER() window query
- indexLatestByModel / indexAvailabilityByModel helpers

Frontend:
- 7 new components: ProviderIcon, MonitorMetricPair, MonitorAvailabilityRow,
  MonitorTimeline, MonitorHero, MonitorCard, MonitorCardGrid
- ChannelStatusView 381→~180 lines (delegated to subcomponents)
- AbortController reload concurrency protection
- HSL 0-120° availability color mapping
- Replace emoji with Icon component (bolt / globe)
- i18n: monitorCommon.* shared namespace, channelStatus.hero.*

Bump VERSION to 0.1.114.24
2026-04-20 23:38:59 +08:00
erio
20a4e41872 feat(monitor): admin channel monitor MVP with SSRF protection and batch aggregation
新增 admin「渠道监控」模块(参考 BingZi-233/check-cx),独立于现有 Channel 体系。
admin 配置 + 后台定时调用上游 LLM chat completions 健康检查 + 所有登录用户只读可见。

后端:
- ent: channel_monitor + channel_monitor_history(AES-256-GCM 加密 api_key)
- service 按职责拆分:service/aggregator/validate/checker/runner/ssrf
- provider strategy map 替代 switch(openai/anthropic/gemini)
- repository batch 聚合(ListLatestForMonitorIDs + ComputeAvailabilityForMonitors)消除 N+1
- runner: ticker(5s) + pond worker pool(5) + inFlight 防并发 + TrySubmit 防雪崩
  + 凌晨 3 点 cron 清理 30 天历史
- SSRF 防护:强制 https + 私网/loopback/云元数据 IP 拒绝(127/8、10/8、172.16/12、
  192.168/16、169.254/16、100.64/10、::1、fc00::/7、fe80::/10)+ DialContext
  在 socket 层防 DNS rebinding
- API key sanitize:擦除 url.Error 与上游响应 body 中的 sk-/sk-ant-/AIza/JWT 模式
- APIKeyDecryptFailed 标志位 + 单 monitor 路径检测,避免空 key 调用上游

handler:
- admin: CRUD + 手动触发 + 历史接口(api_key 脱敏)
- user: 只读列表 + 状态详情(去除 api_key/endpoint)
- ParseChannelMonitorID 共用 + dto.ChannelMonitorExtraModelStatus 共用

前端:
- 路由 /admin/channels/{pricing,monitor} + /monitor(用户只读)
- AppSidebar 父项 expandOnly 支持
- ChannelMonitorView 拆为 8 个子组件 + ChannelStatusView 拆出 detail dialog
- composables/useChannelMonitorFormat + constants/channelMonitor 共享
- i18n monitorCommon namespace 消除 admin/user 两 view 重复

合规:所有文件符合 CLAUDE.md(Go ≤ 500 行 / Vue ≤ 300 行 / 函数 ≤ 30 行)
CI: go build / gofmt / golangci-lint(0 issues) / make test-unit / pnpm build 全绿
2026-04-20 20:21:02 +08:00
shaw
45065c23d5 fix(ci): run 108a migration before 109 in backfill integration test 2026-04-22 18:36:44 +08:00
IanShaw027
9de7a72cce fix(upgrade): close payment and oidc compatibility gaps 2026-04-22 18:01:51 +08:00
IanShaw027
ad4600964e fix(ci): clean up lint and dead code 2026-04-22 16:38:36 +08:00
IanShaw027
36aed35957 fix(auth): harden oauth identity upgrade paths 2026-04-22 14:56:56 +08:00
IanShaw027
01a991f56f fix(test): restore identity repo integration imports 2026-04-22 13:22:33 +08:00
IanShaw027
81c827ee51 fix(profile): stabilize identity binding management 2026-04-22 13:19:28 +08:00
IanShaw027
06136af805 fix(upgrade): preserve legacy auth and payment compatibility 2026-04-22 13:18:10 +08:00
IanShaw027
1ffebbb568 fix(migrations): keep auth identity and payment upgrades safe 2026-04-22 12:29:52 +08:00
IanShaw027
18481a100b fix(migrations): defer online ddl follow-ups safely 2026-04-22 11:17:45 +08:00
IanShaw027
7fbd5177c2 fix(ci): make legacy migration cleanup resilient 2026-04-22 09:15:39 +08:00
IanShaw027
fdf72eb511 fix(ci): repair integration repository tests 2026-04-22 02:42:43 +08:00
IanShaw027
b13e34f831 fix(ci): align auth and payment verification tests 2026-04-22 02:32:53 +08:00
IanShaw027
525a320424 Fix user profile writes on postgres conflicts 2026-04-21 10:13:28 -07:00
IanShaw027
d4c0a99114 feat(auth): support unbinding third-party identities 2026-04-22 00:54:38 +08:00
IanShaw027
0d87f94cb7 Harden adoption decision reassignment 2026-04-21 09:53:15 -07:00
IanShaw027
da1d26001f Merge branch 'main' into rebuild/auth-identity-foundation 2026-04-22 00:35:34 +08:00
IanShaw027
d5819181ea feat(auth): reclaim stale identities and refresh profile UI 2026-04-21 07:49:40 -07:00
IanShaw027
d08757ce9e refactor(admin): remove auth migration reports 2026-04-21 17:34:18 +08:00
IanShaw027
c624cce88e fix: unblock auth identity compat backfill migration 2026-04-21 15:56:30 +08:00
IanShaw027
0a461d8248 fix: harden auth identity legacy migrations 2026-04-21 01:30:37 +08:00
IanShaw027
ea27ac6fd7 fix: unify email identity sync and retry first-bind defaults 2026-04-21 01:00:59 +08:00
IanShaw027
7a9488ff37 Add legacy identity safety remediation migration 2026-04-21 00:59:20 +08:00
IanShaw027
bf3ef2d19a add admin user last used support 2026-04-21 00:22:17 +08:00
IanShaw027
beeab54ae3 Implement latest-used user repo queries 2026-04-21 00:17:48 +08:00
IanShaw027
5d58c7c6fb Add auth identity legacy backfill and email sync 2026-04-21 00:13:40 +08:00
IanShaw027
31d0183d45 fix: normalize repository email lookups 2026-04-20 21:51:57 +08:00
IanShaw027
c6d8592484 feat: add profile auth identity binding flow 2026-04-20 18:28:44 +08:00
IanShaw027
e9de839d87 feat: rebuild auth identity foundation flow 2026-04-20 17:39:57 +08:00
erio
6579f28b64 fix: delete scheduled test plans when account is deleted
Accounts use soft-delete (setting deleted_at), so PostgreSQL's
ON DELETE CASCADE on scheduled_test_plans.account_id never fires.
Add plan deletion to the existing account deletion transaction
to ensure atomicity.

Closes Wei-Shaw/sub2api#1728
2026-04-19 20:38:57 +08:00
Wesley Liddick
e6e73b4f52 Merge pull request #1690 from KnowSky404/fix/ws-codex-scheduler-cache-1662
fix: preserve openai ws flags in scheduler cache
2026-04-16 17:21:32 +08:00
KnowSky404
3944b3d216 fix: preserve openai ws flags in scheduler cache 2026-04-16 02:01:50 +00:00
erio
db27e8f000 feat(usage): add account cost to breakdown sub-table and admin usage log
- UserBreakdownItem: add AccountCost field + SQL aggregation
- UserBreakdownSubTable: add orange account cost column
- Admin usage table: add account_cost column (after cost, default visible)
- Column settings: add account_cost toggle option
2026-04-15 15:40:40 +08:00
erio
22680dc602 test(usage): add unit tests for account_cost and fix gofmt
- Fix mock for GetModelStatsWithFilters: add account_cost column
- Add assertion: GetStatsWithFilters always returns TotalAccountCost
- New test: GetModelStatsAccountCostColumn verifies scan of AccountCost
- New test: GetGroupStatsAccountCostColumn verifies scan of AccountCost
- New test: GetStatsWithFiltersAlwaysReturnsAccountCost (no AccountID filter)
- Integration test: add TotalAccountCost/TodayAccountCost assertions
- Fix gofmt alignment in usage_log_types.go
2026-04-15 15:02:21 +08:00
erio
6ade6d30a8 feat(usage): add account cost display to admin dashboard and usage pages
- Add account_cost column to dashboard aggregation tables (migration 107)
- DashboardStats: add TotalAccountCost/TodayAccountCost fields
- ModelStat/GroupStat: add AccountCost field with SQL aggregation
- GetStatsWithFilters: always return TotalAccountCost (remove accountID filter)
- Dashboard Token cards: show user(green)/cost(orange)/standard(gray)
- Usage stats card: show account cost and standard below main value
- Model/Group distribution tables: add orange cost column
2026-04-15 15:02:21 +08:00
erio
58677dd53f fix: merge 5 PR-related improvements
- gateway_handler: pass ParsedRequest to RecordUsage + set in gin.Context
- channel_handler: add FeaturesConfig to CRUD (WebSearch channel toggle)
- channel_repo: features_config JSONB persistence (Create/Get/Update/List)
- security_headers: add Stripe CSP domains (script-src + frame-src)
2026-04-14 18:34:57 +08:00
erio
6ac8ccde46 fix: merge 30 general improvements from release branch
Bug fixes:
- Detached context for GetAccountConcurrencyBatch (prevent all-zero on request cancel)
- Filter soft-deleted users in GetByGroupID
- Stripe CSP policy (allow Stripe.js in script-src and frame-src)
- WebSearch API key validation on save
- RECHARGING status in payment result success check
- Windows test fixes (logger Sync deadlock, config path escaping)

Feature enhancements:
- Webhook multi-instance dispatch (extractOutTradeNo + GetWebhookProvider)
- EasyPay mobile H5 payment (device param + PayURL2)
- SSE error propagation in WebSearch emulation
- AccountStatsCost DTO field for admin usage logs
- Plans sort by sort_order instead of created_at
- UsageMapHook for streaming response usage data
- apicompat Instructions field passthrough
- EffectiveLoadFactor for ops concurrency/metrics
- Usage billing RETURNING balance for notify system
- BulkUpdate mixed channel warning with details
- println to slog migration in auth cache
- Wire ProviderSet cleanup
- CI cache-dependency-path optimization

Frontend:
- Refund eligibility check per provider (canRequestRefund)
- Plan sort_order editing
- Dead code cleanup (simulate_claude_max, client_affinity)
- GroupsView platform switch guard
- channels features_config API type
- UsageView account_stats_cost export
2026-04-14 17:35:27 +08:00
erio
d6965b0676 fix: resolve cherry-pick conflicts and restore compilation
- Restore gateway_cache.go to upstream (no lua embeds)
- Restore payment_order.go to upstream (use out_trade_no lookup)
- Restore payment_fulfillment.go to upstream (same reason)
- Add FeaturesConfig field and IsWebSearchEmulationEnabled to Channel
- Add applyAccountStatsCost wrapper function
- Add SettingKeyWebSearchEmulationConfig constant
- Add WebSearchEmulationEnabled to SystemSettings
- Add notify code rate limiting methods to EmailCache interface
- Remove AllowUserRefund references (ent schema not present)
- Fix duplicate import in payment_handler.go
- Fix wire_gen.go argument mismatches
2026-04-14 10:18:39 +08:00
erio
0a4ece5f5b fix: audit round-3 — proxy safety, intervals persistence, SMTP timeout, sort fix
- Skip websearch provider when ProxyID is set but proxy not found (prevent
  silent direct connection bypass)
- Fix sortByStableRandomWeight: pair factors with items so sort.Slice swap
  keeps weights aligned
- Allow empty platform in account_stats_pricing_rules (wildcard matching),
  only force anthropic default for main model_pricing
- Add channel_account_stats_pricing_intervals table and repo layer support
  for interval-based pricing in account stats rules
- calculateTokenStatsCost now uses interval pricing when available
- Replace smtp.SendMail/tls.Dial with net.Dialer timeout (10s dial, 20s IO)
  to prevent goroutine leak on SMTP hang
- Fix gofmt formatting issues
- Web Search label: black text with red warning hint
2026-04-14 09:35:20 +08:00
erio
a9880ee7b9 fix: round-2 audit fixes — security, code quality, and UI improvements
Security (HIGH):
- Normalize all Redis cache keys to lowercase (verifyCode, passwordReset)
- Fix verify code TTL renewal on failed attempts: use remaining TTL via
  ExpiresAt field instead of resetting to full 15-minute window
- Add 3 missing fields to diffSettings audit log (promo_code, invitation_code,
  custom_endpoints)

Code quality (MEDIUM):
- Extract filterVerifiedEmails shared helper (balance_notify_service.go)
- Add Pricing array non-empty validation for channel pricing rules
- Add platform token semantics comment in gateway_service.go
- Complete validatePlanPatch test coverage (+10 test cases)
- Replace string types with QuotaThresholdType/QuotaResetMode across frontend
- Remove duplicate getPlatformTextColor/getRateBadgeClass in ChannelsView
- Return EMAIL_NOT_FOUND error on RemoveNotifyEmail miss

UI improvements:
- Reorder cost tooltip: user billing above separator, account billing below
- Add NaN guard to accountBilled function
- Move timezone selector inline into reset-mode row (no longer standalone)
2026-04-14 09:35:05 +08:00
erio
74f8a30f86 fix: address audit findings for websearch, email verification, and pricing
- Fix websearch provider failover: proxy error from provider-specific proxy
  now continues to next provider instead of aborting the entire loop
- Fix SMTP failure locking users out: send email first, then write cache
  and increment rate counter
- Fix notify email cache key case sensitivity: normalize to lowercase
- Add OriginalPrice validation to validatePlanPatch and validatePlanRequired
- Add empty scope validation for channel pricing rules (group_ids/account_ids)
- Add platform color to account search dropdown in channel pricing rules
2026-04-14 09:33:53 +08:00
erio
ed8a9d975b fix: batch 1 audit fixes — quota SQL fixed mode, public recharge URL, WebSearch bool fallback, UpdatePlan validation
H1: incrementUsageBillingAccountQuota now uses shared dailyExpiredExpr/weeklyExpiredExpr
    constants (supporting fixed reset mode) instead of hardcoded '24 hours'/'168 hours'
H4: public settings endpoint now maps balance_low_notify_recharge_url
H6: GetWebSearchEmulationMode tolerates legacy bool values (true→enabled)
H7: UpdatePlan validates non-nil patch fields (rejects negative price, empty name, etc.)
H8: UsageTable accountBilled() helper with total_cost ?? 0 null guard
H9: AdminUsageLog TS type adds channel_id + billing_tier
M2: account.go "fixed" literals replaced with thresholdTypeFixed constant
M13: SystemSettings TS type adds web_search_emulation_enabled
UI: QuotaLimitCard title labels now use flex-1 to align with flex-1 input boxes
2026-04-14 09:32:11 +08:00
erio
b7fb2e4387 fix: audit fixes for websearch, notifications, and channel pricing
P0: fix wildcard matching test assertion (config order, not longest prefix)
P0: add TotalRecharged to auth cache snapshot (v5) for percentage threshold
P1: move pricing rules into per-platform sections in ChannelsView
P1: populate account name cache when editing existing channel rules
P1: sanitize email subject headers to prevent SMTP injection
P1: make Redis INCR+EXPIRE idempotent for rate limiting
P1: deep copy FeaturesConfig in Channel.Clone()
P2: clean up stale email="" placeholder comments
P2: replace log.Printf with slog in email_service.go
2026-04-14 09:26:32 +08:00
erio
915b7a4a56 feat(notify): convert email lists to NotifyEmailEntry struct with toggle support
- Change balance_notify_extra_emails and account_quota_notify_emails
  from []string to []NotifyEmailEntry{email, disabled, verified}
- Add per-email enable/disable toggle for both user and admin notifications
- Add PUT /user/notify-email/toggle API endpoint
- Fix critical bug: API key auth cache snapshot missing balance notify
  fields (Email, Username, BalanceNotifyEnabled, etc.), causing
  notifications to never fire on cached request paths
- Bump cache snapshot version 3→4 to invalidate stale entries
- Add SQL migration 104 to convert old format data
- Backward compatible: parseNotifyEmails auto-detects old/new format
- User balance notify: max 3 emails (primary + 2 extra)
- Admin quota notify: unlimited emails, each with toggle
2026-04-14 09:26:07 +08:00
erio
4e96a6faec fix: address audit findings for notify, websearch and security
- Fix GetByKeyForAuth missing user.FieldEmail and user.FieldUsername (notifications sent to empty address)
- Guard against empty email in collectBalanceNotifyRecipients
- Remove non-atomic TotalRecharged read-modify-write in admin balance adjustment
- HTML-escape userName/siteName/accountName in notification email templates
- Fix timer leak in ProfileBalanceNotifyCard (add onUnmounted cleanup)
- Add warning log on websearch proxy URL resolution failure
2026-04-14 09:25:49 +08:00
erio
9e33d0c4c0 fix: address audit findings for websearch and balance notification
- Fix GetByKeyForAuth not selecting balance notify fields (notifications
  never triggered in gateway path)
- Fix provider-level ProxyURL never resolved: inject ProxyRepository into
  SettingService, resolve proxy URLs when building Manager
- Fix admin manual balance adjustment not updating total_recharged
- Add threshold_type input validation (reject invalid values)
- Fix user threshold_type inheritance: custom threshold defaults to "fixed"
  instead of inheriting global type (prevents $5 being treated as 5%)
- Add try-catch for clipboard.writeText (fails on non-HTTPS)
- Add SetTotalRecharged to user Update for admin balance operations
2026-04-14 09:24:58 +08:00
erio
f694afbbf4 feat(notify): add percentage threshold type for balance low notification
- Add threshold_type field (fixed/percentage) to system and user settings
- Add total_recharged field to users table, auto-incremented on balance credit
- Percentage mode: effective threshold = total_recharged × percentage / 100
- User-level threshold_type inherits from system default when not set
- Update admin settings UI with radio selector (fixed amount / percentage)
- Migration: 102_add_balance_notify_threshold_type.sql
2026-04-14 09:24:17 +08:00
erio
b32d1a2c9f feat(notify): add balance low & account quota notification system
- User balance low notification: email alert when balance drops below
  configurable threshold (user email + verified extra emails)
- Account quota notification: broadcast email to admin-configured
  recipients when daily/weekly/total quota usage exceeds alert threshold
- Admin settings: global enable/disable, default threshold, quota
  notification email list (Email Settings tab)
- User profile: enable/disable, custom threshold, add/remove extra
  notification emails with verification code flow
- Account quota: per-dimension alert toggle and threshold in quota
  control card
- Trigger logic: first-crossing only (old >= threshold && new < threshold
  for balance; old < threshold && new >= threshold for quota), naturally
  prevents duplicate notifications without Redis dedup
2026-04-14 09:23:02 +08:00