Commit Graph

307 Commits

Author SHA1 Message Date
erio
eb385457b2 fix(channel): 全平台渠道映射覆盖 + 公共函数抽取 + 死代码清理
- 4个缺失handler入口添加渠道映射+限制检查(ChatCompletions/Responses/Gemini)
- 模型限制错误信息优化,区分"模型不可用"和"无账号"
- OpenAI RecordUsage RequestedModel 改用 OriginalModel
- ResolveChannelMappingAndRestrict/ReplaceModelInBody 抽取到 ChannelService 消除跨service重复
- validateNoDuplicateModels 按 platform:model 去重
- 删除 Channel.ResolveMappedModel 死代码和 CalculateCostWithChannel Deprecated方法
- 移除冗余nil检查,抽取 validatePricingBillingMode 公共校验
2026-04-04 11:13:56 +08:00
erio
4ea8b4cb4f refactor(channel): 抽取渠道映射公共函数 + OpenAI映射到body + 空响应修复 + 清理日志
- 抽取 ResolveChannelMappingAndRestrict 统一入口(5处→1个方法)
- 抽取 BuildModelMappingChain 到 ChannelMappingResult 方法(5处→1行调用)
- OpenAI 三入口 Forward 前应用渠道映射到请求体
- OpenAI Responses/Messages 限制检查添加错误响应
- 清理前端 3 处 console.log 调试日志
2026-04-04 11:13:56 +08:00
erio
632035aabd feat(billing): 网关计费迁移到 CalculateCostUnified + 模型限制错误统一
- GatewayService/OpenAIGatewayService 注入 ModelPricingResolver
- RecordUsage 从旧路径迁移到 CalculateCostUnified(支持 per_request/image 模式)
- 无渠道时自动回退旧路径,保持原有行为
- 长上下文双倍计费仅在无渠道定价时生效
- CostBreakdown 新增 BillingMode 字段,使用日志记录实际计费模式
- 模型限制错误改为与"无可用账号"相同的 503 响应
2026-04-04 11:12:21 +08:00
erio
a51e0047b7 feat(usage): 使用记录增加计费模式字段 — 记录/展示/筛选 token/按次/图片
- DB: usage_logs 表新增 billing_mode VARCHAR(20) 列
- 后端: RecordUsage 写入时根据 image_count 判定计费模式
- 前端: 使用记录表格新增计费模式 badge 列 + 筛选下拉
2026-04-04 11:11:06 +08:00
erio
ebac0dc628 feat(channel): 缓存扁平化 + 网关映射集成 + 计费模式统一 + 模型限制
- 缓存重构为 O(1) 哈希结构 (pricingByGroupModel, mappingByGroupModel)
- 渠道模型映射接入网关流程 (Forward 前应用, a→b→c 映射链)
- 新增 billing_model_source 配置 (请求模型/最终模型计费)
- usage_logs 新增 channel_id, model_mapping_chain, billing_tier 字段
- 每种计费模式统一支持默认价格 + 区间定价
- 渠道模型限制开关 (restrict_models)
- 分组按平台分类展示 + 彩色图标
- 必填字段红色星号 + 模型映射 UI
- 去除模型通配符支持
2026-04-04 11:09:01 +08:00
erio
91c9b8d062 feat(channel): 渠道管理系统 — 多模式定价 + 统一计费解析
Cherry-picked from release/custom-0.1.106: a9117600
2026-04-04 11:00:55 +08:00
Wesley Liddick
83a16dec19 Merge pull request #1407 from DaydreamCoding/feat/cache-driven-rpm-buffer
feat(gateway): Cache-Driven RPM Buffer
2026-03-31 14:01:23 +08:00
QTom
72e5876c64 feat(gateway): Cache-Driven RPM Buffer
- buffer 公式从 baseRPM/5 改为 concurrency + maxSessions
  保留 baseRPM/5 作为 floor 向后兼容
- 粘性路径 fallback 新增 [StickyCacheMiss] 结构化日志
  reason: rpm_red / gate_check / session_limit / wait_queue_full / account_cleared
- session_limit 路径跳过 wait queue 重试(RegisterSession 拒绝无副作用)
- 典型配置 buffer 从 3 提升至 13,大幅减少高峰期 Prompt Cache Miss

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 13:24:22 +08:00
QTom
aeed2eb9ad feat(group-filter): 分组账号过滤控制 — require_oauth_only + require_privacy_set
为 OpenAI/Antigravity/Anthropic/Gemini 分组新增两个布尔控制字段:
- require_oauth_only: 创建/更新账号绑定分组时拒绝 apikey 类型加入
- require_privacy_set: 调度选号时跳过 privacy 未成功设置的账号并标记 error

后端:Ent schema 新增字段 + 迁移、Group CRUD 全链路透传、
      gateway_service 与 openai_account_scheduler 两套调度路径过滤
前端:创建/编辑表单 toggle 开关(OpenAI/Antigravity/Anthropic/Gemini 平台可见)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 13:04:55 +08:00
shaw
ab3e44e4bd fix: 适配X-Claude-Code-Session-Id头 2026-03-30 11:43:07 +08:00
shaw
b65275235f feat: Anthropic oauth/setup-token账号支持自定义转发URL 2026-03-30 09:10:57 +08:00
QTom
c729ee425f fix(gateway): 修复 OpenAI→Anthropic 转换路径 system prompt 被静默丢弃的 bug
injectClaudeCodePrompt 和 systemIncludesClaudeCodePrompt 的 type switch
无法匹配 json.RawMessage 类型(Go typed nil 陷阱),导致 ForwardAsResponses
和 ForwardAsChatCompletions 路径中用户 system prompt 被替换为仅 Claude Code
banner。新增 normalizeSystemParam 将 json.RawMessage 转为标准 Go 类型。

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 14:51:36 +08:00
shaw
1854050df3 feat(tls-fingerprint): 新增 TLS 指纹 Profile 数据库管理及代码质量优化
新增功能:
- 新增 TLS 指纹 Profile CRUD 管理(Ent schema + 迁移 + Admin API + 前端管理界面)
- 支持账号绑定数据库中的自定义 TLS Profile,或随机选择(profile_id=-1)
- HTTPUpstream.DoWithTLS 接口从 bool 改为 *tlsfingerprint.Profile,支持按账号指定 Profile
- AccountUsageService 注入 TLSFingerprintProfileService,统一 usage 场景与网关的 Profile 解析逻辑

代码优化:
- 删除已被 TLSFingerprintProfileService 完全取代的 registry.go 死代码(418 行)
- 提取 3 个 dialer 的重复 TLS 握手逻辑为 performTLSHandshake() 共用函数
- 修复 GetTLSFingerprintProfileID 缺少 json.Number 处理的 bug
- gateway_service.Forward 中 ResolveTLSProfile 从重试循环内重复调用改为预解析局部变量
- 删除冗余的 buildClientHelloSpec() 单行 wrapper 和 int64(e.ID) 无效转换
- tls_fingerprint_profile_cache.go 日志从 log.Printf 改为 slog 结构化日志
- dialer_capture_test.go 添加 //go:build integration 标签,防止 CI 失败
- 去重 TestProfileExpectation 类型至共享 test_types_test.go
- 修复 9 个测试文件缺少 tlsfingerprint import 的编译错误
- 修复 error_policy_integration_test.go 中 handleError 回调签名被错误替换的问题
2026-03-27 14:33:05 +08:00
shaw
d571f300e5 feat(rectifier): 请求整流器增加 API Key 账号签名整流支持
新增独立开关控制 API Key 账号的签名整流功能,支持配置自定义
匹配关键词以捕获不同格式的上游错误响应。

- 新增 apikey_signature_enabled 开关(默认关闭)
- 新增 apikey_signature_patterns 自定义关键词配置
- 内置签名检测规则对 API Key 账号同样生效
- 自定义关键词对完整响应体做不区分大小写匹配
- 重试二阶段检测仅做模式匹配,不重复校验开关
- Handler 层校验关键词数量(≤50)和长度(≤500)
- API 响应 nil patterns 统一序列化为空数组
- OAuth/SetupToken/Upstream/Bedrock 账号行为不变
2026-03-26 16:43:38 +08:00
shaw
b20e142249 feat: 网关请求头 wire casing 保持、转发行为开关、调试日志增强及 accept-encoding 恢复
- 新增 header_util.go,通过 setHeaderRaw/getHeaderRaw/addHeaderRaw 绕过
  Go 的 canonical-case 规范化,保持真实 Claude CLI 抓包的请求头大小写
  (如 "x-app" 而非 "X-App","X-Stainless-OS" 而非 "X-Stainless-Os")
- 新增管理后台开关:指纹统一化(默认开启)和 metadata 透传(默认关闭),
  使用 atomic.Value + singleflight 缓存模式,60s TTL
- 调试日志从控制台 body 打印升级为文件级完整快照
  (按真实 wire 顺序输出 headers + 格式化 JSON body + 上下文元数据)
- 恢复 accept-encoding 到白名单,在 http_upstream.go 新增 decompressResponseBody
  处理 gzip/brotli/deflate 解压(Go 显式设置 Accept-Encoding 时不会自动解压)
- OAuth 服务 axios UA 从 1.8.4 更新至 1.13.6
- 测试断言改用 getHeaderRaw 适配 raw header 存储方式
2026-03-26 11:17:25 +08:00
Wesley Liddick
02046744eb Merge pull request #1212 from alfadb/fix/filter-empty-text-blocks-nested
fix(gateway): 修复 tool_result 嵌套内容中空 text block 导致上游 400 错误
2026-03-24 15:19:01 +08:00
Ethan0x0000
bcf84cc153 fix(service): normalize user agent for sticky session hashes
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-03-24 09:31:32 +08:00
alfadb
70a9d0d3a2 fix(gateway): strip empty text blocks from nested tool_result content
Empty text blocks inside tool_result.content were not being filtered,
causing upstream 400 errors: 'text content blocks must be non-empty'.

Changes:
- Add stripEmptyTextBlocksFromSlice helper for recursive content filtering
- FilterThinkingBlocksForRetry now recurses into tool_result nested content
- Add StripEmptyTextBlocks pre-filter on initial request path to avoid
  unnecessary 400+retry round-trips
- Add unit tests for nested empty text block scenarios
2026-03-22 17:26:44 +08:00
Ethan0x0000
db9021f9c1 feat(ops): propagate endpoint/request-type context in handlers; add UpstreamURL to upstream error events 2026-03-21 23:47:39 +08:00
Wesley Liddick
186e36752d Merge pull request #1194 from Ethan0x0000/feat/requested-upstream-model-semantics
feat(usage): 统一使用记录中的请求模型与上游模型语义
2026-03-21 14:02:10 +08:00
Dave King
c64ed46d05 fix: correct log levels for thinking block signature retry flow
LegacyPrintf uses inferStdLogLevel() to infer log level from message
text. Any message containing the word "error" is classified as ERROR
level, causing the entire signature-retry recovery flow (which succeeds)
to produce spurious ERROR log entries.

Changes:
- Remove noisy [SignatureCheck] debug logs inside isThinkingBlockSignatureError
  that were logging every detected signature check as ERROR
- Change retry-start log to WARN level via [warn] prefix
- Change retry-success log to INFO level by removing "error" from message

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 00:38:07 +00:00
Ethan0x0000
4edcfe1f7c fix(usage): preserve requested model in gateway billing paths
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-03-21 01:23:54 +08:00
shaw
a6764e82f2 修复 OAuth/SetupToken 转发请求体重排并增加调试开关 2026-03-19 16:56:18 +08:00
Wesley Liddick
241023f3fc Merge pull request #1097 from Ethan0x0000/pr/upstream-model-tracking
feat(usage): 新增 upstream_model 追踪,支持按模型来源统计与展示
2026-03-18 15:36:00 +08:00
alfadb
7d26b81075 fix: address review - add missing whitespace patterns and narrow error matching 2026-03-18 14:31:57 +08:00
alfadb
b8ada63ac3 fix: strip empty text blocks in retry filter and fix error pattern matching
Empty text blocks ({"type":"text","text":""}) cause Anthropic upstream to
return 400: "text content blocks must be non-empty". This was not caught
by the existing error detection pattern in isThinkingBlockSignatureError,
nor handled by FilterThinkingBlocksForRetry.

- Add empty text block stripping to FilterThinkingBlocksForRetry
- Fix isThinkingBlockSignatureError to match new Anthropic error format
- Add fast-path byte patterns to avoid unnecessary JSON parsing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 14:20:00 +08:00
Ethan0x0000
cfaac12af1 Merge upstream/main into pr/upstream-model-tracking
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-03-18 14:16:50 +08:00
shaw
a14babdc73 fix: 兼容 Claude Code v2.1.78+ 新 JSON 格式 metadata.user_id
Claude Code v2.1.78 起将 metadata.user_id 从拼接字符串改为 JSON:
旧: user_{hex}_account_{uuid}_session_{uuid}
新: {"device_id":"...","account_uuid":"...","session_id":"..."}

新增集中解析/格式化模块 metadata_userid.go:
- ParseMetadataUserID: 自动识别两种格式,提取 DeviceID/AccountUUID/SessionID
- FormatMetadataUserID: 根据 UA 版本输出对应格式(>= 2.1.78 输出 JSON)
- ExtractCLIVersion: 从 UA 提取版本号,消除与 ClaudeCodeValidator.ExtractVersion 的重复

修改消费者统一使用新模块:
- claude_code_validator: 用 ParseMetadataUserID 替代只匹配旧格式的 userIDPattern
- identity_service: RewriteUserID/WithMasking 增加 fingerprintUA 参数,
  解析用 ParseMetadataUserID,输出用 FormatMetadataUserID(版本感知)
- gateway_service: GenerateSessionHash 用 ParseMetadataUserID 提取 session_id,
  buildOAuthMetadataUserID 用 FormatMetadataUserID 输出版本匹配格式,
  两处 RewriteUserIDWithMasking 调用传入 fp.UserAgent
- account_test_service: generateSessionString 改用 FormatMetadataUserID,
  自动跟随 DefaultHeaders UA 版本

删除三个旧正则: userIDPattern, userIDRegex, sessionIDRegex
统一 hex 匹配为 [a-fA-F0-9],修复旧 userIDRegex 只匹配小写的不一致
2026-03-18 11:08:58 +08:00
Ethan0x0000
2e4ac88ad9 feat(service): record upstream model across all gateway paths
Propagate UpstreamModel through ForwardResult and OpenAIForwardResult in Anthropic direct, API-key passthrough, Bedrock, and OpenAI gateway flows. Extract optionalNonEqualStringPtr and optionalTrimmedStringPtr into usage_log_helpers.go. Store upstream_model only when it differs from the requested model.

Also introduces anthropicPassthroughForwardInput struct to reduce parameter count.
2026-03-17 19:25:35 +08:00
Ethan0x0000
1b79b0f3ff feat: add InboundEndpoint/UpstreamEndpoint fields to non-OpenAI usage records
Extend RecordUsageInput and RecordUsageLongContextInput structs with InboundEndpoint and UpstreamEndpoint so that Claude, Gemini, and Sora handlers can record endpoint info alongside OpenAI handlers.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-03-15 22:13:22 +08:00
erio
cfe72159d0 feat(ops): add ignore insufficient balance errors toggle and extract error constants
- Add 5th error filter switch IgnoreInsufficientBalanceErrors to suppress
  upstream insufficient balance / insufficient_quota errors from ops log
- Extract hardcoded error strings into package-level constants for
  shouldSkipOpsErrorLog, normalizeOpsErrorType, classifyOpsPhase, and
  classifyOpsIsBusinessLimited
- Define ErrNoAvailableAccounts sentinel error and replace all
  errors.New("no available accounts") call sites
- Update tests to use require.ErrorIs with the sentinel error
2026-03-15 17:26:18 +08:00
YanzheL
1bff2292a6 fix: extract and log Claude output_config.effort in usage records
Claude's output_config.effort parameter (low/medium/high/max) was not
being extracted from requests or logged in the reasoning_effort column
of usage logs. Only the OpenAI path populated this field.

Changes:
- Extract output_config.effort in ParseGatewayRequest
- Add ReasoningEffort field to ForwardResult
- Populate reasoning_effort in both RecordUsage and RecordUsageWithLongContext
- Guard against overwriting service-set effort values in handler
- Update stale comments that described reasoning_effort as OpenAI-only
- Add unit tests for extraction, normalization, and persistence
2026-03-15 12:55:37 +08:00
InCerry
8f0ea7a02d Merge branch 'main' into fix/enc_coot 2026-03-14 18:46:33 +08:00
SsageParuders
4644af2ccc refactor: merge bedrock-apikey into bedrock with auth_mode credential
Consolidate two separate channel types (bedrock + bedrock-apikey) into
a single "AWS Bedrock" channel. Authentication mode is now distinguished
by credentials.auth_mode ("sigv4" | "apikey") instead of separate types.

Backend:
- Remove AccountTypeBedrockAPIKey constant
- IsBedrock() simplified; IsBedrockAPIKey() checks auth_mode
- Add IsAPIKeyOrBedrock() helper to eliminate repeated type checks
- Extend pool mode, quota scheduling, and billing to bedrock
- Add RetryableOnSameAccount to handleBedrockUpstreamErrors
- Add "bedrock" scope to Beta Policy for independent control

Frontend:
- Merge two buttons into one "AWS Bedrock" with auth mode radio
- Badge displays "Anthropic | AWS"
- Pool mode and quota limit UI available for bedrock
- Quota display in account list (usage bars, capacity badges, reset)
- Remove all bedrock-apikey type references
2026-03-14 17:13:30 +08:00
InCerry
e4a4dfd038 Merge remote-tracking branch 'origin/main' into fix/enc_coot
# Conflicts:
#	backend/internal/service/openai_gateway_service.go
2026-03-14 13:04:24 +08:00
InCerry
2666422b99 fix: handle invalid encrypted content error and retry logic. 2026-03-14 11:42:42 +08:00
Wesley Liddick
e6d59216d4 Merge pull request #975 from Ylarod/aws-bedrock
sub2api: add bedrock support
2026-03-14 10:52:24 +08:00
Ylarod
e90ec847b6 fix lint 2026-03-13 19:15:27 +08:00
Ylarod
11f7b83522 sub2api: add bedrock support 2026-03-13 17:00:16 +08:00
ius
6a685727d0 fix: harden usage billing idempotency and backpressure 2026-03-12 18:38:09 +08:00
ius
8d4d3b03bb fix: remove unused gateway usage helpers 2026-03-12 17:08:57 +08:00
ius
b764d3b8f6 Merge remote-tracking branch 'origin/main' into feat/billing-ledger-decouple-usage-log-20260312 2026-03-12 16:53:28 +08:00
ius
611fd884bd feat: decouple billing correctness from usage log batching 2026-03-12 16:53:18 +08:00
amberwarden
6e90ec6111 fix: 为 Anthropic Messages API 流式转发添加下游 keepalive ping
Anthropic Messages API 的流式转发路径(gateway_service.go)在上游长时间
无数据时(如 Opus extended thinking 阶段)不会向下游发送任何内容,导致
Cloudflare Tunnel 等代理因连接空闲而断开。

复用已有的 StreamKeepaliveInterval 配置(默认 10 秒),在 select 循环中
添加 keepalive 分支,定时发送 Anthropic 原生格式的 ping 事件保活,与
OpenAI 兼容路径的实现模式保持一致。

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 18:43:03 +08:00
shaw
00a0a12138 feat: Anthropic平台可配置 anthropic-beta 策略 2026-03-10 11:20:10 +08:00
shaw
a461538d58 fix: 修复gpt->claude转换无法命中codex缓存问题 2026-03-09 15:08:37 +08:00
shaw
ebe6f418f3 fix: gpt->claude格式转换对齐effort映射和fast 2026-03-09 11:42:35 +08:00
Wesley Liddick
6cb8980404 Merge pull request #807 from touwaeriol/fix/openai-passthrough-v2
fix(openai): remove misplaced passthrough check from isModelSupportedByAccount
2026-03-09 09:06:35 +08:00
erio
91ef085d7d fix: increase SSE scanner max line size from 40MB to 500MB
4K image base64 data can exceed 40MB limit, causing "bufio.Scanner:
token too long" errors. Scanner is adaptive (starts at 64KB, grows
as needed), so increasing the cap has no impact on normal responses.
2026-03-09 08:56:54 +08:00
Wesley Liddick
97aaa24733 Merge pull request #858 from james-6-23/fix/pool-mode-03bf3485
支持 API Key 上游池模式的同账号重试次数配置与自定义错误策略
2026-03-09 08:48:53 +08:00