From 7fdc2b2d29281fff6bbbd6f0ed2762f9eb872aed Mon Sep 17 00:00:00 2001 From: IanShaw <131567472+IanShaw027@users.noreply.github.com> Date: Fri, 2 Jan 2026 17:47:49 +0800 Subject: [PATCH] Fix/multiple issues (#24) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(gemini): 修复 google_one OAuth 配置和 scopes 问题 - 修复 google_one 类型在 ExchangeCode 和 RefreshToken 中使用内置客户端 - 添加 DefaultGoogleOneScopes，包含 generative-language 和 drive.readonly 权限 - 在 EffectiveOAuthConfig 中为 google_one 类型使用专门的 scopes - 将 docker-compose.override.yml 重命名为 .example 并添加到 .gitignore - 完善 docker-compose.override.yml.example 示例文档 解决问题： 1. google_one OAuth 授权后 API 调用返回 403 权限不足 2. 缺少访问 Gemini API 所需的 generative-language scope 3. 缺少获取 Drive 存储配额所需的 drive.readonly scope * fix(antigravity): 完全跳过 Claude 模型的所有 thinking 块 问题分析： - 当前代码尝试保留有 signature 的 thinking 块 - 但 Vertex AI 的 signature 是完整性令牌，无法在本地验证 - 导致 400 错误：Invalid signature in thinking block 根本原因： 1. thinking 功能已对非 Gemini 模型禁用 (isThinkingEnabled=false) 2. Vertex AI 要求原样重放 (thinking, signature) 对或完全不发送 3. 本地无法复制 Vertex 的加密验证逻辑 修复方案： - 对 Claude 模型完全跳过所有 thinking 块（无论是否有 signature） - 保持 Gemini 模型使用 dummy signature 的行为不变 - 更新测试用例以反映新的预期行为 影响： - 消除 thinking 相关的 400 错误 - 与现有的 thinking 禁用策略保持一致 - 不影响 Gemini 模型的 thinking 功能 测试： - ✅ TestBuildParts_ThinkingBlockWithoutSignature 全部通过 - ✅ TestBuildTools_CustomTypeTools 全部通过 参考：Codex review 建议 * fix(gateway): 修复 count_tokens 端点 400 错误 问题分析： - count_tokens 请求包含 thinking 块时返回 400 错误 - 原因：thinking 块未被过滤，直接转发到上游 API - 上游 API 拒绝无效的 thinking signature 根本原因： 1. /v1/messages 请求通过 TransformClaudeToGemini 过滤 thinking 块 2. count_tokens 请求绕过转换，直接转发原始请求体 3. 导致包含无效 signature 的 thinking 块被发送到上游 修复方案： - 创建 FilterThinkingBlocks 工具函数 - 在 buildCountTokensRequest 中应用过滤（1 行修改） - 与 /v1/messages 行为保持一致 实现细节： - FilterThinkingBlocks: 解析 JSON，过滤 thinking 块，重新序列化 - 失败安全：解析/序列化失败时返回原始请求体 - 性能优化：仅在发现 thinking 块时重新序列化 测试： - ✅ 6 个单元测试全部通过 - ✅ 覆盖正常过滤、无 thinking 块、无效 JSON 等场景 - ✅ 现有测试不受影响 影响： - 消除 count_tokens 的 400 错误 - 不影响 Antigravity 账号（仍返回模拟响应） - 适用于所有账号类型（OAuth、API Key） 文件修改： - backend/internal/service/gateway_request.go: +62 行（新函数） - backend/internal/service/gateway_service.go: +2 行（应用过滤） - backend/internal/service/gateway_request_test.go: +62 行（测试） * fix(gateway): 增强 thinking 块过滤逻辑 基于 Codex 分析和建议的改进： 问题分析： - 新错误：signature: Field required（signature 字段缺失） - 旧错误：Invalid signature（signature 存在但无效） - 两者都说明 thinking 块在请求中是危险的 Codex 建议： - 保持 Option A：完全跳过所有 thinking 块 - 原因：thinking 块应该是只输出的，除非有服务端来源证明 - 在无状态代理中，无法安全区分上游来源 vs 客户端注入 改进内容： 1. 增强 FilterThinkingBlocks 函数 - 过滤显式的 thinking 块：{"type":"thinking", ...} - 过滤无 type 的 thinking 对象：{"thinking": {...}} - 保留 tool_use 等其他类型块中的 thinking 字段 - 修复：只在实际过滤时更新 content 数组 2. 扩展过滤范围 - 将 FilterThinkingBlocks 应用到 /v1/messages 主路径 - 之前只应用于 count_tokens，现在两个端点都过滤 - 防止所有端点的 thinking 相关 400 错误 3. 改进测试 - 新增：过滤无 type discriminator 的 thinking 块 - 新增：不过滤 tool_use 中的 thinking 字段 - 使用 containsThinkingBlock 辅助函数验证 测试： - ✅ 8 个测试用例全部通过 - ✅ 覆盖各种 thinking 块格式 - ✅ 确保不误伤其他类型的块 影响： - 消除 signature required 和 invalid signature 错误 - 统一 /v1/messages 和 count_tokens 的行为 - 更健壮的 thinking 块检测逻辑 参考：Codex review 和代码改进 * refactor: 根据 Codex 审查建议进行代码优化 基于 Codex 代码审查的 P1 和 P2 改进： P1 改进（重要问题）： 1. 优化日志输出 - 移除 thinking 块跳过时的 log.Printf - 避免高频请求下的日志噪音 - 添加注释说明可通过指标监控 2. 清理遗留代码 - 删除未使用的 isValidThoughtSignature 函数（27行） - 该函数在改为完全跳过 thinking 块后不再需要 P2 改进（性能优化）： 3. 添加快速路径检查 - 在 FilterThinkingBlocks 中添加 bytes.Contains 预检查 - 如果请求体不包含 "thinking" 字符串，直接返回 - 避免不必要的 JSON 解析，提升性能 技术细节： - request_transformer.go: -27行（删除函数），+1行（优化注释） - gateway_request.go: +5行（快速路径 + bytes 导入） 测试： - ✅ TestBuildParts_ThinkingBlockWithoutSignature 全部通过 - ✅ TestFilterThinkingBlocks 全部通过（8个测试用例） 影响： - 减少日志噪音 - 提升性能（快速路径） - 代码更简洁（删除未使用代码） 参考：Codex 代码审查建议 * fix: 修复 golangci-lint 检查问题 - 格式化 gateway_request_test.go - 使用 switch 语句替代 if-else 链（staticcheck QF1003） * fix(antigravity): 修复 thinking signature 处理并实现 Auto 模式降级 问题分析： 1. 原先代码错误地禁用了 Claude via Vertex 的 thinkingConfig 2. 历史 thinking 块的 signature 被完全跳过，导致验证失败 3. 跨模型混用时 dummy signature 会导致 400 错误 修复内容： **request_transformer.go**： - 删除第 38-43 行的错误逻辑（禁用 thinkingConfig） - 引入 thoughtSignatureMode（Preserve/Dummy）策略 - Claude 模式：透传真实 signature，过滤空/dummy - Gemini 模式：使用 dummy signature - 支持 signature-only thinking 块 - tool_use 的 signature 也透传 **antigravity_gateway_service.go**： - 新增 isSignatureRelatedError() 检测 signature 相关错误 - 新增 stripThinkingFromClaudeRequest() 移除 thinking 块 - 实现 Auto 模式：检测 400 + signature 关键词时自动降级重试 - 重试时完全移除 thinking 配置和消息中的 thinking 块 - 最多重试一次，避免循环 **测试**： - 更新并新增测试覆盖 Claude preserve/Gemini dummy 模式 - 新增 tool_use signature 处理测试 - 所有测试通过（6/6） 影响： - ✅ Claude via Vertex 可以正常使用 thinking 功能 - ✅ 历史 signature 正确透传，避免验证失败 - ✅ 跨模型混用时自动过滤无效 signature - ✅ 错误驱动降级，自动修复 signature 问题 - ✅ 不影响纯 Claude API 和其他渠道 参考：Codex 深度分析和实现建议 * fix(lint): 修复 gofmt 格式问题 * fix(antigravity): 修复 stripThinkingFromClaudeRequest 遗漏 untyped thinking blocks 问题： - Codex 审查指出 stripThinkingFromClaudeRequest 只移除了 type="thinking" 的块 - 没有处理没有 type 字段的 thinking 对象（如 {"thinking": "...", "signature": "..."}） - 导致重试时仍包含无效 thinking 块，上游 400 错误持续 修复： - 添加检查：跳过没有 type 但有 thinking 字段的块 - 现在会移除两种格式： 1. {"type": "thinking", "thinking": "...", "signature": "..."} 2. {"thinking": "...", "signature": "..."}（untyped） 测试：所有测试通过 参考：Codex P1 审查意见 --- .gitignore | 1 + .../pkg/antigravity/request_transformer.go | 110 +++++++------- .../antigravity/request_transformer_test.go | 106 +++++++++++--- backend/internal/pkg/geminicli/constants.go | 4 + backend/internal/pkg/geminicli/oauth.go | 9 +- .../repository/gemini_oauth_client.go | 5 +- .../service/antigravity_gateway_service.go | 133 ++++++++++++++++- backend/internal/service/gateway_request.go | 83 +++++++++++ .../internal/service/gateway_request_test.go | 113 +++++++++++++++ backend/internal/service/gateway_service.go | 7 + deploy/docker-compose.override.yml | 21 --- deploy/docker-compose.override.yml.example | 137 ++++++++++++++++++ 12 files changed, 627 insertions(+), 102 deletions(-) delete mode 100644 deploy/docker-compose.override.yml create mode 100644 deploy/docker-compose.override.yml.example diff --git a/.gitignore b/.gitignore index 6d636c8d..c33cde99 100644 --- a/.gitignore +++ b/.gitignore @@ -118,3 +118,4 @@ docs/ code-reviews/ AGENTS.md backend/cmd/server/server +deploy/docker-compose.override.yml diff --git a/backend/internal/pkg/antigravity/request_transformer.go b/backend/internal/pkg/antigravity/request_transformer.go index 83b87a32..3af6579c 100644 --- a/backend/internal/pkg/antigravity/request_transformer.go +++ b/backend/internal/pkg/antigravity/request_transformer.go @@ -20,12 +20,18 @@ func TransformClaudeToGemini(claudeReq *ClaudeRequest, projectID, mappedModel st // 检测是否启用 thinking requestedThinkingEnabled := claudeReq.Thinking != nil && claudeReq.Thinking.Type == "enabled" - // 为避免 Claude 模型的 thought signature/消息块约束导致 400（上游要求 thinking 块开头等）， - // 非 Gemini 模型默认不启用 thinking（除非未来支持完整签名链路）。 - isThinkingEnabled := requestedThinkingEnabled && allowDummyThought + // antigravity(v1internal) 下，Gemini 与 Claude 的 “thinking” 都可能涉及 thoughtSignature 链路： + // - Gemini：支持 dummy signature 跳过校验 + // - Claude：需要透传上游签名（否则容易 400） + isThinkingEnabled := requestedThinkingEnabled + + thoughtSignatureMode := thoughtSignatureModePreserve + if allowDummyThought { + thoughtSignatureMode = thoughtSignatureModeDummy + } // 1. 构建 contents - contents, err := buildContents(claudeReq.Messages, toolIDToName, isThinkingEnabled, allowDummyThought) + contents, err := buildContents(claudeReq.Messages, toolIDToName, isThinkingEnabled, thoughtSignatureMode) if err != nil { return nil, fmt.Errorf("build contents: %w", err) } @@ -34,15 +40,7 @@ func TransformClaudeToGemini(claudeReq *ClaudeRequest, projectID, mappedModel st systemInstruction := buildSystemInstruction(claudeReq.System, claudeReq.Model) // 3. 构建 generationConfig - reqForGen := claudeReq - if requestedThinkingEnabled && !allowDummyThought { - log.Printf("[Warning] Disabling thinking for non-Gemini model in antigravity transform: model=%s", mappedModel) - // shallow copy to avoid mutating caller's request - clone := *claudeReq - clone.Thinking = nil - reqForGen = &clone - } - generationConfig := buildGenerationConfig(reqForGen) + generationConfig := buildGenerationConfig(claudeReq) // 4. 构建 tools tools := buildTools(claudeReq.Tools) @@ -131,7 +129,7 @@ func buildSystemInstruction(system json.RawMessage, modelName string) *GeminiCon } // buildContents 构建 contents -func buildContents(messages []ClaudeMessage, toolIDToName map[string]string, isThinkingEnabled, allowDummyThought bool) ([]GeminiContent, error) { +func buildContents(messages []ClaudeMessage, toolIDToName map[string]string, isThinkingEnabled bool, thoughtSignatureMode thoughtSignatureMode) ([]GeminiContent, error) { var contents []GeminiContent for i, msg := range messages { @@ -140,11 +138,13 @@ func buildContents(messages []ClaudeMessage, toolIDToName map[string]string, isT role = "model" } - parts, err := buildParts(msg.Content, toolIDToName, allowDummyThought) + parts, err := buildParts(msg.Content, toolIDToName, thoughtSignatureMode) if err != nil { return nil, fmt.Errorf("build parts for message %d: %w", i, err) } + allowDummyThought := thoughtSignatureMode == thoughtSignatureModeDummy + // 只有 Gemini 模型支持 dummy thinking block workaround // 只对最后一条 assistant 消息添加（Pre-fill 场景） // 历史 assistant 消息不能添加没有 signature 的 dummy thinking block @@ -183,37 +183,19 @@ func buildContents(messages []ClaudeMessage, toolIDToName map[string]string, isT // 参考: https://ai.google.dev/gemini-api/docs/thought-signatures const dummyThoughtSignature = "skip_thought_signature_validator" -// isValidThoughtSignature 验证 thought signature 是否有效 -// Claude API 要求 signature 必须是 base64 编码的字符串，长度至少 32 字节 -func isValidThoughtSignature(signature string) bool { - // 空字符串无效 - if signature == "" { - return false - } +// buildParts 构建消息的 parts +type thoughtSignatureMode int - // signature 应该是 base64 编码，长度至少 40 个字符（约 30 字节） - // 参考 Claude API 文档和实际观察到的有效 signature - if len(signature) < 40 { - log.Printf("[Debug] Signature too short: len=%d", len(signature)) - return false - } - - // 检查是否是有效的 base64 字符 - // base64 字符集: A-Z, a-z, 0-9, +, /, = - for i, c := range signature { - if (c < 'A' || c > 'Z') && (c < 'a' || c > 'z') && - (c < '0' || c > '9') && c != '+' && c != '/' && c != '=' { - log.Printf("[Debug] Invalid base64 character at position %d: %c (code=%d)", i, c, c) - return false - } - } - - return true -} +const ( + thoughtSignatureModePreserve thoughtSignatureMode = iota + thoughtSignatureModeDummy +) // buildParts 构建消息的 parts -// allowDummyThought: 只有 Gemini 模型支持 dummy thought signature -func buildParts(content json.RawMessage, toolIDToName map[string]string, allowDummyThought bool) ([]GeminiPart, error) { +// thoughtSignatureMode: +// - dummy: 用 dummy signature 跳过 Gemini thoughtSignature 校验 +// - preserve: 透传输入中的 signature（主要用于 Claude via Vertex 的签名链路） +func buildParts(content json.RawMessage, toolIDToName map[string]string, thoughtSignatureMode thoughtSignatureMode) ([]GeminiPart, error) { var parts []GeminiPart // 尝试解析为字符串 @@ -239,7 +221,9 @@ func buildParts(content json.RawMessage, toolIDToName map[string]string, allowDu } case "thinking": - if allowDummyThought { + signature := strings.TrimSpace(block.Signature) + + if thoughtSignatureMode == thoughtSignatureModeDummy { // Gemini 模型可以使用 dummy signature parts = append(parts, GeminiPart{ Text: block.Thinking, @@ -249,20 +233,27 @@ func buildParts(content json.RawMessage, toolIDToName map[string]string, allowDu continue } - // Claude 模型：仅在提供有效 signature 时保留 thinking block；否则跳过以避免上游校验失败。 - signature := strings.TrimSpace(block.Signature) + // Claude via Vertex： + // - signature 是上游返回的完整性令牌；本地不需要/无法验证，只能透传 + // - 缺失/无效 signature（例如来自 Gemini 的 dummy signature）会导致上游 400 if signature == "" || signature == dummyThoughtSignature { - log.Printf("[Warning] Skipping thinking block for Claude model (missing or dummy signature)") continue } - if !isValidThoughtSignature(signature) { - log.Printf("[Debug] Thinking signature may be invalid (passing through anyway): len=%d", len(signature)) + + // 兼容：用 Claude 的 "thinking" 块承载两类东西 + // 1) 真正的 thought 文本（thinking != ""）-> Gemini thought part + // 2) 仅承载 signature 的空 thinking 块（thinking == ""）-> Gemini signature-only part + if strings.TrimSpace(block.Thinking) == "" { + parts = append(parts, GeminiPart{ + ThoughtSignature: signature, + }) + } else { + parts = append(parts, GeminiPart{ + Text: block.Thinking, + Thought: true, + ThoughtSignature: signature, + }) } - parts = append(parts, GeminiPart{ - Text: block.Thinking, - Thought: true, - ThoughtSignature: signature, - }) case "image": if block.Source != nil && block.Source.Type == "base64" { @@ -287,10 +278,15 @@ func buildParts(content json.RawMessage, toolIDToName map[string]string, allowDu ID: block.ID, }, } - // 只有 Gemini 模型使用 dummy signature - // Claude 模型不设置 signature（避免验证问题） - if allowDummyThought { + switch thoughtSignatureMode { + case thoughtSignatureModeDummy: part.ThoughtSignature = dummyThoughtSignature + case thoughtSignatureModePreserve: + // Claude via Vertex：透传 tool_use 的 signature（如果有） + // 注意：跨模型混用时可能出现 dummy signature，这里直接丢弃以避免 400。 + if sig := strings.TrimSpace(block.Signature); sig != "" && sig != dummyThoughtSignature { + part.ThoughtSignature = sig + } } parts = append(parts, part) diff --git a/backend/internal/pkg/antigravity/request_transformer_test.go b/backend/internal/pkg/antigravity/request_transformer_test.go index 56eebad0..845ae033 100644 --- a/backend/internal/pkg/antigravity/request_transformer_test.go +++ b/backend/internal/pkg/antigravity/request_transformer_test.go @@ -8,11 +8,11 @@ import ( // TestBuildParts_ThinkingBlockWithoutSignature 测试thinking block无signature时的处理 func TestBuildParts_ThinkingBlockWithoutSignature(t *testing.T) { tests := []struct { - name string - content string - allowDummyThought bool - expectedParts int - description string + name string + content string + thoughtMode thoughtSignatureMode + expectedParts int + description string }{ { name: "Claude model - skip thinking block without signature", @@ -21,20 +21,20 @@ func TestBuildParts_ThinkingBlockWithoutSignature(t *testing.T) { {"type": "thinking", "thinking": "Let me think...", "signature": ""}, {"type": "text", "text": "World"} ]`, - allowDummyThought: false, - expectedParts: 2, // 只有两个text block - description: "Claude模型应该跳过无signature的thinking block", + thoughtMode: thoughtSignatureModePreserve, + expectedParts: 2, // 只有两个text block + description: "Claude模型应该跳过无signature的thinking block", }, { - name: "Claude model - keep thinking block with signature", + name: "Claude model - preserve thinking block with signature", content: `[ {"type": "text", "text": "Hello"}, - {"type": "thinking", "thinking": "Let me think...", "signature": "valid_sig"}, + {"type": "thinking", "thinking": "Let me think...", "signature": "sig_real_123"}, {"type": "text", "text": "World"} ]`, - allowDummyThought: false, - expectedParts: 3, // 三个block都保留 - description: "Claude模型应该保留有signature的thinking block", + thoughtMode: thoughtSignatureModePreserve, + expectedParts: 3, + description: "Claude模型应透传带 signature 的 thinking block（用于 Vertex 签名链路）", }, { name: "Gemini model - use dummy signature", @@ -43,16 +43,27 @@ func TestBuildParts_ThinkingBlockWithoutSignature(t *testing.T) { {"type": "thinking", "thinking": "Let me think...", "signature": ""}, {"type": "text", "text": "World"} ]`, - allowDummyThought: true, - expectedParts: 3, // 三个block都保留，thinking使用dummy signature - description: "Gemini模型应该为无signature的thinking block使用dummy signature", + thoughtMode: thoughtSignatureModeDummy, + expectedParts: 3, // 三个block都保留，thinking使用dummy signature + description: "Gemini模型应该为无signature的thinking block使用dummy signature", + }, + { + name: "Claude model - signature-only thinking block becomes signature-only part", + content: `[ + {"type": "text", "text": "Hello"}, + {"type": "thinking", "thinking": "", "signature": "sig_only_456"}, + {"type": "text", "text": "World"} + ]`, + thoughtMode: thoughtSignatureModePreserve, + expectedParts: 3, + description: "Claude模型应将空 thinking + signature 映射为 signature-only part，便于 roundtrip", }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { toolIDToName := make(map[string]string) - parts, err := buildParts(json.RawMessage(tt.content), toolIDToName, tt.allowDummyThought) + parts, err := buildParts(json.RawMessage(tt.content), toolIDToName, tt.thoughtMode) if err != nil { t.Fatalf("buildParts() error = %v", err) @@ -61,10 +72,71 @@ func TestBuildParts_ThinkingBlockWithoutSignature(t *testing.T) { if len(parts) != tt.expectedParts { t.Errorf("%s: got %d parts, want %d parts", tt.description, len(parts), tt.expectedParts) } + + switch tt.name { + case "Claude model - preserve thinking block with signature": + if len(parts) != 3 { + t.Fatalf("expected 3 parts, got %d", len(parts)) + } + if !parts[1].Thought || parts[1].ThoughtSignature != "sig_real_123" { + t.Fatalf("expected thought part with signature sig_real_123, got thought=%v signature=%q", + parts[1].Thought, parts[1].ThoughtSignature) + } + case "Claude model - signature-only thinking block becomes signature-only part": + if len(parts) != 3 { + t.Fatalf("expected 3 parts, got %d", len(parts)) + } + if parts[1].Thought || parts[1].Text != "" || parts[1].ThoughtSignature != "sig_only_456" { + t.Fatalf("expected signature-only part, got thought=%v text=%q signature=%q", + parts[1].Thought, parts[1].Text, parts[1].ThoughtSignature) + } + case "Gemini model - use dummy signature": + if len(parts) != 3 { + t.Fatalf("expected 3 parts, got %d", len(parts)) + } + if !parts[1].Thought || parts[1].ThoughtSignature != dummyThoughtSignature { + t.Fatalf("expected dummy thought signature, got thought=%v signature=%q", + parts[1].Thought, parts[1].ThoughtSignature) + } + } }) } } +func TestBuildParts_ToolUseSignatureHandling(t *testing.T) { + content := `[ + {"type": "tool_use", "id": "t1", "name": "Bash", "input": {"command": "ls"}, "signature": "sig_tool_abc"} + ]` + + t.Run("Claude preserve tool_use signature", func(t *testing.T) { + toolIDToName := make(map[string]string) + parts, err := buildParts(json.RawMessage(content), toolIDToName, thoughtSignatureModePreserve) + if err != nil { + t.Fatalf("buildParts() error = %v", err) + } + if len(parts) != 1 || parts[0].FunctionCall == nil { + t.Fatalf("expected 1 functionCall part, got %+v", parts) + } + if parts[0].ThoughtSignature != "sig_tool_abc" { + t.Fatalf("expected tool signature sig_tool_abc, got %q", parts[0].ThoughtSignature) + } + }) + + t.Run("Gemini uses dummy tool_use signature", func(t *testing.T) { + toolIDToName := make(map[string]string) + parts, err := buildParts(json.RawMessage(content), toolIDToName, thoughtSignatureModeDummy) + if err != nil { + t.Fatalf("buildParts() error = %v", err) + } + if len(parts) != 1 || parts[0].FunctionCall == nil { + t.Fatalf("expected 1 functionCall part, got %+v", parts) + } + if parts[0].ThoughtSignature != dummyThoughtSignature { + t.Fatalf("expected dummy tool signature %q, got %q", dummyThoughtSignature, parts[0].ThoughtSignature) + } + }) +} + // TestBuildTools_CustomTypeTools 测试custom类型工具转换 func TestBuildTools_CustomTypeTools(t *testing.T) { tests := []struct { diff --git a/backend/internal/pkg/geminicli/constants.go b/backend/internal/pkg/geminicli/constants.go index 63f48727..14cfa3a1 100644 --- a/backend/internal/pkg/geminicli/constants.go +++ b/backend/internal/pkg/geminicli/constants.go @@ -26,6 +26,10 @@ const ( // https://www.googleapis.com/auth/generative-language.retriever (often with cloud-platform). DefaultAIStudioScopes = "https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/generative-language.retriever" + // DefaultScopes for Google One (personal Google accounts with Gemini access) + // Includes generative-language for Gemini API access and drive.readonly for storage tier detection + DefaultGoogleOneScopes = "https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/generative-language.retriever https://www.googleapis.com/auth/drive.readonly https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/userinfo.profile" + // GeminiCLIRedirectURI is the redirect URI used by Gemini CLI for Code Assist OAuth. GeminiCLIRedirectURI = "https://codeassist.google.com/authcode" diff --git a/backend/internal/pkg/geminicli/oauth.go b/backend/internal/pkg/geminicli/oauth.go index f93d99b9..c75b3dc5 100644 --- a/backend/internal/pkg/geminicli/oauth.go +++ b/backend/internal/pkg/geminicli/oauth.go @@ -172,14 +172,19 @@ func EffectiveOAuthConfig(cfg OAuthConfig, oauthType string) (OAuthConfig, error if effective.Scopes == "" { // Use different default scopes based on OAuth type - if oauthType == "ai_studio" { + switch oauthType { + case "ai_studio": // Built-in client can't request some AI Studio scopes (notably generative-language). if isBuiltinClient { effective.Scopes = DefaultCodeAssistScopes } else { effective.Scopes = DefaultAIStudioScopes } - } else { + case "google_one": + // Google One accounts need generative-language scope for Gemini API access + // and drive.readonly scope for storage tier detection + effective.Scopes = DefaultGoogleOneScopes + default: // Default to Code Assist scopes effective.Scopes = DefaultCodeAssistScopes } diff --git a/backend/internal/repository/gemini_oauth_client.go b/backend/internal/repository/gemini_oauth_client.go index bac8736b..b1c86853 100644 --- a/backend/internal/repository/gemini_oauth_client.go +++ b/backend/internal/repository/gemini_oauth_client.go @@ -30,13 +30,14 @@ func (c *geminiOAuthClient) ExchangeCode(ctx context.Context, oauthType, code, c // Use different OAuth clients based on oauthType: // - code_assist: always use built-in Gemini CLI OAuth client (public) + // - google_one: same as code_assist, uses built-in client for personal Google accounts // - ai_studio: requires a user-provided OAuth client oauthCfgInput := geminicli.OAuthConfig{ ClientID: c.cfg.Gemini.OAuth.ClientID, ClientSecret: c.cfg.Gemini.OAuth.ClientSecret, Scopes: c.cfg.Gemini.OAuth.Scopes, } - if oauthType == "code_assist" { + if oauthType == "code_assist" || oauthType == "google_one" { oauthCfgInput.ClientID = "" oauthCfgInput.ClientSecret = "" } @@ -77,7 +78,7 @@ func (c *geminiOAuthClient) RefreshToken(ctx context.Context, oauthType, refresh ClientSecret: c.cfg.Gemini.OAuth.ClientSecret, Scopes: c.cfg.Gemini.OAuth.Scopes, } - if oauthType == "code_assist" { + if oauthType == "code_assist" || oauthType == "google_one" { oauthCfgInput.ClientID = "" oauthCfgInput.ClientSecret = "" } diff --git a/backend/internal/service/antigravity_gateway_service.go b/backend/internal/service/antigravity_gateway_service.go index 267d7548..be908189 100644 --- a/backend/internal/service/antigravity_gateway_service.go +++ b/backend/internal/service/antigravity_gateway_service.go @@ -307,6 +307,74 @@ func (s *AntigravityGatewayService) unwrapV1InternalResponse(body []byte) ([]byt return body, nil } +// isSignatureRelatedError 检测是否为 signature 相关的 400 错误 +func isSignatureRelatedError(statusCode int, body []byte) bool { + if statusCode != 400 { + return false + } + + bodyStr := strings.ToLower(string(body)) + keywords := []string{ + "signature", + "thought_signature", + "thoughtsignature", + "thinking", + "invalid signature", + "signature validation", + } + + for _, keyword := range keywords { + if strings.Contains(bodyStr, keyword) { + return true + } + } + return false +} + +// stripThinkingFromClaudeRequest 从 Claude 请求中移除所有 thinking 相关内容 +func stripThinkingFromClaudeRequest(req *antigravity.ClaudeRequest) *antigravity.ClaudeRequest { + // 创建副本 + stripped := *req + + // 移除 thinking 配置 + stripped.Thinking = nil + + // 移除消息中的 thinking 块 + if len(stripped.Messages) > 0 { + newMessages := make([]antigravity.ClaudeMessage, 0, len(stripped.Messages)) + for _, msg := range stripped.Messages { + newMsg := msg + + // 如果 content 是数组，过滤 thinking 块 + var blocks []map[string]any + if err := json.Unmarshal(msg.Content, &blocks); err == nil { + filtered := make([]map[string]any, 0, len(blocks)) + for _, block := range blocks { + // 跳过有 type="thinking" 的块 + if blockType, ok := block["type"].(string); ok && blockType == "thinking" { + continue + } + // 跳过没有 type 但有 thinking 字段的块（untyped thinking blocks） + if _, hasType := block["type"]; !hasType { + if _, hasThinking := block["thinking"]; hasThinking { + continue + } + } + filtered = append(filtered, block) + } + if newContent, err := json.Marshal(filtered); err == nil { + newMsg.Content = newContent + } + } + + newMessages = append(newMessages, newMsg) + } + stripped.Messages = newMessages + } + + return &stripped +} + // Forward 转发 Claude 协议请求（Claude → Gemini 转换） func (s *AntigravityGatewayService) Forward(ctx context.Context, c *gin.Context, account *Account, body []byte) (*ForwardResult, error) { startTime := time.Now() @@ -414,11 +482,70 @@ func (s *AntigravityGatewayService) Forward(ctx context.Context, c *gin.Context, respBody, _ := io.ReadAll(io.LimitReader(resp.Body, 2<<20)) s.handleUpstreamError(ctx, account, resp.StatusCode, resp.Header, respBody) - if s.shouldFailoverUpstreamError(resp.StatusCode) { - return nil, &UpstreamFailoverError{StatusCode: resp.StatusCode} + // Auto 模式：检测 signature 错误并自动降级重试 + if isSignatureRelatedError(resp.StatusCode, respBody) && claudeReq.Thinking != nil { + log.Printf("[Antigravity] Detected signature-related error, retrying without thinking blocks (account: %s, model: %s)", account.Name, mappedModel) + + // 关闭原始响应，释放连接（respBody 已读取到内存） + _ = resp.Body.Close() + + // 移除 thinking 块并重试一次 + strippedReq := stripThinkingFromClaudeRequest(&claudeReq) + strippedBody, err := antigravity.TransformClaudeToGemini(strippedReq, projectID, mappedModel) + if err != nil { + log.Printf("[Antigravity] Failed to transform stripped request: %v", err) + // 降级失败，返回原始错误 + if s.shouldFailoverUpstreamError(resp.StatusCode) { + return nil, &UpstreamFailoverError{StatusCode: resp.StatusCode} + } + return nil, s.writeMappedClaudeError(c, resp.StatusCode, respBody) + } + + // 发送降级请求 + retryReq, err := antigravity.NewAPIRequest(ctx, action, accessToken, strippedBody) + if err != nil { + log.Printf("[Antigravity] Failed to create retry request: %v", err) + if s.shouldFailoverUpstreamError(resp.StatusCode) { + return nil, &UpstreamFailoverError{StatusCode: resp.StatusCode} + } + return nil, s.writeMappedClaudeError(c, resp.StatusCode, respBody) + } + + retryResp, err := s.httpUpstream.Do(retryReq, proxyURL, account.ID, account.Concurrency) + if err != nil { + log.Printf("[Antigravity] Retry request failed: %v", err) + if s.shouldFailoverUpstreamError(resp.StatusCode) { + return nil, &UpstreamFailoverError{StatusCode: resp.StatusCode} + } + return nil, s.writeMappedClaudeError(c, resp.StatusCode, respBody) + } + + // 如果重试成功，使用重试的响应（不要 return，让后面的代码处理响应） + if retryResp.StatusCode < 400 { + log.Printf("[Antigravity] Retry succeeded after stripping thinking blocks (account: %s, model: %s)", account.Name, mappedModel) + resp = retryResp + } else { + // 重试也失败，返回重试的错误 + retryRespBody, _ := io.ReadAll(io.LimitReader(retryResp.Body, 2<<20)) + _ = retryResp.Body.Close() + log.Printf("[Antigravity] Retry also failed with status %d: %s", retryResp.StatusCode, string(retryRespBody)) + s.handleUpstreamError(ctx, account, retryResp.StatusCode, retryResp.Header, retryRespBody) + + if s.shouldFailoverUpstreamError(retryResp.StatusCode) { + return nil, &UpstreamFailoverError{StatusCode: retryResp.StatusCode} + } + return nil, s.writeMappedClaudeError(c, retryResp.StatusCode, retryRespBody) + } } - return nil, s.writeMappedClaudeError(c, resp.StatusCode, respBody) + // 不是 signature 错误，或者已经没有 thinking 块，直接返回错误 + if resp.StatusCode >= 400 { + if s.shouldFailoverUpstreamError(resp.StatusCode) { + return nil, &UpstreamFailoverError{StatusCode: resp.StatusCode} + } + + return nil, s.writeMappedClaudeError(c, resp.StatusCode, respBody) + } } requestID := resp.Header.Get("x-request-id") diff --git a/backend/internal/service/gateway_request.go b/backend/internal/service/gateway_request.go index fbec1371..32e9ffba 100644 --- a/backend/internal/service/gateway_request.go +++ b/backend/internal/service/gateway_request.go @@ -1,6 +1,7 @@ package service import ( + "bytes" "encoding/json" "fmt" ) @@ -70,3 +71,85 @@ func ParseGatewayRequest(body []byte) (*ParsedRequest, error) { return parsed, nil } + +// FilterThinkingBlocks removes thinking blocks from request body +// Returns filtered body or original body if filtering fails (fail-safe) +// This prevents 400 errors from invalid thinking block signatures +func FilterThinkingBlocks(body []byte) []byte { + // Fast path: if body doesn't contain "thinking", skip parsing + if !bytes.Contains(body, []byte("thinking")) { + return body + } + + var req map[string]any + if err := json.Unmarshal(body, &req); err != nil { + return body // Return original on parse error + } + + messages, ok := req["messages"].([]any) + if !ok { + return body // No messages array + } + + filtered := false + for _, msg := range messages { + msgMap, ok := msg.(map[string]any) + if !ok { + continue + } + + content, ok := msgMap["content"].([]any) + if !ok { + continue + } + + // Filter thinking blocks from content array + newContent := make([]any, 0, len(content)) + filteredThisMessage := false + for _, block := range content { + blockMap, ok := block.(map[string]any) + if !ok { + newContent = append(newContent, block) + continue + } + + blockType, _ := blockMap["type"].(string) + // Explicit Anthropic-style thinking block: {"type":"thinking", ...} + if blockType == "thinking" { + filtered = true + filteredThisMessage = true + continue // Skip thinking blocks + } + + // Some clients send the "thinking" object without a "type" discriminator. + // Vertex/Claude still expects a signature for any thinking block, so we drop it. + // We intentionally do not drop other typed blocks (e.g. tool_use) that might + // legitimately contain a "thinking" key inside their payload. + if blockType == "" { + if _, hasThinking := blockMap["thinking"]; hasThinking { + filtered = true + filteredThisMessage = true + continue // Skip thinking blocks + } + } + + newContent = append(newContent, block) + } + + if filteredThisMessage { + msgMap["content"] = newContent + } + } + + if !filtered { + return body // No changes needed + } + + // Re-serialize + newBody, err := json.Marshal(req) + if err != nil { + return body // Return original on marshal error + } + + return newBody +} diff --git a/backend/internal/service/gateway_request_test.go b/backend/internal/service/gateway_request_test.go index 5d411e2c..eb8af1da 100644 --- a/backend/internal/service/gateway_request_test.go +++ b/backend/internal/service/gateway_request_test.go @@ -1,6 +1,7 @@ package service import ( + "encoding/json" "testing" "github.com/stretchr/testify/require" @@ -38,3 +39,115 @@ func TestParseGatewayRequest_InvalidStreamType(t *testing.T) { _, err := ParseGatewayRequest(body) require.Error(t, err) } + +func TestFilterThinkingBlocks(t *testing.T) { + containsThinkingBlock := func(body []byte) bool { + var req map[string]any + if err := json.Unmarshal(body, &req); err != nil { + return false + } + messages, ok := req["messages"].([]any) + if !ok { + return false + } + for _, msg := range messages { + msgMap, ok := msg.(map[string]any) + if !ok { + continue + } + content, ok := msgMap["content"].([]any) + if !ok { + continue + } + for _, block := range content { + blockMap, ok := block.(map[string]any) + if !ok { + continue + } + blockType, _ := blockMap["type"].(string) + if blockType == "thinking" { + return true + } + if blockType == "" { + if _, hasThinking := blockMap["thinking"]; hasThinking { + return true + } + } + } + } + return false + } + + tests := []struct { + name string + input string + shouldFilter bool + expectError bool + }{ + { + name: "filters thinking blocks", + input: `{"model":"claude-3-5-sonnet-20241022","messages":[{"role":"user","content":[{"type":"text","text":"Hello"},{"type":"thinking","thinking":"internal","signature":"invalid"},{"type":"text","text":"World"}]}]}`, + shouldFilter: true, + }, + { + name: "handles no thinking blocks", + input: `{"model":"claude-3-5-sonnet-20241022","messages":[{"role":"user","content":[{"type":"text","text":"Hello"}]}]}`, + shouldFilter: false, + }, + { + name: "handles invalid JSON gracefully", + input: `{invalid json`, + shouldFilter: false, + expectError: true, + }, + { + name: "handles multiple messages with thinking blocks", + input: `{"messages":[{"role":"user","content":[{"type":"text","text":"A"}]},{"role":"assistant","content":[{"type":"thinking","thinking":"think"},{"type":"text","text":"B"}]}]}`, + shouldFilter: true, + }, + { + name: "filters thinking blocks without type discriminator", + input: `{"messages":[{"role":"assistant","content":[{"thinking":{"text":"internal"}},{"type":"text","text":"B"}]}]}`, + shouldFilter: true, + }, + { + name: "does not filter tool_use input fields named thinking", + input: `{"messages":[{"role":"user","content":[{"type":"tool_use","id":"t1","name":"foo","input":{"thinking":"keepme","x":1}},{"type":"text","text":"Hello"}]}]}`, + shouldFilter: false, + }, + { + name: "handles empty messages array", + input: `{"messages":[]}`, + shouldFilter: false, + }, + { + name: "handles missing messages field", + input: `{"model":"claude-3"}`, + shouldFilter: false, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + result := FilterThinkingBlocks([]byte(tt.input)) + + if tt.expectError { + // For invalid JSON, should return original + require.Equal(t, tt.input, string(result)) + return + } + + if tt.shouldFilter { + require.False(t, containsThinkingBlock(result)) + } else { + // Ensure we don't rewrite JSON when no filtering is needed. + require.Equal(t, tt.input, string(result)) + } + + // Verify valid JSON returned (unless input was invalid) + var parsed map[string]any + err := json.Unmarshal(result, &parsed) + require.NoError(t, err) + }) + } +} diff --git a/backend/internal/service/gateway_service.go b/backend/internal/service/gateway_service.go index f735d2d8..d78507b6 100644 --- a/backend/internal/service/gateway_service.go +++ b/backend/internal/service/gateway_service.go @@ -1136,6 +1136,10 @@ func (s *GatewayService) buildUpstreamRequest(ctx context.Context, c *gin.Contex } } + // Filter thinking blocks from request body (prevents 400 errors from missing/invalid signatures). + // We apply this for the main /v1/messages path as well as count_tokens. + body = FilterThinkingBlocks(body) + req, err := http.NewRequestWithContext(ctx, "POST", targetURL, bytes.NewReader(body)) if err != nil { return nil, err @@ -1862,6 +1866,9 @@ func (s *GatewayService) buildCountTokensRequest(ctx context.Context, c *gin.Con } } + // Filter thinking blocks from request body (prevents 400 errors from invalid signatures) + body = FilterThinkingBlocks(body) + req, err := http.NewRequestWithContext(ctx, "POST", targetURL, bytes.NewReader(body)) if err != nil { return nil, err diff --git a/deploy/docker-compose.override.yml b/deploy/docker-compose.override.yml deleted file mode 100644 index d877ff50..00000000 --- a/deploy/docker-compose.override.yml +++ /dev/null @@ -1,21 +0,0 @@ -# ============================================================================= -# Docker Compose Override for Local Development -# ============================================================================= -# This file automatically extends docker-compose-test.yml -# Usage: docker-compose -f docker-compose-test.yml up -d -# ============================================================================= - -services: - # =========================================================================== - # PostgreSQL - 暴露端口用于本地开发 - # =========================================================================== - postgres: - ports: - - "127.0.0.1:5432:5432" - - # =========================================================================== - # Redis - 暴露端口用于本地开发 - # =========================================================================== - redis: - ports: - - "127.0.0.1:6379:6379" diff --git a/deploy/docker-compose.override.yml.example b/deploy/docker-compose.override.yml.example new file mode 100644 index 00000000..297724f5 --- /dev/null +++ b/deploy/docker-compose.override.yml.example @@ -0,0 +1,137 @@ +# ============================================================================= +# Docker Compose Override Configuration Example +# ============================================================================= +# This file provides examples for customizing the Docker Compose setup. +# Copy this file to docker-compose.override.yml and modify as needed. +# +# Usage: +# cp docker-compose.override.yml.example docker-compose.override.yml +# # Edit docker-compose.override.yml with your settings +# docker-compose up -d +# +# IMPORTANT: docker-compose.override.yml is gitignored and will not be committed. +# ============================================================================= + +# ============================================================================= +# Scenario 1: Use External Database and Redis (Recommended for Production) +# ============================================================================= +# Use this when you have PostgreSQL and Redis running on the host machine +# or on separate servers. +# +# Prerequisites: +# - PostgreSQL running on host (accessible via host.docker.internal) +# - Redis running on host (accessible via host.docker.internal) +# - Update DATABASE_PORT and REDIS_PORT in .env file if using non-standard ports +# +# Security Notes: +# - Ensure PostgreSQL pg_hba.conf allows connections from Docker network +# - Use strong passwords for database and Redis +# - Consider using SSL/TLS for database connections in production +# ============================================================================= + +services: + sub2api: + # Remove dependencies on containerized postgres/redis + depends_on: [] + + # Enable access to host machine services + extra_hosts: + - "host.docker.internal:host-gateway" + + # Override database and Redis connection settings + environment: + # PostgreSQL Configuration + DATABASE_HOST: host.docker.internal + DATABASE_PORT: "5678" # Change to your PostgreSQL port + # DATABASE_USER: postgres # Uncomment to override + # DATABASE_PASSWORD: your_password # Uncomment to override + # DATABASE_DBNAME: sub2api # Uncomment to override + + # Redis Configuration + REDIS_HOST: host.docker.internal + REDIS_PORT: "6379" # Change to your Redis port + # REDIS_PASSWORD: your_redis_password # Uncomment if Redis requires auth + # REDIS_DB: 0 # Uncomment to override + + # Disable containerized PostgreSQL + postgres: + deploy: + replicas: 0 + scale: 0 + + # Disable containerized Redis + redis: + deploy: + replicas: 0 + scale: 0 + +# ============================================================================= +# Scenario 2: Development with Local Services (Alternative) +# ============================================================================= +# Uncomment this section if you want to use the containerized postgres/redis +# but expose their ports for local development tools. +# +# Usage: Comment out Scenario 1 above and uncomment this section. +# ============================================================================= + +# services: +# sub2api: +# # Keep default dependencies +# pass +# +# postgres: +# ports: +# - "127.0.0.1:5432:5432" # Expose PostgreSQL on localhost +# +# redis: +# ports: +# - "127.0.0.1:6379:6379" # Expose Redis on localhost + +# ============================================================================= +# Scenario 3: Custom Network Configuration +# ============================================================================= +# Uncomment if you need to connect to an existing Docker network +# ============================================================================= + +# networks: +# default: +# external: true +# name: your-existing-network + +# ============================================================================= +# Scenario 4: Resource Limits (Production) +# ============================================================================= +# Uncomment to set resource limits for the sub2api container +# ============================================================================= + +# services: +# sub2api: +# deploy: +# resources: +# limits: +# cpus: '2.0' +# memory: 2G +# reservations: +# cpus: '1.0' +# memory: 1G + +# ============================================================================= +# Scenario 5: Custom Volumes +# ============================================================================= +# Uncomment to mount additional volumes (e.g., for logs, backups) +# ============================================================================= + +# services: +# sub2api: +# volumes: +# - ./logs:/app/logs +# - ./backups:/app/backups + +# ============================================================================= +# Additional Notes +# ============================================================================= +# - This file overrides settings in docker-compose.yml +# - Environment variables in .env file take precedence +# - For more information, see: https://docs.docker.com/compose/extends/ +# - Check the main README.md for detailed configuration instructions +# =============================================================================