Commit Graph

3593 Commits

Author SHA1 Message Date
t0ng7u
d951485431 feat: enhance soft-delete handling & boost pricing cache performance
Summary
-------
This commit unifies soft-delete behaviour across meta tables and
introduces an in-memory cache for model pricing look-ups to improve
throughput under high concurrency.

Details
-------
Soft-delete consistency
• PrefillGroup / Vendor / Model
  – Added `gorm.DeletedAt` field with `json:"-" gorm:"index"`.
  – Replaced plain `uniqueIndex` with partial unique indexes
    `uniqueIndex:<name>,where:deleted_at IS NULL`
    allowing duplicate keys after logical deletion while preserving
    uniqueness for active rows.
• Imports updated to include `gorm.io/gorm`.
• JSON output now hides `deleted_at`, matching existing tables.

High-throughput pricing cache
• model/pricing.go
  – Added thread-safe maps `modelEnableGroups` & `modelQuotaTypeMap`
    plus RW-mutex for O(1) access.
  – `updatePricing()` now refreshes these maps alongside `pricingMap`.
• model/model_extra.go
  – Rewrote `GetModelEnableGroups` & `GetModelQuotaType` to read from
    the new maps, falling back to automatic refresh via `GetPricing()`.

Misc
• Retained `RefreshPricing()` helper for immediate cache invalidation
  after admin actions.
• All modified files pass linter; no breaking DB migrations required
  (handled by AutoMigrate).

Result
------
– Soft-delete logic is transparent, safe, and allows record “revival”.
– Pricing-related queries are now constant-time, reducing CPU usage and
  latency under load.
2025-08-05 22:26:19 +08:00
t0ng7u
f3a1f98add 🐛 fix(models): eliminate vendor column flicker by loading vendors before models
Why:
• The vendor list API is separate from the models API, causing the “Vendor” column in `ModelsTable` to flash (rendering `'-'` first, then updating) after the table finishes loading.
• This visual jump degrades the user experience.

What:
• Updated `web/src/hooks/models/useModelsData.js`
  – In the initial `useEffect`, vendors are fetched first with `loadVendors()` and awaited.
  – Only after vendors are ready do we call `loadModels()`, ensuring `vendorMap` is populated before the table renders.

Outcome:
• The table now renders with complete vendor data on first paint, removing the flicker and providing a smoother UI.
2025-08-04 22:11:13 +08:00
t0ng7u
1ccc728e5d 💄 fix(pricing-card): align skeleton responsive grid with actual card layout
- Update PricingCardSkeleton grid classes from 'sm:grid-cols-2 lg:grid-cols-3'
  to 'xl:grid-cols-2 2xl:grid-cols-3' to match PricingCardView layout
- Ensures consistent column count between skeleton and actual content
  at same screen sizes
- Improves loading state visual consistency across different breakpoints
2025-08-04 22:03:12 +08:00
t0ng7u
11ee80d377 🍎 chore: modify the JSONEditor component import path 2025-08-04 21:58:10 +08:00
t0ng7u
512850e83d Merge remote-tracking branch 'origin/alpha' into refactor/model-pricing 2025-08-04 21:37:38 +08:00
t0ng7u
0e9c3cde7c 🏗️ refactor: Replace model categories with vendor-based filtering and optimize data structure
- **Backend Changes:**
  - Refactor pricing API to return separate vendors array with ID-based model references
  - Remove redundant vendor_name/vendor_icon fields from pricing records, use vendor_id only
  - Add vendor_description to pricing response for frontend display
  - Maintain 1-minute cache protection for pricing endpoint security

- **Frontend Data Flow:**
  - Update useModelPricingData hook to build vendorsMap from API response
  - Enhance model records with vendor info during data processing
  - Pass vendorsMap through component hierarchy for consistent vendor data access

- **UI Component Replacements:**
  - Replace PricingCategories with PricingVendors component for vendor-based filtering
  - Replace PricingCategoryIntro with PricingVendorIntro in header section
  - Remove all model category related components and logic

- **Header Improvements:**
  - Implement vendor intro with real backend data (name, icon, description)
  - Add text collapsible feature (2-line limit with expand/collapse functionality)
  - Support carousel animation for "All Vendors" view with vendor icon rotation

- **Model Detail Modal Enhancements:**
  - Update ModelHeader to use real vendor icons via getLobeHubIcon()
  - Move tags from header to ModelBasicInfo content area to avoid SideSheet title width constraints
  - Display only custom tags from backend with stringToColor() for consistent styling
  - Use Space component with wrap property for proper tag layout

- **Table View Optimizations:**
  - Integrate RenderUtils for description and tags columns
  - Implement renderLimitedItems for tags (max 3 visible, +x popover for overflow)
  - Use renderDescription for text truncation with tooltip support

- **Filter Logic Updates:**
  - Vendor filter shows disabled options instead of hiding when no models match
  - Include "Unknown Vendor" category for models without vendor information
  - Remove all hardcoded vendor descriptions, use real backend data

- **Code Quality:**
  - Fix import paths after component relocation
  - Remove unused model category utilities and hardcoded mappings
  - Ensure consistent vendor data usage across all pricing views
  - Maintain backward compatibility with existing pricing calculation logic

This refactor provides a more scalable vendor-based architecture while eliminating
data redundancy and improving user experience with real-time backend data integration.
2025-08-04 21:36:31 +08:00
Calcium-Ion
faaa5a2949 Merge pull request #1499 from QuantumNous/multi-key-manage
feat: improve layout and pagination handling in MultiKeyManageModal
2025-08-04 20:17:22 +08:00
CaIon
c00f5a17c8 feat: improve layout and pagination handling in MultiKeyManageModal 2025-08-04 20:16:51 +08:00
Calcium-Ion
9c079d04a8 Merge pull request #1487 from seefs001/feature/2fa
feat: implement two-factor authentication (2FA) support with user login and settings integration
2025-08-04 19:54:31 +08:00
Calcium-Ion
c9d4cdc57e Merge pull request #1498 from QuantumNous/multi-key-manage
feat: add multi-key management
2025-08-04 19:53:52 +08:00
CaIon
12b4e80d4b feat: add status filtering and bulk enable/disable functionality in multi-key management 2025-08-04 19:51:58 +08:00
CaIon
6e2a04f374 fix: correct option value for pagination in MultiKeyManageModal 2025-08-04 19:33:24 +08:00
CaIon
8357b15fec feat: enhance multi-key management with pagination and statistics 2025-08-04 17:15:32 +08:00
CaIon
ecdd9d1ccb feat: add multi-key management 2025-08-04 16:52:31 +08:00
t0ng7u
fc69f4f757 feat: add model name matching rules with priority-based lookup
Add flexible model name matching system to support different matching patterns:

Backend changes:
- Add `name_rule` field to Model struct with 4 matching types:
  * 0: Exact match (default)
  * 1: Prefix match
  * 2: Contains match
  * 3: Suffix match
- Implement `FindModelByNameWithRule` function with priority order:
  exact > prefix > suffix > contains
- Add database migration for new `name_rule` column

Frontend changes:
- Add "Match Type" column in models table with colored tags
- Add name rule selector in create/edit modal with validation
- Auto-set exact match and disable selection for preconfigured models
- Add explanatory text showing priority order
- Support i18n for all new UI elements

This enables users to define model patterns once and reuse configurations
across similar models, reducing repetitive setup while maintaining exact
match priority for specific overrides.

Closes: #[issue-number]
2025-08-04 16:01:56 +08:00
t0ng7u
5e70274003 💰 feat: Add model billing type (quota_type) support across backend & frontend
Summary
• Backend
  1. model/model_meta.go
     – Added `QuotaType` field to `Model` struct (JSON only, gorm `-`).
  2. model/model_groups.go
     – Implemented `GetModelQuotaType(modelName)` leveraging cached pricing map.
  3. controller/model_meta.go
     – Enhanced `fillModelExtra` to populate `QuotaType` using new helper.

• Frontend
  1. web/src/components/table/models/ModelsColumnDefs.js
     – Introduced `renderQuotaType` helper that visualises billing mode with coloured tags (`teal = per-call`, `violet = per-token`).
     – Added “计费类型” column (`quota_type`) to models table.

Why
Providing the billing mode alongside existing pricing/group information gives administrators instant visibility into whether each model is priced per call or per token, aligning UI with new backend metadata.

Notes
No database migration required – `quota_type` is transient, delivered via API. Frontend labels/colours can be adjusted via i18n or theme tokens if necessary.
2025-08-04 15:38:01 +08:00
t0ng7u
57b194c63f Merge remote-tracking branch 'origin/alpha' into refactor/model-pricing 2025-08-04 10:00:27 +08:00
Xyfacai
10b04416c1 fix: 修复gemini2openai 没有返回 usage 2025-08-04 09:06:57 +08:00
t0ng7u
9f6027325c feat: Add prefill group management system for models
- Add new PrefillGroup model with CRUD operations
  * Support for model, tag, and endpoint group types
  * JSON storage for group items with GORM datatypes
  * Automatic database migration support

- Implement backend API endpoints
  * GET /api/prefill_group - List groups by type with admin auth
  * POST /api/prefill_group - Create new groups
  * PUT /api/prefill_group - Update existing groups
  * DELETE /api/prefill_group/:id - Delete groups

- Add comprehensive frontend management interface
  * PrefillGroupManagement component for group listing
  * EditPrefillGroupModal for group creation/editing
  * Integration with EditModelModal for auto-filling
  * Responsive design with CardTable and SideSheet

- Enhance model editing workflow
  * Tag group selection with auto-fill functionality
  * Endpoint group selection with auto-fill functionality
  * Seamless integration with existing model forms

- Create reusable UI components
  * Extract common rendering utilities to models/ui/
  * Shared renderLimitedItems and renderDescription functions
  * Consistent styling across all model-related components

- Improve user experience
  * Empty state illustrations matching existing patterns
  * Fixed column positioning for operation buttons
  * Item content display with +x indicators for overflow
  * Tooltip support for long descriptions
2025-08-04 02:54:37 +08:00
t0ng7u
b64c8ea56b 🚀 feat: expose “Enabled Groups” for models with real-time refresh
Backend
• model/model_meta.go
  – Added `EnableGroups []string` to Model struct
  – fillModelExtra now populates EnableGroups

• model/model_groups.go
  – New helper `GetModelEnableGroups` (reuses Pricing cache)

• model/pricing_refresh.go
  – Added `RefreshPricing()` to force immediate cache rebuild

• controller/model_meta.go
  – `GetAllModelsMeta` & `SearchModelsMeta` call `model.RefreshPricing()` before querying, ensuring groups / endpoints are up-to-date

Frontend
• ModelsColumnDefs.js
  – Added `renderGroups` util and “可用分组” table column displaying color-coded tags

Result
Admins can now see which user groups can access each model, and any ability/group changes are reflected instantly without the previous 1-minute delay.
2025-08-04 00:00:51 +08:00
t0ng7u
e74d3f4a8f feat: polish “Missing Models” UX & mobile actions layout
Overview
• Re-designed `MissingModelsModal` to align with `ModelTestModal` and deliver a cleaner, paginated experience.
• Improved mobile responsiveness for action buttons in `ModelsActions`.

Details
1. MissingModelsModal.jsx
   • Switched from `List` to `Table` for a more structured view.
   • Added search bar with live keyword filtering and clear icon.
   • Implemented pagination via `MODEL_TABLE_PAGE_SIZE`; auto-resets on search.
   • Dynamic rendering: when no data, show unified Empty state without column header.
   • Enhanced header layout with total-count subtitle and modal corner rounding.
   • Removed unused `Typography.Text` import.

2. ModelsActions.jsx
   • Set “Delete Selected Models” and “Missing Models” buttons to `flex-1 md:flex-initial`, placing them on the same row as “Add Model” on small screens.

Result
The “Missing Models” workflow now offers quicker discovery, a familiar table interface, and full mobile friendliness—without altering API behavior.
2025-08-03 22:51:24 +08:00
t0ng7u
8a2aebf845 feat(edit-vendor-modal): add icon-library reference link & tidy status switch
Highlights
• Introduced `Typography.Text` link with `IconLink` in `extraText` for the **icon** field, pointing to LobeHub’s full icon catalogue; only “请点击我” is clickable for clarity.
• Added required imports for `Typography` and `IconLink`.
• Removed unnecessary `size="large"` prop from the status `Form.Switch` to align with default form styling.

These tweaks improve user guidance when selecting vendor icons and refine the modal’s visual consistency.
2025-08-03 19:45:58 +08:00
t0ng7u
984c8ee477 ♻️ refactor(models-table): extract reusable renderLimitedItems for list popovers
Introduce a generic `renderLimitedItems` helper within `ModelsColumnDefs.js` to eliminate duplicated logic for list-style columns.

Key changes
• Added `renderLimitedItems` to handle item limiting, “+N” indicator, and popover display.
• Migrated `renderTags`, `renderEndpoints`, and `renderBoundChannels` to use the new helper.
• Removed redundant inline implementations, reducing complexity and improving readability.
• Preserved previous UX: first 3 items shown, overflow accessible via popover.

This refactor streamlines code maintenance and ensures consistent behavior across related columns.
2025-08-03 19:31:29 +08:00
Seefs
398ae7156b refactor: improve error handling and database transactions in 2FA model methods 2025-08-03 10:49:55 +08:00
Seefs
d85eeabf11 fix: coderabbit review 2025-08-03 10:41:00 +08:00
t0ng7u
6a62654759 Merge branch 'alpha' into refactor/model-pricing 2025-08-02 22:26:40 +08:00
CaIon
c056a7ad7c feat: add support for multi-key channels in RelayInfo and access token caching 2025-08-02 22:12:15 +08:00
Seefs
c784a70277 feat: implement two-factor authentication (2FA) support with user login and settings integration 2025-08-02 14:53:28 +08:00
Calcium-Ion
e6c87907d5 Merge pull request #1486 from nekohy/fix-get-google-models
fix: correct Gemini channel model retrieval logic
2025-08-02 14:52:22 +08:00
Nekohy
71e9290142 fix: correct Gemini channel model retrieval logic 2025-08-02 14:19:32 +08:00
CaIon
74ec34da67 fix: improve error handling and readability in ability.go 2025-08-02 14:06:12 +08:00
CaIon
7188749cb3 feat: truncate abilities table before processing channels 2025-08-02 13:39:53 +08:00
CaIon
c28add55db feat: add caching for keys in channel structure and retain polling index during sync 2025-08-02 13:16:30 +08:00
CaIon
78f34a8245 feat: retain polling index for multi-key channels during sync 2025-08-02 13:04:48 +08:00
CaIon
97d6f10f15 feat: enhance ConvertGeminiRequest to set default role and handle YouTube video MIME type 2025-08-02 12:53:58 +08:00
Calcium-Ion
afefc4caca Merge pull request #1484 from QuantumNous/ConvertGeminiRequest
feat: Convert gemini request
2025-08-02 12:20:39 +08:00
CaIon
6abbd036f8 feat: add recordErrorLog option to NewAPIError for conditional error logging 2025-08-02 11:07:50 +08:00
CaIon
ef0db0f914 feat: implement key mode for multi-key channels with append/replace options 2025-08-02 10:57:03 +08:00
creamlike1024
e01986fdd4 Merge remote-tracking branch 'origin/alpha' into ConvertGeminiRequest 2025-08-01 22:42:48 +08:00
creamlike1024
a0c6ebe2d8 chore: remove debug log 2025-08-01 22:29:19 +08:00
creamlike1024
d2183af23f feat: convert gemini format to openai chat completions 2025-08-01 22:23:35 +08:00
CaIon
953f1bdc3c feat: add admin info to error logging with multi-key support 2025-08-01 18:19:28 +08:00
CaIon
e2429f20f8 fix: ensure ChannelIsMultiKey context key is set to false for single key retries 2025-08-01 18:09:20 +08:00
CaIon
f0945da4fb refactor: simplify streamResponseGeminiChat2OpenAI by removing hasImage return value and optimizing response text handling 2025-08-01 17:58:21 +08:00
CaIon
8df3de9ae5 fix: update JSONEditor to default to manual mode for invalid JSON and add error message for invalid data 2025-08-01 17:21:25 +08:00
Calcium-Ion
277cc1cac8 Merge pull request #1481 from seefs001/revert-1445-feature/claude-code
Revert "feat: add Claude Code channel support with OAuth integration"
2025-08-01 17:05:22 +08:00
CaIon
07a92293e4 fix: handle case where no response is received from Gemini API 2025-08-01 17:04:16 +08:00
t0ng7u
9730b9ba2d feat(ui): enhance tag input
1. EditModelModal quality-of-life
   • Added comma parsing to `Form.TagInput`; users can now paste
     `tag1, tag2 , tag3` to bulk-create tags.
   • Updated placeholder copy to reflect the new capability.

All files pass linting; no runtime changes outside the intended UI updates.
2025-08-01 03:00:12 +08:00
t0ng7u
508799c452 🎨 style(sidebar): unify highlight color & assign unique icon for Models
• Removed obsolete `sidebarIconColors` map and `getItemColor` util from
  SiderBar/render; all selected states now use the single CSS variable
  `--semi-color-primary` for both text and icons.
• Simplified `getLucideIcon`:
  – Added `Package` to Lucide imports.
  – Switched “models” case to `<Package />`, avoiding duplication with
    the Layers glyph.
  – Replaced per-key color logic with `iconColor` derived from the new
    uniform highlight color.
• Stripped any unused imports / dead code paths after the refactor.
• Lint passes; sidebar hover/focus behavior unchanged while visual
  consistency is improved.
2025-08-01 02:50:06 +08:00
t0ng7u
5e81ef4a44 ♻️ refactor(hooks/models): deduplicate useModelsData and optimize vendor-tab counts
Highlights
──────────
1. Removed code duplication
   • Introduced `extractItems` helper to safely unwrap API payloads.
   • Simplified `getFormValues` to a single-line fallback expression.
   • Replaced repeated list-extraction code in `loadModels`, `searchModels`,
     and `refreshVendorCounts` with the new helper.

2. Vendor tab accuracy & performance
   • Added `refreshVendorCounts` to recalc counts via a single lightweight
     request; invoked only when必要 (current tab ≠ "all“) to avoid redundancy.
   • `loadModels` still updates counts instantly when viewing "all", ensuring
     accurate numbers on initial load and page changes.

3. Misc clean-ups
   • Streamlined conditional URL building and state updates.
   • Confirmed all async branches include error handling with i18n messages.
   • Ran linter → zero issues.

Result: leaner, easier-to-maintain hook with correct, real-time vendor counts
and no repeated logic.
2025-08-01 02:39:12 +08:00