Commit Graph

498 Commits

Author SHA1 Message Date
t0ng7u
7cfeb6e87c Merge remote-tracking branch 'origin/alpha' into refactor/model-pricing 2025-08-07 11:09:28 +08:00
Xyfacai
0c0caad827 refactor: 调整模型匹配 2025-08-06 20:09:22 +08:00
t0ng7u
7c814a5fd9 🚀 refactor: migrate vendor-count aggregation to model layer & align frontend logic
Summary
• Backend
  – Moved duplicate-name validation and total vendor-count aggregation from controllers (`controller/model_meta.go`, `controller/vendor_meta.go`, `controller/prefill_group.go`) to model layer (`model/model_meta.go`, `model/vendor_meta.go`, `model/prefill_group.go`).
  – Added `GetVendorModelCounts()` and `Is*NameDuplicated()` helpers; controllers now call these instead of duplicating queries.
  – API response for `/api/models` now returns `vendor_counts` with per-vendor totals across all pages, plus `all` summary.
  – Removed redundant checks and unused imports, eliminating `go vet` warnings.

• Frontend
  – `useModelsData.js` updated to consume backend-supplied `vendor_counts`, calculate the `all` total once, and drop legacy client-side counting logic.
  – Simplified initial data flow: first render now triggers only one models request.
  – Deleted obsolete `updateVendorCounts` helper and related comments.
  – Ensured search flow also sets `vendorCounts`, keeping tab badges accurate.

Why
This refactor enforces single-responsibility (aggregation in model layer), delivers consistent totals irrespective of pagination, and removes redundant client queries, leading to cleaner code and better performance.
2025-08-06 01:40:08 +08:00
t0ng7u
d61a862fa2 Merge remote-tracking branch 'origin/alpha' into refactor/model-pricing 2025-08-05 23:19:24 +08:00
t0ng7u
327a0ca323 🚀 refactor: refine pricing refresh logic & hide disabled models
Summary
-------
1. Pricing generation
   • `model/pricing.go`: skip any model whose `status != 1` when building
     `pricingMap`, ensuring disabled models are never returned to the
     front-end.

2. Cache refresh placement
   • `controller/model_meta.go`
     – Removed `model.RefreshPricing()` from pure read handlers
       (`GetAllModelsMeta`, `SearchModelsMeta`).
     – Kept refresh only in mutating handlers
       (`Create`, `Update`, `Delete`), guaranteeing data is updated
       immediately after an admin change while avoiding redundant work
       on every read.

Result
------
Front-end no longer receives information about disabled models, and
pricing cache refreshes occur exactly when model data is modified,
improving efficiency and consistency.
2025-08-05 23:18:12 +08:00
t0ng7u
d951485431 feat: enhance soft-delete handling & boost pricing cache performance
Summary
-------
This commit unifies soft-delete behaviour across meta tables and
introduces an in-memory cache for model pricing look-ups to improve
throughput under high concurrency.

Details
-------
Soft-delete consistency
• PrefillGroup / Vendor / Model
  – Added `gorm.DeletedAt` field with `json:"-" gorm:"index"`.
  – Replaced plain `uniqueIndex` with partial unique indexes
    `uniqueIndex:<name>,where:deleted_at IS NULL`
    allowing duplicate keys after logical deletion while preserving
    uniqueness for active rows.
• Imports updated to include `gorm.io/gorm`.
• JSON output now hides `deleted_at`, matching existing tables.

High-throughput pricing cache
• model/pricing.go
  – Added thread-safe maps `modelEnableGroups` & `modelQuotaTypeMap`
    plus RW-mutex for O(1) access.
  – `updatePricing()` now refreshes these maps alongside `pricingMap`.
• model/model_extra.go
  – Rewrote `GetModelEnableGroups` & `GetModelQuotaType` to read from
    the new maps, falling back to automatic refresh via `GetPricing()`.

Misc
• Retained `RefreshPricing()` helper for immediate cache invalidation
  after admin actions.
• All modified files pass linter; no breaking DB migrations required
  (handled by AutoMigrate).

Result
------
– Soft-delete logic is transparent, safe, and allows record “revival”.
– Pricing-related queries are now constant-time, reducing CPU usage and
  latency under load.
2025-08-05 22:26:19 +08:00
Calcium-Ion
306a1a3f57 Merge pull request #1507 from QuantumNous/multi-key-manage
feat: implement channel-specific locking for thread-safe polling
2025-08-05 20:40:26 +08:00
t0ng7u
512850e83d Merge remote-tracking branch 'origin/alpha' into refactor/model-pricing 2025-08-04 21:37:38 +08:00
t0ng7u
0e9c3cde7c 🏗️ refactor: Replace model categories with vendor-based filtering and optimize data structure
- **Backend Changes:**
  - Refactor pricing API to return separate vendors array with ID-based model references
  - Remove redundant vendor_name/vendor_icon fields from pricing records, use vendor_id only
  - Add vendor_description to pricing response for frontend display
  - Maintain 1-minute cache protection for pricing endpoint security

- **Frontend Data Flow:**
  - Update useModelPricingData hook to build vendorsMap from API response
  - Enhance model records with vendor info during data processing
  - Pass vendorsMap through component hierarchy for consistent vendor data access

- **UI Component Replacements:**
  - Replace PricingCategories with PricingVendors component for vendor-based filtering
  - Replace PricingCategoryIntro with PricingVendorIntro in header section
  - Remove all model category related components and logic

- **Header Improvements:**
  - Implement vendor intro with real backend data (name, icon, description)
  - Add text collapsible feature (2-line limit with expand/collapse functionality)
  - Support carousel animation for "All Vendors" view with vendor icon rotation

- **Model Detail Modal Enhancements:**
  - Update ModelHeader to use real vendor icons via getLobeHubIcon()
  - Move tags from header to ModelBasicInfo content area to avoid SideSheet title width constraints
  - Display only custom tags from backend with stringToColor() for consistent styling
  - Use Space component with wrap property for proper tag layout

- **Table View Optimizations:**
  - Integrate RenderUtils for description and tags columns
  - Implement renderLimitedItems for tags (max 3 visible, +x popover for overflow)
  - Use renderDescription for text truncation with tooltip support

- **Filter Logic Updates:**
  - Vendor filter shows disabled options instead of hiding when no models match
  - Include "Unknown Vendor" category for models without vendor information
  - Remove all hardcoded vendor descriptions, use real backend data

- **Code Quality:**
  - Fix import paths after component relocation
  - Remove unused model category utilities and hardcoded mappings
  - Ensure consistent vendor data usage across all pricing views
  - Maintain backward compatibility with existing pricing calculation logic

This refactor provides a more scalable vendor-based architecture while eliminating
data redundancy and improving user experience with real-time backend data integration.
2025-08-04 21:36:31 +08:00
CaIon
8cce3cc84a feat: implement channel-specific locking for thread-safe polling 2025-08-04 20:44:19 +08:00
Calcium-Ion
9c079d04a8 Merge pull request #1487 from seefs001/feature/2fa
feat: implement two-factor authentication (2FA) support with user login and settings integration
2025-08-04 19:54:31 +08:00
CaIon
8357b15fec feat: enhance multi-key management with pagination and statistics 2025-08-04 17:15:32 +08:00
CaIon
ecdd9d1ccb feat: add multi-key management 2025-08-04 16:52:31 +08:00
t0ng7u
fc69f4f757 feat: add model name matching rules with priority-based lookup
Add flexible model name matching system to support different matching patterns:

Backend changes:
- Add `name_rule` field to Model struct with 4 matching types:
  * 0: Exact match (default)
  * 1: Prefix match
  * 2: Contains match
  * 3: Suffix match
- Implement `FindModelByNameWithRule` function with priority order:
  exact > prefix > suffix > contains
- Add database migration for new `name_rule` column

Frontend changes:
- Add "Match Type" column in models table with colored tags
- Add name rule selector in create/edit modal with validation
- Auto-set exact match and disable selection for preconfigured models
- Add explanatory text showing priority order
- Support i18n for all new UI elements

This enables users to define model patterns once and reuse configurations
across similar models, reducing repetitive setup while maintaining exact
match priority for specific overrides.

Closes: #[issue-number]
2025-08-04 16:01:56 +08:00
t0ng7u
5e70274003 💰 feat: Add model billing type (quota_type) support across backend & frontend
Summary
• Backend
  1. model/model_meta.go
     – Added `QuotaType` field to `Model` struct (JSON only, gorm `-`).
  2. model/model_groups.go
     – Implemented `GetModelQuotaType(modelName)` leveraging cached pricing map.
  3. controller/model_meta.go
     – Enhanced `fillModelExtra` to populate `QuotaType` using new helper.

• Frontend
  1. web/src/components/table/models/ModelsColumnDefs.js
     – Introduced `renderQuotaType` helper that visualises billing mode with coloured tags (`teal = per-call`, `violet = per-token`).
     – Added “计费类型” column (`quota_type`) to models table.

Why
Providing the billing mode alongside existing pricing/group information gives administrators instant visibility into whether each model is priced per call or per token, aligning UI with new backend metadata.

Notes
No database migration required – `quota_type` is transient, delivered via API. Frontend labels/colours can be adjusted via i18n or theme tokens if necessary.
2025-08-04 15:38:01 +08:00
t0ng7u
9f6027325c feat: Add prefill group management system for models
- Add new PrefillGroup model with CRUD operations
  * Support for model, tag, and endpoint group types
  * JSON storage for group items with GORM datatypes
  * Automatic database migration support

- Implement backend API endpoints
  * GET /api/prefill_group - List groups by type with admin auth
  * POST /api/prefill_group - Create new groups
  * PUT /api/prefill_group - Update existing groups
  * DELETE /api/prefill_group/:id - Delete groups

- Add comprehensive frontend management interface
  * PrefillGroupManagement component for group listing
  * EditPrefillGroupModal for group creation/editing
  * Integration with EditModelModal for auto-filling
  * Responsive design with CardTable and SideSheet

- Enhance model editing workflow
  * Tag group selection with auto-fill functionality
  * Endpoint group selection with auto-fill functionality
  * Seamless integration with existing model forms

- Create reusable UI components
  * Extract common rendering utilities to models/ui/
  * Shared renderLimitedItems and renderDescription functions
  * Consistent styling across all model-related components

- Improve user experience
  * Empty state illustrations matching existing patterns
  * Fixed column positioning for operation buttons
  * Item content display with +x indicators for overflow
  * Tooltip support for long descriptions
2025-08-04 02:54:37 +08:00
t0ng7u
b64c8ea56b 🚀 feat: expose “Enabled Groups” for models with real-time refresh
Backend
• model/model_meta.go
  – Added `EnableGroups []string` to Model struct
  – fillModelExtra now populates EnableGroups

• model/model_groups.go
  – New helper `GetModelEnableGroups` (reuses Pricing cache)

• model/pricing_refresh.go
  – Added `RefreshPricing()` to force immediate cache rebuild

• controller/model_meta.go
  – `GetAllModelsMeta` & `SearchModelsMeta` call `model.RefreshPricing()` before querying, ensuring groups / endpoints are up-to-date

Frontend
• ModelsColumnDefs.js
  – Added `renderGroups` util and “可用分组” table column displaying color-coded tags

Result
Admins can now see which user groups can access each model, and any ability/group changes are reflected instantly without the previous 1-minute delay.
2025-08-04 00:00:51 +08:00
t0ng7u
e74d3f4a8f feat: polish “Missing Models” UX & mobile actions layout
Overview
• Re-designed `MissingModelsModal` to align with `ModelTestModal` and deliver a cleaner, paginated experience.
• Improved mobile responsiveness for action buttons in `ModelsActions`.

Details
1. MissingModelsModal.jsx
   • Switched from `List` to `Table` for a more structured view.
   • Added search bar with live keyword filtering and clear icon.
   • Implemented pagination via `MODEL_TABLE_PAGE_SIZE`; auto-resets on search.
   • Dynamic rendering: when no data, show unified Empty state without column header.
   • Enhanced header layout with total-count subtitle and modal corner rounding.
   • Removed unused `Typography.Text` import.

2. ModelsActions.jsx
   • Set “Delete Selected Models” and “Missing Models” buttons to `flex-1 md:flex-initial`, placing them on the same row as “Add Model” on small screens.

Result
The “Missing Models” workflow now offers quicker discovery, a familiar table interface, and full mobile friendliness—without altering API behavior.
2025-08-03 22:51:24 +08:00
Seefs
398ae7156b refactor: improve error handling and database transactions in 2FA model methods 2025-08-03 10:49:55 +08:00
Seefs
d85eeabf11 fix: coderabbit review 2025-08-03 10:41:00 +08:00
t0ng7u
6a62654759 Merge branch 'alpha' into refactor/model-pricing 2025-08-02 22:26:40 +08:00
Seefs
c784a70277 feat: implement two-factor authentication (2FA) support with user login and settings integration 2025-08-02 14:53:28 +08:00
CaIon
74ec34da67 fix: improve error handling and readability in ability.go 2025-08-02 14:06:12 +08:00
CaIon
7188749cb3 feat: truncate abilities table before processing channels 2025-08-02 13:39:53 +08:00
CaIon
c28add55db feat: add caching for keys in channel structure and retain polling index during sync 2025-08-02 13:16:30 +08:00
CaIon
78f34a8245 feat: retain polling index for multi-key channels during sync 2025-08-02 13:04:48 +08:00
t0ng7u
eb42eb6f27 🐛 fix(model): preserve created_time on Model update and streamline field maintenance
The update operation for Model previously overwrote `created_time` with zero
because GORM included every struct field in the UPDATE statement.
This commit adjusts `Model.Update()` to:

* Call `Omit("created_time")` so the creation timestamp is never modified.
* Refresh `UpdatedTime` with `common.GetTimestamp()` before persisting.
* Delegate the remainder of the struct to GORM, eliminating the need to
  maintain an explicit allow-list whenever new fields are introduced.

No API contract is changed; existing CRUD endpoints continue to work
normally while data integrity for historical records is now guaranteed.
2025-08-01 02:21:14 +08:00
t0ng7u
232612898b 🔄 fix: improve vendor-tab filtering & counts, resolve SQL ambiguity, and reload data correctly
Backend
• model/model_meta.go
  – Import strconv
  – SearchModels: support numeric vendor ID filter vs. fuzzy name search
  – Explicitly order by `models.id` to avoid “ambiguous column name: id” error

Frontend
• hooks/useModelsData.js
  – Change vendor-filter API to pass vendor ID
  – Automatically reload models when `activeVendorKey` changes
  – Update vendor counts only when viewing “All” to preserve other tab totals
• Add missing effect in EditModelModal to refresh vendor list only when modal visible
• Other minor updates to keep lints clean

Result
Tabs now:
1. Trigger API requests on click
2. Show accurate per-vendor totals
3. Filter models without resetting other counts
Backend search handles both vendor IDs and names without SQL errors.
2025-07-31 23:30:45 +08:00
t0ng7u
6a37efb871 Merge branch 'alpha' into refactor/model-pricing 2025-07-31 22:28:59 +08:00
t0ng7u
af59b61f8a 🚀 feat: Introduce full Model & Vendor Management suite (backend + frontend) and UI refinements
Backend
• Add `model/model_meta.go` and `model/vendor_meta.go` defining Model & Vendor entities with CRUD helpers, soft-delete and time stamps
• Create corresponding controllers `controller/model_meta.go`, `controller/vendor_meta.go` and register routes in `router/api-router.go`
• Auto-migrate new tables in DB startup logic

Frontend
• Build complete “Model Management” module under `/console/models`
  - New pages, tables, filters, actions, hooks (`useModelsData`) and dynamic vendor tabs
  - Modals `EditModelModal.jsx` & unified `EditVendorModal.jsx`; latter now uses default confirm/cancel footer and mobile-friendly modal sizing (`full-width` / `small`) via `useIsMobile`
• Update sidebar (`SiderBar.js`) and routing (`App.js`) to surface the feature
• Add helper updates (`render.js`) incl. `stringToColor`, dynamic LobeHub icon retrieval, and tag color palettes

Table UX improvements
• Replace separate status column with inline Enable / Disable buttons in operation column (matching channel table style)
• Limit visible tags to max 3; overflow represented as “+x” tag with padded `Popover` showing remaining tags
• Color all tags deterministically using `stringToColor` for consistent theming
• Change vendor column tag color to white for better contrast

Misc
• Minor layout tweaks, compact-mode toggle relocation, lint fixes and TypeScript/ESLint clean-up

These changes collectively deliver end-to-end model & vendor administration while unifying visual language across management tables.
2025-07-31 22:28:09 +08:00
CaIon
196bafff03 fix: 修复被禁用的渠道无法测试的问题 2025-07-31 10:56:51 +08:00
CaIon
54447bf227 fix: remove debug print statement 2025-07-30 23:29:45 +08:00
CaIon
fc09051d8b fix: 修复缓存开启下自动禁用失效 2025-07-30 23:26:09 +08:00
Xyfacai
1f5ef24ecd feat: 显式指定 error 跳过重试 2025-07-30 22:35:31 +08:00
CaIon
8e3cf2eaab feat: support claude convert to gemini 2025-07-26 13:31:33 +08:00
CaIon
756a8c50d6 fix: improve error messages for channel retrieval failures in distributor and relay 2025-07-23 16:32:52 +08:00
CaIon
4d0037a40c fix(topup): update TradeNo field to include varchar type and index for better database performance 2025-07-20 19:20:37 +08:00
Xyfacai
7aa2972c3f fix(price): 未设置价格,错误返回模型价格匹配的名字 2025-07-20 10:12:36 +08:00
Xyfacai
986558fea7 fix(db): 修复 db migration 报错过多可能卡住的隐患 2025-07-20 10:11:35 +08:00
IcedTangerine
952b679ca3 Merge pull request #1352 from wzxjohn/feature/simple_stripe
Add stripe support and fix wrong top up loading state
2025-07-18 22:00:52 +08:00
feitianbubu
e269b3bfdd fix: mySQL does not support default false 2025-07-18 13:52:16 +08:00
t0ng7u
15f65bb558 🌐 feat: add configurable USD exchange-rate support across backend & frontend
Backend
- setting/payment.go: introduce default `USDExchangeRate` (7.3)
- model/option.go:
  • inject `USDExchangeRate` into `InitOptionMap`
  • persist & sync value in `updateOptionMap`
- controller/misc.go: expose `usd_exchange_rate` via `/api/status`

Frontend
- OperationSetting.js & SettingsGeneral.js:
  • extend state/inputs with `USDExchangeRate`
  • add form field “美元汇率 (non-top-up rate, pricing only)”
- ModelPricing.js already consumes `status.usd_exchange_rate`; no change needed

API
- Administrators can update the rate via `PUT /api/option` (key: `USDExchangeRate`)
- All clients receive the latest rate through `GET /api/status`

This closes the end-to-end flow for displaying model prices in both USD and CNY based on a configurable exchange rate.
2025-07-17 23:04:45 +08:00
CaIon
b161d6831f fix: 修复playground优先级失效 2025-07-17 22:26:38 +08:00
wzxjohn
fb9b5d31e8 Merge branch 'alpha' into feature/simple_stripe 2025-07-16 10:39:11 +08:00
t0ng7u
06ad5e3f8c 🐛 fix: multi-key channel sync and Vertex-AI key-upload edge cases
Backend
1. controller/channel.go
   • Always hydrate `ChannelInfo` from DB in `UpdateChannel`, keeping `IsMultiKey` true so `MultiKeySize` is recalculated.

2. model/channel.go
   • getKeys(): accept both newline-separated keys and JSON array (`[ {...}, {...} ]`).
   • Update(): reuse new parser-logic to recalc `MultiKeySize`; prune stale indices in `MultiKeyStatusList`.

Frontend
1. pages/Channel/EditChannel.js
   • `handleVertexUploadChange`
     – Reset `vertexErroredNames` on every change so the “ignored files” prompt always re-appears.
     – In single-key mode keep only the last file; in batch mode keep all valid files.
     – Parse files, display “以下文件解析失败,已忽略:…”.
   • Batch-toggle checkbox
     – When switching from batch→single while multiple files are present, show a confirm dialog and retain only the first file (synchronises state, form and local caches).
   • On opening the “new channel” side-sheet, clear `vertexErroredNames` to restore error prompts.

Result
• “已启用 x/x” count updates immediately after editing multi-key channels.
• Vertex-AI key upload works intuitively: proper error feedback, no duplicated files, and safe down-switch from batch to single mode.
2025-07-15 12:02:04 +08:00
t0ng7u
d22ee5d451 🐛 fix: preserve key data when editing channels & update MultiKeySize correctly
Backend
• `model/channel.go`
  – On multi-key channel updates, recalculate `MultiKeySize` from the current key list.
  – If the request omits `key`, fetch existing keys from DB to avoid resetting the count to `0/0`.
  – Remove out-of-range entries from `MultiKeyStatusList` to keep data consistent.

Frontend
• `web/src/pages/Channel/EditChannel.js`
  – Vertex AI channels no longer require re-uploading a key file in edit mode.
  – When no new key is provided during editing, exclude the `key` field from the payload to prevent overwriting the stored value.

These changes ensure that editing a channel no longer zeroes out the enabled key counter and that Vertex AI channels can be modified without mandatory key re-submission.
2025-07-13 00:09:27 +08:00
t0ng7u
203edaed50 🐛 fix: synchronize MultiKeySize after editing multi-key channels
1. Recalculate `ChannelInfo.MultiKeySize` based on the current key list when a multi-key channel is updated.
2. Purge any indexes in `MultiKeyStatusList` that exceed the new key count to avoid stale or out-of-range data.

This ensures the front-end display (`enabled x/x`) always reflects the real number of keys after users add or remove keys in edit mode.
2025-07-12 23:57:59 +08:00
CaIon
20607b0b5c feat(logs): add multi-key support in LogsTable and enhance log info generation 2025-07-12 15:14:55 +08:00
CaIon
50b76f4466 feat(channel): improve channel cache handling and add error checks for disabled channels 2025-07-12 14:20:59 +08:00
CaIon
23e4e25e9a feat(channel): implement thread-safe polling 2025-07-12 11:17:08 +08:00