feat(migrations): 改进校验和错误提示和文档
- 增强迁移校验和不匹配的错误信息,提供具体解决方案 - 添加 migrations/README.md 文档说明迁移最佳实践 - 明确迁移不可变原则和正确的修改流程
This commit is contained in:
@@ -127,7 +127,15 @@ func applyMigrationsFS(ctx context.Context, db *sql.DB, fsys fs.FS) error {
|
|||||||
if existing != checksum {
|
if existing != checksum {
|
||||||
// 校验和不匹配意味着迁移文件在应用后被修改,这是危险的。
|
// 校验和不匹配意味着迁移文件在应用后被修改,这是危险的。
|
||||||
// 正确的做法是创建新的迁移文件来进行变更。
|
// 正确的做法是创建新的迁移文件来进行变更。
|
||||||
return fmt.Errorf("migration %s checksum mismatch (db=%s file=%s)", name, existing, checksum)
|
return fmt.Errorf(
|
||||||
|
"migration %s checksum mismatch (db=%s file=%s)\n"+
|
||||||
|
"This means the migration file was modified after being applied to the database.\n"+
|
||||||
|
"Solutions:\n"+
|
||||||
|
" 1. Revert to original: git log --oneline -- migrations/%s && git checkout <commit> -- migrations/%s\n"+
|
||||||
|
" 2. For new changes, create a new migration file instead of modifying existing ones\n"+
|
||||||
|
"Note: Modifying applied migrations breaks the immutability principle and can cause inconsistencies across environments.",
|
||||||
|
name, existing, checksum, name, name,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
continue // 迁移已应用且校验和匹配,跳过
|
continue // 迁移已应用且校验和匹配,跳过
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -26,10 +26,5 @@ UPDATE accounts
|
|||||||
SET credentials = credentials - 'tier_id'
|
SET credentials = credentials - 'tier_id'
|
||||||
WHERE platform = 'gemini'
|
WHERE platform = 'gemini'
|
||||||
AND type = 'oauth'
|
AND type = 'oauth'
|
||||||
AND jsonb_typeof(credentials) = 'object'
|
AND credentials->>'oauth_type' = 'code_assist';
|
||||||
AND credentials->>'tier_id' = 'LEGACY'
|
|
||||||
AND (
|
|
||||||
credentials->>'oauth_type' = 'code_assist'
|
|
||||||
OR (credentials->>'oauth_type' IS NULL AND credentials->>'project_id' IS NOT NULL)
|
|
||||||
);
|
|
||||||
-- +goose StatementEnd
|
-- +goose StatementEnd
|
||||||
|
|||||||
178
backend/migrations/README.md
Normal file
178
backend/migrations/README.md
Normal file
@@ -0,0 +1,178 @@
|
|||||||
|
# Database Migrations
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This directory contains SQL migration files for database schema changes. The migration system uses SHA256 checksums to ensure migration immutability and consistency across environments.
|
||||||
|
|
||||||
|
## Migration File Naming
|
||||||
|
|
||||||
|
Format: `NNN_description.sql`
|
||||||
|
- `NNN`: Sequential number (e.g., 001, 002, 003)
|
||||||
|
- `description`: Brief description in snake_case
|
||||||
|
|
||||||
|
Example: `017_add_gemini_tier_id.sql`
|
||||||
|
|
||||||
|
## Migration File Structure
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- +goose Up
|
||||||
|
-- +goose StatementBegin
|
||||||
|
-- Your forward migration SQL here
|
||||||
|
-- +goose StatementEnd
|
||||||
|
|
||||||
|
-- +goose Down
|
||||||
|
-- +goose StatementBegin
|
||||||
|
-- Your rollback migration SQL here
|
||||||
|
-- +goose StatementEnd
|
||||||
|
```
|
||||||
|
|
||||||
|
## Important Rules
|
||||||
|
|
||||||
|
### ⚠️ Immutability Principle
|
||||||
|
|
||||||
|
**Once a migration is applied to ANY environment (dev, staging, production), it MUST NOT be modified.**
|
||||||
|
|
||||||
|
Why?
|
||||||
|
- Each migration has a SHA256 checksum stored in the `schema_migrations` table
|
||||||
|
- Modifying an applied migration causes checksum mismatch errors
|
||||||
|
- Different environments would have inconsistent database states
|
||||||
|
- Breaks audit trail and reproducibility
|
||||||
|
|
||||||
|
### ✅ Correct Workflow
|
||||||
|
|
||||||
|
1. **Create new migration**
|
||||||
|
```bash
|
||||||
|
# Create new file with next sequential number
|
||||||
|
touch migrations/018_your_change.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Write Up and Down migrations**
|
||||||
|
- Up: Apply the change
|
||||||
|
- Down: Revert the change (should be symmetric with Up)
|
||||||
|
|
||||||
|
3. **Test locally**
|
||||||
|
```bash
|
||||||
|
# Apply migration
|
||||||
|
make migrate-up
|
||||||
|
|
||||||
|
# Test rollback
|
||||||
|
make migrate-down
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Commit and deploy**
|
||||||
|
```bash
|
||||||
|
git add migrations/018_your_change.sql
|
||||||
|
git commit -m "feat(db): add your change"
|
||||||
|
```
|
||||||
|
|
||||||
|
### ❌ What NOT to Do
|
||||||
|
|
||||||
|
- ❌ Modify an already-applied migration file
|
||||||
|
- ❌ Delete migration files
|
||||||
|
- ❌ Change migration file names
|
||||||
|
- ❌ Reorder migration numbers
|
||||||
|
|
||||||
|
### 🔧 If You Accidentally Modified an Applied Migration
|
||||||
|
|
||||||
|
**Error message:**
|
||||||
|
```
|
||||||
|
migration 017_add_gemini_tier_id.sql checksum mismatch (db=abc123... file=def456...)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solution:**
|
||||||
|
```bash
|
||||||
|
# 1. Find the original version
|
||||||
|
git log --oneline -- migrations/017_add_gemini_tier_id.sql
|
||||||
|
|
||||||
|
# 2. Revert to the commit when it was first applied
|
||||||
|
git checkout <commit-hash> -- migrations/017_add_gemini_tier_id.sql
|
||||||
|
|
||||||
|
# 3. Create a NEW migration for your changes
|
||||||
|
touch migrations/018_your_new_change.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
## Migration System Details
|
||||||
|
|
||||||
|
- **Checksum Algorithm**: SHA256 of trimmed file content
|
||||||
|
- **Tracking Table**: `schema_migrations` (filename, checksum, applied_at)
|
||||||
|
- **Runner**: `internal/repository/migrations_runner.go`
|
||||||
|
- **Auto-run**: Migrations run automatically on service startup
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
1. **Keep migrations small and focused**
|
||||||
|
- One logical change per migration
|
||||||
|
- Easier to review and rollback
|
||||||
|
|
||||||
|
2. **Write reversible migrations**
|
||||||
|
- Always provide a working Down migration
|
||||||
|
- Test rollback before committing
|
||||||
|
|
||||||
|
3. **Use transactions**
|
||||||
|
- Wrap DDL statements in transactions when possible
|
||||||
|
- Ensures atomicity
|
||||||
|
|
||||||
|
4. **Add comments**
|
||||||
|
- Explain WHY the change is needed
|
||||||
|
- Document any special considerations
|
||||||
|
|
||||||
|
5. **Test in development first**
|
||||||
|
- Apply migration locally
|
||||||
|
- Verify data integrity
|
||||||
|
- Test rollback
|
||||||
|
|
||||||
|
## Example Migration
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- +goose Up
|
||||||
|
-- +goose StatementBegin
|
||||||
|
-- Add tier_id field to Gemini OAuth accounts for quota tracking
|
||||||
|
UPDATE accounts
|
||||||
|
SET credentials = jsonb_set(
|
||||||
|
credentials,
|
||||||
|
'{tier_id}',
|
||||||
|
'"LEGACY"',
|
||||||
|
true
|
||||||
|
)
|
||||||
|
WHERE platform = 'gemini'
|
||||||
|
AND type = 'oauth'
|
||||||
|
AND credentials->>'tier_id' IS NULL;
|
||||||
|
-- +goose StatementEnd
|
||||||
|
|
||||||
|
-- +goose Down
|
||||||
|
-- +goose StatementBegin
|
||||||
|
-- Remove tier_id field
|
||||||
|
UPDATE accounts
|
||||||
|
SET credentials = credentials - 'tier_id'
|
||||||
|
WHERE platform = 'gemini'
|
||||||
|
AND type = 'oauth'
|
||||||
|
AND credentials->>'tier_id' = 'LEGACY';
|
||||||
|
-- +goose StatementEnd
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Checksum Mismatch
|
||||||
|
See "If You Accidentally Modified an Applied Migration" above.
|
||||||
|
|
||||||
|
### Migration Failed
|
||||||
|
```bash
|
||||||
|
# Check migration status
|
||||||
|
psql -d sub2api -c "SELECT * FROM schema_migrations ORDER BY applied_at DESC;"
|
||||||
|
|
||||||
|
# Manually rollback if needed (use with caution)
|
||||||
|
# Better to fix the migration and create a new one
|
||||||
|
```
|
||||||
|
|
||||||
|
### Need to Skip a Migration (Emergency Only)
|
||||||
|
```sql
|
||||||
|
-- DANGEROUS: Only use in development or with extreme caution
|
||||||
|
INSERT INTO schema_migrations (filename, checksum, applied_at)
|
||||||
|
VALUES ('NNN_migration.sql', 'calculated_checksum', NOW());
|
||||||
|
```
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Migration runner: `internal/repository/migrations_runner.go`
|
||||||
|
- Goose syntax: https://github.com/pressly/goose
|
||||||
|
- PostgreSQL docs: https://www.postgresql.org/docs/
|
||||||
Reference in New Issue
Block a user