179 lines
4.4 KiB
Markdown
179 lines
4.4 KiB
Markdown
# Database Migrations
|
|
|
|
## Overview
|
|
|
|
This directory contains SQL migration files for database schema changes. The migration system uses SHA256 checksums to ensure migration immutability and consistency across environments.
|
|
|
|
## Migration File Naming
|
|
|
|
Format: `NNN_description.sql`
|
|
- `NNN`: Sequential number (e.g., 001, 002, 003)
|
|
- `description`: Brief description in snake_case
|
|
|
|
Example: `017_add_gemini_tier_id.sql`
|
|
|
|
## Migration File Structure
|
|
|
|
```sql
|
|
-- +goose Up
|
|
-- +goose StatementBegin
|
|
-- Your forward migration SQL here
|
|
-- +goose StatementEnd
|
|
|
|
-- +goose Down
|
|
-- +goose StatementBegin
|
|
-- Your rollback migration SQL here
|
|
-- +goose StatementEnd
|
|
```
|
|
|
|
## Important Rules
|
|
|
|
### ⚠️ Immutability Principle
|
|
|
|
**Once a migration is applied to ANY environment (dev, staging, production), it MUST NOT be modified.**
|
|
|
|
Why?
|
|
- Each migration has a SHA256 checksum stored in the `schema_migrations` table
|
|
- Modifying an applied migration causes checksum mismatch errors
|
|
- Different environments would have inconsistent database states
|
|
- Breaks audit trail and reproducibility
|
|
|
|
### ✅ Correct Workflow
|
|
|
|
1. **Create new migration**
|
|
```bash
|
|
# Create new file with next sequential number
|
|
touch migrations/018_your_change.sql
|
|
```
|
|
|
|
2. **Write Up and Down migrations**
|
|
- Up: Apply the change
|
|
- Down: Revert the change (should be symmetric with Up)
|
|
|
|
3. **Test locally**
|
|
```bash
|
|
# Apply migration
|
|
make migrate-up
|
|
|
|
# Test rollback
|
|
make migrate-down
|
|
```
|
|
|
|
4. **Commit and deploy**
|
|
```bash
|
|
git add migrations/018_your_change.sql
|
|
git commit -m "feat(db): add your change"
|
|
```
|
|
|
|
### ❌ What NOT to Do
|
|
|
|
- ❌ Modify an already-applied migration file
|
|
- ❌ Delete migration files
|
|
- ❌ Change migration file names
|
|
- ❌ Reorder migration numbers
|
|
|
|
### 🔧 If You Accidentally Modified an Applied Migration
|
|
|
|
**Error message:**
|
|
```
|
|
migration 017_add_gemini_tier_id.sql checksum mismatch (db=abc123... file=def456...)
|
|
```
|
|
|
|
**Solution:**
|
|
```bash
|
|
# 1. Find the original version
|
|
git log --oneline -- migrations/017_add_gemini_tier_id.sql
|
|
|
|
# 2. Revert to the commit when it was first applied
|
|
git checkout <commit-hash> -- migrations/017_add_gemini_tier_id.sql
|
|
|
|
# 3. Create a NEW migration for your changes
|
|
touch migrations/018_your_new_change.sql
|
|
```
|
|
|
|
## Migration System Details
|
|
|
|
- **Checksum Algorithm**: SHA256 of trimmed file content
|
|
- **Tracking Table**: `schema_migrations` (filename, checksum, applied_at)
|
|
- **Runner**: `internal/repository/migrations_runner.go`
|
|
- **Auto-run**: Migrations run automatically on service startup
|
|
|
|
## Best Practices
|
|
|
|
1. **Keep migrations small and focused**
|
|
- One logical change per migration
|
|
- Easier to review and rollback
|
|
|
|
2. **Write reversible migrations**
|
|
- Always provide a working Down migration
|
|
- Test rollback before committing
|
|
|
|
3. **Use transactions**
|
|
- Wrap DDL statements in transactions when possible
|
|
- Ensures atomicity
|
|
|
|
4. **Add comments**
|
|
- Explain WHY the change is needed
|
|
- Document any special considerations
|
|
|
|
5. **Test in development first**
|
|
- Apply migration locally
|
|
- Verify data integrity
|
|
- Test rollback
|
|
|
|
## Example Migration
|
|
|
|
```sql
|
|
-- +goose Up
|
|
-- +goose StatementBegin
|
|
-- Add tier_id field to Gemini OAuth accounts for quota tracking
|
|
UPDATE accounts
|
|
SET credentials = jsonb_set(
|
|
credentials,
|
|
'{tier_id}',
|
|
'"LEGACY"',
|
|
true
|
|
)
|
|
WHERE platform = 'gemini'
|
|
AND type = 'oauth'
|
|
AND credentials->>'tier_id' IS NULL;
|
|
-- +goose StatementEnd
|
|
|
|
-- +goose Down
|
|
-- +goose StatementBegin
|
|
-- Remove tier_id field
|
|
UPDATE accounts
|
|
SET credentials = credentials - 'tier_id'
|
|
WHERE platform = 'gemini'
|
|
AND type = 'oauth'
|
|
AND credentials->>'tier_id' = 'LEGACY';
|
|
-- +goose StatementEnd
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Checksum Mismatch
|
|
See "If You Accidentally Modified an Applied Migration" above.
|
|
|
|
### Migration Failed
|
|
```bash
|
|
# Check migration status
|
|
psql -d sub2api -c "SELECT * FROM schema_migrations ORDER BY applied_at DESC;"
|
|
|
|
# Manually rollback if needed (use with caution)
|
|
# Better to fix the migration and create a new one
|
|
```
|
|
|
|
### Need to Skip a Migration (Emergency Only)
|
|
```sql
|
|
-- DANGEROUS: Only use in development or with extreme caution
|
|
INSERT INTO schema_migrations (filename, checksum, applied_at)
|
|
VALUES ('NNN_migration.sql', 'calculated_checksum', NOW());
|
|
```
|
|
|
|
## References
|
|
|
|
- Migration runner: `internal/repository/migrations_runner.go`
|
|
- Goose syntax: https://github.com/pressly/goose
|
|
- PostgreSQL docs: https://www.postgresql.org/docs/
|