Added auth functionality, initial work with views and field types
This commit is contained in:
Francisco Gaona
302
TENANT_MIGRATION_GUIDE.md
Normal file
302
TENANT_MIGRATION_GUIDE.md
Normal file
@@ -0,0 +1,302 @@
|
||||
# Tenant Migration Guide
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Create a New Migration
|
||||
```bash
|
||||
cd backend
|
||||
npm run migrate:make add_your_feature_name
|
||||
```
|
||||
|
||||
Edit the generated file in `backend/migrations/tenant/`
|
||||
|
||||
### Test on Single Tenant
|
||||
```bash
|
||||
npm run migrate:tenant acme-corp
|
||||
```
|
||||
|
||||
### Apply to All Tenants
|
||||
```bash
|
||||
npm run migrate:all-tenants
|
||||
```
|
||||
|
||||
## Available Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `npm run migrate:make <name>` | Create a new migration file |
|
||||
| `npm run migrate:tenant <slug>` | Run migrations for a specific tenant |
|
||||
| `npm run migrate:all-tenants` | Run migrations for all active tenants |
|
||||
| `npm run migrate:latest` | Run migrations (default DB - rarely used) |
|
||||
| `npm run migrate:rollback` | Rollback last migration (default DB) |
|
||||
|
||||
## Architecture
|
||||
|
||||
### Multi-Tenant Database Structure
|
||||
|
||||
```
|
||||
┌─────────────────────────┐
|
||||
│ Central Database │
|
||||
│ │
|
||||
│ - tenants table │
|
||||
│ - users table │
|
||||
│ - (encrypted creds) │
|
||||
└─────────────────────────┘
|
||||
│
|
||||
│ manages
|
||||
│
|
||||
┌───────┴────────┐
|
||||
│ │
|
||||
┌───▼────┐ ┌───▼────┐
|
||||
│ Tenant │ │ Tenant │
|
||||
│ DB1 │ ... │ DBN │
|
||||
│ │ │ │
|
||||
│ - users│ │ - users│
|
||||
│ - roles│ │ - roles│
|
||||
│ - apps │ │ - apps │
|
||||
│ - ... │ │ - ... │
|
||||
└────────┘ └────────┘
|
||||
```
|
||||
|
||||
### How Migrations Work
|
||||
|
||||
1. **New Tenant Provisioning** (Automatic)
|
||||
- User creates tenant via API
|
||||
- `TenantProvisioningService.provisionTenant()` is called
|
||||
- Database is created
|
||||
- All migrations in `migrations/tenant/` are automatically run
|
||||
- Tenant status set to ACTIVE
|
||||
|
||||
2. **Existing Tenants** (Manual)
|
||||
- Developer creates new migration file
|
||||
- Tests on single tenant: `npm run migrate:tenant test-tenant`
|
||||
- Applies to all: `npm run migrate:all-tenants`
|
||||
- Each tenant database is updated independently
|
||||
|
||||
### Migration Scripts
|
||||
|
||||
#### `migrate-tenant.ts`
|
||||
- Accepts tenant slug or ID as argument
|
||||
- Fetches tenant from central database
|
||||
- Decrypts database password
|
||||
- Creates Knex connection to tenant DB
|
||||
- Runs pending migrations
|
||||
- Reports success/failure
|
||||
|
||||
#### `migrate-all-tenants.ts`
|
||||
- Fetches all ACTIVE tenants from central DB
|
||||
- Iterates through each tenant
|
||||
- Runs migrations sequentially
|
||||
- Collects success/failure results
|
||||
- Provides comprehensive summary
|
||||
- Exits with error if any tenant fails
|
||||
|
||||
## Security
|
||||
|
||||
### Password Encryption
|
||||
|
||||
Tenant database passwords are encrypted using **AES-256-CBC** and stored in the central database.
|
||||
|
||||
**Required Environment Variable:**
|
||||
```bash
|
||||
DB_ENCRYPTION_KEY=your-32-character-secret-key!!
|
||||
```
|
||||
|
||||
This key must:
|
||||
- Be exactly 32 characters (256 bits)
|
||||
- Match the key used by backend services
|
||||
- Be kept secure (never commit to git)
|
||||
- Be the same across all environments accessing tenant DBs
|
||||
|
||||
### Encryption Flow
|
||||
|
||||
```
|
||||
Tenant Creation:
|
||||
Plain Password → Encrypt → Store in Central DB
|
||||
|
||||
Migration Time:
|
||||
Encrypted Password → Decrypt → Connect to Tenant DB → Run Migrations
|
||||
```
|
||||
|
||||
## Example Workflow
|
||||
|
||||
### Adding a New Field to All Tenants
|
||||
|
||||
```bash
|
||||
# 1. Create migration
|
||||
cd backend
|
||||
npm run migrate:make add_priority_to_tasks
|
||||
|
||||
# 2. Edit the migration file
|
||||
# migrations/tenant/20250127120000_add_priority_to_tasks.js
|
||||
|
||||
# 3. Test on staging tenant
|
||||
npm run migrate:tenant staging-company
|
||||
|
||||
# 4. Verify it worked
|
||||
# Connect to staging DB and check schema
|
||||
|
||||
# 5. Apply to all tenants
|
||||
npm run migrate:all-tenants
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
🚀 Starting migration for all tenants...
|
||||
|
||||
📋 Found 5 active tenant(s)
|
||||
|
||||
🔄 Migrating tenant: Acme Corp (acme_corp_db)
|
||||
✅ Acme Corp: Ran 1 migrations:
|
||||
- 20250127120000_add_priority_to_tasks.js
|
||||
|
||||
🔄 Migrating tenant: TechStart (techstart_db)
|
||||
✅ TechStart: Ran 1 migrations:
|
||||
- 20250127120000_add_priority_to_tasks.js
|
||||
|
||||
...
|
||||
|
||||
============================================================
|
||||
📊 Migration Summary
|
||||
============================================================
|
||||
✅ Successful: 5
|
||||
❌ Failed: 0
|
||||
|
||||
🎉 All tenant migrations completed successfully!
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Error: "Cannot find module '../prisma/generated-central/client'"
|
||||
|
||||
**Solution:** Generate Prisma client
|
||||
```bash
|
||||
cd backend
|
||||
npx prisma generate --schema=prisma/schema-central.prisma
|
||||
```
|
||||
|
||||
### Error: "Invalid encrypted password format"
|
||||
|
||||
**Solution:** Check `DB_ENCRYPTION_KEY` environment variable matches the one used for encryption.
|
||||
|
||||
### Error: "Migration failed: Table already exists"
|
||||
|
||||
**Cause:** Migration was partially applied or run manually
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check migration status in tenant DB
|
||||
mysql -h <host> -u <user> -p<pass> <dbname> -e "SELECT * FROM knex_migrations"
|
||||
|
||||
# If migration is listed, it's already applied
|
||||
# If not, investigate why table exists and fix manually
|
||||
```
|
||||
|
||||
### Migration Hangs
|
||||
|
||||
**Possible causes:**
|
||||
- Network connection to database lost
|
||||
- Database server down
|
||||
- Migration has long-running query
|
||||
|
||||
**Solution:** Add timeout to migration and check database connectivity
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. ✅ **Test first**: Always test migrations on a single tenant before applying to all
|
||||
2. ✅ **Rollback ready**: Write `down()` functions for every migration
|
||||
3. ✅ **Idempotent**: Use `IF NOT EXISTS` clauses where possible
|
||||
4. ✅ **Backup**: Take database backups before major migrations
|
||||
5. ✅ **Monitor**: Watch the output of `migrate:all-tenants` carefully
|
||||
6. ✅ **Version control**: Commit migration files to git
|
||||
7. ✅ **Document**: Add comments explaining complex migrations
|
||||
|
||||
8. ❌ **Don't skip testing**: Never run untested migrations on production
|
||||
9. ❌ **Don't modify**: Never modify existing migration files after they're deployed
|
||||
10. ❌ **Don't forget down()**: Always implement rollback logic
|
||||
|
||||
## Integration with TenantProvisioningService
|
||||
|
||||
The migrations are also used during tenant provisioning:
|
||||
|
||||
```typescript
|
||||
// src/tenant/tenant-provisioning.service.ts
|
||||
|
||||
async provisionTenant(tenantId: string): Promise<void> {
|
||||
// ... create database ...
|
||||
|
||||
// Run migrations automatically
|
||||
await this.runTenantMigrations(tenant);
|
||||
|
||||
// ... update tenant status ...
|
||||
}
|
||||
|
||||
async runTenantMigrations(tenant: any): Promise<void> {
|
||||
const knexConfig = {
|
||||
client: 'mysql2',
|
||||
connection: {
|
||||
host: tenant.dbHost,
|
||||
port: tenant.dbPort,
|
||||
user: tenant.dbUser,
|
||||
password: decryptedPassword,
|
||||
database: tenant.dbName,
|
||||
},
|
||||
migrations: {
|
||||
directory: './migrations/tenant',
|
||||
},
|
||||
};
|
||||
|
||||
const knexInstance = knex(knexConfig);
|
||||
await knexInstance.migrate.latest();
|
||||
await knexInstance.destroy();
|
||||
}
|
||||
```
|
||||
|
||||
This ensures every new tenant starts with the complete schema.
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
### Docker Compose
|
||||
```yaml
|
||||
services:
|
||||
backend:
|
||||
image: your-backend:latest
|
||||
command: sh -c "npm run migrate:all-tenants && npm run start:prod"
|
||||
environment:
|
||||
- DB_ENCRYPTION_KEY=${DB_ENCRYPTION_KEY}
|
||||
```
|
||||
|
||||
### Kubernetes Job
|
||||
```yaml
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: tenant-migrations
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: migrate
|
||||
image: your-backend:latest
|
||||
command: ["npm", "run", "migrate:all-tenants"]
|
||||
env:
|
||||
- name: DB_ENCRYPTION_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: db-secrets
|
||||
key: encryption-key
|
||||
restartPolicy: OnFailure
|
||||
```
|
||||
|
||||
## Further Documentation
|
||||
|
||||
- [Backend Scripts README](backend/scripts/README.md) - Detailed script documentation
|
||||
- [Multi-Tenant Implementation](MULTI_TENANT_IMPLEMENTATION.md) - Architecture overview
|
||||
- [Multi-Tenant Migration](MULTI_TENANT_MIGRATION.md) - Migration strategy
|
||||
|
||||
## Support
|
||||
|
||||
For questions or issues:
|
||||
1. Check the [Backend Scripts README](backend/scripts/README.md)
|
||||
2. Review existing migration files in `backend/migrations/tenant/`
|
||||
3. Check Knex documentation: https://knexjs.org/guide/migrations.html
|
||||
Reference in New Issue
Block a user