WIP - custom migrations when object is created
This commit is contained in:
324
CUSTOM_MIGRATIONS_IMPLEMENTATION.md
Normal file
324
CUSTOM_MIGRATIONS_IMPLEMENTATION.md
Normal file
@@ -0,0 +1,324 @@
|
||||
# Custom Migrations Implementation
|
||||
|
||||
## Overview
|
||||
|
||||
This implementation adds a database-stored migration system for dynamically created objects. Migrations are recorded in a `custom_migrations` table in each tenant database, allowing them to be replayed or used for environment replication in the future.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Components
|
||||
|
||||
#### 1. CustomMigrationService
|
||||
**Location:** `backend/src/migration/custom-migration.service.ts`
|
||||
|
||||
Handles all migration-related operations:
|
||||
|
||||
- **`generateCreateTableSQL(tableName, fields)`** - Generates SQL for creating object tables with standard fields
|
||||
- **`createMigrationRecord()`** - Stores migration metadata in the database
|
||||
- **`executeMigration()`** - Executes a pending migration and updates its status
|
||||
- **`createAndExecuteMigration()`** - Creates and immediately executes a migration
|
||||
- **`getMigrations()`** - Retrieves migration history with filtering
|
||||
- **`ensureMigrationsTable()`** - Ensures the `custom_migrations` table exists
|
||||
|
||||
#### 2. MigrationModule
|
||||
**Location:** `backend/src/migration/migration.module.ts`
|
||||
|
||||
Provides the CustomMigrationService to other modules.
|
||||
|
||||
#### 3. Updated ObjectService
|
||||
**Location:** `backend/src/object/object.service.ts`
|
||||
|
||||
- Injects CustomMigrationService
|
||||
- Calls `createAndExecuteMigration()` when a new object is created
|
||||
- Generates table creation migrations with standard fields
|
||||
|
||||
### Database Schema
|
||||
|
||||
#### custom_migrations Table
|
||||
|
||||
```sql
|
||||
CREATE TABLE custom_migrations (
|
||||
id UUID PRIMARY KEY,
|
||||
tenantId UUID NOT NULL,
|
||||
name VARCHAR(255) NOT NULL,
|
||||
description TEXT,
|
||||
type ENUM('create_table', 'add_column', 'alter_column', 'add_index', 'drop_table', 'custom'),
|
||||
sql TEXT NOT NULL,
|
||||
status ENUM('pending', 'executed', 'failed') DEFAULT 'pending',
|
||||
executedAt TIMESTAMP NULL,
|
||||
error TEXT,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
INDEX idx_tenantId (tenantId),
|
||||
INDEX idx_status (status),
|
||||
INDEX idx_created_at (created_at)
|
||||
)
|
||||
```
|
||||
|
||||
#### Generated Object Tables
|
||||
|
||||
When a new object is created (e.g., "Account"), a table is automatically created with:
|
||||
|
||||
```sql
|
||||
CREATE TABLE accounts (
|
||||
id VARCHAR(36) PRIMARY KEY,
|
||||
ownerId VARCHAR(36),
|
||||
name VARCHAR(255),
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
-- Custom fields added here
|
||||
)
|
||||
```
|
||||
|
||||
**Standard Fields:**
|
||||
- `id` - UUID primary key
|
||||
- `ownerId` - User who owns the record
|
||||
- `name` - Primary name field
|
||||
- `created_at` - Record creation timestamp
|
||||
- `updated_at` - Record update timestamp
|
||||
|
||||
### Field Type Mapping
|
||||
|
||||
Custom fields are mapped to SQL column types:
|
||||
|
||||
| Field Type | SQL Type | Notes |
|
||||
|---|---|---|
|
||||
| TEXT, STRING | VARCHAR(255) | |
|
||||
| LONG_TEXT | TEXT | Large text content |
|
||||
| NUMBER, DECIMAL | DECIMAL(18, 2) | |
|
||||
| INTEGER | INT | |
|
||||
| BOOLEAN | BOOLEAN | Defaults to FALSE |
|
||||
| DATE | DATE | |
|
||||
| DATE_TIME | DATETIME | |
|
||||
| EMAIL | VARCHAR(255) | |
|
||||
| URL | VARCHAR(2048) | |
|
||||
| PHONE | VARCHAR(20) | |
|
||||
| CURRENCY | DECIMAL(18, 2) | |
|
||||
| PERCENT | DECIMAL(5, 2) | |
|
||||
| PICKLIST, MULTI_PICKLIST | VARCHAR(255) | |
|
||||
| LOOKUP, BELONGS_TO | VARCHAR(36) | References foreign record ID |
|
||||
|
||||
## Usage Flow
|
||||
|
||||
### Creating a New Object
|
||||
|
||||
1. **User creates object definition:**
|
||||
```
|
||||
POST /api/objects
|
||||
{
|
||||
"apiName": "Account",
|
||||
"label": "Account",
|
||||
"description": "Customer account records"
|
||||
}
|
||||
```
|
||||
|
||||
2. **ObjectService.createObjectDefinition() executes:**
|
||||
- Inserts object metadata into `object_definitions` table
|
||||
- Generates create table SQL
|
||||
- Creates migration record with status "pending"
|
||||
- Executes migration immediately
|
||||
- Updates migration status to "executed"
|
||||
- Returns object definition
|
||||
|
||||
3. **Result:**
|
||||
- Object is now ready to use
|
||||
- Table exists in database
|
||||
- Migration history is recorded for future replication
|
||||
|
||||
### Migration Execution Flow
|
||||
|
||||
```
|
||||
createAndExecuteMigration()
|
||||
├── createMigrationRecord()
|
||||
│ └── Insert into custom_migrations (status: pending)
|
||||
└── executeMigration()
|
||||
├── Fetch migration record
|
||||
├── Execute SQL
|
||||
├── Update status: executed
|
||||
└── Return migration record
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
Migrations track execution status and errors:
|
||||
|
||||
- **Status: pending** - Not yet executed
|
||||
- **Status: executed** - Successfully completed
|
||||
- **Status: failed** - Error during execution
|
||||
|
||||
Failed migrations are logged and stored with error details for debugging and retry:
|
||||
|
||||
```typescript
|
||||
{
|
||||
id: "uuid",
|
||||
status: "failed",
|
||||
error: "Syntax error in SQL...",
|
||||
executedAt: null,
|
||||
updated_at: "2025-12-24T11:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
## Future Functionality
|
||||
|
||||
### Sandbox Environment Replication
|
||||
|
||||
Stored migrations enable:
|
||||
|
||||
1. **Cloning production environments** - Replay all migrations in new database
|
||||
2. **Data structure export/import** - Export migrations as SQL files
|
||||
3. **Audit trail** - Complete history of schema changes
|
||||
4. **Rollback capability** - Add down migrations for reverting changes
|
||||
5. **Dependency tracking** - Identify object dependencies from migrations
|
||||
|
||||
### Planned Enhancements
|
||||
|
||||
1. **Add down migrations** - Support undoing schema changes
|
||||
2. **Migration dependencies** - Track which migrations depend on others
|
||||
3. **Batch execution** - Run pending migrations together
|
||||
4. **Version control** - Track migration versions and changes
|
||||
5. **Manual migration creation** - API to create custom migrations
|
||||
6. **Migration status dashboard** - UI to view migration history
|
||||
|
||||
## Integration Points
|
||||
|
||||
### ObjectService
|
||||
|
||||
- Uses `getTenantKnexById()` for tenant database connections
|
||||
- Calls CustomMigrationService after creating object definitions
|
||||
- Handles migration execution errors gracefully (logs but doesn't fail)
|
||||
|
||||
### TenantDatabaseService
|
||||
|
||||
- Provides database connections via `getTenantKnexById()`
|
||||
- Connections are cached with prefix `id:${tenantId}`
|
||||
|
||||
### Module Dependencies
|
||||
|
||||
```
|
||||
ObjectModule
|
||||
├── imports: [TenantModule, MigrationModule]
|
||||
└── providers: [ObjectService, CustomMigrationService, ...]
|
||||
|
||||
MigrationModule
|
||||
├── imports: [TenantModule]
|
||||
└── providers: [CustomMigrationService]
|
||||
```
|
||||
|
||||
## API Endpoints (Future)
|
||||
|
||||
While not yet exposed via API, these operations could be added:
|
||||
|
||||
```typescript
|
||||
// Get migration history
|
||||
GET /api/migrations?tenantId=xxx&status=executed
|
||||
|
||||
// Get migration details
|
||||
GET /api/migrations/:id
|
||||
|
||||
// Retry failed migration
|
||||
POST /api/migrations/:id/retry
|
||||
|
||||
// Export migrations as SQL
|
||||
GET /api/migrations/export?tenantId=xxx
|
||||
|
||||
// Create custom migration
|
||||
POST /api/migrations
|
||||
{
|
||||
name: "add_field_to_accounts",
|
||||
description: "Add phone_number field",
|
||||
sql: "ALTER TABLE accounts ADD phone_number VARCHAR(20)"
|
||||
}
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Manual Testing Steps
|
||||
|
||||
1. **Create a new object:**
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/api/objects \
|
||||
-H "Authorization: Bearer <token>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"apiName": "TestObject",
|
||||
"label": "Test Object",
|
||||
"description": "Test object creation"
|
||||
}'
|
||||
```
|
||||
|
||||
2. **Verify table was created:**
|
||||
```bash
|
||||
# In tenant database
|
||||
SHOW TABLES LIKE 'test_objects';
|
||||
DESCRIBE test_objects;
|
||||
```
|
||||
|
||||
3. **Check migration record:**
|
||||
```bash
|
||||
# In tenant database
|
||||
SELECT * FROM custom_migrations WHERE name LIKE '%test_objects%';
|
||||
```
|
||||
|
||||
4. **Create a record in the new object:**
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/api/test-objects \
|
||||
-H "Authorization: Bearer <token>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "My Test Record"
|
||||
}'
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Migration Fails with SQL Error
|
||||
|
||||
1. Check `custom_migrations` table for error details:
|
||||
```sql
|
||||
SELECT id, name, error, status FROM custom_migrations
|
||||
WHERE status = 'failed';
|
||||
```
|
||||
|
||||
2. Review the generated SQL in the `sql` column
|
||||
|
||||
3. Common issues:
|
||||
- Duplicate table names
|
||||
- Invalid field names (reserved SQL keywords)
|
||||
- Unsupported field types
|
||||
|
||||
### Table Not Created
|
||||
|
||||
1. Verify `custom_migrations` table exists:
|
||||
```sql
|
||||
SHOW TABLES LIKE 'custom_migrations';
|
||||
```
|
||||
|
||||
2. Check object service logs for migration execution errors
|
||||
|
||||
3. Manually retry migration:
|
||||
```typescript
|
||||
const migration = await tenantKnex('custom_migrations')
|
||||
.where({ status: 'failed' })
|
||||
.first();
|
||||
await customMigrationService.executeMigration(tenantKnex, migration.id);
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- **Table creation** is synchronous and happens immediately
|
||||
- **Migrations are cached** in custom_migrations table per tenant
|
||||
- **No file I/O** - all operations use database
|
||||
- **Index creation** optimized with proper indexes on common columns (tenantId, status, created_at)
|
||||
|
||||
## Security
|
||||
|
||||
- **Per-tenant isolation** - Each tenant's migrations stored separately
|
||||
- **No SQL injection** - Using Knex query builder for all operations
|
||||
- **Access control** - Migrations only created/executed by backend service
|
||||
- **Audit trail** - Complete history of all schema changes
|
||||
|
||||
## Related Files
|
||||
|
||||
- [backend/src/object/object.service.ts](backend/src/object/object.service.ts)
|
||||
- [backend/src/migration/custom-migration.service.ts](backend/src/migration/custom-migration.service.ts)
|
||||
- [backend/src/migration/migration.module.ts](backend/src/migration/migration.module.ts)
|
||||
@@ -0,0 +1,29 @@
|
||||
exports.up = function (knex) {
|
||||
return knex.schema.createTable('custom_migrations', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
table.uuid('tenantId').notNullable();
|
||||
table.string('name', 255).notNullable();
|
||||
table.text('description');
|
||||
table.enum('type', [
|
||||
'create_table',
|
||||
'add_column',
|
||||
'alter_column',
|
||||
'add_index',
|
||||
'drop_table',
|
||||
'custom',
|
||||
]).notNullable();
|
||||
table.text('sql').notNullable();
|
||||
table.enum('status', ['pending', 'executed', 'failed']).defaultTo('pending');
|
||||
table.timestamp('executedAt').nullable();
|
||||
table.text('error').nullable();
|
||||
table.timestamps(true, true);
|
||||
|
||||
table.index(['tenantId']);
|
||||
table.index(['status']);
|
||||
table.index(['created_at']);
|
||||
});
|
||||
};
|
||||
|
||||
exports.down = function (knex) {
|
||||
return knex.schema.dropTableIfExists('custom_migrations');
|
||||
};
|
||||
@@ -43,8 +43,9 @@ function decryptPassword(encryptedPassword: string): string {
|
||||
function createTenantKnexConnection(tenant: any): Knex {
|
||||
const decryptedPassword = decryptPassword(tenant.dbPassword);
|
||||
|
||||
// Replace 'db' hostname with 'localhost' when running outside Docker
|
||||
const dbHost = tenant.dbHost === 'db' ? 'localhost' : tenant.dbHost;
|
||||
// Use Docker hostname 'db' when running inside container
|
||||
// The dbHost will be 'db' for Docker connections or 'localhost' for local development
|
||||
const dbHost = tenant.dbHost;
|
||||
|
||||
return knex({
|
||||
client: 'mysql2',
|
||||
@@ -82,7 +83,7 @@ async function migrateTenant(tenant: any): Promise<void> {
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`❌ ${tenant.name}: Migration failed:`, error.message);
|
||||
console.error(`❌ ${tenant.name}: Migration failed:`, error);
|
||||
throw error;
|
||||
} finally {
|
||||
await tenantKnex.destroy();
|
||||
|
||||
306
backend/src/migration/custom-migration.service.ts
Normal file
306
backend/src/migration/custom-migration.service.ts
Normal file
@@ -0,0 +1,306 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
import { Knex } from 'knex';
|
||||
|
||||
export interface CustomMigrationRecord {
|
||||
id: string;
|
||||
tenantId: string;
|
||||
name: string;
|
||||
description: string;
|
||||
type: 'create_table' | 'add_column' | 'alter_column' | 'add_index' | 'drop_table' | 'custom';
|
||||
sql: string;
|
||||
status: 'pending' | 'executed' | 'failed';
|
||||
executedAt?: Date;
|
||||
error?: string;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
}
|
||||
|
||||
@Injectable()
|
||||
export class CustomMigrationService {
|
||||
private readonly logger = new Logger(CustomMigrationService.name);
|
||||
|
||||
/**
|
||||
* Generate SQL to create a table with standard fields
|
||||
*/
|
||||
generateCreateTableSQL(
|
||||
tableName: string,
|
||||
fields: {
|
||||
apiName: string;
|
||||
type: string;
|
||||
isRequired?: boolean;
|
||||
isUnique?: boolean;
|
||||
defaultValue?: string;
|
||||
}[] = [],
|
||||
): string {
|
||||
// Start with standard fields
|
||||
const columns: string[] = [
|
||||
'`id` VARCHAR(36) PRIMARY KEY',
|
||||
'`ownerId` VARCHAR(36)',
|
||||
'`name` VARCHAR(255)',
|
||||
'`created_at` TIMESTAMP DEFAULT CURRENT_TIMESTAMP',
|
||||
'`updated_at` TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP',
|
||||
];
|
||||
|
||||
// Add custom fields
|
||||
for (const field of fields) {
|
||||
const column = this.fieldToColumn(field);
|
||||
columns.push(column);
|
||||
}
|
||||
|
||||
// Add foreign key and index for ownerId
|
||||
columns.push('INDEX `idx_owner` (`ownerId`)');
|
||||
|
||||
return `CREATE TABLE IF NOT EXISTS \`${tableName}\` (
|
||||
${columns.join(',\n ')}
|
||||
)`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert field definition to SQL column definition
|
||||
*/
|
||||
private fieldToColumn(field: {
|
||||
apiName: string;
|
||||
type: string;
|
||||
isRequired?: boolean;
|
||||
isUnique?: boolean;
|
||||
defaultValue?: string;
|
||||
}): string {
|
||||
const columnName = field.apiName;
|
||||
let columnDef = `\`${columnName}\``;
|
||||
|
||||
// Map field types to SQL types
|
||||
switch (field.type.toUpperCase()) {
|
||||
case 'TEXT':
|
||||
case 'STRING':
|
||||
columnDef += ' VARCHAR(255)';
|
||||
break;
|
||||
case 'LONG_TEXT':
|
||||
columnDef += ' LONGTEXT';
|
||||
break;
|
||||
case 'NUMBER':
|
||||
case 'DECIMAL':
|
||||
columnDef += ' DECIMAL(18, 2)';
|
||||
break;
|
||||
case 'INTEGER':
|
||||
columnDef += ' INT';
|
||||
break;
|
||||
case 'BOOLEAN':
|
||||
columnDef += ' BOOLEAN DEFAULT FALSE';
|
||||
break;
|
||||
case 'DATE':
|
||||
columnDef += ' DATE';
|
||||
break;
|
||||
case 'DATE_TIME':
|
||||
columnDef += ' DATETIME';
|
||||
break;
|
||||
case 'EMAIL':
|
||||
columnDef += ' VARCHAR(255)';
|
||||
break;
|
||||
case 'URL':
|
||||
columnDef += ' VARCHAR(2048)';
|
||||
break;
|
||||
case 'PHONE':
|
||||
columnDef += ' VARCHAR(20)';
|
||||
break;
|
||||
case 'CURRENCY':
|
||||
columnDef += ' DECIMAL(18, 2)';
|
||||
break;
|
||||
case 'PERCENT':
|
||||
columnDef += ' DECIMAL(5, 2)';
|
||||
break;
|
||||
case 'PICKLIST':
|
||||
case 'MULTI_PICKLIST':
|
||||
columnDef += ' VARCHAR(255)';
|
||||
break;
|
||||
case 'LOOKUP':
|
||||
case 'BELONGS_TO':
|
||||
columnDef += ' VARCHAR(36)';
|
||||
break;
|
||||
default:
|
||||
columnDef += ' VARCHAR(255)';
|
||||
}
|
||||
|
||||
// Add constraints
|
||||
if (field.isRequired) {
|
||||
columnDef += ' NOT NULL';
|
||||
} else {
|
||||
columnDef += ' NULL';
|
||||
}
|
||||
|
||||
if (field.isUnique) {
|
||||
columnDef += ' UNIQUE';
|
||||
}
|
||||
|
||||
if (field.defaultValue !== undefined && field.defaultValue !== null) {
|
||||
columnDef += ` DEFAULT '${field.defaultValue}'`;
|
||||
}
|
||||
|
||||
return columnDef;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a custom migration record in the database
|
||||
*/
|
||||
async createMigrationRecord(
|
||||
tenantKnex: Knex,
|
||||
data: {
|
||||
tenantId: string;
|
||||
name: string;
|
||||
description: string;
|
||||
type: 'create_table' | 'add_column' | 'alter_column' | 'add_index' | 'drop_table' | 'custom';
|
||||
sql: string;
|
||||
},
|
||||
): Promise<CustomMigrationRecord> {
|
||||
// Ensure custom_migrations table exists
|
||||
await this.ensureMigrationsTable(tenantKnex);
|
||||
|
||||
const id = require('crypto').randomUUID();
|
||||
const now = new Date();
|
||||
|
||||
await tenantKnex('custom_migrations').insert({
|
||||
id,
|
||||
tenantId: data.tenantId,
|
||||
name: data.name,
|
||||
description: data.description,
|
||||
type: data.type,
|
||||
sql: data.sql,
|
||||
status: 'pending',
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
});
|
||||
|
||||
return tenantKnex('custom_migrations').where({ id }).first();
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a pending migration and update its status
|
||||
*/
|
||||
async executeMigration(
|
||||
tenantKnex: Knex,
|
||||
migrationId: string,
|
||||
): Promise<CustomMigrationRecord> {
|
||||
try {
|
||||
// Get the migration record
|
||||
const migration = await tenantKnex('custom_migrations')
|
||||
.where({ id: migrationId })
|
||||
.first();
|
||||
|
||||
if (!migration) {
|
||||
throw new Error(`Migration ${migrationId} not found`);
|
||||
}
|
||||
|
||||
if (migration.status === 'executed') {
|
||||
this.logger.log(`Migration ${migrationId} already executed`);
|
||||
return migration;
|
||||
}
|
||||
|
||||
// Execute the SQL
|
||||
this.logger.log(`Executing migration: ${migration.name}`);
|
||||
await tenantKnex.raw(migration.sql);
|
||||
|
||||
// Update status
|
||||
const now = new Date();
|
||||
await tenantKnex('custom_migrations')
|
||||
.where({ id: migrationId })
|
||||
.update({
|
||||
status: 'executed',
|
||||
executedAt: now,
|
||||
updated_at: now,
|
||||
});
|
||||
|
||||
this.logger.log(`Migration ${migration.name} executed successfully`);
|
||||
return tenantKnex('custom_migrations').where({ id: migrationId }).first();
|
||||
} catch (error) {
|
||||
this.logger.error(`Failed to execute migration ${migrationId}:`, error);
|
||||
|
||||
// Update status with error
|
||||
const now = new Date();
|
||||
await tenantKnex('custom_migrations')
|
||||
.where({ id: migrationId })
|
||||
.update({
|
||||
status: 'failed',
|
||||
error: error.message,
|
||||
updated_at: now,
|
||||
});
|
||||
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create and execute a migration in one step
|
||||
*/
|
||||
async createAndExecuteMigration(
|
||||
tenantKnex: Knex,
|
||||
tenantId: string,
|
||||
data: {
|
||||
name: string;
|
||||
description: string;
|
||||
type: 'create_table' | 'add_column' | 'alter_column' | 'add_index' | 'drop_table' | 'custom';
|
||||
sql: string;
|
||||
},
|
||||
): Promise<CustomMigrationRecord> {
|
||||
// Create the migration record
|
||||
const migration = await this.createMigrationRecord(tenantKnex, {
|
||||
tenantId,
|
||||
...data,
|
||||
});
|
||||
|
||||
// Execute it immediately
|
||||
return this.executeMigration(tenantKnex, migration.id);
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure the custom_migrations table exists in the tenant database
|
||||
*/
|
||||
private async ensureMigrationsTable(tenantKnex: Knex): Promise<void> {
|
||||
const hasTable = await tenantKnex.schema.hasTable('custom_migrations');
|
||||
|
||||
if (!hasTable) {
|
||||
await tenantKnex.schema.createTable('custom_migrations', (table) => {
|
||||
table.uuid('id').primary();
|
||||
table.uuid('tenantId').notNullable();
|
||||
table.string('name', 255).notNullable();
|
||||
table.text('description');
|
||||
table.enum('type', ['create_table', 'add_column', 'alter_column', 'add_index', 'drop_table', 'custom']).notNullable();
|
||||
table.text('sql').notNullable();
|
||||
table.enum('status', ['pending', 'executed', 'failed']).defaultTo('pending');
|
||||
table.timestamp('executedAt').nullable();
|
||||
table.text('error').nullable();
|
||||
table.timestamps(true, true);
|
||||
|
||||
table.index(['tenantId']);
|
||||
table.index(['status']);
|
||||
table.index(['created_at']);
|
||||
});
|
||||
|
||||
this.logger.log('Created custom_migrations table');
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all migrations for a tenant
|
||||
*/
|
||||
async getMigrations(
|
||||
tenantKnex: Knex,
|
||||
tenantId: string,
|
||||
filter?: {
|
||||
status?: 'pending' | 'executed' | 'failed';
|
||||
type?: string;
|
||||
},
|
||||
): Promise<CustomMigrationRecord[]> {
|
||||
await this.ensureMigrationsTable(tenantKnex);
|
||||
|
||||
let query = tenantKnex('custom_migrations').where({ tenantId });
|
||||
|
||||
if (filter?.status) {
|
||||
query = query.where({ status: filter.status });
|
||||
}
|
||||
|
||||
if (filter?.type) {
|
||||
query = query.where({ type: filter.type });
|
||||
}
|
||||
|
||||
return query.orderBy('created_at', 'asc');
|
||||
}
|
||||
}
|
||||
10
backend/src/migration/migration.module.ts
Normal file
10
backend/src/migration/migration.module.ts
Normal file
@@ -0,0 +1,10 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { CustomMigrationService } from './custom-migration.service';
|
||||
import { TenantModule } from '../tenant/tenant.module';
|
||||
|
||||
@Module({
|
||||
imports: [TenantModule],
|
||||
providers: [CustomMigrationService],
|
||||
exports: [CustomMigrationService],
|
||||
})
|
||||
export class MigrationModule {}
|
||||
@@ -5,9 +5,10 @@ import { SetupObjectController } from './setup-object.controller';
|
||||
import { SchemaManagementService } from './schema-management.service';
|
||||
import { FieldMapperService } from './field-mapper.service';
|
||||
import { TenantModule } from '../tenant/tenant.module';
|
||||
import { MigrationModule } from '../migration/migration.module';
|
||||
|
||||
@Module({
|
||||
imports: [TenantModule],
|
||||
imports: [TenantModule, MigrationModule],
|
||||
providers: [ObjectService, SchemaManagementService, FieldMapperService],
|
||||
controllers: [RuntimeObjectController, SetupObjectController],
|
||||
exports: [ObjectService, SchemaManagementService, FieldMapperService],
|
||||
|
||||
@@ -1,13 +1,18 @@
|
||||
import { Injectable, NotFoundException } from '@nestjs/common';
|
||||
import { TenantDatabaseService } from '../tenant/tenant-database.service';
|
||||
import { CustomMigrationService } from '../migration/custom-migration.service';
|
||||
|
||||
@Injectable()
|
||||
export class ObjectService {
|
||||
constructor(private tenantDbService: TenantDatabaseService) {}
|
||||
constructor(
|
||||
private tenantDbService: TenantDatabaseService,
|
||||
private customMigrationService: CustomMigrationService,
|
||||
) {}
|
||||
|
||||
// Setup endpoints - Object metadata management
|
||||
async getObjectDefinitions(tenantId: string) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const objects = await knex('object_definitions')
|
||||
.select('object_definitions.*')
|
||||
@@ -28,7 +33,8 @@ export class ObjectService {
|
||||
}
|
||||
|
||||
async getObjectDefinition(tenantId: string, apiName: string) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const obj = await knex('object_definitions')
|
||||
.where({ apiName })
|
||||
@@ -69,15 +75,104 @@ export class ObjectService {
|
||||
isSystem?: boolean;
|
||||
},
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const [id] = await knex('object_definitions').insert({
|
||||
id: knex.raw('(UUID())'),
|
||||
// Resolve tenant ID in case a slug was passed
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Generate UUID for the new object
|
||||
const objectId = require('crypto').randomUUID();
|
||||
|
||||
// Create the object definition record
|
||||
await knex('object_definitions').insert({
|
||||
id: objectId,
|
||||
...data,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
|
||||
return knex('object_definitions').where({ id }).first();
|
||||
const objectDef = await knex('object_definitions').where({ id: objectId }).first();
|
||||
|
||||
// Create standard field definitions (only if they don't already exist)
|
||||
const standardFields = [
|
||||
{
|
||||
apiName: 'ownerId',
|
||||
label: 'Owner',
|
||||
type: 'LOOKUP',
|
||||
description: 'The user who owns this record',
|
||||
isRequired: true,
|
||||
isUnique: false,
|
||||
referenceObject: null,
|
||||
},
|
||||
{
|
||||
apiName: 'name',
|
||||
label: 'Name',
|
||||
type: 'TEXT',
|
||||
description: 'The primary name field for this record',
|
||||
isRequired: true,
|
||||
isUnique: false,
|
||||
referenceObject: null,
|
||||
},
|
||||
{
|
||||
apiName: 'created_at',
|
||||
label: 'Created At',
|
||||
type: 'DATE_TIME',
|
||||
description: 'The timestamp when this record was created',
|
||||
isRequired: true,
|
||||
isUnique: false,
|
||||
referenceObject: null,
|
||||
},
|
||||
{
|
||||
apiName: 'updated_at',
|
||||
label: 'Updated At',
|
||||
type: 'DATE_TIME',
|
||||
description: 'The timestamp when this record was last updated',
|
||||
isRequired: true,
|
||||
isUnique: false,
|
||||
referenceObject: null,
|
||||
},
|
||||
];
|
||||
|
||||
// Insert standard field definitions that don't already exist
|
||||
for (const field of standardFields) {
|
||||
const existingField = await knex('field_definitions')
|
||||
.where({
|
||||
objectDefinitionId: objectDef.id,
|
||||
apiName: field.apiName,
|
||||
})
|
||||
.first();
|
||||
|
||||
if (!existingField) {
|
||||
await knex('field_definitions').insert({
|
||||
id: knex.raw('(UUID())'),
|
||||
objectDefinitionId: objectDef.id,
|
||||
...field,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Create a migration to create the table
|
||||
const tableName = this.getTableName(data.apiName);
|
||||
const createTableSQL = this.customMigrationService.generateCreateTableSQL(tableName);
|
||||
|
||||
try {
|
||||
await this.customMigrationService.createAndExecuteMigration(
|
||||
knex,
|
||||
resolvedTenantId,
|
||||
{
|
||||
name: `create_${tableName}_table`,
|
||||
description: `Create table for ${data.label} object`,
|
||||
type: 'create_table',
|
||||
sql: createTableSQL,
|
||||
},
|
||||
);
|
||||
} catch (error) {
|
||||
// Log the error but don't fail - migration is recorded for future retry
|
||||
console.error(`Failed to execute table creation migration: ${error.message}`);
|
||||
}
|
||||
|
||||
return objectDef;
|
||||
}
|
||||
|
||||
async createFieldDefinition(
|
||||
@@ -94,7 +189,8 @@ export class ObjectService {
|
||||
defaultValue?: string;
|
||||
},
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
const obj = await this.getObjectDefinition(tenantId, objectApiName);
|
||||
|
||||
const [id] = await knex('field_definitions').insert({
|
||||
@@ -134,7 +230,8 @@ export class ObjectService {
|
||||
userId: string,
|
||||
filters?: any,
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Verify object exists
|
||||
await this.getObjectDefinition(tenantId, objectApiName);
|
||||
@@ -163,7 +260,8 @@ export class ObjectService {
|
||||
recordId: string,
|
||||
userId: string,
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Verify object exists
|
||||
await this.getObjectDefinition(tenantId, objectApiName);
|
||||
@@ -193,7 +291,8 @@ export class ObjectService {
|
||||
data: any,
|
||||
userId: string,
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Verify object exists
|
||||
await this.getObjectDefinition(tenantId, objectApiName);
|
||||
@@ -226,7 +325,8 @@ export class ObjectService {
|
||||
data: any,
|
||||
userId: string,
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Verify object exists and user has access
|
||||
await this.getRecord(tenantId, objectApiName, recordId, userId);
|
||||
@@ -246,7 +346,8 @@ export class ObjectService {
|
||||
recordId: string,
|
||||
userId: string,
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Verify object exists and user has access
|
||||
await this.getRecord(tenantId, objectApiName, recordId, userId);
|
||||
|
||||
@@ -169,6 +169,36 @@ export class TenantDatabaseService {
|
||||
return domainRecord.tenant;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve tenant by ID or slug
|
||||
* Tries ID first, then falls back to slug
|
||||
*/
|
||||
async resolveTenantId(idOrSlug: string): Promise<string> {
|
||||
const centralPrisma = getCentralPrisma();
|
||||
|
||||
// Try by ID first
|
||||
let tenant = await centralPrisma.tenant.findUnique({
|
||||
where: { id: idOrSlug },
|
||||
});
|
||||
|
||||
// If not found, try by slug
|
||||
if (!tenant) {
|
||||
tenant = await centralPrisma.tenant.findUnique({
|
||||
where: { slug: idOrSlug },
|
||||
});
|
||||
}
|
||||
|
||||
if (!tenant) {
|
||||
throw new Error(`Tenant ${idOrSlug} not found`);
|
||||
}
|
||||
|
||||
if (tenant.status !== 'active') {
|
||||
throw new Error(`Tenant ${tenant.name} is not active`);
|
||||
}
|
||||
|
||||
return tenant.id;
|
||||
}
|
||||
|
||||
async disconnectTenant(tenantId: string) {
|
||||
const connection = this.tenantConnections.get(tenantId);
|
||||
if (connection) {
|
||||
|
||||
Reference in New Issue
Block a user